Holographic Doctors can now Provide Medical Advice on the International Space Station

In recent years, the practice of “telemedicine” has grown considerably. Similar to “telepresence” and “telecommuting,” this technology relies on high-speed internet and live-video streaming to allow a person in one part of the world to interact with people in another without being physically present. The technology has come in handy during the two-year COVID-19 pandemic, where doctors were in high demand, but physical travel was restricted.

This process has also allowed for the emergence of “holoportation,” which relies on holographic technology and 3-D modeling for the same purpose. In October 2021, the first “holoportation handshake” was conducted between Earth and space and demonstrated the technology’s potential for future missions. On this occasion, NASA flight surgeon Dr. Josef Schmid, AEXA Aerospace CEO Fernando De La Pena Llaca, and their teams had a two-way conversation with ESA astronaut Thomas Pesquet aboard the International Space Station (ISS).

To break it down, holoportation is a type of technology that allows high-quality 3D models of people to be captured, reconstructed, compressed, and live-transmitted anywhere in real-time. Combined with mixed reality displays, it allows users to see, hear, and interact with other participants as if they were present in the same physical space. While Microsoft has used holoportation since 2016, this was the first time it was used to connect people between Earth and space.

In this case, Schmid and Pesquet connected using the Microsoft Hololens Kinect camera and a personal computer with custom software from Aexa Aerospace. This Texas-based company is dedicated to the development of holoportation technology to enhance personal communication, training, and real-time operations support. With this “handshake,” NASA and Aexa successfully demonstrated the technology and how it could enable telemedicine in space! As Schmid described it in a recent NASA press release:

“This is completely new manner of human communication across vast distances. Furthermore, it is a brand-new way of human exploration, where our human entity is able to travel off the planet. Our physical body is not there, but our human entity absolutely is there. 

“It doesn’t matter that the space station is traveling 17,500 mph and in constant motion in orbit 250 miles above Earth, the astronaut can come back three minutes or three weeks later and with the system running, we will be there in that spot, live on the space station.”

In the near future, NASA hopes to send astronauts back to the Moon, Mars, and other locations in deep space. There are also extensive plans for commercializing Low Earth Orbit (LEO), involving everything from space stations that will replace the ISS to orbital habitats (like the Orbital Reef or the Voyager Station). Given the distances involved, astronauts will be unable to depend on regular resupply missions as the ISS does.

In addition, there is currently no legal framework or certification process for “space medicine,” which creates a bit of a problem for sending doctors along on deep-space missions. Dr. Maurice Ramirez, a doctor of osteopathic medicine and a former senior physician with the Florida Department of Health (DOH) and the National Disaster Medical System (NDMS), explained this disparity to Universe Today via email:

“Currently, none of the three organizations that provide board certifications to physicians provide Board Certification in Space Medicine. [Only] one boarding organization is currently working towards a Certificate of Added Qualification (CAQ). The CAQ would add the designation ‘in Space’ to existing certification. Thus my certification in Emergency Medicine plus the CAQ would become ‘Board Certified in Emergency Medicine in Space.'”

For these reasons, holoportation and tools like could have significant implications for future missions to deep space. Naturally, a few obstacles need to be overcome for this technology to be practical, foremost of which is the communication delays for missions to Mars. Depending on the distance between Earth and Mars, it takes between 5 and 20 minutes for communications to reach robotic missions on or around the Red Planet.

This issue will become all the more pressing when crewed missions are sent to Mars, which will need to be able to establish contact with Mission Control at all times. But with the right strategies, holoportation could become a regular feature for missions destined for deep space, whether it’s for the purpose of mission communications, medical or technical support, or so astronaut crews can stay in touch with their loved ones.

The successful demonstration of this technology is a precursor for more extensive use on future missions. Looking ahead, NASA plans to pair it with two-way communication so that people on Earth are holoported to space and astronauts are holoported back to Earth. Beyond that, they intend to combine holoportation with augmented reality to enable full-scale “tele-mentoring.” Said Schmidt:

“We’ll use this for our private medical conferences, private psychiatric conferences, private family conferences and to bring VIPs onto the space station to visit with astronauts. Imagine you can bring the best instructor or the actual designer of a particularly complex technology right beside you wherever you might be working on it.

“Furthermore, we will combine augmented reality with haptics. You can work on the device together, much like two of the best surgeons working during an operation. This would put everyone at rest knowing the best team is working together on a critical piece of hardware.” 

As with all space-based research and innovation, developments in holoportation also have direct applications here on Earth. In extreme environments such as Antarctica, offshore oil rigs, or military bases far from home, this technology could allow for communications where it feels like people are physically present. The implications this would have for medicine, education, work, and missions that take astronauts far beyond Low Earth Orbit (LEO) would be nothing short of revolutionary!

Further Reading: NASA