Avatars Return to the Movies — and Find a Real-Life Foothold

Thirteen years after the original “Avatar” movie came out, the idea of human minds inhabiting alien bodies is returning for an amped-up sequel — and since 2009, real-life efforts to create robotic avatars have advanced at least as much as computer-aided filmmaking has.

Oscar-winning director James Cameron’s “Avatar: The Way of Water” returns to Pandora, a far-off exomoon where the peaceful, blue-skinned Na’vi people are menaced by human invaders who are capable of getting into their skin. The film is a visual mind-blower, combining elements of underwater documentaries, video games and the movie that earned Cameron his Oscar: “Titanic.”

The idea of a human taking charge of an alien body via virtual reality is pure science fiction — but if you replace the fictional Na’vi with a robot, you get the premise for the ANA Avatar XPRIZE, which gave out its top awards at the $10 million competition’s finals in November.

In the latest episode of the Fiction Science podcast, we focus on the parallels between the science-fiction vision embodied in the Avatar movies and the future-tech vision that roboticists are pursuing through the Avatar XPRIZE and other efforts. Someday, robotic avatars could well transform space exploration as well as life back here on Earth.

“I recently re-watched the ‘Avatar’ movie, and I was surprised how similar some of the things they were talking about in there were to the work that we were doing,” said Henry Mayne — one of the members of Boston-based Team Northeastern, which won the $1 million third-place prize in the Avatar XPRIZE competition. “I think it’s funny, because we didn’t necessarily have the ‘Avatar’ movie in mind while we were developing the avatar robot, but it just happened to come about in a similar way in the movie. So maybe James Cameron saw into the future.”

The origins of avatars go back to Hindu spiritual tradition, which used the word to refer to divine beings who came down to earth in human or animal form. More recently, the term has been applied to the virtual-reality manifestations of video-game players, or to the portraits linked to social-media accounts. But Jacquelyn Ford Morie, a pioneer in the study of virtual worlds who served as technical adviser for the Avatar XPRIZE, says there’s something lacking in a purely digital avatar.

“What was always missing was the physicality,” Morie told me. “So if we want to send our sense of presence somewhere else, we get a lot of it through digital means, but we don’t get that physicality. And so what we’re looking at with the Avatar XPRIZE and new forms of avatars is that physicality.”

The XPRIZE competition, sponsored by Japan’s All Nippon Airways, was aimed at boosting the development of robotic avatars that could explore an environment under the control of a human operator.

Teams were required to build a robot that was capable of navigating through an obstacle course under remote control, sending back the sights and sounds of an environment, picking up and examining objects it found, and even sending back a virtual-reality sense of touch.

Germany’s Team NimbRo won the $5 million first prize, and France’s Pollen Robotics took the $2 million second prize. Team Northeastern, based at Northeastern University, was the top U.S. finisher out of a field of 17 finalists.

So what lessons were learned? Morie said the teams found out that it’s not necessary for a robotic avatar to have a complete human form. For example, it doesn’t need to have legs, as long as it has some way to get around. But it does need to have a face. “For that human-to-human connection, a face is kind of critical,” Morie said. “We want to see how that person we’re interacting with is responding to us.”

It’s important to keep the human-to-avatar interface as intuitive as possible. “For the person operating the avatar, there has to be a low cognitive load,” Morie said. “They can’t be having to think about every single thing they’re doing. … If they have to think about every little thing, they’re losing that sense of presence at a distance.”

The physical load was also a factor. Morie said the winning team’s robot didn’t look very human. “But when the operator was in it, it was almost weightless … so they didn’t have to worry about operating these big massive arms and getting their muscles fatigued,” she said.

During the XPRIZE finals, operators had to use gloves, pedals and other moving parts to control their avatars mechanically via a wireless communication system. Team Northeastern’s Mayne said a brain-computer interface like the one being developed by Elon Musk’s Neuralink venture would have made operating an avatar easier.

“There are a lot of limitations with coupling to the body — because, say you don’t have functionality in one of your arms. It would be useful if you could still control the avatar without that arm,” he said. “And that’s where the really good use cases for avatars start coming in.”

One of Mayne’s teammates, Rui Luo, said telesurgery could be one of the first applications for emerging avatar technologies. “You would probably want to use such a system to do complex surgeries for patients in different places,” he said.

Another member of Team Northeastern, David Nguyen, said avatars could play a role in “any scenario where you’d want to send somebody, but it’s too dangerous or expensive for that person to go.”

“This avatar could be out somewhere, doing some dangerous work, and get completely smooshed by a boulder or something — and then whoever’s using it is completely fine, but you still get that skill transfer from human intelligence,” Nguyen said.

Eventually, remote-controlled avatars could become caregivers, companions or service workers, Morie said.

“One of the applications that is actually being tested in Japan right now is for people with limited mobility, with physical problems, to inhabit one of these robotic avatars to serve people in cafes,” she said. “People don’t realize that this is a person with limited mobility who is actually behind that robot serving them.”

If controlling avatars became a simple enough task, they could also provide ways to experience far-off destinations or meet up with far-off loved ones through virtual reality. This was a big reason why ANA became a backer of the Avatar XPRIZE, and why the airline spun out a subsidiary called Avatarin.

“We’re looking at a future where you can take your vacation in the tropics and feel the sand in your fingers and smell the ocean,” Morie said. “But that’s going to be a ways out.”

---

• XPRIZE Blog: When we can’t be there in person
• Vignettes from a robotic future, by Jacquelyn Morie

---

When it comes to way-out applications, space exploration is likely to be high on the list, Mayne said.

“It makes sense that avatars would probably be first deployed on space missions,” he said. “This is because it’s very risky to send an astronaut outside a moon base … so if you could have that astronaut experience and have the ability to transfer your skills to the outside of your vessel, it could be really useful in setting up, say, a moon base.”

It might not be practical to have the avatar controller back on Earth, due to the time lag for signals to travel back and forth, but Mayne said the controller could be stationed on an outpost orbiting above.

“You wouldn’t have to waste resources or risk losing your astronaut while you’re setting up the base,” he said. “There were a lot of people at the XPRIZE conference that were interested in that use case.”

So maybe it’s not so crazy to imagine avatars colonizing alien planets, under the control of humans safely stationed somewhere else. Future avatars are more likely to be machines than blue-skinned aliens — but other than that, the long-term vision for telepresence may not be all that much different from the sci-fi vision of “Avatar.”

“Perhaps the biggest difference is, Hollywood is usually 50 years ahead of the technology of today,” said Taskin Padir, who’s the director of Northeastern University’s Institute for Experiential Robotics and one of the faculty advisers for Team Northeastern. “So some of the things that we see in the movie are not within reach yet, in terms of the technology being ready.”

“Avatar: The Way of Water” made its theatrical debut this week and has already earned millions of dollars at the box office. Disney+ is streaming the original “Avatar” movie from 2009. Three more sequels are scheduled for release in 2024, 2026 and 2028.

This item was originally published on Cosmic Log. Stay tuned for future episodes of the Fiction Science podcast via Anchor,  Apple, Google,  Overcast, Spotify, Breaker, Pocket Casts, Radio Public and Reason. If you like Fiction Science, please rate the podcast and subscribe to get alerts for future episodes.