A Glitch Caused Curiosity to Freeze in Place. But It’s Better Now

Article updated at 3:40 pm CST, 1/24/20.

NASA’s Curiosity Mars rover experienced a technical glitch last week, causing it to temporarily lose its sense of direction and freeze in its tracks. But the talented rover repair team back on Earth enabled a fix, and Curiosity is now back in action.

“We believe this was a repetition of a specific issue we observed once years ago on the mission,” Andrew Good from JPL’s media office told Universe Today. “During the rover’s execution of a series of standard error-checking steps, the orientation measurement subsystem temporarily failed a single error check during bootup. By design, if all steps haven’t been passed, the rover no longer trusts its orientation knowledge, and some rover motions are precluded until re-enabled by the operations team. This ensures that the rover won’t take any actions that might cause damage to itself. In this event, the rover’s estimate of its orientation remained correct, but this needed to be confirmed by ground operators.”

In a blog post update on January 20, Dawn Sumner, a planetary geologist at UC Davis and Curiosity science team member, wrote, “Partway through its last set of activities, Curiosity lost its orientation. Some knowledge of its attitude was not quite right, so it couldn’t make the essential safety evaluation.”

Specialized fault protection software runs throughout the rover’s modules and instruments (somewhat similar to the ground fault circuit interrupter in your bathroom), and when a problem occurs, the rover stops and sends data called ‘event records’ to Earth. When this happens, Curiosity is programmed not to move until it hears back from Earth.

The event records include images taken of its surroundings that provide details about the nature of the terrain and clues of the rover’s position. Other information sent by the rover during this fault event allowed the team to determine what happened so they could develop a recovery plan.

“The engineers on the team built a plan to inform Curiosity of its attitude and to confirm what happened,” Sumner said in the blog post. In a subsequent post on January 21, MSL team member Scott Guzewich from NASA’s Goddard Space Flight Center wrote that the plan enacted to ensure Curiosity had enough knowledge of its orientation to proceed with arm activities and mobility was successful. Curiosity is now back to its regularly scheduled science activities.

In an email to Universe Today, Sumner she the team at JPL are still analyzing the data, as well as working to prevent a similar problem in the future.

Since the engineering team can’t go to Mars and repair a problem, everything is fixed either by sending software updates to the rover or by changing operational procedures. Over the years since Curiosity landed on Mars in August of 2012, the rover team has upgraded the rover’s software to provide much more efficiency, fault protections, and system robustness.  

Detailed in Emily Lakdawalla’s excellent book, “The Design and Engineering of Curiosity: How the Mars Rover Performs Its Job,” Curiosity has two redundant sets of avionics controlling all of its functions, referred to as the A-side and B-side. Two redundant rover power analog modules (RPAMs) function like the rover’s cerebellum, controlling all of its essential life support functions: power distribution, system fault protection, and wakeups/shutdowns.

This recent event is not the first time the rover team has had to work through problems. For example, as early as the rover’s 200^th day on Mars, a rover had a problem with its flash memory on the A-side, and the rover couldn’t properly shut down for the day. In order not to deplete the batteries, the rover team worked around the problem by instructing the A-side computer not to use half of its flash memory.

“The software was updated to handle these conditions more gracefully,” Lakdawalla wrote. “The rover has used the B-side rover compute element as its primary computer ever since. Engineers patched the flight software to return the A-side computer to service as a reliable backup after sol 772.”

During the seven-and-a-half-year mission, Curiosity has overcome other issues such as a short in the electronics for its drill, problems with the wheels and other memory problems.

“It’s really impressive how well the team can diagnose and recover from glitches in rover operations on another planet,” Sumner told Universe Today. “I have immense respect for our engineering team. In particular, they have really effective processes for working together to identify the best path forward when faced with something unknown.”

Sumner added that when she has sat through the engineering team’s discussions, she has been fascinated by how they share data, create hypotheses, challenge each other’s assumptions, and focus on solving the problem, identifying the uncertainties, and deciding what action to take.

The ingenuity of the team and the resilience of the rover has allowed the mission to be so successful for so long, allowing the rover to be the eyes and hands for an international team of about 500 earthbound scientists. Their goal is to figure out how Mars evolved over billions of years and determine if it once was — or even now is — capable of supporting microbial life.

Curiosity is currently climbing a 3.4 mile (5.5 km) -high Mars mountain scientists call Mt. Sharp (formally known as Aeolis Mons) that sits in the middle of Gale Crater, a 96-mile (155-km) diameter impact basin.

Follow more mission updates at NASA’s Curiosity mission update site.