How the Air Force and SpaceX Saved Dragon from Doom

The picture perfect docking of the SpaceX Dragon capsule to the International Space Station (ISS) on March 3 and the triumphant ocean splashdown last week on March 26 nearly weren’t to be – and it all goes back to a microscopic manufacturing mistake in the oxidizer tank check valves that no one noticed long before the vessel ever took flight.

Barely 11 minutes after I witnessed the spectacular March 1 blastoff of the Dragon atop the SpaceX Falcon 9 rocket from Cape Canaveral, Florida, everyone’s glee suddenly turned to disbelief and gloom with the alarming news from SpaceX Mission Control that contact had been lost.

I asked SpaceX CEO and founder Elon Musk to explain what caused the failure and how they saved the drifting, uncontrolled Dragon capsule from doom – just in the nick of time.

Applying the space version of the Heimlich maneuver turned out to be the key. But if you can’t talk to the patient – all is lost.

Right after spacecraft separation in low Earth orbit , a sudden and unexpected failure of the Dragon’s critical thrust pods had prevented three out of four from initializing and firing. The oxidizer pressure was low in three tanks. And the propulsion system is required to orient the craft for two way communication and to propel the Dragon to the orbiting lab complex.

So at first the outlook for the $133 million Dragon CRS-2 cargo resupply mission to the ISS appeared dire.

Then, SpaceX engineers and the U.S Air Force sprang into action and staged an amazing turnaround.

“The problem was a very tiny change to the check valves that serve the oxidizer tanks on Dragon.” Musk told Universe Today

“Three of the check valves were actually different from the prior check valves that had flown – in a very tiny way. Because of the tiny change they got stuck.”

SpaceX engineers worked frantically to troubleshoot the thruster issues in an urgent bid to overcome the serious glitch and bring the crucial propulsion systems back on line.

“What we did was we were able to write some new software in real time and upload that to Dragon to build pressure upstream of the check valves and then released that pressure- to give it a kind of a kick,” Musk told me at a NASA media briefing.

“For the spacecraft you could call it kind of a Heimlich maneuver. Basically that got the valves unstuck and then they worked well”

“But we had difficulty communicating with the spacecraft because it was in free drift in orbit.”

“So we worked closely with the Air Force to get higher intensity, more powerful dishes to communicate with the spacecraft and upload the software to do the Heimlich pressure maneuver.”

Just how concerned was Musk?

“Yes, definitely it was a worrying time,” Musk elaborated.

“It was a little frightening,” Musk had said right after the March 1 launch.

Later in the briefing Musk explained that there had been a small design change to the check valves by the supplier.

“The supplier had made mistakes that we didn’t catch,” said Musk. “You would need a magnifying glass to see the difference.”

SpaceX had run the new check valves through a series of low pressurization systems tests and they worked well and didn’t get stuck. But SpaceX had failed to run the functional tests at higher pressures.

“We’ll make sure we don’t repeat that error in the future,” Musk stated.

Musk added that SpaceX will revert to the old check valves and run tests to make sure this failure doesn’t happen again.

SpaceX, along with Orbital Sciences Corp, are both partnered with NASA’s Commercial Resupply Services program to replace the cargo up mass capability the US lost following the retirement of NASA’s space shuttle orbiters in 2011.

Orbital’s Antares rocket could blast off on its first test mission as early as April 17.

Of course the Dragon CRS-2 flight isn’t the first inflight space emergency, and surely won’t be the last either.

So, for some additional perspective on the history of reacting to unexpected emergencies in space on both human spaceflight and robotic science probes, Universe Today contacted noted space historian Roger Launius, of the Smithsonian National Air & Space Museum (NASM).

Roger provided these insights to Universe Today editor Nancy Atkinson – included here:

“There are many instances in the history of spaceflight in which the mission had difficulties that were overcome and it proved successful,” said Launius.

“Let’s start with Hubble Space Telescope which had a spherical aberration on its mirror and the first reports in 1990 were that it would be a total loss, but the engineers found workarounds that allowed it to be successful even before the December 1993 servicing mission by a shuttle crew that really turned it into a superb scientific instrument.”

“Then what about Galileo, the Jupiter probe, which had a problem with its high gain antenna. It never did fully deploy but the engineers found ways to overcome that problem with the communication system and the spacecraft turned into a stunning success.”

“If you want to feature human spaceflight let’s start with the 1999 shuttle flight with Eileen Collins as commander that had a shutdown of the SSMEs prematurely and it failed to reach its optimum orbit. It still completed virtually all of the mission requirements.”

“That says nothing about Apollo 13,… I could go on and on. In virtually every mission there has been something potentially damaging to the mission that has happened. Mostly the folks working the mission have planned for contingencies and implement them and the public rarely hears about it as it looks from the outside like a flawless operation.”

“Bottom line, the recovery of the Dragon capsule was not all that amazing. It was engineers in the space business doing what they do best,” said Launius.

Ken Kremer

…………….

Learn more about SpaceX, Antares, Curiosity and NASA missions at Ken’s upcoming lecture presentations:

April 20/21 : “Curiosity and the Search for Life on Mars – (in 3-D)”. Plus Orion, SpaceX, Antares, the Space Shuttle and more! NEAF Astronomy Forum, Suffern, NY

April 28: “Curiosity and the Search for Life on Mars – (in 3-D)”. Plus the Space Shuttle, SpaceX, Antares, Orion and more. Washington Crossing State Park, Titusville, NJ, 130 PM

SpaceX Falcon 9 rocket and Dragon capsule poised to blast off from Cape Canaveral Air Force Station, Florida on a commercial resupply mission to the ISS. Credit: Ken Kremer/www.kenkremer.com