Kennedy Space Center – Barely 11 minutes after the spectacular Friday morning, March 1 launch of the SpaceX Falcon 9 rocket and unmanned Dragon capsule bound for the International Space Station (ISS), absolute glee suddenly threatened to turn to total gloom when the mission suffered an unexpected failure in the critical propulsion system required to propel the Dragon to the Earth orbiting outpost.
An alarming issue with the Dragons thrust pods prevented three out of four from initializing and firing.
For several hours the outlook for the $133 million mission appeared dire, but gradually began to improve a few hours after launch.
“It was a little frightening,” said SpaceX CEO Elon Musk at a Friday afternoon media briefing for reporters gathered at the Kennedy Space Center, commenting on the moments after the glitch appeared out of nowhere.
“We noticed after separation that only one of the four thruster pods engaged or was ready to engage,” Musk explained. “And then we saw that the oxidizer pressure in three of the four tanks was low.”
The situation progressed onto the road to recovery after SpaceX engineers immediately sprang into action and frantically worked to troubleshoot the thruster problems in an urgent bid to try and bring the crucial propulsion systems back on line and revive the mission.
By late Saturday afternoon sufficient recovery work had been accomplished to warrant NASA, ISS and SpaceX managers to give the go-ahead for the Dragon to rendezvous with the station early Sunday morning, March 3.
“The station’s Mission Management Team unanimously agreed that Dragon’s propulsion system is operating normally along with its other systems and ready to support the rendezvous two days after Friday’s launch on a Falcon 9 rocket from the Cape Canaveral Air Force Station in Florida,” NASA announced in a statement on Saturday, March 2.
A failure to ignite the thrusters within 1 or 2 days would have resulted in unacceptable orbital decay and a quick and unplanned fiery reentry into the earth’s atmosphere, said Musk.
Reentry would cause a total loss of the mission – carrying more than a ton of vital supplies, science gear, research experiments, spare parts, food, water and provisions to orbit for the stations six man crew.
Shortly after the Dragon achieved orbit and separated from the second stage, the solar arrays failed to deploy and the live webcast stopped prematurely.
During the course of the Friday afternoon briefing, Musk and NASA officials received continuous updates indicating the situating was changing and slowly improving.
Musk confirmed that SpaceX was able to bring all four of Dragon’s thruster pods back up and running. Engineers were able to identify and correct the issue, normalizing the pressure in the oxidation tanks.
The problem may have been caused by stuck valves or frozen oxidizer in the lines. Dragon has four oxidizer tanks and four fuel tanks.
“We think there may have been a blockage of some kind or stuck check valves going from the helium pressure tank to the oxidizer tank,” Musk hypothesized. “Whatever that blockage is seems to have alleviated.”
Three of the four thruster pods must be active before the Dragon would be permitted to dock, said Mike Suffredini, NASA program manager for the ISS. There are a total of 18 Draco thrusters.
SpaceX and the ISS partners conducted a thorough review process to assure that the thrusters will work as advertised and allow the Dragon to safely enter the stations keep out zone and physically dock at the berthing port onto the Earth-facing port of the Harmony module.
“SpaceX said it has high confidence there will be no repeat of the thruster problem during rendezvous, including its capability to perform an abort, should that be required,” NASA said in a statement.
Dragon is now slated to be grappled early Sunday morning at 6:31 a.m. by NASA Expedition 34 Commander Kevin Ford and NASA Flight Engineer Tom Marshburn – that’s one day past the originally planned Saturday morning docking.
Video: Falcon 9 SpaceX CRS-2 launch on March 1, 2013 bound for the ISS – shot from the roof of the Vehicle Assembly Building. Credit: Matthew Travis/Spacearium
NASA says that despite the one-day docking delay, the Dragon unberthing and parachute assisted return to Earth will still be the same day as originally planned on March 25.
There are numerous docking opportunities available in the coming days if SpaceX and NASA determined that more time was needed to gain confidence that Dragon could safely carry out an attempt.
Musk said the Dragon could stay on orbit for several additional months if needed.
We have to review the data with NASA before docking to make sure it’s safe, Musk emphasized on Friday.
The mission dubbed CRS-2 will be only the 2nd commercial resupply mission ever to berth at the ISS. SpaceX is under contract to NASA to conduct a dozen Dragon resupply flight to the ISS over the next few years at a cost of about $1.6 Billion.
NASA TV coverage of rendezvous and grapple on Sunday, March 3 will begin at 3:30 a.m. EST. Coverage of berthing operations on NASA TV will begin at 8 a.m.
SpaceX’s live coverage at http://www.spacex.com/webcast begins at 6:00 a.m. Eastern.
This mission is a testimony to SpaceX’s wisdom at making their systems redundant enough to survive a very serious technical failure. Still the robustness of the thruster design has to be called into question since there was such a serious failure with so few missions launched. I am sure that they will work to discover and fix the bug rather than continue to tolerate it like NASA did with the falling ice/foam problem.
We may not agree, but if a single unexpected blockage was cleared by calling on the remedy of a “pressure hammer” procedure, it seems overly dramatic to say “failure”, much less “very serious technical failure”.
Perhaps judgement can be reserved until the mission is complete and we see a report.
There is that “pressure hammer ” claim again.
Will you use that pressure hammer too when you need to accelerate entering a highway and need speed to accelerate to the minimal highway speed?
A pressure hammer could also damage the oxidizer line or oxidizer injectors.
Again?
These procedures are not likely doing damage, rapid pressurization is something these systems may have to handle.
Specifically here, the pressurant and oxidant systems are separated by the tank that is to be pressurized. (And further mechanically dampened by a separating diaphragm, if I understand the descriptions correctly.)
But be as it may, a non-nominal situation are by necessity treated by non-nominal (if standard) actions. As someone said elsewhere, this mission was hitting nominal values more than most.
The redundancy clearly failed.
If redundancy would have worked then minimal 3 thrusters would have worked.
Currently Dragon is berthing, being able to do that after such a technical failure makes me think the redundancy worked.
I think we have a different understanding of what is meant with redundancy. Redundancy means for me that during critical moments it works reliably.
Fixing the problem with a hammer manually means that you did not need the redundancy anyway. This is ok if you are on route to Pluto and have tons of time and nothing to collide into.
I do not know if the oxidizer lines were redundant, because if they both froze then the redundant line clearly failed. And not 1 time but 3 times (3 Thrusters).
That was more intense than your average movie 🙂
Dragon CRS-2 is now officially berthed.
Here is a link to the best source of information I have found to date about the nature of the problem. Fortunately it seems that it a fix for the problem would be limited to the helium tank and it’s connections to the oxidizer tanks. Space X will just have to determine why the helium froze and devise a fix for it.
http://www.spacesafetymagazine.com/2013/03/02/dragon-glitch-delays-iss-delivery/