A erospace s a fet y a dv isory pa n e l

Vice Admiral Joseph W. Dyer, USN (Ret.), Chair

Dr. James P. Bagian Aerospace Safety
John C. Frost
Deborah L. Grubbe, P.E.
Advisory Panel
John C. Marshall
Annual Report
Joyce A. McDevitt, P.E.
for 2011
Dr. Donald P. McErlean
Dr. George C. Nield
 NASA AEROSPACE SAFETY ADVISORY PANEL
National Aeronautics and Space Administration
Washington, DC 20546
VADM Joseph W. Dyer, USN (Ret.), Chair

January 25, 2012

The Honorable Charles F. Bolden, Jr.

Administrator
National Aeronautics and Space Administration
Washington, DC 20546

Dear Mr. Bolden:

Pursuant to Section 106(b) of the National Aeronautics and Space Administration Authorization Act of 2005 (P.L. 109-155),
the Aerospace Safety Advisory Panel (ASAP) is pleased to submit the ASAP Annual Report for 2011 to the U.S. Congress
and to the Administrator of the National Aeronautics and Space Administration (NASA).

This report is based on the Panel’s 2011 fact-finding and quarterly public meetings; “insight” visits and meetings; direct
observations of NASA operations and decision-making; discussions with NASA management, employees, and contractors;
and the Panel members’ past experiences.
In our report we highlight issues on cost, schedule, resources, requirements and acquisition strategy that impact safety. Further,
we again note the uncertainty regarding goals and objectives and the need for clarity and constancy of purpose. Importantly,
we also acknowledge several of NASA’s many accomplishments during calendar year 2011. We observe that transparency, the
evolution of a safe and open culture, and key process advances have all significantly improved under your leadership.

A key and honest question that we pose is: “How safe is safe enough?” The pursuit of great reward often comes hand in hand
with great risk—so it has always been with explorers. The answer to the question must come from a balance between risk
and reward and should reflect a consensus among the American people, the White House, the Congress, and NASA. It is not
our intent or purpose to answer the question; however, we point to areas where we believe the stated requirement may not
produce the requisite safety. We especially invite your attention to the section “Reassessment of Space Shuttle Risks” and
the historical gap between anticipated and deployed systems risk.

In this report, we have assumed that the purpose of the commercial crew initiative is to facilitate the near-term development
of a U.S. commercial space transportation capability to achieve safe, reliable, and cost-effective access to and from low-
Earth orbit (LEO). However, some among the stakeholders believe human transport to LEO is not the primary intent of the
commercial crew initiative. They believe the intent is, instead, to foster a domestic U.S. space industry over a longer time
horizon. We note that attention to and investment in safety are critical in developing near-term transport to LEO but may not
be as significant in seeding a future industry. Some of the funding decisions and the resultant shifts in acquisition strategy
give credence to those who believe the objective has indeed changed. We believe clarity is needed, and constancy of purpose
must follow in either regard. We strongly believe those setting national strategy, providing resources, and planning for
execution must all share in acknowledging and shouldering the risk. To speak more plainly, if NASA attempts to execute an
underfunded program to an unrealistic schedule, the accountability for accepting the associated risk must be shared.
NASA’s senior leaders and staff members offered significant cooperation to support the completion of this document. I
therefore submit the ASAP Annual Report for 2011 with respect and appreciation.

Sincerely,

VADM Joseph W. Dyer, USN (Ret.)

Chair, Aerospace Safety Advisory Panel

Enclosure
 NASA AEROSPACE SAFETY ADVISORY PANEL
National Aeronautics and Space Administration
Washington, DC 20546
VADM Joseph W. Dyer, USN (Ret.), Chair

January 25, 2012

The Honorable Joseph R. Biden, Jr.

President of the Senate
Washington, DC 20510

Dear Mr. President:

Pursuant to Section 106(b) of the National Aeronautics and Space Administration Authorization Act of 2005 (P.L. 109-155),
the Aerospace Safety Advisory Panel (ASAP) is pleased to submit the ASAP Annual Report for 2011 to the U.S. Congress
and to the Administrator of the National Aeronautics and Space Administration (NASA).

This report is based on the Panel’s 2011 fact-finding and quarterly public meetings; “insight” visits and meetings; direct
observations of NASA operations and decision-making; discussions with NASA management, employees, and contractors;
and the Panel members’ past experiences.
In our report we highlight issues on cost, schedule, resources, requirements and acquisition strategy that impact safety.
Further, we again note the uncertainty regarding goals and objectives and the need for clarity and constancy of purpose.
Importantly, we also acknowledge several of NASA’s many accomplishments during calendar year 2011. We observe that
transparency, the evolution of a safe and open culture, and key process advances have all significantly improved under
NASA Administrator Charles F. Bolden, Jr.’s leadership.

A key and honest question that we pose is: “How safe is safe enough?” The pursuit of great reward often comes hand in hand
with great risk—so it has always been with explorers. The answer to the question must come from a balance between risk
and reward and should reflect a consensus among the American people, the White House, the Congress, and NASA. It is not
our intent or purpose to answer the question; however, we point to areas where we believe the stated requirement may not
produce the requisite safety. We especially invite your attention to the section “Reassessment of Space Shuttle Risks” and
the historical gap between anticipated and deployed systems risk.

In this report, we have assumed that the purpose of the commercial crew initiative is to facilitate the near-term development
of a U.S. commercial space transportation capability to achieve safe, reliable, and cost-effective access to and from low-
Earth orbit (LEO). However, some among the stakeholders believe human transport to LEO is not the primary intent of the
commercial crew initiative. They believe the intent is, instead, to foster a domestic U.S. space industry over a longer time
horizon. We note that attention to and investment in safety are critical in developing near-term transport to LEO but may not
be as significant in seeding a future industry. Some of the funding decisions and the resultant shifts in acquisition strategy
give credence to those who believe the objective has indeed changed. We believe clarity is needed, and constancy of purpose
must follow in either regard. We strongly believe those setting national strategy, providing resources, and planning for
execution must all share in acknowledging and shouldering the risk. To speak more plainly, if NASA attempts to execute an
underfunded program to an unrealistic schedule, the accountability for accepting the associated risk must be shared.
NASA’s senior leaders and staff members offered significant cooperation to support the completion of this document. I
therefore submit the ASAP Annual Report for 2011 with respect and appreciation.

Sincerely,

VADM Joseph W. Dyer, USN (Ret.)

Chair, Aerospace Safety Advisory Panel

Enclosure
 NASA AEROSPACE SAFETY ADVISORY PANEL
National Aeronautics and Space Administration
Washington, DC 20546
VADM Joseph W. Dyer, USN (Ret.), Chair

January 25, 2012

The Honorable John A. Boehner

Speaker of the House of Representatives
Washington, DC 20510

Dear Mr. Speaker:

Pursuant to Section 106(b) of the National Aeronautics and Space Administration Authorization Act of 2005 (P.L. 109-155),
the Aerospace Safety Advisory Panel (ASAP) is pleased to submit the ASAP Annual Report for 2011 to the U.S. Congress
and to the Administrator of the National Aeronautics and Space Administration (NASA).

This report is based on the Panel’s 2011 fact-finding and quarterly public meetings; “insight” visits and meetings; direct
observations of NASA operations and decision-making; discussions with NASA management, employees, and contractors;
and the Panel members’ past experiences.
In our report we highlight issues on cost, schedule, resources, requirements and acquisition strategy that impact safety.
Further, we again note the uncertainty regarding goals and objectives and the need for clarity and constancy of purpose.
Importantly, we also acknowledge several of NASA’s many accomplishments during calendar year 2011. We observe that
transparency, the evolution of a safe and open culture, and key process advances have all significantly improved under
NASA Administrator Charles F. Bolden, Jr.’s leadership.

A key and honest question that we pose is: “How safe is safe enough?” The pursuit of great reward often comes hand in hand
with great risk—so it has always been with explorers. The answer to the question must come from a balance between risk
and reward and should reflect a consensus among the American people, the White House, the Congress, and NASA. It is not
our intent or purpose to answer the question; however, we point to areas where we believe the stated requirement may not
produce the requisite safety. We especially invite your attention to the section “Reassessment of Space Shuttle Risks” and
the historical gap between anticipated and deployed systems risk.

In this report, we have assumed that the purpose of the commercial crew initiative is to facilitate the near-term development
of a U.S. commercial space transportation capability to achieve safe, reliable, and cost-effective access to and from low-
Earth orbit (LEO). However, some among the stakeholders believe human transport to LEO is not the primary intent of the
commercial crew initiative. They believe the intent is, instead, to foster a domestic U.S. space industry over a longer time
horizon. We note that attention to and investment in safety are critical in developing near-term transport to LEO but may not
be as significant in seeding a future industry. Some of the funding decisions and the resultant shifts in acquisition strategy
give credence to those who believe the objective has indeed changed. We believe clarity is needed, and constancy of purpose
must follow in either regard. We strongly believe those setting national strategy, providing resources, and planning for
execution must all share in acknowledging and shouldering the risk. To speak more plainly, if NASA attempts to execute an
underfunded program to an unrealistic schedule, the accountability for accepting the associated risk must be shared.
NASA’s senior leaders and staff members offered significant cooperation to support the completion of this document. I
therefore submit the ASAP Annual Report for 2011 with respect and appreciation.

Sincerely,

VADM Joseph W. Dyer, USN (Ret.)

Chair, Aerospace Safety Advisory Panel

Enclosure
.
Aerospace Safety Advisory Panel

I. Introductory Remarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
A. The Aerospace Safety Advisory Panel . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
B. ASAP Observations on NASA Accomplishments in 2011 . . . . . . . . . . . . . . . . . . . . . . . . . . 1
1. Shuttle Program Completion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
2. International Space Station (ISS) Assembly Complete . . . . . . . . . . . . . . . . . . . . . . . . . . . 1
3. Noteworthy Launches . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
4. Public Communications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2
5. Progress on ASAP Recommendations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

II. Issues and Concerns . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3

A. Human Spaceflight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
1. International Space Station . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
2. Soyuz Return to Flight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3. Commercial Crew . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6
4. Space Launch System (SLS)/Multi-Purpose Crew Vehicle (MPCV) . . . . . . . . . . . . . . . . . . . 7
B. Risk Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
1. Reassessment of Space Shuttle Risks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8
2. How Safe Is Safe Enough? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
3. Preparing for Future Low-Probability Events . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10
4. Knowledge Management . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11
5. Facilities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12
C. Transparency/Culture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
1. Insight/Oversight . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13
2. Timely and Accurate Communications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15
3. Alcohol Use and Testing Policy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

III. Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16
Appendix: Summary and Status of ASAP 2011 Recommendations . . . . . . . . . . . . . . . . . . . . 19

CD Table of Contents
Attachment 1: Charter of the Aerospace Safety Advisory Panel
Attachment 2: ASAP 2011 Recommendations, NASA Responses, and Status
Attachment 3: ASAP 2011 Quarterly Meeting Minutes
Attachment 4: 2011 Activities of the Aerospace Safety Advisory Panel
Attachment 5: Aerospace Safety Advisory Panel Members and Staff

Annual Report for 2011 v

.
Aerospace Safety Advisory Panel

I. Introductory Remarks
A. The Aerospace Safety Advisory Panel (ASAP)
The ASAP was established by Congress in 1968 to provide advice and make recommendations to the
NASA Administrator on safety matters.1 The Panel holds quarterly fact-finding and public meetings and
makes one or more “insight” visits per year to NASA Field Centers or other related sites. It reviews safety
studies and operations plans and advises the NASA Administrator and Congress on hazards related to
proposed or existing facilities and operations, safety standards and reporting, safety and mission assurance
aspects on ongoing or proposed programs, and NASA management and culture related to safety. Although
the Panel may perform other duties and tasks as requested by either the NASA Administrator or Congress,
the ASAP members normally do not engage in specialized studies or detailed technical analyses.
This report highlights the issues and concerns that were identified or raised by the Panel during its activities
over the past year. The Panel recommendations submitted to the Administrator during 2011 are summarized
in the Appendix at the end of this report.2 They are based upon the ASAP fact-finding and quarterly public
meetings; “insight” visits and meetings; direct observations of NASA operations and decision-making;
discussions with NASA management, employees, and contractors; and the Panel members’ expertise.

B. ASAP Observations on NASA Accomplishments in 2011

1. Shuttle Program Completion
NASA safely concluded the Space Shuttle Program with the landing of the orbiter Atlantis on July 21.
This historic program and its vehicles have inspired people around the world and flown more people to
space than any other spacecraft. The five vehicles have been workhorses for NASA and the international
community for the past 30 years. They spent a combined total of 1,332 days in space and completed
21,152 Earth orbits covering 548.2 million miles. All five orbiters deployed a total of 66 satellites and
rendezvoused with the International Space Station (ISS) 37 times. They carried a total of 355 individual
astronauts and cosmonauts into space, many of whom flew multiple times, bringing the total number of
“crewmember transports” to 833. Although the Shuttle Program came to an end in 2011, its legacy and the
experience of those who worked on it will benefit future human exploration programs for years to come.

2. International Space Station (ISS) Assembly Complete

With the installation of the Alpha Magnetic Spectrometer II in May, the assembly and outfitting of
the U.S. On-orbit Segment of the ISS was completed, allowing the opportunity for full utilization of
its research capabilities. The ISS represents an unparalleled engineering feat—construction of a highly
complex spacecraft with components built in many nations, launched from four different space centers,
and assembled on orbit by over 160 astronaut spacewalks. It has been visited by more than 200 people
and has been continuously crewed for over 11 years. The ISS provides a valuable laboratory for research
related to exploration requirements and human space exploration.

1. The ASAP Charter is included as Attachment 1 on the enclosed CD.

2. The full text of all the 2011 recommendations is included as Attachment 2 on the enclosed CD.

Annual Report for 2011 1

 Aerospace Safety Advisory Panel

3. Noteworthy Launches
NASA safely and successfully launched several important robotic missions in 2011:
• Aquarius was launched on June 9 from Vandenberg Air Force Base (AFB) on a Delta II. The joint
U.S.-Argentine mission will map ocean surface salinity, which is critical to understanding the
water cycle and ocean circulation—two major components of Earth’s climate system.
• The Juno probe to Jupiter, the second mission in NASA’s New Frontiers Program, was launched
on August 5 from Cape Canaveral on an Atlas V. Juno will take 5 years to reach the planet and
will be the first spacecraft to orbit Jupiter since the Galileo probe was de-orbited in 2003.
• The Gravity Recovery and Interior Laboratory (GRAIL), a two-spacecraft Discovery mission, was
launched on September 10 from Cape Canaveral on a Delta II. GRAIL will be used to study the
Moon’s gravitational field and learn more about its internal structure and thermal evolution.
• The National Polar-orbiting Operational Environmental Satellite System (NPOESS) Preparatory
Project (NPP) was launched on October 28 from Vandenberg AFB on a Delta II. NPP is the first of
a new generation of satellites that will observe many facets of our changing Earth.
• The Mars Science Laboratory, with the Curiosity Rover, was launched on November 26 from Cape
Canaveral on an Atlas V on its quest to determine if Mars is, or ever was, capable of supporting
microbial life. Curiosity is scheduled to arrive at Mars in August 2012.

4. Public Communications
Public communications on two potentially negative events were handled very well. In March, the Glory
spacecraft failed to reach orbit after launch on a Taurus XL from Vandenberg AFB. NASA’s Office of
Communications released a thorough message very quickly; several NASA personnel were on television
and radio discussing the mishap scenario and presenting the information that was available at that time.
There were frequent updates throughout the day. In September, 6 years after the end of its productive
scientific life, the Upper Atmosphere Research Satellite (UARS) broke into pieces during reentry, and
most of it burned up in the atmosphere. NASA’s chief scientist for orbital debris discussed the reentry
process and status on NASA’s website, and NASA’s Office of Communications was in close contact with
the media. From the time that it appeared that UARS would be de-orbiting, NASA did an outstanding
job of keeping the public informed on the satellite’s status and associated risks in an exemplary manner
that has not always been the case with other programs. This was done by not only supplying very
technical numerical analyses, but also by providing comparative real-life analogies to other risks to
which people are exposed in their everyday life; thus, people could make meaningful comparisons.

5. Progress on ASAP Recommendations

NASA has continued to show good progress responding to ASAP recommendations. For 2010, 27
recommendations were closed out during the year, resulting in 30 recommendations open at the end of
the year. For 2011, 25 recommendations were closed out during the year, resulting in 18 open at the end
of the year. In 2011, the ASAP generated 13 recommendations (9 fewer than in 2010), of which 7 have
already been closed. There are still 12 open recommendations from years prior to 2011.

2 Annual Report for 2011

Aerospace Safety Advisory Panel

II. Issues and Concerns

Note on color bars: Red highlights what the ASAP considers to be a long-standing concern or
an issue that is not being adequately addressed by NASA. Yellow highlights an important ASAP
concern or issue, but one that is currently being addressed by NASA. Green indicates a positive
aspect or a concern that is being adequately addressed by NASA but continues to be followed by the
Panel. A heading with no color bar represents an issue or concern that is relatively new and that the
ASAP will be addressing in the upcoming year.

A. Human Spaceflight
Calendar year 2011 saw the Space Shuttle era draw to a close with the successful completion of the
STS-135 mission. The human spaceflight safety issues that were related to the Space Shuttle Program
are now no longer NASA’s most pressing concerns; however, there still are a number of areas related
to the ISS, Soyuz, the Commercial Crew Program (CCP), and potentially the Space Launch System
(SLS) that require continuing attention. The areas of particular interest include Micro-Meteoroid and
Orbital Debris (MMOD) risk to the ISS, plans regarding ground impact risks after de-orbit at the end
of the ISS’s useful life, and the risks of inadequate resources for programs under development. Risk is an
unavoidable component of any program. A critical characteristic of a successful program is the ability
to identify hazards and their risk of causing harm; comparing that to the level of risk allowable by the
program, the Agency, and the Congress; and prioritizing the work to mitigate the known risks to an
acceptable level.
In this Report, the ASAP provides specific comments on the ISS, Soyuz, the CCP, and the SLS/Multi-
Purpose Crew Vehicle (MPCV).

1. International Space Station

NASA had great success in completing the ISS construction. The technical and organizational
challenges that had to be overcome to achieve this result are often not appreciated and can lead to a
perception that this success was easily achieved. Such a misperception can cause complacency in regard
to the hazards and associated risks that are always present, such as MMOD. Failing to realize the hostile
environment in which the ISS must operate and not giving adequate attention to these hazards can
result in an unintended compromise to crew safety. Constant vigilance and a continual attention to
detail are essential to the safe sustainment of crew operations on the ISS.
Analyses presented to the ASAP on several occasions, most recently in May 2011, stated that the
probabilistic risk assessment (PRA) related to ISS Loss of Mission (LOM) was 1 in 55 for a 180-day
mission. Since there are approximately 20 180-day missions in the currently projected ISS Program,
this means that there is a greater-than-30-percent chance that the ISS could sustain a LOM sometime
during its projected operating life. Even though PRA numbers have uncertainties associated with them,
one cannot escape the conclusion that the risk of an ISS LOM is more than an outside possibility.
Should such an event occur, the result could arise that the ISS would have to be abandoned—potentially
without the possibility of a return to nominal operation. Thus, a premature ISS de-orbit is one potential

Annual Report for 2011 3

 Aerospace Safety Advisory Panel

outcome. This can occur in either a controlled or an uncontrolled manner, with the latter clearly the
more dangerous. While this possibility has been known for some time, NASA has not yet shared with
the Panel an explicit plan to deal with this situation. ISS End of Life (EOL) is inevitable, and the ISS
will ultimately de-orbit; therefore, it is not too soon for EOL planning to begin in earnest and its
consequences to be understood by all stakeholders. Action plans, contingent on various circumstances,
should be created and shared with all agencies responsible for executing some part of that plan. This
would especially be true should resources (such as Soyuz) be required to complete the action plan.
The lack of advanced planning was illustrated by the activities surrounding the Russian Progress failure
that is described more fully in the next section of this report. This failure necessitated major schedule
changes due to the delay of Soyuz launches to the ISS. While NASA had always recognized a possible
interruption of scheduled Soyuz availability and, from a safety perspective, there was no immediate
impact on the crew, the subsequent flurry of activity showed a lack of maturity in the planning to
handle this eventuality. The potential loss of Soyuz availability—to bring new crew to the ISS as well
as to provide vehicles to be available 24/7 as potential “lifeboats”—raised the very real potential that
the ISS may have had to be abandoned. Fortunately, in this instance, time was available to create the
required plan; however, had this been an MMOD hit or other more immediate hazard, there may not
have been this luxury of time.
It is a foregone conclusion that at some time in the future, the ISS will have to be de-orbited. This fact
strongly supports the argument that NASA should have detailed plans already worked out with the
international partners on how this de-orbit will be accomplished. The hazards and risk implications
for those on the ground must be analyzed, and the resulting conclusions and plans must be available to
mitigate those risks.

2. Soyuz Return to Flight

On August 24, the third-stage engine of a Russian Progress cargo vehicle failed to operate properly,
and the vehicle and its supplies were destroyed. The Progress cargo vehicle, which carried no crew,
is closely related to the Soyuz and is used to resupply the ISS with dry cargo, propellant, water, and
oxygen. It is also used to boost the ISS orbit and control Station orientation. The launch vehicle
involved is used for both Progress and Soyuz spacecraft and is a time-tested design that has been
flying for many decades.
Since the launch vehicle is used for both Progress and Soyuz, a detailed failure investigation was ordered
to be completed prior to any crew being transported on Soyuz flights. Our Russian partners formed
a commission to investigate the anomaly and, as has been the case with previous investigations, kept
NASA well informed about the progress and the review results.
In September, the ISS partners baselined a new Progress and Soyuz flight plan based on the results of the
Russian Accident Board (called a State Commission) that was chartered to investigate the root cause of
the Progress failure and to recommend recovery and remediation activities. The report on the accident
investigation was fully presented to members of the ASAP (among other participants) in a daylong
briefing by Marshall Space Flight Center (MSFC) and a discussion session held at Johnson Space Center

4 Annual Report for 2011

Aerospace Safety Advisory Panel

(JSC). Following the detailed discussion on the Russian investigation and its conclusions, the MSFC
team then explained its independent Risk/Failure Tree analysis and computer modeling. This was very
thorough, especially considering the fact that the team’s knowledge of the system and its history is not as
detailed as the Russian Commission’s. However, regardless of some obvious differences in background
on the system, they were able to replicate all of the critical Russian results.
Both teams concluded that the most likely event was a “quality escape” resulting in debris entering
and clogging the fuel system. This result was based on the investigation and considerable history of
this engine (some 2,000 engine runs) without ever seeing this failure before. This conclusion was also
supported by audits on the assembly, build, and test process. In short, the MSFC team agreed with the
conclusions of the Russian team and felt that the failure scenario was plausible. MSFC was able to reach
its conclusions independently and also felt that a quality escape was the most probable cause. The MSFC
team concluded that the Russians were on track to put into place measures to mitigate any recurrence.
In the ASAP’s view, the two teams did an exemplary job examining the cause of the Progress accident.
The Russians treated this incident very seriously and put the kind of expertise on the team that had
technical knowledge and background as well as the seniority in the Russian system to act and speak
independently. The MSFC team’s work was very impressive, both its analysis of the Russian work and
its independent work.
While no absolutely definitive physical evidence is available to prove the failure cause, the completeness
and competence with which this investigation took place gave the confidence in the subsequent return
to flight decision. The successful Soyuz mission commencing November 14, 2011, took place without
incident. NASA should continue to closely examine Soyuz operations so as to be alert to any information
that might bear on future operational decisions.
It is well to remember that the Soyuz spacecraft, an evolutionary vehicle that has been flying since
1967, currently provides the sole transportation to and from the ISS for the Expedition crews.
Equally important to know is that since Soyuz has the capability to remain docked to the Station for
6-month periods, it provides a “lifeboat” function. This supports the crew by providing an on-orbit
rescue capability in the event of a contingency aboard the ISS. It has been long-standing ISS policy
that the Station can host six crewmembers on long-duration missions only with the availability of
immediate de-orbit capability for the entire on-board crew. Therefore, two Soyuz spacecraft are
required to be docked at Station for the six-person ISS crew. In September of this year, when one of
the docked Soyuz vehicles reached the end of its on-orbit certified life, NASA and Russia made the
decision to return three crewmembers to Earth on Soyuz 26. It must be recognized that there could
be circumstances where the Soyuz 6-month on-orbit limit could result in a de-crewing of the ISS. In
addition, recently projected slippages in the Commercial Crew Program will require renegotiation
with the Russians to provide Soyuz transportation beyond the currently agreed 2016 deadline. Such
renegotiations could be problematic. Failure to renegotiate the agreement could result in a period of
time without U.S. access to the Station. The ASAP will continue its focus on these issues and NASA’s
plans over the next year.

Annual Report for 2011 5

 Aerospace Safety Advisory Panel

3. Commercial Crew
We believe the objective of the commercial crew initiative is to facilitate the development of a U.S.
commercial space transportation capability with the goal of achieving safe, reliable, and cost-effective
astronaut access to and from low-Earth orbit (LEO) and the ISS. (We do not believe the objective is
just to fund and develop a commercial space industry; however, we note that some stakeholders would
debate this.) Once that capability has matured, NASA plans to purchase commercial services to meet
the ISS crew transportation needs. To take maximum advantage of the limited funding available, and
in recognition of the urgent need for this capability, NASA is using nontraditional acquisition and
partnering approaches during the early phases of the program and had planned to switch to a more
standard acquisition process during later (post–critical design review) program phases. Competition is
considered to be a fundamental aspect of the strategy in order to incentivize performance, support cost
effectiveness, and eliminate dependence on a single provider.
Because the 2010 NASA Authorization Act stated that commercially provided services should be used
as the means for ISS crew transportation to the maximum extent practicable, the ASAP has been closely
following NASA’s progress on this program and has requested status updates at each quarterly meeting.
Some program challenges that have been identified to date include making sure that the available funds
are appropriate to the objectives, working toward a realistic schedule, developing the safety certification
processes that will be used, and selecting the proper design targets for safety and reliability.
NASA has recently baselined and published the design requirements in its 1100-series documents.
It has also defined a streamlined control board process that should contribute to expedited decision-
making. To increase its insight into the commercial development efforts, NASA has put in place
Partner Integration Teams that will have the ability to work side by side with the various partners to
understand their approaches and progress. Although such an approach can certainly be beneficial, it will
be important to ensure that the tendency to “over-identify” with the contractor does not result in a lack
of objectivity by the NASA representatives.
According to NASA program managers, the top program risk is inadequate budget; however, without
an accurate NASA estimate of how much it will cost to develop and test a system, it is not possible to
precisely know the program’s budgetary requirements or if there is a risk of underfunding. In this case,
the difficulty is compounded by the fact that the vehicles are being developed by commercial entities,
using nontraditional procurement strategies. It is not surprising that when NASA asked proposers on
the second phase of the Commercial Crew Development (CCDev) Program for rough estimates of
the funds needed to complete the development of a commercial crew system, it received a wide range
of figures from the various companies. NASA is understandably reluctant to publicize the details of
those estimates due to the proprietary nature of the figures in the ongoing competition. Nevertheless, it
appears to the ASAP that the fiscal year (FY) 2012 funding level approved by Congress, which was less
than half of what was requested by the Administration, will not allow commercial crew transportation
to the ISS by 2016. In fact, if the new funding level continues into the future, it is the ASAP’s belief that

6 Annual Report for 2011

Aerospace Safety Advisory Panel

the program is in jeopardy, thus extending the current lack of a U.S. human spaceflight capability and
resulting in no alternative to reliance on Russia to obtain access to the ISS.
The ASAP considers the lack of a credible and appropriately funded plan to develop a U.S. capability to
launch its astronauts to the ISS to be an issue with significant safety implications. If the development
program is continued without adequate funding, it will increase the likelihood that safety-related testing
and modifications to correct any design deficiencies would not be made. Alternatively, terminating the
development program would result in continued reliance on the Russian Soyuz, a system with an
uncertain long-term future.
In mid-December, however, just before this report went to publication, NASA announced plans to
change its acquisition strategy for the integrated design phase of the CCP from a fixed-price, Federal
Acquisition Regulation (FAR)-based contracting approach to one utilizing Space Act Agreements
(SAAs). Previously, NASA had made a strong safety case for using conventional contracting on the
next phase of the CCDev Program, an approach that was viewed as well reasoned and appropriate by
the ASAP. The ASAP acknowledges NASA’s assertion that the change is primarily driven by funding
uncertainties and the need to maintain more than one provider for commercial crew transportation
services. However, we believe that the sudden change in acquisition strategy in an effort to salvage the
CCP may have significantly increased the risk to safety that the previous plan had begun to address. The
lack of the ability to incorporate firm safety requirements using an SAA procurement exposes NASA to
new risks if, at the conclusion of the developmental phase, the proposed designs do not meet minimum
safety requirements. In that event, NASA will have to either (1) expend additional time and money
having the designs modified and retested or (2) accept the risk associated with flying its astronauts on
systems that do not meet the currently articulated minimum safety requirements. If NASA is deciding
to take on more risk because the cost is otherwise prohibitive, then the Agency should be clear about
that increased level of risk acceptance and develop approaches to manage that risk. While it is possible
that NASA can find a way to accomplish the assigned mission with the available budget, at this point in
time the Panel has serious concerns about the likelihood of such an outcome. The ASAP plans to closely
examine the SAA approach in 2012 and will be most interested in the plan for transitioning the designs
into certified systems before their use as crew transport.

4. Space Launch System (SLS)/Multi-Purpose Crew Vehicle (MPCV)

To provide the capability for human exploration beyond Earth orbit, NASA plans to develop the
SLS and the Orion MPCV. The SLS is an advanced, heavy-lift launch vehicle that will incorporate
technological investments from the Space Shuttle Program and the Constellation Program. It will have
an initial 70-metric-ton lift capability and will be evolvable to a 130-metric-ton capability. The first
test flight is targeted for the end of 2017. The Orion MPCV will serve as the primary crew vehicle for
missions beyond LEO and as a backup system for ISS cargo and crew delivery.
Because the SLS and Orion MPCV will be NASA’s primary vehicles for carrying out its exploration
mission over the next several decades, it will be important that they initially be designed to be as safe
and reliable as possible and that they take advantage of the lessons learned during the Space Shuttle

Annual Report for 2011 7

 Aerospace Safety Advisory Panel

and Constellation Programs. The ASAP plans to review the SLS and Orion MPCV programs during
the coming year.
A key question involves the selection of an appropriate mission. For the purposes of determining risk,
should NASA assume the vehicles will be used in a mission to the Moon, to an asteroid, to a Lagrange
point, or to some other destination? Or should the vehicles be designed for all of those missions? Other
areas of interest include reexamining the design targets and thresholds for LOM and Loss of Crew
(LOC), the plan for program control boards, the appropriate magnitude of needed budget margins,
and the potential impacts of a decline in the U.S. aerospace industrial base on long-term logistics and
support. NASA will need to give each of these questions due consideration in 2012. This is not a new
challenge. Prior ASAP reports have highlighted the requirement for clarity and constancy of purpose
regarding goals and objectives for NASA.

B. Risk Management
At least in our lifetime, travel by humans to orbital velocity and beyond and returning to Earth
through our atmosphere will always entail significant risks. The sheer amount of energy required
to reach these velocities and the space environment’s unforgiving nature dictate that extraordinary
efforts must be expended to identify potential hazards and either design them out or provide positive
measures to minimize the probability of their occurrence and control the results if they do occur.
The residual risk that remains is measured by the probability of the various failure scenarios and the
severity of their outcomes. The most serious of these outcomes is known as LOC (loss of crew). In
using a variety of analytical tools to identify, assess, and manage these risks, NASA remains at the
forefront of organizations conducting high-consequence operations. The ASAP feels that the risk
targets must be prudently selected, based on past experience, and explicitly articulated. The foundation
upon which the ultimate assessment must be made is the acceptable level of risk. In other words—how
safe is safe enough?

1. Reassessment of Space Shuttle Risks

By the end of the longest-running human spaceflight program in U.S. history, the Space Shuttle PRA
grew to be a highly refined tool for predicting flight risk. The ASAP asked that an analysis be made of
the lessons learned in predicting risk in a complex space vehicle. The reason the ASAP requested this
analysis is because, as the longest-running human spaceflight program, the Shuttle Program will become
the basis of everything we do in the future. Whatever human spaceflight program proceeds forward, it
is inevitable that people will ask: What did the Shuttle do? It is very important to capture that database.
NASA responded with an outstanding “Space Shuttle Launch and Re-entry Risk Study.” This study
analyzed the many years of Shuttle flights by taking “snapshots” of the various Shuttle configurations
that existed over time and what the flight risk was thought to be for each at the time. Then, based on
what is now known about actual failures and the failure mechanisms that were there all along but had
not yet manifested themselves, a revised assessment of the true risk on each flight was calculated. The
results of this analysis are depicted on the following page.

8 Annual Report for 2011

Aerospace Safety Advisory Panel

SPACE SHUTTLE PROGRAM

Space Shuttle Safety and Mission Assurance Office
NASA Johnson Space Center, Houston, Texas

RESULTS SUMMARY
• SSME risk increase due to higher power level
• APU risk reduction post STS-9 (process improvement) • MMOD risk reduction due to addition of late inspection
• Orbiter flight software using OI-2 • Ascent debris risk reduction improved debris
• Ejection Seats Disabled
environment and improved repair
• APU risk reduction post STS-9 (re-design) • SSME uncontained risk reduction with Block II engine
0.12 • Orbiter flight software using OI -7 with AHMS

1:10 1:10 • SRM risk reduction post Challenger

• SSME risk reduction with Phase II engine
1:10 • Orbiter flight software using OI-8B
• Ascent debris and TPS Debond risk reduction
0.1 with inspection, repair and crew rescue
• Orbiter flight software risk using OI-30
1:12 • Ascent Debris risk increase due to new ET
foam application process
0.08 • Orbiter flight software using OI-26
• SSME Risk reduction with Block IIA engines
• Ascent debris risk
Probability

1:17 reduction from SRB • Ascent debris risk reduction due to

0.06 nose cap TPS re- venting holes in ET foam
design post STS -27
1:21 1:21 • Orbiter flight software using OI -26B

• Orbiter flight software using OI-21 • SSME risk slight increase with
• Risk reductions due to IAPU Block II engines
0.04 • Orbiter flight software using OI-29
1:36 1:37 1:38
1:47 1:47
0.02 • SSME Risk reduction with
Block I & IA engines 1:73 1:90
• Orbiter flight software
using OI-24
0
1 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100 105 110 115 120 125 130
STS-1 STS-41B STS-51L, STS-26 and STS-49 STS-77 STS-86 STS-103 STS-110 STS-114 STS-133
STS-5 STS-29 STS-89

Flight Sequence #

The ASAP is very pleased with NASA’s work in this area. Many things were learned through this
analysis. One key finding was that the risk on a new system that has not been flown before and thus
has not been through the rigors of real-life flight is probably much higher than what the initial risk
assessments show. The reason for this difference is that at the beginning of operations, all the failure
mechanisms are not fully known. In the language of risk analysis, such unknown failure mechanisms
are often called “unknown-unknowns.” In the Shuttle’s case, the first flight risk as now retrospectively
calculated was in actuality 1 in 12 for LOC, yet at least one analysis that existed at the time of the
initial launch estimated the risk to be 1 in 1,000 or better. In other words, the system was almost 100
times more dangerous than the early analysis indicated. This type of disparity must be remembered
when future targets for reliability and LOC numbers are chosen for new programs. One thing that has
always been said in the design business is that engineering design standards take care of the “knowns”;
factors of safety take care of the “known-unknowns”; and margin is what takes care of the “unknown-
unknowns.” A significant margin for error should be allowed for the unknown-unknowns as well as to
create a robust design.

Annual Report for 2011 9

 Aerospace Safety Advisory Panel

In any discussion of spaceflight risk, the perceived versus actual risks experienced by the Space Shuttle
clearly should be taken into account. As already mentioned, the Space Shuttle, unbeknownst to the
team at the time, started at a LOC risk level of 1 in 12, and there was a 92-percent chance that a crew
would be lost in the first 25 missions. By constant improvements, that risk was lowered to 1 in 90 by the
last flight, which is still a high number compared to many endeavors.

2.How Safe Is Safe Enough?

The ASAP applauds the overall review undertaken by NASA to establish a new methodology to set safety
risk tolerances for human spaceflight. The acceptable mission risk for LOC is now to be expressed in
terms of three levels: (1) the Agency acquisition threshold, which is the highest risk level to be tolerated
by the Agency; breaching this level would normally result in program cancellation; (2) the Program
Design/Mission Requirement risk level, which is the “build to” level and is somewhat more conservative
than the Agency threshold to allow a margin of buffer; and (3) the Agency long-term maturity goal,
which includes continuous-improvement upgrades and represents the long-term mission goal.
While the Panel applauds the effort to establish safety thresholds, we are concerned that the specific
levels chosen by NASA for these criteria unfortunately are significantly less conservative than those that
were being used for the now-cancelled Constellation Program. For example, the Exploration Program
requirement for probability of LOC on an ISS mission has changed from 1 in 1,000 to 1 in 270. This
new Agency criterion for future human spaceflight missions is less than one-third as safe as the old
criterion. This is especially worrisome considering the fact that the criterion only considers the risks that
are already known, not the always-present hazards that have not yet been discovered. This observation is
compounded by the fact that recent detailed analysis on Shuttle, as noted above, revealed that the initial
flights were not nearly as safe as predicted. Thankfully, those flights did not result in crew loss, but the
risk they posed illustrates a profound problem. When estimating probabilities of failure in areas where
there is no history, limited experience, and only a partial understanding of what can go wrong, analysts
tend to produce optimistic numbers. If a design process is initiated using a high value of acceptable loss
criteria, this tendency is exacerbated by setting goals too low and hence creates a larger potential for
failure than might be anticipated. The ASAP continues to recommend that NASA reconsider its criteria
for future human spaceflight.

3. Preparing for Future Low-Probability Events

While NASA has historically focused great attention on meticulous preparation for upcoming events
such as launches, Space Station assembly, and exploration on other planets, the Panel has noted a less
aggressive preparation effort for some low-probability events that can be postulated to occur at some
future time. One example is the present lack of a fully vetted, detailed procedure for emergency ISS
de-crewing as noted in the Human Spaceflight section of this report. The eventual safe de-orbiting of
the ISS at EOL or a potential catastrophic failure are future events that the ASAP believes have not been
given sufficient consideration and planning. The Panel continues to encourage NASA to expend the
effort required now to prepare for such future events.

10 Annual Report for 2011

Aerospace Safety Advisory Panel

In order for risk to be managed appropriately at any level, but especially at a programmatic level, the
various component risks and their overall impact must be clearly communicated. A critical question
involves not only what technical information should be communicated, but also what entities need
to receive this communication and the manner in which the information is communicated. If this
clear communication of risk is not accomplished, then safety, appropriate use of resources, and overall
confidence and support of programs and NASA can be negatively impacted.
Examples— UARS and ISS
There are examples where this has been done in productive and less than productive ways. As noted in
Section I of this Report, NASA communicated the UARS reentry hazard in a multipronged manner
and did a very good job in communicating the level of risk and explaining that real damage or injury
could result. The proactive, well-thought-out and -executed communication of risk was accomplished
well in advance of the actual reentry event and ultimately resulted in an appropriate public response
rather than an emotional alarmist response.
In the case of the ISS, however, ASAP feels this level of proactive, clear communication of risk has
not been accomplished. As noted earlier, NASA provided information to ASAP regarding the risk of
LOM for ISS at a level of 1 in 55 for a single, 180-day mission. The ASAP thinks that it would be more
informative to state the risk as a greater-than-30-percent chance that the ISS could sustain a LOM
sometime during its projected operating life. LOM is not an inconsequential risk, and it has not been
openly communicated either inside or outside of NASA. The point here is not the 1 in 55 number,
but that it may not be understood in the same manner as expressing it as an approximately 30-percent
chance of LOM during the ISS’s currently projected life. Failure to clearly communicate this level of risk
in a manner that the various stakeholders conceptually understand can result in an inappropriate loss of
support and confidence in NASA in the event that an ISS LOM occurs. For example, it could be argued
that in the case of the Challenger mishap, the failure to communicate openly and effectively about risk
undercut the confidence and support for NASA that might otherwise have been present.
Attempting to execute programs with insufficient funding often leads to compromise detrimental to
safety. Therefore, we believe transparency is essential if NASA, the Congress, and the White House are
to collectively shoulder the risk and the responsibility.

4. Knowledge Management
Knowledge management is the collection of processes that govern the creation, dissemination, and
utilization of knowledge. In this discussion, the ASAP is using the following definition of “knowledge”:
the ability, capability, and willingness to assemble information in such a way as to advance learning,
improve on current mechanisms, and advance civilization. Knowledge management takes time, effort,
expertise, and the willingness to be curious, search, think, and experiment. It is a nontrivial, critical
task that must be undertaken by serious personnel who are competent in the process. The ASAP has
observed such personnel within the NASA Centers and compliments the personnel and the programs at
JSC and Goddard Space Flight Center, for example. In these locations (and perhaps others), competent
and enthusiastic personnel are making excellent progress at both cataloging and managing NASA

Annual Report for 2011 11

 Aerospace Safety Advisory Panel

critical knowledge. We have also seen excellent examples in regard to specific programs; for example, the
Constellation Program just published a two-volume report on lessons learned, and the Shuttle Program
has an equivalent program in process.
These examples, while excellent and laudable, do not constitute an approach that ensures the
identification and capture of critical NASA implicit and explicit knowledge Agency-wide in a manner
that would allow any NASA employee (or, under some circumstances, NASA partners and contractors)
a single process or tool to locate and then access all of the information resources.
The ASAP has recommended that NASA establish a single focal point—a “Chief Knowledge Officer”—
within the Agency to develop the policy and requirements necessary to integrate knowledge capture
across programs, projects, and Centers. Additionally, the ASAP has recommended that NASA consider
establishing Chief Knowledge Officer positions at all NASA Centers and in all Mission Directorates to
ensure standardization of programs and lessons learned as we move forward. A single focal point within
the Agency provides clear responsibility and authority to ensure an integrated Agency-wide process and
archive for knowledge capture. A similar focal point at each NASA Center and each Mission Directorate
would facilitate this function at the local level.
The ASAP believes that one overarching and fundamental purpose of NASA is to create knowledge.
The Agency remains the sole repository within the U.S. for a rich history of knowledge on human
spaceflight. Its Centers contain much of the world’s information on planetary science, knowledge of the
cosmos, and many related scientific fields. Ensuring that this knowledge is captured and available to
future generations is more than an obligation; it is a sacred trust.

5. Facilities
For the past 3 years, the ASAP has been monitoring the condition of NASA facilities and infrastructure
with an eye toward safety and mission accomplishment. In this regard, during each Center or
installation visit, a facilities tour is performed to derive a sense of the changes that have occurred since
the last visit and to gain an appreciation of the general condition of the facilities’ components. This is
not a detailed engineering inspection or assessment, nor is it a comprehensive review, covering every
facility or all areas of the Center; rather, it is an overall impression of the facilities’ condition, including
pressure vessels, boilers, hoists and lifts, hangars, test stands, electrical systems, etc.
The Panel has noted in past annual reports and continues today to believe that, considering their
age, most NASA facilities are in relatively acceptable condition. However, during each visit we have
observed firsthand or have identified areas where the infrastructure or a particular condition could have
an undesirable mission effect. Worse yet, it could present a safety hazard that, if not addressed, could
result in NASA or contractor personnel injury. Regretfully, but not surprisingly, the number of such
conditions or infrastructures is on the rise, and the overall facility condition-index trend is downward.
In light of this downward trend, for the past 2 years, the ASAP has asked NASA to identify the process
used Agency-wide to identify, characterize, and prioritize facilities or infrastructure requiring critical
repair or replacement. The response thus far is that the burden of such characterization and repairs is the

12 Annual Report for 2011

Aerospace Safety Advisory Panel

responsibility of each Center’s engineering organizations, using standardized codes to identify the most
critical repair or replacement. Funding to address these needs routinely is provided by project funds, if
available. For those Centers well endowed with an abundance of projects, this approach has been, for
the short term, satisfactory; however, for Centers not so fortunate, this methodology has been less than
successful. In some cases, instead of repairs, the Agency has earmarked facilities for replacement. This
approach has resulted in a significant near-term request for construction that even under the best of
circumstances is not likely to be funded.
Considering the current and anticipated budget environment, the ASAP believes that NASA must
develop and implement a process that compares risk at each Center, then integrates, prioritizes, and
allocates dollars for facility repairs Agency-wide for the most critical areas. To do otherwise will allow
further deterioration of critical facilities at some Centers while possibly over-improving facilities or
infrastructure at other Centers. Both results are undesirable. In other words, these decisions need to be
made from the perspective of what is best for NASA overall rather than what is best for an individual
NASA Center.

The ASAP continues to believe that it is critical for the Agency to constantly evaluate its transparency
and culture. In this context, “transparency” means open access to information, participation, and
decision-making, which ultimately creates a higher level of trust among stakeholders; “culture” is a
collection of values and norms that are shared by people in the organization and that control the way
they interact with each other and with stakeholders outside the organization. An open culture makes
it easier to identify risk and perform insight and oversight; it will also improve communication within
the Agency and with key partners and contractors. The ASAP has addressed culture frequently since
the Columbia Accident Investigation Board (CAIB) report and will continue to do so. With respect to
transparency and culture, this past year the ASAP made recommendations on insight/oversight, timely
and accurate communication, and development of a NASA alcohol use and testing policy.

1. Insight/Oversight
As NASA transitions into utilizing commercial services for both cargo and crew transportation to LEO
destinations while developing a new SLS for exploration, both insight and oversight will be essential to
maintaining the safety of various systems and the crew that occupy them.
The ASAP has been monitoring the transition to commercially based programs for the delivery of cargo
and eventually crew to the ISS and other LEO destinations and has discussed the type of information
that is needed to provide additional insight. In accordance with our recommendations from prior
reports, we are pleased to note that NASA has recently baselined and published technical requirements
for any provider interested in offering transportation for NASA astronauts to LEO. This publication is
fundamental to establishing transparency in the relationship between the Agency and its suppliers. The
ASAP believes it will be equally helpful for NASA to provide oversight to a validation and verification
matrix that outlines how each provider will provide assurance that the design meets those requirements.

Annual Report for 2011 13

 Aerospace Safety Advisory Panel

Establishing requirements to guide the design and then overseeing the process that validates that the
intent is being met is fundamental to being able to certify these systems upon entering into follow-on
development phases after the SAA design phase.
The ASAP feels that oversight must continue to be provided in order to ensure that the vehicle is
manufactured, is assembled, and will be operated in accordance with the requirements and the design
constraints. Any manufacturer that is involved in producing or providing systems where human safety
is a critical concern should expect that a level of oversight is a necessary requirement and take the
appropriate steps to integrate this oversight into the program plans.
Even though the SAA prevents NASA from issuing requirements directly under a contract-type
arrangement, there should be no restriction on NASA seeking assurance information to make certain
that the provider designs to meet the human rating requirements, validates the design, builds to the
design, operates the build within the design limits, and maintains it to ensure that no degradation
takes place.
Another related area where insight must be provided is risk. Technical risk represents perhaps the most
controllable risk. NASA is providing a set of requirements that list the objectives that must be achieved
as well as any known approach for achieving them; in addition, all of NASA’s engineering standards are
being provided. While technical risk can never be fully eliminated, this approach mitigates such risk to
the extent possible. We believe that NASA should seek the maximum opportunity to closely oversee what
the contractors are developing during the SAA phase of the CCP. Schedule risk is a more difficult risk
to mitigate because, like cost, it tends to be a “victim” of whatever else goes wrong. The ASAP believes
that the best approach to handling this risk is to develop an agreed-to integrated plan and schedule that
calls out specific, measurable events that are easily discernable by all parties so that progress is clearly
measurable and evident. Financial risk is, without a doubt, the most contentious risk category, from
both the funder’s perspective and the performer’s perspective. The funding uncertainty makes this risk
more difficult to manage. The tendency to “promise beyond ability” and to “expect beyond capability”
is strong in the program culture. Under the recently announced change from traditional contracts to
SAAs, the ASAP continues to stress that insight and oversight into program execution are essential to
ensuring that the SAA phase of the program yields designs that can be ultimately certified as meeting
safety criteria.
Several things can ease the ASAP’s concerns regarding the programmatic and thus the safety risk
associated with developing the commercial space transportation system and any future space launch
system for NASA astronauts:
1. An independent and credible cost estimate;
2. A realistic schedule based on the resources made available;
3. Sufficient resources to fund the acquisition approach, with historically realistic management reserve;
4. Completion of NASA’s safety certification requirements and process; and
5. Provisions in the SAAs negotiated with suppliers that provide NASA access to and insight into the
design and validation of the vehicles under development.

14 Annual Report for 2011

Aerospace Safety Advisory Panel

Transparent communication, constructive feedback, professional trust, and flexibility will be necessary
to assure all stakeholders that these programs are on a path to success. The insight/oversight process
must revolve around the development of a long-standing, mutually trusting relationship between all
stakeholders. It is essential to the safe and efficient execution of any programs or partnerships. When
issues arise, they need to be openly and thoroughly discussed. Withholding information, hiding
concerns, and keeping secrets are clearly signs of trouble in any relationship, but especially one in
which the end product is designed for human transport to space, which is a very hazardous endeavor.
The ASAP believes this must be worked on by both sides until an open and transparent relationship
is established. While the SAAs pose the potential for increased risk, this can be partially overcome by
making sure that both sides of the process are fully knowledgeable about the ability of the design to
meet NASA’s human rating requirements.

2. Timely and Accurate Communications

Given the 24/7 news cycle, the 15-second sound bite, and the social media of Facebook and Twitter,
NASA needs to work even harder to communicate more complex scientific information, relative risk,
and test results. When working through and with commercial entities, it becomes more critical for
NASA to effectively communicate all events of interest to external stakeholders. This involves very
close coordination between NASA leadership, the broader NASA organization, NASA’s Office of
Communications, NASA’s program offices, and external providers. Timely and accurate information
needs to flow seamlessly and rapidly. Clear, timely, and effective communication is essential, as poor
performance in this area leads to loss of reputation, distrust, and perhaps even contractual issues.
Accuracy in communicating NASA’s activities involves structuring the information to meet the needs
of several different levels of understanding in the target audience. The general public needs to be aware
of the benefits and the risks involved in the endeavor, expressed in terms that can be understood by a
nonscientific audience. At the same time, NASA must keep its scientific audience satisfied with much
more detailed technical information. This challenge makes the communication job more complex but
in no way makes it less critical. NASA and its contractors and partners must work to achieve these
multiple levels of understanding if they ever hope to gain a national consensus in favor of their activities.
In the past year, the Panel believes that there have been communication missteps between the
Commercial Cargo and Crew offices and NASA’s top leaders. These transparency and communication
issues have caused precious time to be wasted and confidence lost as the individuals involved strove for
clarity. In one instance, a very public legal action was taken that detracted from the overall program. In
another instance, “absolute” versus more “measured” language led to more questions, larger issues, and
some loss of credibility.
To NASA’s credit, there has been progress in this area within the past year, as a few anomalies were
handled quickly and candidly (e.g., the Glory failure and the UARS reentry cited in Section I of this
report). While actual performance has indicated a few bright spots, there is more work to be done to
build more trust into the accuracy of communications. The ASAP will continue to monitor and assess
accuracy and timeliness over the next year.

Annual Report for 2011 15

 Aerospace Safety Advisory Panel

3. Alcohol Use and Testing Policy

In the early afternoon of March 17, 2006, a fatal fall from a roof occurred at a NASA facility. Subsequent
investigation yielded information that pointed to an elevated blood alcohol level in the deceased.
Falls like this are tragic and are totally preventable. All stakeholders—the family, the employer, the
community, and the Agency—suffer in the aftermath.
The ASAP, after learning the details from the internal mishap investigation, also learned that NASA
had no formal alcohol or drug testing policy for employees or contractors. In the spirit of improved
safety, the Panel recommended the following in the third quarter of 2006:
Random Drug and Alcohol Testing—Recent mishap investigation revelations indicate that
there does not seem to be an Agency-wide requirement for random drug and alcohol testing
among contractors. ASAP recommends that expanding both random pre-incident and targeted
post-incident testing would be well advised for contractors as well as NASA civil servants.
In the past 5 years, there have been various and periodic discussions on this subject between the ASAP
and NASA’s Office of Safety and Mission Assurance (OSMA). NASA now has in place a drug testing
policy for civil servants and contractor employees that addresses the drug use and testing portion of the
recommendation. Unfortunately, NASA has yet to make appreciable progress on a formal policy on alcohol
use and testing. Finally, in October 2011, NASA shared with the ASAP that the whole effort had become
“lost” with the recent retirements and challenges at the Agency, and the OSMA will now “start over.”
The ASAP appreciates NASA’s frankness and transparency on the status; however, little to no progress
in this area over 5 years continues to be troubling. NASA’s work is serious, and any benchmarking with
industry indicates that NASA is behind the power curve in this area. The ASAP is looking forward to
hearing more from NASA on this subject in 2012.
III. Conclusion
NASA is moving forward into 2012 with many challenges, the foremost of which is virtually the same
as last year—a lack of clarity and constancy of purpose among the White House, Congress, and NASA.
Despite laudable progress on space systems including the ISS, NASA’s human exploration mission
remains unclear and its budget and schedule uncertain. The ASAP feels it is vital to the national interest
that all the principal stakeholders reach a clear and unambiguous understanding of the U.S. human
spaceflight program’s goals and objectives. The risks to reach the goals must be measured both in
resources and in human lives. These risks must be properly estimated by using the most knowledgeable
analysts and the most sophisticated and accurate tools. The assessment should be unbiased and accepted
by all parties in the endeavor; it must include the risks to both those who make the journey and those
who stay behind. Perhaps most importantly, the risks must be made fully transparent to and understood
by the American people. The appropriate budget must then be provided to carry out the actions needed
to reach that goal.
In regard to human spaceflight, the ASAP has concluded that the following three critical questions must
be answered in the near term for any program to any destination:

16 Annual Report for 2011

Aerospace Safety Advisory Panel

• What is the mission? We must clearly articulate the goals and objectives of the U.S. human
spaceflight program, both within and beyond LEO.
• What will it cost? We must measure the risk in both resources and human lives to reach the goal.
Said another way—Is what we will get worth what it may cost?
• Whom will you tell? The risk must be made fully transparent to the stakeholders and the
American people.
The time for either unfounded pessimism or unbridled optimism is over. Human space exploration,
should we choose to pursue it, is expensive, time-consuming, demanding of the highest levels of
technology, and inherently dangerous. If we cannot accept these fundamental facts, then we must
consider whether or not we should go. It is a choice, and that choice should not be postponed.
Expanded commercial activities and how best to acquire them remains a topic of importance. The
ASAP had previously noted that fixed-price contracting potentially sets up a conflict between cost,
schedule, and performance that can affect safety. Significant insight and oversight by NASA will be
required to ensure that this inherent conflict is appropriately managed. NASA’s very recently announced
plan to change the acquisition approach for commercial crew transportation services from FAR-based
contracting to SAAs may have negative implications for the safety of NASA crew. The ASAP plans to
closely examine this approach and its safety issues in 2012.
In our current media environment, NASA needs to work harder to clearly express more complex
scientific information, relative risk, and test results. When working through and with commercial
entities, it becomes more critical for NASA to effectively communicate to all external stakeholders.
The ASAP believes that one overarching and fundamental purpose of NASA is to create knowledge.
Public information, no matter how skillfully done, does not ensure the identification and capture of
critical NASA knowledge. The Agency remains the sole U.S. repository for a rich history of knowledge
on human spaceflight, and its Centers contain much of the world’s information on space science and
related scientific fields. Ensuring that this knowledge is captured, retained, and available to future
generations is essential. Therefore, the ASAP has recommended that NASA establish a single focal
point—a “Chief Knowledge Officer”—within the Agency to develop the policy and requirements
necessary to integrate knowledge capture across programs, projects, and Centers.
NASA has now announced that, to provide the future capability for human exploration beyond Earth
orbit, it plans to develop the SLS and the Orion MPCV. Since these systems will be NASA’s primary
vehicles for carrying out its exploration mission over the next several decades, it is essential that they be
designed to be as safe and reliable as possible. The ASAP plans to initiate a more detailed review of these
programs during the coming year.

Annual Report for 2011 17

.
Appendix:
Summary and Status of ASAP 2011 Recommendations

19
 Aerospace Safety Advisory Panel

Rec. # DescRiption of RecommenDation S

2011-01-01 NASA Alcohol Use and Testing Policy. NASA should NASA response
implement a post-mishap alcohol and drug testing program and updates
for all personnel in sensitive positions that are involved in received;
Class A and B mishaps. That includes NASA contractors, civil OPEN pending
servants, political appointees, and all affected visitors. schedule with
completion and
implementation
dates

2011-01-02 Safety and Mission Assurance Role Descriptions. NASA NASA response
should begin to draft a role description as well as some key job received 6/27/11;
requirements, such as educational background and experience, OPEN pending
for the personnel who have to specify, manage, and assure the briefing on study
S&MA activities under the new program direction. findings

2011-01-03a Safety Metrics. The NSC should expand mishap analysis NASA response
to include all types of mishaps. As this process develops received 6/27/11;
and matures, and as the comparisons make the data more CLOSED,
meaningful, the NSC should brief the senior leadership of the with quarterly
Centers and the Agency on the results. monitoring

2011-01-03b IRIS Support. NASA should describe how the IRIS supports NASA response
causal analysis and include the causations in the periodic received 6/27/11;
reports together with their associated mitigation actions and OPEN, with
schedules for completion to management. NASA should progress report at
take steps to have the system do the analysis and reporting 1st qtr. mtg. in
automatically. 2012

2011-01-04 Document Title for “Commercial Crew Transportation NASA response

System Certification Requirements for NASA Low Earth received 4/22/11;
Orbit Missions.” NASA should change the title to one that CLOSED
clearly indicates that the document applies to NASA crew
transport to LEO only.

2011-02-01 Commercial Crew Program. NASA needs to apply NASA response

appropriate resources to the CCP to ensure that it meets or received 8/24/11;
beats the 2016 goal while maintaining NASA’s high standards CLOSED
for quality and safety. NASA should seek additional resources
either within the existing budget or through additional
appropriations.

20 Annual Report for 2011

Aerospace Safety Advisory Panel

Rec. # DescRiption of RecommenDation status

2011-02-02 Space Shuttle Launch and Re-entry Risk Study. NASA NASA response
should perform an analytical study on the Space Shuttle received
launch and re-entry risk to both crew and public safety. This 8/8/11; briefing
study should be done using a consistent set of assumptions 10/20/11;
over the total launch history. CLOSED

2011-02-03 SOMD/ESMD Organizational Merger. OSMA should NASA response

review the current reorganizational plans to ensure that no received 8/16/11;
current critical safety and mission assurance (SMA) aspects, CLOSED
particularly programmatic, are inadvertently eliminated or
disrupted due to the merger.

2011-02-04 SMA Software Assurance. OSMA should do an analysis on NASA response

what the impact is to NASA’s critical programs by not doing received 8/16/11;
100 percent IV&V for software assurance. OPEN pending
briefing

2011-03-01 Abort Effectiveness Requirement. Requirements for abort NASA response

system effectiveness should be retained as a safety requirement. received 10/6/11;
CLOSED, with
monitoring

2011-03-02 Partner Integration Team Rotation. The CCP should develop NASA response
a written policy specifying team rotation schedules based on received 10/6/11;
tour of duty, milestones, or other appropriate criteria, to ensure OPEN, pending
a fresh set of eyes are always protecting the government’s interest receipt of policy
for the insight portion of the acquisition strategy. or procedure

2011-03-03 Responsibility, Authority, and Accountability for System NASA response

Requirement Approval and Design Risk Acceptance. received 10/6/11;
NASA’s Chief of OSMA, Chief Engineer, and AA for CLOSED, with
ESMD should clarify who has responsibility, authority, and monitoring
accountability to approve system requirements and accept
design risks associated with the CCP.

2011-04-01 Chief Knowledge Officer Positions. NASA should establish a OPEN

single focal point (a Chief Knowledge Officer) within the Agency
to develop the policy and requirements necessary to integrate
knowledge capture across programs, projects, and Centers; NASA
should consider establishing Chief Knowledge Officer positions
at all NASA Centers and in all Mission Directorates to ensure
standardization of programs and lessons learned.

Annual Report for 2011 21

.