NASA/SP-2011-3421 Probabilistic Risk Assessment Procedures Guide for NASA Managers and Practitioners

431 Pages·5.967 MB·English

NASA/SP-2011-3421
Second Edition
December 2011

Probabilistic Risk Assessment
Procedures Guide for NASA
Managers and Practitioners
NASA Scientific and Technical (STI) Program ... in Profile
Since its founding, NASA has been dedicated CONFERENCE PUBLICATION.
to the advancement of aeronautics and space Collected papers from scientific and technical
science. The NASA scientific and technical conferences, symposia, seminars, or other
information (STI) program plays a key part in meetings sponsored or co-sponsored by NASA.
helping NASA maintain this important role.
SPECIAL PUBLICATION. Scientific, technical,
The NASA STI program operates under the or historical information from NASA programs,
auspices of the Agency Chief Information Officer. projects, and missions, often concerned with
It collects, organizes, provides for archiving, and subjects having substantial public interest.
disseminates NASA’s STI. The NASA STI
program provides access to the NASA TECHNICAL TRANSLATION.
Aeronautics and Space Database and its public English-language translations of foreign
interface, the NASA Technical Report Server, scientific and technical material pertinent to
thus providing one of the largest collections of NASA’s mission.
aeronautical and space science STI in the world.
Results are published in both non-NASA channels Specialized services also include organizing
and by NASA in the NASA STI Report Series, and publishing research results, distributing
which includes the following report types: specialized research announcements and
feeds, providing help desk and personal search
TECHNICAL PUBLICATION. Reports of support, and enabling data exchange services.
completed research or a major significant phase
of research that present the results of NASA For more information about the NASA STI
Programs and include extensive data or program, see the following:
theoretical analysis. Includes compila
tions of significant scientific and technical data Access the NASA STI program home page at
and information deemed to be of continuing http://www.sti.nasa.gov
reference value. NASA counter-part of peer-
reviewed formal professional papers but has less E-mail your question via the Internet to
stringent limitations on manuscript length and [email protected]
extent of graphic presentations.
Fax your question to the NASA STI Help Desk
TECHNICAL MEMORANDUM. at 443-757-5803
Scientific and technical findings that are
preliminary or of specialized interest, Phone the NASA STI Help Desk at
e.g., quick release reports, working 443-757-5802
papers, and bibliographies that contain minimal
annotation. Does not contain extensive analysis. Write to:
NASA STI Help Desk
CONTRACTOR REPORT. Scientific and NASA Center for AeroSpace Information
technical findings by NASA-sponsored 7115 Standard Drive
contractors and grantees. Hanover, MD 21076-1320
NASA/SP-2011-3421

Probabilistic Risk Assessment Procedures
Guide for NASA Managers and Practitioners

NASA Project Managers:

Michael Stamatelatos, Ph.D., and
Homayoon Dezfuli, Ph.D.

NASA Headquarters
Washington, DC

Second Edition

December 2011

Acknowledgments

The individuals responsible for this document, who managed this project, and were also
authors are:

Michael Stamatelatos, NASA Headquarters (HQ), Washington, DC, and
Homayoon Dezfuli, NASA HQ, Washington, DC

The following individuals, listed in alphabetic order, are principal contributors to the present
and/or previous edition of this document:

George Apostolakis, previously at Massachusetts Institute of Technology (MIT), now at
United States Nuclear Regulatory Commission (NRC)
Chester Everline, NASA Jet Propulsion Laboratory (JPL)
Sergio Guarro, Aerospace Corporation
Donovan Mathias, NASA Ames Research Center (ARC)
Ali Mosleh, University of Maryland (UMD)
Todd Paulos, Alejo Engineering
David Riha, Southwest Research Institute
Curtis Smith, Idaho National Laboratory (INL)
William Vesely, NASA HQ
Robert Youngblood, INL

Additional contributors to this or the previous version of this document are: Harold
Blackman, Ron Boring, and David Gertman, INL; Scott Dixon and Michael Yau, ASCA Inc.;
Parviz Moieni, Southern California Edison; Hamed Nejad, Science and Technology Corp.; Pete
Rutledge, Quality Assurance & Risk Management Services; Frank Groen and Faith Chandler,
NASA HQ; Ken Gee, ARC; Susie Go, ARC; Scott Lawrence, ARC; Ted Manning, ARC; Patrick
McCabe and Kurt Vedros, INL; and Shantaram Pai, Glenn Research Center.

Reviewers who provided comments on the drafts leading up to this revision are: Allan
Benjamin and Christopher Everett, Information Systems Laboratories; Tim Barth, NASA
Engineering and Safety Center (NESC); Mark Bigler, Johnson Space Center (JSC); Michael
Blythe, NESC; Roger Boyer, JSC; Alfredo Colón, NASA HQ; Charles Ensign, Kennedy Space
Center (KSC); Amanda Gillespie, KSC; Teri Hamlin, JSC; Curtis Larsen, JSC; Mike Lutomski,
JSC; Mark Monaghan, KSC; Bruce Reistle, JSC; Henk Roelant, JSC.

Document available from:

NA SA Center for AeroSpace Information National Technical Information Service
7115 Standard Drive 5301 Shawnee Road
Hanover, MD 21076-1320 Alexandria, VA 22312
443-757-5802 703-605-6000

ii
i
ii
Contents
Acknowledgments .......................................................................................................................... i
Acronyms and Abbreviations .................................................................................................... xviii
1. Introduction ......................................................................................................................... 1-1
1.1 Purpose and Scope of This Procedures Guide ........................................................... 1-2
1.2 Knowledge Background ............................................................................................... 1-3
1.3 Application Recommendation ...................................................................................... 1-3
1.4 References .................................................................................................................. 1-3
2. Risk Management ............................................................................................................... 2-1
2.1 Definition of Risk .......................................................................................................... 2-1
2.2 Risk Management at NASA ......................................................................................... 2-2
2.2.1 Risk-Informed Decision Making Process (RIDM) ................................................... 2-4
2.2.2 Continuous Risk Management (CRM) .................................................................... 2-7
2.3 References ................................................................................................................ 2-11
3. Probabilistic Risk Assessment Overview ............................................................................ 3-1
3.1 Historical Background .................................................................................................. 3-1
3.1.1 Design Basis Evaluation vs. Risk Evaluation ......................................................... 3-1
3.1.2 From Regulation Based on Design Basis Review to Risk-Informed Regulation .... 3-2
3.1.3 Summary of PRA Motivation .................................................................................. 3-3
3.1.4 Use of PRA in the Formulation of a Risk-Informed Safety Case (RISC) ................ 3-4
3.1.5 Management Considerations ................................................................................. 3-4
3.2 Example ....................................................................................................................... 3-5
3.2.1 Propellant Distribution Module Example ................................................................ 3-5
3.2.2 Selected Results .................................................................................................... 3-6
3.2.3 High-Level Application of Results ........................................................................... 3-8
3.2.4 Summary ................................................................................................................ 3-9
3.3 Elements of PRA ....................................................................................................... 3-10
3.3.1 Identification of Initiating Events ........................................................................... 3-11
3.3.2 Application of Event Sequence Diagrams and Event Trees ................................. 3-13
3.3.3 Modeling of Pivotal Events ................................................................................... 3-17
3.3.4 Quantification of (Assignment of Probabilities or Frequencies to) Basic Events .. 3-19
3.3.5 Uncertainties: A Probabilistic Perspective ............................................................ 3-21
3.3.6 Formulation and Quantification of the Integrated Scenario Model ....................... 3-23
3.3.7 Overview of PRA Task Flow ................................................................................. 3-25
3.4 Summary ................................................................................................................... 3-26
3.4.1 Current State of Practice ...................................................................................... 3-26
3.4.2 Prospects for Future Development ....................................................................... 3-27
3.5 References ................................................................................................................ 3-27
4. Scenario Development ....................................................................................................... 4-1
4.1 System Familiarization ................................................................................................ 4-1
4.2 Success Criteria .......................................................................................................... 4-3
iii
4.2.1 Mission Success Criteria ........................................................................................ 4-3
4.2.2 System Success Criteria ........................................................................................ 4-4
4.3 Developing a Risk Model ............................................................................................. 4-5
4.3.1 IE Development ...................................................................................................... 4-7
4.3.2 Accident Progression ........................................................................................... 4-10
4.3.3 Fault Tree Modeling ............................................................................................. 4-17
4.4 References ................................................................................................................ 4-20
5. Data Collection and Parameter Estimation ......................................................................... 5-1
5.1 PRA Parameters .......................................................................................................... 5-1
5.2 Sources of Information ................................................................................................ 5-3
5.2.1 Generic Data Sources ............................................................................................ 5-3
5.2.2 System-Specific Data Collection and Classification ............................................... 5-5
5.3 Parameter Estimation Method ..................................................................................... 5-9
5.4 Prior Distributions ...................................................................................................... 5-10
5.5 Selection of the Likelihood Function .......................................................................... 5-11
5.6 Development of the Posterior Distribution ................................................................. 5-12
5.7 Sequential Updating .................................................................................................. 5-15
5.8 Developing Prior Distributions from Multiple Sources of Generic Information ........... 5-15
5.9 Guidance for Bayesian Inference Calculations .......................................................... 5-16
5.10 References ................................................................................................................ 5-16
6. Uncertainties in PRA ........................................................................................................... 6-1
6.1 The Model of the World ............................................................................................... 6-1
6.2 The Epistemic Model ................................................................................................... 6-2
6.3 A Note on the Interpretation of Probability ................................................................... 6-3
6.4 Presentation and Communication of the Uncertainties ............................................... 6-7
6.5 The Lognormal Distribution ......................................................................................... 6-8
6.6 Assessment of Epistemic Distributions ...................................................................... 6-10
6.6.1 Bayes’ Theorem ................................................................................................... 6-10
6.6.2 A Simple Example: The Discrete Case ............................................................... 6-11
6.6.3 A Simple Example: The Continuous Case ........................................................... 6-12
6.6.4 Conjugate Families of Distributions ...................................................................... 6-15
6.7 The Prior Distribution ................................................................................................. 6-17
6.8 The Method of Maximum Likelihood .......................................................................... 6-18
6.9 References ................................................................................................................ 6-19
7. Modeling and Quantification of Common Cause Failures .................................................. 7-1
7.1 Importance of Dependence in PRA ............................................................................. 7-1
7.2 Definition and Classification of Dependent Events ...................................................... 7-1
7.3 Accounting for Dependencies in PRAs ........................................................................ 7-2
iv
7.4 Modeling Common Cause Failures ............................................................................. 7-4
7.5 Procedures and Methods for Treating CCF Events ..................................................... 7-6
7.6 Preliminary Identification of Common Cause Failure Vulnerabilities (Screening
Analysis) ................................................................................................................................. 7-6
7.6.1 Qualitative Screening ............................................................................................. 7-6
7.6.2 Quantitative Screening ........................................................................................... 7-8
7.7 Incorporation of CCFs into System Models (Detailed Analysis) ................................ 7-10
7.7.1 Identification of CCBEs ........................................................................................ 7-10
7.7.2 Incorporation of CCBEs into the Component-Level Fault Tree ............................ 7-11
7.7.3 Development of Probabilistic Models of CCBEs .................................................. 7-13
7.7.4 Estimation of CCBE Probabilities ......................................................................... 7-15
7.8 Generic Parameter Estimates ................................................................................... 7-16
7.9 Treatment of Uncertainties ........................................................................................ 7-17
7.10 References ................................................................................................................ 7-18
8. Human Reliability Analysis (HRA) ...................................................................................... 8-1
8.1 Basic Steps in the HRA Process ................................................................................. 8-1
8.2 Classifications of Human Interactions and Associated Human Errors ......................... 8-3
8.2.1 Pre-Initiator, Initiator, and Post-Initiator HSIs ......................................................... 8-3
8.2.2 Skill, Rule, and Knowledge-Based Response ........................................................ 8-3
8.2.3 Error of Omission and Error of Commission ........................................................... 8-4
8.3 General Modeling of Pre-Initiator, Initiator, and Post-Initiator HSIs in a PRA .............. 8-4
8.4 Quantification of Human Interactions (or Errors) ......................................................... 8-4
8.4.1 Qualitative Screening ............................................................................................. 8-5
8.4.2 Quantitative Screening ........................................................................................... 8-6
8.5 HRA Models ................................................................................................................ 8-6
8.5.1 Technique for Human Error Rate Prediction (THERP) ........................................... 8-6
8.5.2 Cognitive Reliability and Error Analysis Method (CREAM) .................................. 8-11
8.5.3 Nuclear Action Reliability Assessment (NARA) .................................................... 8-15
8.5.4 Standard Plant Analysis Risk HRA Method (SPAR-H) ......................................... 8-18
8.6 Guidelines on Uses of HRA Models .......................................................................... 8-21
8.7 HRA Examples .......................................................................................................... 8-22
8.7.1 Example for a Post-Initiator HSI ........................................................................... 8-22
8.7.2 Example for a Pre-Initiator HSI ............................................................................. 8-25
8.8 References ................................................................................................................ 8-28
9. Software Risk Assessment ................................................................................................. 9-1
9.1 Concept of Software Risk and Related Definitions ...................................................... 9-2
9.1.1 Basic Definitions ..................................................................................................... 9-3
9.1.2 Software Defects and Software Failures ................................................................ 9-3
9.2 Lessons Learned from Software Failures in Space Systems ...................................... 9-5
9.3 Classification of Software Failures for Risk Modeling .................................................. 9-8
v
9.3.1 Conditional vs. Unconditional Failures ................................................................... 9-8
9.3.2 Recoverable vs. Mission-critical Failures ............................................................... 9-9
9.4 Context-based Software Risk Model (CSRM) ........................................................... 9-10
9.4.1 Conceptual Formulation ....................................................................................... 9-10
9.4.2 Key Objectives and Characteristics of CSRM Application ................................... 9-12
9.4.3 Application Process .............................................................................................. 9-15
9.4.4 Examples of Application ....................................................................................... 9-17
9.4.5 CSRM Modeling Detail and Representation of Software Failure Modes .............. 9-31
9.4.6 Software Risk Quantification ................................................................................ 9-33
9.5 Use of Software Risk Information .............................................................................. 9-39
9.5.1 Conditional Scenarios and Risk-informed Software Testing Strategies ............... 9-39
9.5.2 Integration of Results into Pre-existing PRA Models ............................................ 9-40
9.6 Definitions .................................................................................................................. 9-41
9.7 References ................................................................................................................ 9-42
10. Physical and Phenomenological Models ....................................................................... 10-1
10.1 Role of Phenomenological Methods in Risk Assessment ......................................... 10-2
10.2 Phenomenological Modeling During the Design Process .......................................... 10-2
10.3 Stress-Strength Formulation of Physical Models ....................................................... 10-4
10.4 Range Safety Phenomenological Models .................................................................. 10-6
10.4.1 Inert Debris Impact Models............................................................................... 10-7
10.4.2 Blast Impact Models ......................................................................................... 10-8
10.4.3 Re-Entry Risk Models ..................................................................................... 10-12
10.5 MMOD Risk Modeling .............................................................................................. 10-14
10.5.1 Risk from Orbital Debris ................................................................................. 10-14
10.5.2 MMOD Risk Modeling Framework.................................................................. 10-14
10.5.3 Probability of MMOD Impact P ...................................................................... 10-15
I
10.5.4 Probability of MMOD Impact Affecting Critical SV Components, P ............ 10-15
C/ I
10.5.5 Probability of Critical Component Damage, P ............................................ 10-16
D/C
10.6 Ground-Based Fire PRA .......................................................................................... 10-16
10.7 A Launch Vehicle Ascent Abort Model .................................................................... 10-23
10.8 Summary ................................................................................................................. 10-24
10.9 References .............................................................................................................. 10-24
11. Probabilistic Structural Analysis .................................................................................... 11-1
11.1 Basic Concepts of Probabilistic Structural Analysis .................................................. 11-1
11.2 Probabilistic Structural Response Modeling .............................................................. 11-2
11.2.1 Limit State Formulation .................................................................................... 11-2
11.2.2 Assigning Uncertainty Models to Random Variables ........................................ 11-4
11.3 Stress Versus Strength Modeling .............................................................................. 11-4
11.3.1 Normal Distributions ......................................................................................... 11-5
11.3.2 Lognormal Distributions .................................................................................... 11-6
11.4 Monte Carlo Simulation and Most Probable Locus Approaches ............................... 11-8
vi