DO-178C: A New Standard For Software Safety Certification DO-178C: A New Standard For Software Safety Certification
DO-178C: A New Standard For Software Safety Certification DO-178C: A New Standard For Software Safety Certification
page EU
European Headquarters:
46 rue d
d’Amsterdam
Amsterdam
75009 Paris
France
+33-1-4970-6716 (voice)
+33-1-4970-0552 (FAX)
Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and
maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information,
including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington
VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it
does not display a currently valid OMB control number.
16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF 18. NUMBER 19a. NAME OF
ABSTRACT OF PAGES RESPONSIBLE PERSON
a. REPORT b. ABSTRACT c. THIS PAGE Same as 42
unclassified unclassified unclassified Report (SAR)
3
DO-178B Basics
Software Considerations in Airborne Systems and Equipment Certification,
December 1992, published by RTCA*
EUROCAE** / ED-12B in Europe
Comprises a set of 66 “objectives” (“guidelines”) for production of software
for airborne systems
• Reliability: System does what it is supposed to do Ö no failures
Can trace each requirement
q to its implementing
p g code and verification
No missing functionality
• Safety: System does not do what it is not supposed to do Ö no hazards
Can trace each piece of code back to a requirement
No additional functionality, no “dead code”
• Requires appropriate configuration management, quality assurance
“Level”
Level of software establishes which objectives apply
* RTCA (www.rtca.org) is a U.S. Federal Advisory Committee whose recommendations guide FAA policy
** European Organisation for Civil Aviation Equipment (www.eurocae.org) 4
Software Levels Based on System Safety Assessment
Level A
“Safety- • Anomalous behavior Ö catastrophic failure condition
Critical” “prevent continued safe flight and landing”
Levell B
• Anomalous behavior Ö hazardous / severe-major failure condition
“serious or potentially fatal injuries to a small number of … occupants”
Level C
• Anomalous behavior Ö major failure condition
“discomfort to occupants,
p p
possibly
y including
g injuries”
j
Level D
• Anomalous behavior Ö minor failure condition
“some
some inconvenience to occupants”
occupants
Level E
• Anomalous behavior Ö no effect on aircraft operational capability
or pilot workload
Not addressed
in DO-178B
5
Structure / Goals / Usage
DO-178B guidelines organized into three major categories, each with a
specified set of output artifacts
• Software Planning Process
• Software Development Processes
• “Integral” Processes
Appears oriented around new development efforts
• But mayy be applied
pp to previously
p y developed
p software,, COTS,, etc.
Strong emphasis on traceability
Implies traditional / static program build model
• Compile,
Compile link
link, e
execute
ec te
Used by FAA to approve software for commercial aircraft
• Developer organization supplies certification material
• Designated Engineering Representative (“DER”) evaluates for compliance with DO-178B
“In a nutshell, what does this DO-178B specification really do?”*
• “It specifies that every line of code be directly traceable to a requirement and a test routine,
and that no extraneous code outside of this process be included in the build”*
Software QA Plan
Plan for Software Software Config Mgmt Plan
Software Aspects of Planning Software Verification Plan
Certification
Process Software Development Plan
Concurrent
Activities
High-Level Requirements
Software Development Integral Processes
Processes Derived Requirements
• Software Verification
• Requirements
Design Description • Software Configuration Mgmt
• Design
Low-Level Requirements • Software Quality Assurance
• Coding
Derived Requirements • Certification Liaison
• Integration
Source Code
Object Code
Totals 66 65 57 28
11
Sample DO-178B Objectives [Table A-7]
Verification of Verification Process Results
Objective Level Output
Test procedures are correct ABC Software Verification
Cases and Procedures
Test results are correct and discrepancies explained ABC
Test coverage of high-level requirements is achieved ABCD
Test coverage of low
low-level
level requirements is achieved ABC
Test coverage of software structure (modified A
condition/decision coverage) is achieved
Test coverage of software structure (decision coverage) is AB Software
S ft V
Verification
ifi ti
achieved Results
Test coverage of software structure (statement coverage) is ABC
achieved
Test coverage of software structure (data coupling and ABC
control coupling) is achieved
Statement Coverage
* Every statement has been invoked at least once 9 9 9
Decision Coverage
* Described below 9 9
14
Decision Coverage
“Condition”
• A Boolean expression containing no Boolean operators; e.g., X>0
“Decision”
• A Boolean expression composed of conditions and zero or more Boolean operators; e.g.,
X>0 and Y=2
• If a condition appears more than once in a decision, each occurrence is a distinct condition
X>0 and X>0 has two conditions
Decision coverage
• Every point of entry and exit in the program has been invoked at least once
• Every decision in the program has taken all possible outcomes at least once
15
Modified Condition / Decision Coverage
MC / DC = Decision coverage + additional requirements
• Every condition in a decision in the program has taken all possible outcomes at least once
• Each condition in a decision has been shown to independently affect that decision's
outcome
With all other conditions constant, changing the truth value of the condition
changes the result of the decision
* In general, at least n+1 tests are needed for a decision with n conditions 16
Related Standards and Other Material
DO-248B
• Provides clarification of DO-178B guidance material (does not introduce new guidance)
12 errata corrections
76 Frequently-Asked Questions (FAQ)
15 Discussion Papers
CAST Papers (Certification Authority Software Team)
• Further clarification of DO-178B
DO-278
• Guidelines for Communication, Navigation, Surveillance, and Air Traffic Management
(CNS/ATM) Systems Software Integrity Assurance
• Applies to software DO-278 Assurance Level DO-178B Software Level
in CNS/ATM
AL1 A
systems
y used in
ground- or space- AL2 B
based applications AL3 C
that affects aircraft AL4 No equivalent
q
safety
AL5 D
• Largely based on
DO-178B AL6 E
17
Navigating the Software Safety Certification Waters
Software
D
Developers
l
DO-178B
DO-278
DER
DO-248B:
DO 248B:
• Errata in DO-178B
• FAQ Certification
• Discussion Papers Authorities
S ft
Software Team
T
(CAST) Papers
18
Why Revise DO-178B?
Address / accommodate “new” software technologies
• Object-Oriented Programming
• Model-based design / automatic code generators
• COTS software and tools, including real-time operating systems
Recognize that there is more to software verification than testing
• Formal methods
• Abstract interpretation
p
Consider combining with ground-based safety standard (DO-278)
Take into account the supplementary papers / commentaries on DO-178B
• Certification
C tifi ti Authorities
A th iti Software
S ft Team
T (CAST) papers
• Issues Papers (IPs)
Try to remove/reduce the need for DERs to make subjective judgments
Correct errors
• Example: requirements on development tool qualification don’t distinguish the host
environment (in which the tool runs) from the target environment (in which the system runs)
19
DO-178C “Terms of Reference”
Objectives
• Promote safe implementation of aeronautical software
• Provide clear and consistent ties with the systems and safety processes
• Address emerging software trends and technologies
• Implement an approach that can change with the technology
Activities (partial)
• Modify
y DO-178B
• Develop supplements to document technology-specific or method-specific guidance and
guidelines
• Develop
p and document rationale for each DO-178B objective
j
Other considerations (partial)
• Maintain technology-independent nature of DO-178B objectives
• Modifications to DO-178B
DO 178B should:
Strive to minimize changes to existing text
Consider economic impact relative to system certification without compromising
system safety
Address clear errors or inconsistencies / fill any clear gaps in DO-178B
Meet a documented need to a defined assurance benefit
20
Initial Controversy over DO-178C approach (1)
C. Michael Holloway (NASA), message posted to SC205/WG71 e-mail forum,
5 April 2005: Are We Heading in the Right Direction?
“The trend in the world (at least outside the U.S.) seems to be away from
prescriptive standards and towards goal-based standards,
standards which require
the presentation of cogent arguments for safety (and other desired attributes),
rather than the simple completion of certain prescribed processes.
21
Initial Controversy over DO-178C approach (2)
Martyn Thomas (Praxis), message posted to SC205/WG71 e-mail forum,
10 August 2005: The Right Direction?
“I strongly support the proposal that safety standards should move away from
prescribing specific processes to requiring evidence that the delivered system
has the required properties. The relationship between development processes
and the properties of the resulting systems is simply too weak to justify any
confidence that a system will be safe just because it has been developed in the
specific way prescribed by some standard.
22
Rationale for DO-178C’s Approach
Alternatives
• Major surgery on DO-178B
Remove prescriptive content, making document goal-oriented
Add supplements for requirements for specific approaches
• Minor surgery
Address new technologies in supplements when prescribed approaches may be
replaced or augmented by others (e.g., formal verification)
Decision was to go with “minor surgery”
• Core document fixes known issues and adds clarifications (e.g. from DO-248B)
gy p
• Technology-specific supplements
pp “add,, delete or otherwise modify
y objectives,
j , activities,,
explanatory text, and software life cycle data in DO-178C/ED-12C”
Reasoning
• DO
DO-178B
178B works
No commercial aviation deaths caused by software that had been certified to Level A
• Experience with goal-based approach has been mixed
Change of mindset required for both developer and regulator
Safety cases tend to focus on individual hazards, but most accidents are due to a
combination of factors
23
DO-178C Development Framework (1)
Joint RTCA Special Committee 205 and EUROCAE* Working Group 71
• “SC-205 WG-71”, officially abbreviated as “SCWG”
• Co-chairs: Jim Krodel (US), Gérard Ladier (Europe)
• Secretaries: Leslie Alford (US), Ross Hannan (Europe)
• Government agency representatives: Barbara Lingberg (FAA), Jean-Luc Delamaide (EASA)
Web site (registration required)
• ultra.pr.erau.edu/SCAS/
p
Subgroups
• SG1: SCWG Document Integration (Marty Gasiorowski / Ron Ashpole)
• SG2: Issues and Rationale (Fred Moyer / Ross Hannan)
• SG3: Tool Qualification (Leanna Rierson / Frédéric Pothon)
• SG4: Model Based Design and Verification (Mark Lillis / Pierre Lionne)
• SG5:
SG5 Obj
Object-Oriented
t O i t d Technology
T h l (G
(Greg Milli
Millican / Jan-Hendrik
J H d ik Boelens)
B l )
• SG6: Formal Methods (Kelly Hayhurst / Duncan Brown)
• SG7: CNS/ATM and Safety (Don Heck / David Hawken)
“CNS/ATM” = Communications, Navigation and Surveillance / Air Traffic Management
Formal
DO-178B MBD OOT
Clarification DO-178C methods
fixes Suppl. Suppl. Suppl.
core
Overriding Supplements
12.3
DO 248B
DO-248B Clarification
fixes
DO 248C
DO-248C
Tool
Qual.
Suppl.
26
Example of Change in Core Document: Figure 2-1
System User /
Functional Requirements
Software
Software Life Cycle Proceesses
Planning Process
Derived
High Level Software Requirements
Process
DO-178C Requirements
Software
Verification Process
(IP 50, Rev. K) Derived
Low Level
Software Design
Process
Requirements
and Software
S
Architecture S f
Software C
Coding
di and d
Integration Processes
27
Example of Change: Section 2.2
DO-178B
2.2 Failure Condition and Software Level
…
The failure condition of a system is established by determining the severity
of failure conditions on the aircraft and its occupants. An error in the software
may cause a fault that contributes to a failure condition. Thus, the level
of software integrity necessary for safe operation is related to the system failure
conditions.
System boundary
Fig. 2-2 Chain of Events for Software Error
Software boundary Leading to Failure Condition
28
Tool Qualification Supplement (1)
DO-178B O C
DO-178C
29
Tool Qualification Supplement (2)
30
Tool Qualification Supplement (3)
Software Criteria
Level
1 2 3 TQL 1: DO-178 level A
31
Model-Based Development and Verification
A model is an abstract representation of system characteristics
• Specification model expresses high-level requirements (eg functions, performance, safety)
• Design model expresses low-level requirements / architecture
Data structures, data flow, control flow
• Verification model represents life cycle data of verification process
Benefits
• Unambiguous
g notation
• Supports use of automated code generation, automated test generation
Model-Based Development/Verification Supplement
• Acknowledges need for “Higher
Higher-Level
Level Requirements”
Requirements from which development model is
derived
• Model usage affects most of the DO-178C-specified life cycle processes
• Traceability,
Traceability etc
etc, required for source program are required for design model
Demonstrate property for model, show that the property is preserved in the source
code
• Model simulator may be useful tool
32
Object-Oriented Technology (“OOT”)
What is OOT?
• Software development methodology supported by language features
Primary focus is on data elements and their relationships
Secondary focus is on the processing that is performed
• Applicable during entire software “life cycle”
Language concepts (OOP, or “Object-Oriented Programming”)
• Object
j = state ((“attributes”)) + operations
p ((“methods”))
• Class = module + object creation template Object-Oriented Design (“OOD”)
also known as
• Encapsulation = separation of interface (spec for
Object-Based Programming
methods)) from implementation
p ((state,, algorithms)
g )
• Inheritance = specialization (“is-a”) relationship between classes
Extend a class, adding new state and adding/ overriding operations
• Polymorphism = ability of a variable to reference objects from different classes at different
times
• Dynamic binding (“dispatching”) = interpretation of operation applied to polymorphic
variable based on current class of referenced object
j
33
Object-Oriented Technology (“OOT”), cont’d.
Additional OOP elements
• Single versus multiple inheritance
“Interface” for a simple form of multiple inheritance
• Use of constructors / destructors
Related language features
• Method overloading
yp conversion
• Type
• Inline expansion
Other modern language features that complicate safety certification
• Generic templates
• Exceptions
• Concurrency
34
OO Technology & Safety Certification
Why consider OOT for safety-critical software?
• Data-centric approach eases maintenance of many large systems
• Model-driven architecture / UML tools may generate OO code to be certified
• Many programmers know OO languages such as C++, Java, or Ada 95
• Languages (such as Ada) used for safety-critical systems have OO features
• May want to take OO legacy code and apply DO-178 à posteriori
• Issues explored at NASA/FAA Object-Oriented Technology in Aviation (OOTiA) workshops
What’s the catch?
• Paradigm clash
OOT’s
OOT s distribution of functionality across classes
classes, versus safety analysis’s
analysis s focus on
tracing between requirements and implemented functions
• Technical issues
The features that are the essence of OOP complicate safety certification and raise
security issues (e.g. ensuring integrity of “vtable”)
Dynamic memory allocation, VM implementations
• Cultural issues
Many DERs / TOE evaluation personnel are not language experts and are (rightfully)
concerned about how to deal with unfamiliar technology
35
OOT Supplement
New OOT-specific objectives and activities for Software Verification Process
• Based on Liskov Substitution Principle:
“Let q(x) be a property provable about objects x of type T. Then q(y) should be true for
objects of type S where S is a subtype of T”
OO.6.5 Local Type Consistency Verification
The use of inheritance with method overriding and dynamic dispatch requires additional verification
activities that can be done either by testing or by formal analysis.
• New g
guidance for dynamic
y memory
y management
g verification
Need to verify absence of problems such as dangling references, fragmentation,
storage exhaustion, unbounded allocation or deallocation time
New objectives and activities related to virtualization (OO.E.1.7)
• Issue of code versus data (objectives generally apply to code, not data)
Virtualization defines an intermediate execution platform…. One must ensure that all data that is
used as executable code for the execution platform be verified as executable code. 36
Formal Methods Supplement: Concepts
Formal Method = Formal Model + Formal Analysis
• May be applied at various stages in software development
• Formal methods are used as a verification technique
Formal Model
• System abstraction with unambiguous, mathematically defined syntax and semantics
• Examples
Graphical
p models ((state machines))
Text-based (Z, set theory, programming language subsets)
• May be used to capture some system properties (e.g. Worst-Case Execution Time)
Formal Analysis
• Provides guarantees/proofs of software properties, compliance with requirements
• An analysis method can only be regarded as formal if it is sound
It never asserts
t th
thatt a property
t hholds
ld if it iis possible
ibl ffor th
the property
t tto nott h
hold
ld
Converse is a usability issue
• Kinds of formal analysis
Deductive (theorem proving)
Model checking Implemented in (qualified) tools
Abstract interpretation 37
Formal Methods Supplement*
Mainly adapts Software Verification Process section of DO-178C
• Goal is to prevent and eliminate requirements, design and code errors throughout the
software development processes
Formal methods are complementary to testing
• Testing shows that functional requirements are satisfied and detects errors
• Formal methods can increase confidence that no anomalous behavior will occur
May find faults that are not detected by testing
Formal methods cannot establish verification evidence for the target hardware
• Therefore testing on the target is still required
• But: formal analysis of source code can be used to reach [compliance with Low-Level
Low Level
Requirements] provided that complementary analyses show the property preservation
between source code and object code
Uses of Formal Methods
• Formal specification of life-cycle artifacts
• Formal derivation of life-cycle artifacts
• Formal analysis of life
life-cycle
cycle artifacts
* This slide largely derived from Working towards DO-178C/ED-12C, DO-248C/ED-94C and
DO-278A/ED-109A, invited presentation by James Chelini (Verocel) at ACM SIGAda 2009
38
Navigating the Software Safety Certification Waters
Software
DO-178C Developers
DO-278A
DER
Tool Qualification
Supplement
Model-Based Development
and Verification Supplement
Object-Oriented
Object Oriented Technology
Supplement
40
Acronyms and Abbreviations
ATM Air Traffic Management
CAST Certification Authority Software Team
CNS Communications, Navigation, and Surveillance
COTS Commercial Off-The-Shelf
Off The Shelf
DO-178B, DO-178C [Not acronyms, these are names/numbers of documents]
DER Designated Engineering Representative
EUROCAE European Organisation for Civil Aviation Equipment
FAA Federal Aviation Administration
MC/DC Modified Condition/Decision Coverage
g
OOP Object-Oriented Programming
OOT Object-Oriented Technology
OOTiA Object-Oriented
Obj t O i t d Technology
T h l in
i Aviation
A i ti
RTCA [Not an acronym, this is the name of an organization]
41