100% found this document useful (1 vote)
1K views43 pages

DO-178C: A New Standard For Software Safety Certification DO-178C: A New Standard For Software Safety Certification

This document provides an overview of DO-178C, a new standard for software safety certification that updates DO-178B. Key changes include a reorganization of the standard into a core document and supplemental materials on new technologies. The presentation will cover the organization of DO-178C, the rationale for changes from DO-178B, and updates to the core document and new supplements on topics like model-based design and formal methods.

Uploaded by

Yohan Mekonnen
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
100% found this document useful (1 vote)
1K views43 pages

DO-178C: A New Standard For Software Safety Certification DO-178C: A New Standard For Software Safety Certification

This document provides an overview of DO-178C, a new standard for software safety certification that updates DO-178B. Key changes include a reorganization of the standard into a core document and supplemental materials on new technologies. The presentation will cover the organization of DO-178C, the rationale for changes from DO-178B, and updates to the core document and new supplements on topics like model-based design and formal methods.

Uploaded by

Yohan Mekonnen
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 43

Presentation cover

page EU

DO-178C: A New Standard for


Software Safety Certification
SSTC 2010
North American Headquarters:
104 Fifth Avenue, 15th Floor
Salt Lake City, Utah
New York, NY 10011
USA Track 1
+1-212-620-7300 (voice) Monday, 26 April 2010
+1-212-807-0162 (FAX)
3:30 – 4:15 pm

European Headquarters:
46 rue d
d’Amsterdam
Amsterdam
75009 Paris
France
+33-1-4970-6716 (voice)
+33-1-4970-0552 (FAX)

Ben Brosgol y [email protected]


www.adacore.com Cyrille Comar y [email protected]
Form Approved
Report Documentation Page OMB No. 0704-0188

Public reporting burden for the collection of information is estimated to average 1 hour per response, including the time for reviewing instructions, searching existing data sources, gathering and
maintaining the data needed, and completing and reviewing the collection of information. Send comments regarding this burden estimate or any other aspect of this collection of information,
including suggestions for reducing this burden, to Washington Headquarters Services, Directorate for Information Operations and Reports, 1215 Jefferson Davis Highway, Suite 1204, Arlington
VA 22202-4302. Respondents should be aware that notwithstanding any other provision of law, no person shall be subject to a penalty for failing to comply with a collection of information if it
does not display a currently valid OMB control number.

1. REPORT DATE 3. DATES COVERED


2. REPORT TYPE
26 APR 2010 00-00-2010 to 00-00-2010
4. TITLE AND SUBTITLE 5a. CONTRACT NUMBER
DO-178C: A New Standard for Software Safety Certification 5b. GRANT NUMBER

5c. PROGRAM ELEMENT NUMBER

6. AUTHOR(S) 5d. PROJECT NUMBER

5e. TASK NUMBER

5f. WORK UNIT NUMBER

7. PERFORMING ORGANIZATION NAME(S) AND ADDRESS(ES) 8. PERFORMING ORGANIZATION


REPORT NUMBER
AdaCore,North American Headquarters,104 Fifth Avenue, 15th
Floor,New York,NY,10011
9. SPONSORING/MONITORING AGENCY NAME(S) AND ADDRESS(ES) 10. SPONSOR/MONITOR’S ACRONYM(S)

11. SPONSOR/MONITOR’S REPORT


NUMBER(S)

12. DISTRIBUTION/AVAILABILITY STATEMENT


Approved for public release; distribution unlimited
13. SUPPLEMENTARY NOTES
Presented at the 22nd Systems and Software Technology Conference (SSTC), 26-29 April 2010, Salt Lake
City, UT. Sponsored in part by the USAF. U.S. Government or Federal Rights License
14. ABSTRACT

15. SUBJECT TERMS

16. SECURITY CLASSIFICATION OF: 17. LIMITATION OF 18. NUMBER 19a. NAME OF
ABSTRACT OF PAGES RESPONSIBLE PERSON
a. REPORT b. ABSTRACT c. THIS PAGE Same as 42
unclassified unclassified unclassified Report (SAR)

Standard Form 298 (Rev. 8-98)


Prescribed by ANSI Std Z39-18
Outline
DO-178B
• Summary
• Levels
• Life-Cycle Model
• Objectives
• Role of Testing
• Related documents
DO-178C
• Organization of revision effort
• Terms of Reference / rationale for approach
• Changes to Core Document
• Technology Supplements*
ƒ Tool Qualification
ƒ Model-Based Design and Verification
ƒ Object-Oriented Technology
ƒ Formal
F l Methods
M th d

* Based on information available in February 2010


1
Safety-Critical Software: Background
What is “safety critical” software?
• Failure can cause loss of human life or have other catastrophic consequences
How does safety criticality affect software development?
• Regulatory agencies require compliance with certification requirements
• Safety-related standards may apply to finished product, development process, or both
Prescriptive
• Specify requirements on the process by which software is developed and fielded
ƒ Sound process adds confidence in soundness of result
• Example: DO-178B
Goal based
Goal-based
• Developer provides safety cases
ƒ Claims concerning system’s safety-relevant attributes
ƒ Arguments justifying those claims
ƒ Evidence backing up the arguments
• Example: UK Defense Standard 00-56
ƒ “A Safety Case is a structured argument, supported by a body of evidence, that
provides a compelling, comprehensible and valid case that a system is safe for a given
application in a given environment”
2
Perspectives on DO-178B’s Process-Based Approach
Quote from Gérard Ladier (Airbus), FISA-2003 conference
• “It is not feasible to assess the number or kinds of software errors, if any, that may remain
after the completion of system design, development, and test”
• “Since dependability cannot be guaranteed from an
assessment off the
h software
f product,
d iit iis necessary
to have assurance on its development process”
• “You can’t deliver clean water in a dirty pipe”

Quote from John Rushby,


y HCSS Aviation Safety
y Workshop,
p Oct 2006
• “Because we cannot demonstrate how well
we’ve done, we’ll show how hard we’ve tried”

3
DO-178B Basics
Software Considerations in Airborne Systems and Equipment Certification,
December 1992, published by RTCA*
ƒ EUROCAE** / ED-12B in Europe
Comprises a set of 66 “objectives” (“guidelines”) for production of software
for airborne systems
• Reliability: System does what it is supposed to do Ö no failures
ƒ Can trace each requirement
q to its implementing
p g code and verification
ƒ No missing functionality
• Safety: System does not do what it is not supposed to do Ö no hazards
ƒ Can trace each piece of code back to a requirement
ƒ No additional functionality, no “dead code”
• Requires appropriate configuration management, quality assurance
“Level”
Level of software establishes which objectives apply

* RTCA (www.rtca.org) is a U.S. Federal Advisory Committee whose recommendations guide FAA policy
** European Organisation for Civil Aviation Equipment (www.eurocae.org) 4
Software Levels Based on System Safety Assessment

Level A
“Safety- • Anomalous behavior Ö catastrophic failure condition
Critical” ƒ “prevent continued safe flight and landing”
Levell B
• Anomalous behavior Ö hazardous / severe-major failure condition
ƒ “serious or potentially fatal injuries to a small number of … occupants”
Level C
• Anomalous behavior Ö major failure condition
ƒ “discomfort to occupants,
p p
possibly
y including
g injuries”
j
Level D
• Anomalous behavior Ö minor failure condition
ƒ “some
some inconvenience to occupants”
occupants
Level E
• Anomalous behavior Ö no effect on aircraft operational capability
or pilot workload
Not addressed
in DO-178B

5
Structure / Goals / Usage
DO-178B guidelines organized into three major categories, each with a
specified set of output artifacts
• Software Planning Process
• Software Development Processes
• “Integral” Processes
Appears oriented around new development efforts
• But mayy be applied
pp to previously
p y developed
p software,, COTS,, etc.
Strong emphasis on traceability
Implies traditional / static program build model
• Compile,
Compile link
link, e
execute
ec te
Used by FAA to approve software for commercial aircraft
• Developer organization supplies certification material
• Designated Engineering Representative (“DER”) evaluates for compliance with DO-178B
“In a nutshell, what does this DO-178B specification really do?”*
• “It specifies that every line of code be directly traceable to a requirement and a test routine,
and that no extraneous code outside of this process be included in the build”*

* Esterel Technologies, DO-178B FAQs, www.esterel-technologies.com/do-178b/ 6


Other Aspects of DO-178B
Not specific to aviation
• Could be applied, in principle, to other domains (medical devices, etc.)
Includes glossary of terms
• “dead code”, “deactivated code”, “verification”, ...
Does not dictate
• Particular development process, design approach or notation
• Particular approach to hazard assessment (fault tree analysis
analysis, etc)
• Specific programming language(s) or software tools
• Requirements for personnel training
• Format for artifacts
Tool qualification
• Ensures that tool provides confidence at least equivalent to that of the process(es)
eliminated,
li i t d reduced
d d or automated
t t d
• Can qualify as a verification tool (bug may fail to detect errors but won’t introduce any) or
as a development tool (bug may introduce errors)
Wh t about
What b t security?
it ?
• No specific security-related objectives in DO-178B
• Work in progress under RTCA (SC-216) and EUROCAE (WG-72) 7
DO-178B Software Life Cycle Model

Software QA Plan
Plan for Software Software Config Mgmt Plan
Software Aspects of Planning Software Verification Plan
Certification
Process Software Development Plan

Concurrent
Activities
High-Level Requirements
Software Development Integral Processes
Processes Derived Requirements
• Software Verification
• Requirements
Design Description • Software Configuration Mgmt
• Design
Low-Level Requirements • Software Quality Assurance
• Coding
Derived Requirements • Certification Liaison
• Integration
Source Code

Object Code

Executable Object Code +


Link / Load Data
8
Summary of DO-178B “Objectives”
Safety Level
Process A B C D
Software Planning Process 7 7 7 2

Software Development Process 7 7 7 7

Verification of Outputs of Software Requirements Process 3(ind) + 4 3(ind) + 4 6 3

Verification of Outputs of Software Design Process 6(ind) + 7 3(ind) + 10 9 1

Verification of Outputs of Software Coding & Integration 3(ind) + 4 1(ind) + 6 6 0


Processes
Testing of Outputs of Integration Processes 2(ind) + 3 1(ind) + 4 5 3

Verification of Verification Process Results 8(ind) 3(ind) + 4 6 1

Software Configuration Management Process 6 6 6 6

Software Quality Assurance Process 3(ind) 3(ind) 2(ind) 2(ind)

Certification Liaison Process 3 3 3 3

Totals 66 65 57 28

Table shows number of objectives per process category


“ind” Ö need to show that objective is satisfied “with independence” 9
Sample DO-178B Objectives [Table A-5]
Verification of Outputs
p of
Software Coding and Integration Processes
Objective Level Output
Source Code complies with low-level ABC
requirements

Source Code complies with software ABC


architecture
Source Code is verifiable AB
Source Code conforms to standards ABC Software Verification Results

Source Code is traceable to low-level ABC


requirements
Source Code is accurate and consistent ABC
Output of software integration process is ABC
complete and correct

Underlining of level Ö “objective should be satisfied with independence” 10


Some Issues related to Table A-5 Objectives [§6.3.4]
Reviews and Analyses of the Source Code
Conformance to standards
• Complexity restrictions
• Code constraints consistent with system safety objectives
Accuracy and consistency
• Stack usage
• Fixed
Fixed-point
point arithmetic overflow and resolution
• Resource contention
• Worst-case execution timing
• Exception handling
• Use of uninitialized variables or constants
• Unused variables or constants
• Data
D t corruption
ti d due tto ttask
k or iinterrupt
t t conflicts
fli t

11
Sample DO-178B Objectives [Table A-7]
Verification of Verification Process Results
Objective Level Output
Test procedures are correct ABC Software Verification
Cases and Procedures
Test results are correct and discrepancies explained ABC
Test coverage of high-level requirements is achieved ABCD
Test coverage of low
low-level
level requirements is achieved ABC
Test coverage of software structure (modified A
condition/decision coverage) is achieved
Test coverage of software structure (decision coverage) is AB Software
S ft V
Verification
ifi ti
achieved Results
Test coverage of software structure (statement coverage) is ABC
achieved
Test coverage of software structure (data coupling and ABC
control coupling) is achieved

Underlining of level Ö “objective should be satisfied with independence” 12


Role of Testing in Software Verification
Test cases are to be derived from software requirements
• Requirements-based hardware/software integration testing
• Requirements-based software integration testing
• Requirements-based low-level testing
Test cases must fully cover the code
• Unexercised code may be due to any of several reasons
ƒ Missing
g requirement
q Ö Add new requirement
q
ƒ Missing test Ö Add new test case
ƒ Dead code Ö Remove it
ƒ Deactivated code Ö Show that it will not be executed
• Coverage on source versus object code
ƒ May be demonstrated on Source Code for Levels B and below
ƒ May be demonstrated on Source Code for Level A unless compiler generates object
code not directly traceable to Source Code
• Then need additional verification on object code to establish correctness of such
generated code
Structural coverage is not “white box” testing
• Need to show that all exercised code is traceable to requirements
13
Required Coverage Depends on Safety Level

Level Level Level Level


A B C D

Statement Coverage
* Every statement has been invoked at least once 9 9 9

Decision Coverage
* Described below 9 9

Modified Condition / Decision Coverage


* Described below 9

14
Decision Coverage
“Condition”
• A Boolean expression containing no Boolean operators; e.g., X>0
“Decision”
• A Boolean expression composed of conditions and zero or more Boolean operators; e.g.,
X>0 and Y=2
• If a condition appears more than once in a decision, each occurrence is a distinct condition
ƒ X>0 and X>0 has two conditions
Decision coverage
• Every point of entry and exit in the program has been invoked at least once
• Every decision in the program has taken all possible outcomes at least once

One test sufficient for statement coverage


if X>0 and Y=2 then X = 1, Y = 2 Ö True Ö if statement, assignment
Z := X+Y;
end if;
Two tests sufficient for decision coverage
X = 1,, Y = 2 Ö True Ö if statement,, assignment
g
X = 0, Y = 2 Ö False Ö if statement

15
Modified Condition / Decision Coverage
MC / DC = Decision coverage + additional requirements
• Every condition in a decision in the program has taken all possible outcomes at least once
• Each condition in a decision has been shown to independently affect that decision's
outcome
ƒ With all other conditions constant, changing the truth value of the condition
changes the result of the decision

if X>0 and Y=2 then X>0 Y=2 X>0 and Y=2


Z := X+Y; True True True
end if;
True False False
False True False
False False False
Need 3 tests* for MCDC
ƒ (True True)
(True, True), (True
(True, False)
False), (False
(False, False)
Choose one of these test vectors
ƒ (True, True), (False, True), (False, False)
For further information see tutorial NASA / TM-2001-210876
ƒ AP
Practical
ti l TTutorial
t i l on M
Modified
difi d C
Condition
diti / D
Decision
i i C Coverage, by
b KKelly
ll Hayhurst,
H h t
Dan Veerhusen, John Chilenski, Leanna Rierson

* In general, at least n+1 tests are needed for a decision with n conditions 16
Related Standards and Other Material
DO-248B
• Provides clarification of DO-178B guidance material (does not introduce new guidance)
ƒ 12 errata corrections
ƒ 76 Frequently-Asked Questions (FAQ)
ƒ 15 Discussion Papers
CAST Papers (Certification Authority Software Team)
• Further clarification of DO-178B
DO-278
• Guidelines for Communication, Navigation, Surveillance, and Air Traffic Management
(CNS/ATM) Systems Software Integrity Assurance
• Applies to software DO-278 Assurance Level DO-178B Software Level
in CNS/ATM
AL1 A
systems
y used in
ground- or space- AL2 B
based applications AL3 C
that affects aircraft AL4 No equivalent
q
safety
AL5 D
• Largely based on
DO-178B AL6 E
17
Navigating the Software Safety Certification Waters

Software
D
Developers
l
DO-178B

DO-278
DER

DO-248B:
DO 248B:
• Errata in DO-178B
• FAQ Certification
• Discussion Papers Authorities
S ft
Software Team
T
(CAST) Papers

18
Why Revise DO-178B?
Address / accommodate “new” software technologies
• Object-Oriented Programming
• Model-based design / automatic code generators
• COTS software and tools, including real-time operating systems
Recognize that there is more to software verification than testing
• Formal methods
• Abstract interpretation
p
Consider combining with ground-based safety standard (DO-278)
Take into account the supplementary papers / commentaries on DO-178B
• Certification
C tifi ti Authorities
A th iti Software
S ft Team
T (CAST) papers
• Issues Papers (IPs)
Try to remove/reduce the need for DERs to make subjective judgments
Correct errors
• Example: requirements on development tool qualification don’t distinguish the host
environment (in which the tool runs) from the target environment (in which the system runs)

19
DO-178C “Terms of Reference”
Objectives
• Promote safe implementation of aeronautical software
• Provide clear and consistent ties with the systems and safety processes
• Address emerging software trends and technologies
• Implement an approach that can change with the technology
Activities (partial)
• Modify
y DO-178B
• Develop supplements to document technology-specific or method-specific guidance and
guidelines
• Develop
p and document rationale for each DO-178B objective
j
Other considerations (partial)
• Maintain technology-independent nature of DO-178B objectives
• Modifications to DO-178B
DO 178B should:
ƒ Strive to minimize changes to existing text
ƒ Consider economic impact relative to system certification without compromising
system safety
ƒ Address clear errors or inconsistencies / fill any clear gaps in DO-178B
ƒ Meet a documented need to a defined assurance benefit
20
Initial Controversy over DO-178C approach (1)
C. Michael Holloway (NASA), message posted to SC205/WG71 e-mail forum,
5 April 2005: Are We Heading in the Right Direction?

“The trend in the world (at least outside the U.S.) seems to be away from
prescriptive standards and towards goal-based standards,
standards which require
the presentation of cogent arguments for safety (and other desired attributes),
rather than the simple completion of certain prescribed processes.

This is a good trend. Although DO-178B/ED-12B has some attributes of a


goal-based standard, for the most part it is a prescriptive standard of the sort
that is increasingly difficult to justify on technical grounds.

The TOR's [Terms of Reference for DO-178C] insistence on minimizing changes


to the current document is likely to ensure that DO-178C/ED-12C remains a
mostly prescriptive standard. This is not, in my opinion, a good thing.”

21
Initial Controversy over DO-178C approach (2)
Martyn Thomas (Praxis), message posted to SC205/WG71 e-mail forum,
10 August 2005: The Right Direction?

“I strongly support the proposal that safety standards should move away from
prescribing specific processes to requiring evidence that the delivered system
has the required properties. The relationship between development processes
and the properties of the resulting systems is simply too weak to justify any
confidence that a system will be safe just because it has been developed in the
specific way prescribed by some standard.

...the core of safety certification must be formal reasoning about properties


(ideally assisted by automated analysis tools)
tools). That
That, in turn
turn, requires that the
safety requirements are stated with mathematical rigour.

...certification should depend on a mathematically sound argument that the


d li
delivered d system
t h
has th
the necessary safety
f t properties…”
ti ”

22
Rationale for DO-178C’s Approach
Alternatives
• Major surgery on DO-178B
ƒ Remove prescriptive content, making document goal-oriented
ƒ Add supplements for requirements for specific approaches
• Minor surgery
ƒ Address new technologies in supplements when prescribed approaches may be
replaced or augmented by others (e.g., formal verification)
Decision was to go with “minor surgery”
• Core document fixes known issues and adds clarifications (e.g. from DO-248B)
gy p
• Technology-specific supplements
pp “add,, delete or otherwise modify
y objectives,
j , activities,,
explanatory text, and software life cycle data in DO-178C/ED-12C”
Reasoning
• DO
DO-178B
178B works
ƒ No commercial aviation deaths caused by software that had been certified to Level A
• Experience with goal-based approach has been mixed
ƒ Change of mindset required for both developer and regulator
ƒ Safety cases tend to focus on individual hazards, but most accidents are due to a
combination of factors
23
DO-178C Development Framework (1)
Joint RTCA Special Committee 205 and EUROCAE* Working Group 71
• “SC-205 WG-71”, officially abbreviated as “SCWG”
• Co-chairs: Jim Krodel (US), Gérard Ladier (Europe)
• Secretaries: Leslie Alford (US), Ross Hannan (Europe)
• Government agency representatives: Barbara Lingberg (FAA), Jean-Luc Delamaide (EASA)
Web site (registration required)
• ultra.pr.erau.edu/SCAS/
p
Subgroups
• SG1: SCWG Document Integration (Marty Gasiorowski / Ron Ashpole)
• SG2: Issues and Rationale (Fred Moyer / Ross Hannan)
• SG3: Tool Qualification (Leanna Rierson / Frédéric Pothon)
• SG4: Model Based Design and Verification (Mark Lillis / Pierre Lionne)
• SG5:
SG5 Obj
Object-Oriented
t O i t d Technology
T h l (G
(Greg Milli
Millican / Jan-Hendrik
J H d ik Boelens)
B l )
• SG6: Formal Methods (Kelly Hayhurst / Duncan Brown)
• SG7: CNS/ATM and Safety (Don Heck / David Hawken)
ƒ “CNS/ATM” = Communications, Navigation and Surveillance / Air Traffic Management

* European Organisation for Civil Aviation Equipment (www.eurocae.org) 24


DO-178C Development Framework (2)
Revision effort began in 2005, expected to complete in December 2010
• Two plenary meetings per year, alternating between US and Europe
The group is completely open, based on volunteer’s participation
About 110 regular
g participants
p p in 2009
• Many one-time visitors
• Some plenaries had more than 200 participants
About half from the US and half from Europe
• Europe: mostly Great Britain, France, Germany, Sweden
• Some participation from Brazil and recently China
Th
Three main
i kinds
ki d off organizations
i ti represented
t d
• Airframe industry and contractors (Boeing, Airbus, Lockheed, Rockwell Collins, Honeywell,
Thales, Eurocopter, …)
• Government agencies / certification authorities and DERs (FAA, EASA, NASA, Transport
Canada, DGAC…)
• Tool vendors (LDRA, Esterel, Mathworks, AdaCore, Verocel, …)
Plenary vote for acceptance
• “Almost-unanimity” is the rule
25
Expected Result(s)

DO-278 Changes DO-278A Merging DO-278 into DO-178C was


equivalent initially proposed but eventually
(CNS/ATM) to DO-178C core rejected as too big a change

Formal
DO-178B MBD OOT
Clarification DO-178C methods
fixes Suppl. Suppl. Suppl.
core
Overriding Supplements
12.3

DO 248B
DO-248B Clarification
fixes
DO 248C
DO-248C
Tool
Qual.
Suppl.
26
Example of Change in Core Document: Figure 2-1

System User /
Functional Requirements

Safety Assessment Process


System Approval
Functional Hazard Analysis Activities
Prel. System Safety Assessment
System
Safety
DO-178B Assessment Functional/Req. System V&V
Allocations Activities
System Dessign and V&V process / S

Other Requirements System requirements allocated to S/W


S/W Development Assurance Level
Hardware Definition Compliance Data
Other processes

Software
Software Life Cycle Proceesses

Planning Process

Derived
High Level Software Requirements
Process
DO-178C Requirements
Software
Verification Process
(IP 50, Rev. K) Derived
Low Level
Software Design
Process
Requirements
and Software
S

Architecture S f
Software C
Coding
di and d
Integration Processes

27
Example of Change: Section 2.2
DO-178B
2.2 Failure Condition and Software Level

The failure condition of a system is established by determining the severity
of failure conditions on the aircraft and its occupants. An error in the software
may cause a fault that contributes to a failure condition. Thus, the level
of software integrity necessary for safe operation is related to the system failure
conditions.

DO-178C (IP 50, Rev. K)


2.2 System Safety Assessment Process and Software Development Assurance Level

The SWDAL of a software component is determined as part of the system safety
assessment process by establishing how an error in a software component relates to
the system failure condition(s) and the severity of that failure condition(s).

System boundary
Fig. 2-2 Chain of Events for Software Error
Software boundary Leading to Failure Condition

Software error Fault System Failure condition


(latent) failure effect

28
Tool Qualification Supplement (1)

Qualification needed when processes are eliminated,


eliminated
reduced or automated without manual verification

DO-178B O C
DO-178C

2 cases 3 criteria and 5 levels (TQL)

Development Tool Criterion 1 Could insert an error


Could fail to detect an error
and is used to reduce other
C it i 2
Criterion development or verification
activities

Verification Tool Criterion 3 Could fail to detect an error

29
Tool Qualification Supplement (2)

Example: Criterion 2 versus Criterion 3

Proof Tool Source code verification Criterion 3


+
Reduce robustness testing Criterion 2

Static Analysis Tool Source code review Criterion 3


+
Remove defensive code Criterion 2

30
Tool Qualification Supplement (3)

Mostly for Formal Methods & Model-Based Design

Software Criteria
Level
1 2 3 TQL 1: DO-178 level A

A TQL-1 TQL-4 TQL-5 TQL 2: DO-178 level B

TQL3: DO-178 level C


B TQL-2 TQL-4 TQL-5
TQL4 : Complete requirements
C TQL 3
TQL-3 TQL 5
TQL-5 TQL 5
TQL-5 Describe architecture
More verifications
D TQL-4 TQL-5 TQL-5
TQL5 : TOR verification

Tool Qualification Supplement has same structure as DO-178C


• Tool Planning Process
Process, Tool Development Process
Process, etc
• TQL (Tool Quality Level) analogous to SWDAL

31
Model-Based Development and Verification
A model is an abstract representation of system characteristics
• Specification model expresses high-level requirements (eg functions, performance, safety)
• Design model expresses low-level requirements / architecture
ƒ Data structures, data flow, control flow
• Verification model represents life cycle data of verification process
Benefits
• Unambiguous
g notation
• Supports use of automated code generation, automated test generation
Model-Based Development/Verification Supplement
• Acknowledges need for “Higher
Higher-Level
Level Requirements”
Requirements from which development model is
derived
• Model usage affects most of the DO-178C-specified life cycle processes
• Traceability,
Traceability etc
etc, required for source program are required for design model
ƒ Demonstrate property for model, show that the property is preserved in the source
code
• Model simulator may be useful tool

32
Object-Oriented Technology (“OOT”)
What is OOT?
• Software development methodology supported by language features
ƒ Primary focus is on data elements and their relationships
ƒ Secondary focus is on the processing that is performed
• Applicable during entire software “life cycle”
Language concepts (OOP, or “Object-Oriented Programming”)
• Object
j = state ((“attributes”)) + operations
p ((“methods”))
• Class = module + object creation template Object-Oriented Design (“OOD”)
also known as
• Encapsulation = separation of interface (spec for
Object-Based Programming
methods)) from implementation
p ((state,, algorithms)
g )
• Inheritance = specialization (“is-a”) relationship between classes
ƒ Extend a class, adding new state and adding/ overriding operations
• Polymorphism = ability of a variable to reference objects from different classes at different
times
• Dynamic binding (“dispatching”) = interpretation of operation applied to polymorphic
variable based on current class of referenced object
j

33
Object-Oriented Technology (“OOT”), cont’d.
Additional OOP elements
• Single versus multiple inheritance
ƒ “Interface” for a simple form of multiple inheritance
• Use of constructors / destructors
Related language features
• Method overloading
yp conversion
• Type
• Inline expansion
Other modern language features that complicate safety certification
• Generic templates
• Exceptions
• Concurrency

34
OO Technology & Safety Certification
Why consider OOT for safety-critical software?
• Data-centric approach eases maintenance of many large systems
• Model-driven architecture / UML tools may generate OO code to be certified
• Many programmers know OO languages such as C++, Java, or Ada 95
• Languages (such as Ada) used for safety-critical systems have OO features
• May want to take OO legacy code and apply DO-178 à posteriori
• Issues explored at NASA/FAA Object-Oriented Technology in Aviation (OOTiA) workshops
What’s the catch?
• Paradigm clash
ƒ OOT’s
OOT s distribution of functionality across classes
classes, versus safety analysis’s
analysis s focus on
tracing between requirements and implemented functions
• Technical issues
ƒ The features that are the essence of OOP complicate safety certification and raise
security issues (e.g. ensuring integrity of “vtable”)
ƒ Dynamic memory allocation, VM implementations
• Cultural issues
ƒ Many DERs / TOE evaluation personnel are not language experts and are (rightfully)
concerned about how to deal with unfamiliar technology
35
OOT Supplement
New OOT-specific objectives and activities for Software Verification Process
• Based on Liskov Substitution Principle:
ƒ “Let q(x) be a property provable about objects x of type T. Then q(y) should be true for
objects of type S where S is a subtype of T”
OO.6.5 Local Type Consistency Verification
The use of inheritance with method overriding and dynamic dispatch requires additional verification
activities that can be done either by testing or by formal analysis.

OO.6.5.1 Local Type Consistency Verification Objective


Verify that all type substitutions are safe by testing or formal analysis.

OO.6.5.2 Local Type Consistency Verification Activity


For each subtype where substitution is used, perform one of the following:
• formally
formall verify
erif ssubstitutability,
bstit tabilit
• ensure that each class passes all the tests of all its parent types which the class can replace, or
• for each call point, test every method that can be invoked at that call point (pessimistic testing).

• New g
guidance for dynamic
y memory
y management
g verification
ƒ Need to verify absence of problems such as dangling references, fragmentation,
storage exhaustion, unbounded allocation or deallocation time
New objectives and activities related to virtualization (OO.E.1.7)
• Issue of code versus data (objectives generally apply to code, not data)
Virtualization defines an intermediate execution platform…. One must ensure that all data that is
used as executable code for the execution platform be verified as executable code. 36
Formal Methods Supplement: Concepts
Formal Method = Formal Model + Formal Analysis
• May be applied at various stages in software development
• Formal methods are used as a verification technique
Formal Model
• System abstraction with unambiguous, mathematically defined syntax and semantics
• Examples
ƒ Graphical
p models ((state machines))
ƒ Text-based (Z, set theory, programming language subsets)
• May be used to capture some system properties (e.g. Worst-Case Execution Time)
Formal Analysis
• Provides guarantees/proofs of software properties, compliance with requirements
• An analysis method can only be regarded as formal if it is sound
ƒ It never asserts
t th
thatt a property
t hholds
ld if it iis possible
ibl ffor th
the property
t tto nott h
hold
ld
ƒ Converse is a usability issue
• Kinds of formal analysis
ƒ Deductive (theorem proving)
ƒ Model checking Implemented in (qualified) tools
ƒ Abstract interpretation 37
Formal Methods Supplement*
Mainly adapts Software Verification Process section of DO-178C
• Goal is to prevent and eliminate requirements, design and code errors throughout the
software development processes
Formal methods are complementary to testing
• Testing shows that functional requirements are satisfied and detects errors
• Formal methods can increase confidence that no anomalous behavior will occur
ƒ May find faults that are not detected by testing
Formal methods cannot establish verification evidence for the target hardware
• Therefore testing on the target is still required
• But: formal analysis of source code can be used to reach [compliance with Low-Level
Low Level
Requirements] provided that complementary analyses show the property preservation
between source code and object code
Uses of Formal Methods
• Formal specification of life-cycle artifacts
• Formal derivation of life-cycle artifacts
• Formal analysis of life
life-cycle
cycle artifacts

* This slide largely derived from Working towards DO-178C/ED-12C, DO-248C/ED-94C and
DO-278A/ED-109A, invited presentation by James Chelini (Verocel) at ACM SIGAda 2009
38
Navigating the Software Safety Certification Waters

Software
DO-178C Developers

DO-278A
DER

Tool Qualification
Supplement
Model-Based Development
and Verification Supplement
Object-Oriented
Object Oriented Technology
Supplement

Formal Methods Technology


Supplement
39
References / Web Resources
DO-178B
• RTCA SC-167 / EUROCAE WG-12. RTCA/DO-178B – Software Considerations in Airborne
Systems and Equipment Certification, December 1992
• Certification Authority Software Team (CAST):
www.faa.gov/aircraft/air_cert/design_approvals/air_software/cast/cast_papers/
f / i f/ i /d i l / i f / / /
DO-178C
• SC-205/WG-71 website: forum.pr.erau.edu/SCAS
Open-DO
• Community initiative to promote open-source software and lean/agile methods in
developing and certifying high-integrity systems
• www.open-do.org
University of York (UK) Safety-Critical Mailing List Archives
• www.cs.york.ac.uk/hise/safety-critical-archive
www.cs.york.ac.uk/hise/safety critical archive
Object-Oriented Technology and Safety Certification
• Handbook for Object-Oriented Technology in Aviation (OOTiA), October 2004.
www faa gov/aircraft/air cert/design approvals/air software/oot
www.faa.gov/aircraft/air_cert/design_approvals/air_software/oot

40
Acronyms and Abbreviations
ATM Air Traffic Management
CAST Certification Authority Software Team
CNS Communications, Navigation, and Surveillance
COTS Commercial Off-The-Shelf
Off The Shelf
DO-178B, DO-178C [Not acronyms, these are names/numbers of documents]
DER Designated Engineering Representative
EUROCAE European Organisation for Civil Aviation Equipment
FAA Federal Aviation Administration
MC/DC Modified Condition/Decision Coverage
g
OOP Object-Oriented Programming
OOT Object-Oriented Technology
OOTiA Object-Oriented
Obj t O i t d Technology
T h l in
i Aviation
A i ti
RTCA [Not an acronym, this is the name of an organization]

41

You might also like