Best IV &V Practices

Pradeep P. Oak, MTech, CQA

Oak Systems Pvt Ltd, Bengaluru

[email protected]
Dec 2011 Best I V&V Practices IVV/OAK/2.0.1 1
www.oaksys.net
 Agenda
• Introduction
• Software characteristics
• IV&V best practices
• Q&A

IVV

Dec 2011 Best I V&V Practices IVV/OAK/2.0.1 2

www.oaksys.net
 Oak Systems 1998 – 2011
– Specialist Independent Software V&V/Testing company
– Established in 1998
– ISO 9001 certified since 2003
– Over 100 V&V specialists
– CEMILAC approved V&V company – since 2006
– Experience on IV&V/Testing under IEEE12207, MIL2167A,
DO178B, CMMI frameworks
– Offices in Bangalore, Pune
– Established 3 CoEs for Research with Engg colleges
– More details at www.oaksys.net

Dec 2011 Best I V&V Practices IVV/OAK/2.0.1 3

www.oaksys.net
 Contribution of Software

Contribution of software in implementing a system is

ever increasing.
 Moving from conventional mechanical systems to intelligent, adaptive Control systems

User demands more …

• Reduced operator workload
• Enhanced mission capability
• Fail safe system
• User friendly HMI
• Improved reliability
• Improved maintainability
Dec 2011 Best I V&V Practices IVV/OAK/2.0.1 4
www.oaksys.net
 Software/System Characteristics
Challenges in handling software
• Software is a logical entity
• It is easy to change software (impact can be varied)

Software systems may be of varied complexity ..

• Safety critical / Mission critical /Failsafe
• Supporting functions
• Utility functions
• Maintenance functions
• etc..

Software Dependability cannot be guaranteed from an assessment

of the finished software product
Answer is to develop software in a controlled and disciplined environment
Dec 2011 Best I V&V Practices IVV/OAK/2.0.1 5
www.oaksys.net
Software - controlled & disciplined approach
Number of standards and guidelines have
emerged to facilitate construction in a
disciplined and controlled manner in order
to better its ‘fitness for purpose’

• DO178 - guideline
• IEEE12207/ IEEE1012 - standard
• MIL 2167A - standard
• Ddpmas - guideline
•…
These are not just for V&V purposes ..
Dec 2011 Best I V&V Practices IVV/OAK/2.0.1 6
www.oaksys.net
 DO – 178B

"Software Considerations in Airborne

Systems and Equipment Certification"
• is a process-oriented document. For each process,
objectives are defined and a means of satisfying these
objectives are described.

• is intended to provide the aviation community with

guidance for determining, in a consistent manner and
with an acceptable level of confidence, that the
software aspects of airborne systems and equipment
comply with airworthiness requirements
Dec 2011 Best I V&V Practices IVV/OAK/2.0.1 7
www.oaksys.net
 DO-178B Software Levels
Software level is based upon the contribution of software to potential failure
conditions as determined by the system safety assessment process.

Dec 2011 Best I V&V Practices IVV/OAK/2.0.1 8

www.oaksys.net
 DO-178B Processes and Outputs
DO-178B is divided into six main processes:
Software Planning Processes
Software Development Processes
Software Verification Processes
Software Configuration Management Processes
Software Quality Assurance Processes
Certification Liaison Processes

Each process has a set of expected documented outputs

Dec 2011 Best I V&V Practices IVV/OAK/2.0.1 9

www.oaksys.net
 DO-178B Software Life Cycle Model

Dec 2011 Best I V&V Practices IVV/OAK/2.0.1 10

www.oaksys.net
 Requirements Phase IEEE 12207 - Development and IV&V
•System Reqts
Analysis
Design Phase Activities
•S/W Reqts Analysis •Design Analysis Code Phase
•Interface Analysis •Interface Analysis
•Process Analysis •Test Program •Code Analysis Test Phase
•Technical Reviews & Analysis •Test Program
Audits •Supportability Analysis •Test Program
Analysis •Supportability Analysis
•Process Analysis Analysis •Independent Test
•Technical Reviews & •Process Analysis •Supportability
Audits •Technical Reviews & Analysis
Audits •Technical Reviews &
Audits

Verify

Verify

Verify
Validate
Catastrophic/Critical/High Risk Functions List
Traceability Analysis
Issues Tracking
Metrics Assessment
Loading Analysis
Change Impact Analysis
Special Studies
Dec 2011 Best I V&V Practices IVV/OAK/2.0.1 11
www.oaksys.net
 Goal of IV &V
To produce high quality, reliable software.
Use Independent Verification and Validation
(IV&V) in an independent, systematic evaluation
process throughout the software life cycle.
Using the IV&V process; locate, identify, and
correct software problems and errors early in the
development cycle.

Dec 2011 Best I V&V Practices IVV/OAK/2.0.1 12

www.oaksys.net
 Independent Verification & Validation
Software IV&V is a systems engineering process
employing rigorous methodologies for evaluating the
correctness and quality of the software product throughout
the software life cycle
Independent
– Technical: IV&V prioritizes its own efforts
– Managerial: Independent reporting route to Program Management
– Financial: Budget is allocated by program and controlled at high level such
that IV&V effectiveness is not compromised

Verification (Are we building the product right?)

Validation (Are we building the right product?)

Dec 2011 Best I V&V Practices IVV/OAK/2.0.1 13
www.oaksys.net
 Verification Process
Verification process to provide objective evidence whether
software & associated products & processes -

Conform to requirements (e.g., for correctness, completeness,
consistency, accuracy) for all life cycle activities during each
life cycle process (acquisition, supply, development,
operation, and maintenance)

Satisfy standards, practices, and conventions during life cycle
processes

Successfully complete each life cycle activity and satisfy all
the criteria for initiating succeeding lifecycle activities (e.g.,
building the software correctly)

Dec 2011 Best I V&V Practices IVV/OAK/2.0.1 14

www.oaksys.net
 Validation Process
Validation process to provide evidence whether the
software and its associated products and
processes

Satisfy system requirements allocated to software at
the end of each life cycle activity

Solve the right problem (e.g., correctly model
physical laws, implement business rules, use the
proper system assumptions)

Satisfy intended use and user needs

Dec 2011 Best I V&V Practices IVV/OAK/2.0.1 15

www.oaksys.net
 IV&V Approach
Traditional Software Development
Req Design Code Test (Verification & Validation) V&V
Unit Integration Acceptance

Req Design Code Testing

Unit
Clean Room Approach
iV&V
Test (Verification & Validation)
Integration Acceptance

Req Design Code Test (Verification & Validation)

Unit Integration Acceptance
IV&V
IV&V Implementation
Dec 2011 Best I V&V Practices IVV/OAK/2.0.1 16
www.oaksys.net
 Software Verification Process

The purpose is to identify and report any errors resulting from the
development process.

Verification is not just Testing.

The verification process objectives are satisfied through a

combination of reviews, analysis, execution of walkthroughs,
unit testing, integration testing, and more.

Reviews, walkthrus & Analyses are spread throughout the SDLC

Dec 2011 Best I V&V Practices IVV/OAK/2.0.1 17

www.oaksys.net
DO178B – Software Verification Process - Testing

Dec 2011 Best I V&V Practices IVV/OAK/2.0.1 18

www.oaksys.net
 DO178B – S/w Verification – Types of testing

Hardware/software integration testing: To verify

correct operation of the software in the target
computer environment.
Software integration testing: To verify the
interrelationships between software
requirements and components and to verify the
implementation of the software requirements and
software components within the software
architecture.
Low-level testing: To verify the implementation of
software low-level requirements.
Dec 2011 Best I V&V Practices IVV/OAK/2.0.1 19
www.oaksys.net
 DO178B – Software Verification Process - Testing

Sl no. Objective Level

A B C D E
1 Test procedures are correct. √ √ √
2 Test results are correct and √ √ √
discrepancies explained.
3 Test coverage of high-level √ √ √ √
requirements is achieved.
4 Test coverage of low-level √ √ √
requirements is achieved.
5 Test coverage of software structure √
(modified condition/decision) is
achieved.
6 Test coverage of software structure √ √
(decision coverage) is achieved.
7 Test coverage of software structure √ √ √
(statement coverage) is achieved.
8 Test coverage of software structure √ √ √
(data coupling and control coupling) is
achieved.
Dec 2011 Best I V&V Practices IVV/OAK/2.0.1 20
www.oaksys.net
 Software V&V Plan
IEEE12207 / IEEE1012
V&V processes
− Management of V&V
− Acquisition support V&V
− Development
− Planning V&V
− Concept V&V
− Requirements V&V
− Design V&V
− Implementation V&V
− Test V&V
− Installation and checkout V&V
− Operation V&V
− Maintenance V&V

Dec 2011 Best I V&V Practices IVV/OAK/2.0.1 21

www.oaksys.net
 IV & V – Technical rationale
IV&V has proven to be a necessary function for
software projects of all sizes, but is particularly
important in large, critical software packages.

Although the level of IV&V performed will be

proportional to project size, its performance in
embedded /flight software projects is widely
considered to be mandatory.
Dec 2011 Best I V&V Practices IVV/OAK/2.0.1 22
www.oaksys.net
 IV&V Benefits

Technical Management

Better Visibility into

Better software/system
Development
Performance

Higher Confidence in Better Decision Criteria

Software Reliability
Second Source Technical
Alternative
Compliance between
Specs & Code
Reduced maintenance cost
Criteria for Program
Reduced Frequency of
Acceptance
Operational Change

Dec 2011 Best I V&V Practices IVV/OAK/2.0.1 23

www.oaksys.net
 IV & V - Impact of Nonpractice

Failure to perform IV&V for software projects

could result in software system weaknesses,
performance of unintentional functions, and
failure of the system and the mission.

Anything less than a methodical, systematic

rigorous treatment of IV&V could cause loss of
mission, life, and valuable resources.

Dec 2011 Best I V&V Practices IVV/OAK/2.0.1 24

www.oaksys.net
 Conclusion
• Applied early in the software development process, IV&V
can reduce overall Project cost
• Org. policy should provide the management process for
assuring that the right level of IV&V is applied
• IV&V Implementation Criteria provide a quantitative
approach for determining the right level based on mission
risk
• IV&V CANNOT replace Quality assurance but must
supplement it to be successful
• IV&V Requires a strong Quality assurance base

Dec 2011 Best I V&V Practices IVV/OAK/2.0.1 25

www.oaksys.net
 Any Questions ?

Dec 2011 Best I V&V Practices IVV/OAK/2.0.1 26

www.oaksys.net
 Thank You

#27, NGEF Lane, Off 100 Feet Road,

Indiranagar, Bengaluru – 560 038, INDIA
Tel : +91-80-41 26 76 81 Email: [email protected]
URL: http://www.oaksys.net

Dec 2011 Best I V&V Practices IVV/OAK/2.0.1 27

www.oaksys.net