ITEA2 Project

Call 6 11025
2012 – 2015
OETCS/WP4/D4.1V01.00

Work Package 4: “Validation & Verification Strategy”

openETCS Validation & Verification Plan

Version 01.00

Hardi Hungar (Ed.) 04 October 2013

Contributions by:
Frederic Badeau (Systerel), Marc Behrens (DLR),
Cecile Braunstein (U Bremen), Cyril Cornu (All4Tec),
Christophe Gaston (CEA), Jens Gerlach (Fraunhofer),
Ainhoa Gracia (SQS), Hardi Hungar (DLR),
Stephan Jagusch (AEbt), Alexander Nitsch (U Rostock),
Jan Peleska (U Bremen), Marielle Petit-Doche (Systerel),
Virgile Prevosto (CEA), Stefan Rieger (TWT),
Izaskun de la Torre (SQS), Jan Welte(TU-BS)
openETCS

This work is licensed under the "openETCS Open License Terms" (oOLT) dual Licensing:
This page is intentionally left blank
OETCS/WP4/D4.1V01.00 i

Work Package 4: “Validation & Verification Strategy” OETCS/WP4/D4.1V01.00

04 October 2013

openETCS Validation & Verification Plan

Version 01.00

Hardi Hungar (Ed.)

Contributions by:
Frederic Badeau (Systerel), Marc Behrens (DLR),
Cecile Braunstein (U Bremen), Cyril Cornu (All4Tec),
Christophe Gaston (CEA), Jens Gerlach (Fraunhofer),
Ainhoa Gracia (SQS), Hardi Hungar (DLR),
Stephan Jagusch (AEbt), Alexander Nitsch (U Rostock),
Jan Peleska (U Bremen), Marielle Petit-Doche (Systerel),
Virgile Prevosto (CEA), Stefan Rieger (TWT),
Izaskun de la Torre (SQS), Jan Welte(TU-BS)
DLR
Lilienthalplatz 7
38108 Brunswick, Germany
eMail:[email protected]

Deliverable

Prepared for openETCS@ITEA2 Project

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 ii

Abstract: This document describes strategy and plan of the verification and validation activ-
ities in the project openETCS. As the goals of the project include the selection, adaption and
construction of methods and tools for a FLOSS development in addition to performing actual
development steps, differing from the plan for a full development project, the plan covers also
activities evaluating the suitability of methods and tools, and it makes provisions for incorporation
of V&V of partial developments which are actually done.

The overall strategy is to support the design process as specified in D2.3 and its partial instan-
tiations within openETCS. In accordance with the project approach, V&V shall be done in a
FLOSS style, and it has to suit a model-based development. A further main consideration shall
be to strive for conformance with the requirements of the standards (EN 50128 and further). This
means that the contribution of all activities to a complete verification and validation shall be
defined and assessed.

The plan details how to perform verification & validation for a complete development which
follows the process sketch from D2.3, so that the result conforms to the requirements of the
standards for a SIL 4 development. This includes a definition of activities, the documentation
to be produced, the organisation structure, roles, a selection of methods and tools, a format for
describing design artifacts subject to V&V, and a feedback format for the findings during V&V.

As D2.3 gives only a rough description of the development steps and not yet a complete list of
design artifacts, nor one of methods applied and formats to be used, this first version of the V&V
plan will also lack detail which will to be added in later revisions as these informations become
more concrete.

Besides the usual purpose of verification & validation activities, namely evaluating and proving
the suitability of design artifacts, V&V in openETCS will also generate information on the
suitability of the methods and tools employed. For that purpose, a format for describing methods
and tools to be used in V&V and one for summarizing the findings about the suitability are
defined.

The plan also contains partial instantiations of V&V which match partial developments that are
realised within openETCS.

Disclaimer: This work is licensed under the "openETCS Open License Terms" (oOLT) dual Licensing: European Union Public
Licence (EUPL v.1.1+) AND Creative Commons Attribution-ShareAlike 3.0 – (cc by-sa 3.0)

THE WORK IS PROVIDED UNDER openETCS OPEN LICENSE TERMS (oOLT) WHICH IS A DUAL LICENSE AGREEMENT IN-
CLUDING THE TERMS OF THE EUROPEAN UNION PUBLIC LICENSE (VERSION 1.1 OR ANY LATER VERSION) AND THE
TERMS OF THE CREATIVE COMMONS PUBLIC LICENSE ("CCPL"). THE WORK IS PROTECTED BY COPYRIGHT AND/OR
OTHER APPLICABLE LAW. ANY USE OF THE WORK OTHER THAN AS AUTHORIZED UNDER THIS OLT LICENSE OR COPY-
RIGHT LAW IS PROHIBITED.

BY EXERCISING ANY RIGHTS TO THE WORK PROVIDED HERE, YOU ACCEPT AND AGREE TO BE BOUND BY THE TERMS
OF THIS LICENSE. TO THE EXTENT THIS LICENSE MAY BE CONSIDERED TO BE A CONTRACT, THE LICENSOR GRANTS
YOU THE RIGHTS CONTAINED HERE IN CONSIDERATION OF YOUR ACCEPTANCE OF SUCH TERMS AND CONDITIONS.

http://creativecommons.org/licenses/by-sa/3.0/
http://joinup.ec.europa.eu/software/page/eupl/licence-eupl

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 iii

Table of Contents
Figures and Tables................................................................................................................... v

Document Control................................................................................................................... vi

1 Introduction....................................................................................................................... 1

1.1 Purpose ..................................................................................................................... 1

1.2 Document Structure...................................................................................................... 1
1.3 Background Information................................................................................................. 1
1.3.1 Definitions ......................................................................................................... 2

2 Document Evolution ........................................................................................................... 4

3 Verification & Validation in the Design Process ...................................................................... 5

4 Verification & Validation Strategy ......................................................................................... 6

4.1 Verification & Validation Strategy for a Full Development ...................................................... 6

4.1.1 Verification Strategy for a Full Development .............................................................. 6
4.1.2 Validation Strategy for a Full Development ............................................................... 8
4.1.3 Safety Interface ................................................................................................. 10
4.2 Verification & Validation Strategy for openETCS ................................................................. 10

5 Verification & Validation Plan for a Full Development ............................................................. 11

5.1 Verification & Validation Plan Overview ............................................................................ 11

5.1.1 Verification & Validation Organisation ..................................................................... 11
5.1.2 Verification & Validation Activity Overview ............................................................... 11
5.1.3 Schedule ......................................................................................................... 12
5.1.4 Verification & Validation Resources........................................................................ 12
5.1.5 Responsibilities ................................................................................................. 12
5.2 Requirements Base ..................................................................................................... 12
5.3 Verification & Validation Methods and Tools ...................................................................... 13
5.3.1 On the Notion of “Formal Methods” ........................................................................ 13
5.3.2 Reviews and Inspections ..................................................................................... 13
5.3.3 Software Architecture Analysis Method (SAAM) ....................................................... 14
5.3.4 Architecture Tradeoff Analysis Method (ATAM) ......................................................... 14
5.3.5 Model Based Testing Method ............................................................................... 15
5.3.6 Characterisation of Formal Methods ...................................................................... 21
5.3.7 Formal Analysis Methods .................................................................................... 21
5.3.8 Correct by Construction Formal Methods ................................................................ 23
5.3.9 Verification with Formal Methods ........................................................................... 23
5.3.10 Verification with Model-Based Simulation ................................................................ 30
5.4 Verification Acitivities for a Full Development ..................................................................... 31
5.4.1 DAS2V Verification............................................................................................. 31
5.4.2 SSRS Verification (1c) ........................................................................................ 31
5.4.3 SFM Verification (2c) .......................................................................................... 32
5.4.4 System Verification ............................................................................................ 32
5.5 Verification Reporting ................................................................................................... 53
5.5.1 Structure of the Verification Report ........................................................................ 53

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 iv

5.6 Administrative Procedures............................................................................................. 56

5.6.1 Problem Report ................................................................................................. 56
5.7 Validation for a Full Development .................................................................................... 56
5.7.1 DAS2V Validation .............................................................................................. 56
5.7.2 SFM Validation (3d)............................................................................................ 57
5.7.3 Final Validation (tbd)........................................................................................... 57
5.8 Implementation of Verification & Validation........................................................................ 58

6 Verification & Validation Plan for openETCS ......................................................................... 59

6.1 Verification Plan for openETCS ...................................................................................... 59

6.1.1 Verification Overview .......................................................................................... 59
6.1.2 Verification Activities—User Sories ........................................................................ 60
6.1.3 Verification Activities—Timeline ............................................................................ 64
6.1.4 Verification Activities—Process View...................................................................... 65
6.1.5 Verification Reporting ......................................................................................... 65
6.1.6 Administrative procedures ................................................................................... 65
6.2 Validation Plan for openETCS ........................................................................................ 66
6.2.1 Validation Overview............................................................................................ 66
6.2.2 Validation Activities—User Sories.......................................................................... 66
6.2.3 Validation Activities—Timeline .............................................................................. 66
6.2.4 Validation Activities—Process View ....................................................................... 67

Appendix A: Requirements on Verification & Validation ............................................................... 68

A.1 Requirements on Verification & Validation from D2.9 ........................................................... 68

A.2 General Requirements on Verification .............................................................................. 69
A.3 Glossary ................................................................................................................... 71

Appendix: References ............................................................................................................. 72

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 v

Figures and Tables

Figures

Figure 1. openETCS Process (rough view)...................................................................................... 5

Figure 2. Compositionnal Testing ................................................................................................. 27
Figure 3. First (faulty) version of a timed automaton for processing emergency messages ........................ 29
Figure 4. Corrected version of the timed automaton for processing emergency messages ........................ 29
Figure 5. Software phase description ............................................................................................ 33
Figure 6. Transformation Verification ............................................................................................. 43

Tables

Table 1. General Verification & Validation Responsibilities ................................................................. 12

Table 2. Requirements Trace Table ............................................................................................... 35
Table 3. SW Requirements Verification Tools, Techniques, Methods and Measures................................. 35
Table 4. SW Architecture Verification preparation table - Example of contents ........................................ 37
Table 5. SW Architecture, Design and Modelling Verification Tools, Techniques, Methods and Measures ..... 39
Table 6. SW Component and Modelling Verification Tools, Techniques, Methods and Measures................. 42
Table 7. SW Code Generation Verification Tools, Techniques, Methods and Measures............................. 45
Table 8. Traceability Matrix Verification Tools, Techniques, Methods and Measures ................................. 47
Table 9. Test Verification Tools, Techniques, Methods and Measures ................................................... 49
Table 10. SW Coverage Verification Tools, Techniques, Methods and Measures ..................................... 52
Table 11. SW Verification Activities Breakdown ............................................................................... 54
Table 12. Tool Chain Verification Summary..................................................................................... 55

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 vi

Document Control
Document information
Work Package WP4
Deliverable ID or doc. ref. D4.1
Document title openETCS Validation & Verification Plan
Document version 00.09
Document authors (org.) Marc Behrens (DLR), Cecile Braunstein (U Bremen),
Cyril Cornu (All4Tec), Christophe Gaston (CEA), Jens
Gerlach (Fraunhofer), Ainhoa Gracia (SQS), Hardi Hun-
gar (DLR), Stephan Jagusch (AEbt), Alexander Nitsch
(U Rostock), Jan Peleska (U Bremen), Virgile Prevosto
(CEA), Stefan Rieger (TWT), Izaskun de la Torre (SQS),
Frederic Badeau (Systerel),
Marielle Petit-Doche (Sys-
terel)

Review information
Last version reviewed –
Main reviewers –

Approbation
Name Role Date
Written by Hardi Hungar WP4-T4.1 Task Leader September
2013
Approved by Marc Behrens WP4 Leader November
2013

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 vii

Document evolution
Version Date Author(s) Comment
00.01 11/06/2013 H. Hungar Document creation based on draft by S.
Jagusch
00.02 14.06.2013 J. Gerlach, H. Hungar Completed one requirement table from
draft, added detail to V&V plan for full
dev.,
00.03 20.06.2013 J. Gerlach, H. Hungar Major revisions and additions
00.04 19.07.2013 C. Braunstein, C. Additions to Sec. 5.3
Cornu, C. Gaston, V.
Prevosto, S. Rieger
00.05 24.07.2013 A. Gracia, H. Hungar, Ch. 6: Additions and restructuring
I. de la Torre
00.06 29.07.2013 H. Hungar Restructuring
00.07 31.07.2013 J. Welte Ch. 5: Description of Safety Interfaces
00.08 01.08.2013 H. Hungar Streamlining, placeholders replaced by
text, strategy detailed, several minor ad-
ditions and changes
00.09 04.09.2013 C. Braunstein, C. Elaboration of Sec. 4.1 (Cornu), BMC
Cornu, M. Petit-Doche, (Braunstein), B-method (Petit-Doche),
H. Hungar Adaptations, References (Hungar)
00.10 04.10.2013 A. Gracia, I. de la Torre Additions to Sec 5.4.4.1
01.00 20.11.2013 M. Behrens Review and approval

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 viii

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 1

1 Introduction
1.1 Purpose

The purpose of this document is to define the verification & validation activities in the project
openETCS.

This document describes strategy and plan of the verification and validation activities in the
project openETCS. As the goals of the project include the selection, adaption and construction of
methods and tools for a FLOSS development in addition to performing actual development steps,
differing from the plan for a full development project, the plan covers also activities evaluating
the suitability of methods and tools, and it makes provisions for incorporation of V&V of partial
developments which are actually done.

This document describes which verification and validation activities are needed for a full FLOSS
development of the EVC software. It describes how the work performed within the project
openETCS is to be organised to contribute to such a task, and how to demonstrate that it can be
realised.

The document is only valid in conjunction with the Quality Assurance plan [1]. It is supplemented
by the safety plan, which focuses on the safety aspect. Verification and validation play an
important role in the safety case. This document identifies the V&V activities which do contribute
and refers to the safety plan for further details on the additional requirements to be met and a
precise statement of what has to be established.

1.2 Document Structure

This document comprises both verification and validation plan, as these activities share some of
their methods and tools, and in some case are applied to the same design artifacts. Nevertheless,
these activities are intended to be and remain independent.

There are three main issues which make this plan different from an ordinary V&V plan for a
software to develop. First, openETCS is not only concerned with the software part of the EVC.
As part of the activities, a semi-formal model for SS 026 is to be developed and to be verified.
As the SS 026 covers parts of the ETCS system beyond the software, also process steps on the
system (not just software) level are to be performed. And in particular the design does not start
with a clearly defined set of requirements on the software.

As a second point, openETCS will not only do development, but shall also be concerned with
processes, methods and tools with the goal of being able to propose a complete SIL 4 compliant
approach. As part of this, it has to be defined how to handle verification and validation. This is
done in the sections addressing a “full development”. Due to the limited resources of the project,
actually performing such a full development is out of the project’s scope. Instead, only some
functions will be implemented, and only partial lines of development will be realised. V&V
related to these activities is to be planned in the specific sections dedicated to “openETCS”.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 2

1.3 Background Information

1.3.1 Definitions

Verification

Verification is an activity which has to be performed at each step of the design. It has to be
verified that the design step achieved its goals. This consists at least of two parts:

• that the artifacts produced in the step are of the right type and contain allthe information they
should. E.g., that the SSRS identifies all components addressed in SS 026, specifies their
interfaces in sufficient detail and has allocated the functions to the components (this should
just serve an example and is based on a guess what the SSRS should do)

• that the artifact correctly implements the input requirements of the design step. These
typically include the main output artifacts of the previous step. “Correctly implements”
includes requirement coverage (tracing). This can and should be supported by some tools.
Adequacy of such tools depends on things like format compatibility, degree of automation,
functionality (e.g., ability to handle m-to-n relations). Depending on the design step (and the
nature of the artifacts) different forms of verification will complement requirement coverage,
with different levels of support. The step from SS 026 to the SSRS will mainly consist of
manual activities besides things like coverage checks. Verifying a formal (executable) model
against the SSRS can be supported by animation or simulation to e.g. execute test cases which
have been designed to check compliance with the SSRS. Even formal proof tools may be
employed to check or establish properties. Model-to-code steps offer far more options (and
needs) for tool support. And tools or tool sets for unit test will support dynamic testing for
requirement or code coverage. This may include test generation, test execution with report
generation, test result evaluation and so on. Also, code generator verification (or qualification)
may play a role, here. Integration steps mandate still other testing (or verification) techniques.

Summarizing, one may say that verification subsumes highly diverse activities, and may be
realized in very many different forms.

Validation

Validation is name for the activity by which the compliance of the end result with the initial
requirements is shown. In the case of openETCS, this means that the demonstrator (or parts of it)
are checked against the SS 026 or one of its close descendants (i.e., SSRS), taking also further
sources of requirements from operational scenarios and TSIs into accoutn. This will consist of
testing the equipment according to a test plan derived from the requirements and detailed into
concrete test cases at some later stage. Tool support for validation will thus mainly concern
test execution and evaluation, perhaps supplemented by test derivation or test management.
Ambitious techniques like formal proof are most likely not applicable here.

Thus, the tool support for validation will not differ substantially from that for similar verification
activities.

One might also consider “early” validation activities, e.g. “validating” an executable model
against requirements from the SS 026. These are not mandated by the standards and can per se
not replace verification of design steps. They may nevertheless be worthwhile as means for early
defect detection.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 3

Further (mostly complementary) information on V&V can be found in the report on the CEN-
ELEC standards (D2.2).

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 4

2 Document Evolution
The verification and validation plan shall be revised in the course of the project as the design
progresses and gets detailed and experiences with verification and validation are made. This is in
accordance with the EN 50128, where it is required that the plan shall be maintained throughout
the development cycle.

V01, T0+13: First version of the plan.

V02, T0+17: First revision, based on the 1st V&V interim reports on applicability of the V&V
approach to model and implementation/code (D4.2.1, D4.2.2)

V03, T0+25: Second revision, based on the internal reports on the applicability of the V&V
approach to prototypes of design models and code

V04, T0+36: Final version as part of the final V&V report (D4.4)

The first version of the plan is based on the available information of the design process. This is
not yet very detailed as also the description in Chapter 3 of this report shows. In particular, the
nature of the SSRS is yet to be defined precisely, and the architecture description including the
HW/SW partitioning needs to be revised.

Concrete plans of activities are thus still to be made, and methods and tools to be applied will
have to be selected. Only the first phase of V&V activities is described in Sec. 6.1.3.1.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 5

3 Verification & Validation in the Design Process

D2.3 defines the openETCS process on an abstract level. It already defines the main steps. A
slightly more detailed picture than the one given in D2.3 is given in Fig. 1.

Figure 1. openETCS Process (rough view)

As a basis for planning the V&V activities, the process sketch permits to name the main phases.
Planning will need a better definition of the stages (scope of the design artifacts, level of detail,
respective system boundaries), detailed planning a specification of the artifacts to be produced.
Version V01 of this document will in these respects be of an accordingly preliminary nature.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 6

4 Verification & Validation Strategy

The overall strategy is to support the design process as specified in D2.3 and its partial instan-
tiations within openETCS. In accordance with the project approach, V&V shall be done in a
FLOSS style, and it has to suit a model-based development. A further main consideration shall
be to strive for conformance with the requirements of the standards (EN 50128 and further). Of
particular importance in that respect is of course the interface to the safety considerations and the
contribution of V&V to the safety case.

W.r.t. verification & validation, openETCS shall achieve two goals:

1. Design a tool-supported method with which the EVC software can be developed and main-
tained so that it is suitable for integration in (SIL-4) certified products. This will be described
in Sec. 4.1, “Verification & Validation Strategy for a Full Development”.
2. Perform part of the development (including tool and method evaluation and the study of
tool qualification questions) on representative parts of the design. The plans for that will be
detailed in Sec. 4.2, “Verification & Validation Strategy for openETCS”.

In more detail, the verification and validation have to consider the following aspects throughout
the development process.

• functionalities of the system and the sub-system,

• system and sub-system architecture,
• external and internal interfaces of the sub-system,
• software components,
• performance and Safety objectives and constraints,
• functional properties,
• safety properties.

4.1 Verification & Validation Strategy for a Full Development

Here, the ideal shall be described: How will the V&V part of a FLOSS development of the
open-source EVC software look like, what are its constituents, and how do they act together in
developing and maintaining this software within the openETCS ecosystem.

4.1.1 Verification Strategy for a Full Development

This section defines the strategy for verifying a full development of the EVC software from the
requirements source (ss 026+TSIs+. . . ). This ends with the verification of the software/hardware
integration. In current view, the API defines the interface and relevant properties of the hardware.
Thus, SW/HW integration for openETCS will most probably done virtually with an instantiation
of the API playing the role of the hardware.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 7

4.1.1.1 System Test Campaigns

Testing is one of the main means for verification. The three main features of test campaigns
which need to be specified are:

• Definition of the different campaigns for each design phase and their characteristics.

• Definition of the campaign objectives, and of the kind of test to be performed in order to
achieve these objectives.

• Traceability matrix between characteristics to check/ validate and test cases.

4.1.1.2 V&V Actions Criteria

These criteria shall define the acceptance of system or component modification, and the inter-
ruption criteria shall define the condition under which a test campaign should be stopped for
instance. These criteria have to bet set during the V&V plan redaction. During the verification
phase, these items are verified thanks to the actions criteria list described in the subpart related to
the development phases and items verification.

4.1.1.3 Organization Charts and Responsibilities

The plan shall define the roles and responsibilities of people involved in the V&V activities
(according to standards and project needs). Must also be defined the teams compositions, and the
deliverables product owners, as well as the group and committee for modification. These items
are verified during the verification phase.

4.1.1.4 Internal Means and Tools

The plan shall list and define the means and tools needed for V&V actions (also when they are
needed). These items are verified during the verification phase.

4.1.1.5 External Means and Tools

The plan shall list and define the means and the tools used by the interfaces with the project, and
which company or structure is using them. These items are verified during the verification phase.

4.1.1.6 Development Phases and Item Verification

The plan for the development phases verification is based on requiement coverage at system and
sub-system level. The requirements are usually decomposed and gathered in a table, and this
table constitutes a road-map for verification activities, and is closely followed in order to figure
out if the required items are covered, and if so, in which document. The following items give
an overview of items checked during verification phases, regarding the kind of prescriptions
checked, and the appropriate related development phase.

Introduction: This part gives an overview of Verification general purposes, of main verification
phases (Safety Specification of Software, architecture, and overall test campaign), and
verification support formalism (harmonized against different Verification activities, can be a
simple table).

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 8

Software Security/Safety Prescriptions Specification Phase: Gives a list of prescriptions (spec-

ifications top level from a top level specification providing the customer needs), and trace
their coverage (justification and in which document this justification is provided). Here is
an example: “Prescription specification related to Software safety should come out from
upper level system safety prescriptions and from prescriptions from Security schedule. This
information should be communicated to the Software developer.”

Design Phases and Software Development: This phase is based on the same approach of the
previous part, and prescriptions are gathered in a table. The different categories considered
are defined in the following parts: (), , detailed design and development prescriptions, coding
prescriptions, Software modular tests and Software integration tests prescriptions.

General Requiements: For instance: design representations shall be based on a clearly defined
notations, or limited to clearly stated characteristics

Software Architecture Related Requirements: . For instance: Any System Safety prescrip-
tion modification related to security should be documented and acknowledged by the devel-
oper.

Support Tools and Coding Languages Requirements: . For instance: According to the type
of development, the compliance with the given prescriptions is under the software supplier
responsibility.

Detailed Design and Development Prescriptions: For instance: “The Software shall be pro-
duced in order to insure the required modularity, testability and modification ease.”

Coding Prescriptions: For instance: “Each Software code module shall be reviewed”

Software Modular Tests Prescriptions: For instance: “Modular tests shall be documented”

Software Integration Tests Requirements: For instance: “Integration test specification shall
encompass: integration groups easily manageable, test cases and test data, test kinds, en-
vironment, tools configuration on test program, test acceptance criteria, corrective actions
procedures”

Software Security Validation Planning: The same formalism as previous parts can be used,
and this part sums up the different requirements/specification for the planning regarding the
safety activity. For instance: “Planning shall be managed in order to specify technical and
process steps necessary to prove that the software is compliant with safety prescriptions”.

Programmed Electronic Components (Hardware and Software): This part is specifically re-
lated to the integration conditions. This part is not supposed to be considered in openETCS,
as no integration is planned in the project so far.

Software Safety Validation: For instance: “Validation activities shall be performed as specified
in the validation plan”.

Conclusion: (related to the requirements coverage). In our case, the prescriptions are the
requirements explained in the WP4 (these prescriptions have to be refined from the different
project inputs, such as CENELEC, SRS, FPP. . . ).

4.1.2 Validation Strategy for a Full Development

Validation, according to the standard, starts after SW/HW integration. In this section, it shall be
detailed how this should look like for the openETCS architecture approach (with SSRS and API).

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 9

Ideal would be a description of how a full openETCS EVC software could be taken up by some
manufacturer and brought to life in a product (validation aspect only, of course). Validation will
use tests covering operational scenarios.

Not-so-classical validation can start earlier when executable models become available. If a model
can be animated to run an operational scenario (perhaps with some additional environment/rest-
of-system modeling), design defects may get unveiled before the real validation. This is, however,
not an activity which is mentioned as a development activity in the standard EN 50128. Thus,
to use results of “early validation” in a validation report requires a definition of its role and an
argument for its usefulness.

4.1.2.1 Validation Case

The aim of the Validation Report is to demonstrate that the objectives of validation which have
been set in the validation plan have been achieved. This concerns the adequacy of the software
design documentation and that components and system behavior is compliant with the software
requirements.

The inputs for validation at system level are:

• Software Requirement Specification (design, architecture),

• Sub-System Requirement Specification (encompassing the architecture, interface description

and requirement allocation),

• Application Programming Interface,

• Internal and External constraints,

• Validation constraints list,

• Validation Plan,

The inputs for validation at software level are:

• Components Requirements Specification,

• Integration Specification,

• Integration Report,

• Test Specifications,

• Test reports.

The outputs are:

• Software Validation Report (Verdict on software ability to fulfill the objectives and function-
alities defined in the requirement specification.),

• Software Validation Verification Report,

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 10

4.1.3 Safety Interface

The safety activities in the software development process are closely connected to the verification
& validation activities as these provide the overall system safety requirements, derive correspond-
ing safety related design specifications and collect the verification & validation documentation to
build the safety case. The safety activities are identifying unwanted accidents resulting in harm
and analyses the potential hazards, which could lead to the harm. Resulting from this in depth
analysis certain requirements are derived, which provide the overall safety goals for the system.
If the initial risk of a hazard resulting in harm has to be reduced or the possibility of harm shall be
eliminated at all specific safety designs are developed to reach the required risk. The verification
& validation process has to verify these specifications and validated that the requirements hold
for the developed software. Respectively, a complete documentation is needed to show in the
safety case that this steps have been done according to the overall verification & validation plan.

Therefore three main interfaces to the safety activities have to be maintained in the verification &
validation process:

• Verification of safety design specification: Based on risk control measures documented in

the Hazard Log specific safety design specifications are derived and written in the backlogs
for model and code development. As it is done for the overall model and code specification
the verification activities have to demonstrate that all of the safety specifications have been
implemented properly.

• Validation of safety requirements: The overall software validation has to demonstrated that
the safety goals are met by the developed product. Therefore the safety requirements which
have been stated based on potential accidents and accepted risk levels have to be validated.
As these in many cases requires proof for the absence of certain conditions, it is important
that verification & validation activities work in a close iterations with the safety activities to
ensure that the safety requirements are stated in a way that can be validated.

• Creation of verification & validation documentation: As the safe case has to provide the
complete argumentation that all needed steps to ensure a qualified and safe development
process have been performed properly, it is important that the verification & validation plan
and all resulting verification & validation reports are coherent and allow to show a close
chain of arguments.

4.2 Verification & Validation Strategy for openETCS

The project will only perform part of the development, and thus also only a part of the V&V
activities. These need to be defined and planned. The overall approach shall be to try out the
different constituents on and in representative samples, so that the realisability of the verification
& validation strategy for a full development can covincingly be demonstrated.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 11

5 Verification & Validation Plan for a Full

Development
The section is going to instantiate the generic Verification & Validation plan from the standard to
the EVC development in openETCS-style (FLOSS). This entails the organisation, a definition of
the requirements, generic schedule covering all design steps, resources, responsibilities, tools,
techniques, and methodologies to be deployed in order to perform the verification activities, and
all the documents which are to be produced.

The result must conform to the requirements of the standards for a SIL 4 development.

As D2.3 gives only a rough description of the development steps and not yet a complete list of
design artifacts, nor one of methods applied and formats to be used, this first version (V01) of
the V&V plan will also lack detail which will to be added in later revisions as these informations
become more concrete.

5.1 Verification & Validation Plan Overview

This section gives an overview of the verification & validation plan for a full development.

5.1.1 Verification & Validation Organisation

This section defines the relationship of verification and validation to other efforts such as devel-
opment, project management, quality assurance, and configuration management. It defines the
lines of communication within the verification & validation, the authority for resolving issues,
and the authority for approving verification & validation deliverables. Here, the verification &
validation aspect of the “openETCS-ecosystem” should be outlined.

5.1.2 Verification & Validation Activity Overview

This section gives a short overview of the activities (Verification or Validation) which happen at
the respective development steps, to be detailed in the subsequent sections. The numbering (e.g.
2e) refers to Fig. 1. Abbreviations used are defined in the glossary, Sec. A.3.

SSRS—Verification (1c): verification that the SSRS the requirements consistently extends the
requirements base.

SSRS—Validation (1c): Deriving a sub-system test specification

SFM—Verification (2c): Verification that the model formalises the requirements

SFM—Validation (2c): Detailing the test specification, perhaps validating the model (e.g. via
animation)

SW-SFM—Verification (3d): Verifying the SW-HW architecture definition (should be some-

where) and the software model

SW-SFM—Validation (3d): Perhaps validation of the software model

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 12

SW-FFM—Verification (3d): verification, employing also formal methods/tools

SW-FFM—Validation (3d): validation, may e.g. employ model checkers

Code—Verification (3e): verification depends on the code generation method (manual, gener-
ated, generated with validated tool), unit test requirements have to be met, afterwards code
integration tests

Code—Validation (3e): no specific activities foreseen

The following step descriptions are preliminary and shall serve as a concept which is to be
detailed in later revisions (V02 and up).

EVC Software—Verification (tbd): Perform software system verification

EVC Software—Validation (tbd): Validation against user requirements/scenarios, broken down

to software functionality

SW/HW integration (tbd): Use the API in a simulation environment as a replacement of actual
SW/HW integration

Final Validation (tbd): Apply user (railway operator) requirements and scenarios (based on the
sub-system test specification)

5.1.3 Schedule

The overview of the activities given above shalle be detailed in this section. The objective here is
to define an orderly flow of material between project activities and verification tasks.

5.1.4 Verification & Validation Resources

In a regular development project, resources needed to perform verification tasks, including

staffing, facilities, tools, finances, and special procedural requirements such as security, access
rights, and documentation control, have to be defined. Here, where we define merely a pattern
of a V&V plan, the emphasis will lie on spelling out requirements and proposing principal
solutions.

5.1.5 Responsibilities

Table 1. General Verification & Validation Responsibilities

Role Name of the person Affiliation Activity Code

Verification
Team Manager
Verifier

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 13

5.2 Requirements Base

This section shall provide references to all requirements against which the design is to be verified
and validated. It does not include process requirements. For the latter, see Sec. A.

The requirements on the EVC software origin in the SS-026 and TSI specifications.

5.3 Verification & Validation Methods and Tools

The project shall select / develop / describe a chain of methods and tools for doing verification
& validation in a full development. In Version V01, this section collects proposals, which will
be evaluated and from which those suitable for real-life development and maintenance of the
open-source EVC software will be selected.

Each proposal described here shall be classified for which tasks and activities in teh development
it is intended, i.e., which role it should take.

5.3.1 On the Notion of “Formal Methods”

As FLOSS relies to a large extent on the use of tools for generating, verifying and validating a
design, “formal methods” necessarily will play an important role in the openETCS activities.

In common language, the notion “formal” is often used in a broad sense, meaning everything
that can be described by rules, even if they are rather vague. Contrary to that, we use “formal” in
the narrow sense of EN-50128 [2, Section D.28], meaning strictly mathematical techniques and
methods. Since the Aerospace Standard DO-178C [3] follows a similar understanding, but gives
more elaborate explanation in its supplementary document devoted to formal methods [4], our
presentation closely follows the terminology of the latter.

Formal methods are mathematically based techniques for the specification, development,
and verification of software aspects of digital systems. The mathematical basis of formal
methods consists of formal logic, discrete mathematics, and computer-readable languages.
The use of formal methods is motivated by the expectation that, as in other engineering
disciplines, performing appropriate mathematical analyses can contribute to establishing
the correctness and robustness of a design.
[4, Section 1.0, p.1]

5.3.2 Reviews and Inspections

Not everything can be done by formal methods. The introduction of reviews in the openETCS
project will enable the adherence to design specifications as well as ensuring that appropriate
coding techniques—for automatic generation as well as observation of relevant coding standards—
have been used. The use of an appropriate review technique will also help to ensure the
consistency of approaches across development teams and may result in improved standards
through the identification of best practice solutions. Software reliability will be increased due
to the removal of a larger percentage of the errors that would otherwise remain in the software.
Specific analysis techniques exist for predicting the reliability of software.

For verifying any product, whether it is a piece of software, a design specification, test script
or anything else, it is essential to carry out some sort of review or inspection process. The

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 14

introduction of formal review processes provides us with definite points in time when we can
carry out these essential assessments.

It is possible to review just about anything. In the openETCS project all written documents,
specifications, models and code can be reviewed. It is also important to include all documents
concerned with the creation and delivery of the openETCS product. This means that strategies,
plans, approaches, operation and maintenance manual, user guides, the contract that will initiate
the work should all be reviewed in a structured way.

Inspection

On the other hand, an inspection is a visual examination of a software product to detect and
identify software anomalies, including errors and deviations from standards and specifications
[5]. Inspections are peer examinations led by impartial facilitators who are trained in inspection
techniques. Determination of remedial or investigative action for an anomaly is a mandatory
element of a software inspection, although the solution should not be determined in the inspection
meeting.

Both review and inspection methods shall be used in different contexts during the execution of
the Verification activities.

More information about the reviews can be found in Quality Assurance Plan [1].

5.3.3 Software Architecture Analysis Method (SAAM)

SAAM [6] is one of the simpler methods for a scenario-based architecture evaluation, and it was
the first to be published. SAAM is suitable for the testing of software architectures with regard to
quality attributes (qualitative requirements), such as

• Modifiability,

• Portability,

• Growth Potential,

• Performance,

• Reliability,

but also for the evaluation of the functionality (functional requirements) of a software architecture.

In a SAAM evaluation basically scenarios are developed, prioritized and assigned to those parts
of the software architecture to be tested that are affected by them. This may be sufficient to
indicate problems in the architecture.

5.3.4 Architecture Tradeoff Analysis Method (ATAM)

ATAM [7] is used to review the design decisions of the architecture. It is checked whether
the design decisions satisfactorily support the requirements concerning quality. Risks and
compromises included in the architecture are identified and documented.

The process includes two phases.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 15

• In the first phase the necessary components are presented. Then the architecture is checked
and analyzed.
• In the second phase it is tested whether the analysis and the test were correct and complete.
Then the results are summed up.

5.3.5 Model Based Testing Method

5.3.5.1 Model Based Testing Strategy - generalities

Testing consists in executing the System Under Test (SUT) for some particular inputs and in
assessing whether or not the corresponding SUT executions conform to some requirements.
Whatever the testing technique used is, one has to define test cases to be submitted to the SUT
and associate to them a decision procedure called oracle. The oracle allows the tester to compute
verdicts according to what the executions of SUT (resulting from the test case submission) reveal
about its correctness. This correctness is measured with respect to requirements. Model based
testing is a particular kind of testing technique in which requirements are described by models
which are executable specifications. Their execution traces (or “traces” for short) are sequences of
stimulations of the SUT and resulting observations of the SUT reactions. Test cases are sequences
of stimulations that are selected from the test model. A sequence corresponding to an input
test data can be obtained by considering a trace of the model and ”forgetting” all observations
occurring in it. For functional testing, SUT is considered as a black bbox: the tester (a human or
a test bench) can only stimulate the SUT and observe its reactions. Interactions between the tester
and the SUT result on the definition of traces. Therefore, a SUT can be seen as a set of traces
that is not known (since SUT is a black box) but the tester may discover some of those traces by
interacting with SUT. The oracle is based on a so called conformance relation. A conformance
relation is a mathematical relation between the set of traces of the SUT and the set of traces of the
model. When these sets of traces fulfill the relation we say that the SUT conforms to the model.
The oracle takes as inputs traces representing an interaction between the tester and the SUT and
compute verdicts. Whatever the testing techniqu e is, the set of possible verdicts always contain
the verdict Fail which is emitted whenever the trace taken as input demonstrates that the SUT
does not conform to the model. Depending on the testing technique used there may be different
verdicts emitted when Fail is not emitted. These different verdicts reflect different traceability
information related to interaction trace taken as input. In this section we briefly discuss two
model based testing tools that we will use conjointly in the OpenETCS project.

Model based testing (MBT) may apply at different level during the lifecycle:

• System integration testing

• Software integration testing
• High level code verification
• Object code verification

High-level code verification may be performed on any host perform whereas object code verifica-
tion intends to test the running code on the target hardware.

Model-Based System Integration Testing

The objectives of MBT on system integration level are to

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 16

• validate the correctness and completeness of the development model,

• verify that the generated code components cooperate correctly on the target HW, in order to
achieve the system-level capabilities.

The first objective implies that the test model and the original development model are separate
entities; otherwise the system integration test would just validate that all logical errors still
residing in the openETCS development model are really implemented in the code. Even in
presence of a formally validated development model, in which high confidence can be placed,
we prefer to create a separate test model, because

• the test model may use a higher level of abstraction since only the SUT behaviour visible at
the system interfaces is relevant,

• the test model may specify different interfaces to the SUT, depending on the observable
interfaces in a test suite; the observation level ranges from black-box (only the “real” SUT
system interfaces are visible) to grey-box level (some global variables may be monitored or
even manipulated by the testing environment, some task or object communications may be
observed etc.),

• the development model may contain errors that are only revealed during HW/SW integration
(for example, calculations failing due to inadequate register word size, or deadlines missed
due to insufficient CPU resources).

Software integration is performed by software-in-the-loop technology on host computers. The

software components as well as the complete software are tested on host computer with a
testing environment that simulates the hardware behavior and the operational environment. The
main advantage is that all software properties can be easily simulated and tested. The tests for
software-in-the-loop may be generated from a model but at this level unite test for each software
functionalities may also be performed.

Model-Based Testing of Generated High-Level Code

Another application of MBT aims at the verification of generated high-level code (for openETCS,
the target language will be C). If model-to-text transformations are not formally verified, it is
necessary to verify the outcome of each transformation. Since the transformation source is a
model M, MBT suites can be derived automatically from this model to show that the generated
code conforms to M.

Observe that in contrast to system-level MBT no redundant model is used for this objective, but
the same model M used for code generation can be used: we just have to verify the consistency
between code and M, without validating M’s correctness and completeness. The latter task is
separately performed by means of

• property checking or

• simulation.

The model-based testing (MBT) approach can be used to create test suites conforming to the
highest criticality level of the applicable CENELEC standards, in order to justify that the

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 17

generated code is consistent to its model [8, 9, 10]. Furthermore, the generated result may be
formally verified against the model. This formal verification task is easier than proving the
correctness of a generator or compiler as a whole, because now just one concrete artefact (the
generated code) has to be checked against the transformation source. The theoretical foundations
of object code verification, as well as its proof of concept have been established in [11]. In
[12, 13] these concepts have been refined and applied to the railway domain.

The main advantage of this approach in comparison to performing V&V for generators and
compilers is that the latter do not have to be re-verified after improvements and extensions.
Therefore we advocate the test-based code verification approach to be applied in openETCS for
verifying generated high-level source code or object code of SIL-4 applications.

The HW/SW integration testing is out of the scope of this project. Nevertheless, Model-Based
Testing of Compiled Object Code or/and Alternative Unit test on target HW may also be
performed.

5.3.5.2 Model Based Testing applied to Open ETCS V&V

According to the previous activity on defining the project process, the Open ETCS process is
based on 3 main inputs for methodology and product lifecycle: the SCRUM methodology, the
Model Driven Design and the Cenelec software development V cycle. Traditionally, system
requirements are directly translated into formal specifications on which verification and proof
techniques are applied. The use of formal specifications and formal language allows then to
derive the models using dedicated languages (B for instance) in order to guaranty conservation
of properties along the design process. The main difficulty in this context is to be sure that the
interpretation of rules has correctly been captured in the formalized specification which is not
easy to check by the regulators. For this reason, the model based testing has been chosen as
testing and verification technique within the V&V activities of the OpenETCS project.

Moreover, we suggest to create test models on the basis of the ETCS standard (subset 026) and
the existing high-level test suites made available in subset 076. The latter test cases should be
feasible computations of the test model, so that the test model really creates a superset of the
existing test suite from subset 076.

This technique application will be explained in the following paragraphs through the description
of different model-based testing tools: MaTeLo, Diversity and RT-tester.

5.3.5.3 Matelo Model Based Testing solution

MaTeLo purpose is to generate test cases for systems whose expected usage and behavior are
described by a probabilistic model. MaTeLo tool is based on its own test model called "usage
model" and uses, among other characteristics, usage profiles for test case generation. This
usage model describes the possibilities regarding the use of the soft (in our case; operating
scenario) during its whole lifecycle. This usage model is performed thanks to a Matelo specific
modeler, it allows to generate test cases that will then be plugged to the SUT which will be the
software semi-formal model realized in the frame of WP3 activities. MaTeLo has three main
functionalities: test modeler, test cases generator and test campaign analyser. Even if MaTeLo
is mainly a test case generation tool, we can consider that this tool performs also analysis for
different reasons:

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 18

• A test model can be considered as a development artifact the same way as a system model for
example. So analysis on it could identify some ambiguous or erroneous points in test model
(i.e. in the future test campaign) or in the specifications (because MaTeLo mode l is built
from system specifications).

• Even whether test campaign analysis is mainly based on testing activities, analysis techniques
have to be used as wellThe limit between a model to perform test and a model to perform
analysis is not so obvious.

Because its test case generation is based on a model, MaTeLo belongs to the family of Model-
Based Testing solutions. MaTeLo model basically uses Markov Chains to describe the test model
of the SUT implemented for "Black Box Testing" in all xIL steps (MIL, SIL, PIL, HIL). MaTeLo
Usage Model edition facility allows for implementing test models that describe the use cases of
the SUT completed with the tester point of view, and then, Matelo testing facility can generate
automatically the test cases generated by the tool. Thanks to the numerous validation steps,
MaTeLo Test Campaign Analysis provides information such as test coverage (requirements,
model) or reliability of the SUT. Once the MaTeLo test model is performed and the testing
strategy is defined with MaTeLo profiles faciulities, MaTeLo generates test cases. For that,
MaTeLo Testor contains several test generation algorithms that can be used for different purposes.
Different test case generators are based on a Usage profile approach, considering the occurrence
probability of each model transition. Other are deterministic (most probable execution path, or
all the transitions are covered). In the case of Open ETCS project, the SUT model is an on-board
EVC, designed according to the SRS Subset 026. This specification itself is not sufficient to
cover all functional aspects, and tests depend strongly on the operating rules to be considered on
the observed track. The principle for the MaTeLo model would be to encompass all the possible
states and transitions that can be considered in a well-defined perimeter (based on Subset026,
signaling and exploitation rules to consider). Then, the test could be precisely defined by the
usage profile to adapt it to a track oriented testing campaign.

5.3.5.4 Diversity Model Based Testing solution

DIVERSITY is a model based testing tool developed at CEA LIST. Its underlying technology is
symbolic execution. Symbolic execution has been first defined for programs. The goal of this
technique is to identify, for each possible execution of the program, the constraints to be satisfied
in order to follow it. The main idea consists in executing the program, not for concrete numerical
values but for symbolic parameters, and to characterize constraints on these parameters at each
step of the execution. In that sens, DIVERSITY is a white box testing tool. In the frame of the
openETCS project we plan to use DIVERSITY to extract test cases from models defined in the
first phases of the system design. Our goal is to extract test cases dedicated to abstract safety
requirements. More precisely we focus on safety requirements dealing with communication
between sub systems. For that purpose we will use the language of sequence diagrams extended
with timing constraints to specify such requirements. With sequence diagrams, one may describe
execution scenarios in terms of partially ordered message passing between subsystems. Message
passing can be structured thanks to operators expressing sequencing, parallelism, choice, loop...
It is possible to automatically analyze sequence diagrams with DIVERSITY in order to extract
test cases. The originality is that, thanks to projection mechanisms, it is possible to extract test
cases, not only for the entire system, but also for any sub systems composing it. Because of
this mechanism, sub systems can be tested as soon as they are implemented, even though the
entire system is not yet implemented. In such a process we perform a particular kind of unitary
testing in which unit test cases are built according to the usage that will be made of the sub
system in the entire system. In the frame of OpenETCS, this functionality could be useful in

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 19

order to realize the unitary and modular tests. The first step consists in defining a requirement
model in the form of a sequence diagram or a Matelo Test scenario. The requirement model is
analyzed with DIVERSITY in step 2. This analysis results on a so-called symbolic tree, who’s
each path denotes a possible (symbolic) execution of the sequence diagram. Such trees may be
theoretically infinite due to the possible occurrences of the "loop" operator of sequence diagrams.
Therefore, DIVERSITY uses various stopping criteria to stop the computation (typically based on
message coverage notions). The symbolic tree computed in step 2 characterizes executions of the
whole system model. However, because testing the whole system may be complicated in terms
of testing architecture, or simply because one wants to test some sub systems before the whole
system is implemented, we offer a mechanism to extract symbolic trees for each distinguished
sub system. This is based on so-called projection techniques. This operation is realized in step
3. In step 4, each identified sub system is tested thanks to a real time off-line testing algorithm.
Then, we can relate correctness of sub systems and correctness of the whole system by using
a compositionality theorem. The compositionality theorem expresses that, the conformance of
each subsystems to all their projections guarantees the conformance of the whole system to the
sequence diagram. A direct consequence is that any faults of the whole system can be discovered
as a fault of at least one of its sub systems. This implies that testing the whole system mainly
comes to test each of its sub systems after a short test integration phase testing that each sub
system is correctly connected. We believe that such an approach will be very useful for ETCS
systems which are by nature very distributed and thus hardly observable and controllable as a
whole. The share of OBU EVC kernel in sub-system is the role of the SSRS model, and this
refinement to diversity will be possible once this functional decomposition of the EVC will be
released.

5.3.5.5 Complementary use of the DIVERSITY and MaTeLo

The use of the two tools can be done in a complementary way that would allow a more efficient
test case set generation. MaTeLo would start from the test model, and generate automatically all
the use cases that can be encountered in CBTC use. MaTeLo tool analyses the models as black
box, and generates tests according to a stochastic approach. DIVERSITY will analyze these
scenarios, based on a symbolic execution of the semi-formal SysML model (white box testing),
in order to filter the tests generated by MaTeLo and to reduce the test case set. As discussed in
previous Sections, the two tools DIVERSITY and MaTeLo handle different kinds of models. The
version of DIVERSITY that we will use in the project handles high level models in the form
of sequence diagrams. Such models can be used to specify requirements on communication
scenario between subsystems of a reference system under test. Models handled in MaTeLo are
automata labeled by transfer functions and probabilities. Such models are useful to describe
executable behaviors very close to the actual implementation, and based on operating scenarii.
Clearly these two levels of modeling are useful in design processes of safety critical applications
such as ETCS implementations, and can be combined in different ways for improving the test
coverage of our EVC Software kernel. Indeed, ETCS systems have such a level of complexity,
that it is difficult to describe them in a model straight from the requirements. Therefore, the
refinements provided by two modeling levels are very helpful. Moreover, it is mandatory to
maintain a good traceability between these two levels of modeling, in order to fulfill the safety
requirements. The complementarity of these tools takes place in some refinement processes in
which high level requirements can be implemented into executable models. However, it is crucial
to assess whether executable models correctly implement requirements. In practice this may be a
difficult question because it requires to efficiently explore the executable model, which by nature
is generally huge because it represents in a precise manner the functional behaviors of the actual
implementation. In order to overcome this problem we plan to take benefits from the fact that
executable models of ETCS will be described in the form of communicating executable models.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 20

This fact permits to see the model as a collection of communicating subsystems. This permits
to take benefits of the compositional result described in the Diversity, and use it for white box
testing (the internal behavior of functional modules and blocks defined in the kernel can then be
precisely tested)..

5.3.5.6 The RT-tester

The RT-Tester test automation tool, made by Verified [14], performs automatic test generation,
test execution and real-time test evaluation. It supports different testing approach such as unit
testing, software integration testing for component, hardware/software integration testing and
system integration testing. The RT-Tester version follows the model-based testing approach [15]
and it provides the following features :

• Automated Test Case Generation

• Automated Test Data Generation

• Automated Test Procedure Generation

• Automated Requirement Tracing

• Test Management system

Starting from a test model design with UML/SYML, the RT-tester fully automatically generates
test cases. They are then specified as test data (sequences of stimuli with timing constraints) and
used to stimulate the SUT and run concurently with the generated test oracles. The test procedure
is the combination of the test oracles and the SUT that can be compiled and executed.

The tool supports test cases/data generation for structural testing. It automatically generates
reach statement coverage, branch coverage and modified condition/decision coverage (MC/DC)
as far as this is possible. The test cases may all be linked to requirements ensuring a complete
requirement traceability. Additionally RT-tester may produce test cases/data from a LTL formula,
since a LTL formula describes a possible run of the model.

Taking advantage of SysML requirements diagram, the test cases and test procedures are directly
linked to the requirements. It is then possible to perform test campaign guided by requirements.

Finally the tool may produce the documentation of tests for certification purposes. For each test
cases the following document are produced :

• Test procedure: that specifies how one test case can be executed, its associated test data
produced and how the SUT reactions are evaluated against the expected results.

• Test report: that summarizes all relevant information about the test execution.

In [16], a general approach on how to qualify model-based testing tool according to the standard
ISO 26262 ad RTCA DO178C has been proposed and applied with success to the RT-tester tool.
Following the same approach compatibility with the CENELEC EN50128 may be easily done.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 21

5.3.6 Characterisation of Formal Methods

Based on rigorous mathematical notions, formal methods may be used to describe software
systems’ requirements in an unambiguous way, thus supporting precise communication between
engineers. Formally specified requirements can be checked for consistency and completeness by
appropriate tools; also, compliance between different representation levels of specification can be
verified. Formal methods allow one to check software properties like:

• Freedom from exceptions

• Freedom from deadlock

• Non-interference between different levels of criticality

• Worst case resource usage (execution time, stack, . . . )

• Correct synchronous or asynchronous behaviour, including absence of unintended behaviour

• Absence of run-time error

• Consistency of a formal model

• Correctness of a formal model

In order to subsume this variety of applications under a single paradigm, the DO-178C considers
a formal method to consist in applying a formal analysis to a formal model. Both analysis
and model differs depending on the particular method. For most methods, the model is just
identical to the source code; however, it may also be e.g. a tool-internally generated abstract state
space (used in the Abstract Interpretation method, cf. Section 5.3.7.1 below). For most methods,
analysis tools need human advice; however, they may also be fully automatic (e.g. for Abstract
Interpretation or Model Checking, cf. 5.3.7.3).

5.3.7 Formal Analysis Methods

In this section we present the three most common methods for formal analysis. The foundation
of these analysis methods are well understood and they have been applied to many practical
problems.

5.3.7.1 Abstract Interpretation

The abstract interpretation method [17] builds at every point of a given program a conservative1
abstraction of the set of all possible states that may occur there during any execution run. Such
a representation is also called an over-approximation, in the sense that it captures all possible
concrete behaviours of the program, while the abstraction might lead to consider states that
cannot occur in a concrete execution. Abstract interpretation determines particular effects of the
program relevant for the properties to be analysed, but does not actually execute it. This allows
one to statically determine dynamic properties of infinite-state programs. The main application is
to check the absence of runtime errors, like e.g. dereferencing of null-pointers, zero-divides, and
out-of-bound array accesses. While conventional ad-hoc static analysis tools such as PCLint or
QAC++ are well-tailored for quick, but incomplete analyses, abstract-interpretation based tools
while requiring more computation time, are safe in the sense that they guarantee that all potential
1
i.e. guaranteeing soundness

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 22

runtime errors are detected. On the other hand, such a tool might report spurious warnings, related
to states that are included in the abstraction but do not correspond to concrete executions. Such
false alarms can be avoided to some extent by increasing the precision of the abstraction [18],
at the expense of the computation time of the analysis. However, human intervention is often
required to improve the approximation accuracy w.r.t. those program points where false alarms
have to be removed.

5.3.7.2 Deductive Verification

Deductive methods [19] [20][21] [22] perform mathematical proofs to establish formally specified
properties of a given program, thus providing rigorous evidence. Its primary use is to verify
functional properties of the program. This method is based on the Hoare logic [23, 24], or
axiomatic semantics, in which functions are seen as predicate transformers. In summary, a
function f is given a state described by a given predicate P and transforms it into a new state,
described by another predicate f (P). In this context, the specification of f is given by a contract,
which defines the predicate R that f requires from its callers and the predicate E that it ensures
upon return. Verifying the implementation against such a specification amounts to proving that
for each P such that P ⇒ R (i.e. that satisfies the requirement of f), then E ⇒ f (P) (i.e. the
concrete final state is implied by what f ensures).

Tools based on deductive verification usually extract proof obligations from program code and
property specifications and attempt to prove them, either automatically or interactively. Some
tools are tightly coupled to a given theorem prover such as Atelier B [25] or Rodin [26], while
other such as Why3 [27] promote a cooperation across a wide range of provers. In addition to the
contracts of the function, it is often required to provided additional annotations in order to be able
to use deductive verification. In particular, for each loop in the code, a suitable loop invariant
has to be provided. A loop invariant is a property that is true when encountering the loop for the
first time and, if true at the beginning of a loop step, stays true at the end of this step. From both
hypotheses, it is then possible to inductively conclude that the invariant is true for any number of
step, and in particular at the end of the loop. While it is possible to synthesize automatically loop
invariant in some simple cases, in particular thanks to abstract interpretation, this activity must
most of the time be done manually.

Similarly, some proof obligations are too complicated to be handled by automated theorem
provers, and must be discharged interactively via proof assistants [28, 29]. Deductive verification
is thus much less automated than abstract interpretation. On the other hand, it is much more
flexible for functional properties verification, in the sense that it can be used to prove any property
that can be expressed in the specification language of the tool (usually any first-order logic
property), while abstract interpretation is limited to the properties that fit within the abstract
setting that has been chosen.

5.3.7.3 Model Checking

Model checking [30] explores all possible behaviours of a program to determine whether a
specified property is satisfied. It is applicable only to programs with reasonable small state
spaces; the specifications are usually about temporal properties. If a property is unsatisfied, a
counter-example can be generated automatically, showing a use case leading to property violation.

Bounded Model checking [32] (BMC) is mainly used for finding bugs more than proving
properties. The basic idea of BMC is to find a counter-example-trace of a bounded length . The
transition relation of the system is unrolled to a bounded length symbolically and check against a

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 23

propositional formula with a SAT or a SMT-solver. If the formula is satisfiable, it exists a feasible
path in the system that validate the formula, the SAT solver returns a satisfying assignment that
is transformed into a counter-example. Otherwise the bound is increased and the process is
repeated. Some more recent works [33, 34] add the use of induction techniques and interpolant
to prove the properties.

5.3.8 Correct by Construction Formal Methods

5.3.8.1 Event B for system analysis

5.3.8.2 Classical B for software development

Classical B [21] is used for software development. It is supported by Atelier B from ClearSy[25],
a free partially open source tool. The B software development starts with software requirements
usually expressed in a natural language document. Classical B uses a mathematical language
suitable both to model the requirements and to write implementable code. In a first step called
formal general designed, requirements are modeled into B specification modules. The correctness
of this step is partially covered by proof and partially by human verification. Then in a second
step called formal detailed design, B specification modules are refined by breaking down modules
until the B model is fully implemented. The correctness of this step is entirely covered by proof.

A B module contains state data and treatments both abstract and concrete. Those data are based
on integers, sets, relations and functions. The properties of data are expressed with classical first
order predicates. The treatments, which are called operations, are expressed with substitutions,
which may be abstract (for instance one may specify that some state variables become such that
some predicate should hold), or concrete like mere statements (assignment, if, case, while).

A B module should follow a refinement process. During this process, specification information is
gradually added, design choices are made, and eventually code is written by breaking down the
module in other modules. The refinement consistency is handled by dedicated proof obligations,
generated by the tool. Those proof obligations should be proved using specific proof tools: some
automatic provers try to discharge proof obligations and when they fail, the user should help to
build a demonstration in the interactive prover.

During the proof activity of large models, the model should be tuned to build a complete and
consistent model where everything should fit together. Through this process, the user becomes
aware of the precise properties needed to satisfied every part of the model.

In the end, B model code is translated into C or Ada code by automatic translator tools. This
translation is basically a syntax transformation. However, as translating is here safely critical,
usually 2 independent tools are used.

With classical B, the strength of a consistent and fully proved model is so high that it gives a very
high level of trust in the code obtained. The proof activity makes it useless to go through a unit
test activity, since symbolic proof covers all possible use of each operation, whereas unit tests
only cover a limited number of cases.

5.3.8.3 B predicate evaluation for data verification and validation

5.3.9 Verification with Formal Methods

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 24

In the railway domain, the standard EN 50128 highly recommends use of formal methods in
requirements specification ([2, Table A.2]), software architecture (A.3), software design and
implementation (A.4), verification and testing (A.5), data preparation (A.11), and modelling
(A.17) for Safety Integrity Level SIL 3 and above. However, functional/black-box testing is still
mandatory in verification; this constraint may be considered as discouraging from the use of
formal methods.

Until recently, the situation was quite similar in the aerospace domain. J. Joyce, a member of
the RTCA standardisation committee SC-205, described Airbus’ problems in certifying their
“unit-proof for unit-test” approach:

“Formal methods were used for certification credit in development of the A380, but
apparently it was not a trivial matter to persuade certification authorities that this was
acceptable even with the reference to formal methods in DO-178B as an alternative
method.”

Such experiences eventually caused the more detailed treatment of formal method issues in the
revision C of DO-178 that appeared in late 2011. The DO-178C considers formal methods as
special cases of reviews and analyses; thus incorporating them without major structural changes
of the software development recommendations. For an employed formal method, the standard
requires to justify its unambiguity, its soundness2 , and any additional assumptions3 needed by the
method. The DO-178C admits formal property verification on object code as well as on source
code, the latter additionally needing evidence about property preservation of the source-to-object
code compiler. However, “functional tests executed in target hardware are always required
to ensure that the software in the target computer will satisfy the high-level requirements” [4,
FM.12.3.5].

As a consequence of subsuming formal methods under general reviews and analyses, no deviating
special rules to qualify tools are necessary: “Any tool that supports the formal analysis should
be assessed under the tool qualification guidance required by DO-178C and qualified where
necessary.” [4, FM.1.6.2]. Of course, for the railway domain, the rules of EN 50128 for
supporting software tools and languages must be taken into account [2, Section 6.7].

During the last 15 years, formal methods have grown out of academic playgrounds and become
practically relevant in several applications domains. Below, we sketch a few different tools, also
to indicate the variety of issues formal methods can be applied for. Many of the tools mentioned
below provide formal verification for programs written in C. There is currently insufficient support
for the programming language C++, which is predominantly used in Thales’ RBC product. A
list of free software tools for formal verification can be found at [35]. The list is not meant to be
complete. It is structured by tool purpose, and each tool is briefly introduced.

5.3.9.1 The Frama-C Source Code Analysis Suite

Frama-C [36] is a suite of tools from CEA LIST and INRIA Saclay, dedicated to the analysis
of C source code. Frama-C gathers several static analysis techniques in a single collaborative
framework. Frama-C also features a formal specification language, ACSL [37], in which the
contract of each function of the program can be written (see section 5.3.7.2), as well as assertions
that are supposed to hold at a given program point.
2
i.e., that the method never asserts a property to be true when it actually may be not true
3
e.g. data range limits

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 25

Frama-C’s kernel as well as many analysis plug-ins are available under the LGPL Open-Source
licence from [38]. Other plug-ins have been developed by third-party developers, either in an
academic [39] or an industrial [20] background. The remainder of this section only deals with
the plugins that are released with Frama-C’s kernel and are the most relevant for OpenETCS.

Value Analysis

Value analysis is based on abstract interpretation (section 5.3.7.1). This plugin analyses a
complete application, starting from a given entry point, and gives at each program point an
over-approximation of the values that can appear in each memory location at this point. For each
operation, Value also checks that whether the abstract value of the operands guarantees that the
operation is safe. If this is not the case, it emits an alarm, in the form of an ACSL assertion,
and attempts to reduce its abstract state to represent only safe concrete values. If all concrete
values are unsafe, then either the corresponding branch of the code is dead (and was only taken
because of the over-approximation), or there is a real error in the code. Otherwise, the analysis
resumes with the reduced state. Conversely, if no alarm is emitted by Value, the analysed code is
guaranteed not to lead to a run-time error.

Value can also be used to check whether ACSL annotations hold or not. However, it is restricted
to the subset of the ACSL language that fits well within the abstract representation that is used.

Finally, Value can be tweaked in various ways to increase the precision of the results (leading to
fewer false alarms), generally at the expense of the computation time and amount of memory
used by the analysis. These options are described in more detail in Value’s reference manual [40].

WP

WP is a plugin dedicated to deductive verification (see section 5.3.7.2). It uses different models
to represent C memory states in the logic. More abstract models lead to easier proof obligations,
but cannot be used in presence of low-level pointer arithmetic, while more concrete ones are able
to deal with any C construction, at the expense of far more complex proof obligations.

WP has two native interfaces to discharge proof obligations. The first one calls the Alt-Ergo [41]
automated theorem prover, while the second let the user do the proof within the Coq [28]
interactive proof assistant. In both cases, the original formulas are first run through an internal
simplifier, that can directly discharge the simplest proof obligations, without the need for a
call to an external tool. In addition, WP can also call the Why3 [27] back-end, through which
it has access to a variety of automated provers. Alt-Ergo, Coq and Why3 are available under
Open-Source licences (Cecill-C and LGPL). The various possible settings of WP are described
in its user manual [42].

While WP’s primary usage is to prove functional properties expressed as function contracts, it
can also be used to prove the absence of runtime error, either by discharging the alarms emitted
by Value Analysis, or by generating proof obligations for all operations that might lead to a
runtime error (without having to use Value first). The latter case is done through the use of the
RTE plugin, that generates an ACSL assertion for each potentially dangerous operation. WP can
then generate proof obligations for these assertions as usual.

Aoraï

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 26

While Value and WP are used to verify program properties, the Aoraï plugin is dedicated to
generate ACSL specifications (which can then be proved by Value or WP). More precisely, it
takes as input an automaton describing the sequence of function calls that are allowed during the
execution of a program (from a given entry point). From that automaton, Aoraï instruments the
code and provides ACSL contract for each function so that if all the contracts hold, then the code
is behaving according to the automaton.

Transitions of the automaton can be guarded by conditions over the state of the program at a
given call point. Full syntax of Aoraï’s input language is described in [43].

5.3.9.2 The Diversity Symbolic Execution Tool

DIVERSITY is a symbolic execution tool developed at CEA − LIS T . Its underlying technology
is symbolic execution. Symbolic execution has been first defined for programs [44, 45, 46]. The
goal of this technique is to identify, for each possible execution of the program, the constraints
to be satisfied in order to follow it. The main idea consists in executing the program, not for
concrete numerical values but for symbolic parameters, and to characterize constraints on those
parameters at each step of the execution. For instance let us consider that at a given step of
an execution the next instruction ins to be executed is i f (x > 14)x := x + 1. Moreover let us
suppose that from the previous steps we have computed a couple (x → a, a < 45) meaning that
before the execution of ins, the value of x is represented by the symbolic parameter a, with the
constraint that a < 45. Executing ins results on a new context (x → a + 1, a < 45 ∧ a > 14)
taking into account both the constraints so that the instruction is executable (x has to be greater
than 14 and since x value is a it means that a has to be greater than 14) and the variable updates
induced by the instruction (the result of the execution of x := x + 1 is that x value is now a + 1).
a < 45 ∧ a > 14 is called a path condition. Generating test data to follow some executions
comes the to use solvers to find values satisfying such path conditions. Symbolic execution
has been later adapted to modeling formalisms like Input Output Symbolic Transition Systems
([47, 48]), later to timed version of Input Output Symbolic Transition Systems ([49, 50]) and
also to various industrial modeling languages like the sequence diagrams of the U ML ([51]).
Those symbolic execution adaptations have been used in model based testing contexts. System
under test are compared to their models by means of two conformance relations namely ioco
([52]) and its timed extension tioco ([53]). Those two conformance relations are among the most
widely accepted conformance relations. Several testing algorithms were defined based on those
conformance relations ([54, 49, 50]).

In the frame of the openETCS project we plan to use DIVERSITY to extract test cases from
models defined in the first phases of the system design. Our goal is to extract test cases dedicated
to abstract safety requirements. More precisely we focus on safety requirements dealing with
communication between sub systems. For that purpose we will use the language of sequence
diagrams extended with timing constraints to specify such requirements. With sequence di-
agrams, one may describe execution scenarios in terms of partially ordered message passing
between subsystems. Message passing can be structured thanks to powerful operators expressing
sequencing, parallelism, choice, loop... In [51] we show how to automatically analyze such
sequence diagrams with DIVERSITY in order to extract test cases. The originality is that is that,
thanks to projection mechanisms, it is possible to extract test cases, not only for the entire system,
but also for any of its distinguished sub systems. Thanks to this mechanism, sub systems can be
tested as soon as they are implemented, even though the entire system is not yet implemented.
In such a process we perform a particular kind of unitary testing in which unit test cases are
built according to the usage that will be made of the sub system in the entire system. Faults
identified with such an approach are very relevant because we know that they will be activated in

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 27

(1) specify system intended behaviors

as a UML Sequence Diagram
with MARTE time annotations (SD) (2) m1,m2

C1 C2 C3
m1 (3)

alt m2

m1 m1,m2 m2

(5) if each component (sub-system)

conforms to its elicited behaviors (4)
then the system as a whole
is conform to the sequence diagram
C1 C2 C3 C1 C2 C3

(2) compute system behaviors using symbolic techniques as symbolic trees

(3) elicit unitary behaviors using projection techniques as symbolic sub trees
(4) perform unit testing on each component (sub-system) in isolation

Figure 2. Compositionnal Testing

the system. The process is illustrated in Figure 2. The first step consists in defining a requirement
model in the form of a sequence diagram. The requirement model is analyzed with DIVERSITY
in step (2). This analysis results on a so-called symbolic tree, whose each path denotes a possible
(symbolic) execution of the sequence diagram. Such trees may be theoretically infinite due to the
possible occurrences of the "loop" operator of sequence diagrams. Therefore, DIVERSITY uses
various stopping criteria to stop the computation (typically based on message coverage notions).
The symbolic tree computed in step (2) characterizes executions of the whole system model.
However because testing the whole system may be complicated in terms of testing architecture,
or simply because one wants to test some sub systems before the whole system is implemented,
we offer a mechanisms to extract symbolic trees for each distinguished sub system. This is based
on so-called projection techniques ([55, 49]). This operation is realized in step (3). In step (4)
Each identified sub system is tested thanks to a real time off-line testing algorithm ([50]). Thanks
to a compositionality theorem ([56]) we can relate correctness of sub systems and correctness of
the whole system (see step 5). The compositionality theorem expresses that, the conformance of
each subsystems to all their projections guarantees the conformance of the whole system to the
sequence diagram. A direct consequence is that any faults of the whole system can be discovered
as a fault of at least one of its sub systems. This implies that testing the whole system mainly
comes to test each of its sub systems regardless of a very simple test integration phase in which
one only tests that each sub system is correctly connected. We believe that such an approach will
be very useful for ETCS systems which are by nature very distributed and thus hardly observable
and controllable as a whole. We plan to identify with experts how to partition them into several
sub systems that will be more easily observable and controllable at the testing phase.

5.3.9.3 Microsoft’s Verifier for Concurrent C (VCC)

VCC is a tool from Microsoft Research to prove correctness of annotated concurrent C programs.
It was mainly developed to verify Microsoft’s Hyper-V hypervisor. It supports an own annotation
language providing e.g. contracts, pre- and postconditions, and type invariants. It uses the Boogie
tool to generate proof obligations, and the automatic prover Z3 to prove them. If an obligation is
violated, the Model Viewer tool can generate a counter-example use case. VCC is available for
non-commercial use from [57].

5.3.9.4 The Proof Assistants Coq and Isabelle

Coq is an interactive theorem prover and proof checker, developed at INRIA, and based on
higher-order logic and the natural deduction calculus. It provides the formal language Gallina, in
which mathematical definitions can be expressed as well as executable algorithms and theorems.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 28

The supporting tool for tactics-based semi-interactive development of proofs is available from
[28].

Isabelle, maintained at Cambridge University, and its predecessor HOL4 , are similar tactic-
oriented interactive theorem provers. Isabelle is available from [29]. While Isabelle is not yet
supported in the Frama-C environment, Coq is.

5.3.9.5 The Model Checker NuSMV

SMV5 has been the first model checker based on binary decision diagrams. NuSMV is a reimple-
mentation by the Fondazione Bruno Kessler that is in addition capable of performing SAT-based
model-checking. It supports both Linear Temporal Logic (LTL) and Computation Tree Logic
(CTL). NuSMV’s source code is available under an LGPL license from [58].

5.3.9.6 Formal Verification of Real-Time Aspects based on Timed Automata

Verifying system properties involving time is difficult with traditional model checking methods.
Commonly used temporal logics, such as LTL or CTL catch discrete and qualitative aspects of
time and allow to formulate properties such as6

X At the next point in time a property holds.

F At some future point in time a property holds.
G Always/generally (now and at any future point in time) a property holds.
U A property p holds until a property q holds.

While it is possible to state properties that must be satisfied at individual (discrete) points in time,
continuous and quantitative aspects of time as in the safety requirement

“The delay between receiving an emergency message and the issuing of a brake order is
less than 1 second.”

are a real challenge. The problem does not stem from discrete vs. continuous time, as any
physical realisation of a real-time system is inherently discretised by its clock. Instead, an
operator for expressing arbitrary temporal quantities or differences is missing. Thus, for LTL and
a clock of 1 kHz, it would be required to use the operator X 1000 times. This notation is rather
unhandy, as it enforces to express a functional property relative to a particular system.

This lack of expressivity is not merely a matter of notation, i.e., LTL or CTL, but also of the
underlying semantics. Before introducing a better suited logic we will first consider a formalism
that serves as this logic’s semantics – namely timed automata.

Timed Automata

Timed automata [59] are essentially finite automata extended with a finite set of clocks that all
proceed at the same rate. Clocks may be individually reset to zero. Clock variables can be part of
4
Higher Order Logic
5
Symbolic Model Verifier
6
The prefixes are the corresponding linear time operators.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 29

emg_msg, x := 0 x < 1, proc_msg

start wait rcvd prcd

x < 1, brake

Figure 3. First (faulty) version of a timed automaton for processing emergency messages

emg_msg, x := 0 proc_msg
start wait rcvd prcd

x<1 x<1

brake

Figure 4. Corrected version of the timed automaton for processing emergency messages

constraint expressions that may be used as transition guards. A transition can only be taken if its
guard is fulfilled. Similarly, it is possible to specify an invariant for a state that must be satisfied
when the automaton is in this state. Thus, we can enforce time constraints for the runs of the
automaton.

An Example

The timed automaton in Figure 3 depicts an automaton representing an over-simplified version of

an OBU subsystem processing emergency messages. It has three states, one clock x and three
actions, emg_msg (reception of an emergency message), proc_msg (processing the message, e.g.
raising an alarm) and brake (issueing the brake order). Upon receiving an emergency message
the clock x is reset. The proc_msg-transition is guarded by the clock constraint x < 1 preventing
the transition to be taken if x ≥ 1. The same holds for the brake-transition.

One might think that the automaton from Figure 3 thus fulfills the safety requirement stated
above. Due to the operational semantics of timed automata this is not true: a timed automaton in a
given state can either take a transition or wait for an arbitrary amount of time. Thus, if automaton
waits in state rcvd and x exceeds one second, the system will deadlock as the next transition is
guarded by the constraint x < 1. A run of the automaton that could serve as counterexample is,
e.g.

(wait, 0) → (wait, 0.5) → (rcvd, 0.5) → (rcvd, 2) → DEADLOCK

A solution to this problem is to force the automaton to proceed by placing progress constraints
on the states. This has been done in Figure 4. The transition guards have been omitted as they
are not necessary anymore. Now the safety property “The delay between receiving an emergency
message and the issuing of a brake order is less than 1 second.” is fulfilled.

UPPAAL

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 30

Uppaal is a tool for modelling and verifying timed automata developed by the universities of
Uppsala and Aalborg [60]. This toolkit is under constant development and comes with an
academic as well as a commercial licence. Moreover, there is a comparably large body of
literature featuring Uppaal, providing introductory and industrial examples.

Uppaal extends timed automata with synchronisation enabling concurrent, communicating au-
tomata representing different parts of a system. In addition, variables other than clocks are
supported making the modelling language more powerful. The logic used for expressing real-
time properties is a subset of TCTL (Timed Computation Tree Logic). Nesting of temporal
operators is not supported leading to a restriction in expressiveness.

From SysML/UML to Timed Automata

Timed automata and statecharts in SysML/UML are both based on the concept of finite automata.
Thus, it seems reasonable to extract timed automata from existing statecharts which is addressed
in the literature [61, 62, 63]. In this way, safety properties – formalised as TCTL formulae – can
be verified in an automated fashion for a given statechart. However, there remain challenges:

• Translating hierarchical states to timed automata is not straight-forward and complicates

matters significantly. If an hierarchical state-chart is flattened, structural information is lost
and makes the timed automaton more difficult to read and understand. Thus, it is advisable to
retain some kind of hierarchy, possibly by using synchronisation mechanisms.

• Special statechart features, such as history nodes that have a partially undefined semantics
according to the current SysML/UML standard [64], introduce problems. As they are not
used very often, they can possibly left out in a first iteration.

5.3.10 Verification with Model-Based Simulation

This section addresses verification based on simulation. By building an executable model of

system components its real-time behaviour can by analysed and evaluated before actually building
the entire system.

5.3.10.1 Modelling with SysML and SystemC

Specifications in natural language are difficult to handle. Breaking down an overall system
description into small comprehensible parts reduces complexity and eases interdisciplinary
communication to be more efficient in performing development tasks.

SysML, developed by the OMG (Object Management Group), is a simple but powerful general-
purpose graphical modeling language that does not directly support executable models. However,
there is a variety of tools for code generation from UML/SysML, especially for the Eclipse
platform and the Papyrus framework that will be used in the project.

To enable model execution and especially real-time simulation it is considered to generate semi-
formal SystemC code from that abstract SysML models. It has to be investigated whether the the
SysML model have to be adapted to a domain or language specific version. Concrete analysis
will show whether this is feasible.

SystemC is a C++ library providing an event-driven simulation interface suitable for electronic
system design at various abstraction levels (from high level down to individual hardware com-

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 31

ponents). It enables a system designer to simulate concurrent processes. SystemC processes

can communicate in a simulated real-time environment, using channels of different datatypes
(all C++ types and user defined types are supported). SystemC supports hardware and software
synthesis (with the corresponding tools). SystemC models are executable.

5.3.10.2 Model execution and simulation

The aim of the execution of an SystemC model is to ensure that the working capacity (perfor-
mance) of the underlying hardware system is sufficient to meet the system requirements. It has
to be analysed which hardware resources will be needed for the OBU to avoid excessive delays
and to ensure adequate response times in critical situations. Because of the integrated simulation
enviroment, SystemC enables scheduling analysis for average and worst-case conditions and
provides analyses of process resources for individual system functions.

In addition, by creating an executable system model from SysML or UML, the (real time)
behaviour of the system can be analysed which is not feasible at the SysML level.

5.4 Verification Acitivities for a Full Development

for each of the verification steps identified in the plan overview, the following has to be instanti-
ated:

5.4.1 DAS2V Verification

5.4.1.1 Task

5.4.1.2 Documents to Be Produced

5.4.1.3 Phase Specific Activities

5.4.1.4 Techniques and Measures

Here the verification plan begins

5.4.2 SSRS Verification (1c)

5.4.2.1 Task

The SSRS (sub-system requiement specification) outlines the subsystem which is going to be
modeled within the project. The SSRS describes the architecture of the subsystem (functions
and their I/O) and the requirements allocated to these functions. If necessary, the requirements
are rewritten in order to address the I/O and to correspond to the allocation. It also provides the
classification into vital and non vital requirements and data streams. The architecture part is
described in a semi-formal language, and the requirements are described in natural language.

The SSRS is to be viewed as a supplement to the SS-026 and the TSIs and is not intended to
replace them. The verification has to check that a complete and consistent set of functionalities
have been identified and that the architecture is adequate.

5.4.2.2 Documents to Be Produced

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 32

SSRS verification report.

5.4.2.3 Phase Specific Activities

5.4.2.4 Techniques and Measures

Due to the informal nature of the SSRS, mainly manual techniques are to be applied.

¿Review?

5.4.3 SFM Verification (2c)

5.4.3.1 Task

5.4.3.2 Documents to Be Produced

5.4.3.3 Phase Specific Activities

5.4.3.4 Techniques and Measures

5.4.4 System Verification

SSRS Verification(1c)

The SSRS Verification phase refers to the task already defined in the Sec. 5.4.2 of this document
that involves the full development. For further details about the activities involved, please go to
the mentioned section.

5.4.4.1 Software Verification

This section describes software verification activities in greater detail than the description above.
This presentation of the material shall be unified in due course.

The SW process is detailed in the D.2.3 OpenETCS process. Bearing in mind that the tasks
described in this openETCS Verification plan are strongly linked to the Software process defined
in that document, the following figure is included in order to have present the key points to be
covered in the plan.

SW Requirements Verification

Task

In the SW Requirements Specification the SSRS shall be taken as starting point, redefining it
to ensure the software constraints are considered. The SW Requirements Verification phase
then shall verify that the proposed Software Requirements cover as much as possible of the
SSRS and provide a correct implementation of the System Requirements in the Software context.
On the other hand, the Verification done in this phase shall include the assessment of the SW
Requirements modelling, the objective is to ensure the representation of the requirements is
coherent with their specification as well as complete, explicit and implementable, as well as
traceable to the semi-formal and/or formal models defined in the system phase.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 33

Figure 5. Software phase description

Documents to Be Produced

• SW Requirements Verification Report

Activities

Some activities shall be performed to ensure the requirements are written in a readable and testable
manner. The Verification of the Software Requirements shall assess whether the requirements
met the following aspects:

• Deterministic: Given an initial system state and a set of inputs, you must be able to predict
exactly what the outputs will be.

• Unambiguous: All openETCS project members must get the same meaning from the require-
ments; otherwise they are ambiguous.

• Correct: The relationships between causes and effects are described correctly.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 34

• Complete: All requirements are included. There are no omissions.

• Non-redundant: Just as the Software modelling (semi-formal and formal) provides a non-
redundant set of data, the requirements should provide a non-redundant set of functions and
events.

• Lends itself to change control: Requirements, like all other deliverables of the openETCS
project, should be placed under change control.

• Traceable: SW Requirements must be traceable to each other, to the SSRS, to the objectives,
to the design, to the test cases, and to the code.

• Readable by all project team members: The project stakeholders, including the users, experts
and testers, must each arrive at the same understanding of the requirements.

• Written in a consistent style: Requirements should be written in a consistent style to make

them easier to understand.

• Explicit: Requirements must never be implied.

• Logically consistent: There should be no logic errors in the relationships between causes and
effects.

• Lends itself to reusability: Good requirements can be reused on future projects based on
openETCS.

• Succinct: Requirements should be written in a brief manner, with as few words as possible.

• Annotated for criticality: Each SW requirement should note the level of criticality to the
openETCS project. In this way, the priority of each requirement can be determined, and the
proper amount of emphasis placed on developing and testing each requirement.

• Feasible: If the software design is not capable of delivering the requirements, then the
requirements are not feasible.

The activities that shall ensure these aspects are successfully met are the following:

• SWReq-Ver-Act1. Compliance with SSRS: the SW Requirements are based on the SSRS, so it
shall be ensured that their specification is compliant and coherent with regard to the SSRS;
the SW requirements complement, adjust and enlarge the scope delimited by the SSRS in the
Software phase.

• SWReq-Ver-Act2. Accuracy, Consistency, Completeness, Correctness assurance: A signif-

icant problem with recording requirements as text is the difficulty of analyzing them for
completeness, consistency, and correctness. The Specification of the SW Requirements has
precise syntactic and semantic rules (e.g., data type consistency) and the specification of the
requirements shall be compared against those established rules to ensure these aspects are
correctly covered.

• SWReq-Ver-Act3. Testability assurance: The two objectives for testing the openETCS
safety-critical SW are the demonstration that the software satisfies its requirements and the
demonstration that errors that could lead to unacceptable failures have been removed.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 35

• SWReq-Ver-Act4. Verificability assurance: The SW Requirements shall identify the main

functionality and describe the functional breakdown and data flows from top-level functions
to low-level functions. The description provided as well as the flows identified shall be
verifiable.

• SWReq-Ver-Act5. Compliance with Standards: Compliance with CENELEC Standards

(EN50126, EN50128 and EN50129) shall be verified.

• SWReq-Ver-Act6. Traceability with SSRS: Each SSRS allocated to software must map to one
or more software requirement. Traceability analysis, which can be automated, determines
the completeness of the mapping of SSRS to software. A table similar to this one shall be
obtained, either manually or automatically to assess the conformance with this expected
activity.

• SWReq-Ver-Act7. Requirements modelling correctness: Considering "the result of the mod-

elling activities is a document that represents a thorough understanding of the problem the
proposed software is intended to solve"[1. Cho, Chin-Kuei, Quality Programming, John
Wiley & Sons, Inc, 1987, p 21.], during the SW Requirements verification, it shall be possible
to capture all required information, including additional information for safety-critical aspects
with the support of the models. These Requirements representations shall be compared to
their original specification and assess whether they are complete and enough to cover all the
information provided by them.

Table 2. Requirements Trace Table

Source SSRS ID SW Requirement ID

SSRS-xxxx SWR-yyy
SSRS-xxxx SWR-yyy

Tools, Techniques, Methods and Measures

This section shall identify the special software tools, techniques, and methodologies to be
employed by the verification team. The purpose of each should be defined and plans for the
acquisition, training, support, and qualification of each shall be described in this section.

The following table summarizes the Techniques, methods, measures or tools proposed for the
identified activities.

Table 3. SW Requirements Verification Tools, Techniques, Methods and Measures

Activity Techniques/ Methods/ Measures Tools

This compliance is verified by peer review. The first
SWReq- version of SW Requirements is usually incomplete and
<To be defined>
Ver-Act1 composed primarily of top-level adjustments. The mean-
ing of the SW requirements is described textually.
Continued on next page

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 36

Table 3 – continued from previous page

Activity Techniques/ Methods/ Measures Tools
Since the modelling at this stage is provided, verification
is mostly based on review and with a tools-based mod-
SWReq-
elling approach. Some consistency checks between the <To be defined>
Ver-Act2
SW Requirements and both the semi-formal modelling
and the formal modelling can be automated.
Requirements-based testing has been found to be effec-
tive at revealing errors early in the testing process Re-
quirement models created with standard UML use-case Test cases can be auto-
SWReq- notations are not robust enough to create requirements matically generated from
Ver-Act3 based test cases that met the safety-critical aspects, so “test-ready” use-case mod-
the standard use-case model notation shall be extended els. <To be defined>
to allow the capture of additional information required
to determine test cases.
SWReq-
Verification is mostly based on review <To be defined>
Ver-Act4
SWReq-
Verification is mostly based on review <To be defined>
Ver-Act5
Integrated tools can gener-
ate software to system re-
A traceability matrix between SSRS and SW Require- quirements traceability ta-
SWReq-
ments shall be prepared either manually or automatically bles showing which soft-
Ver-Act6
and assessed by review ware requirements are al-
located to system require-
ments. <To be defined>
SWReq-
Verification is mostly based on review <To be defined>
Ver-Act7

SW Architecture, Design and Modelling Verification

Task

This phase aims to ensure that the software architecture design adequately fulfils the software
requirements specification already verified in the SW Requirements Verification phase. The
software architecture defines the major elements and subsystems of the openETCS software, how
they are interconnected, and how the required (safety integrity) attributes will be achieved. It
also defines the overall behaviour of the software, and how the software elements interact. To
carry this phase the information from the current software lifecycle phase shall be verified. All
essential information should be available and must be verified; the information should include the
adequacy of the specifications, design and plans in the current phase. The architecture and design
shall be modelled and the coherence of those models with regard to their specification shall be
verified to assess whether the constraints are correctly treated. The verification configuration
should be precisely defined and the verification activities shall be repeatable.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 37

Documents to Be Produced

• SW Architecture, Design and Modelling Verification Report

Activities

Software architecture verification should consider whether the software architecture design
adequately fulfils the SW requirements specification.

The essential properties to be verified during this phase are:

• What is the software supposed to do: including the SW specification and the system capabili-
ties

• What is the software not supposed to do: be aware of the unexpected emergent behaviour,
boundary specifications, inhibits

• What is the software supposed to do under adverse conditions: system biases, tolerances

• What are the safety considerations

• What are the integration or interfacing considerations: subsystems, modules, components,

relationship between hardware and software, partitions, parallelism, compatibility

• What are the dependability considerations: performance, capacity, complexity, stability

Considering these aspects, a table shall be prepared to list a relation of verification dimensions
regarding to the Architecture and Design that facilitates the process. An example is given below:

Table 4. SW Architecture Verification preparation table - Example of contents

Verification Intended Unacceptable Adverse

Safety Integration Dependability
dimension behaviour behaviour conditions
Consequences
of service
Distributed
Performance Service failure, un-
redundancy
analysis delivery expected
and fail-over
emergent
behaviour
Safety
and
Blown per- Re-
Timeline
formance cover-
Distributed manage-
Timing Re- normal margins, abil-
redundancy ment, phase
quirements behaviour missing ity,
and fail-over transition
timing re- tim-
interlocks
quirements ing
mar-
gins
Continued on next page
This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 38

Table 4 – continued from previous page

Verification Intended Unacceptable Adverse
Safety Integration Dependability
dimension behaviour behaviour conditions
Redundant
Compromised
Security Re- Secure com- commu-
communica-
quirements munication nication
tion
channels

The software architecture modelling, which consists of determining and connecting software
components, requires a phase of analysis to be able to validate the representation carried out, as
errors can occur in the representation. It is thus necessary to be able to identify and to provide
them to the designer.

Modelling must also bring answers in term of feasibility and the constraint shall be analyzed so
the interdependent functions should not overlap.

The verifications done at the software architecture level are related to specification and execution
model coherency.

The expected activities for this phase are the following:

• SWArch-Ver-Act1. Check the software architecture design: The SW architecture and design
should fulfil the SW requirements specification. From a safety point of view, the software
architecture phase is where the basic safety strategy for the software is developed.

• SWArch-Ver-Act2. Handle attributes: The attributes of major elements and subsystems

should be adequate with reference to the feasibility of the safety performance required,
testability for further verification, readability by the development and verification team, and
safe modification to permit further evolution. With the attributes, on which we rely once the
software is specified, it is possible to check if the model of execution and the coherence of
the periods of the various software components are corrects.

• SWArch-Ver-Act3. Check incompatibilities: The incompatibilities between design and specifi-

cation should be checked.

• SWArch-Ver-Act4. Model coherency: Verify the representation carried out with the modelling.
The errors identified can be related to the symmetry of the inter-connected functions. A
certain number of constraints of coherence must thus be analyzed.

• SWArch-Ver-Act5. Constraints analysis: Ensure system dysfunctions do not occur. Some

components can also execute an operation depending on a signal resulting from another
component, which results in constraints of synchronization that must also be taken into
account.

Tools, Techniques, Methods and Measures

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 39

The specific software tools, techniques, and methodologies to be employed by the verification
team are to be identified here. The purpose of each should be defined and plans for the acquisition,
training, support, and qualification of each shall be described in this section.

The following table summarizes the Techniques, methods, measures or tools proposed for the
identified activities.

Table 5. SW Architecture, Design and Modelling Verification Tools, Techniques, Methods and Measures

Activity Techniques/ Methods/ Measures Tools

SWArch-
<To be defined>
Ver-Act1
SWArch-
<To be defined>
Ver-Act2
SWArch-
<To be defined>
Ver-Act3
SWArch-
<To be defined>
Ver-Act4
SWArch-
<To be defined>
Ver-Act5

SW Component and Modelling Verification

Task

This phase analyzes whether the SW components defined in the openETCS Software process are
verifiable by design, with a focus on the formal verification of data aspects of components and
components modelling. This task requires a detailed understanding not only of the corresponding
specification, but also the environments and methodology in which the verification component
will be used.

This phase delivers methods and tools to

• Verify the component models by model-checking techniques, with a focus on behavioural

aspects such as attributes and data-dependent properties

• Verify the specification of the components is coherent to the SW architecture and design
previously verified. This activity shall be based on the existing models in the component
stage as well as previous phases, in close connection with the model checking activities.

Documents to Be Produced

• SW Component and Modelling Verification Report

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 40

Activities

This phase aims to assess whether the specification and modelling of components are reusable,
configurable, and implementable in the expected code automatically generated environment.

• Within the context of the models that contain the components identified for the SW phase, the
verification of components within a control system model shall be required. As standalone
components — For a high level of confidence in the component algorithm, verify the compo-
nent in isolation from the rest of the components and sub-systems identified. This approach
is called component analysis.

• Verifying standalone components provides several advantages:

– The analysis can be be used to focus on portions of the design that cannot be tested
because of the physical limitations of the system being controlled.
– This approach can be used for open-loop simulations to test the plant model without
feedback control.

The components shall be identified at the points of minimal coupling (minimal control and/or
information exchange) and decompose the specification. The smaller components might be
amenable to be verified and how to abstract these parts of the specification shall be analysed.

The rest of the specification could be exposed to formal analysis, such as model checking or
theorem proving. The abstraction reduces the number of states that need to be checked by an
automated verification technique and helps in avoiding the state explosion problem, which occurs
in traditional model checking.

An approach to specification decomposition is slicing the system specifications represented with

Petri Nets. In this way, the specification shall improve the understanding of the complexity of the
system and high-risk components can be identified earlier.

The activities that shall ensure these aspects are successfully met are the following:

• SWComp-Ver-Act1. Verify components with minimal coupling independently of the model:

Within this activity a component analysis shall be performed to verify model blocks, atomic
subsystems and statechart atomic subcharts. After extracting the blocks, or contents of the
subsystems and subcharts, a harness model shall be created for the referenced model and the
extracted model with the contents of the subsystem o subchart. A Test suite shall be prepared
and executed to record coverage and output values. The Code Generation Verification (CGV)
shall be invoked to execute the test cases on the generated code for the model that contains
the component.

• SWComp-Ver-Act2. Verify components with minimal coupling in the context of the model: A
similar process shall be conducted in this case but performing a system analysis to verify the
model blocks in the context of the model.

• SWComp-Ver-Act3. Formal analysis of components: Perform model checking of components

against properties; the results obtained shall highlight that: the component satisfies the
properties for the environment; the component violates the properties for the environment;
characterization of those environments in which the components satisfied each property.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 41

Regarding the Modeling Verification, the activities involved shall ensure that:

• The models have been designed correctly and covers all the parts involved in the openETCS
project

• The algorithms have been implemented properly

• The models do not contain errors, oversights, or bugs

• The specification is complete and that mistakes have not been made in implementing the
model

Each of the models defined in the full development context shall be developed for a specific
purpose and its validity is determined with respect to that purpose. It shall be considered that
a model that may be valid for one set of conditions, can be invalid in another; because of that,
the model’s output variables of interest shall be clearly identified and their required amount of
accuracy be specified.

It is costly and time consuming to determine whether a model is valid over the complete domain
of the full development context. Instead, tests and evaluations are conducted until sufficient
confidence and certainty is obtained that a model can be considered valid. In the model verification
activities, tests are performed, errors are identified, and corrections are made to the underlying
models while the models are exercised for the identified cases. Furthermore, the verification of
the models can result in retesting requirements to ensure code integrity.

If the verification process determines that a model does not have sufficient accuracy for any
one of the sets of identified conditions, then the model is invalid. However, determining that a
model has sufficient accuracy for numerous conditions does not guarantee that a model is valid
everywhere in its applicable domain.

• SWModel-Ver-Act1.

• SWModel-Ver-Act2.

• SWModel-Ver-Act3.

Tools, Techniques, Methods and Measures

The specific software tools, techniques, and methodologies to be employed by the verification
team are to be identified here. The purpose of each should be defined and plans for the acquisition,
training, support, and qualification of each shall be described in this section.

The following table summarizes the Techniques, methods, measures or tools proposed for the
identified activities.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 42

Table 6. SW Component and Modelling Verification Tools, Techniques, Methods and Measures

Activity Techniques/ Methods/ Measures Tools

SWComp-
<To be defined>
Ver-Act1
SWComp-
<To be defined>
Ver-Act2
SWComp-
<To be defined>
Ver-Act3
SWModel-
<To be defined>
Ver-Act1
SWModel-
<To be defined>
Ver-Act2
SWModel-
<To be defined>
Ver-Act3

SW code generation Verification

Task

This phase analyzes whether the generated code from the model is correct, robust, complete,
coherent and reliable, with a focus on the formal verification and on the equivalence of the model
and the generated code.

This phase delivers methods and tools to

• Verify the equivalence between the model and the generated code by equivalence testing and
dynamic testing techniques

• Verify the correctness of the translation algorithm and its implementation

• Verify the complexity, coding guidelines and syntax of the generated code by static analysis
techniques

• Verify the robustness of the generated code

Documents to Be Produced

• SW Code Generation Verification Report

Activities

When in the other phase the confidence in the correctness of the model has been verified, it is
necessary to have high confidence in the generated code.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 43

To achieve verified code generation, it is not sufficient to only verify the code generation algorithm
due to the implementation of the algorithm might introduce errors as well. It is necessary to
distinguish between the correctness of the translation algorithm itself and the correctness of its
implementation.

Within the context of the Code Generation Verification, the code generation verification approach
shall utilize translation testing to demonstrate that the execution semantics of the model are
preserved during production code generation, compilation and linking

The validity of the translation process, i.e. whether or not the semantics of the model have been
preserved during code generation, compilation and linking, shall be determined by comparing
the system reactions

The activities that shall ensure these aspects are successfully met are the following:

• SWCode-Ver-Act1. Semantic Properties Verification: For safety reasons, it is necessary that

the generated code is semantically equivalent to the original model specification to ensure
correct software and system behavior.
During this activity, the semantics of the original model is preserved during the code genera-
tion shall be demostrated. This will verify that the transformation algorithm is correct.

Figure 6. Transformation Verification

To prove the semantical correctness of such a transformation, a suitable model semantics is

needed.
As a basic prerequisite, the semantics of the model and code must be comparable and it is
also required a semantics for both model and code with the same semantic domain.
The first steps will be define the commom both model and code semantics. A multitude of
approaches can be taken to define such a semantics. This definintion of the semantics poses
subtle difficulties due to inherent ambiguities. Once this is done the coverage over semantics
shall be identified.
Then, it will be necessary to establish a correspondence between the model and code, based
on refinement, and to prove it by simulation.
A refinement mapping correlating the control points of the model and code, and indicating how
the relevant model variables correspond to the code variables or expressions at each control
point shall be established. For some transformation optimizations it is often impossible to
apply the refinement based rule since there are often no control points where the states of the

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 44

model and code can be compared. So, a large class of these optimizations shall be identified
to allow their effective transformation verification.
After this, the simulation method shall be used for transformation verification.
This transformation from model to code will be considered semantically correct if the
semantics of model and code are always the same.
Using the simulation principle to show that are equal, a simulation relation in which they are
contained shall be found and defined. Note that the definition of a simulation relation is an
artificial construct for conducting proofs.

• SWCode-Ver-Act2. Functional Correctness Verification: Within this activity, as a first

approach for functional properties correctness verification, the automatic test cases generated
from the model will be executed under the automatic generated code from the same model,
so that, the coherence, functionality and coverage of the code shall be verified.
Taking into account the first obtained results, it will be possible to check the coverage of both
code and test cases, detect the necessary functional properties not generated automatically,
and they shall be inserted manually into the code.
When discrepances are found, an analysis of both test cases and code shall be done to check
in which part the problem is taking into account the techniques and tools of the specific
verification phase activities.

• SWCode-Ver-Act3. Quality Properties (Complexity, Syntax, coding guidelines) Verification:

Within this activity, a static analysis shall be performed locating potentially vulnerable code.
The static analysis involves no dynamic execution and will detect possible faults such as
unreachable code, undeclared variables, parameter type mismatches, uncalled functions and
procedures, possible array bound violations, standards fulfillment i.e. MISRA standard, etc.
The static analysis techniques will also include:

– data flow analysis: it consists in a technique to follow the path that various specific
items of data take through the code, looking for possible anomalies in the way the code
handles the data items
– control flow analysis: technique for determining the control flow of the code. This type
of analysis will identify infinite loops, unreachable code, cyclomatic complexity that
identifies the number of decision points in a control flow, other complexity metrics like
number of nested statements, etc.

With this analysis, errors in the structure and syntax of the code are going to be found but
run-time errors (errors that only occur when the code is executed) will not be detected as we
are not running the code.
Therefore this will not find things like memory leaks or check whether a value in a variable is
correct.

• SWCode-Ver-Act4. Equivalence Verification: This activity shall use equivalence testing and
other techniques like dynamic testing to demonstrate equivalence between the model and the
generated code.
During this phase, the execution of the model used for code generation and the object code
derived from it with the same input stimuli followed by a comparison of the outputs shall be
performed. The validity of the translation process, i.e. whether or not the semantics of the
model have been preserved during code generation, compilation and linking, is determined
by comparing the system reactions (results, which are the outputs resulting from stimulation
with identical timed test inputs) of the model and the generated code.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 45

If the coverage achieved with the existing test inputs is not sufficient w.r.t. the selected model
coverage metrics, additional test inputs that execute the model elements not covered shall
be created. In practice, the tester can iteratively extend the set of test inputs using model
coverage analysis until the chosen level of model coverage has been achieved. If full coverage
for the selected metric(s) cannot be achieved, the uncovered parts shall be assessed and
justification for uncovered parts shall be provided.

• SWCode-Ver-Act5. Robustness Verification: Within this activity a code analysis shall be

performed to verify its robustness so it behaves reasonably in exceptional situations; wrong
execution paths, computational errors or erroneous inputs and memory alterations coming
from the environment(errors caused by interactions between the software and its environ-
ment(hardware and software))
The exceptional situations coming from the software itself can be automatically proven using
static analyzers based on abstract interpretation such as Frama-C or RSM
After this analysis, there will be necessary to check the exceptional situations coming from the
environment. The inconsistencies between the specification and the user constrains present
in the code shall be detected. To ensure the robustness of a piece of code, the value of each
un-trusted input must be checked against its correctness domain after its production and
before its consumption
The robustness review of the generated code can be done by manual code review to verify
that all checks are present at right locations and that the checked domains are strictly included
in the correctness domains.

Tools, Techniques, Methods and Measures

The specific software tools, techniques, and methodologies to be employed by the verification
team are to be identified here. The purpose of each should be defined and plans for the acquisition,
training, support, and qualification of each shall be described in this section.

The following table summarizes the Techniques, methods, measures or tools proposed for the
identified activities.

Table 7. SW Code Generation Verification Tools, Techniques, Methods and Measures

Activity Techniques/ Methods/ Measures Tools

SWCode-Ver-Act1 <To be defined>
SWCode-Ver-Act2 <To be defined>
SWCode-Ver-Act3 <To be defined>
SWCode-Ver-Act4 <To be defined>
SWCode-Ver-Act5 <To be defined>

Traceability matrix Verification

Task

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 46

The Traceability Matrix is a resource to ensure that the project’s scope, requirements, modeling
and generated code remain as are expected to be when compared to the objectives to be reached.
The matrix traces the different elements by establishing a thread for each requirement to the
corresponding model and component, the generated code that consists on its implementations
and the test cases defined.

• The specifications shall be verified following the traces between the elements defined in the
matrix
• With the matrix verification it shall be ensured that all the required information is included in
the specification, such as process models and data models
• Requirements that are not addressed by configuration items during design and code reviews
can be identified; in a similar way, extra configuration items that are not required can be
highlighted
• The verification of the matrix shall provide input to change requests and future project plans
when missing elements are identified

Documents to Be Produced

• Traceability Matrix Verification Report

Activities

Taking the time to cross-reference each element involved in the project to another element ensures
that the results obtained by each phase of the project are consistent with the requirements, the
modeling and the code.

To accomplish this objective it shall be verified that the traceability matrix is in the following
format:

• The matrix generated is a two-dimensional table,

• There is one column per identified element (such as requirement, model, components, or
code),
• A check mark at the end of the table shall be put if all the row is well traced, coherent and
unambiguous
• Various traceability matrices may be utilized throughout the project life cycle.
• At least the matrices shall contain the following elements:
– Formal specification of Requirements: It shows that each requirement (obtained from
SSRS) has been covered in an appropriate section of the formal specification.
– Design specification through modeling to functional specification, this verifies that each
function has been covered in the design.
– System test plan to functional specification ensures the test case or test scenario for
each process, component and each requirement in the functional specification have been
identified

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 47

Tools, Techniques, Methods and Measures

The specific software tools, techniques, and methodologies to be employed by the verification
team are to be identified here. The purpose of each should be defined and plans for the acquisition,
training, support, and qualification of each shall be described in this section.

The following table summarizes the Techniques, methods, measures or tools proposed for the
identified activities.

Table 8. Traceability Matrix Verification Tools, Techniques, Methods and Measures

Activity Techniques/ Methods/ Measures Tools

SWMatrix-
<To be defined>
Ver-Act1
SWMatrix-
<To be defined>
Ver-Act2
SWMatrix-
<To be defined>
Ver-Act3

Test Verification

Task

Verification of the test suites designed for testing, the conformance of the implementation of the
specifications, the design, the models and/or the code generated against the formal description
of the openETCS system shall involve the verification of some relevant aspects of the test cases
such as:

• expected input/output of the test behaviour

• test results

• test purpose

The properties of the test cases shall be defined as they were liveness properties, providing
notations to specify the test purpose formally. All these properties expressed with formal notation
can be verified using model checking, for example, on an extended state machine diagram, so the
behaviour of the test case is represented. With this methodology, errors in the specification of the
test cases can be found easily.

Documents to Be Produced

• Test Verification Report

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 48

Activities

There are two approaches when designing a test suite:

• Semi-automatic generation of test-cases: the formal specification of the system is used to

generate the test cases. Previously there has been identified some test design techniques that
generate test cases. However there could be some features, like the robustness capabilities of
implementation, that cannot be derived to test cases using semi-automatic techniques.

• Human design of test cases: human designed test suites have some advantages over the semi-
automatic ones that shall be taken into account, for example that the test case can be designed
with a specific test purpose, the test cases can be grouped into various categories (basic
interconnection tests, capability tests, valid behaviour tests, robustness/invalid behaviour
tests, etc.) and the test cases can be manually designed for complex architectures. On the
other hand these test cases, due to they have been designed manually, can be error-prone.

The test suites designed during the openETCS project shall combine both strategies, so in general
terms some of them shall be generated semi-automatically following different techniques to be
reviewed and adjusted later by the test engineers to ensure their appropriateness, and for the most
specific contexts, the required experience and knowledge shall involve a manual design and a
methodology to verify the correctness of those test cases against the formal specification.

Considering these aspects when designing the test suites, the following activities shall be done to
verify the correctness and completeness of the tests cases generated:

• SWTest-Ver-Act1. Verification of semi-automatic generated test cases: The verification of the

test case shall imply to check whether the essential information has been correctly generated.
For example: are sufficient input combinations exercised?, are the tests complete and enough
to find defects?, the conditions have been collected in the test cases, the test cases test one
specific thing each to simplify the tracking of errors and obtain high coverage, the test cases
say how the system works, the test cases are correctly organised in test suites and they keep
the consistency so it will be easy to locate and add new test cases in the future, and finally
assess whether the test cases are:

– Fast: The test cases shall be fast to execute, the running should not take much time.
– Independent: The test cases can be run in any order.
– Repeatable: The result of the test case should be always the same, no matter how many
times it has been executed.
– Small: Small test cases are easy to understand and change, are also likely to be faster.
– Transparent: It should be clear what the purpose of each test case is.

• SWTest-Ver-Act2. Verification of manual designed test cases: the verification process is

similar to the process already defined in the verification of semi-automatic generated test
cases activity. Besides this, some aspects related to the manual specification of test cases
shall be taken into account like the peer-review done to find typical human errors, robustness
and completeness checking or general coherence.

Tools, Techniques, Methods and Measures

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 49

The specific software tools, techniques, and methodologies to be employed by the verification
team are to be identified here. The purpose of each should be defined and plans for the acquisition,
training, support, and qualification of each shall be described in this section.

The following table summarizes the Techniques, methods, measures or tools proposed for the
identified activities.

Table 9. Test Verification Tools, Techniques, Methods and Measures

Activity Techniques/ Methods/ Measures Tools

SWTest-
<To be defined>
Ver-Act1
SWTest-
<To be defined>
Ver-Act2
SWTest-
<To be defined>
Ver-Act3

Coverage Verification

(futher development: requirements, model and code coverage description in task and activities)

Task

Test coverage is a measure of test effectiveness and it aims to reach a sufficient fault removal with
contained testing effort. Considering higher levels of coverage are associated with better quality,
this aspect raises an important practical question of how hard it may be to increase coverage.

The test coverage shall be influenced by:

• Modeling complexity

• Code Complexity

• Type of functionality

• Testing team experience in the area to be covered

All these factors were related to the level of coverage and quality, with coverage having an
effect even after these adjustments. Furthermore, the test effort increases exponentially with
test coverage, but the reduction in field problems increases linearly with test coverage. This
suggests that for most cases the optimal levels of coverage are likely to be well short of 100%.
The level of coverage needed shall be analysed in each phase of the project and adjusted when
more information and details are obtained.

Documents to Be Produced

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 50

• Coverage Verification Report

Activities

One of the key tasks to take into account in the preparation of the V&V plan is that the correctness
of openETCS system shall be verified with respect to its specification, while is is also assessed
how complete the specification is and whether it really covers all the expected behaviours. Doing
an exhaustive verification process is a great challenge considering we are speaking in terms of
simulation-based verification, where test suites shall be prepared with finite subsets of input
sequences.

In this context, it is essential to measure the exhaustiveness of the test suite with the support
of numerous coverage metrics that reflect different aspects of the appropriateness of the test
suite designed. The activities to be done to measure the coverage shall be focused not only on
state-based coverage, but simulation-based verification to the formal verification setting as well.

Then, coverage metrics are used in order to monitor progress of the verification process, estimate
whether more input sequences are needed, and direct simulation towards unexplored areas of the
design and modeling. The metrics proposed shall measure the part of the design that has been
activated by the input sequences.

The basic approach to coverage in testing, which is recording which parts of the design were
exercised during the execution, cannot be used in formal verification because formal methods are
exhaustive, and this shall be taken into account when designing the coverage strategy and dealing
with formal methods.

The activities that shall ensure these aspects are successfully met are the following:

• SWCover-Ver-Act1. Coverage in model checking: Important advantages of model based

testing are formal test specifications that are close to requirements, traceability of these
requirements to test cases, and the automation of test case design. In model checking,
coverage is a standard measure in testing, but is considerable difficult to compute in the
context of formal verification. With this activity the correctness of the system shall be verified
with respect to the desired behaviour by checking whether the involved models satisfy the
formal specifications. The correctness and exhaustiveness of the specifications made shall
have direct influence in the exhaustiveness of the model checking; the coverage metrics
are the way to check this exhaustiveness and address the verification process to unexplored
areas of the design. One technique to be applied in the activity shall be to measure coverage
checking whether each output is fully determined by the specification, given a combination
of input values. There are other possibilities, like do a model check for each of the mutant
designs, and although a way to simplify the process is to use non-deterministic variables for
mutations, this technique is still time consuming and costly, so it can be proposed for the
most critical scenarios. The following sub-activities shall be included.

– SWCover-Ver-Act1.1. Coverage of the properties defined: Model Checking is a method

for deciding whether a given design satisfies a given formal property: in case the design
violates the property, the Model Checker provides a trace that demonstrates how the
property can be violated. On the other hand, when the answer to the Model Checking
query is positive, most tools terminate with no further feedback. However, there can be
situations where the properties are not as complete as expected and here the skills of the

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 51

verification engineer are essential. For this reason, a sanity check for the completeness
of the set of properties is to measure the coverage of them.
– SWCover-Ver-Act1.2. Code-based coverage metrics: This activity shall imply the
execution of syntactic coverage that shall be supported with the syntactic representation
of the design with respect to which the coverage is measured. It covers the measures
identified to assess the degree to which the source code obtained by automatic generation
from the models is tested by particular test suites.
Among others, the number of code lines executed during the simulation shall be
measured (statement coverage), as well as the functions executed (function coverage),
whether all the branches defined in each control structure present in the code (branch
coverage) are executed at least once, the points of entry and exit are invoked at least once
(decision coverage), the boolean sub-expressions (condition coverage), or combination
of some of them like condition/decision coverage where both decision and condition
coverage are checked to assess whether both have been satisfied in the generated code.
– SWCover-Ver-Act1.3. State-based model coverage: It measures all the states that have
been reached and checked. It is the essential measure to take in formal verification. The
approach proposed is to combine coverage criteria, to use model transformations for
testing, and to combine state machines with the other test models in order to reach a
whole overview of the correctness and effectiveness of the models and whether these
specifications are close to requirements.
– SWCover-Ver-Act1.4. Semantic coverage: Semantic coverage metrics measure the
part of the functionality of the design exercised by the set of input sequences. In this
context, the effect of the mutations shall be checked mainly on the satisfaction of the
specification. The influence of mutations and omissions shall be checked on the result
of model checking of the specifications, in such a way that the influence of omission
or the changes of values of output variables shall be assessed on the satisfaction of the
specification in the design.
Besides this, in path coverage the influence of omitting or mutating a finite path on the
satisfaction of the specification in the design shall be assessed. Within this task, all
possible mutations can be introduced consistently on each occurrence of the mutated
element, on exactly one occurrence, or on a subset of occurrences, thus resulting in
structure, node, or tree coverage, respectively. The application of mutations shall be
done, as said before, in the most critical elements of the design.

• SWCover-Ver-Act2. Functional Test Coverage: It refers to metrics that create reports on the
measurement of functionality exercised in the design and models obtained. In this area the
metrics proposed are the control-flow coverage that indicates how completely the logical
flow of the functional model is traversed, and the data path coverage, that indicates how
completely the data paths have been exercised over a specified set of observable values. A
list of assertions referring to the variables of the design shall be obtained, describing those
assertions the conditions that may be satisfied during the execution or a state of the design.
These kind of coverage metrics shall measure what assertions are covered by a given set of
input sequences.

• SWCover-Ver-Act3. Simulation-based verification: Each of the metrics identified for the

Simulation-based verification is addressed to a specific representation of the design or a
specific verification goal. This activity shall be conducted taking into consideration the
syntactic coverage and semantic coverage metrics perspective and shall contain the following
sub-activities:

– SWCover-Ver-Act3.1. Syntactic coverage metrics: Syntactic coverage metrics shall

assume a specific formalism for the description of the design and this kind of metrics

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 52

shall also measure the syntactic part of the design when executing a given input sequence.
In this context, it shall be considered as a precondition to reach a high degree of coverage
according to syntactic-based metrics before moving to other type of coverage metrics to
apply when performing Simulation-based verification.
The syntactic coverage metrics, as in the Model Checking perspective, shall include the
Code coverage; the task shall be conducted in a similar way with statement and branch
coverage as the basis, where the coverage is calculated on the basis of when these items
(branches or statements) are executed at least once during the execution of a sequence.
The expression coverage shall be also applied to check boolean expressions.
– SWCover-Ver-Act3.1. Semantic coverage metrics: Semantic coverage metrics require
the support of the experts involved in the design of the system and are more sophisticated
than syntactic coverage metrics. Similarly to code coverage, a state or a transition shall
be covered if it is visited during the execution of the input sequence. Limited-path
coverage metrics shall be applied to check what expected sequences of behaviour are
exercised; on the other hand, transition coverage shall be done as a special case of path
coverage, but for paths of length 1. As said before, mutation coverage is the metric that
inspired some important work on coverage in model checking; and it can be applied
in the contexts of semantic coverage and in this specific case, for simulation-based
verification. In mutation coverage, a small change or mutation ins introduced to the
design, and it shall be checked whether the change leads to an erroneous behaviour.
The coverage shall be measured in terms of the percentage of the mutant designs that
fail. The goal here shall be to find a set of input sequences such that for each mutant
design there exists at least one test that fails on it.

Low coverage indicates a possible incompleteness in the specification, which may lead to missed
bugs in the non-covered parts of the design, so the levels of coverage shall be continuously
monitored in order to assure the correct flow of the project.

Tools, Techniques, Methods and Measures

The specific software tools, techniques, and methodologies to be employed by the verification
team are to be identified here. The purpose of each should be defined and plans for the acquisition,
training, support, and qualification of each shall be described in this section.

The following table summarizes the Techniques, methods, measures or tools proposed for the
identified activities.

Table 10. SW Coverage Verification Tools, Techniques, Methods and Measures

Activity Techniques/ Methods/ Measures Tools

SWCover-Ver-Act1 <To be defined>
SWCover-Ver-Act2 <To be defined>
SWCover-Ver-Act3 <To be defined>
SWCover-Ver-Act4 <To be defined>
SWCover-Ver-Act5 <To be defined>
SWCover-Ver-Act6 <To be defined>

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 53

Safety Verification

5.4.4.2 Tool chain Verification

Verification of the identified list of tools that have been collected in the tool chain.

5.5 Verification Reporting

This section describes how the verification activities have to be documented. Verification
reporting will occur throughout the software life cycle. The content, format, and timing of all
verification reports shall be specified in this section.

The following kinds of reports will be generated during the verification process:

• Anomaly reports:

• Phase Summary Verification reports:

• Final report:

5.5.1 Structure of the Verification Report

The following sketches the structure of the verification report. It shall cover the following central
topics:

Header containing all information to identify, this report, the authors, the approbation and
reviewing entities.
Executive Summary giving an overview of the major elements from all sections.
Problem Statement describing the challenges to be answered by Verification & Validation as
well as the decisions to be taken based on the V&V results as well as how to cope with
potentially faulty output. It further describes the accreditation scope based on the risk
assessment done on V&V-level.
V&V Requirements Traceability Matrix links every V&V artifact back to the requirements
to measure e.g. test coverage and to directly link V&V results to the requirements.
Acceptability Criteria, describing the criteria for acceptance of the artifact into the Verification
& Validation process e.g. as the direct translation of the requirements into metrics to measure
success, are used e.g. for burndown charts within the process.
Assumptions that are identified during the design of the verification and validation strategy and
how these assumptions have an impact on the verdict by listing capabilities and limitations.
Risks and Impacts that come across the execution of V&V tasks together with the impacts
foreseen.
V&V Design states how the V&V process builds up including data preparation, execution and
evaluation.
V&V Methodologies giving a step-by-step walkthrough of all possible V&V activities including
the assumptions, and verdict-relevant limitations and criteria for, e.g., model verification,

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 54

model-to-code verification, unit testing, integration testing and final validation (according to
the standard, this involves running the software on the target hardware).
V&V Issues describing unsolved V&V issues and their impact on the affected proof or verdict.
Peer Reviews going into details on how the community can take part and how official bodies
and partners are integrated into the development and review process.
Test Plan Definition going into the details of testing by describing among other things:

Title as a unique identifier to the test plan.

Description of the test and the test-item giving information about version and revision.
Features to be tested and not to be tested in combination are listed together with information
background.
Entry Criteria which have to be met by the EVC before a test can be started, e.g. that the
EVC has to be in level 3 limited supervision with the order to switch to level 2.
Suspension criteria and resumption requirements are the central key to a smooth automa-
tion of the tests covering topics like when exiting this test before step 10, which entry
criteria does it comply to or which resumption sequence has to be executed to continue
testing.
Walkthrough covering a step-by-step approach of the test plan.
Environmental requirements going into the details of what is needed concerning the test
environment, e.g. tools, adapter, data preparation.

Discrepancy Reports identifying the defects.

Key Participants describing the assignment and task for each role involved.

Accreditation of Participants describing who was accredited to which role during the
Verification & Validation phase.
V&V Participants listing the partners participating in V&V activities,
Other participants including other interest groups such as reviewer by affiliate partners7 .

Timeline giving the timeline for the baselines as input to the V&V process and identifying when
each artifact should be created.

An example of a summary table of verification activities performed is given in Table 11. Such a
table should enter the verification report.

Table 11. SW Verification Activities Breakdown

Activity Code Activity Responsibility Status/Link

Phase 0: SW Requirements Verification
SWReq-Ver-
Compliance with SSRS verification
Act1
SWReq-Ver- Accuracy, Consistency, Completeness,
Act2 Correctness assurance
Continued on next page
7
affiliate partners are non-funded companies who signed the project cooperation agreement and with it get read
access to the repositories starting from incubation phase to contribute e.g. by reviewing

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 55

Table 11 – continued from previous page

Activity Code Activity Estimated Delivery Real Delivery
SWReq-Ver-
Testability assurance
Act3
SWReq-Ver-
Verificability assurance
Act4
SWReq-Ver-
Compliance with Standards
Act5
SWReq-Ver-
Traceability with SSRS verification
Act6
SWReq-Ver-
Requirements modelling correctness
Act7
Phase 1: SW Architecture and Design Verification
SWArch-
software architecture design verification
Ver-Act1
SWArch-
Handle attributes verification
Ver-Act2
SWArch-
incompatibilities checking
Ver-Act3
SWArch-
Model coherency verification
Ver-Act4
SWArch-
Constraints analysis verification
Ver-Act5
SWArch-
Traceability verification
Ver-Act6
SWArch-
Compliance with Standards verification
Ver-Act7
Phase 2: SW Component Design

Also a summary of the tool verification activities is to be included, as indicated in Table 12.

Table 12. Tool Chain Verification Summary

Activity Code Activity Estimated Delivery Real Delivery

Phase 0: Tool chain Requirements

Phase 1:

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 56

5.6 Administrative Procedures

This section identifies the existing administrative procedures that are to be implemented as part
of the Verification Plan. Verification efforts consist of both management and technical tasks.
Furthermore, it is the task of the SQA team to monitor whether the procedures as defined in the
management plans ([1], [SCMP], [Review and Revision processes]) are followed.

5.6.1 Problem Report

The problem reporting procedure is described within the document Change/Problem Management
Process.

Any problem, failure and error encountered during the review activities (QA. Verification,
Validation, Assessment) planned in the software development life-cycle, problems reported by
users and customers as well as change requests initiated by any of the system stakeholders
will be reported and managed following the Change/Problem Management Process detailed in
[governance] and through the Change/Problem Management Tool.

5.6.1.1 Task Iteration Process

Any change in the requirements (system, sub-systems, sw or components) require repeated

verification and validation activities.

Once the change is accepted following the change/problem management procedure, the phases
and items affected by it must be evaluated. These tests will be redesigned to reflect the change in
the requirement and will be executed again.

In turn, a new analysis of the Software Integrity Level will involve the analysis of the activities
requirements and documentation presented by the EN50128 standard and include such activities
in the SVVP if necessary.

5.6.1.2 Deviation Process

The Quality Manager will be informed in the case of detection of a deviation regarding Verification
Plan. In addition, he/she also be informed if it is deemed necessary by an amendment to the Plan,
whether or not motivated by a deviation

The Quality Manager will report such incidents to the Project Managers and with whom shall act
appropriately. All persons listed in the responsibilities section (Sec. 5.1.5) shall be informed of a
change in the Verification Plan

5.6.1.3 Control Procedure

Control procedures are specified in the Configuration Management Plan [SCMP]

5.7 Validation for a Full Development

For each of the validation steps identified in the plan overview, the following has to be instantiated:

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 57

5.7.1 DAS2V Validation

5.7.1.1 Task

5.7.1.2 Documents to Be Produced

5.7.1.3 Phase Specific Activities

5.7.1.4 Techniques and Measures

5.7.2 SFM Validation (3d)

5.7.2.1 Task

The formalisation of the requirements in form of a semi-formal model enables a systematic check
of the completeness and consistency of the system test specification.

The model itself can perhaps be animated (depending on the concrete form which is not yet
fixed). This offers the chance to an early (preliminary) validation of the design.

5.7.2.2 Documents to Be Produced

1. Revised System Test Specification

2. SFM validation report

5.7.2.3 Phase Specific Activities

5.7.2.4 Techniques and Measures

5.7.3 Final Validation (tbd)

5.7.3.1 Task

The final validation shall ascertain that the end result of the development—the EVC software in
its specified environment—behaves as required.

5.7.3.2 Documents to Be Produced

1. System Test Definition (based on System Test Specification)

2. System Validation Report

5.7.3.3 Phase Specific Activities

Testing the software against the user requirements.

5.7.3.4 Techniques and Measures

A technique to be considered is testing in a validated testbed (including API animation/simulation)

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 58

5.8 Implementation of Verification & Validation

The verification & validation has to be performed in cooperation with WP 3, which produces
DAS2Vs (models and code), and with WP 7, where methods and tools are defined and developed.

To exchange information with WP 3, formats are needed for collecting information about DAS2Vs
(V&V tasks) and for giving back information about the results of V&V activities. Similarly, with
WP 7 communication shall use formats to describe V&V methods and tools (input from WP 7)
and the results of evaluations of V&V methods and tools.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 59

6 Verification & Validation Plan for openETCS

Besides the usual purpose of verification & validation activities, namely evaluating and proving
the suitability of design artifacts, V&V in openETCS will also generate information on the
suitability of the methods and tools employed. For that purpose, a format for describing methods
and tools to be used in V&V and one for summarizing the findings about the suitability are
defined.

The plan also contains partial instantiations of V&V which match partial developments that are
realised within openETCS.

• verification & validation for partial developments

• evaluation
• demonstration story of capabilities

6.1 Verification Plan for openETCS

6.1.1 Verification Overview

Describe the organization, schedule, resources, responsibilities, tools, techniques, and method-
ologies to be deployed in order to perform the verification activities.

6.1.1.1 Organisation

Define the relationship of verification to other efforts such as development, project management,
quality assurance, and configuration management. Define the lines of communication within the
verification effort, the authority for resolving issues, and the authority for approving verification
deliverables.

Organisation: a format for describing design artifacts subject to V&V, and a feedback format for
the findings during V&V.

6.1.1.2 Schedule

The schedule summarizes the various verification tasks and their relationship to the overall
openETCS project. It describes the project life cycle and project milestones including completion
dates. Summarize the schedule of verification tasks and how verification results provide feedback
to the whole openETCS process to support overall project management functions. The objective
of this section is to define an orderly flow of material between project activities and verification
tasks.

According to the Description of Work of WP 4 [65], the verification activities will be structured
into three main phases:

1. First Level: Verification of prototypical system and API model and prototypical code,

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 60

2. Second Level: Verification of system model, functional API prototype model, code architec-
ture and system API prototype

3. Third Level: Verification of final system and API model, final code and the functional API
model

6.1.2 Verification Activities—User Sories

The term “User Story” as used in openETCS stands for any kind application of tools, not just for
the end user application of the system (EVC software) which is to be developed. This section
shall describe such “user stories” of verifiers, i.e., it shall describe where which method or tool
is to going to be applied to what artifact(s) (DAS2Vs). It thus shall tell coherently the story of
verification activities. Later (sub)sections provide the organisational detail: When (Timeline,
Sec. 6.1.3) and Contribution (which of the verification obligations from Sec. 5.4 are tackled by
what approach.

6.1.2.1 Reviews and Inspections

Reviews

In the openETCS project all written documents, specifications, models and code can be reviewed.
It is also important to include all documents concerned with the creation and delivery of the
openETCS product. This means that strategies, plans, approaches, operation and maintenance
manual, user guides, the contract that will initiate the work should all be reviewed in a structured
way.

Information about the reviews planned within openETCS is given in the QA plan [1].

Inspections

Inspection shall be applied to design artifacts whose correctness is important for the demonstration
of the project results, and also to validate results of innovative verification & validation methods
and/or tools.

6.1.2.2 Software Architecture Analysis Method (SAAM)

Potential targets for SAAM are parts and instances the SFM (semi-formal model) of the software,
to ascertain the viability of the software architecture decision. The scenarios with which the
analysis shall be performed are to be developed from the operator requirements.

6.1.2.3 Architecture Tradeoff Analysis Method (ATAM)

ATAM may be applied to the SW architecture.

6.1.2.4 Formal Verification at Software Level

This section describes CEA LIST’s and Fraunhofer FOKUS’ plans regarding the use of formal
methods to assess properties at the C code level. Section 5.3.9.1 above describes the main
plug-ins of the Frama-C tool suite that are envisaged for that, while the theoretical background
is summarized in sections 5.3.7.1 and 5.3.7.2. Namely, two main categories of properties can
be dealt with. First, we can focus on functional properties, that is establishing that a given

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 61

function is conforming to its (formal) specification. Second, it is also possible to analyze a whole
application to check the absence of potential run-time errors (arithmetic overflows, division by 0,
invalid dereference of pointers, buffer overflow, use of uninitialized variables, undefined order
of evaluation, ...). A case study partly based on previous experiments is developed further in
OpenETCS and has been presented in [66]. Existing code from OpenETCS partners, namely
Siemens and ERSA has also been identified has a good target for such activities.

6.1.2.5 Formal Model Verification (TWT)

This section describes TWT’s plans regarding the formal verification of the system model. While
TWT’s original approach has been focused primarily on real-time aspects by employing timed
automata and the UPPAAL tool as described in Section 5.3.9.6 (page 28) our activities are now
addressing formal model verification on a broader scale.

TWT’s work will be based on the following action items:

1. We will investigate the suitability of the open source toolkit CPN Tools[67] for formal
verfication (model checking) and simulation. The evaluation of CPN Tools will be aligned
with the secondary toolchain assessment in WP7. A first step in the tool evaluation is the
construction of an example model (currently “Start of Mission”) from the ETCS specification.

2. We will analyze whether/how UML/SysML statecharts and possibly activity diagrams can
be transformed to colored Petri nets to permit their simulation and the formal verification of
requirements.

3. We will analyse the expressive power of the CPN Tools property language and whether the
modeling formalism can replace UPPAAL for modelling and verifying timed aspects of
ETCS components.

4. We will provide feedback regarding ambiguities, inconsistencies and errors in the current
ETCS standard based on our formalisation and the analysis of other models based on our
approach.

5. As the University of Braunschweig has experience with formal Petri net modelling, we will
align our efforts with them.

6. We will investigate, how our approach can be integrated in the Eclipse environment and
develop such integration if feasible with our efforts.

7. We plan to publish our results on action item 1.

Please note that the above list may be subject to future change.

6.1.2.6 Verification with Model-Based Simulation using SystemC (TWT, URO)

In Section 5.3.10 (page 30) the basics of SystemC and the SysML/SystemC joint approach have
been described. TWT and the University of Rostock (URO) will work on this concept based on
the following action items:

1. TWT will analyse methods for generating SystemC code from SysML models. In first
investigations the Acceleo tool (Eclipse plugin, based on OMG standard) seems to be a
promising candidate for model-to-text transformation.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 62

2. URO will build a modular and executable SystemC model for braking curves that is suitable
for real-time simulation. An accompanying high-level SysML model will be constructed as
well and will be means to test the transformation methods to be developed in the context of
action item 1.

3. TWT and URO plan to investigate which other parts of the ETCS specification can benefit
from real-time simulation and build models accordingly.

4. URO will investigate whether performance analysis based on the underlying hardware system
is feasible within the openETCS project. This will allow to scale the hardware resources of
the OBU system accordingly.

5. In addition, using the results from action item 1, TWT and URO plan to transform existing
SysML models (to be developed in WP3) to SystemC for real-time simulation.

6. Evaluation (and possibly implementation) of an Eclipse integration

Please note that the above list may be subject to future change.

6.1.2.7 System Integration Testing (Uni Bremen/DLR)

Section 5.3.5 describes the main objective of system integration testing. Uni Bremen will focus
on the following items :

1. Create a test model in SysML. From the model evaluation activities, the management of the
radio communication is already available. To cover different aspect of the specification, the
ceiling speed monitoring model will be also provided.

2. Generate test cases according to the defined interface given by DLR. (RT-Tester)

3. Provide simulation environment for the track-to-train simulation (including braking curves/speed
profiles)8 along routes used for testing

4. Study the automatic generation of these track layouts and speed simulations; if feasible,
implement a generator and integrated it in the DLR laboratory environment

5. Set up a test environments for

• Hardware-in-the-loop Testing within DLR laboratory.

• Software-in-loop testing with code provided by SCADE (Siemens)

Track simulation

OpenETCS system testing for the EVC on-board computer requires test execution in real physical
time and also track layout with realistic speed profiles. We also want to contribute to the track
and Speed Simulation by automatically generates “relevant” layout for OpenETCS.
8
OpenETCS system testing for the EVC on-board computer requires test execution in real physical time and
also track layout with realistic speed profiles. We also want to contribute to the track and Speed Simulation by
automatically generates “relevant” layout for OpenETCS.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 63

Test cases generation

We also plan different activities to ensure the pertinence of our test cases.

1. Check the test model (SysML) : RTT-BMC or other model checkers

2. Add relevant LTL properties if needed

3. Test case analysis by

• Structural coverage
• Requirement coverage
• Mutation coverage
• Data coverage

Test cases analysis – comparison to Subset 76

1. Provide techniques and Howto describing how test cases from Subset 76 can be executed in
the RT-Tester environment, either as SW integration test or as HiL test in the DLR simulation
environment

2. Create new set of test cases for the ceiling speed monitoring (As far as we know,they do not
yet exist in Subset 76)

3. Compare new test cases created by RT-Tester to new test cases for ceiling speed monitoring
provided by ERTMS standardization group, as soon as available; suggest improvements for
the Subset 76 test cases.

Exchange Formats

Test models represented in XMI/Ecore are used as SysML test modeling standard. RT-Tester
model parsers are extended to cope with this format.

Test procedures will be represented in a general abstract syntax format, so that procedures
generated by RT-tester can be run on any test execution platform.

Test results (test execution logs) will be represented in a general format, so that exchange of test
results between tools (for example, for simulation purposes) becomes possible.

A first part of these activities have already been done for model of radio management communi-
cation (see the model-evaluation for EA/RT-tester).

6.1.2.8 Model Verification by applying the openETCS Verification Tool Chain (Siemens)

Siemens will focus the activities on the verification of its contributions to the openETCS work
packages 3 and 7. Currently available is the MoRC model provided for the primary tool chain
for WP 7. This modeling will be continued for WP 3. The existing formal model will be
complemented by a semi-formal model on top. Both, the semi-formal as well as the formal
model need to be verified in WP 4 and will serve as verification objects.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 64

The intention is to apply the preferred and most valuable methods and parts of the openETCS
verification tool chain to the models and by giving feedback into the openETCS project contribute
to increase their useability and adequacy.

In more detail, these activities are planned:

• Support the integration of the MoRC model into the openETCS test environment / DLR
laboratory.

• Support the integration of the MoRC model into the RT-Tester environment, provided by Uni
Bremen.

• Model based test case generation and execution by using the RT-Tester environment provided
by Uni Bremen.

• Determination of the structural test coverage on the model.

• Determination of the requirements coverage regarding subset-026 documents, SSRS, imple-

mentation model, test model, test case and test execution issues.

• Determine the feasibility of proving safety properties for the model.

6.1.3 Verification Activities—Timeline

This section lists per partner/activity in which of the project phases according to Sec. 6.1.3 (first,
second or third level of verification) a particular activity is planned. In the first version of the
V&V plan, only the first level needs to be detailed.

6.1.3.1 First Level of Verification

This section gives only a short description, which should refer to an activity detailed earlier. Most
probably the material will be organised in a table.

TWT

TWT will continue with the analysis of methods for generating SystemC code from SysML
models. In first investigations the Acceleo tool (Eclipse plugin, based on OMG standard) seems
to be a promising candidate for model-to-text transformation. The approach will be aligned with
URO and their SystemC model of the braking curves. See also Sct. 6.1.2.6.

In addition, TWT will start with the analysis on whether/how UML/SysML statecharts can
be transformed to timed automata in a sensible manner while retaining as much structural
information as possible. Timed automata provide a means for model checking real-time properties
of systems. See also Sct. 6.1.2.5.

URO

URO will continue their work on a modular and executable SystemC model for braking curves
suitable for real-time simulation. An accompanying high-level SysML model will be constructed
as well and aligned with TWT’s activities on SysML → SystemC code generation. See also
Sct. 6.1.2.6.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 65

Uni. Bremen

Uni Bremen will start by developing and setting a simulation environment for the test of the EVC.
The test models from the model-evaluation will be completed and the breaking curves will be
added. The set of tests generated will be then compared to the available ones of SUBSET-076.

6.1.3.2 Second Level of Verification

6.1.3.3 Third Level of Verification

6.1.4 Verification Activities—Process View

This section provides the detailed plan for the verification tasks throughout the openETCS project
life cycle. It summarizes the activities performed by the project partners in relating them to the
overall definition of verification activities in Sec. 5.4.

6.1.5 Verification Reporting

This section describes how the results of implementing the Verification Plan will be documented.
Verification reporting will occur throughout the software life cycle. The content, format, and
timing of all verification reports shall be specified in this section.

The following reports will be generated during the verification process:

• Anomaly reports:

• Phase Summary Verification reports:

• Final report:

The structure of the Verification report is already defined in the 5.5.1 section of this document

6.1.6 Administrative procedures

This section identifies the existing administrative procedures that are to be implemented as part
of the Verification Plan. Verification efforts consist of both management and technical tasks.
Furthermore, it is the task of the SQA team to monitor whether the procedures as defined in the
management plans ([QAplan], [SCMP], [Review and Revision processes]) are followed.

6.1.6.1 Problem Report

The problem reporting procedure is described within the document Change/Problem Management
Process.

Any problem, failure and error encountered during the review activities (QA. Verification,
Validation, Assessment) planned in the software development life-cycle, problems reported by
users and customers as well as change requests initiated by any of the system stakeholders
will be reported and managed following the Change/Problem Management Process detailed in
[governance] and through the Change/Problem Management Tool.

6.1.6.2 Task Iteration Process

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 66

Any change in the requirements (system, sub-systems, sw or components) require repeated

verification and validation activities.

Once the change is accepted following the change/problem management procedure, the phases
and items affected by it must be evaluated. These tests will be redesigned to reflect the change in
the requirement and will be executed again.

In turn, a new analysis of the Software Integrity Level will involve the analysis of the activities
requirements and documentation presented by the EN50128 standard and include such activities
in the SVVP if necessary.

6.1.6.3 Deviation Process

The Quality Manager will be informed in the case of detection of a deviation regarding Verification
Plan. In addition, he/she also be informed if it is deemed necessary by an amendment to the Plan,
whether or not motivated by a deviation

The Quality Manager will report such incidents to the Project Managers and with whom shall act
appropriately. All persons listed in the Responsibilities section (Sec. 5.1.5) shall be informed of
a change in the Verification Plan

6.1.6.4 Control Procedure

Control procedures are specified in the Configuration Management Plan [SCMP]

6.2 Validation Plan for openETCS

6.2.1 Validation Overview

This section describes the validation activities which are planned to be performed within the
project.

The timeline of validation activities parallels that of the verfication. I.e., there are the same three
levels as in Sec. 6.1.1.

6.2.2 Validation Activities—User Sories

Like Sec. 6.1.2, this section describes the validation activities to be performed within the project.
It shall tell coherently the content (method, tool, result quality) of validation activities. Later
(sub)sections provide the organisational detail: When (Timeline, Sec. 6.2.3) and Contribution
(which of the validation obligations from Sec. 5.7 are tackled by what approach.

6.2.3 Validation Activities—Timeline

This section lists per partner/activity in which of the project phases according to Sec. 6.2.3 (first,
second or third level of validation) a particular activity is planned. In the first version of the V&V
plan, only the first level needs to be detailed.

6.2.3.1 First Level of Validation

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 67

A short description which may refer to an activity detailed earlier. Most probably the material
will be organised in a table.

6.2.3.2 Second Level of Validation

6.2.3.3 Third Level of Validation

6.2.4 Validation Activities—Process View

This section provides the detailed plan for the validation tasks throughout the openETCS project
life cycle. It summarizes the activities performed by the project partners in relating them to the
overall definition of validation activities in Sec. 5.7.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 68

Appendix A: Requirements on Verification &

Validation
• Requirements from D2.9.

• Take the lists from the draft from 121207, retain the structure (at least preliminarily).

A.1 Requirements on Verification & Validation from D2.9

The already provided requirements require a safety plan compliant to the CENELEC EN 50126,
50128 and 50129. This pulls a number of requirements on V&V, including Verification and
Validation plans. On the topic of compliance to EN 50128, one shall also refer to the D2.2
document.

R-WP2/D2.6-02-061 A Verification plan shall be issued and complied with.

R-WP2/D2.6-02-061.01 The verification plan shall provide a method to demonstrate the re-
quirements covering all the development artifacts.

R-WP2/D2.6-02-061.02 The verification plan shall state all verification activities required for
each of these development artifacts.

R-WP2/D2.6-02-062 A Validation Plan shall be issued and complied with.

R-WP2/D2.6-02-062.01 The validation plan shall provide a method to validate all functional
and safety requirements over all development artifacts.

R-WP2/D2.6-02-062.02 The validation plan shall state all validation activities required for
each of these development artifacts.

R-WP2/D2.6-01-021 The test plan shall comply the mandatory documents of the SUBSET-076,
restricted to the scope of the OpenETCS project.

Justification. It will possibly be difficult to model all the tests in the course of the project,
but the test plan should at least be complete.

R-WP2/D2.6-02-063 Each design artifact needs a reference artifact which it implements (e.g.
code to detailed model, SFM to SSRS model. . . )

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 69

R-WP2/D2.6-02-063.01 The implementation between them relation shall be specified in detail.

e.g. for state machine and a higher level state machine mapping of interfaces, states and transition
is required. This includes additional invariants, input assumptions and further restrictions. This
informaiton is the basis for verification activities.

R-WP2/D2.6-02-063.02 The design of the artifacts shall be made such to allow verifiability as
far as possible.

R-WP2/D2.6-02-064 The findings from the verification shall be traced, and will be adequately
addressed (taken into consideration, or postponed or discarded with a justification).

A.2 General Requirements on Verification

Excerpt from Requirement Project Relevance

EN 50128:2011
[N01]
5.3.2.7 For each document, traceability shall be pro- fully applicable
vided in terms of a unique reference number
and a defined and documented relationship
with other documents.
5.3.2.8 Each term, acronym or abbreviation shall
have the same meaning in every document.
If, for historical reasons, this is not possible,
the different meanings shall be listed and the
references given.
5.3.2.9 Except for documents relating to pre-existing
software (see 7.3.4.7), each document shall
be written according to the following rules:
• it shall contain or implement all applica-
ble conditions and requirements of the
preceding document with which it has a
hierarchical relationship;
• it shall not contradict the preceding doc-
ument.

5.3.2.10 Each item or concept shall be referred to by

the same name or description in every docu-
ment.
6.5.4.14 Traceability to requirements shall be an im-
portant consideration in the validation of a
safety-related system and means shall be
provided to allow this to be demonstrated
throughout all phases of the lifecycle.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 70

Excerpt from Requirement Project Relevance

EN 50128:2011
[N01]
6.5.4.15 Within the context of this European Standard,
and to a degree appropriate to the speci-
fied software safety integrity level, traceability
shall particularly address
a) traceability of requirements to the design
or other objects which fulfil them,
b) traceability of design objects to the imple-
mentation objects which instantiate them.
c) traceability of requirements and design
objects to the tests (component, integra-
tion, overall test) and analyses that verify
them.
Traceability shall be the subject of configura-
tion management.
6.5.4.16 In special cases, e.g. pre-existing software This requirement does not apply to the
or prototyped software, traceability may be project.
established after the implementation and/or
documentation of the code, but prior to verifi-
cation/validation. In these cases, it shall be
shown that verification/validation is as effec-
tive as it would have been with traceability
over all phases.
6.5.4.17 Objects of requirements, design or implemen-
tation that cannot be adequately traced shall
be demonstrated to have no bearing upon
the safety or integrity of the system.

Excerpt from Requirement

EN 50128:2011
[N01]
6.1.4.1 Tests performed by other parties such as the Requirements Manager, Designer or Im-
plementer, if fully documented and complying with the following requirements, may be
accepted by the Verifier.
6.1.4.2 Measurement equipment used for testing shall be calibrated appropriately. Any tools,
hardware or software, used for testing shall be shown to be suitable for the purpose.
6.1.4.3 Software testing shall be documented by a Test Specification and a Test Report, as defined
in the following.
6.2.4.2 A Software Verification Plan shall be written, under the responsibility of the Verifier, on the
basis of the necessary documentation.
6.2.4.3 The Software Verification Plan shall describe the activities to be performed to ensure proper
verification and that particular design or other verification needs are suitably provided for
6.2.4.4 During development (and depending upon the size of the system) the plan may be sub-
divided into a number of child documents and be added to, as the detailed needs of
verification become clearer.
6.2.4.5 The Software Verification Plan shall document all the criteria, techniques and tools to be
used in the verification process. The Software Verification Plan shall include techniques
and measures chosen from Table A.5, Table A.6, Table A.7 and Table A.8. The selected
combination shall be justified as a set satisfying 4.8, 4.9 and 4.10
6.2.4.6 The Software Verification Plan shall describe the activities to be performed to ensure
correctness and consistency with respect to the input to that phase. These include
reviewing, testing and integration.
6.2.4.7 In each development phase it shall be shown that the functional, performance and safety
requirements are met.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 71

Excerpt from Requirement

EN 50128:2011
[N01]
6.2.4.8 The results of each verification shall be retained in a format defined or referenced in the
Software Verification Plan.
6.2.4.9 The Software Verification Plan shall address the following:
a) the selection of verification strategies and techniques (to avoid undue complexity in the
assessment of the verification and testing, preference shall be given to the selection of
techniques which are in themselves readily analysable);
b) selection of techniques from Table A.5, Table A.6, Table A.7 and Table A.8;
c) the selection and documentation of verification activities;
d) the evaluation of verification results gained;
e) the evaluation of the safety and robustness requirements;
f) the roles and responsibilities of the personnel involved in the verification process;
g) the degree of the functional based test coverage required to be achieved;
h) the structure and content of each verification step, especially for the Software Require-
ment Verification (7.2.4.22), Software Architecture and Design Verification (7.3.4.41,
7.3.4.42), Software Components Verification (7.4.4.13), Software Source Code Verifi-
cation (7.5.4.10) and Integration Verification (7.6.4.13) in a way that facilitates review
against the Software Verification Plan.

A.3 Glossary

API: Application Programming Interface. In the project, the API defines the interface of the
EVC software to the operating system and hardware. The exact nature of the API still needs
to be defined, whether it should be seen as a specification or as an implementation has yet to
be resolved.

ATAM: Architecture Tradeoff Analysis Method

DAS2V: Design Artifact Subject to Verification or Validation, e.g. some model or code fragment
which has to be verified against its specification.

EVC: European Vital Computer

FLOSS: Free/Libre/Open Source Software

FFM: Fully Formal Model. Sometimes called “Strictly Formal Model”. A model of a part of
the design which has a fully formal semantics and can thus be subjeced to rigorous analysis
methods from the domain of mathematical or computational logic.

HW: Hardware

SAAM: Software Architecture Analysis Method

SFM: Semi Formal Model. A model of some part of the design whose semantical interpretation
is either not fully fixed or is similar to that of a program. I.e., the interpretation might depend
on variations in the code generation or compilation, or it does not resolve “semantic variation
points” (UML).

SW: Software

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 72

Appendix: References
[1] I. de la Torre. Project quality assurance plan. openETCS Deliverables D1.3.1.

[2] CENELEC, European Committee for Electrotechnical Standardization. EN 50128: Railway ap-
plications - Communication, signalling and processing systems - Software for railway control and
protection systems, June 2011.

[3] RTCA SC-205. Software Considerations in Airborne Systems and Equipment Certification (DO-
178C). Radio Technical Commission for Aeronautics (RTCA Inc.), Washington/DC, Dec 2011.

[4] RTCA SC-205. Formal Methods Supplement to DO-178C and DO-278A (DO-333). Radio Technical
Commission for Aeronautics (RTCA Inc.), Washington/DC, Dec 2011.

[5] M. E. Fagan. Design and code inspections to reduce errors in program development. IBM Syst. J.,
38(2-3):258–287, 1999. Reprint from 1976.

[6] Rick Kazman, Gregory Abowd, Len Bass, and Paul Clements. Scenario-based analysis of software
architecture. IEEE Softw., 13(6):47–55, 1996.

[7] Carnegie Mellon University Software Engineering Institute. Architecture tradeoff analysis method.
http://www.sei.cmu.edu/architecture/tools/evaluate/atam.cfm.

[8] Jan Peleska, Elena Vorobev, and Florian Lapschies. Automated test case generation with smt-solving
and abstract interpretation. In Bobaru et al. [68], pages 298–312.

[9] Jan Peleska, Artur Honisch, Florian Lapschies, Helge Löding, Hermann Schmid, Peer Smuda, Elena
Vorobev, and Cornelia Zahlten. A real-world benchmark model for testing concurrent real-time
systems in the automotive domain. In Burkhart Wolff and Fatiha Zaidi, editors, Testing Software and
Systems. Proceedings of the 23rd IFIP WG 6.1 International Conference, ICTSS 2011, volume 7019
of LNCS, pages 146–161, Heidelberg Dordrecht London New York, November 2011. IFIP WG 6.1,
Springer.

[10] Helge Löding and Jan Peleska. Timed moore automata: test data generation and model checking.
In Proc. 3rd International Conference on Software Testing, Verification and Validation (ICST’10).
IEEE Computer Society, 2010.

[11] A. Pnueli, O. Shtrichman, and M. Siegel. The code validation tool CVT: Automatic verification of a
compilation process. International Journal on Software Tools for Technology Transfer, 2(2):192–201,
1998.

[12] J. Peleska, J. Feuser, and A. E. Haxthausen. Railway Safety, Reliability and Security: Technologies
and Systems Engineering, chapter The Model-Driven openETCS Paradigm for Secure, Safe and
Certifiable Train Control Systems, pages 22–52. In Flammini [69], 2012.

[13] Anne Elisabeth Haxthausen, Jan Peleska, and Sebastian Kinder. A formal approach for the construc-
tion and verification of railway control systems. Formal Asp. Comput., 23(2):191–219, 2011.

[14] Verified Systems International GmbH. Verified :: Products. http://www.verified.de/en/products.

[15] Jan Peleska, Elena Vorobev, and Florian Lapschies. Automated test case generation with smt-solving
and abstract interpretation. In Mihaela Bobaru, Klaus Havelund, GerardJ. Holzmann, and Rajeev
Joshi, editors, NASA Formal Methods, volume 6617 of Lecture Notes in Computer Science, pages

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 73

298–312. Springer Berlin Heidelberg, 2011.

[16] Jörg Brauer, Jan Peleska, and Uwe Schulze. Efficient and trustworthy tool qualification for model-
based testing tools. In Brian Nielsen and Carsten Weise, editors, Testing Software and Systems,
volume 7641 of Lecture Notes in Computer Science, pages 8–23. Springer Berlin Heidelberg, 2012.

[17] P. Cousot and R. Cousot. Static determination of dynamic properties of programs. In Proc. 2nd Int.
Symp. on Programming, pages 106–130, Paris, 1976. Dunot.

[18] Jean Souyris and David Delmas. Experimental Assessment of Astrée on Safety-Critical Avionics
Software. In Proc. Int. Conf. Computer Safety, Reliability, and Security, SAFECOMP 2007, volume
4680 of LNCS. Springer, September 2007.

[19] Bernhard Beckert and Claude Marché, editors. Formal Verification of Object-Oriented Software -
International Conference, FoVeOOS 2010, Paris, France, June 28-30, 2010, Revised Selected Papers,
volume 6528 of Lecture Notes in Computer Science. Springer, 2010.

[20] Dillon Pariente and Emmanuel Ledinot. Formal Verification of Industrial C Code using Frama-C: A
Case Study, pages 205–219. Volume 6528 of Beckert and Marché [19], 2010.

[21] Jean-Raymond Abrial. The B-book - assigning programs to meanings. Cambridge University Press,
2005.

[22] Jean-Raymond Abrial. Modeling in Event-B - System and Software Engineering. Cambridge
University Press, 2010.

[23] C. A. R. Hoare. An axiomatic basis for computer programming. Communications of the ACM,
12(10):576–580 and 583, October 1969.

[24] C.A.R. Hoare and Niklaus Wirth. An axiomatic definition of the programming language Pascal.
Acta Informatica, 2:335 – 355, 1973.

[25] Atelier B. http://www.atelierb.eu/.

[26] Event-B. http://www.event-b.org/.

[27] Prover Platform. http://why3.lri.fr.

[28] Coq Prover. http://coq.inria.fr.

[29] Isabelle Theorem Prover. http://www.cl.cam.ac.uk/research/hvg/isabelle.

[30] Edmund M. Clarke and Bernd-Holger Schlingloff. Model checking. In Robinson and Voronkov [31],
pages 1637–1790.

[31] Alan Robinson and Andrei Voronkov, editors. Handbook of Automated Reasoning. Elsevier, 2001.

[32] Armin Biere, Alessandro Cimatti, Edmund Clarke, and Yunshan Zhu. Symbolic model checking
without BDDs. In Proceedings of the 5th International Conference on Tools and Algorithms for
Construction and Analysis of Systems, TACAS’99, pages 193—207. Springer-Verlag, 1999.

[33] A Bradley. SAT-based model checking without unrolling. \ldots , Model Checking, and Abstract
Interpretation, 2011.

[34] K.L. McMillan. Interpolation and sat-based model checking. In Jr. Hunt, WarrenA. and Fabio
Somenzi, editors, Computer Aided Verification, volume 2725 of Lecture Notes in Computer Science,
pages 1–13. Springer Berlin Heidelberg, 2003.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 74

[35] List of free formal verification tools. http://gulliver.eu.org/free_software_for_formal_

verification.

[36] Pascal Cuoq, Florent Kirchner, Nikolai Kosmatov, Virgile Prevosto, Julien Signoles, and Boris
Yakobowski. Frama-C: a Software Analysis Perspective. In Proceedings of Software Engineering
and Formal Methods (SEFM), volume 7504 of LNCS, pages 233–247. Springer, 2012.

[37] Patrick Baudin, Jean-Christophe Filliâtre, Claude Marché, Benjamin Monate, Yannick Moy, and
Virgile Prevosto. ACSL: ANSI/ISO C Specification Language, 1.7 edition, April 2013. available at
http://frama-c.com/download/acsl.pdf.

[38] Frama-C: Source Code Analysis Suite. http://frama-c.com/.

[39] A. Bouajjani, C. Dragoi, C. Enea, and M. Sighireanu. On inter-procedural analysis of programs

with lists and data. In Proceedings of Programming Languages Design and Implementation (PLDI),
2011.

[40] Pascal Cuoq, Boris Yakobowski, and Virgile Prevosto. Frama-C’s value analysis plug-in. CEA
LIST, fluorine-20130601 edition, June 2013. available at http://frama-c.com/download/
frama-c-value-analysis.pdf.

[41] Theorem Prover. http://alt-ergo.lri.fr.

[42] Patrick Baudin, Loïc Correnson, and Zaynah Dargaye. WP Plugin. CEA LIST, 0.7 for fluorine-
20130601 edition, 2013. available at http://frama-c.com/download/frama-c-wp-manual.
pdf.

[43] Nicolas Stouls and Virgile Prevosto. Aoraï Plugin Tutorial. INSA Lyon and CEA LIST, 2013.
Available at http://frama-c.com/download/frama-c-aorai-manual.pdf.

[44] J.-C. King. A new approach to program testing. Proceedings of the international conference on
Reliable software, Los Angeles, California, 21-23:228–233, April 1975.

[45] L.-A. Clarke. A system to generate test data and symbolically execute programs. IEEE Transactions
on software engineering, 2(3):215–222, September 1976.

[46] C.-V. Ramamoorthy, S.-F. Ho, and W.-T. Chen. On the automated generation of program test data.
IEEE Transactions on software engineering, 2(4):293–300, September 1976.

[47] N. Rapin, C. Gaston, A. Lapitre, and J.-P. Gallois. Behavioral Unfolding of Formal Specifications
Based on Communicating automata. In Proceedings of the 1st workshop on Automated Technology
for Verification and Analysis (ATVA), 2003.

[48] C. Gaston, M. Aiguier, and P. Le Gall. Algebraic Treatment of Feature-oriented Systems. In

Language Constructs for Describing Features. Springer LNCS, 2000.

[49] J.P. Escobedo, C. Gaston, and P. Le Gall. Timed Conformance Testing for Orchestrated Service
Discovery. In Proceedings of the 8th International Symposium on Formal Aspects of Component
Software (FACS). Springer LNCS, 2011.

[50] B. Bannour, J.P. Escobedo, C. Gaston, and P. Le Gall. Off-line test case generation for timed
symbolic model-based conformance testing. In Proceedings of the 23rd International Conference on
Testing Software and Systems (ICTSS). Springer LNCS, 2012.

[51] B. Bannour, C. Gaston, and D. Servat. Eliciting unitary constraints from timed Sequence Diagram
with symbolic techniques: application to testing. In Proceedings of the 18th Asian-Pacific Software
Engineering Conference (APSEC). IEEE Computer Society, 2011.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 75

[52] J. Tretmans. Conformance Testing with Labelled Transition Systems: Implementation Relations and
Test Generation. Computer Networks and ISDN Systems, 29:49–79, 1996.

[53] M. Krichen and S. Tripakis. Black-box time systems. In Proc. of Int. SPIN Workshop Model
Checking of Software. Springer, 2004.

[54] C. Gaston, P. Le Gall, N. Rapin, and A. Touil. Symbolic execution techniques for test purpose
definition. In Proceedings of the 18th International Conference on Testing Communicating Systems
(TestCom). Springer LNCS, 2006.

[55] A. Faivre, C. Gaston, and P. Le Gall. Symbolic Model Based Testing for Component Oriented
Systems. In Proceedings of the 19th International Conference on Testing Communicating Systems
(TestCom/FATES). Springer LNCS, 2007.

[56] B. Bannour. Symbolic analysis of scenario based timed models for component based systems:
Compositionality results for testing. PhD thesis, CEA LIST / École Centrale Paris, 2012. http:
//www.cti.ecp.fr/~bannourb/PhDthesis.pdf.

[57] Microsoft’s Verifier for Concurrent C. http://research.microsoft.com/en-us/projects/

vcc.

[58] NuSMV Model Checker. http://nusmv.fbk.eu.

[59] Rajeev Alur and David L. Dill. A theory of timed automata. Theor. Comput. Sci., 126(2):183–235,
April 1994.

[60] Gerd Behrmann, Alexandre David, and Kim G. Larsen. A tutorial on uppaal. In Marco Bernardo and
Flavio Corradini, editors, Formal Methods for the Design of Real-Time Systems: 4th International
School on Formal Methods for the Design of Computer, Communication, and Software Systems,
SFM-RT 2004, number 3185 in LNCS, pages 200–236. Springer–Verlag, September 2004.

[61] Alexandre David, M.Oliver Möller, and Wang Yi. Formal verification of uml statecharts with real-
time extensions. In Ralf-Detlef Kutsche and Herbert Weber, editors, Fundamental Approaches to
Software Engineering, volume 2306 of Lecture Notes in Computer Science, pages 218–232. Springer,
2002.

[62] Alexander Knapp, Stephan Merz, and Christopher Rauh. Model checking - timed uml state machines
and collaborations. In Proceedings of the 7th International Symposium on Formal Techniques in
Real-Time and Fault-Tolerant Systems: Co-sponsored by IFIP WG 2.2, FTRTFT ’02, pages 395–416,
London, UK, UK, 2002. Springer.

[63] Karsten Diethers and Michaela Huhn. Vooduu: Verification of object-oriented designs using uppaal.
In Kurt Jensen and Andreas Podelski, editors, Tools and Algorithms for the Construction and Analysis
of Systems, volume 2988 of Lecture Notes in Computer Science, pages 139–143. Springer, 2004.

[64] Harald Fecher, Jens Schönborn, Marcel Kyas, and Willem-Paul de Roever. 29 new unclarities in
the semantics of UML 2.0 state machines. In Kung-Kiu Lau and Richard Banach, editors, Formal
Methods and Software Engineering, number 3785 in Lecture Notes in Computer Science, pages
52–65. Springer Berlin Heidelberg, January 2005.

[65] M. Behrens, H. Hungar, A. Cavalli, J. Gerlach, H. Manz, and C. Cornu. openETCS validation and
verification strategy work package: Description of work, May 2013.

[66] Jens Gerlach, Virgile Prevosto, Jochen Burghardt, Kerstin Hartig, Kim Völlinger, and Hans Pohl.
Formal Specification and Automated Verification of Railway Software with Frama-C. In IEEE
International Conference on Industrial Informatics (INDIN). IEEE Xplore, July 2013.

[67] CPN Tools. http://cpn-tools.org.

This work is licensed under the "openETCS Open License Terms" (oOLT).
OETCS/WP4/D4.1V01.00 76

[68] Mihaela Gheorghiu Bobaru, Klaus Havelund, Gerard J. Holzmann, and Rajeev Joshi, editors. NASA
Formal Methods - Third International Symposium, NFM 2011, Pasadena, CA, USA, April 18-20,
2011. Proceedings, volume 6617 of Lecture Notes in Computer Science. Springer, 2011.

[69] Francesco Flammini, editor. Railway Safety, Reliability and Security: Technologies and Systems
Engineering. Information Science Reference, 2012.

This work is licensed under the "openETCS Open License Terms" (oOLT).