A Unified Approach for Verification and Validation of Systems and Software

Engineering Models

Luay Alawneh† Mourad Debbabi† Fawzi Hassaı̈ne‡ Yosr Jarraya†

Andrei Soeanu†
†
Computer Security Laboratory, Concordia Institute for Information Systems Engineering,
Concordia University, Montreal, Canada.
‡
Future Forces Synthetic Environments Section,
Defence Research and Development Canada, Ottawa, Ontario, Canada.

Abstract Verification is defined as the confirmation by examina-

tion and provision of objective evidence that the specified
We present in this paper a unified paradigm for the veri- requirements have been fulfilled, whereas validation is de-
fication and validation of software and systems engineering fined as proving that the particular requirements for a spe-
design models expressed in UML 2.0 or SysML. This para- cific intended use are fulfilled [19].
digm relies on an established synergy between three salient The V&V phase can be a major bottleneck in the de-
approaches, which are model-checking, program analysis, velopment life cycle of any complex software or systems
and software engineering techniques. To illustrate the ac- engineering product since it may range from 50 to 80%
complishment of our results, we have designed and imple- of the total design effort [2]. In addition, designed prod-
mented an integrated and automated computer-aided as- ucts are required to meet a very high-level of reliability, se-
sessment tool. We provide three case studies for sequence, curity, and performance especially in safety-critical areas.
state machine, and class and package diagrams to demon- Therefore, ensuring that such systems meet their predefined
strate the benefits of our methodology. requirements and that they perform as expected is a chal-
lenging issue. However, in many modern engineering disci-
plines, conventional V&V methods such as testing and sim-
ulation have become less useful and are not always applica-
1. Motivations
ble. Conversely, using formal techniques complementary to
simulation provides a certain level of confidence since they
The continuous advancement in software and systems are rigorous and complete.
design led to the emergence of modern modeling languages, The research presented hereby, aims at providing a uni-
including the most prominent ones, namely UML [16] and fied paradigm for V&V in software and systems engineer-
SysML [17]. Both modeling languages are playing an im- ing fields. This effort is supported by the Collaborative
portant role in software and systems engineering. While Capability Definition, Engineering and Management (Cap-
software engineering (IEEE) [12] is defined as the appli- DEM) project which is an R&D initiative within the Cana-
cation of a systematic approach to the development, op- dian Department of National Defence1 . The latter aims
eration, and maintenance of software, systems engineer- to the development of a Systems-of-Systems engineering
ing (INCOSE) [7] is an interdisciplinary approach that en- process and relies heavily on modeling and simulation. The
ables the realization of successful systems focusing on the underlying models need to be subjected to formal V&V as
system as a whole. Ubiquitous systems such as ATMs or a complementary technique. In this context, our research
portable electronics as well as advanced technologies like
1 This research is the result of a fruitful collaboration between the Com-
aero-space, radars, defence or telecommunications are im-
puter Security Laboratory (CSL) at Concordia University and Defence Re-
portant application fields of systems engineering. In this
search and Development Canada (DRDC) at Ottawa. The research is sup-
context, the process of design and development mandates a ported by the CapDEM (Collaborative Capability Definition, Engineering
strong and sound Verification and Validation (V&V) phase. and Management) project.

Proceedings of the 13th Annual IEEE International Symposium and Workshop on Engineering of Computer Based Systems (ECBS’06)
0-7695-2546-6/06 $20.00 © 2006 IEEE
Authorized licensed use limited to: ISAE. Downloaded on April 02,2023 at 19:04:39 UTC from IEEE Xplore. Restrictions apply.
 provides the means to achieve such an objective. have been proposed. Van der Aalst in [23] used interval
The main contribution of this paper is to introduce a uni- timed colored Petri nets [22] as an ascribed semantics to ac-
fied paradigm for V&V in software and systems engineer- tivity diagrams. E. Börger et al. in [3] provided an Abstract
ing. It is based on an established synergy between three State Machine (ASM) semantics for activity diagrams. R.
major techniques: Formal verification, program analysis, Eshuis et al. [8] mapped activity diagrams to equivalent
and software engineering techniques. By formal verifica- activity hypergraphs by flattening their structure. Then, ac-
tion, we mean mainly model-checking. By program analy- tivity hypergraphs are mapped to a Clocked Labeled Kripke
sis, we mean flow analysis (data and control), type-based Structure (CLKS).
analysis, and abstract interpretation. By software engineer- In [5], the authors propose a rich trace-based seman-
ing, we point out those techniques that are used to measure tics for UML 2.0 interactions (sequence diagram). Harald
quality attributes in software such as metrics. Our key idea Störrle presents in [20] a partial order semantics for time
is to harmoniously combine these techniques in a synergetic constrained interaction diagrams.
way to achieve V&V of software and systems engineering Class and package diagrams have been investigated in
models. Furthermore, to the best of our knowledge, this is several research initiatives. In [1], metrics have been used
a pioneering endeavor in using these three techniques to- to measure the quality attributes of class and package di-
gether. In addition, the suggested approach is rigorous, for- agrams. These metrics could be classified into two cate-
mal and cost-effective since it is entirely automatic. gories. The first one deals with traditional metrics such as
The remainder of the paper is structured as follows. We cyclomatic complexity while the second, which is specif-
start by briefly surveying some related work. Then, Section ically related to object oriented systems, involves metrics
3 introduces our approach and shows how it can contribute such as coupling, depth of inheritance, and the number of
to high-quality, practical, and formal V&V of software and children. In [9], the authors illustrated the use of several
systems engineering design models. Thereafter, we present object oriented metrics to assess the complexity of class di-
the assessment of three case studies. Section 4 is dedicated agrams at the initial phases of the development life cycle.
to the assessment of a sample sequence diagram (interaction More recently, topics like validation by applying audits and
view). Section 5 contains the assessment of state machine metrics to UML models are addressed in [10]. Audits refer
diagram (behavioral view). In Section 6, we give some in- to conformance to standards while metrics are viewed as
sights into metrics and the usage thereof with a relevant ex- numerical measurements that allow the analysis of a model
ample of class and package diagrams. Since the case studies with respect to an already established scale indicating the
are only meant to depict the structure of some real-life sys- quality attributes of the design.
tems, we will not be concerned about the underlying engi-
neering discipline. Finally, we conclude with some remarks 3 Approach
and discuss future work in Section 7.
As previously stated, the main trait of our V&V para-
2 Related Work digm lies in the harmonious synergy between three well-
established techniques: Formal verification, software engi-
In this section, we survey the state of the art in terms neering techniques, and program analysis. The experiments
of V&V of software and systems engineering design mod- that we conducted demonstrate the viability of this approach
els. Particularly, we focus on UML 2.0 and SysML design as it will be exemplified later in this paper. In Figure 1, we
models. It is worthy to mention that only subsets of the depict the synoptic of our approach.
syntax and semantics specified by the considered standards The foundation of our paradigm is sustained by three
are covered in the related work. In the sequel, we present distinctive layers. First, model-checking, as a formal tech-
V&V of the most prominent UML diagrams namely state nique, can be fully automated and has been successfully
machine, activity, sequence, class and package diagrams. used in the verification of real applications (e.g. digital
Several approaches to the verification of UML state ma- circuits, communication protocols, etc). Hence, for each
chine diagrams have been advanced. D. Latella et al. [13] behavioral diagram, we derive a formal semantic model
as well as E. Mikk et al. [15] address the formal verifica- reflecting its characteristics and we express the properties
tion of a subset of UML state machine using SPIN, based that the design must satisfy as temporal logic formulas. A
on an operational semantics as described in [14]. The trans- widely used model-checker within the scientific commu-
lation process is done in two phases. First, the state ma- nity is SMV. It has interesting features like branching time
chine is converted into an Extended Hierarchical Automa- logic for expressing properties (CTL). We are currently us-
ton (EHA). Then, the latter is modeled in PROMELA and ing NuSMV [6] model-checker (a modified version of the
subjected to model-checking. original SMV) that has been used in the verification of hard-
For activity diagrams, several verification approaches ware and other systems modeled as FSMs (e.g. software).

Proceedings of the 13th Annual IEEE International Symposium and Workshop on Engineering of Computer Based Systems (ECBS’06)
0-7695-2546-6/06 $20.00 © 2006 IEEE
Authorized licensed use limited to: ISAE. Downloaded on April 02,2023 at 19:04:39 UTC from IEEE Xplore. Restrictions apply.
 The second layer consists of fifteen metrics adopted from provided as a counterexample for the unsatisfied properties.
software engineering field and applied directly on UML and Additionally, the graph provides a visual appraisal of the
SysML models. Thus, they can be used to assess quality at- diagram complexity with respect to the number of nodes
tributes of various models independently of their underlying and edges of the corresponding CTS. Furthermore, it can
discipline (software or systems engineering). For instance, also be used as a quick feedback when applying corrective
G.C. Tugwell et al. [21] outline the importance of metrics measures giving some insights about the resulting increase
in systems engineering especially those related to complex- or decrease in the diagram complexity.
ity measurement. Furthermore, the metric concept can also The NuSMV code generated for a given CTS will
be applied on the semantic model derived from different slightly vary depending on the type of the original diagram.
behavioral diagrams. Some examples include cyclomatic This is necessary in order to trace back the unsatisfied prop-
complexity, length of critical path, and others. Thus, the erties to the original diagram. In order to grasp the idea,
quality assessment of a given design can combine both the we give the following edifying example. Though one might
static and dynamic perspectives. initially think that it is pretty sound to consider that each
The third layer is represented by program analysis tech- configuration can represent a distinct entity in the NuSMV
niques such as flow analysis, type systems, and abstract in- model, one would quickly stumble on some shortcomings
terpretation. We advocate their use in verifying important when considering the case of a CTS derived from a state
model properties such as data dependencies, control de- machine diagram. The fact that the states within a state ma-
pendencies, invariants, anomalous behavior, reliability, and chine diagram share a hierarchical containment relation can
compliance to certain specifications. lead one to portray a scenario wherein a compound state is
never exited while allowing for transitions among its sub-
states. It follows that we have a state that when reached,
Formal
Language Requirements is never left, though the system is continuously progress-
ing. In such a case, one would be unable to specify a prop-
Architecture
erty that would detect such a problem without considering
Formalization Compilation
an independent entity for every state that might belong to
Design
different configurations of the corresponding CTS. Equiva-
Systems Engineering
lently, one can specify only the individual states as entities
Semantic
Properties
Models Models in the transition system and not the configurations them-
selves. This amounts to expressing the evolution of each
state by specifying all the conditions required for its ac-
Verification And Validation Module
Software
tivation or deactivation. Thus, even models exhibiting a
Formal Program Design
Analysis Analysis
Engineering Assessment wild dynamics and having complex CTS graphs, can be ef-
Techniques ficiently subjected to model-checking.
Accordingly, the core component responsible with the
Figure 1. Synoptic of the V&V Process. semantic model generation procedure represents an impor-
tant aspect of our framework. However, the main intent
of this paper is to present our results when applying the
proposed paradigm for V&V on different UML diagrams.
In the following, we briefly present some considerations
Thus, we will concentrate in the remainder of the paper in
with respect to the behavioral diagrams and their corre-
presenting some relevant case studies without detailing the
sponding semantic models. Any system that exhibits a dy-
theoretical aspects underlaying the procedure that is used
namic of some kind can be abstracted to one that evolves
for obtaining the corresponding configuration systems of
within a discrete state space. Such a system is able to
the behavioral diagrams.
evolve through its state space assuming different configura-
tions where a configuration is understood as the set of states
3.1 System Aspects and System Properties
wherein the system abides at any particular moment. All
the possible configurations summed up by the dynamics of
There are many systems engineering aspects that we tar-
the system and the transition thereof can be coalesced into
get: Requirements, time, concurrency, structure, interface,
a configuration transition system (CTS) or more simply put
control, and performance. In the sequel, we briefly present
a configuration system. The latter is characterized by a set
the main ones:
of configurations, a transition relation and a set of events.
Moreover, any CTS can be processed such that it can be • Requirements: They are a description of what a system
graphically visualized using an external graph drawing tool should do and are captured by requirement diagrams in
(e.g. daVinci [4]). The latter can be used to follow the trace SysML or using sequence and use case diagrams.

Proceedings of the 13th Annual IEEE International Symposium and Workshop on Engineering of Computer Based Systems (ECBS’06)
0-7695-2546-6/06 $20.00 © 2006 IEEE
Authorized licensed use limited to: ISAE. Downloaded on April 02,2023 at 19:04:39 UTC from IEEE Xplore. Restrictions apply.
 • Concurrency: It identifies how activities, events, and • Reusability: It measures the easiness and rapidity with
processes are composed (sequence, branching, alter- which a part (or more) of a system design and/or im-
native, parallel, etc.). It could be specified using se- plementation can be reused.
quence, activity, and timing diagrams.
• Coupling: It measures how strongly system parts de-
• Performance: It is the total effectiveness of the sys- pend on each other. Generally, a loose coupling is
tem. It makes reference to the timeliness aspects of sought in a high-quality design. Moreover, there is a
how systems behave including different types of qual- strong correlation between coupling and other quality
ity of service characteristics like latency and through- attributes (e.g. complexity, reusability, etc).
put. Timing and sequence diagrams depict perfor-
mance aspects. • Cohesion: It refers to the degree to which system com-
ponents are functionally related. Generally, a strong
• Structure: It is shown in class and composite structure cohesion is sought in a high-quality system design.
diagrams. The class diagram shows the relationships
between different classes of the system. The composite 3.2 V&V Framework
structure diagram shows the internal structure of the
building blocks of the system and how these blocks
Our V&V framework requires an underlying modeling
are interfacing with other components of the system.
tool wherefrom various models can be fetched and assessed.
Verification and validation contribute to the design as- We chose ARTiSAN Real-time Studio [18] which is a mod-
sessment by detecting the unsatisfied properties. Hence, eling tool that supports UML and SysML designs. Addi-
system developers will know if the design is flawed and ap- tionally, it provides component-based development specif-
ply corrective measures. We consider the following prop- ically for real-time systems. The current version of our
erties in our V&V framework: Latency, liveness, safety, framework is composed of three core components, as shown
deadlock, livelock, precedence, complexity, maintainabil- in Figure 2. First, we have the semantic compilation compo-
ity, reusability, coupling, and cohesion. Due to space limi- nent responsible for deriving the semantic model of a spe-
tations, we present the main ones, as follows: cific diagram. It communicates with the model-checker by
providing the semantic model along with the properties to
• Latency: It is the measure of the temporal delay be- be verified. Second, we have the metric computation com-
tween the request for the execution of an operation and ponent that is used for applying metric algorithms. We
the reply to this request. Detecting latency contributes have provided an interface that accesses the object repos-
to V&V by analyzing the efficiency of the system. itory of the modeling tool and retrieves the needed infor-
mation about the diagrams. Finally, the assessment results
• Liveness: It asserts that under certain conditions, a
component is devoted to the presentation of interpreted re-
given event will occur. It is known as “something
sults. Should a specified property fail, the trace provided by
good will always happen”. Liveness analysis consists
the model-checker is analyzed and the relevant information
of checking whether some important or crucial events
is provided as feedback.
may or may not eventually happen in the system.
• Safety: It means that nothing bad can occur with re-
spect to the design of the system. In other words, it is
a judgment of the acceptability of risk, which implies
that no harm will occur under the specified conditions.
• Deadlock: It describes a state wherein a process is
waiting for some event that will never happen. It could
be waiting for a resource that another process is hold-
ing indefinitely. Thus, the system would not progress. Figure 2. Architecture of the Framework.
• Reachability: It consists of checking whether a par-
ticular state is reachable in a design, starting from an
entry point. Unreachable states negatively impact the In the following paragraphs, we give a glimpse about
quality design since they denote dead entities. V&V with respect to class, package, and state machine dia-
grams. Subsequently, we detail the employed methodology
• Complexity: It is the quality of being intricate and for the V&V of the behavioral diagrams.
compounded, measuring the degree to which a design The quality of an object oriented system depends on
is difficult to be understood and/or to be implemented. different attributes such as complexity, understandability,

Proceedings of the 13th Annual IEEE International Symposium and Workshop on Engineering of Computer Based Systems (ECBS’06)
0-7695-2546-6/06 $20.00 © 2006 IEEE
Authorized licensed use limited to: ISAE. Downloaded on April 02,2023 at 19:04:39 UTC from IEEE Xplore. Restrictions apply.
 maintainability, stability, and others. According to the type
of diagram, we have a class of metrics for structural dia-
grams and another for behavioral ones. In the literature,
many metrics were developed to measure the quality of
software systems, especially for structural diagrams namely
class and package ones. However, until now, they were ne-
glected in the verification and validation of systems engi-
neering designs. We adopted a set of fifteen metrics includ-
ing Coupling Between Object classes (CBO), Depth of In-
heritance Tree (DIT), and Instability (I). CBO measures the
interrelationship between objects. DIT measures the level
of a class in the class inheritance hierarchy. The instability
metric (I) measures the rate of instability of a package. A
package is unstable if it depends more on other packages
than they depend on it. With the respect to behavioral di-
agrams, we are currently in early stages of experimenting
with metrics like the cyclomatic complexity and the length
of critical path. However, nominal ranges can not be ab-
solute but are tailored to the intended system’s characteris-
tics such as size, specialization, and redundancy.
Concerning the behavioral diagrams, we addressed the
V&V of sequence, activity, and state machine diagrams.
We will illustrate our methodology with respect to verify-
ing behavioral diagrams in the case study sections for se- Figure 3. Sequence Diagram Example.
quence and state machine diagrams. After converting each
diagram into its corresponding semantic model, we can au-
tomatically specify CTL properties for deadlock and reach-
the analysis of the state machine case study.
ability. Manual specification of properties can also be spec-
ified by using macros, with intuitive names (e.g. ALWAYS,
MAYREACH etc.), that are systematically expanded to cor- 4 Sequence Diagram Case Study
responding CTL properties. Thus, the designers can easily
express properties without being required to know formal Figure 3 shows a sample sequence diagram that we use
logics or temporal formulas. as case study. The diagram encompasses three actors inter-
The sequence diagram depicts object interactions acting in order to accomplish an arbitrated request/delivery
arranged in a time sequence. Furthermore, it can be used service. The actors are the requestor, the dispatcher, and the
to capture attributes such as latency and precedence. Using handler. The envisioned interaction is as follows: the re-
sequence diagrams, one can check whether a system meets questor (A) is sending a request to the dispatcher (D). After
some specified properties by techniques such as reachability receiving the request, D is acknowledging A while dispatch-
analysis and model-checking. It follows that by extracting ing at the same time (in parallel) the request to the handler
all possible execution paths of a given sequence diagram, (H). Subsequently, we have the parallel composition of two
we can construct a corresponding transition system that can groups of messages. The first group is composed of an al-
be analyzed by a model-checker. ternative between two cases: in the first case, A is sending
State machines can be used to specify the behavior of itself a wait message; in the second case, A is sending to
the various elements that are going to be modeled. A state D a cancel message, followed by a canceling message of
machine is a specification that thoroughly describes all pos- the dispatched request from D to H. The second group of
sible behaviors of some dynamic model. We were initially messages is composed of an alternative between two other
inspired by Latella et al. [13], however, we preferred the cases: in the first case, H is sending to D an accept message
NuSMV model-checker (since it supports branching time). in parallel with a self processing message, followed by a
We also recently succeeded in efficiently mapping the state grant message that D is sending to A and a subsequent de-
machine diagram to a kind of Labeled Transition System livery message from H to A. In the second case, H is sending
in the form of CTS. This allowed us to unify the semantic a busy message to D, followed by the refuse message from
model of the behavioral diagrams. In the sequel, we give D to A.
some insights with respect to our methodology and present In order to assess the diagram, we convert it to a cor-

Proceedings of the 13th Annual IEEE International Symposium and Workshop on Engineering of Computer Based Systems (ECBS’06)
0-7695-2546-6/06 $20.00 © 2006 IEEE
Authorized licensed use limited to: ISAE. Downloaded on April 02,2023 at 19:04:39 UTC from IEEE Xplore. Restrictions apply.
 start

evt1

A_req_D

evt2

D_ack_A, D_disp_H

evt7 evt3 evt10 evt13

A_cancel_req_D, H_busy_D A_cancel_req_D, H_accept_D, H_process_req_H A_wait_A, H_accept_D, H_process_req_H A_wait_A, H_busy_D

evt8 evt4 evt11 evt14

D_cancel_H, D_refuse_A D_cancel_H, D_grant_A D_grant_A D_refuse_A

evt5 evt12
evt9 evt15
H_deliver_A

evt6

end

Figure 4. CTS of the Sequence Diagram.

responding semantic model which is a kind of a transition terexample for the third user specified property. The in-
system (configuration system) wherein the elements repre- terpreted result of the trace provided by the model-checker
sent sets (possibly singleton) of interaction messages. In consisted in the following path in the CTS:
order to construct it, we first explore all possible execution A req D; D ack A,D disp H;
paths of the diagram. The corresponding CTS is depicted in A cancel req D,H accept D,H process req H;
Figure 3. D cancel H,D grant A; H deliver A
For the sequence diagram, we are mainly interested in
the manual property specification as aside from deadlock The identified path contains a series of messages (semi-
and reachability, one can not infer automatically the re- colon separated) that are exchanged between the actors. For
quired properties to be verified. Thus, we will exemplify readability reasons, we have by convention each message
the usage of manual specification of properties. For the pre- label starts with the sender actor and terminates with the
sented sequence diagram, we present some relevant proper- receiving one. Also, whenever two or more messages are
ties that reflect the intended interactions. We have the fol- being sent in parallel (thus part of the same configuration in
lowing properties presented in both simplified macro nota- the transition system), a comma is separating them.
tion along with their corresponding CTL notation.
The first one asserts that it is always the case that if A is 5 State Machine Diagram Case Study
sending a request to D, then there should be case where a
corresponding grant will be sent from D to A:
In this section, we present an interesting example that
ALWAYS A req D → MAYREACH D grant A
represents an abstracted model for an Automated Teller Ma-
CTL: SPEC AG((A req D → E[!(end) U D grant A]))
chine (ATM) system. As depicted in Figure 5, the model is
The second one asserts that it is always the case that if D based on a hypothetical behavior and is meant only as an
is sending a grant message to A, then it should be possible example that outlines the usefulness of our proposed V&V
for H to deliver to A at the next step: paradigm. The diagram has several states that are going to
ALWAYS D grant A → POSSIB H deliver A be presented in accordance to the diagram containment hi-
CTL: SPEC AG((D grant A → EX(H deliver A))) erarchy. The top state ATM is composed of four substates:
The third one is stating that it is always the case that if A idle, verif, eject, and operation. Initially, the
is canceling the request to D, then the system may not reach system waits for a potential user to use the ATM in the
the point where the handler H is sending to A: idle state. The verif state represents the verification of
ALWAYS A cancel req D → !MAYREACH( H deliver A) the card validness and authorization. The eject state de-
CTL: SPEC AG((A cancel req D → !(E[!(end) U picts the phase of termination of the user transaction. The
H deliver A]))) operation state is also a composite state that includes the
When subjecting the sequence diagram to our V&V tool, states that capture several functions related to banking oper-
we found that only the first two properties are satisfied. ations. These are the account, payment and transac.
However, the model-checker was able to produce a coun- The account state is where the account corresponding to

Proceedings of the 13th Annual IEEE International Symposium and Workshop on Engineering of Computer Based Systems (ECBS’06)
0-7695-2546-6/06 $20.00 © 2006 IEEE
Authorized licensed use limited to: ISAE. Downloaded on April 02,2023 at 19:04:39 UTC from IEEE Xplore. Restrictions apply.
 











idle

insert

verif

cardOk badCard

operation, account, info, balance, proc, selAccount eject

back

 back

Figure 5. ATM State Machine Diagram. operation, payment, cashAdv

next
select next
back

the card is selected and the user can receive information operation, transac, chkBal operation, payment, billPay

about balance and statement. The payment state has two insuf
balOk select
substates for cash advancing and bill payment respectively.
Finally, the transac state captures the transaction phase
operation, transac, modify operation, transac, debit
and includes three substates for checking the balance, mod-
ifying the amount (if necessary) and debiting the account.
When applying our paradigm to assess the presented Figure 6. CTS of the ATM.
state machine diagram, the steps that are being followed are
similar to those involved in assessing the sequence diagram.
That is, we first converted the diagram to its corresponding is taken as soon as there is a configuration containing the
semantic model which in this case is a CTS, as depicted account state (there is no trigger labeling the transition).
in Figure 6, wherein each element is represented by a set
In addition to the automatically generated properties, we
(possibly singleton) of states of the state machine diagram.
have the following manual specifications that represent an
Thereafter, we automatically specify deadlock and reacha-
intended behavior. We subsequently present both the macro
bility properties for every state. Furthermore, we also pro-
and the CTL notations.
vide some manual property specification in both macro and
The first property states that it is always the case that
CTL notations. After finishing the model-checking proce-
if verif is reached then it should be possible to reach
dure for the automatically generated properties, the results
operation at the next step:
that have been obtained pinpointed some interesting prob-
ALWAYS verif → POSSIB operation
lems in the ATM state machine design.
CTL: SPEC AG((verif → EX(operation)))
The model-checker determined that the operation
state exhibits deadlock, meaning that once entered, it is The second one asserts that it is always the case that
never left. This was found to be caused by the fact that if operation state is reached then from that point
in UML state machine diagrams, the transitions that have payment state should be reachable:
the same trigger are given higher priority when the source ALWAYS operation → MAYREACH payment
state is deeper in the containment hierarchy. Moreover, the CTL: SPEC AG((operation → (E[!(idle) U
transitions that have no event are fired as soon as the state payment])))
machine reaches a configuration that is containing the cor- The next property asserts that it is always the case that
responding source state. This is precisely the case with the after reaching state operation it should be unavoidable
transition from account to payment. Thus, there is no to reach state eject at a later point:
case that allows the operation state to be exited. This can ALWAYS operation → INEVIT eject
also be seen by looking at the corresponding CTS where CTL: SPEC AG((operation → (A[!(idle) U
we can notice that once a configuration that contains the eject])))
operation state is reached, there is no transition to a con- The last one states that chkBal must precede debit:
figuration that does not contain it. chkBal PRECEDE debit
Another property that failed was with respect to the CTL: SPEC (!E[!(chkBal) U debit])))
reachability of state statement. The problem is caused The first two manually input specifications turned out
by the fact that the transition from account to payment to be satisfied when running the model-checker. However,

Proceedings of the 13th Annual IEEE International Symposium and Workshop on Engineering of Computer Based Systems (ECBS’06)
0-7695-2546-6/06 $20.00 © 2006 IEEE
Authorized licensed use limited to: ISAE. Downloaded on April 02,2023 at 19:04:39 UTC from IEEE Xplore. Restrictions apply.
 the last two properties failed. The failure of the third one a trigger such as select to the transition from state
was not unexpected as from the automatic specifications we account to state payment. This will fix both the dead-
noticed that state operation has deadlock and does not lock and the reachability problems. The second modifi-
have state eject as a substate. The failure of the last prop- cation targets the last manual specification and consists in
erty was accompanied by the following trace provided by changing the target of the transition from state billPay
the model-checker. Though the model-checker provided to state debit to a new target state namely transac. Af-
other counterexamples for previous failed properties, we ter re-executing the verification phase for the fixed diagram,
present this last one as it captures a critical unintended be- a modified CTS was obtained, depicted in Figure 7, and all
havior: the specifications, both automatic and the manually speci-
idle; fied properties were satisfied.
verif;
operation,account,info,balance,
6 Class and Package Diagrams Case Study
proc,selAccount;
operation,payment,cashAdv;
operation,payment,billPay; The proposed example of class and package diagrams,
operation,transac,debit depicted in Figure 8, illustrates a real-time heart monitor-
ing system consisting of three packages. One contains the
The presented counterexample follows exactly the same windows components that display the monitoring results.
notation as the sequence diagram counterexample. For the Another package contains the platform specific heart moni-
toring tools, whereas the last one contains the heart monitor-
ing components. This is a good example to be tested due to
idle
the different types of relationships among the classes. Our
insert tool implements a set of fifteen metrics [11] for class and
package diagrams. In the following, we briefly present our
verif results when applying these metrics on this diagram. The
analysis results indicate that some classes are rather com-
cardOk
badCard plex thus having a weak reusability potential.
operation, account,
info, balance, proc, Package Name A I DMS
selAccount
next
Platform Specific HM 0 49 51
next HM 0 49 51
back Windows Components 0 50 50
back select back
Average 0 50 50
operation, account,
Nominal Range - - 50 - 100%
info, statement, proc,
selAccount
The table above shows the metrics related to package
select back diagrams. The distance from the main sequence metric
(DMS) measures the balance between the abstraction and
operation, payment, cashAdv eject
instability levels in the package. As shown in the table, the
next
next
three packages in the diagram fall within the nominal range
select
back of DMS. However the Abstraction and Instability metrics do
operation, payment, billPay not have nominal ranges due to the difference in the design
perspectives. Thus, DMS is a compromise between their
select
values. In the same table, the zero abstractness value for
operation, transac, chkBal
the three packages shows a weak potential for extendibility
insuf
and modifiability. Also, the elevated value of the instability
balOk metric in the second column indicates that the three pack-
ages are subject to change if other packages do change.
operation, transac, modify operation, transac, debit
Table 1 presents the analysis results of the class diagram
inheritance related metrics. The Depth of Inheritance Tree
Figure 7. CTS of the Fixed ATM. (DIT) metric shows a proper use of inheritance as it has
no negative impact on the complexity level of the diagram.
Furthermore, our results show that the diagram has a shal-
presented state machine diagram, we also provide the nec- low inheritance tree indicating a good level of understand-
essary changes that will fix the problems identified by the ability and testability. With respect to the Number Of Chil-
model-checker. The first modification consists in adding dren (NOC), the analysis shows that only four classes in the

Proceedings of the 13th Annual IEEE International Symposium and Workshop on Engineering of Computer Based Systems (ECBS’06)
0-7695-2546-6/06 $20.00 © 2006 IEEE
Authorized licensed use limited to: ISAE. Downloaded on April 02,2023 at 19:04:39 UTC from IEEE Xplore. Restrictions apply.
 









Figure 8. Class and Package Diagrams Example.

Class Name DIT NOC NOM NOA NMA NMO Class Name CR CCRC CBO PMR
Cardio 1 0 10 1 6 4 Cardio 0 200 1 100
Rate Hdlr 1 0 3 1 2 1 Rate Hdlr 0 200 1 100
Gain Hdlr 1 0 3 1 2 1 Gain Hdlr 0 200 1 100
Power Hdlr 1 0 3 1 2 1 Power Hdlr 0 200 1 100
TE PlotTimer 1 0 5 2 2 3 TE PlotTimer 0 100 0 100
Cardio Proxy 1 0 11 0 5 4 Cardio Proxy 0 700 5 100
Abstract Cardio 0 2 4 0 4 0 Abstract Cardio 0 100 1 100
WMutex 0 0 3 1 3 0 WMutex 0 0 0 100
Event 0 3 1 0 1 0 Event 0 0 0 100
TimedEvent 0 1 3 0 3 0 TimedEvent 0 0 0 100
Active Object 0 1 2 0 2 0 Active Object 0 0 0 100
Queue 0 0 3 0 3 0 Queue 0 0 0 100
Input Handler 0 0 5 0 5 0 Input Handler 0 200 2 100
Average 0.46 0.88 4.31 0.54 3.08 1.08 Average 0 146 0.92 100
Nominal Range 1-4 1-4 3-7 2-5 0-4 0-5% Nominal Range 20 - 75% 150 - 350% 1-4 5 - 50%

Table 1. Class Diagram Inheritance Metrics Table 2. Class Diagram General Metrics

diagram have a good NOC value. In a class diagram, the high NMA may be difficult to reuse, whereas, classes with
number of children is an indication of the class reusability. no specialization and having large number of methods may
The analysis results also show that five classes have weak impede other classes from reusing their functionality requir-
Number Of Methods (NOM) value. On the other hand, the ing the decomposition into smaller specialized classes in or-
class diagram has an overall NOM that lies in the nomi- der to improve the design.
nal range. The problem of unsuitable NOM values may The Coupling Between Object (CBO) classes metric
be solved by modifying the class diagram by decompos- measures the level of coupling between classes, denoting
ing the existing classes to smaller new classes to share the an increase in the complexity for high coupling. Table 2
number of methods that exceed the nominal range. Conse- shows seven classes outside the CBO nominal range while
quently, classes in the class diagram will be more reusable. six classes are falling within it. This shows an increased
Table 1 shows only one class in the Number Of Attributes complexity and suggests further modification by reducing
(NOA) nominal range. This invites to further enhancement the number of relationships between the classes.
by adding new attributes to the non-abstract classes, though The Class Category Relational Cohesion (CCRC) mea-
the size of the class increases with its number of attributes. sures the cohesion of classes within the diagram. This
The Number of Methods Added (NMA) measures the metric reflects the diagram’s architecture strength. Table
inheritance usefulness degree. Three classes have a high 2 shows a good CCRC level for only five classes whereas
NMA value indicating a misuse of inheritance. Classes with the remaining eight classes have a weak CCRC level. We

Proceedings of the 13th Annual IEEE International Symposium and Workshop on Engineering of Computer Based Systems (ECBS’06)
0-7695-2546-6/06 $20.00 © 2006 IEEE
Authorized licensed use limited to: ISAE. Downloaded on April 02,2023 at 19:04:39 UTC from IEEE Xplore. Restrictions apply.
 can also see that the average CCRC is outside the nominal Systems Development with UML (CSDUML ’04, Proceed-
range indicating a lack of cohesion between the classes. The ings), pages 85–99. Technische Universität München, 2004.
Class Responsibility (CR) results in Table 2 show that none [6] A. Cimatti, E. M. Clarke, F. Giunchiglia, and M. Roveri.
of the classes in the diagram is implementing pre-conditions Nusmv: A New Symbolic Model Verifier. In cav, 1999.
[7] C. Committee. The International Council on Systems
and/or post-conditions. CR is measured in the cases when
Engineering (incose). http://www.incose.org/
a class method should be responsible to check whether a practice/whatissystemseng.aspx.
message is correct before taking any action. In the current [8] R. Eshuis. Semantics and Verification of UML Activity Di-
example, the CR value can be enhanced by adding pre/post- agrams for Workflow Modelling. PhD thesis, University of
conditions to the methods that need to check the validity Twente, 2002.
of messages prior or after the execution of an action. In [9] M. Genero, M. Piattini, and C. Calero. Early Measures for
the design of a class diagram, the use of pre/post-conditions UML Class Diagrams. L’OBJET, 6(4), 2000.
should be carefully considered. Therefore, this metric is [10] R. Gronback. Model Validation: Applying Audits and Met-
useful to check systems with real-time messaging. rics to UML Models. In BorCon 2004 Proceedings, 2004.
[11] U. R. Group. Object Oriented Model Met-
Finally, Table 2 shows that all methods in the class dia-
rics. Technical report, The United States Air
gram are accessible, which inhibits encapsulation in the dia- Force Space and Warning Product-Line Systems,
gram. This requires some adjustments for the access control http://www.cin.ufpe.br/ inspector/relacionados/Object-
level for all the classes in the diagram. oriente%20Model%20Metrics%20Document.htm, 1996.
[12] IEEE. Standard 610.12.
7 Conclusion and Future Work [13] D. Latella, I. Majzik, and M. Massink. Automatic Ver-
ification of a Behavioural Subset of UML Statechart dia-
grams using the spin model-checker. Formal Asp. Comput.,
In this paper, we proposed a unified paradigm for V&V 11(6):637–664, 1999.
in software and systems engineering with three distinc- [14] D. Latella, I. Majzik, and M. Massink. Towards a For-
tive features: model-checking, program analysis, and soft- mal Operational Semantics of UML Statechart Diagrams.
ware engineering techniques. With respect to the latter, we In Proceedings of the IFIP TC6/WG6.1 Third International
showed a real life example of class and package diagrams Conference on Formal Methods for Open Object-Based Dis-
assessment. Also, we showed two case studies for sequence tributed Systems (FMOODS), page 465. Kluwer, B.V., 1999.
[15] E. Mikk, Y. Lakhnech, M. Siegel, and G. J. Holzmann. Im-
and state machine diagrams respectively accompanied by
plementing Statecharts in PROMELA/SPIN. In WIFT ’98:
the analysis based on formal verification and the benefits of
Proceedings of the Second IEEE Workshop on Industrial
our approach in term of the assessment results. Strength Formal Specification Techniques, page 90. IEEE
As future work, we intend to cover the complete set of Computer Society, 1998.
UML and SysML behavioral diagrams using the presented [16] O. M. G. (OMG). UML 2.0 Superstructure Specification,
methodology. Moreover, we intend to fully pursue the in- 2003.
tegration of the metrics related to the semantic models that [17] S. Partners. System Modeling Language: SysML, 2004.
are derived from various behavioral diagrams. [18] A. Software. ARTiSAN Real-time Studio.
http://www.artisansw.com/pdflibrary/
Rts_5.0_datasheet.pdf. Datasheet.
References [19] D. E. Stevenson. Verification and Validation of Complex
Systems. In Proceedings of ANNIE 2002, Smart Engineer-
[1] N. Aeronautics and S. Administration. Software Quality ing System Design. ASME, November 2002.
Metrics for Object Oriented System Environments. Tech- [20] H. Störrle. Semantics of Interactions in UML 2.0. In HCC,
nical Report SATC-TR-95-1001, National Aeronautics and pages 129–136, 2003.
Space Administration, Goddard Space Flight Center, Green- [21] G. Tugwell, J. Holt, C. Neill, and C. Jobling. Metrics for Full
belt Maryland 20771, JUNE 1995. Systems Engineering Lifecycle Activities (MeFuSELA). In
[2] I. Averant. Static Functional Verification with Solidify, a Proceedings of the Ninth International Symposium of the In-
New Low-Risk Methodology for Faster Debug of ASICs ternational Council on Systems Engineering (INCOSE 99),
and Programmable Parts. Technical report, Averant, Inc.s, Brighton, U.K., 1999.
2001. [22] W. M. P. van der Aalst. Interval Timed Coloured Petri Nets
[3] E. Börger, A. Cavarra, and E. Riccobene. An ASM Seman- and their Analysis. In Application and Theory of Petri Nets,
tics for UML Activity Diagrams. In AMAST, pages 293– pages 453–472, 1993.
308, 2000. [23] W. M. P. van der Aalst. The Application of Petri Nets to
[4] U. Bremen. udraw(graph)tool. http://www. Workflow Management. Journal of Circuits, Systems, and
informatik.uni-bremen.de/uDrawGraph/ Computers, 8(1):21–66, 1998.
en/index.html.
[5] M. V. Cengarle and A. Knapp. UML 2.0 Interactions: Se-
mantics and Refinement. In 3rd Intl. Workshop on Critical

Proceedings of the 13th Annual IEEE International Symposium and Workshop on Engineering of Computer Based Systems (ECBS’06)
0-7695-2546-6/06 $20.00 © 2006 IEEE
Authorized licensed use limited to: ISAE. Downloaded on April 02,2023 at 19:04:39 UTC from IEEE Xplore. Restrictions apply.