Introduction to Software

Testing
(2nd edition)
Chapter 7.1, 7.2

Overview Graph Coverage

Criteria
(active class version)

Paul Ammann & Jeff Offutt

http://www.cs.gmu.edu/~offutt/softw
aretest/

Update, October 2016

 Ch. 7 : Graph Coverage
Four Structures for
Modeling Software

Input Graph Logic Synta

Space Applie s Applied x
d to to
Applie
Sourc FSMs
d to
e
Specs DNF

Source Specs Sourc Model

e s
Design Use Integ Input
Introduction to Software Testing, Edition 2 (Ch 07) cases © Ammann & Offutt 2
 Covering Graphs (7.1)
• Graphs are the most commonly used structure
for testing

• Graphs can come from many sources

– Control flow graphs
– Design structure
– FSMs and statecharts
– Use cases

• Tests usually are intended to “cover” the graph

in some way

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 3

 Definition of a Graph
• A set N of nodes, N is not empty

• A set N of initial nodes, N is not empty

0 0

• A set N of final nodes, N is not empty

f f

• A set E of edges, each edge from one node to

another
– ( ni , nj ), i is predecessor, j is successor
N0 =
Is this a { 1} Yes
graph? 1
Nf =
{1}
Introduction to Software Testing, Edition 2 (Ch 07)
E={} © Ammann & Offutt 4
 Example Graphs
1 1 2 3 1

Not a
valid
graph
2 3 4 5 6 7 2 3

4 8 9 10 4

N0 = { 1 } N0 = { 1, 2, 3 } N0 = { }
Write down Write down Write down
the Nfinitial
= { 4and } Nthe
f = initial
{ 8, 9, and10 } theNfinitial
= { 4and
}
final
E = nodes, { (1, 2), final (1,5),
E = { (1,4), nodes,(2,5), (3,6), final
E =nodes,
{ (1,2),
and
(1,3), the (2,4),edges (3, 7),and
(4,the
8), edges
(5,8), (5,9), and the (2,4),
(1,3), edges
Introduction(3,4) } Edition 2 (Ch 07) (6,2), (6,10), (7,10)
to Software Testing, (9,6)
© Ammann & Offutt } (3,4) } 5
 Paths in Graphs
• Path : A sequence of nodes – [n , n , …, n ]
1 2 M
– Each pair of nodes is an edge
• Length : The number of edges
– A single node is a path of length 0
• Subpath : A subsequence of nodes in p is a
subpath of p

1 2 3
A Few
Paths
Write
[ 1, 4, 8 ]
4 5 6 7 down
[three
2, 5, 9, 6, 2 ]
paths
[ 3, 7, 10in]
this graph
8 9 10
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 6
 Test Paths and SESEs
• Test Path : A path that starts at an initial node
and ends at a final node
• Test paths represent execution of test cases
– Some test paths can be executed by many tests
– Some test paths cannot be executed by any tests
• SESE graphs : All test paths start at a single
node and end at another node
– Single-entry, single-exit
– N0 and Nf have exactly one node
Double-diamond
2 5 graph
Four test paths
1 4 7
[1, 2, 4,
Write down5, 7]
3 6 [1,the
all 2, 4,test
6, 7]
[1, 3, 4,in5,this
paths 7]
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt
[1, 3, 4, 6, 7]
graph 7
 Visiting and Touring
• Visit : A test path p visits node n if n is in p
A test path p visits edge e if e is in p
• Tour : A test path p tours subpath q if q is a
subpath of p
Test path [ 1, 2, 4, 5, 7 ]
Visits nodes1,?2, 4, 5, 7
Visits edges(1,2),
? (2,4), (4, 5), (5, 7)
Tours subpaths ?
[1,2,4], [2,4,5], [4,5,7], [1,2,4,5],
[2,4,5,7], [1,2,4,5,7]

(Also, each edge is technically a subpath)

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 8
 Tests and Test Paths
• path (t) : The test path executed by test t
• path (T) : The set of test paths executed by
the set of tests T
• Each test executes one and only one test path
– Complete execution from a start node to an final
node
• A location in a graph (node or edge) can be
reached from another location if there is a
sequence of edges from the first location to
the second
– Syntactic reach : A subpath exists in the graph
– Semantic reach : A test exists that can execute that
subpath
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 9
 Tests and Test Paths
test many-to-
1 one
Test
test
Path
2
test
3
Deterministic software–test always executes the
same test path
many-to-many
test Test Path
1 1
test Test Path
2 2
test Test Path
3 3
Non-deterministic software–the same test can
execute different test paths
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 10
Testing and Covering Graphs (7.2)
• We use graphs in testing as follows :
– Develop a model of the software as a graph
– Require tests to visit or tour specific sets of nodes,
edges or subpaths
• Test Requirements (TR) : Describe properties of test
paths
• Test Criterion : Rules that define test requirements
• Satisfaction : Given a set TR of test requirements for a
criterion C, a set of tests T satisfies C on a graph if
• and only ifCoverage
Structural for every Criteria
test requirement in TR,
: Defined on therejust
a graph is a
test pathof
in terms innodes
path(T) that
and meets the test requirement tr
edges
• Data Flow Coverage Criteria : Requires a graph to be
annotated with references to variables

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 11

 Node and Edge Coverage
• The first (and simplest) two criteria require
that each node and edge in a graph be
executed
Node Coverage (NC) : Test set T satisfies node
coverage on graph G iff for every syntactically
reachable node n in N, there is some path p in
path(T) such that p visits n.
• This statement is a bit cumbersome, so we abbreviate
it in terms of the set of test requirements

Node Coverage (NC) : TR contains each

reachable node in G.
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 12
 Node and Edge Coverage
• Edge coverage is slightly stronger than node
coverage
Edge Coverage (EC) : TR contains each
reachable path of length up to 1, inclusive, in
G.
• The phrase “length up to 1” allows for graphs
with one node and no edges
• NC and EC are only different when there is an
edge and another subpath between a pair of
nodes (as in an “if-else” statement)
Node Coverage TR : ? = { 1, 2, 3 }
1 Test Path = [ 1, 2,
3]
2 Edge Coverage TR : ? = { (1, 2), (1, 3), (2, 3)
}
3
Test Paths = [ 1, 2, 3 ]
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt [ 1, 3 ] 13
 Paths of Length 1 and 0
• A graph with only one node will not have any
edges
1

• It may seem trivial, but formally, Edge

Coverage needs to require Node Coverage on
• this graph Edge Coverage will not subsume
Otherwise,
Node Coverage
– So we define “length up to 1” instead of simply
“length 1”
• We have the same issue with
1
graphs that only have one edge
– for Edge-Pair Coverage …
2

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 14

 Covering Multiple Edges
• Edge-pair coverage requires pairs of edges, or
subpaths of length 2
Edge-Pair Coverage (EPC) : TR contains each
reachable path of length up to 2, inclusive, in
G.
• The phrase “length up to 2” is used to include
graphs that have less than 2 edges
1 5
Edge-Pair Coverage : ?
2 4 TR = { [1,4,5], [1,4,6], [2,4,5],
[2,4,6], [3,4,5], [3,4,6] }
3 6

• The logical extension is to require all paths …

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 15

 Covering Multiple Edges

Complete Path Coverage (CPC) : TR contains all

paths in G.

Unfortunately, this is impossible if the graph has

a loop, so a weak compromise makes the tester
decide which paths:

Specified Path Coverage (SPC) : TR contains a

set S of test paths, where S is supplied as a
parameter.

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 16

 Structural Coverage Example
Node Coverage
TR = { 1, 2, 3, 4, 5, 6, 7 }
Test Paths: [ 1, 2, 3, 4, 7 ] [ 1, 2, 3, 5, 6, 5, 7 ]
Write
1
Edge Coverage down the
TR = { (1,2), (1, 3), (2, 3), (3, 4), (3, 5), (4,TRs7), and
(5, 6), (5, 7),
2 (6, 5) } Test Paths
Test Paths: [ 1, 2, 3, 4, 7 ] [1, 3, 5, 6, 5,for 7 ]these
criteria
3
Edge-Pair Coverage
TR = { [1,2,3], [1,3,4], [1,3,5], [2,3,4], [2,3,5], [3,4,7],
4 5 [3,5,6], [3,5,7], [5,6,5], [6,5,6], [6,5,7] }
Test Paths: [ 1, 2, 3, 4, 7 ] [ 1, 2, 3, 5, 7 ] [ 1, 3, 4, 7 ]
[ 1, 3, 5, 6, 5, 6, 5, 7 ]
6
7
Complete Path Coverage
Test Paths: [ 1, 2, 3, 4, 7 ] [ 1, 2, 3, 5, 7 ] [ 1, 2, 3, 5, 6, 5,
7 ] [ 1, 2, 3, 5, 6, 5, 6, 5, 7 ] [ 1, 2, 3, 5, 6, 5, 6, 5, 6, 5, 7 ] …
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 17
 Handling Loops in Graphs
• If a graph contains a loop, it has an infinite
number of paths

• Thus, CPC is not feasible

• SPC is not satisfactory because the results are

subjective and vary with the tester

• Attempts to “deal with” loops:

– 1970s : Execute cycles once ([4, 5, 4] in previous example,
informal)
– 1980s : Execute each loop, exactly once (formalized)
– 1990s : Execute loops 0 times, once, more than once (informal
description)
– 2000s : Prime paths (touring, sidetrips, and detours)
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 18
 Simple Paths and Prime Paths
• Simple Path : A path from node ni to nj is
simple if no node appears more than once,
except possibly the first and last nodes are the
same
– No internal loops
– A loop is a simple path
• Prime Path : A simple path that does not
appear as a proper subpath of any other simple
Simple Paths : [1,2,4,1], [1,3,4,1], [2,4,1,2], [2,4,1,3],
path 1 [3,4,1,2], [3,4,1,3], [4,1,2,4], [4,1,3,4], [1,2,4], [1,3,4],
Write
[2,4,1], [3,4,1], [4,1,2], down
[4,1,3], [1,2], [1,3], [2,4], [3,4],
2 3 [4,1], [1], [2], [3], [4]the simple
and prime
4
Prime Paths : [2,4,1,2], paths[2,4,1,3],
for this[1,3,4,1], [1,2,4,1],
graph [3,4,1,3]
[3,4,1,2], [4,1,3,4], [4,1,2,4],

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 19

 Prime Path Coverage
• A simple, elegant and finite criterion that
requires loops to be executed as well as
skipped
Prime Path Coverage (PPC) : TR contains each
prime path in G.

• Will tour all paths of length 0, 1, …

• That is, it subsumes node and edge coverage
• PPC almost, but not quite, subsumes EPC …

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 20

 PPC Does Not Subsume EPC
• If a node n has an edge to itself (self
edge), EPC requires [n, n, m] and [m, n,
n]
• [n, n, m] is not prime
• Neither [n, n, m] nor [m, n, n] are simple
paths (not prime)
EPC Requirements : ?
1 TR = { [1,2,3], [1,2,2], [2,2,3],
[2,2,2] }

2 PPC Requirements
:TR
? = { [1,2,3],

3 [2,2] }

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 21

 Prime Path Example
• The previous example has 38 simple paths
• Only nine prime paths

1
Prime Paths
2 [1, 2, 3, 4, 7]
[1, 2, 3,down
Write 5, 7] Execute
[1, 9
all 2, prime
3, 5, 6] loop 0 times
3 [1, 3, 4, 7]
paths
[1, 3, 5, 7] Execute
4 5 [1, 3, 5, 6] loop once
[6, 5, 7]
6 [6, 5, 6] Execute loop
7 [5, 6, 5] more than once

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 22

 Touring, Sidetrips, and Detours
• Prime paths do not have internal loops … test
paths might
• Tour : A test path p tours subpath q if q is a
subpath of p
• Tour With Sidetrips : A test path p tours
subpath q with sidetrips iff every edge in q is
also in p in the same order
• The tour can include a sidetrip, as long as it comes
back to the same node
• Tour With Detours : A test path p tours
subpath q with detours iff every node in q is
also in p in the same order
• The tour can include a detour from node ni, as long
as it comes back to the prime path at a successor
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 23
 Sidetrips and Detours Example
1 2 3 4
1 2 3 5 6

Touring the prime

path [1, 2, 3, 5, 6] 4
without sidetrips or
detours 1 2 5 6
1 2 3 5 6
3 4
Touring with
a sidetrip 4

1 2 5
1 2 3 5 6
3
Touring with 4
4
a detour

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 24

 Infeasible Test Requirements
• An infeasible test requirement cannot be
satisfied
– Unreachable statement (dead code)
• Most testthat
– Subpath criteria
can onlyhave somewith
be executed infeasible test(X > 0
a contradiction
and X < 0)
requirements
• It is usually undecidable whether all test
requirements are feasible
• When sidetrips are not allowed, many
structural criteria have more infeasible test
requirements
• However, always allowing sidetrips weakens
Practical recommendation—Best Effort
the test criteria Touring
– Satisfy as many test requirements as possible
without sidetrips
– Allow
Introduction to Software Testing, Edition 2 sidetrips
(Ch 07) to try to satisfy remaining
© Ammann & Offutt test 25
 Simple & Prime Path Example
‘!’ means path
Simpl Len Len 1 Len 2
terminates Len 3
e 0
[1] [1, 2] [1, 2, 3] [1, 2,‘*’
3, means
4] path
paths Write
[2] [1, 3] [1, 3, 4] [1, 2, 3, 5]cycles
1 [3] [2, 3] [1, 3, 5] Write
[1, 3, 4, 7] !
paths Write
[4] [3, 4] Write
[2, 3, 4] paths
[1, 3, 5, 7] !
of paths
2 [5] [3, 5] paths
[2, 3, 5] of3, 5, 6] !
[1,
length of
[4, 7] ! of 4, 7] ! length
0[6] length [3, [2, 3, 4, 7] !
length 3 3, 5, 6] !
3
[7] ! 1[5, 7] ! [3, 5, 7] ! [2,
[5, 6] 2 5, 6] !
[3, [2, 3, 5, 7] !
[6, 5] [5, 6, 5] *
4 5 [6, 5, 7] !
[6, 5, 6] *
6
7 Len 4
[1,Write
2, 3, 4, 7] !
[1,paths
2, 3, 5, 7] ! Prime Paths ?
[1,of
2, 3, 5, 6] !

Introduction to Software Testing, Edition 2 (Ch 07)

length © Ammann & Offutt 26
 Round Trips
• Round-Trip Path : A prime path that starts and
ends at the same node
Simple Round Trip Coverage (SRTC) : TR
contains at least one round-trip path for each
reachable node in G that begins and ends a
round-trip path.
Complete Round Trip Coverage (CRTC) : TR
contains all round-trip paths for each
reachable node in G.
• These criteria omit nodes and edges that are
not in round trips
• Thus, they do not subsume edge-pair, edge, or
node coverage
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 27
 Data Flow Criteria
Goal : Ensure that values are computed and
used correctly
• Definition (def) : A location where a value for a
variable is stored into memory
• Use : A location where a variable’s value is
accessed Z = X*2 Defs: def (1) =X{
2 5 } Fill
X= Z
def (5) = { in
42
1 4 7 Z
} thes
3 6 X e
def (6) = { sets
Z = X- } X
8
Uses: use (5) = {
The values given in defs should
} reach at least
one, some, or all possibleuse
uses
(6) = {
Introduction to Software Testing, Edition 2 (Ch 07)
}
© Ammann & Offutt 28
 DU Pairs and DU Paths
• def (n) or def (e) : The set of variables that are
defined by node n
or edge e
• use (n) or use (e) : The set of variables that are used
by node n or
• edge
DU pair e : A pair of locations (li, lj) such that a variable
v is
defined at li and used at lj
• Def-clear : A path from li to lj is def-clear with respect
to variable
v if v is not given another value on any of the nodes or
edges in
the path
• Reach : If there is a def-clear path from li to lj with
• respect
du-pathto :A v,simple subpath that is def-clear with
respect
the def of tovvat li reaches the use at lj
from a def of v to a use of v
• du (ni, nj, v) – the set of du-paths from ni to nj
• du
Introduction to Software (n
Testing,
i, v) – the set of du-paths that start at ni
Edition 2 (Ch 07) © Ammann & Offutt 29
 Touring DU-Paths

• A test path p du-tours subpath d with respect

to v if p tours d and the subpath taken is def-
clear with respect to v

• Sidetrips can be used, just as with previous

touring

• Three criteria
– Use every def
– Get to every use
– Follow all du-paths
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 30
 Data Flow Test Criteria
• First, we make sure every def reaches a use

All-defs coverage (ADC) : For each set of du-

paths S = du (n, v), TR contains at least one
path d in S.
• Then we make sure that every def reaches all
possible uses
All-uses coverage (AUC) : For each set of du-
paths to uses S = du (ni, nj, v), TR contains at
least one path d in S.
• Finally, we cover all the paths between defs
and uses
All-du-paths coverage (ADUPC) : For each set
S = du (ni, nj, v), TR contains every path d in
S.
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 31
 Data Flow Testing Example
Z = X*2
2 5
X = 42
1 4 7
3 6
Z = X-8

All-defs for All-uses for All-du-paths

X 4, 5 ]
[ 1, 2, X 4, 5 ]
[ 1, 2, [ 1,for
2, 4,X5 ]
Write
down [Write
1, 2, 4, 6 ] [Write
1, 3, 4, 5 ]
down
paths to down
satisfy paths to
[paths to 6 ]
1, 2, 4,
satisfy
ADC satisfy [ADUPC
1, 3, 4, 6 ]
AUC

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 32

 Graph Coverage Criteria
Subsumption Complete
Path
Coverage
CPC
Prime Path
Coverage
PPC
All-DU-
Paths
Coverage Edge-
ADUP Pair
Coverag
e
All-uses Complete
Coverage EPC Round Trip
Edge Coverage
AUC
Coverag CRTC
e
All-defs EC Simple Round
Coverage Trip Coverage
ADC Node SRTC
Coverage
NC
Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 33
 Summary 7.1-7.2
• Graphs are a very powerful abstraction for
designing tests
• The various criteria allow lots of cost / benefit
tradeoffs
• These two sections are entirely at the “design
abstraction level” from chapter 2
• Graphs appear in many situations in software
– As discussed in the rest of chapter 7

Introduction to Software Testing, Edition 2 (Ch 07) © Ammann & Offutt 34