Introduction to Software Testing

Chapter 7.1
Engineering Criteria for Technologies

Paul Ammann & Jeff Offutt

http://www.cs.gmu.edu/~offutt/softwaretest/
 The Technologies
 Chapters 1-5 emphasize criteria on four models of software

 Emphasis in each chapter was first on the criteria, then on how

to construct the models from different software artifacts

 This chapter discusses how to apply the criteria to specific

technologies
– Most of the ideas in this chapter were developed after the year 2000
– Thus they are still evolving

Introduction to Software Testing (Ch 7) © Ammann and Offutt 2

 Chapter 7 Outline

1. Object-Oriented Software
2. Web Applications and Web Services
3. Graphical User Interfaces
4. Real-Time and Embedded Software

Introduction to Software Testing (Ch 7) © Ammann and Offutt 3

 Section 7.1 Outline
1. Overview
2. Types of Object-Oriented Faults
1. Example
2. The Yo-Yo Graph and Polymorphism
3. Categories of Inheritance Faults
4. Testing Inheritance, Polymorphism and Dynamic
Binding
5. Object-Oriented Testing Criteria

Introduction to Software Testing (Ch 7) © Ammann and Offutt 4

 Inheritance

Allows common features of many

classes to be defined in one class

A derived class has everything

its parent has, plus it can:
 Enhance derived features (overriding)
 Restrict derived features
 Add new features (extension)

Introduction to Software Testing (Ch 7) © Ammann and Offutt 5

 Inheritance (2)

Declared type: The type given when an object

reference is declared
Clock w1; // declared type Clock

Actual type: The type of the current object

w1 = new Watch(); // actual type Watch

A
In Java, the method that is executed is
the lowest version of the method defined B
between the actual and declared types in
the inheritance hierarchy C

Introduction to Software Testing (Ch 7) © Ammann and Offutt 6

 Access Control (in Java)
Class 1
Package
public members

protected members Class 3

default

private members

inheritance

Class 2

Class 5 Class 4
Introduction to Software Testing (Ch 7) © Ammann and Offutt 7
 Polymorphism
 The same variable can have different types depending on the
program execution

 If B inherits from A, then an object of type B can be used when

an object of type A is expected

 If both A and B define the same method M (B overrides A), then

the same statement might call either A’s version of M or B’s
version

Introduction to Software Testing (Ch 7) © Ammann and Offutt 8

 Subtype and Subclass Inheritance
 Subtype Inheritance : If B inherits from A, any object of type B
can be substituted for an object of type A
– A laptop “is a” special type of computer animal
– Called substitutability
human deer

 Subclass Inheritance : Objects of type B may not be substituted

for objects of type A
– Objects of B may not be “type compatible”
– In Java’s collection framework, a Stack inherits from a Vector …
convenient for implementation, but a stack is definitely not a vector

human
This gives a
deer access
to “hands” !
deer
Introduction to Software Testing (Ch 7) © Ammann and Offutt 9
 Testing OO Software

1. Intra-method testing : Testing individual methods

within classes
2. Inter-method testing : Multiple methods within a
class are tested in concert
3. Intra-class testing : Testing a single class, usually
using sequences of calls to methods within the class
4. Inter-class testing : More than one class is tested at
the same time (integration)

Introduction to Software Testing (Ch 7) © Ammann and Offutt 10

 Section 7.1 Outline
1. Overview
2. Types of Object-Oriented Faults
1. Example
Example
2. The Yo-Yo Graph and Polymorphism
3. Categories of Inheritance Faults
4. Testing Inheritance, Polymorphism and
Dynamic Binding
5. Object-Oriented Testing Criteria

Introduction to Software Testing (Ch 7) © Ammann and Offutt 11

 Example DU Pairs and Anomalies
Consider what happens when an
A overriding method has a different
-u def-use
-v def-set than the overridden method
-w
+h() Method Defs Uses
+i()
+j() A::h () {A::u, A::w}
+l()
A::i () {A::u}
A::j () {A::v} {A::w}
B
-x
A::l() {A::v} DU
+h () B::h() {B::x} anomaly
+i ()
B::i() {B::x}
C::i()
C
-y
C::j() {C::y} {C::y}
+i ()
+j ()
A::h() calls j(), DU
def-use
B::h() does not anomaly
Introduction to Software Testing (Ch 7) © Ammann and Offutt 12
 Section 7.1 Outline
1. Overview
2. Types of Object-Oriented Faults
1. Example
2. The Yo-Yo
The Yo-Yo Graph
Graph andand Polymorphism
Polymorphism
3. Categories of Inheritance Faults
4. Testing Inheritance, Polymorphism and
Dynamic Binding
5. Object-Oriented Testing Criteria

Introduction to Software Testing (Ch 7) © Ammann and Offutt 13

 Polymorphism Headaches (Yo-Yo)
A Actual
+d () A d() g() h() i() j() l()
+g () type
+h ()
B h() i() k()
+i ()
+j () C i() j() l()
+l ()

Object is of actual type A

A::d ()

d() calls g(), which calls h(), which

calls i(), which calls j()

Introduction to Software Testing (Ch 7) © Ammann and Offutt 14

 Polymorphism Headaches (Yo-Yo)
A
+d () A d() g() h() i() j() l()
+g ()
+h ()
B h() i() k()
+i ()
+j () C i() j() l()
+l ()

A d() g() h() i() j() l()

B
Actual
+h () B h() i() k()
+i () type
+k ()
C i() j() l()

Object is of actual type B

B::d ()

Introduction to Software Testing (Ch 7) © Ammann and Offutt 15

 Polymorphism Headaches (Yo-Yo)
A
+d () A d() g() h() i() j() l()
+g ()
+h ()
B h() i() k()
+i ()
+j () C i() j() l()
+l ()

A d() g() h() i() j() l()

B
+h () B h() i() k()
+i ()
+k ()
C i() j() l()

A d() g() h() i() j() l()

B h() i() k()

C
Actual
+i () C i() j() l()
+j () type
+l ()
Object is of actual type C, C::d ()
Introduction to Software Testing (Ch 7) © Ammann and Offutt 16
 Section 7.1 Outline
1. Overview
2. Types of Object-Oriented Faults
1. Example
2. The Yo-Yo Graph and Polymorphism
3. Categories
Categories ofof Inheritance
Inheritance Faults
Faults
4. Testing Inheritance, Polymorphism and
Dynamic Binding
5. Object-Oriented Testing Criteria

Introduction to Software Testing (Ch 7) © Ammann and Offutt 17

 Potential for Faults in OO Programs

• Complexity is relocated to the connections among components

• Less static determinism – many faults can now only be detected
at runtime
• Inheritance and Polymorphism yield vertical and dynamic
integration
• Aggregation and use relationships are more complex
• Designers do not carefully consider visibility of data and
methods

Introduction to Software Testing (Ch 7) © Ammann and Offutt 18

 Object-oriented Faults
 Only consider faults that arise as a direct result of OO language
features:
– inheritance
– polymorphism
– constructors
– visibility
 Language independent (as much as possible)

Introduction to Software Testing (Ch 7) © Ammann and Offutt 19

 OO Faults and Anomalies

Acronym Fault / Anomaly

ITU Inconsistent Type Use
SDA State Definition Anomaly
SDIH State Definition Inconsistency Examples
SDI State Defined Incorrectly shown

IISD Indirect Inconsistent State Definition

ACB1 Anomalous Construction Behavior (1)
ACB2 Anomalous Construction Behavior (2)
IC Incomplete Construction
SVA State Visibility Anomaly
Introduction to Software Testing (Ch 7) © Ammann and Offutt
20
 Inconsistent Type Use (ITU)
 No overriding (no polymorphism)
 C extends T, and C adds new methods (extension)
 An object is used “as a C”, then as a T, then as a C
 Methods in T can put object in state that is inconsistent for C
s.push (“Steffi”);
Vector s.push (“Joyce”);
-array s.push (“Andrew”);
+insertElementAt() dumb (s);
+removeElementAt() s.pop();
s.pop();
s.pop(); // Stack is empty!
Stack
call void dumb (Vector v)
+pop (): Object {
call v.removeElementAt (v.size()-1);
+push (): Object
}
Introduction to Software Testing (Ch 7) © Ammann and Offutt 21
 State Definition Anomaly (SDA)
 X extends W, and X overrides some methods
 The overriding methods in X fail to define some variables that
the overridden methods in W defined

W
v
u • W::m () defines v and W::n() uses v
m()
n() • X::n () uses v

X • Y::m () does not define v

x For an object of actual type Y, a data flow
n() anomaly exists and results in a fault if m() is
called, then n()
Y
w
m()
Introduction to Software Testing (Ch 7) © Ammann and Offutt 22
 State Definition Inconsistency (SDIH)
 Hiding a variable, possibly accidentally
 If the descendant’s version of the variable is defined, the
ancestor’s version may not be

W
v
u • Y overrides W’s version of v
m()
n() • Y::m() defines Y::v

X • X::n() uses v … getting W’s version of v

x For an object of actual type Y, a data flow
n() inconsistency may exist and result in a fault if
overrides
m() is called, then n()
Y
(hides)
v
m()
Introduction to Software Testing (Ch 7) © Ammann and Offutt 23
 Anomalous Construction Behavior (ACB1)
 Constructor of W calls a method f()
 A child of W, X, overrides f()
 X::f() uses variables that should be defined by X’s constructor

W()
When an object of type X is constructed,
Calls W() is run before X().
f()
Overrides When W() calls X::f(), x is used, but has
X not yet been given a value!
x
Uses
X()
f()
Introduction to Software Testing (Ch 7) © Ammann and Offutt 24
 State Visibility Anomaly (SVA)
 A private variable v is declared in ancestor W, and v is defined
by W::m()
 X extends W and Y extends X
 Y overrides m(), and calls W::m() to define v
W W
-v -v
m() Overrides m()
Overrides,
calls
X X
X::m() is added later
X::m() does not call W::m()
Overrides, m()
calls Y:m() can no longer call W::m(),
Y Y
so v is not defined!

m() m()

Introduction to Software Testing (Ch 7) © Ammann and Offutt 25

 Section 7.1 Outline
1. Overview
2. Types of Object-Oriented Faults
1. Example
2. The Yo-Yo Graph and Polymorphism
3. Categories of Inheritance Faults
4. Testing Inheritance,
Testing Inheritance, Polymorphism
Polymorphism and and
Dynamic
DynamicBinding
Binding
5. Object-Oriented Testing Criteria

Introduction to Software Testing (Ch 7) © Ammann and Offutt 26

 Coupling Sequences
 Pairs of method calls within body of method under test:
– Made through a common instance context
– With respect to a set of state variables that are commonly referenced by
both methods
– Consists of at least one coupling path between the two method calls with
respect to a particular state variable
 Represent potential state space interactions between the called
methods with respect to calling method
 Used to identify points of integration and testing requirements

Introduction to Software Testing (Ch 7) © Ammann and Offutt 27

 Types of Def-Use Pairs
def
def last-def
A A ()
use ()
inter-procedural
B () first-use data flow
A use B ()
()
intra-procedural data flow
(within the same unit) full coupling

def
A( M A() def
A()
) () A()
M()
B( use N(
) ) B() use
B()
N() B()
F F(
() )
object-oriented direct object-oriented indirect
coupling data flow coupling data flow
Introduction to Software Testing (Ch 7) © Ammann and Offutt 28
 Coupling-Based Testing (from Ch 2)
 Test data and control
connections
Caller
 Derived from previous work x = 14 last-def-before-call
F
for procedural programs
y = G (x) call site
 Based on insight that print (y) first-use-after-call
integration occurs through
couplings among software Callee
G (a) print (a) first-use-in-callee
artifacts
b = 42 last-def-before-
return
return (b)

Introduction to Software Testing (Ch 7) © Ammann and Offutt 29

 Polymorphic Call Set

public void f ( W o )
{
…
Set of methods that can potentially
execute as result of a method call j o.m();
through a particular instance …
context
l o.l();
pcs (o.m) = {W::m, Y::m, X::m} …
k o.n();
}

Introduction to Software Testing (Ch 7) © Ammann and Offutt

30
 Example Coupling Sequence
Client f
W
o bound to h def (o)
-v : m ()
-u : instance of W i o.m()
+m()
+n()
def (W::v)
+l()

l ()
j o.l()
X Z
-x :
def (W::u)
+n() +m()
+n()
n ()
k o.n()
Y Coupling
-w : Couplingwith
sequence use (W::u)
+m()
+l()
sequence
respect with
to W::u use (W::v)
respect to W::v

Introduction to Software Testing (Ch 7) © Ammann and Offutt 31

 Example Coupling Sequence (2)
Client f
W
o bound to h def (o)
-v : m ()
-u : instance of Z i o.m()
+m()
+n()
def (Z::x)
+l()

l ()
j o.l()
X Z
-x : -x :
def (W::u)
+n() +m()
+n()
n ()
k o.n()
Y
-w : Coupling use (Z::x)
+m()
+l()
sequence with use (Z::x)
respect to Z::x

Introduction to Software Testing (Ch 7) © Ammann and Offutt 32

 Section 7.1 Outline
1. Overview
2. Types of Object-Oriented Faults
1. Example
2. The Yo-Yo Graph and Polymorphism
3. Categories of Inheritance Faults
4. Testing Inheritance, Polymorphism and
Dynamic Binding
Object-Oriented
5. Object-Oriented Testing
Testing Criteria
Criteria

Introduction to Software Testing (Ch 7) © Ammann and Offutt 33

 Testing Goals
 We want to test how a method can interact with
instance bound to object o:
– Interactions occur through the coupling sequences
 Need to consider the set of interactions that can occur:
– What types can be bound to o?
– Which methods can actually execute? (polymorphic call sets)
 Test all couplings with all type bindings possible

Introduction to Software Testing (Ch 7) © Ammann and Offutt 34

 All-Coupling-Sequences
All-Coupling-Sequences (ACS) : For every coupling sequence Sj
in f(), there is at least one test case t such that there is a coupling
path induced by Sj,k that is a sub-path of the execution trace of f(t)

 At least one coupling path must be executed

 Does not consider inheritance and polymorphism

Introduction to Software Testing (Ch 7) © Ammann and Offutt 35

 All-Poly-Classes
All-Poly-Classes (APC) : For every coupling sequence Sj,k in
method f(), and for every class in the family of types defined by
the context of Sj,k, there is at least one test case t such that when
f() is executed using t, there is a path p in the set of coupling
paths of Sj,k that is a sub-path of the execution trace of f(t)

 Includes instance contexts of calls

 At least one test for every type the object can bind to
 Test with every possible type substitution

Introduction to Software Testing (Ch 7) © Ammann and Offutt 36

 All-Coupling-Defs-Uses
All-Coupling-Defs-Uses (ACDU) : For every coupling variable v
in each coupling Sj,k of t, there is a coupling path induced by Sj,k
such that p is a sub-path of the execution trace of f(t) for at last
one test case t

 Every last definition of a coupling variable reaches every

first use

 Does not consider inheritance and polymorphism

Introduction to Software Testing (Ch 7) © Ammann and Offutt 37

 All-Poly-Coupling-Defs-and-Uses
All-Poly-Coupling-Defs-and-Uses (APDU) : For every coupling
sequence Sj,k in f(), for every class in the family of types defined
by the context of Sj,k, for every coupling variable v of Sj,k, for
every node m that has a last definition of v and every node n
that has a first-use of v, there is at least one test case t such that
when f() is executed using t, there is a path p in the coupling
paths of Sj,k that is a sub-path of the trace of f()

 Every last definition of a coupling variable reaches every first

use for every type binding
 Combines previous criteria
 Handles inheritance and polymorphism
 Takes definitions and uses of variables into account

Introduction to Software Testing (Ch 7) © Ammann and Offutt 38

 OO Coverage Criteria Subsumption

All-Poly-Coupling
Defs-Uses
APDU

All-Poly- All-Coupling-
Classes Defs-Uses
APC ACDU

All-Coupling-
Sequences
ACS

Introduction to Software Testing (Ch 7) © Ammann and Offutt 39

 Conclusions
 A model for understanding and analyzing faults that occur as
a result of inheritance and polymorphism
– Yo-yo graph
– Defs and Uses of state variables
– Polymorphic call set

 Technique for identifying data flow anomalies in class

hierarchies

 A fault model and specific faults that are common in OO

software

 Specific test criteria for detecting such faults

Introduction to Software Testing (Ch 7) © Ammann and Offutt 40