Techniques

Design
EECS4313 Integration Testing
Unit Testing
Automation
Software Engineering Testing Software Engineering Testing
Report & Documentation
Dr. Marzieh Ahmadzadeh Verification
Validation Theory
Practice
Integration

The content of this slides is a copyrighted material of

Software Testing: A craftsman’s approach” by Paul C. Jorgensen; Byron DeVries; from Francis & Taylor publisher
Testing Computer Software, Cem Kaner, Jack Falk, Hung Q. Nguyen 1
Outline

▪ Last Lecture
▪ Introduction to the significance of testing
▪ Introducing key terms
▪ Fault, Error, Failure
▪ Validation vs. Verification

▪ This week
▪ Category of unit testing (Verification)
▪ Black-box testing
▪ Boundary Value

2
TESTING TECHNIQUES; UNIT TESTING

3
A Unit of Work

▪ Depending on the paradigm of the programming, a unit of work can be interpreted differently:
▪ A function
▪ A body of code that is presented as a unit in Work Breakdown Structure
▪ An independent body of code that can be compiled and executed by itself.
▪ A class in OOP and all its methods.
▪ The authors of the reference book believe that a unit is a body of software that can be completed by one or two
persons.

▪ For simplicity, here we assume that a unit is a function/procedure/method (depending on which

programming paradigm we refer to) or an independent body of code.

4
 Two Categories of Unit Testing

Specification / functional Code-based / structure

Input Output Input Output

Expected output = Actual output ???

Expected output = Actual output

This Photo by Unknown Author is licensed under CC BY-NC
&&
All lines of code covered by the test cases???

5
Specification vs. Code-based testing

▪ Specification (black box) ▪ Code-based (white box)

▪ Adv.: ▪ Adv.:
▪ Testing is independent of how the software is ▪ Strong background theory like graph theory forms the
implemented. basis for this type of testing. Therefore, test coverage
▪ Test case development can occur in parallel with metric can be used.
implementation.

▪ Disadv.: ▪ Disadv.:
▪ Redundant test cases are possible ▪ Does not recognize if some parts of specification were
▪ Some part of software may remain untested. not implemented.

To [almost] comprehensively test a software, both the

testing should be performed.
6
BLACK BOX UNIT TESTING

7
 Two Categories of Unit Testing

Specification / functional Code-based / structure

Boundary Value Equivalence Class Decision Table

Normal Worst Case Special Value

Robust Robust Worst Case random

8
Functional/Specification Testing

▪ A unit should be seen as a function that maps the domain

of the inputs to its output range.
Output = F(Input) F(a, b)

Da = { A, E, I, O, U}
Db = {1, 2}
▪ The domain of the function is a cartesian product of sets
𝐷1 , 𝐷2, 𝐷3 , … , 𝐷𝑛 , where there are n inputs.
▪ i.e. D= 𝐷1 × 𝐷2 × 𝐷3 × … . × 𝐷𝑛 DF = {(A, 1),(A, 2),(E,
1),(E, 2),(I, 1), (I, 2),
(O, 1), (O, 2), (U, 1),
(U, 2)}

▪ Question: what is the implication of the domain definition, for

functional testing?

9
Boundary Value Testing

▪ Assumption:
▪ F is a function (i.e. program/unit) that gets two inputs x1, x2.
▪ x1 and x2 have some boundaries: 𝑎 ≤ 𝑥1 ≤ 𝑏
▪ A representative from each category is selected. 𝑐 ≤ 𝑥2 ≤ 𝑑
x2
Input Domain of F(x1, x2)

X2 max
d
X2 max-

X2 nom

X2 min+
X2 min
c x1 x1 x1 x1
x1 min min+ nom max- max

x1 10
a b
F(X1, X2)

DX1 = {x1 min , x1 min+ , x1 nom , x1 max- ,x1 max}

DX2 = {X2 min , X2 min+ , X2 nom , X2 max- , X2 max}

DF = {(x1 min , X2 min), (x1 min , X2 min+), (x1 min , X2 nom), (x1 min , X2 max-), (x1 min , X2 max),
(x1 min+ , X2 min), (x1 min+ , X2 min+), (x1 min+ , X2 nom), (x1 min+ , X2 max-), (x1 min+ , X2 max),
(x1 nom , X2 min), (x1 nom , X2 min+), (x1 nom , X2 nom), (x1 nom , X2 max-), (x1 nom , X2 max),
(x1 max- , X2 min), (x1 max- , X2 min+), (x1 max- , X2 nom), (x1 max- , X2 max-), (x1 max- , X2 max),
(x1 max , X2 min), (x1 max , X2 min+), (x1 max , X2 nom), (x1 max , X2 max-), (x1 max , X2 max)}

11
 x2

Normal Boundary Value Testing

d
▪ Test Case Reduced.
▪ It focuses on valid values of the inputs.
▪ A reduced set of test cases can be generated by
c
▪ Holding all inputs at their nominal values except one.
▪ Letting that one input gets all its boundary values.
x1
▪ Repeating this for each input. a b

For two inputs x1, x2

<X1 nom X2 min> <X1 nom X2 min+> <X1 nom X2 nom> <X1 nom X2 max-> <X1 nom X2 max>

<X1 min X2 nom> <X1 min+ X2 nom> <X1 max- X2 nom> <X1 max X2 nom>

▪ In this example, the number of test cases reduced from 25 to 9.

12
Test Case reduction

▪ Boundary testing is important because errors are seen frequently on the boundary values.
▪ It is assumed that two variables rarely hold their extreme values at the same time, when a fault
happens.
▪ Single fault assumptions
▪ a balance between testing comprehensiveness and efficiency

13
STRENGTH AND LIMITATIONS OF NBVT

14
Generalizing Boundary Value Analysis

▪ As long as an order can be defined on the domain of the variable, this method can be used,
because a boundary can be defined.
▪ Nominal variables: can be interpreted as integers.
▪ {Jan, Feb, March, …Dec} -> {1, 2, 3, …12}

▪ Discrete Variables:
▪ Perfect squares less than 100 = {0, 4, 9, 16, 25, 36, 49, 64, 81, 100}

▪ Variables with artificial boundaries

15
Limitation of Boundary Value Analysis

▪ This method is not suitable for variables

1. for which no ordering can be defined.
▪ Colors, Football Teams
2. that are not independent. i.e. the boundary of one variable may change depending on the value of another
variable.
▪ The boundary of days for 1<= February <= 28 on normal years but 1<= February <= 29 for leap years
▪ Example: find the next date:

3. Physical quantity is important

▪ Example: temperature, pressure, wind speed, number of engine spin

16
Expectations & Reading

▪ Expectations:
▪ You should be able to write a complete set of test cases using NBVT technique
▪ To explain what’s the advantage and disadvantages of this method.
▪ You should be mindful of what levels of test maturity you are working, when you write a set of test cases.

▪ Reading:
▪ [Optional]: Chapter 5 of the first reference book up to and including NBVT technique.

17