SOFTWARE TESTING

LECTURE # 6

STRUCTURAL TESTING

26th September, 2023 Dr. Ali Javed

 Contact Information
2

q Course Instructor: Dr. Ali Javed

Associate Professor
Department of Software Engineering
U.E.T Taxila

ü Email: [email protected]
ü Website: http://fms.uettaxila.edu.pk/Profile/ali.javed
ü Research Lab: http://msplab.uettaxila.edu.pk
ü Contact No: +92-51-9047747
ü Office hours:
n Monday, 11:00 - 12:00, Office # 7 S.E.D

q Lab Instructor: Engr. Naazia

q Course TA: Hafsa Ilyas

Dr. Ali Javed
 Course Information
3

q Course Title: Software Testing

q Course Code: SE-411

q Credit Hours: 2+1

q Class: 7th Semester (2k20-Software)

Dr. Ali Javed

 4 Structural Testing
q How to perform structural testing?

q Coverage Types

q Examples

Dr. Ali Javed

 Structural Testing
5

q Structural testing, also known as glass box testing or white box

testing is an approach where the tests are derived from the
knowledge of the software's structure or internal
implementation.

q The other names of structural testing includes clear box testing,

open box testing, logic driven testing, code-based testing or
path driven testing.

q Structural testing focuses on executing test cases to achieve

maximum code coverage

Dr. Ali Javed

 How to perform structural testing?
6

q Structural Testing is based on two factors Control Flow

Testing Theory and Building Control Flow Graphs.

ü The algorithm for all control flow testing consists of a strategy of

converting the code into a control graph.

ü Control graph gives the visual representation of the structure of code

ü After this, the code to be tested is identified, and test cases are created
for the test that is specific to path or condition

ü Test cases generated are based on the structure of the program

Dr. Ali Javed

 Structural Testing Categories
7

q Statement coverage
ü Statement coverage is a measure of the percentage of statements that
have been executed by test cases. Your objective should be to achieve
100% statement coverage through your testing.

q Method coverage
ü Method coverage is a measure of the percentage of methods that have
been executed by test cases. Undoubtedly, your tests should call 100%
of your methods.
ü Method coverage of 50%, for example, means that half of the methods
have been called [1]

q Branch coverage
ü Branch coverage is a measure of the percentage of the decision points
(Boolean expressions) of the program have been evaluated as both
Dr. Ali Javed true and false in test cases.
 Structural Testing Categories
8

q Condition/Multiple Condition Coverage

ü Condition coverage is a measure of percentage of Boolean sub-
expressions of the program that have been evaluated as both true or
false outcome [applies to compound predicate] in test cases.
ü Multiple Condition Coverage is also known as Condition Combination
Coverage.

q Path coverage
ü Path coverage refers to designing test cases such that all linearly
independent paths in the program are executed at least once.

Dr. Ali Javed

 Program Flowcharts
9

q Example of code: main () {

int x, pos, even;

printf("Enter an integer:");
scanf("%d", &x);
pos = 0;
even = 0;
if (x > 0) {pos = 1;}
if (x%2 == 0) {even = 1;}
printf ("%d, %d, %d\n", x, pos, even);
q
return 0;
}

q Intended to set pos = 1 if x is positive, even = 1 if x is even

q Draw a graph of the statements and decisions in the code:
q Statements = boxes
q Decisions = diamonds
q This graph called a “flowchart”
Dr. Ali Javed
 Flowchart Example 1
10

Printf() X>0 T
Pos=1
F
Scanf()
X%2=0 T Even = 1
Pos=0
F

Even=0 Printf()

End
Dr. Ali Javed
 Structural Testing and Flowcharts
11

q If test suite executes all statements, then test suite visits every box and
diamond in the flowchart
q We say the test suite covers every node
q We can also say the test suite covers every statement

q This does not mean that every arrow (edge) in the flowchart has been
followed

q Example: consider the first example flowchart

q We run it on one test case: x=2
q All nodes visited
q However, neither F branch has ever been taken

q If every arrow in the flowchart has been taken by some case in the test suite:
q We say the test suite covers every edge
q We can also say the test suite covers every decision

Dr. Ali Javed

 Edge vs. Node Coverage
12

q Edge cover is more thorough than node coverage

q Catches more problems

q Example: consider first example flowchart

q Change statements pos =0; even = 0 in the first column to pos
= 1; even = 1
q Now, program always sets pos and even to 1
q But we wanted pos to indicate whether x is positive, even to
indicate whether x is even
q If we just test with x = 2

Dr. Ali Javed

 Would 100% Statement/Node Coverage Find
the Mistake with test case x=2?
13

Printf() T
x>0 Pos=1

Coding F
mistake made Scanf()
by a novice
programmer x%2=0 T Even = 1
(initialized
incorrectly)
Pos=1
F

Even=1 Printf()

End
Dr. Ali Javed
 Would 100% Statement/Node Coverage Find
the Mistake with test case x=2?
14

q If we test with both x = 2 and x = -1:

q We cover every edge
q We also force a failure to happen, program reports
wrong results for x = - 1

q Thus, edge coverage can reveal failures where node

coverage cannot.

Dr. Ali Javed

 Coverage Terminology
15

q If a test suite executes 100% of statements in the program, we say it

achieves
q 100% node coverage or 100% statement coverage

q If a test suite follows 90% of edges in a flowchart, we say it achieves 90%

edge coverage (etc)

q “Line Coverage”:
q Any line containing code is measured
q Considered covered if any code on the line is executed
q Not exactly the same thing as statement coverage
q Can have more than one statement on a line
q try: if sam[0] != 'harry': print('hellp', sam)
q except:pass

Dr. Ali Javed

 Strength of Coverage Measures
16

q 100% node coverage implies 100% line coverage

q However, 100% line coverage does not imply 100% node
coverage

q 100% edge coverage implies 100% node coverage Edge

q If followed every edge, must have visited every node (Decision)
q However, as we saw, 100% node coverage does not necessarily
imply 100% edge coverage
Node
q Thus we say: (Statement)
q Node (statement) coverage is strong than line coverage
q Edge (decision) coverage is stronger than node coverage

Line

Dr. Ali Javed

 17
Minimal Test Suites
q By throwing more and more test cases into a test suite, we may achieve
higher coverage. However, longer test suites take longer to run
q Hence, we often want to find some minimal test suite
q Important to define what minimal means i.e., minimal test cases? Minimal
number of inputs used? Minimum time to run?
q Example: first flowchart example:
q Test suite: x=2, x=1, x=-2 covers all edges
q However, it has 3 test cases
q Test suite: x=2, x=-1 covers all edges and has only 2 test cases
q Cannot have a minimal test suite which covers all edges and has only 1
test case
q Have to execute each decision both ways
q Hence, x=2, x=-1 is a minimal test suite for 100% edge coverage
q Minimal in the sense of needing the fewest test cases

Dr. Ali Javed

 Flowcharts: Example 2
18

i = 0; i=0
while (s[i] != 0) {
if (s[i] >= ‘a’) && (s[i] <= ‘z’))
{
s[i]=s[i] – 32;
}
F s[i]!=0?
i++;
} T
s[i]>=‘a’&&
F s[i]<=‘z’?

T
s[i]=s[i]-32

i++

Dr. Ali Javed End

 Coverage and Test Suites: Another
19
Example
q Consider the case-conversion code flowchart (Example 2)

q Find “minimal” test suite which achieves:

q 100 % node coverage

q 100% edge coverage

q Decide what minimal means in this context!

Dr. Ali Javed

 Case Conversion Flowchart:
20
Coverage
q Clearly, we can achieve 100% node coverage with one-char test case
q Say, for example: “a”

q This test case also follows both arrows from the top diamond
q This is because:
q First time through loop (i==0), T branch taken
q Next time condition is evaluated (i==1), s[i] is the null (end of string)
character
q Hence, F branch is also taken

q However, F branch of other diamond not taken

q To cover all edges, we need test suite like

q (A) s= “a”, s=“X”; or
q (B) s = “aX”

Dr. Ali Javed

 Case Conversion Flowchart: Minimality
21

q Which of (A) and (B) is “minimal”

q Depends on how we define minimality
q (A) has 2 test cases, but each has just 1 character
q (B) has just 1 test case, but it has 2 characters

q We should always define precisely what we mean by minimality for a given test
suite
q Here, makes more sense to accept (B) as minimal
q Don’t’ have to run program as many times, incur overhead of starting up
process again

q Hence, for this problem, we should say something like:

q Test suite X will be considered smaller than test suite Y if either:
q X has few test cases than Y; or
q X has the same number of test cases as Y, but the total number of
characters in X is smaller
q Saves us from having to accept s = “aaaaaaXXXXXX” as minimal

Dr. Ali Javed

 Stronger Coverage Measures
22

q So far, we have been writing compound decisions

q (e.g. (c >=‘a’) && (c <=‘z’)) inside a single diamond

q Decision: the whole thing in parentheses after the “if”, while”, etc
q Consider the code:
if ((c>=‘a’) && (c<=‘z’)) {..}
q (c >=‘a’) && (c<=‘z’) is the decision

q Conditions: the individual terms of the condition, connected by logical operators

q (c>=‘a’) && (c<=‘z’) are the two conditions

q If we separate each condition with the decision into a separate diamond:

q We can get a better reflection of what the program does
Dr. Ali Javed
 Splitting Up Diamonds: Flowcharts
23

q Example:
q Instead of writing s[i]>=‘a’
&&
s[i]<=‘z’
F
T

s[i]>=‘a’
q we write F
T
s[i]<=‘z’
F T
q Now we have two new arrows
q To cover all arrows, we have to cover the two ones as well

Dr. Ali Javed

 Short Circuit Condition Coverage
24

q If we split up all diamonds and then cover all arrows:

q We are achieving more thorough coverage than just edge (decision) coverage can
give us
q This is called short circuit (SC) condition coverage
q Recall the definition of short circuit evaluation

q Evaluation of a Boolean expression is cut short as soon as it can be determined

to be true or false
q Applies to language with short circuit evaluation of decisions, e.g., C, C++, Java
q Similar but not identical to another notion called just “condition coverage”
q Condition coverage applies to languages without short-circuit evaluation of
decisions, e.g., Visual Basic

Dr. Ali Javed

 Achieving SC Condition Coverage
25

q Consider the case conversion example

q Test case “aX” is sufficient to achieve 100% edge coverage

q Whole decision (c>=‘a’) && (c<=‘z’) is tested both ways: true for
‘a’ and false for ‘X’

q Is not sufficient to achieve 100% SC condition coverage

q Have not taken the F arrow from the s[i] <= ‘z’ diamond

q To take that arrow we need a character that is:

q Greater than or equal to ‘a’
q Greater than ‘z’

q We need one such ASCII character

q The test case “aX~” will achieve 100% SC condition coverage

Dr. Ali Javed

 Achieving SC Condition Coverage in
General
26

q In general, for a decision A && B to achieve SC condition coverage,

we need to design test cases to make:
q A true, B true
q A true, B false

q A false

q We will not even be able to get to the evaluation of B unless A is true

as in first two cases (due to SC evaluation)
q Similarly, for a decision A || B to achieve SC condition coverage, we
need to design test cases to make:
q A false, B true
q A false, B false

q A true

q We will not even be able to get to the evaluation of B unless A is false

as in the first two cases

Dr. Ali Javed

 SC Condition Coverage vs. Edge
Coverage
27

q If we have achieved SC condition coverage:

q We must have evaluated each condition of an “if”, “while”,
etc. both ways
q Therefore, we must have evaluated each decision both ways

q Therefore, SC condition coverage is stronger than edge

(decision) coverage

q Is there anything stronger than SC condition coverage?

Dr. Ali Javed

 Strength of Coverage Measures
28

Path (strongest)

SC Condition

Edge
(Decision)

Node
(Statement)

Line (weakest)
Dr. Ali Javed Where arrow from A to B means “B is stronger than A”
 Path Coverage: Basic path testing
29

q First proposed by Tom McCabe.

q Used as a guide for defining a basis set of execution path.

q Guarantee to execute every statement in the program at least one time.

q Basis path testing (McCabe, 1976) is a means for ensuring that all independent paths through
a code module have been tested.

q An independent path is any path through the code that introduces at least one new set of
processing statements or a new condition. (Pressman, 2001)

q Basis path testing provides a minimum, lower-bound on the number of test cases that need to
be written.

Dr. Ali Javed

 Basic path testing- Steps
30

Step 1 : Using the design or code as a foundation, draw a

corresponding flow graph.
Step 2: Determine the cyclomatic complexity of the resultant 1 node
flow graph. a b
Step 3: Determine a basis set of linearly independent paths. edge 2 3
For example, d c

path 1: 1-2-4-5-6-7 4
path 2: 1-2-4-7 e
path 3: 1-2-3-2-4-5-6-7 i 5 g
path 4: 1-2-4-5-6-5-6-7 f region
Step 4: Prepare test cases that will force execution of each 6
path in the basis set.
h
Step 5: Run the test cases and check their results
7

Dr. Ali Javed

 31 Coverage-Example

Dr. Ali Javed

 Example Code

Dr. Ali Javed

 Method Coverage
q In the code shown in previous slide, we attain 100% method coverage by calling the
foo method.

q Consider Test Case 1: the method call foo(0, 0, 0, 0, 0.), expected return value of
0.

q If you look at the code, you see that if a has a value of 0, it doesn’t matter what the
values of the other parameters are – so we’ll make it really easy and make them
all 0. Through this one call we attain 100% method coverage.

Dr. Ali Javed

 Statement Coverage
q In Test Case 1, we executed the program statements on lines 1-5 out of 12 lines of
code.

q As a result, we had 42% (5/12) statement coverage from Test Case 1. We can attain
100% statement coverage by one additional test case,

q Test Case 2: the method call foo(1, 1, 1, 1, 1.), expected return value of 1. With this
method call, we have achieved 100% statement coverage because we have now
executed the program statements on lines 6-12.

Dr. Ali Javed

 Branch Coverage
q The small program in Figure 3 has two decision points – one on line 3 and the other on
line 7.
ü Line 3 if (a == 0) {
ü Line 7 if ((a==b) OR ((c == d) AND bug(a) )) {

q For decision/branch coverage, we evaluate an entire Boolean expression as one true-

or-false predicate even if it contains multiple logical-and or logical-or operators – as in
line 7.

q We need to ensure that each of these predicates (compound or single) is tested as both
true and false. Table 1 shows our progress so far:

Dr. Ali Javed

 Branch Coverage
Table 1: Decision Coverage
Line # Predicate True False
3 (a==0) Test Case 1 Test Case 2
foo(0, 0, 0, 0, 0) foo(1, 1, 1, 1, 1)
return 0 return 1
7 ((a==b)OR((c==d)AND bug(a))) Test Case 2
foo(1, 1, 1, 1, 1)
return 1

q Therefore, we currently have executed three of the four necessary conditions; we have achieved 75% branch
coverage thus far.

q We add Test Case 3 to bring us to 100% branch coverage: foo(1, 2, 1, 2, 1). When we look at the code to
calculate an expected return value, we realize that this test case uncovers a previously undetected division-by-zero
problem on line 10!

q In many cases, the objective is to achieve 100% branch coverage in your testing, though in large systems only 75%-
85% is practical. Only 50% branch coverage is practical in very large systems of 10 million source lines of code or
more (Beizer, 1990).

Dr. Ali Javed

 Condition Coverage
q Condition coverage is a measure of percentage of Boolean sub-expressions of the program that
have been evaluated as both true or false outcome [applies to compound predicate] in test cases.
Notice that in line 7 there are three sub-Boolean expressions to the larger statement (a==b),
(c==d), and bug(a).

q Condition coverage measures the outcome of each of these sub-expressions independently tested
as both true and false. We consider our progress thus far in Table 2.

Table 2: Condition Coverage

Predicate True False

(a==b) Test Case 2 Test Case 3
foo(1, 1, x, x, 1) foo(1, 2, 1, 2, 1) Division by zero
return 0
(c==d) Test Case 3
foo(1, 2, 1, 2, 1) Division by zero
bug(a)

Dr. Ali Javed

 Condition Coverage
q At this point, our condition coverage is only 50%. The true condition (c==d) has never been tested.
Additionally, short-circuit Boolean has prevented the method bug(int) from ever being executed. We
examine our available information on the bug method and determine that is should return a value of true
when passed a value of a=1.

q Write Test Case 4 to address test (c==d) as true: foo(1, 2, 1, 1, 1), expected return value 1.

q However, when we actually run the test case, the function bug(a) actually returns false, which causes our
actual return value (division by zero) to not match our expected return value. This allows us to detect an
error in the bug method. Without the addition of condition coverage, this error would not have been
revealed.

q To finalize our condition coverage, we must force bug(a) to be false. We again examine our bug()
information, which informs us that the bug method should return a false value if fed any integer greater
than 1. So, we create Test Case 5, foo(3, 2, 1, 1, 1), expected return value “division by error”. The
condition coverage thus far is shown in Table 3

Dr. Ali Javed

 Condition Coverage
Table 3: Condition Coverage
Predicate True False
(a==b) Test Case 2 Test Case 3
foo(1, 1, 1, 1, 1) foo(1, 2, 1, 2, 1)
return 0 Division by zero
(c==d) Test Case 4 Test Case 3
foo(1, 2, 1, 1, 1) foo(1, 2, 1, 2, 1)
return 1 Division by zero
bug(a) Test Case 4 Test Case 5
foo(1, 2, 1, 1, 1) foo(3, 2, 1, 1, 1)
return 1 Division by zero

Dr. Ali Javed

 References
40

[1] http://docs.ncover.com/best-practices/code-quality-metrics/

Dr. Ali Javed

 For any query Feel Free to ask
41

Dr. Ali Javed