Module 5 – Validation and

Verification
Strategic Approach to Software Testing, Testing Fundamentals Test Plan, Test Design, Test
Execution, Reviews, Inspection and Auditing – Regression Testing – Mutation Testing -
Object oriented testing - Testing Web based System - Mobile App testing – Mobile test
Automation and tools – DevOps Testing – Cloud and Big Data Testing
Software Testing
What Testing shows
Strategic Approach
• To perform effective testing, you should conduct effective technical
reviews. By doing this, many errors will be eliminated before testing
commences
• Testing begins at the component level and works "outward" toward
the integration of the entire computer based system
• Different testing techniques are appropriate for different software
engineering approaches and at different points in time
• Testing is conducted by the developer of the software and (for large
projects) an independent test group
• Testing and debugging are different activities, but debugging must be
accommodated in any testing strategy
Any testing strategy must incorporate
• Test planning,
• Test case design,
• Test execution,
• Resultant data collection, and
• Evaluation
Validation and Verification
• Verification refers to the set of tasks that ensure that software
correctly implements a specific function.

• Validation refers to a different set of tasks that ensure that

the software that has been built is traceable to customer
requirements.

Verification: "Are we building the product right?“

Validation: "Are we building the right product?"
Who tests the software?
Testing strategy
Testing strategy
Testing Strategy
• We begin by ‘testing-in-the-small’ and move toward ‘testing-in-the-large’

• For Conventional Software

• The module (component) is our initial focus

• Integration of modules follows

• For OO Software

• Our focus when “testing in the small” changes from an individual module (the
conventional view) to an OO class that encompasses attributes and operations
and implies communication and collaboration.
Unit Testing
Unit Testing considerations
Unit Test procedures
Integration Testing Strategies
Top Down Integration
Bottom-Up Integration
Sandwich Testing
Regression Testing
• Regression testing is the re-execution of some subset of tests that have already
been conducted to ensure that changes have not propagated unintended side
effects

• Whenever software is corrected, some aspect of the software configuration (the

program, its documentation, or the data that support it) is changed.

• Regression testing helps to ensure that changes (due to testing or for other
reasons) do not introduce unintended behavior or additional errors.

• Regression testing may be conducted manually, by re-executing a subset of all

test cases or using automated capture/playback tools
Object-Oriented Testing
• Begins by evaluating the correctness and consistency of the analysis and design
models

• Testing strategy changes

• the concept of the ʻunitʼ broadens due to encapsulation

• integration focuses on classes and their execution across a ʻthreadʼ or in the

context of a usage scenario

• validation uses conventional black box methods

• Test case design draws on conventional methods, but also encompasses special

features
OO Testing Strategy
• class testing is the equivalent of unit testing
• operations within the class are tested
• the state behavior of the class is examined
• Integration applied three different strategies
• thread-based testing—integrates the set of classes required to
respond to one input or event
• use-based testing—integrates the set of classes required to
respond to one use case
• cluster testing—integrates the set of classes required to
demonstrate one collaboration
WebApp Testing - I
• The content model for the WebApp is reviewed to uncover errors.

• The interface model is reviewed to ensure that all use cases can be
accommodated.

• The design model for the WebApp is reviewed to uncover navigation

errors.

• The user interface is tested to uncover errors in presentation and/or

navigation mechanics.

• Each functional component is unit tested

WebApp Testing - II
• Navigation throughout the architecture is tested.

• The WebApp is implemented in a variety of different environmental

configurations and is tested for compatibility with each configuration.

• Security tests are conducted in an attempt to exploit vulnerabilities in the

WebApp or within its environment.

• Performance tests are conducted.

• The WebApp is tested by a controlled and monitored population of end-users.

The results of their interaction with the system are evaluated for content and
navigation errors, usability concerns, compatibility concerns, and WebApp
reliability and performance.
Mutation Testing
• It is used to perform new software test and also evaluate the quality of already
existing software tests

• It is related to modification of program in small ways

Objectives of Mutation testing
• To identify pieces of code that are not tested properly.

• To identify hidden defects that can’t be detected using other testing methods.

• To discover new kinds of errors or bugs.

• To calculate the mutation score.

• To study error propagation and state infection in the program.

• To assess the quality of the test cases.

Types of Mutation testing
Value Mutations:

• In this type of testing the values are changed to detect errors in the program.

• Basically a small value is changed to a larger value or a larger value is changed to

a smaller value. In this testing basically constants are changed.
Types of Mutation testing
Decision Mutations:

• In decisions mutations are logical or arithmetic operators are changed to detect

errors in the program.
Types of Mutation testing
Statement Mutations:

In statement mutations a statement is deleted or it is replaces by some other

statement.
Internal and External Views
Testing methods

white-box black-box
methods methods

Methods

Strategies
Unit
Integration
system
29
White-Box Testing

... our goal is to ensure that all

statements and conditions have
been executed at least once ...

30
Traditional White Box Testing
• White box testing is further divided into 2 types

• Basis Path Testing

- Flow Graph

- Cyclomatic Complexity

- Graph Matrix

• Control Structure

- Condition Testing

- Data Flow Testing

- Loop Testing
32
33
Flow Chart

34
Flow Graph

35
36
37
38
 1. Basis Path Testing
Module given for testing -> Flowchart -> Flow graph
Cyclomatic complexity = Edges-Nodes+2 (9-7+2 = 4)
= no. of Predictive nodes + 1 (3+1 = 4)
= No. of Closed path +1 (3+1=4)

First, we compute the cyclomatic

complexity:

number of simple decisions + 1

or

number of enclosed areas + 1

In this case, V(G) = 4

39
 Basis Path Testing

Next, we derive the

independent paths:
2
Since V(G) = 4,
there are four paths
3
4 Path 1: 1,2,3,6,7,8
5 6 Path 2: 1,2,3,5,7,8
Path 3: 1,2,4,7,8
Path 4: 1,2,4,7,2,4,...7,8
Finally, we derive test
7
cases to exercise these
paths.

40
 What is the complexity V(G)?
public void howComplex()
{
int i=20;
while (i<10)
{
System.out.printf("i is %d", i);
if (i%2 == 0)
{
System.out.println("even");
} else
{
System.out.println("odd");
}
i--;
}
}

V(G) = 2 decisions + 1 = 3

Coming up: Output from JavaNCSS

2. Graph Matrix
• A graph matrix is a square matrix whose size
represents the number of nodes in the control flow
graph.
• Each row and column in the matrix identifies a node
and the entries in the matrix represent the edges or
links between these nodes.
• Conventionally, nodes are denoted by digits and
edges are denoted by letters.

42
Convert the control flow graph into a graph matrix. Since the graph has 4 nodes, so the graph
matrix would have a dimension of 4 X 4. Matrix entries will be filled as follows :
(1, 1) will be filled with ‘a’ as an edge exists from node 1 to node 1
(1, 2) will be filled with ‘b’ as an edge exists from node 1 to node 2. It is important to note
that (2, 1) will not be filled as the edge is unidirectional and not bidirectional
(1, 3) will be filled with ‘c’ as edge c exists from node 1 to node 3
(2, 4) will be filled with ‘d’ as edge exists from node 2 to node 4
(3, 4) will be filled with ‘e’ as an edge exists from node 3 to node 4
Flow Graph Graph Matrix Connection Matrix

A connection matrix is a matrix defined with edges weight. In simple form, when a connection
exists between two nodes of control flow graph, then the edge weight is 1, otherwise, it is 0.
However, 0 is not usually entered in the matrix cells to reduce the complexity. 43
A connection matrix is used to find the cyclomatic complexity of the control graph.
Following are the steps to compute the cyclomatic complexity :
• Count the number of 1s in each row and write it in the end of the row
• Subtract 1 from this count for each row (Ignore the row if its count is 0)
• Add the count of each row calculated previously
• Add 1 to this total count
• The final sum in Step 4 is the cyclomatic complexity of the control flow graph

Cyclomatic Complexity

e-n+2=5-4+2=3
P+1 = 2 +1 = 3
R +1 = 2 +1 = 3

Region 1: bounded by edges b, c, d, and e

Region 2: bounded by edge a (in loop)

Path 1: 1, 2, 4
Path 2: 1, 3, 4
Path 3: 1, 1, 3, 4
44
 Graph Matrix Connection Matrix Cyclomatic Complexity Calc

1 2 3 4 5 1 2 3 4 5 1 2 3 4 5
1 a 1 1 1 1 1-1= 0
2 2 2
3 d b 3 1 1 3 1 1 2-1 = 1
4 c f 4 1 1 4 1 1 2-1 = 1
5 g e 5 1 1 5 1 1 2-1 = 1
CC = 0=+1+1+1 + 1 = 4
1
Flow Graph a
Path 1: 1, 3, 4, 2
e 3 Path 2: 1, 3, 4, 5, 2
Path 3: 1, 3, 4, 5, 3
b
5 f Path 4: 1, 3, 2
4
d
g c

2
45
3. Control Structure Testing
Control structure testing is used to increase the
coverage area by testing various control structures
present in the program. The different types of testing
performed under control structure testing are as
follows
i. Condition Testing - a test case design method that exercises the
logical conditions contained in a program module
ii. Data Flow Testing - selects test paths of a program according to
the locations of definitions and uses of variables in the program
iii. Loop Testing - It specifically focuses on the validity of loop
construction.

46
 i. Condition Testing
Condition testing is a test case design method, which ensures that the logical condition and
decision statements are free from errors.
The errors present in logical conditions can be incorrect Boolean operators, missing
parenthesis in a Boolean expression, error in relational operators, arithmetic expressions,
and so on.
The common types of logical conditions that are tested using condition testing are -

1. A relation expression, like E1 op E2 where ‘E1’ and ‘E2’ are arithmetic expressions and
‘OP’ is an operator.

2. A simple condition like any relational expression preceded by a NOT (~) operator.
For example, (~E1) where ‘E1’ is an arithmetic expression and ‘ ~ ’ denotes NOT
operator.

3. A compound condition consists of two or more simple conditions, Boolean operator,

and parenthesis.
For example, (E1 & E2)|(E2 & E3) where E1, E2, E3 denote arithmetic expression
and ‘&’ and ‘|’ denote AND or OR operators.

4. A Boolean expression consists of operands and a Boolean operator like ‘AND’, OR,
NOT.
For example, ‘A|B’ is a Boolean expression where ‘A’ and ‘B’ denote operands
and | denotes OR operator. 47
 ii. Data Flow Testing
The data flow test method chooses the test path of a program
based on the locations of the definitions and uses all the
variables in the program.

48
iii. Loop Testing

Simple
loop
Nested
Loops

Concatenated
Loops Unstructured
Loops
49
Simple Loop – The following set of test can be applied to simple loops, where the
maximum allowable number through the loop is n.
• Skip the entire loop.
• Traverse the loop only once.
• Traverse the loop two times.
• Make p passes through the loop where p<n.
• Traverse the loop n-1, n, n+1 times.
Concatenated Loops – If loops are not dependent on each other, contact loops can
be tested using the approach used in simple loops. if the loops are
interdependent, the steps are followed in nested loops
Nested Loops – Loops within loops are called as nested loops. when testing nested
loops, the number of test increases as level nesting increases.
The following steps for testing nested loops are as follows-
• Start with inner loop. set all other loops to minimum values.
• Conduct simple loop testing on inner loop.
• Work outwards.
• Continue until all loops tested.
Unstructured loops – This type of loops should be redesigned, whenever possible,
to reflect the use of unstructured the structured programming
constructs.
50
51
52
Black-Box Testing

requirements

output

input events

53
Black Box Testing
• Black-box testing, also called behavioral testing, focuses on the functional
requirements of the software.

• That is, black-box testing enables the software engineer to derive sets of input
conditions that will fully exercise all functional requirements for a program.

• Black-box testing is not an alternative to white-box techniques. Rather, it is a

complementary approach that is likely to uncover a different class of errors than
white-box methods.

54
Black Box Testing
➢ Black-box testing attempts to find errors in the following
categories:
1. incorrect or missing functions,
2. interface errors,
3. errors in data structures or external data base access,
4. behavior or performance errors, and
5. initialization and termination errors.

➢ Stages of Black Box Testing

1. Graph based Testing
2. Equivalence Partitioning
3. Boundary Value Analysis

55
1.

56
57
58
2. Equivalence partitioning
• Equivalence partitioning is a black-box testing method
that divides the input domain of a program into
classes of data from which test cases can be derived.

• Test case design is based on an evaluation of

equivalence classes for an input condition

• An equivalence class represents a set of valid or

invalid states for input conditions

• From each equivalence class, test cases are selected

so that the largest number of attributes of an
equivalence class are exercise at once.
59
• If an input condition specifies a range, one valid and two invalid
equivalence classes are defined
Input range: 1 – 10 Eq classes: {1..10}, {x < 1}, {x > 10}

• If an input condition requires a specific value, one valid and two

invalid equivalence classes are defined
Input value: 250 Eq classes: {250}, {x < 250}, {x > 250}

• If an input condition specifies a member of a set, one valid and

one invalid equivalence class are defined
Input set: {-2.5, 7.3, 8.4} Eq classes: {-2.5, 7.3, 8.4}, {any other x}

• If an input condition is a Boolean value, one valid and one invalid

class are define
Input: {true condition} Eq classes: {true condition}, {false Condition}

60
3. Boundary Value Analysis
• A greater number of errors occur at the boundaries of
the input domain rather than in the "center“

• Boundary value analysis leads to a selection of test

cases that exercise bounding values.

• Boundary value analysis is a test case design method

that complements equivalence partitioning
➢ It selects test cases at the edges of a class
➢ It derives test cases from both the input domain
and output domain

61
Guidelines for Boundary Value Analysis

1. If an input condition specifies a range bounded by values a and b, test cases

should be designed with values a and b as well as values just above and just below
a and b

2. If an input condition specifies a number of values, test case should be

developed that exercise the minimum and maximum numbers. Values just above
and just below the minimum and
maximum are also tested

3. Apply guidelines 1 and 2 to output conditions; produce output that reflects the
minimum and the maximum values expected; also test the values just below and
just above

4. If internal program data structures have prescribed boundaries (e.g., an array),

design a test case to exercise the data structure at its minimum and maximum
boundaries

62
Boundary Value Analysis

Consider a program with two input variables x and y. These input variables have specified boundaries as:

a≤x≤b
c≤y≤d
Input domain

d
y
c

a b
x
Input domain for program having two input variables
63
The boundary value analysis test cases for our program with two inputs
variables (x and y) that may have any value from 100 to 300 are:
(200,100), (200,101), (200,200), (200,299), (200,300), (100,200),
(101,200), (299,200) and (300,200). This input domain is shown. Each
dot represent a test case and inner rectangle is the domain of
legitimate inputs. Thus, for a program of n variables, boundary value
analysis yield 4n + 1 test cases.
Input domain
400
300
y 200
100

0 100 200 300 400

x
Input domain of two variables x and y with boundaries [100,300] each
64
Advantage’s of Black Box Testing

• More effective on larger units of code than glass box testing

• Tester needs no knowledge of implementation, including specific

programming languages

• Tester and programmer are independent of each other Tests are

done from a user's point of view

• Will help to expose any ambiguities or inconsistencies in the

specifications

• Test cases can be designed as soon as the specifications are

complete
65
Disadvantages of Black Box Testing
• Only a small number of possible inputs can actually be
tested, to test every possible input stream would take
nearly forever

• Without clear and concise specifications, test cases are

hard to design

• There may be unnecessary repetition of test inputs if

the tester is not informed of test cases the programmer
has already tried

66