Introduction

⚫ Making different modules work

together.
⚫ Both together are combined
and tested to evaluate
interactions between them.
⚫ Black box testing
⚫ Integration testing is done to test
the modules/components when
integrated to verify that they
work as expected .
 Integration testing
strategies
 Big Bang: Involves integrating the modules to build a complete
software system. This is considered a high-risk approach because
it requires proper documentation to prevent failure.
 Contd,…

 Top-Down: Involves testing the top integrated modules first.

Subsystems are tested individually. Top-down testing facilitates
detection of lost module branch links.
 Contd,…

 Bottom-Up: Involves low-level component testing, followed by

high-level components. Testing continues until all hierarchical
components are tested. Bottom-up testing facilitates efficient
error detection.
Software Testing Hierarchy
 System Testing

⚫ SYSTEM TESTING is a level of testing that validates

the complete and fully integrated software product.
⚫ The purpose of a system test is to evaluate the end-to-end system
specifications.
⚫ Types of System testing
 Load testing
 Usability Testing
 Regression Testing
Contd,…
 Usability Testing

⚫ Usability Testing can be defined as the kind of testing performed

from the end user's point of view for determining whether the
application you are developing is effortlessly usable or not.
 Acceptance Testing

⚫ A testing technique performed to determine whether or not the

software system has met the requirement specifications.
⚫ The main purpose of this test is to evaluate the system's
compliance with the business requirements and verify if it is has
met the required criteria for delivery to end users.
⚫ There are various forms of acceptance testing:
 User acceptance Testing
 Business acceptance Testing
 Alpha Testing
 Beta Testing
Acceptance Testing - In SDLC
 Regression Testing

⚫ Regression Testing is nothing but a full or partial selection of

already executed test cases which are re-executed to ensure
existing functionalities work fine.
⚫ This testing is done to make sure that new code changes should
not have side effects on the existing functionalities.
⚫ It ensures that the old code still works once the latest code
changes are done.
 Need of Regression Testing

Regression Testing is required when there is a

⚫ Change in requirements and code is modified according to
the requirement
⚫ New feature is added to the software
⚫ Defect fixing
⚫ Performance issue fix
 How to do Regression Testing

⚫ Software maintenance is an activity which includes

enhancements, error corrections, optimization and deletion of
existing features. These modifications may cause the system to
work incorrectly. Therefore, Regression Testing becomes
necessary. Regression Testing can be carried out using the
following techniques:
 Regression Test Selection

⚫ This is a way of selecting the most appropriate test case to ensure

that it delivers the correct result.
⚫ It thus enables a tester to select a valid set of test cases to make
sure the unmodified part of the program is error free.
⚫ So we can categorize the concept of test selection for regression
testing into two broad categories:
Identify the affected parts of the program - When a particular
module is changed, other parts of the program are bound to be
affected. It is time for the identification of bugs in the unmodified
parts of the program.
Test Case Selection -Involves a selection of "test cases" from an
already defined "test suite".
 Contd,…

Regression test selection can further be categorized into three

categories:
⚫ Coverage Techniques: It is based on "coverage" criteria. These
include coverable components for the modified portion in the
program.
⚫ Minimization Technique: Works similar to a coverage technique
however, it is based on the selection of a minimum number of test
cases.
⚫ Safe Techniques: Unlike coverage technique, it selects those test
cases that cause a modified program to produce different output
other than the original program.
 Regression Test Selection Techniques

⚫ The parameters which form the basis of comparison and

evaluation of the selection techniques are as follows:
⚫ Inclusiveness: Analyses the extent to which a test case selection
will cause the modified program to produce different output other
than the original. Helps to capture faults caused as a result of
modifications.
⚫ Precision Those test cases are not selected that won't produce a
different output than the original one.
⚫ Efficiency: Calculates the cost of computing and practicality of
adopting a technique.
⚫ Generality: Ability of a technique to handle varied language
constructs, complex changes in the system.
 Test case prioritization

⚫ Test Prioritization is an extension of software testing, which is

implemented to decide the priority of the test cases.
⚫ This priority of the test cases is decided through various
factors that depend upon the need of the software, which are then
assigned to the test cases for further processes.
⚫ The term "test prioritization" refers to the subjective and difficult
part of testing that allows testers to manage risks, plan tests,
consider cost value, and be analytical about which test to run in
the context of the specific project.
 Contd,…

⚫ This process, is well known as Test Case Prioritization,

which is a method of prioritizing and scheduling test cases.
⚫ This techniques is used in order to run test cases of higher
priority in order to minimize time, cost and efforts
during software testing phase.
 Contd,…

The aspects that are required to be considered while prioritizing

test cases are:
⚫ The tester should consider the usage frequency of a function or
the probability of failure in software use.
⚫ The failure risks of the test cases should be measured.
⚫ Visibility of a failure for the end user is a further criteria
for
prioritization of the test cases.
⚫ Test cases can be chosen as per the priority of the requirements of
all the stakeholders.
 Contd,…

⚫ Quality characteristics may have different importance to

the customer or client.
⚫ Prioritization can also be done from the perspective of
the development or system architecture.
⚫ Areas with complex coding and with past histories of problems
should be taken into account.
 Benefits Of Test Case Prioritization

⚫ Improves the performance of the regression testing.

⚫ Arranges test cases in an order, to obtain maximum available
fault that is detected in a shorter time.
⚫ In this, the priority is given to the test cases that are performed
based on the multiple criteria and hybrid criteria, to enhance the
effectiveness of the time and cost for proposed technique.
⚫ It also ensures that the most important test cases are executed
early in the Software Development Life Cycle (SDLC).
⚫ Allows testers to find defects and errors in the system as soon as
possible.
 Test Case Prioritization Techniques

 Prioritization Based on Average Percentage Fault Detected:

 Prioritization Using Faulty Severity:
⚫ The some of the factors that decide the weight of the requirement
are:
 Business value measure.

 Project change volatility.

 Development complexity.

 Fault proneness of requirement.

 Prioritization in Case of Regression Testing:

 Cond,…

 Random Prioritization:
 Optimal Prioritization:
 Total statement coverage prioritization:
 Additional statement coverage prioritization:
 Total branch coverage prioritization:
 Total branch coverage prioritization:
 Total fault-exposing potential prioritization:
 Additional fault-exposing potential
Prioritization:
 Challenges in Test Prioritization:

⚫ Detecting change and affected parts of the program.

⚫ Scalability to handle complex systems.
⚫ Integrating seamlessly into development process.
 Overview of a test selection
method

Step 1: Given P and test set T, find the execution trace of P for each test in T.

Step 2: Extract test vectors from the execution traces for each node in the
CFG of P

Step 3: Construct syntax trees for each node in the CFGs of P and P’. This step
can be executed while constructing the CFGs of P and P’.

Step 4: Traverse the CFGs and determine the a subset of T appropriate for
regression testing of P’.
 Execution Trace [1]

Let G=(N, E) denote the CFG of program P. N is a finite set of

nodes and E a finite set of edges connecting the nodes. Suppose
that nodes in N are numbered 1, 2, and so on and that Start and
End are two special nodes as discussed in Chapter 1.

Let Tno be the set of all valid tests for P’. Thus, Tno contains only
tests valid for P’. It is obtained by discarding all tests that have
become obsolete for some reason.
 Execution Trace [2]

An execution trace of program P for some test t in Tno is the sequence of nodes in
G traversed when P is executed against t. As an example, consider the following
program.
 Execution Trace [3]

Here is a CFG for our example program

 Execution Trace [4]

Now consider the following set of three tests and the corresponding
trace.
 Test vector

A test vector for node n, denoted by test(n), is the set of tests that
traverse node n in the CFG. For program P we obtain the
following test vectors.
 Syntax trees

A syntax tree is constructed for each node of CFG(P) and

CFG(P’). Recall that each node represents a basic block. Here
sample syntax trees for the example program.
 Test selection [1]

Given the execution traces and the CFGs for P and P’, the
following three steps are executed to obtain a subset T’ of T
for regression testing of P’.
 Test selection [2]

The basic idea underlying the SelectTests procedure is to traverse

the two CFGs from their respective START nodes using a
recursive descent procedure.

The descent proceeds in parallel and the corresponding nodes are

compared. If two two nodes N in CFG(P) and N’ in CFG( P’)
are found to be syntactically different, all tests in test (N) are
added to T’.
 Test selection example

Suppose that function g1 in P is modified as follows.

Try the SelectTests algorithm and check if you get T’={t1,

t3}.
 Issues with Select Tests

Think:

What tests will be selected when only, say, one declaration is

modified?

Can you think of a way to select only tests that correspond to

variables in the modified declaration?
 Levels Of Testing, Integration Testing

Traditional View of Testing Levels

⚫ The traditional model of software development is the Waterfall
model, which is drawn as a V in.
⚫ In this view, information produced in one of the development
phases constitutes the basis for test case identification at that
level.
⚫ Two observations: there is a clear presumption of functional
testing here, and there is an implied “bottom-up” testing order.
 Alternative Life Cycle Models

 Decomposition is a perfect fit both to the top-down

progression of the waterfall model and to the bottom-up testing
order.
 Functional decomposition can only be well done when the system
is completely understood, and it promotes analysis to the near
exclusion of synthesis.
 The result is a very long separation between requirements
specification and a completed system, and during this interval,
there is no opportunity for feedback from the customer.
 Contd,…

Waterfall Spin-offs
⚫ There are three mainline derivatives of the waterfall model:
incremental development, evolutionary development, and the
Spiral model.
 Specification Based Models
 An Object-Oriented Life Cycle Model
 Formulations of the SATM System

⚫ Simple Automatic Teller Machine (SATM) system.

⚫ there are function buttons B1, B2, and B3, a digit keypad with a
cancel key, slots for printer receipts and ATM cards, and doors for
deposits and cash withdrawals.
⚫ The SATM system is described here in two ways: with a
structured analysis approach, and with an object-oriented
approach.
⚫ These descriptions are not complete, but they contain detail
sufficient to illustrate the testing techniques under discussion
SATM with Structured
Analysis
 Contd,…

⚫ The structured analysis approach to requirements specification is

the most widely used method in the world.
⚫ The technique is based on three complementary models: function,
data, and control.
⚫ Here we use data flow diagrams for the functional models,
entity/relationship models for data, and finite state machine
models for the control aspect of the SATM system.
⚫ The portions of the SATM system shown here pertain generally
to the personal identification number (PIN) verification portion
of the system.
Contd,…

Screens for the SATM

System
 Separating Integration and System Testing

⚫ We need this distinction to avoid gaps and redundancies across

levels of testing, to clarify appropriate goals for these levels, and
to understand how to identify test cases at different levels.
⚫ A thread is a construct that refers to execution time behavior;
when we test a system, we use test cases to select (and execute)
threads.
⚫ For example, we might think of a thread as a path through a finite
state machine description of a system, or we might think of a
thread as something that is determined by a data context and a
sequence of port level input events, such as those in the context
diagram of the SATM system.
 A Closer Look at the SATM
System
⚫ This decomposition is the basis for the usual view of integration
testing.
⚫ It is important to remember that such a decomposition is
primarily a packaging partition of the system.
⚫ As software design moves into more detail, the added
information lets us refine the functional decomposition tree into a
unit calling graph.
⚫ That information is captured in the adjacency matrix given below
in Table 2.
Contd,…
 Decomposition Based Integration

We can dispense with the big bang approach most easily: in this
view of integration, all the units are compiled together and tested
at once.
The drawback to this is that when (not if!) a failure is observed,
there are few clues to help isolate the location(s) of the fault.
 Contd,…

Top-Down Integration
 Top-down integration begins with the main program (the root of
the tree). Any lower level unit that is called by the main program
appears as a “stub”, where stubs are pieces of throw-away code
that emulate a called unit.
⚫ first step would be to develop stubs for all the units called by the
main program: Watch Card Slot, Control Card Roller, Screen
Driver, Validate Card, Validate PIN, Manage Transaction
Contd,…
 Contd,…

Bottom-up Integration
⚫ Bottom-up integration is a “mirror image” to the top-down order,
with the difference that stubs are replaced by driver modules that
emulate units at the next level up in the tree.
⚫ In bottom-up integration, we start with the leaves of the
decomposition tree (units like ControlDoor and DispenseCash),
and test them with specially coded drivers.
Sandwich Integration
 Sandwich integration is a combination of top-down and bottom-
up integration. If we think about it in terms of the decomposition
tree, we are really just doing big bang integration on a sub-tree.
Contd,…
Sandwich Integration
 Call Graph Based Integration

⚫ One of the drawbacks of decomposition based integration is that

the basis is the functional decomposition tree.
⚫ If we use the call graph instead, we mitigate this deficiency; we
also move in the direction of behavioral testing.
⚫ Pair-wise Integration
⚫ Neighborhood Integration
Path Based Integrations
 Contd,…

⚫ Wanted: an integration testing level construct

⚫ similar to DD-Paths for unit testing...
⚫ – extend the symbiosis of spec-based and code-based
⚫ testing to the integration level
⚫ – greater emphasis on behavioral threads
⚫ – shift emphasis from interface testing to interactions
(co-
⚫ functions) among units
⚫ • Need some new definitions
⚫ – source and sink nodes in a program graph
⚫ – module (unit ) execution path
⚫ – generalized message
 Contd,…

⚫ A source node in a program is a statement fragment at which

program execution begins or resumes.
⚫ A sink node in a unit is a statement fragment at which program
execution terminates.
⚫ A module execution path is a sequence of statements that
begins with a source node and ends with a sink node, with no
intervening sink nodes.
⚫ A message is a programming language mechanism by which one
unit transfers control to another unit, and acquires a response
from the other unit.
 Contd,…

⚫ The first guideline for MM-Paths: points of quiescence are

“natural” endpoints for an MM-Path. Our second guideline also
serves to distinguish integration from system testing.
⚫ Our second guideline: atomic system functions are an upper limit
for MM-Paths: we don’t want MM-Paths to cross ASF
boundaries.
Contd,…
 Details of the Example MM-Path

⚫ The node sequence in the example MM-Path is:

⚫ <a1, a2, a3, a4>
⚫ message msg1
⚫ <b1, b2>
⚫ message msg2
⚫ <c1, c2, c4, c5, c6, c8, c9>
⚫ msg2 return
⚫ <b3, b4, (b2, b3, b4)*, b5>
⚫ msg1 return
⚫ <a6, a7, a8>
⚫ Note: the (b2, b3, b4)* is the Kleene Star notation for repeated
traversal of the loop.