Software Testing

Chapter 8
Definition
Testing traditionally mean to executing program code
Software testing is done by set of people whose goal is to uncover the
defects in the product before it reaches the customer.
Basically, software testing is the process of executing software in a
controlled manner in order to answer the question “Does the software
behave as specified?”
Simply, it is the process of analyzing a program with the intent of
finding errors.
During testing, the program to be tested is executed with set of
test cases and the output of the program for the test cases is
evaluated.
The evaluation determines whether the program is performing
as expected or not.
Software testing is a complex process which cannot be done instantly.
Since testing covers 30-40% of development cost, the testing process is
carried out in 4 steps.
1. Unit Testing
2. Integration Testing
3. System Testing
4. Acceptance Testing
Unit Testing
It refers to the process of testing modules against the detailed
design specifications.
The inputs to the unit testing are the compiled success modules.
The output of unit testing is the verification of each module is
against the detail design.
In unit testing following checklist are considered.
 Interface: the module interface is tested to ensure that information
properly flows into and out of the program.
Local Data Structures: the local data structure is examined to
ensure that data stored temporarily maintains its integrity.
Boundary Conditions: boundary conditions are tested to ensure
that the module operates properly to the boundaries established to limit
or restrict processing.
Error handling: finally error handling is checked to ensure
description of error, error respondents.
Procedure of Unit Testing
First of all, a review of the design information is used to guide for the
development of test cases.
Each test case should be linked with a set of anticipated results.
Driver and stub are produced for each unit test. Driver and stub are
designed to provide the complete environment for a module.
Stub Driver
A stub procedure is a dummy ● A driver module would contain
procedure that has the same the non-local data structure
input/output parameters of given accessed by the module under
procedure but has a very simplified test and would also have the
behaviour. Stub server to replace code to call the different
modules that are called by the functions of the module with
components to be tested. appropriate values.
Example
Consider a module with units: Login, Home and User
Stub:
Suppose login module is ready for test, but the two minor modules Home and User, which
are called by Login module are not ready yet for testing. At this time, a piece of dummy
code is written, which simulates the called methods of Home and User. These dummy
pieces of code are the stubs.

Driver:
Taking the same example as above. Suppose this time, the User and Home modules are
ready, but the Login module is not ready to test. Now since Home and User return values
from Login module, so a dummy piece of code is written, which simulates the Login
module. This dummy code is then called Driver.
Integration Testing
● After all the individual units have been tested, there is a need to
test how they are to be combined to prevent data loss.
● We need to ensure that one module does not have adverse impact
on another and a function perform incorrectly.
● This is where integration testing comes into play.
● Basically, integration testing is a systematic technique for
constructing the program structure while at the same time
conducting tests to uncover errors associated with interfacing
between modules.
● The primary objective is to test the module that there is no errors
in parameter passing, when one module invokes another.
● Integration testing takes as its input modules that have been unit
tested, groups them in larger aggregates, applies tests defined in an
integration test plan to those aggregates, and delivers as its output
the integrated system ready for system testing.
Approaches to Integration Testing
● Big Bang Approach
● Incremental Approach
● Top-down Integration Approach
● Bottom-up Integration Approach
● Sandwich Integration Approach
● Regression
Big Bang Approach
● In big-bang, most of the developed modules are coupled together to form
a complete software system or major part of the system and then used for
integration testing.
● This method is very effective for saving time in the integration testing
process.
● However, if the test cases and their results are not recorded properly, the
entire integration process will be more complicated and may prevent the
testing team from achieving the goal of integration testing.
● This method require near perfect unit testing because the bugs discovered
would be the defects due to component interfacing
Incremental Approach
● Each individually tested modules are integrated incrementally and
tested to ensure smooth interface and interaction between
modules.
● In this approach, every module is combined incrementally, i.e., one
by one till all modules or components are added logically to make
the required application, instead of integrating the whole system at
once and then performing testing on the end product.
● Integrated modules are tested as a group to ensure successful
integration and data flow between modules.
● Here, the primary focus of doing this testing is to check interface,
integrated links, and flow of information between modules.
● This process is repeated till the modules are combined and tested
successfully.
● Incremental approach can be further classified as
1. Top-down Incremental Testing
2. Bottom-up Incremental Testing
3. Sandwich Testing
Top Down Integration Testing
● It is a method in which integration testing takes place from top to
bottom following the control flow of software system.
● The higher level modules are tested first and then lower level
modules are tested and integrated in order to check the software
functionality.
● Stubs are used for testing if some modules are not ready.
● Depth First Search and Breadth First Search can be applied for this
purpose.
● Module: Website Login aka L
● Module: Order aka O
○ Module Order Summary aka OS
(Not yet developed)
● Module: Payment aka P
○ Module Cash Payment aka CP
○ Module Debit/Credit Payment
aka DP (Not yet developed)
○ Module Wallet Payment aka WP
(Not yet developed)
● Module: Reporting aka R (Not yet
developed)
Process Explanation
● Here, the main control module is used as test driver and stubs are
substituted for all components directly called by main module
● Depending on the strategy (DFS or BFS), subordinate stub are replaces
one at a time with actual components
● Tests are conducted as each unit is integrated
● Following test cases will be derived for “depth-first”:
○ Test Case1: Module L and Module O will be integrated and tested
Test Case2: Module L, O and OS will be integrated and tested
Test Case3: Module L, O, OS, P will be integrated and tested
Test Case4: Module L, O, OS, P, CP will be integrated and tested
Advantages Disadvantages
1. Early exposure of 1. Significant modules are
architecture defects tested late in cycle
2. It outlines the working of 2. It is very challenging to
an application as a whole write test conditions
in early stages and helps in 3. A stub is not a perfect
early disclosure of design implementation of related
defects Module. It just simulates
3. Main control points are the data flow between two
tested early modules
Bottom-Up Integration Testing
● It is a strategy in which the lower level modules are tested first.
● These tested modules are then further used to facilitate the testing
of higher level modules.
● The process continues until all modules at top level are tested.
● Once the lower level modules are tested and integrated, then the
next level of modules are formed.
Process Explanation with an example
● Modules: Rank, Marks, Percentage and Sports
Grade are not yet developed so they will be
substituted with related Driver
● Modules: Practical and Theory - integration

● Test Case1: Unit testing of module Practical and

Theory
Test Case2: Integration and testing of Modules
Marks-Practical-theory
Test Case3: Integration and testing of Modules
Percentage-Marks-Practical-Theory
Test Case4: Unit testing of Module Sports Grade
Test Case5: Integration and testing of Modules
Rank-Sports Grade-Percentage-Marks-Practical-
Theory
Advantages Disadvantages
1. This methodology is very useful 1. Drivers are more difficult to write
for applications where bottom up than stub
design model is used 2. Design defects are caught in the
2. It’s easier to create test conditions later stage
in bottom up approach 3. In this approach, we do not have
3. To start testing at the bottom level working application until the last
of hierarchy means testing of module is build
critical modules or functionality at 4. Driver is not a complete
an early stage. This helps in early implementation of related
discovery of errors Module. It just simulates the data
4. Interface defects are detected at flow between two modules.
early stage
Sandwich Testing
● This approach is a hybrid of top-down and bottom-up methodology.
● Stub and drivers are used for incomplete or not developed modules.
● A middle layer is identified from which bottom-up and top-down testing
are done. This middle layer is also known as target layer
● Target layer is identified as per Heuristic approach, i.e., select a layer
which allows minimal use of Stubs and drivers
● Top-down testing starts from middle layer and moves downwards
towards lower level modules. This layer below middle layer is known as
Bottom layer
● Bottom-up testing also starts from middle layer and move up towards top
layer modules. This layer above middle layer is known as Top layer
● With use of stubs and drivers, user interface
and functions of lower level modules are
tested respectively
● In the end, only middle layer is left for the
execution of the final test

● Test Case1: Test A, X, Y, and Z individually –

where Test A comes under Top layer test and
Test X, Y and Z comes under Bottom layer
tests
Test Case2: Test A, G, H and I
Test Case3: Test G, X, and Y
Test Case4: Test Hand Z
Test Case5: Test A, G, H, I, X, Y, and Z
Advantages Disadvantages
1. It is very beneficial for a big 1. Before module unification,
project which has various subsystems and their interfaces
sub-projects are not tested thoroughly
2. Top-down and bottom-up 2. Higher cost due to involvement
testing methodology can run of both top-down and bottom-up
side by side testing methodology
3. This type of testing is not
advised for a system where
modules are highly inter-
dependent
System Testing
● After completing the integration of the modules, we obtain a final system
● The final system must comprehend the required functionalities as
specified in the system design.
● In other word, the system must produce desired output with given inputs
as a whole.
● For this we need to perform system testing.
● System testing is done to evaluate system compliance with specified
requirements on functional and non-functional requirements.
● It is the only testing where functional as well as non-functional
requirements are tested.
Types of System Testing

Functional based Testing Non-functional Based Testing

1. White Box Testing 1. Performance/ Load Testing
i. Basic Path Testing 2. Recovery Testing
ii. Structural Testing 3. Scalability Testing
iii. Logic Based Testing 4. Security Testing
iv. Fault Based Testing 5. Stress Testing
2. Black Box Testing 6. Reliability Testing
i. Requirement Based Testing 7. Interoperability Testing
ii. Positive and Negative Testing 8. Localization Testing
iii. Boundary Value Analysis
iv. Equivalence Partitioning Testing
1. White Box Testing
● White Box Testing is software testing technique in which internal
structure, design and coding of software are tested to verify flow of
input-output and to improve design, usability and security.
● It examines and checks the program code that realizes external
functionality
● It takes into account the program code, code structure and the
internal design
● It ensures that all independent path within a module have been
exercised at least once
● It verifies all logical decision return either true or false
● It checks all loops are executed at their boundaries and within
their operational bounds internal data structures validity
● Since, each line of code and paths are thoroughly executed, it is
also called as clear box, glass box or open box testing
● White box testing in software engineering is based on the inner
workings of an application and revolves around internal testing.
White box testing includes the following code coverage
● Control Flow testing
● Data flow testing
● Branch testing
● Statement coverage
● Decision coverage
● Modified condition/decision coverage
● Prime path testing
● Path testing
Basic Path Testing
● Allows the design and definition of a basic set of execution paths
● Here each possible path are examined by executing program in
such a way that each statement is executed at least once
● The control structures are represented by flow graph and
cyclomatic complexity is used to determine number of
independent path
Structural Testing
● It examines the source code and analyzes what is present in the code.
● Structural testing is dynamic in nature i.e. the code is executed during
analysis
● For the structural testing, the test cases are derived from the analysis of
the program control flow.
● It covers
○ Statement coverage
○ Branch coverage
○ Condition & Loop coverage
○ Path coverage
○ Data flow testing
○ Domain and boundary testing
Logic Based Testing
● Used when the input domain and resulting processing are
responsive to a decision table representation
● Logic based testing follows following steps:
○ List all actions that can be associated with a specific algorithm
○ List all conditions during execution of the algorithm
○ Associate specific conditions with specific actions eliminating
impossible combinations of conditions
○ Define rules by indicating what actions occur for a set of condition
Fault-based Testing
● It is used to show the absence of certain classes of faults in code
● The code is actually analyzed for uninitialized variables, unreferenced
variables, parameter type checking, etc.
● The main technique is called mutation analysis
● Mutation analysis is a fault based testing for determining the adequacy of
attest suite in terms of its effectiveness
● A mutant is a copy of the original test program with one component, such
as an operand or operator altered to simulate a syntactically correct
programming fault
● Eg: while(index > 10) could be mutated to while(index >=10)
● Mutation analysis simulates simple programming errors
Advantages Disadvantages
1. Code optimization by finding 1. White box testing can be quite complex
hidden errors. and expensive.
2. White box tests cases can be 2. White box testing requires professional
resources, with a detailed understanding
easily automated.
of programming and implementation.
3. Testing is more thorough as
3. White-box testing is time-consuming,
all code paths are usually bigger programming applications take
covered. the time to test fully.
4. It can improve stability and 4. Test cases focuses on software that
reusability of test cases exists, if any functionality missed may
provided that object remains not be discovered.
the same.
2. Black Box Testing
● Black Box Testing is a software testing method in which the
functionalities of software applications are tested without having
knowledge of internal code structure, implementation details and internal
paths.
● Black Box Testing mainly focuses on input and output of software
applications and it is entirely based on software requirements and
specifications.
● It is also known as Behavioural Testing, closed box testing.
● Through black box testing errors like incorrect function, data structure
errors, missing function, external database access errors, etc. are
discovered.
Requirement Based Testing
● Deals with validating the requirements given in the SRS of the software system.

Positive and Negative Testing

● Positive testing tries to prove that a given product does what is supposed to do.
● Positive test checks the product behavior for specified positive and negative
conditions.
● Negative testing is done to show that the product does not fail when an
unexpected input is given
● Negative testing covers the scenario for which the values may not have been
represented or coded.
Boundary Value Testing
● Boundary value testing is focused on the values at boundaries.
● This technique determines whether a certain range of values are
acceptable by the system or not.
● It is very useful in reducing the number of test cases.
● It is most suitable for the systems where an input is within certain
ranges.
Equivalence Partitioning Testing
● Partition is set of input values that generate one single expected
output.

● It is a software testing technique that involves identifying a small

set of representative input values that produce as many different
output condition as possible.

● It is used to minimize the number of possible test cases to an

optimum level while maintains reasonable test coverage.
Advantages Disadvantages
1. Well suited and efficient for large 1. Limited coverage, since only a
code segments. selected number of test scenarios
2. Code access is not required. is actually performed.
3. Clearly separates user's 2. Inefficient testing, due to the fact
perspective from the developer's that the tester only has limited
perspective through visibly knowledge about an application.
defined roles. 3. Blind coverage, since the tester
4. Large numbers of moderately cannot target specific code
skilled testers can test the segments or error prone areas.
application with no knowledge of 4. There may be repetition of test
implementation, programming cases if the tester has not been
language, or operating systems. informed
Gray Box Testing
● Grey Box Testing or Gray box testing is a software testing technique to
test a software product or application with partial knowledge of internal
structure of the application.
● The purpose of grey box testing is to search and identify the defects due
to improper code structure or improper use of applications.
● In this process, context-specific errors that are related to web systems are
commonly identified.
● It increases the testing coverage by concentrating on all of the layers of
any complex system.
● Gray Box Testing is a software testing method, which is a combination of
both white box testing and black box testing method.
Acceptance Testing
● After the system has been tested for its functionality and non-
functionality, now its time for the final testing or acceptance or rejection.
● The Acceptance test is done by either the customer or representatives of
the customer
● Here, the customer defines a set of test cases that will be executed to
qualify and accept the product.
● The test cases for acceptance testing are normally small in number and
not written with the intention of finding defects
● The main purpose of this test is to evaluate the system's compliance with
the business requirements and verify if it is has met the required criteria
for delivery to end users.
Types of Acceptance Test
● User Acceptance Testing
● Business Acceptance Testing
● Contract Acceptance Testing
● Regulation Acceptance Testing
● Operational Acceptance Testing
● Alpha Testing
● Beta Testing
Alpha Testing
● Alpha testing is the first end-to-end testing of a product to ensure it meets
the business requirements and functions correctly.
● It is typically performed by internal employees and conducted in a
lab/stage environment.
● An alpha test ensures the product really works and does everything it’s
supposed to do.
● While “unit testing” and “smoke testing” of various individual
components and features may have been conducted during the
development process, an alpha test is an initial opportunity to evaluate
the performance and comprehensive functionality of a given product
release.
● The main objective of alpha testing is to refine the software product by finding
and fixing the bugs that were not discovered through previous tests.
● It is the final testing stage before the software is released into the real world.
● Alpha testing has two phases,
○ Phase 1: done by in-house developers using either hardware-assisted
debuggers or debugger software. The aim to catch bugs quickly. Usually
while alpha testing, a tester will come across to plenty of bugs, crashes,
missing features, and docs.
○ Phase 2: done by software QA staff, for additional testing in an environment.
It involves both black box and white box testing.
● So, alpha testing can be envisioned as an online application that is not completely
ready for the usage but has been opened up to get some initial feedback.
Beta Test
● Beta testing is an opportunity for real users to use a product in a production
environment to uncover any bugs or issues before a general release.
● Beta testing is the final round of testing before releasing a product to a wide
audience.
● The objective is to uncover as many bugs or usability issues as possible in this
controlled setting.
● Beta testers are “real” users and conduct their testing in a production
environment running on the same hardware, networks, etc., as the final
release.
● This also means it’s the first chance for full security and reliability testing
because those tests can’t be conducted in a lab or stage environment.
● Beta tests can either be open or closed.
● In an open test, anyone can use the product and is usually presented with
some messaging that the product is in beta and given a method for
submitting feedback.
● In closed beta, the testing is limited to a specific set of testers, which may
be composed of current customers, early adopters, and/or paid beta
testers.
● The feedback collected will improve the current release and help drive
priorities for future releases and ensure the roadmap and planning are as
responsive as possible to what’s being learned from the market.
In a product based company In a service based company,
● Alpha test is defined as the ● Alpha testing is the testing
testing which is done at conducted by customers at
developer’s site by the tester. developers site
● Beta test is defined as the ● Beta testing is the process of
testing done by testers by giving the product to the
generating the customer’s customers and let them do the
environment. testing at their own
environment.
Test Case
● A test case is a set of actions executed to verify a particular feature
or functionality of your software application.
● A Test Case contains test steps, test data, precondition, post-
condition developed for specific test scenario to verify any
requirement.
● The test case includes specific variables or conditions, using which
a testing engineer can compare expected and actual results to
determine whether a software product is functioning as per the
requirements of the customer.
● Test scenario and test case are two different things.
● Test scenarios are rather vague and cover a wide range of possibilities.
Testing is all about being very specific.
● For a test scenario : Check Login Functionality there many possible test
cases are:
○ Test Case 1: Check results on entering valid User Id & Password
○ Test Case 2: Check results on entering Invalid User ID & Password
○ Test Case 3: Check response when a User ID is Empty & Login Button is
pressed, and many more
Test Case Sample
Test Case Test Scenario Test Steps Test Data Expected Actual Pass/Fai
ID Results Results l

TU01 Check 1.Go to site http://demo.com Userid = abc User should As Pass
Customer 2.Enter UserId Password = cba Login into an Expected
Login with valid 3.Enter Password application
Data 4.Click Submit

TU02 Check 1.Go to site http://demo.com Userid = abc User should not As Pass
Customer 2.Enter UserId Password = abc Login into an Expected
Login with 3.Enter Password application
invalid Data 4.Click Submit