Ch.

6 Software Testing
Content:
6.1Definition
6.2 Software testing Process
6.3 Unit Testing
6.4 Integration Testing
6.5 System Testing

6.1 Definition:
Software testing is a process of identifying the correctness of software by considering
its all attributes (Reliability, Scalability, Portability, Re-usability, Usability) and
evaluating the execution of software components to find the software bugs or errors or
defects.

Software Testing is a method to assess the functionality of the software program. The
process checks whether the actual software matches the expected requirements and
ensures the software is bug-free. The purpose of software testing is to identify the
errors, faults, or missing requirements in contrast to actual requirements. It mainly
aims at measuring the specification, functionality, and performance of a software
program or application.

 Importance of Software Testing:

 Defects can be identified early:
Software testing is important because if there are any bugs they can be identified early
and can be fixed before the delivery of the software.
 Improves quality of software:
Software Testing uncovers the defects in the software, and fixing them improves the
quality of the software.
 Increased customer satisfaction:
Software testing ensures reliability, security, and high performance which results in
saving time, costs, and customer satisfaction.
 Helps with scalability:
Software testing type non-functional testing helps to identify the scalability issues and
the point where an application might stop working.
 Saves time and money:
After the application is launched it will be very difficult to trace and resolve the issues,
as performing this activity will incur more costs and time. Thus, it is better to conduct
software testing at regular intervals during software development.

 Need for Software Testing:

Software bugs can cause potential monetary and human loss. There are many examples
in history that clearly depicts that without the testing phase in software development
lot of damage was incurred. Below are some examples:

1985: Canada’s Therac-25 radiation therapy malfunctioned due to a software bug and
resulted in lethal radiation doses to patients leaving 3 injured and 3 people dead.

1994: China Airlines Airbus A300 crashed due to a software bug killing 264 people.

1996: A software bug caused U.S. bank accounts of 823 customers to be credited with
920 million US dollars.

1999: A software bug caused the failure of a $1.2 billion military satellite launch.

2015: A software bug in fighter plan F-35 resulted in making it unable to detect targets
correctly.

2015: Bloomberg terminal in London crashed due to a software bug affecting 300,000
traders on the financial market and forcing the government to postpone the 3bn pound
debt sale.

Starbucks was forced to close more than 60% of its outlet in the U.S. and Canada due to
a software failure in its POS system.

Nissan cars were forced to recall 1 million cars from the market due to a software
failure in the car’s airbag sensory detectors.
 Different Types of Software Testing:

6.2 Software testing Process:

These are 11 steps software testing process is an experience based practical approach
for solution to test assignment.
These are explained as following below.
Step-1: Assess Development Plan and Status:
This initiative may be prerequisite to putting together Verification, Validation, and
Testing Plan won’t to evaluate implemented software solution. During this step, testers
challenge completeness and correctness of event plan. Based on extensiveness and
completeness of Project Plan testers can estimate quantity of resources they’re going to
get to test implemented software solution.
Step-2: Develop the Test Plan:
Forming plan for testing will follow an equivalent pattern as any software planning
process. The structure of all plans should be an equivalent, but content will vary
supported degree of risk testers perceive as related to software being developed.
Step-3: Test Software Requirements:
Incomplete, inaccurate, or inconsistent requirements cause most software failures. The
inability to get requirement right during requirements gathering phase can also
increase cost of implementation significantly. Testers, through verification, must
determine that requirements are accurate, complete, and they do not conflict with
another.
Step-4: Test Software Design:
This step tests both external and internal design primarily through verification
techniques. The testers are concerned that planning will achieve objectives of wants,
also because design being effective and efficient on designated hardware.
Step-5: Build Phase Testing:
The method chosen to build software from internal design document will determine
type and extensiveness of testers needed. As the construction becomes more
automated, less testing are going to be required during this phase. However, if software
is made using waterfall process, it’s subject to error and will be verified. Experience has
shown that it’s significantly cheaper to spot defects during development phase, than
through dynamic testing during test execution step.
Step-6: Execute and Record Result:
This involves testing of code during dynamic state. The approach, methods, and tools
laid out in test plan are going to be wont to validate that executable code actually meets
stated software requirements, and therefore the structural specifications of design.
Step-7: Acceptance Test:
Acceptance testing enables users to gauge applicability and usefulness of software in
performing their day-to-day job functions. This tests what user believes software should
perform, as against what documented requirements state software should perform.
Step-8: Report Test Results:
Test reporting is continuous process. It may be both oral and written. It is important
that defects and concerns be reported to the appropriate parties as early as possible, so
that corrections can be made at the lowest possible cost.
Step-9: The Software Installation:
Once test team has confirmed that software is prepared for production use, power to
execute that software during production environment should be tested. This tests
interface to operating software, related software, and operating procedures.
Step-10: Test Software Changes:
While this is often shown as Step 10, within context of performing maintenance after
software is implemented, concept is additionally applicable to changes throughout
implementation process. Whenever requirements changes, test plan must change, and
impact of that change on software systems must be tested and evaluate.
Step-11: Evaluate Test Effectiveness:
Testing improvement can best be achieved by evaluating effectiveness of testing at top
of every software test assignment. While this assessment is primarily performed by
testers, it should involve developers, users of software, and quality assurance
professionals if function exists within the IT organization.

6.3 Unit Testing:

Unit testing involves the testing of each unit or an individual component of the software
application. It is the first level of functional testing. The aim behind unit testing is to
validate unit components with its performance.
A unit is a single testable part of a software system and tested during the development
phase of the application software.
The purpose of unit testing is to test the correctness of isolated code. A unit component
is an individual function or code of the application. White box testing approach used for
unit testing and usually done by the developers.
Whenever the application is ready and given to the Test engineer, he/she will start
checking every component of the module or module of the application independently or
one by one, and this process is known as Unit testing or components testing.

Why Unit Testing?

In a testing level hierarchy, unit testing is the first level of testing done before
integration and other remaining levels of the testing. It uses modules for the testing
process which reduces the dependency of waiting for Unit testing frameworks, stubs,
drivers and mock objects are used for assistance in unit testing.
Generally, the software goes under four level of testing: Unit Testing, Integration
Testing, System Testing, and Acceptance Testing but sometimes due to time
consumption software testers does minimal unit testing but skipping of unit testing may
lead to higher defects during Integration Testing, System Testing, and Acceptance
Testing or even during Beta Testing which takes place after the completion of software
application.

Some crucial reasons are listed below:

o Unit testing helps tester and developers to understand the base of code that
makes them able to change defect causing code quickly.

o Unit testing helps in the documentation.

o Unit testing fixes defects very early in the development phase that's why there is
a possibility to occur a smaller number of defects in upcoming testing levels.

o It helps with code reusability by migrating code and test cases.

Unit Testing Tools:

We have various types of unit testing tools available in the market, which are as follows:

o NUnit
o JUnit
o PHPunit
 o Parasoft Jtest
o EMMA

Unit Testing Techniques:

Unit testing uses all white box testing techniques as it uses the code of software
application:

o Data flow Testing

o Control Flow Testing
o Branch Coverage Testing
o Statement Coverage Testing
o Decision Coverage Testing

How to achieve the best result via Unit testing?

Unit testing can give best results without getting confused and increase complexity by
following the steps listed below:

o Test cases must be independent because if there is any change or enhancement in

requirement, the test cases will not be affected.
o Naming conventions for unit test cases must be clear and consistent.
o During unit testing, the identified bugs must be fixed before jump on next phase of the
SDLC.
o Only one code should be tested at one time.
o Adopt test cases with the writing of the code, if not doing so, the number of execution
paths will be increased.
o If there are changes in the code of any module, ensure the corresponding unit test is
available or not for that module.

Advantages and disadvantages of unit testing

The pros and cons of unit testing are as follows:

Advantages
o Unit testing uses module approach due to that any part can be tested without waiting for
completion of another parts testing.
o The developing team focuses on the provided functionality of the unit and how
functionality should look in unit test suits to understand the unit API.
 o Unit testing allows the developer to refactor code after a number of days and ensure the
module still working without any defect.

Disadvantages
o It cannot identify integration or broad level error as it works on units of the code.
o In the unit testing, evaluation of all execution paths is not possible, so unit testing is not
able to catch each and every error in a program.
o It is best suitable for conjunction with other testing activities.

6.4 Integration testing:

Integration testing is the second level of the software testing process comes after unit
testing. In this testing, units or individual components of the software are tested in a
group. The focus of the integration testing level is to expose defects at the time of
interaction between integrated components or units.

Unit testing uses modules for testing purpose, and these modules are combined and
tested in integration testing. The Software is developed with a number of software
modules that are coded by different coders or programmers. The goal of integration
testing is to check the correctness of communication among all the modules.
Once all the components or modules are working independently, then we need to check
the data flow between the dependent modules is known as integration testing.
Let us see one sample example of a banking application, as we can see in the below
image of amount transfer.

Guidelines for Integration Testing:

o We go for the integration testing only after the functional testing is completed on each
module of the application.
o We always do integration testing by picking module by module so that a proper
sequence is followed, and also we don't miss out on any integration scenarios.
o First, determine the test case strategy through which executable test cases can be
prepared according to test data.
o Examine the structure and architecture of the application and identify the crucial
modules to test them first and also identify all possible scenarios.
o Design test cases to verify each interface in detail.
o Choose input data for test case execution. Input data plays a significant role in testing.
o If we find any bugs then communicate the bug reports to developers and fix defects and
retest.
o Perform positive and negative integration testing.

Here positive testing implies that if the total balance is Rs15, 000 and we are
transferring Rs1500 and checking if the amount transfer works fine. If it does, then the
test would be a pass.
And negative testing means, if the total balance is Rs15, 000 and we are transferring
Rs20, 000 and check if amount transfer occurs or not, if it does not occur, the test is a
pass. If it happens, then there is a bug in the code, and we will send it to the
development team for fixing that bug.

Reason behind Integration Testing:

Although all modules of software application already tested in unit testing, errors still
exist due to the following reasons:

1. Each module is designed by individual software developer whose programming logic

may differ from developers of other modules so; integration testing becomes essential to
determine the working of software modules.
 2. To check the interaction of software modules with the database whether it is an
erroneous or not.
3. Requirements can be changed or enhanced at the time of module development. These
new requirements may not be tested at the level of unit testing hence integration testing
becomes mandatory.
4. Incompatibility between modules of software could create errors.
5. To test hardware's compatibility with software.
6. If exception handling is inadequate between modules, it can create bugs.

Integration Testing Techniques

Any testing technique (Black box, White box, and Grey box) can be used for Integration
Testing; some are listed below:

Black Box Testing

o State Transition technique
o Decision Table Technique
o Boundary Value Analysis
o All-pairs Testing
o Cause and Effect Graph
o Equivalence Partitioning
o Error Guessing

White Box Testing

o Data flow testing
o Control Flow Testing
o Branch Coverage Testing
o Decision Coverage Testing

Types of Integration Testing:

Integration testing can be classified into two parts:

o Incremental integration testing

o Non-incremental integration testing
Incremental Approach:
In the Incremental Approach, modules are added in ascending order one by one or
according to need. The selected modules must be logically related. Generally, two or
more than two modules are added and tested to determine the correctness of functions.
The process continues until the successful testing of all the modules.
OR
In this type of testing, there is a strong relationship between the dependent modules.
Suppose we take two or more modules and verify that the data flow between them is
working fine. If it is, then add more modules and test again.
For example: Suppose we have a Flipkart application, we will perform incremental
integration testing, and the flow of the application would like this:

Flipkart→ Login→ Home → Search→ Add cart→Payment → Logout

Incremental integration testing is carried out by further methods:

o Top-Down approach
o Bottom-Up approach

Top-Down Approach:
The top-down testing strategy deals with the process in which higher level modules are
tested with lower level modules until the successful completion of testing of all the
modules. Major design flaws can be detected and fixed early because critical modules
tested first. In this type of method, we will add the modules incrementally or one by one
and check the data flow in the same order.

In the top-down approach, we will be ensuring that the module we are adding is
the child of the previous one like Child C is a child of Child B and so on as we can see
in the below image:
Advantages:

o Identification of defect is difficult.

o An early prototype is possible.

Disadvantages:

o Due to the high number of stubs, it gets quite complicated.

o Lower level modules are tested inadequately.
o Critical Modules are tested first so that fewer chances of defects.

Bottom-Up Method
The bottom to up testing strategy deals with the process in which lower level modules
are tested with higher level modules until the successful completion of testing of all the
modules. Top level critical modules are tested at last, so it may cause a defect. Or we can
say that we will be adding the modules from bottom to the top and check the data flow
in the same order.
In the bottom-up method, we will ensure that the modules we are adding are the
parent of the previous one as we can see in the below image:

Advantages

o Identification of defect is easy.

 o Do not need to wait for the development of all the modules as it saves time.

Disadvantages

o Critical modules are tested last due to which the defects can occur.
o There is no possibility of an early prototype.

In this, we have one addition approach which is known as hybrid testing.

Hybrid Testing Method

In this approach, both Top-Down and Bottom-Up approaches are combined for testing.
In this process, top-level modules are tested with lower level modules and lower level
modules tested with high-level modules simultaneously. There is less possibility of
occurrence of defect because each module interface is tested.

Advantages

o The hybrid method provides features of both Bottom Up and Top Down methods.
o It is most time reducing method.
o It provides complete testing of all modules.

Disadvantages
 o This method needs a higher level of concentration as the process carried out in both
directions simultaneously.
o Complicated method.

Non- incremental integration testing

We will go for this method, when the data flow is very complex and when it is difficult to
find who is a parent and who is a child. And in such case, we will create the data in any
module bang on all other existing modules and check if the data is present. Hence, it is
also known as the Big bang method.

Big Bang Method

In this approach, testing is done via integration of all modules at once. It is convenient
for small software systems, if used for large software systems identification of defects is
difficult.
Since this testing can be done after completion of all modules due to that testing team
has less time for execution of this process so that internally linked interfaces and high-
risk critical modules can be missed easily.
Advantages:

o It is convenient for small size software systems.

Disadvantages:

o Identification of defects is difficult because finding the error where it came from is a
problem, and we don't know the source of the bug.
o Small modules missed easily.
o Time provided for testing is very less.
o We may miss to test some of the interfaces.

6.5 System Testing:

System testing is a type of software testing that evaluates the overall functionality and
performance of a complete and fully integrated software solution. It tests if the system
meets the specified requirements and if it is suitable for delivery to the end-users. This
type of testing is performed after the integration testing and before the acceptance
testing.

OR
To check the end-to-end flow of an application or the software as a user is known as
System testing. In this, we navigate (go through) all the necessary modules of an
application and check if the end features or the end business works fine, and test the
product as a whole system.

There are mainly two widely used methods for software testing, one is White box
testing which uses internal coding to design test cases and another is black box
testing which uses GUI or user perspective to develop test cases.

o White box testing

o Black box testing

System testing falls under Black box testing as it includes testing of the external
working of the software. Testing follows user's perspective to identify minor defects.

System Testing includes the following steps.

o Verification of input functions of the application to test whether it is producing the

expected output or not.
o Testing of integrated software by including external peripherals to check the interaction
of various components with each other.
o Testing of the whole system for End to End testing.
o Behaviour testing of the application via a user's experience

Types of System Testing

System testing is divided into more than 50 types, but software testing companies
typically uses some of them. These are listed below:
Regression Testing

Regression testing is performed under system testing to confirm and identify that if
there's any defect in the system due to modification in any other part of the system. It
makes sure, any changes done during the development process have not introduced a
new defect and also gives assurance; old defects will not exist on the addition of new
software over the time.

Load Testing

Load testing is performed under system testing to clarify whether the system can work
under real-time loads or not.

Functional Testing

Functional testing of a system is performed to find if there's any missing function in the
system. Tester makes a list of vital functions that should be in the system and can be
added during functional testing and should improve quality of the system.
Recovery Testing

Recovery testing of a system is performed under system testing to confirm reliability,

trustworthiness, accountability of the system and all are lying on recouping skills of the
system. It should be able to recover from all the possible system crashes successfully.

In this testing, we will test the application to check how well it recovers from the
crashes or disasters.

Recovery testing contains the following steps:

o Whenever the software crashes, it should not vanish but should write the crash log
message or the error log message where the reason for crash should be
mentioned. For example: C://Program Files/QTP/Cresh.log
o It should kill its own procedure before it vanishes. Like, in Windows, we have the Task
Manager to show which process is running.
o We will introduce the bug and crash the application, which means that someone will
lead us to how and when will the application crash. Or By experiences, after few
months of involvement on working the product, we can get to know how and when the
application will crash.
o Re-open the application; the application must be reopened with earlier settings.

For example: Suppose, we are using the Google Chrome browser, if the power goes off,
then we switch on the system and re-open the Google chrome, we get a message asking
whether we want to start a new session or restore the previous session. For any
developed product, the developer writes a recovery program that describes, why the
software or the application is crashing, whether the crash log messages are written or
not, etc.

Migration Testing
Migration testing is performed to ensure that if the system needs to be modified in new
infrastructure so it should be modified without any issue.

Usability Testing
The purpose of this testing to make sure that the system is well familiar with the user
and it meets its objective for what it supposed to do.
Software and Hardware Testing
This testing of the system intends to check hardware and software compatibility. The
hardware configuration must be compatible with the software to run it without any
issue. Compatibility provides flexibility by providing interactions between hardware
and software.

Why is System Testing Important?

o System Testing gives hundred percent assurance of system performance as it covers end
to end function of the system.
o It includes testing of System software architecture and business requirements.
o It helps in mitigating live issues and bugs even after production.

System testing uses both existing system and a new system to feed same data in both and then
compare the differences in functionalities of added and existing functions so, the user can
understand benefits of new added functions of the system.