1..

Introduction
Learning Objectives
To study the concept of the Software Testing.
Toknow about the nature of errors.
To learn about the Testing Principles and Testing fundamentals.
Toget information about the debugging.

11 INTRODUCTION TO sOFTWARE TESTING

Definition:
"Software testing is a process used to identify the correctness, completeness and
quality of developed software".
Testing is the process of exercising and evaluating a system or system component by
manual or automated means to verify that it satisfies specified requirements.
the exposure
Primary role of testing isnot demonstration of correct performance, but - Glen Myers
of hidden defects.
concerned with errors, faults, failures and incidents. A test is the act of
Testing is
exercising software with an objective of:
o Finding failure
Demonstrate correct execution -Paul Jorgensen
to ANSI/IEEE 1059 standard, Testing can be defined as -A process of
According
software itemn to detect the differences between existing and required
analyzing a the features of the software
defects/errors/bugs) and to evaluate
conditions (that is
item.
process used to identify the correctness, completeness, and
Software testing is a
computer software. It includes a set of activities conducted with
quality of developed could be corrected before the
finding errors in software so that it
the intent of
product is released to the end users.
OR
 In simple words, software testing is an activity to check whether the actual resu:ts
match the expected results and to ensure that the software system is defect free.
OR
It can also be stated as the process of validating and verifying that a software
program or application or product:
Meets the business and technical requirements that guided it's design and
development.
Worksas expected.
Can be implemented with the desired characteristics.
Benefits of Software Testing:
1. It improves the Quality as testing looks for a class of problems and flags them as
possible candidates for investigation and fixes.
2. Testing demonstrates that software functions appear to be working according to
specifications.
Goals of Software Testing:
The main goal of software testing is to find bugs as early as possible and to fix
bugs and to make sure that the software is bug free.
The goals of software testing may be classified into three major categories as
shown in Fig. 1.1.
Immediate Goals
Bug discOvery
Bug prevention

Long-term Goals
Reliability
Software testing Quality
Customer satisfaction Quality software
Risk management product

Post-implementation Goals
Reduced maintenance cost
Improve testing process
Fig. 1.1: Software Testing Goals
1 Immediate Goals: These goals are
also called as short term goals. These testing
goals are the immediate result after testing. These goals contain:
(i) Bug discovery: This is the immediate goal of
software testing to find errors at
any stage of software development. Numbers of the bugs are
discovered in the
early stage of testing.
(ii) Bug prevention:This is the resultant action of bug discovery.
 Sem. VI) 1.3
Introduction
2. Long Term Goals:
long term. These These testing goals affect the software product quality in the
(i) Quality: Thisinclude:
(ii) goal enhances the quality of the software products.
Customer satisfaction: This goal verifies the customer's satisfaction for
developed software product.
() Rellability: It is a matter of confidence that the software will not rall.
SIOrt rellability means to gain the confidence of the customers by
them a quality product. providing
() Risk management: Risk must be done toreduce the failure of
product ama to
manage risk in different situations.
3. Post Implemented Goals: These goals are important after the software product
released. Some of them are listed below:
(1) Reduce maintenance cost: Post-released errors are costlier to fix
and
difficult to identify.
(ii) Improved software testing process: These goals improve the testing process
for future use of software projects. These goals are known as post
implementation goals.
L1.1 Nature of Errors
Software Bug:
A software bug is the common term used to describe an error, flaw, mistake, failure
or fault in a computer program or system that produces an incorrect or unexpected
result, or which causes the program to perform in an un-intended manner.
Most bugsarise from mistakes and errors in either a program's source code or its
design, but the main cause can be traced to the specification.
ASoftware bug occurs when one or more of the following is true:
The Software does not do something that the product specification says it should
do.
The Software does something that the product specification says it should not do.
1. Errors:
lead to a
The system is in a state such that further processing by the system will
failure.
a software
Anerror is a mistake, misconception or misunderstanding on the part of
a
developer. It might be typographical error, a misleading of specifications, or
misunderstanding of what a subroutine does.
The errors may be:
o Actual bugs in the code.
f the requirements or functional specifications
 Missing commands, Pertormahce,
o Calculation errors.
Errors in handling or interpreting data.
Documentation - The user does not observe the ++operation described in manuals.
o Testing errors.
2. Faults:
Afault occurs when a human error results in a mistake in sonme software product(s).
For example, a developer might misunderstand a user-interface requirement and
therefore create a design that includes misunderstanding. The design fault can also
result in incorrect code, as well as incorrect instructions in user manual.
Lead to
Human Error Fault

Misunderstanding Design Fault Incorrect code

GUI requirements Incorrect instructions in
User manual
Fig.1.2: Flow of Fault
A fault is the difference between an incorrect program and the correct
version. A
single error can result in oneor more faults. One fault can
 CA) Sem. VI) Introduction
1,5
3. Failure:
A fault (bug) can go user or tester
perceives that the undetected until a failure occurs, which iswhen a
systemis not delivering the expected service.
Failures can be defined as :
Deviation of the system from its
The inability of a expected behavior or service.
system or component toperform its required functions within
specified performance requirements.
Failures can be discovered both before and after system
testing as well as in operation. delivery, as they Can oco
Faults represent problems that the developer sees, while failures are
the user sees. problems that
Ideally the life of a bug ends when it is uncovered in testing and
Lead to fixed.
Error Lead to
Fault Failure

Logical Error Incorrect Code Unable to Process

Fig. 1.3: Flow of Failure
For example, consider ATM machine. Developer might have forgotten to fire update
query on Database when user changes the PIN number, and screen displays the
message "Your PIN number has been changed!!" When user tries to access the system
with new PIN number, System gives the message "Sorry unable to process".
Defects:
Defects are :
Abnormal behavior of the software.
Non-conformance to requirements or functional/program specification.
Errors in Testing.
Mistakes in Correction.
For example, Requirements and specification defects, design defects, coding defects
or testing defects.
1.2 TESTING OBJECTIVES
Errors.
Testing is a process of executing a program with the intention of finding
Establishing confidence that a program does what it is supposed to do. It is the
measurement of Software Quality which confirms that a program performs its
intended functions correctly.
Testing identifies the difference between Expected and ActualResult. This is a process
work product.
of trying to discover every conceivable fault or weakness in a
Basic objectives of Testing ar
To find a bug/ any defect/errors in Software.
 Sem. VI)

Software auality which confirms that a program performs its

0 ensure the
intended functions correctly?
o Toexplore thè weaknesses/limitations of a software.
Software ouality in terms of its reliability and working as per the
0 ensure the
customer satisfaction and gives expected results.
To verify the software as per the standards.
To validate the software as per the customer requirements.
complexity by fixing the errors
t1S good to begin testing from the first stage. to avoid
at the last stage.
Wedo software testing for many reasons, such as:
1. Tocheck the reliability of the software.
contain any bug which can become a
Z. To be ensured that the software does not
reason for failure?
3. To check the software was made according to its specification.
4. Tocheck that the software meets its requirements.
5. To check that users are capable of using the software.
6. To check software works with other software and
hardware it needs to work with.
errors or
7. To improve the quality of software by removing maximum possible
defects from it.

1.3 TESTING PRINCIPLES

C As software development techniques have advanced during the last decades, some
basicprinciples of testing have also been established. Describing theoretical ideas and
practical hints, these principles can beseen as abasic guideline for both testing and
coding.
1. Testing shows the presence of errors:
Testing an application can only disclose one or more defects that exist in the
application, however, testing alone cannot prove that the application is error free. It
is important to design effective test cases which find as many defects as possible.
2. Exhaustive testing is impossible:
Unless the application under test (UAT) has a very simple logical structure and
limited input, it is not possible totest all possible combinations of data and scenarios.
Toy this reason. risk and priorities are used to concentrate on the most
important
aspects to test.
3. Early testing:
We can start testing as soon as the lnitial products, such as the requirement or design
Jsmentsare available. It is common for the testing phase to get squeezed at the end
the development lifecycle, 1.e. when development has finished, so by
testing early, we can prepare testang tor each levelof the development lifecvcle.starting
 Sem. VIj Introduction
1.7
When defects are
fix. It is much found earlier in the life cycle, they are much easier and cheaper to
functionality in cheaper
to change an incorrect requirement than having to change
alarge system incorrect
that is not working as requested or as designed!
4. Defect clustering:
During testing, it can be observed that mOst of the
Small number of modules within asvstem Le. a renorted defects are related to a
most of the defects in the small number of modules Contaln
5. The Fading
system.
effectiveness:
If you keep running the same set of tests
over and over again, there chances are that
no more new defects will be
discovered by those test cases. Because as the system
evolves, many of the previously reported defects will have been fixed and the
old test
cases do not apply anymore. Anytime a fault is fixed or a new
functionality added, we
need to do regression testing to make sure the new changed software has not
any other part of the software.
affected
6. Testing depends on context:
Different methodologies, techniques and types of testing are related to the type and
nature of the application. For example, a software application in a medical device
needs more testing than games software. More importantly a medical device software
requires risk based testing, be compliant with medical industry egulators and
possibly specifictest design techniques. By the same token, a very popular website
needs to gothrough rigorous performance testing as well as functionality testing to
make sure the performance is not affected by the load on the servers.
7. Absence of errors myth:
Just because testing did not find any defects in the software, it does not mean that the
software is ready to be shipped. Were the executed tests really designed to catch the
most defects? Or where they designed to see if the software matched the user's
requirements? There are many other factors to be considered before making a
decision to dispatch the software.
14 TESTING FUNDAMENTALS
In testing, we can describe goals as intended outputs of the software testing process.
Primary goals of software testing are:
1. Error detection.
2. Debugging.
3. Verification and Validation.
4. Testcoverage.
5. Quality and Reliability.
1. Error detection:
error isa human action producing an incorrect result. When programmers make
An
errors, they bring in faults to program code. For Example,
Incorrect usage of software by users.
Bad architecture and design by architects and designers.
Bad programming by developers.
Inadequate testing by testers.
by Build Team Member.
Wrong build using incorrect configuration items software
challenging problems in
Software error detection is one of the most
engineering. It involves identifying bugs/errors/defects in software without
assurance background are
correcting it. Normally professionals with a quality
testing phase.
involved in bug's identification. Test execution is performed in the
2. Debugging:
testing detects an
Debugging occurs as a consequence of successful testing. When
cause of failure and removing
error, debugging is an action that results in finding the
its specifications or
it. It determines why a system doesn't mneet one or more of
testing but
requirements and making it meet those specifications. Debugging is not
in the next section.
testing can be a part of debugging. We will see more on debugging
3. Verification and Validation:
Verification:
Verification is the process of confirming whether software meets its specifications.
This process of reviews deliverables throughout the life cycle. It examines Product
Requirements, Specifications, Design, code for errors, faults and failures. It is usually
performed by STATIC testing, or inspecting without execution on a computer.
Inspections, walkthroughs and reviews are examples of verification techniques.
Verification is the assurance that the products of a particular phase in the
development process are consistent with the requirements of that phase. It is a low
level activity. In simple terms, verification makes sure that the "product is being built
right".
For example, one of the purposes of system testing is to give assurance that the
system design isconsistent with the requirements of the system design phase.
Validation:
Validation is a process of con firming whether software meets a user's requirements.
This process covers executing something to see how it behaves. Unit, Integration.
System, and Acceptance testing are examples of validation techniques. Validation is
ollyperformed by DYNAMIC testing, or testing with execution on a computer.
Blackdotion
box testing
is the
and White box testing are examples of test case design technigues.
assurance that the rinal product satisfies the system requirements.
Software Testing (BBA (CA) -Sem. VI) Introduction
1.9
It is high level activity. In simple terms.
is being built". validation makes sure that the rBP
The purpose of validation is to ensure that the system has
requirements, so that each function can be traced back to a particular implemente
requirement. Table 1.1 summarizes the difference between Verification customer
Validation. a
Table 1.1: Difference between Verification and
Sr, No.
Validation
Verification Validation
1 lt 1s a process of confirming It is a process of confirming
whether software meets its whether
software meets the user's
specifications. requirements.
2 It is the process of reviewing/ It is the process of
executing
inspecting deliverables throughout something to see how it behaves.
the life cycle.
3 It checks product requirements, It examines the entire product for
specification, design, code for errors, fault and failures against the
errors, faults and failures. verified requirements,specifications
and design.
4 Verification is usually performed Validation is usually performed by
by Static testing, or inspecting Dynamic testing, or testing with
without execution on a computer. execution on a computer.
5. Verification is the assurance that Validation is the assurance that the
the products of a particular phase final product satisfies the system
in the development process are requirements.
consistent with the defined
standard.
6. It is a low level activity. It is a high level activity.
7. Verification comes into picture Validation comes into picture during
during initial phases of SDLC. later stages of SDLC (0nce code is
ready).
8. Verification methods: Validation methods:
Inspections White box testing
Walkthroughs Black box testing
Reviews
9. Verification makes sure that the Validation makes sure that the "right
"product is being built right". product is being built".
10. Static mode of testing. Dynamic mode of testing.
 Debugging. testing, and verification are mapped to the software life cycle as
Fig. 1.4.
System
shownin
Requirements

Software
Requirements

Analysis

Design Verification

Static
Testing Coding Validat

Testing by
Execution
Testing (Dynamic
Testing)
Debugging
Production/Deployment
Fig. 1.4: Debugging,
4. Test Coverage: Verification and Testing
Exhaustive testing is impossible, not everything can be
subset of everything can be tested. tested. Not even a significant
and prioritize thoroughly. Therefore, testing needs to assign tasks
valid input case. Generally, every feature should be tested at leastreasonably
with one
We can also test input
permutations, invalid input,and non-functional
depending upon the operational of software. Highly present andrequirements
scenarios should have more profilecoverage than frequent use
insignificant scenarios. infrequently encountered and
5. Quality and Reliability:
Software quality is the degree of conformance to
and expectations. explicit or implicit requirements
Explicit: Clearly defined and documented.
o Implicit: Not clearly
defined and documented but indirectly
Requirements: Business/product/software requirements. suggested.
Expectations: End-user expectations.
 Intrt
1.11

Reliabiity ls one aspect of quality. To ensure that a program is of nin 4ue

reliable, asoftware tester must do both verify and validate throughout tne po
development process.
 Sem. V) 1.15

1.5 SOFTWARE REVIEWS. FORMAL TECHNICAL REVIEWS

Review is a kind of meeting for analyzing the product being developed or already
developed. It is a verification process where the product is checked ror de
specifications. Reviews can be done in various ways. It is a kind of static testing
As per IEEE definition - "Review is a formal or informal meeting at which a product
or document is presented to the user, customer, or other interested parties ror
comment and approval."
1.5.1 Reviews
Review is the name given to a testing technique done by people. It is a people based
static technique in which one or more people are involved in examining certain
activity or activities. There are a variety of ways in which reviews can be carried out
across different organizations. Some reviews are done in a formal way, some are very
informal and some reviews lies between these two techniques. Review technique can
be performed individually or in groups.
Review finds faults in specifications and other documents well in advance before
they are used in subsequent phases. These faults detected through review can be fixed
before their use in the next phase of development. Reviews are cost effective.
Review in code is required for:
The necessary improvements in the product well in advance.
To bring simplicity and comfort during the development process, review helps in
understanding the code complexity.
defects.
o Review process reduces the cost impact of Software
removal.
Review helps in defect amplification, defect detection and its
phases. Review is done in
o Review process moves throughout the development
code review.
terms of Requirement Review, Design review, Technical and
Software review method include following techniques:
Formal Review / Technical Review/ Peer Review.
Informal Review.
o Walkthrough.
EGEOFC
Inspection.
Audit.
15.2 Formal Review Technique
to verify all related documents Also
As its name implies. it is a formal review processhas
review who a high degree of formality while
known as technical review or peer issues and
performing the review. This type of review mainly toCuses on the technical
review
technical documents. A peer review would exclude managers from the
members
process. The success of this type of review heavily depends upon the review
 1.16
Software Testing [BBA (CA) - Sem. V
focus and knowledge. They can be very effective and useful i
and their individual sometimes they are very wastek.
manner, otherwise
planned in a systematic This level of documents needs tok.
meetings are not well disciplined.
especially if the documents. It is mandatory under the
planned effectively and so will have some set of
review process that even for a list of issues to be discussed or raised durines
formal
meeting should also be well documented.
faults along with resolvine
The formal revieww is very useful in finding important decisions.
technical problems related to important technical
difficult
execution and the post
The formal review process includes review planning, review
which
participants
review follow up. Review process starts with listing the review
include review leader, 2 or 3 review team members having good knowledge of
document under review and a scribe (could be a review team member) tO note down
the raised issues and corresponding decisions made by the reviewers, so that during
post review follow up process they can be verified.
Activities involved in FTR are:
1. Review guidelines.
2. Review Meeting.
3. Review reporting and Record Keeping.
4. Review checklist.
 Sem. VI) 1.17

1.6 INSPECTION AND WALKTHROUGH

1.6.1 Inspection
Inspection is a "formal evaluation technigue in which software requirements, deslg,
Or code are examined in detail by a nerson or group other than the author to deteC
faults,violations of development standards, and other problems" (BM).
Asoftware formal inspection is defect detection, defect elimination, and correction
verification process carried out during the development life cycle. The goal Or tormi
inspections is to ensure that defects are fixed early in the life cycle rather than late,
when they are harder to find and more costly to fix. Formal inspections are neid
throughout the software development life cycle phases.
Activities:
Inspection is a powerful review technigue to support following activities:
o Detect, identify,and describedefects.
Collect metricsdata.
Verify "fitness for use" in subsequent efforts.
Provide controls that determine the next reviewstep.
Requirements:
Essential requirements for Inspection process include:
o Definition of development process, entry criteria and exit criteria.
Description of the inspection process.
Correct execution of the process.
Authentic Documnentation of Inspection process.
Types of Inspections:
Common types of inspections are:
1. High-level design inspection (Lo): High Level Design (HLD) is the overall system
design covering the system architecture and database design. It describes the
relation between various modules and functions of the system. Data flow, flow
charts and datastructures, ERD are covered under HLD. Converted from SRS.
CEn+H
shipdiagay
Speel icato
 2. Low-level design inspection (L): Low Level Design (LLD) is like detailing the HLD
It defines the actual logic for each and every component/module of the system
under
Class diagrams with all the methods and relation between classes come
LLD. Program specificationss are covered under LLD.
3. Code inspection (L): Code Inspection (CI) is the inspection process of code
developed by the developer based on design specifications. The inspection team
reviews the code for its flow, complexity and development as per the need and
specifications.
Participants of inspection team:
Formal inspections are performed by a team (a moderator, reader, recorder, author,
and other inspectors). The actual team size should consist of four to seven persons,
although a minimum of three is acceptable. Each team member has a specific,
bo anoneibi1ity of +he entire insnection team to find
Goals:
product, not the
Inspections should be used to judge the quality of the software work
quality of the people who create the product. For this reason, managers should
neither participate in nor attend the inspection meeting.
1.6.2 Walkthrough
Walkthroughs can be viewed as presentation reviews in which a review participant,
usually the developer of the software being reviewed, narrates a description of the
software and the remainder of the review group provides their feedback throughout
the presentation.
" These are referred to as presentation reviews because the bulk of the feedback usually
only occurs for the material actually presented at the level it is presented.
 Walkthrough is a Review process in which a participant gives the software
description while the other members provide feedback throughout the presentation.
Here, the development team members are involved in the review process.
Walkthrough Process:
" Participants:
o Reviewer
o Review teamn
Timing:Completion of major milestones
Plan
o Identify walkthrough team
Schedule time, place
o Distribute materials in advance
Walkthrough is an informal way of verification. The review team inspects the
software in an informal way and in friendly environment. The team prepares the
schedule
report after going through the software. The walkthrough team fixes up the
consent. They
and time with the project team for walkthrough process with mutual then execute
members and
distribute the project information in advance to all team
prepared. Report starts with the
the walkthrough process. After the review a report is rer rt. the team
 uction
1.7 TESTING LIFE CYCLE
Once the project is confirmed to start, project development can be divided i
following phases:
Software requirements phase.
Software Design.
Implementation.
o Testing.
Maintenance.
These phases form a model called as a waterfall model, a linear
sequential model or
software development life cycle, which suggests a systematic, sequential
software development. approach to
Software Testing Life Cycle (STLC) model is basically developed to identify
testing activities needs to be carried and what's the best time to which
perform them to
accomplish those test activities. Even though testing differsbetween
there is a testing life cycle. organizations,
Requirement Analysis
Test Planning

Test Analysis

Test Design

Construction and Verification

Testing Cycle

Final Testing

Post Implementation
Fig. 1.8: Software Testing Life Cycle
1. Requirements Analysis:
In this phase, Software testers
analyze the customer requirements and work with
developers during the design phase to see which requirements are
they are going to test those requirements. testable and how
It is very important to start
testing activities from the
because the cost of fixing defects is very less if it is found inrequirements phase itself
this phase rather than in
future phases.
2. Test Planning:
In thisphase, all the planning about testing is done like what needs to be tested, how
the testing will be done, test strategy to be followed, what will be the test
environment, what test methodologies will be followed, hardware and software
availability, resources, risks etc. A high level test plan document is created which
includes all the planning inputs mentioned above and circulated to the stakeholderS.
3. Test Analysis:
After test planning phase is over, test analysis phase starts. In this phase, we need to
dig deeper into project and figure out what testing needs to be carried out in each
SDLC phase. Automation activities are also decided, if automation needs to be done
for software products, how will the automation be done, how much time will it take to
automate and which features need to be automated. Non functional testing areas
(Stress and performance testing) are also analyzed and defined.
4. Test Design:
In this phase, various Black-box and White-box test design techniques are used to
those
design the test cases for testing. Testers start writing test cases by following
scripts
design techniques, if automation testing needs to be done then automation
also need to be written.
5. Test Execution and Validation:
cases designed in step 4 are
In this stage, we need to do all the test activities. All test
verified and then processed with module(s) under test.
6. Testing Cycle:
Test Execution and Bug Reporting, Bug Management.
cases are executed without errors and
In this stage, we need to do test cycles untiltest
our test plans are reached as shown in Fig. 1.9.
Run test cases

Report bugs

Revise test case

Add new test cases

(if needed)

Bug fixing

Re-testing
Fig. 1.9: Testing Cycle
7. Final Testing and Implementation:
In this phase, the final testing is done for the software. Non functional testing like
stress, load and performance testing are performed. Software is also tested in the
production type of environment. In this phase, finaltest execution reports and all test
deliverables are prepared.
8. Post Implementation:
state, the
In this phase, the test environment is cleaned up and restored to default
A document is
process review meetings are done and lessons learnt are documented.
prepared to manage similar problems in future releases.