Chapter-l1

System Testing and

Quality Assurance
Introduction

Aare testing is necessary and important activity of software development

OceSs. It is a very expensive process and consumes one-third to one-half of the
nst of a typical development project. Therefore, a proper strategy is required to
sarny out testing activities systematicaly and effectively.
Software testing strategy provides a framework or set of activities, which are essential
for the sucoess of the project. This may include planning, designing of test cases,
execution of program with test cases, interpretation of the outcome and finally
collection and management of data.
Efective Software testing will contribute to :
Delivery of higher quality software product.
More satisfied users.
Lower maintenance cost.
More accurate and reliable
results.
The testing can be
(a) broadBy divided into two forms :
Verification Testing
Verification testing is used for requirements verification,
Code design verification or
(b) verification
Validation Testingduring various phases.
Validation testing is the process of evaluating a system at the end of
development process to determine whether it satisfies the specified
requirements.
[ 161 )
 There are two fundamental testing strategies at validaion stage. These are :
() Block box testing : Black boxtesting is done without the internal
of product and it aims,to test the functionality of software product knowedge
(ii) White box testing : White box testing is done to testthe internals of
product and hence it requires internal details of product. software

11.2 Definition of Testing

Let us consider three main definitions of testing.

Def I (According to Hetzel) :

"Testing is the process of establishing
system does what it is supposed to do." confidence that a program or |
Def 2 (According to Glen Meyers) :
"Testing is the process of executing a program or
of finding errors." system with the intent|
Def 3(According to IEEE):
"Testing is the process of analyzing a
diferences between existing and required software item to detect Ue
features of the software. item. conditions and to evaluate ulo
code, job control code, control Software item is a code, object
data or a collection ofsource
these iteMs.
11.3 Objectives of Testing
Testing objectives are :
1.
Testing is a process of
executing a program with the intent of finding an
Therefore, objective of testing
eror.

is to find and
2. To ensure that
the software is correct the errors.
3. To according to the requirements of the users.
systematically uncover different a witha
minimum amount of effort.
types of errors in minimumtimeand
4. A good test
case is one that
has a high
a ne s - y e t

undiscovered error. probability ofof finding

 Asuccessful test is one that uncovers an as-yet-undiscovered error.
5.
6.
To deliver high quality software product.

What Do We Test For ?

Aproposed system is subject to a variety of following tests

Correct Output
1.
The first test of a system is to see whether it produces the correct outputs or
not. We must use the test data for testing the correct output. Test data may be
artificial or live (taken from the users files)
2. Online Response
Response time should be less. One way to test this is to input transactions on
the
as many terminals as would normally be used in peak hours and note
establish a true
response time corresponding to each transactions to
performance level.
3. Volume
normally be produced to
In this test, we create as many records as would
verify that the.hardware and software will function correctly.
4. Stress Testing
efficiently during peak of
This test determines whether the system can work
its processing demand or not.
5. Storage Test
system under all possible
This test determines the storage capacity of
situations.
6. Security Test
are secure from unauthorized
This test determines that the data and programs
access.
7. Recovery Test
reason, whether it can be
This test deternmines if data is lost for whatever
recovered successfully or not.
[163 ]
 procedure
8. Usability documentation and
user-friendly nature of the system. This
The usability test verifies the
to nomal operating and error-handling
procedures. relates

11.5 The Test Plan

The first step in system-testing is to prepare a plan that will test all aspects of
system in a way that promotes its credibility among potential users. the

Atest plan entails the following eight activities.

1. Prepare Test Plan
A test plan should be prepared in accordance with established desion
specifications. It contains the following :
) expected output
(i) Crieteria for evaluating outputs.
(ii) A volume of test data.
(iv) PrOcedure for using test data.
(v) Personal and training requirements.
2. Specify Conditions for user Acceptance
Testing
Planning for user acceptance testing calls for the analyst and the User to
agree on the conditions for the test. The test schedule, the test duration and
the persons designated for the test should be included in the agreement. The
start and termination dates for
the test should be specified.
3. Prepare Test Data for Program
Testing
Test data are prepared and documented to ensure that all aspects of the
program are properly tested.
4. Prepare test data for
Transaction PathTesting
The data required for testing every condition and transaction to beintroduced
into the system. The path of each transaction from origin to destination s
carefully tested for reliable
results.
 plan User Training
6.
leer training is designed to
prepare the user for testing and
system. User involvement and training take place converting the
for three reasons: parallel with programming
1. The system analysts has time
available to spend on training while the
programs are being written.
2. Auser-training program gives the systems
analysts a clearer image of
the user's interest in the proposed
system.
3. Atrained user participates more effectively in
system testing.
For user training, preparation of a proper checklist is useful. The trainingplan
is followed by preparation of the user training manual and other text materials.
Acommon procedure is to train supervisors and department heads who, in
turn, train their staff as they see fit.
6. Complle/ Assemble Programs
Allprograms have to be compiled/ assembled for testing. Before this,however,
a complete program description should be available. The purpose of the
program, its use, the programmers who prepared it and the amount of computer
times it takesto run it should be included.
Atest scheme specifies how program software should be debugged. Acommon
approach, called bottom-up programming tests small-scale program modules,
which are linked to ahigher-level module, and so on until the program is
completed. An altemative is the top-down approach, where the general program
a time
IS tested first, followed by the addition of program modules, one level at
to the lowest level.
7. Prepare Job Pertormance Aids
personnel to run the system are
In this activity the materials to be used by
program Codes, a list of input
Specified and scheduled. This contains a list of
written instructions for system operations &
user training.
Codes and
8, Prepare Operational Documents
documents are finalized, including
During the test plan stage, all operational
by the proposed system.
Copies of the operational formats required
[165 ]
1l.6The Testing Techniques
In order to conduct a proper testing, the following order should be performed as

shown infigure 11.1.

Other high order test

Softequsentcse Integration testing

Unit testing

FIGURE 11.1
Unit testing
Integration testing
Other high order tests like system testing and
validation testing.
As shown in fig 11.1 first unit testing is perfomed. Unit testing focuses on the
individual modules of the product. After that integration testing is performed. When
modules are integrated into bigger program structure then new errors aarise often.
Integration testing uncovers those errors. After integration testing. other high order
tests like validation tests and
system test are performed.
In validation testing requirements established as part of software requirements
analysis are validated against the software that has been constructed. Finally, we
arrive at system testing, where the software and other system elements aretested

as a whole.
[ 1 6 6 1
1.6.1 Unit Testing
We know that smallest unit of software design is a
module. Unit testing is performed
to check the functionality of these units. The primary goal of
unit testing is to take
the smallest unit of testable software and to find out
whether it works exactly as
expected. Each unit or module is tested separately before these unit or module are
integrated together to build theoverall system. Since the modules are small in size
individual programmers can do until testing on their respective modules. So unit
testing is basically white box oriented.

1.6.2 Integraion Testing

Unit testing ensure that all modules have been tested and each of them works
properly individually. Unit testing does not guarantee if these modules will work fine
if these are integrated together as a whole system. Integration testing uncovers
errors that arises when modules are integrated to build the overall system.
Integration testing is a systematic technique for constructing the program structure
while conducting tests to uncover error associated with interfacing. The objective
them and build the
is to take unit tested modules, integrate them, find errors, remove
overall program structure as specified by design.
integration of modules :
Followingtypes of errors may arise during the
data coming out of a module is
1. Data can be lost across an interface. That is
module.
not going into the desired
on another.
2. One module can have an adverse affect

Modules when combined, may not produce

the desired major function.
3.
magnified to unacceptable levels.
4. Individually acceptable imprecision may be
create problem.
5. Global data structures can
module is added as part of integration testing, the software changes
Each time a new
established, neW I/O may occur, and
new control logic is
Newdata flow paths are
previously worked
changes may cause problem with functions that
invoked, These module is
Therefore, regression tested is needed every time a new
flawlesslv. re.
context of an integration test strategy, regression testing is the
added. In the ensure thet
some subset of tests that have already been conducted to
execution of
propagated unintended side effects.
changes have not
[167 )
1.6.3 Validation Testing
After the integrationtesting we have an assembled package that is free from modules
of software tests, validation
At this stage afinal series
and interfacing errors.
begin. Validation succeeds when software functions in a manner that can be testing
expected by the customer.
that:
In validation testing, plan and procedures ensure
1. All functional requirements are fulfilled.
2. All the required characteristics are achieved.
3. Documentation is done correctly.
Each validation tests may result in one of the following two possible conditions :
() The software pertormance is accepted.
(ü) deviation from specification is uncovered anda deficiency list is
A
created
Deviation or error discovered at this stage in a
project can rarely be
corrected prior to scheduled completion. It is often
with the customer to establish a
necessary to negotiate
method for resolvingdeficiencies.
I1.6.3.1 Configuration Review
An important elenment of the
validation process is a
software configuration have configuration review. It elisu
that all elements of the
configuration review is also called an been properly developed. The
audit.
I1.6.3.2 Alpha And Beta
When software is Testing
built for one
to enable the customer, a
by customer
customer to validate all series of acceptance tests are conducted

rather than by
developer.
requirements. Acceptance test is Conducted

If software is
to perform developed as a product to be
formal used by many
use a process acceptance tests with each customers, impractical
it is
called alpha and one. Most software product developers
user seems able beta
to find. testing to uncover the
end
errors that only

[ 1681
 Alphatesting is conducted by the customer at the developer's site. The developed
software is used in a natural setting with the developer. The developer records
errors and usage problems. The alpha testing process continues until the software
developer and the customer agrees that the developed software is an acceptable
implementation of the system requirements.
Beta testingis
conducted at one or more customer sites by the end user(s)) of the
nfhware. Here, developer is not present. Therefore, the beta test is a live application
*the software in an environement that cannot be controlled by the developer. The
CUstomer records all problems that are encountered and reports these to the

developer at regular intervals.

The software developer makes modifications and releases the developed software
either for further beta testing or to the entire customer base.

64 System Testing
testing is
System tesating means testing of the entire software. The goal of system
includes a series of
to see if the software meets its requirements. System testing
different tests to fully exercise the computer-based system.
Although each test has a different purpose, all work to
verify that system elements
functions.
have been properly integrated and perform allocated
System testing involves the following types of testing :
() Recovery testing
(i) Security testing
(iü) Stress testing
(iv) Perfcrmance testing

lI.6.4.1Recovery Testing
and completely the system
Recovery testing is designed to examine how easily
can recOver a system failure.
re-initialization mechanisms, data recoven
In case, the recover is automated then
In case, the recover requires human
and restart are evaluated for correctness.
Intervention, the mean time to repair is evaluated
to determine whether it is within
acceptable limits.
minimum human intervention is
Asystem with quick recoveringcapability and with
desirable.
[169 ]
II.6.4.2 Security Testing
mechanism built
built into a system wil
verify that protection
Security testing attempts to
protect it from unauthorized penetration. During security testing, the tester plays the
the individual who desires topenetrate the system.
role of
The role of the system designer is to make penetration Cost greater than tthe value
obtained.
of the information that will be

11.6.4.3 Stress Testing

Stress tests are designed to confront program with abnormalsituations. This testing
is normally used to understand the upper limits of capacity within the annlict
landscape. This kind of test is done to determine the application's robustness in
times of extreme load and helps application administrators to determine it the
application will perform sufficiently if the current load goes wellabove the expected
maximum.

For example
1. Special tests may be designed that generate ten interrupts per second, Wnen
one or two is the average rate.
2. Input data rates may be increased byan order of magnitude to determine
input functions will respond.
3. Test cases that require maximum memory or other resources are executed.

11.6.4.4 Performance Testing

Performance testing is designed to test the run-time performance of Software within
the context of an integrated system. Performance testing occurs throughouttallsteps
in the testing process. Even Imodule

at the unit level, the individual mo

may be assessed as tests are conducted. performance
However. of anal system elements
it is not untill
are fully integrated that
the true performance of a system can be
a s c e r t a i n e d .

Purposes of Pertormance testing

are
(a) Demonstrate
that the system meets
(b) Compare twO performance criterla.
systems to find which performs
(c) Measure what better.
workload cause the
system to perform badly.
 desigl.

12.7 Maintenance
es
required to be properly
he After completion of implementation phase, the software is
maintained, it may fail too. Therefore
sis maintained. If the system is not properly
507% OT Total soTware development time is spent in maintaining
generallymore than
the system.
improvement and updation of software in response to the
Maintenance is actually
organisation.
ses
c h a n g e s in
the
the
l a b o u r - i n t e n s i v e

and expensive phase of a software

is quite
Maintenance

system may become difficult to be mnaintained if it is not

ysis project. A
development pf
properly designed.
[187 ]
186]
 Maintenance task is assigned to a speciâl group Consisting of maintenan nce

programmers. The maintenance programming team's job involves keeping the

preventive measures to keep
software
operational, that is
undertaking
computer
programs running.

Maintenance
12.8 Types of System
three types of System Maintenance:
There are
Corrective Maintenance
1.

2. Adaptive Maintenance

Perfective Maintenance
3.

1. Corrective Maintenance

any errors in the newiy

Corrective Maintenance is performed to correct
following types:
implemented system. The errors that occur may be one of the
validation procedures.
(a) Program logic errors such as inadequate data
data type errors, inconsistent or wrong usage of variable names, infinite
loop etc.
(b) System errors consist of hardware problems such as disk crash, memory
failure etc.

(c) Operational errors occur due to members of the operating stait. For
example, improper backup of computer files, failure to change prnier
ribbons.

(d) User errorsoccur due to deficiencies in training and user guides.

2. Adaptive Maintenance
Adaptive Maintenance is performed to meet with the changing r e q u i r e m e n t s

of the market and organization. It is performed with a view to keeping the

software up-to-date.
In addition, adaptive maintenance also includes other tasks Such as file

copying, back-up, file reorganization, file conversion, table update to

include

revised employee pay rates.

I 188]
 Perfective Maintenance
3
Maintenance is
Perfective

performed to further improve

working system. the efficiency of a
Some ways to improve efficiency are

Modiftying data structures

(i) BacCkup records

i) eliminating temporary work files

(iv) modifying expensive parts of a system

9Activities of A Maintenance Procedure

Primary activities of a maintenance procedure are:

1. Maintenance staff receives a request for service from authorized user.

2 Maintenance staff studies the definition of the required modifications.
3 Maintenance staff acquires source program statements printout from library.
4. Also, acquire the documentation of this source program, if any.
5. Make required changes to programs and systems.
6 Test changes

7. Submit test result for user approval.

the library.
Once approved, the modified documentation is filed with
Aproject completion notice is sent to the
user, signaling the termination of the
project.

10 Reducing Maintenance Costs

development cost.
Costof imaintenance is approximate 30% to 70% of the

Maintenance involves:
code
Understanding the existing software

Understanding the related documents [189]

 changes
effects of
the documents)

and the
Understanding

change
(both code
Making the
(changes)

Testing the
new parts

the side
effect
problem of
Testing
to attack the
ever-growing

software
a plan is
needed
maintenance
reduction plan. Maintenance
Therefore,
organization use
maintenance.
Many

has three
phases :
reductionplan
Maintenance Management Audit
1. maintenance effort. Maintenance
quality of the
evaluates the
This phase
that:
Management Audit ensure
is logged in a
maintenance request log.
maintenance request
(i) Each Corrections, additicnel
worked are spenton error
Percent of total hours
(i) changes/deletions, and improvements.
reducton
well-defined maintenance
Organization currently have a
(ii)
program.

2. Software System Audit :

This phase entails :
assessment of tre
overall view of the system documentation and an
() An reliability,
and system maintainability,
quality of datafiles and data bases
and efficiency. to
system
in the
(i) Functional information gathered on all the programs a s s i g n e d a

is
determine how well they do the job. Each program
preliminary ranking value. mean
time

value, m
(ü) Adetailed program audit, which considers theranking
between failure (MTBF), and size of the maintenance bacivs
3. Software Modification
This phase consists of three steps:
documents.

() Make the changes in both code (or program) and the

(190/
 (i) System level update, which
brings up to date data flow completes system level documentation
references programs.
diagrams or system flowcharts
and errors
(i)
Reaudit oflow-ranking programs to make sure that the
corrected. errors have been
Maintenance reduction plans will
More user satisfaction
contributed to :
Improve response time in correcting
errors.
More reliable software
Reduce maintenance backlog
Reduce maintenance cost
Delivery of high quality software.

ISystem Documentation
Documentation is the process of collecting, organizing,
historical record of programs and other storing and maintaining
documents used or prepared during the
different phases of the life cycle of the software.
System documentation means coordinated
effort, to communicate the information
of the system in written form. Its purpose
is to ensure that the details of the
aie understood by all the persons concerned during system
the development process and
subsequent operation.
ngn quality standard documentation offers the
following advantages :
Aid to analysis
Aid to communications
Aid to training
Aid to debugging
Aid to implementation
Aid to evaluation
Aid to management
Aid to security
(191 ]
 12.12 Characteristics of a Good Documentation
Characteristics of a Good Documentations are

1. Availability
It should be accessible to those for whom it is intended
2. Objectivity
It must be clearly stated in the language that is easily understood
3. Cross-referable

Itshould be possible to refer to other documents.

4. Easy to maintain
When the system gets modified it should be easyto update the
5.
documentation
Completeness
It should contain everything needed, so that those who are
reading it carefuly
understand the system.

12.13 Forms of Documentation

There are three commonly used forms of documentation:
(i) Comments
() User Manual

(i) System Manual

All these forms of documentation are necessary to properly and completely
docui
a software.
1. Comments
Comments are natural language statemnents put assist

within a program to the

anyone reading the soure program listing in understanding the logic of
along
program. All high-level languages provide the facility to write comments
should

with the source code of a program. It is

suggested that programmers

liberally use this facility for proper documentation of their prograi

[ 192)
 )

should not be redundant, incorrect,

Comments

be understood by
incomplete, or written in a
manner that cannot anyone else.
User's Manual

2
e manual is the training guide that Contains
conmplee uescription on how
loment and operate the neW System. It must
contain the following :
1. Hardware Requirements Specifications
2. Software Requirements Specifications
3. Installation and Uninstallation Proedure
4. Starting, running and terminating Procedure
5. Input screen and Output formats
6. Menu and Sub-menus
7. Procedures to execute each option of the Menu
8. DataFile Layouts
9. Data File Layouts
10. Guidelines to operate the System
11. List of errors with explanations and debugging
methods
12. Special checks (if any) and security measures
3.
etc.
System Manual
System manual coontains the following
1.
information :
Defining the objectives of developing the software, and its
2. usefuiness.
The environment in which it
3.
functions.
Its limnitation
4.
5
Put data requirenments and the form and type of output required.
Detailed system flow charts and program floW
6. charts.
7.
Specific program names along with their description and purpose.
8. sGrnption of the prooram listings, and the control procedures.
Source code with documentation.
[ 193]