Software Testing

Testing is the process of

exercising a program with
the specific intent of
finding errors prior to
delivery to the end user.

Don’t view testing as a

“safety net” that will catch
all errors that occurred
because of weak software
engineering practice.

5/22/2024
 Who Test the Software
Develop

Who Test

r
Teste
the
Software?
er

Understands the system but, Must learn about the system,

will test "gently" but, will attempt to break it <Develope
and, is driven by "delivery" and, is driven by quality
r> OR
Testing need a strategy Dev
Testing without plan is of
team needs to work with
[Tester]
no point
Test team, “Egoless
It wastes time and effort
Programming”

5/22/2024
 When to Test the Software?
Com ponentCode Com ponentCode Com ponentCode
Unit Test Unit Test Unit Test

Design Specifications Integration Test Integrated modules

Systemfunctional requirements Function Test Functioning system

Other software requirements Performance Test Verified, validated software

Customer SRS Acceptance Test Accepted system

User environment Installation Test Systemin use!

15

5/22/2024
 Verification &Validation
Verification Verification Validation
Are we building the product Process of evaluating products of a Process of evaluating software at the
right? development phase to find outwhether end of the development to
theym eetthespecified requirem ents. determine whether software meets the
The objective of Verification is to customer expectations and requirements.
make sure that the product being
Activites involved:Reviews,Meetings Activities involved: Testing like black box
develop is as per the requirements testing, white box testing, gray box
and Inspections
and design specifications. testing

Carried outbyQAteam Carried outbytesting team

Validation
Are we building the rightproduct? Execution ofcodeis not comes under Execution of code is comes
Verification under Validation
The objective of Validation is to
make sure that the product Explains whether the Describes whetherthe software is
outputs are according accepted by the user ornot
actually meet up the user’s
to inputs or not
requirements, and check Cost oferrors caughtis less Cost oferrors caughtis high
whether the specifications were
correct in the first place.

16

5/22/2024
 Software Testing Strategy

Unit Testing
It concentrate on
each unit of the
software as
implemented in
UnitTesting source code
Integration Testing It focuses on
Vald
i ation Testing each component
individual, ensuring
System Testing
that it functions
properly as a unit. 17

5/22/2024
 Software Testing Strategy Cont.

Integration Testing Validation Testing SystemTesting

It focus is on design Software is validated The software and other software
and construction of against requirements elements are tested as a whole
software architecture establishe as a part of Software once validated, m ust be
rdequirem entm odeling combine wit other system
Integration testing is the
database
d etc…
h e.g. hardware people
process of testing It give assurance elements , ,
interfac the softwar tha It verifies that all
e between e
informational t meets
functional, all properly and elements
that
softwar two unitsor , behavioral and mesh
e perform ance requirem ents function /perform ance is achieved.
overall system
modules
18

5/22/2024
 Unit Testing
Unit is the smallest part of a software system which is testable.
It may include code files, classes and methods which can be tested individually
for correctness.
Unit Testing validates small building block of a complex system before
testing an integrated large module or whole system
The unit test focuses on the internal processing logic and data
structures within the boundaries of a component.
The module is tested to ensure that information properly flows into and
out of the program unit
Local data structures are examined to ensure that data stored
temporarily
maintains its integrity during execution
All independent paths through the control structures are exercised to
ensure that
all statements in module have been executed at least once
Boundary conditions are tested to ensure that the module operates
properly at 19
boundaries established to limit or restricted processing
5/22/2024 All error handling paths are tested
 Diver &Stub (Unit Testing)
A B C
Let’s take an example to understand it in a better
way.
Suppose there is an application consisting of
three modules say, module A, module B &
module C.
Developer has design in such a way that
module B depends on module A & module C
Com ponent-testing (Unit Testing)m ay be depends on module B
done in isolation from restofthe system
The developer has developed the module B
In such case the missing software is and now wanted to test it.
replace by Stubs and Drivers and
But the m odule A and m odule C has not been
d the interface betw een the
simulat developed yet.
softw are com ponents in a sim ple m anner
e In thatcase to test the module B completely we can
20
replace them odule Aby Driver and m odule C by stub
5/22/2024
 Diver &Stub (Unit Testing) Cont.
Driver Stub
Driver and/or Stub softw are must be Stubs serve to replace
developed foreach unit test modules that are subordinate
A driver is nothing more than a (called by) the component to be
"main program" tested.
It accepts test case data A stub or "dummy subprogram"
Passes such data to the component and Uses the subordinate module's interface
Prints relevant results. May do minimal data manipulation
Driver Prints verification of entry and
Used in Bottom up approach Returns control to the
Lowest modules are tested first. module undergoing testing
Simulates the higher level of components Stubs
Dummy program for Higher level component Used in Top down approach
Top most module is tested first
Simulates the lower level of components
Dummy program of lower level components
21

5/22/2024
 Integration Testing
Integration testing is theprocess of testing theinterface between two software units orm odules
It can be done in 3 ways 1.Big Bang Approach 2.Top Down Approach 3.Bottom Up Approach

Big Bang Approach Top Down Approach

Combining all the modules Testing take place from top to bottom
once and verifying the
High level modules are tested first and then low-level
functionality after
modules and finally integrated the low level modules to high
completion of individual
level to ensure the system is working as intended
module testing
Stubs are used as a temporary module, if a module is not
ready for integration testing
Bottom Up Approach
Testing take place from bottom to up
Lowest level modules are tested first and then high-level modules and finally integrated the
high level modules to low level to ensure the system is working as intended
Drivers are used as a temporary module, if a module is not ready for integration testing
22

5/22/2024
 Regression Testing
When to do regression testing?

Repeated testing of an already When new functionalities are added to

tested program, after modification, the application
to discover any defects introduced or E.g. A website has login functionality with only Email.
uncovered as a result of the changes Now the new features look like “also allow login using
in the software being tested Facebook”
Regression testing is done by When there is a change requirement
re- executing the tests against the Forgot password should be removed from the login page
modified application to evaluate
whether the modified code breaks When there is a defect fix
anything which was working earlier E.g. assume that “Login” button is not working and
tester reports a bug. Once the bug is fixed by developer,
Anytime we modify an application, tester tests using this approach
we should do regression testing
When there is a performance issue
It gives confidence to the E.g. loading a page takes 15 seconds. Reducing load
developers that there is no time to 2 seconds
unexpected side effects after
modification When there is an environment change
E.g. Updating database from MySQL to Oracle

5/22/2024
 Smoke Testing
Smoke Testing is an integrated
testing approach that is commonly used when Build
product software is developed
This test is performed after each Build
Release F1 F2 F3 F4 F5 F6
Smoke testing verifies – Build Stability
This testing is performed by “Tester”
or “Developer” Critical Critical Major Major

This testing is executed for Integration It test the build just to check if any
Testing, System Testing & Acceptance Testing
major or critical functionalities are broken
What to Test?
All major and critical functionalities If there are smoke or Failure in the build
of the application is tested after Test, build is rejected and developer
It does not go into depth team is reported with the issue
to test each functionalities
This does not incudes detailed testing for the build

5/22/2024
 Validation Testing
Theprocess of evaluating software to determine whether it satisfies
specified business requirements (client’s need).
It provides final assurance that software meets all informational, functional,
behavioral, and performance requirements
When custom software is build for one customer, a series of acceptance tests are
conducted to validate all requirements
It is conducted by end user rather then software engineers
If software is developed as a product to be used by many customers, it is
impractical to perform formal acceptance tests with each one
Most software product builders use a process called alpha and beta testing to uncover
errors
that only the end user seems able to find

5/22/2024
 Validation Testing – Alpha &Beta Test
Alpha Test
The alphatestis conducted at the developer’s site byarepresentative group ofend users
The softw are is used in a natural setting with the developer “lo oking over the shoulders”ofthe
users and recording errors and usageproblems
The alphatests are conducted in a controlled environment
Beta Test
The beta testis conducted atoneorm ore end-user sites
Developers are not generaly present
Beta test is a “live”application of the software in an environm entthatcan not be controlled by
thedeveloper
The customer records allproblems and reports to the developers atregularintervals
After modifications,softw are is released forentire customer base

5/22/2024
 SystemTesting
In system testing the software and other system elements are tested.
To test computer software, you spiral out in a clockwise direction along streamlines
that increase the scope of testing with each turn.
System testing verifies that all elements mesh properly
and overall system function/performance is achieved.
System testing is actually a series of different tests whose primary purpose is to fully
exercise the computer-based system.

Types of System Testing

1 Recovery Testing 4 Performance Testing

2 Security Testing 5 Deployment Testing
3 Stress Testing

5/22/2024
 Types of SystemTesting
Recovery Testing
It is a system test that forces the software to fail in a variety of ways
and verifies that recovery is properly performed.
If recovery is automatic (performed by the system itself)
Re-initialization, check pointing mechanisms, data recovery, and restart are evaluated for correctness.
If recovery requires human intervention
The mean-time-to-repair (MTTR) is evaluated to determine whether it is

Security Testing
It attempts to verify software’s protection
mechanisms, which protect it from improper penetration
(access).
During this test, the tester plays the role of the individual who
desires to penetrate the system.

5/22/2024
 Types of SystemTesting Cont.
Stress Testing
It executes a system in a manner that demands
resources in abnormal quantity, frequency or volume.
A variation of stress testing is a technique called sensitivity testing.

Performance Testing
It is designed to test the run-time performance of software.
It occurs throughout all steps in the testing process.
Even at the unit testing level, the performance of an individual module may be tested.

5/22/2024
 Types of SystemTesting Cont.
Deployment Testing
It exercises the software in each environment in
which it is to operate.
In addition, it examines
All installation procedures
Specialized installation software that will be used by customers
All documentation that will be used to introduce the software to end users

5/22/2024
 Acceptance Testing
It is a level of the software testing where a system is tested for acceptability.
The purpose of this test is to evaluate the
system’scompliance with the business requirements.
It is a formal testing conducted to determine whether or not a system satisfies the
acceptance criteria with respect to user needs, requirements, and business processes
It enables the customer to determine, whether or not to accept the system.
It is performed after System Testing and before making the system available for actual use.

5/22/2024
 Views of Test Objects
Black Box Testing White Box Testing Grey Box Testing
Close Box Testing Open Box Testing Partial knowledge
Testing based only Testing based on of source code
on specification actual source code

5/22/2024
 Black Box Testing
Also known asspecification-based testing
Tester hasaccess only to running code and thespecification itis supposed to satisfy
Test cases are written with no knowledge of internal workings ofthecode
No access to source code
So test cases don’t worry aboutstructure
Emphasis is only on ensuring thatthecontract is met
Advantages
Scalable; notdependenton sizeofcode
Testing needs no knowledge ofimplementation
Tester and developercan be truly independent ofeach other
Tests are done with requirements in mind
Doesnot excuse inconsistencies in thespecifications
Test cases can be developed in parallel with code
33

5/22/2024
 Black Box Testing Cont.
Disadvantages Test Case Design
Examine pre-condition, and identify equivalence Test size will have to be small
classes Specifications must be
Allpossible inputs such that all classes clear, concise, and correct
are covered May leave many program paths
Apply the specification to untested
input to write down Weighting of program paths
expected output is not possible
Test Case 1
Input: x1 (sat. X)
Specification Specification- Exp. Output: y2
Operation op
Based Test
Pre: X
Post: Y Case Design Test Case 2
Input: x2 (sat. X)
Exp. Output: y2

5/22/2024
 Black Box Testing Cont.
Exhausting testing is not always possible when there is a large set of input
combinations, because of budget and time constraint.
The special techniques are needed which select test-cases smartly from the all
combination of test-cases in such a way that all scenarios are covered

Two techniques are used

1 EquivalencePartioning 2 BoundaryValueAnalysis (BVA)

Equivalence Partitioning
Input data fora program unit usually falls into a number of partitions,
e.g.allnegativeintegers,zero,allpositive num bers
Each partition ofinputdata makes the programbehave in a similar way
Two test cases based on m em bers from the same partition is likely to
reveal the same bugs

5/22/2024
 Equivalence Partitioning (Black Box Testing)
Byidentifying and testing one member of each partition we gain 'good' coverage with 'small'
num beroftestcases
Testing one member ofapartition should be as good as testing any member ofthepartion

Example - Equivalence Partitioning

For binary search the following partitions exist Pick specific conditions ofthearray
Inputs thatconform to pre-conditions Thearrayhasa single value
Inputs where theprecondition is false Arraylength is even
Inputs where thekey element is a member ofthearray Arraylength is odd
Inputs where thekey element is not a member ofthearray

5/22/2024
 Equivalence Partitioning (Black Box Testing) Cont.

Example - Equivalence Partitioning

Example: Assume that we have to test field which accepts SPI (Semester Performance
Index) as input (SPI range is 0 to 10)

SPI *Accepts value0 to 10

Equivalence Partitioning
Invalid Valid Invalid
<=-1 0 to 10 >=11

Valid Class: 0 – 10,pickanyoneinputtestdata from 0 to 10

Invalid Class 1: <=-1,pickanyone inputtestdata lessthanorequalto -1
Invalid Class 2: >=11,pickanyoneinputtestdata greaterthanorequalto 11

5/22/2024
 Boundary Value Analysis (BVA) (Black Box Testing)
It arises from the fact that most program fail at input boundaries
Boundary testing is the process of testing between extreme ends or boundaries
between partitions of the input values.
In Boundary Testing, Equivalence Class Partitioning plays a good role
Boundary Testing comes after the Equivalence Class Partitioning
The basic idea in boundary value testing is to select input variable values at their:

Justabove them inim um

Just below the Minimu
Justabove them axim um
minimum Just below m
Boundary
the maximum Maximu
m
Boundary Values

5/22/2024
 Boundary Value Analysis (BVA) (Black Box Testing)
Suppose system asksfor“a num berbetw een 100 and 999 inclusive”
The boundaries are 100 and 999
99 100 101 999 999 1000
W etherefore test for values
Lowerboundary Upperboundary
BVA - Advantages
The BVA is easy to use and remember because of the uniformity of identified tests and the
automated nature of this technique.
One can easily control the expenses made on the testing by controlling the number of identified
test cases.
BVA is the best approach in cases where the functionality of a software is based on
numerous variables representing physical quantities.
The technique is best at user input troubles in the software.
The procedure and guidelines are crystal clear and easy when it comes to determining the
test cases through BVA.
The test cases generated through BVA are very small.

5/22/2024
 Boundary Value Analysis (BVA) (Black Box Testing) Cont.
BVA - Disadvantages
This technique sometimes fails to test all the potential input values. And so, the
results are unsure.
The dependencies with BVA are not tested between two inputs.
This technique doesn’t fit well when it comes to Boolean Variables.
It only works well with independent variables that depict quantity.

5/22/2024
 White Box Testing
Also known asstructural testing
W hite Box Testing is a softw are testing method in which the internal
structure/design/implementation ofthem odule being tested is known to the tester
Focusis on ensuring thateven abnormal invocations are handled gracefully
Using white-boxtesting m ethods,you canderive test cases that
Guarantee thatallindependent paths within a m odule havebeen exercised at least once
Exercise alllogical decisions on theirtrue and false sides
Execute allloops attheirboundaries
Exercise internal data structures to ensure theirvalidity

...our goal is to ensure

that all statements and
conditions have been
executed at least once ...

5/22/2024
 White Box Testing Cont.
It is applicable to the following levels of software
testing
Unit Testing: For testing paths within a unit
Integration Testing: For testing paths between units
System Testing: For testing paths between subsystems

Advantages Disadvantages
Testing can be Since tests can be very complex, highly
commenced at an skilled resources are required, with thorough
earlier stage as one need knowledge of programming and implementation
not wait for the GUI to Test script maintenance can be a burden,
be available.
Testing is more thorough, if the implementation changes too frequently
with the possibilty of Since this method of testing is closely tied with
covering most paths the application being testing, tools to cater to every
kind of implementation/platform may not be readily
available
42

5/22/2024
White-box testing strategies
One white-box testing strategy is said to be stronger than another strategy, if all types of
errors detected by the first testing strategy is also detected by the second testing strategy,
and the second testing strategy additionally detects some more types of errors.
White-box testing strategies
1 Statement coverage 2 Branch coverage 3 Path coverage
Statement coverage
It aims to design test cases so that every statement in a program is executed at
least once
Principal idea is unless a statement is executed, it is very hard to determine if an error
exists in that statement
Unless a statement is executed, it is very difficult to observe whether it causes failure
due to some illegal memory access, wrong result computation, etc.

5/22/2024
White-box testing strategies Cont.
ConsidertheEuclid’s GCDcomputation algorithm

int compute_gcd(x, By choosing the test set {(x=3,

y) int x, y; y=3), (x=4, y=3), (x=3, y=4)},
exercisewethe program
can such that all
{ statements are executed at least
1 while (x! = y){ once.
2 if (x>y) then
3 x= x – y; else
4 y= y – x;
5 }
6 return x;
}

5/22/2024
White-box testing strategies Cont.
Branch coverage
In the branch coverage based testing strategy, test cases are designed to make each
branch condition to assume true and false values in turn
It is also known as edge Testing as in this testing scheme, each edge of a program’s
control flow graph is traversed at least once
Branch coverage guarantees statement coverage, so it is stronger strategy
compared to Statement Coverage.
Path Coverage
In this strategy test cases are executed in such a way that every path is executed at
least once
All possible control paths taken, including
All loop paths taken zero, once and multiple items in technique
The test case are prepared based on the logical complexity measure of the procedure design
Flow graph, Cyclomatic Complexity and Graph Metrices are used to arrive at basis
path.

5/22/2024
White-box testing strategies Cont.
Branch coverage Path Coverage
I the branch coverage
n based testing In this strategy test cases are executed
strategy, test cases are designed to in such a way that every path is
make each branch condition to executed at least once
assume true and false values in All possible control paths taken, including
turn
All loop paths taken zero, once and multiple
It is also known as edge Testing as items in technique
in this testing scheme, each edge of a The test case are prepared based on the logical
program’s control flow graph is complexity measure of the procedure design
traversed at least once Flow graph, Cyclomatic
coverage,
Branch so coverage it is stronger strategy Complexity and Graph Metrices are
com pareguarantees statement
d to Statem entCoverage. used to arrive at basis path.

5/22/2024
 Grey Box Testing
Combination ofwhite box and black box testing
Tester hasaccess to source code,butusesitin arestricted manner
Test cases are stilwritten using specifications based on expected outputs forgiven input
Thesetest cases are informed byprogramcode structure

5/22/2024
 Testing Object Oriented Applications
Unit Testing in the OO Context
The concept of the unit testing changes in
object-oriented software
Encapsulation drives the definition of classes and objects
Means, each class and each instance of a class (object) packages
attributes (data) and the operations (methods or services) that
manipulate these data
Rather than testing an individual module, the smallest testable unit is
the encapsulated class
Unlike unit testing of conventional software,
which focuses on the algorithmic detail of a module and the data that flows
across the module interface,
class testing for OO software is driven by the operations encapsulated
by the class and the state behavior of the class

5/22/2024
 Integration Testing in the OOContext
Object-oriented software does not have a hierarchical control structure,
conventional top-down and bottom-up integration strategies have little meaning
There are two different strategies for integration testing of OO systems.
1.Thread-based testing
▪ integrates the set of classes required to respond to one input or event for the system
▪ Each thread is integrated and tested individually
▪ Regression testing is applied to ensure that no side effects occur
2.Use-based testing
▪ begins the construction of the system by testing those classes (called independent classes) that use very few (if
any) of server classes
▪ After the independent classes are tested, the next layer of classes, called dependent classes, that use
the independent classes are tested
Cluster testing is one step in the integration testing of OO software
Here, a cluster of collaborating classes is exercised by designing test cases that attempt
to uncover

5/22/2024
 Validation Testing in an OOContext
At the validation or system level, the details of class connections disappear
Like conventional validation, the validation of OO software focuses on user-visible
actions and user-recognizable outputs from the system
To assist in the derivation of validation tests, the tester should draw upon use cases that
are part of the requirements model
Conventional black-box testing methods can be used to drive validation tests

5/22/2024
 Testing Web Applications
WebApp testing is a collection of related activities with
a single goal to uncover errors in WebApp content,
function, usability, navigability, performance, capacity,
and security
To accomplish this, a testing strategy that encompasses
both
reviews and executable testing is applied.
Dimensions of Quality
Content is evaluated at both a syntactic and semantic
level.
At the syntactic level spelling, punctuation, and
grammar are assessed for text-based documents.
At a semantic level correctness of information
presented, Consistency across the entire content
object and related objects, and lack of ambiguity are all
assessed.
5/22/2024
 Dimensions of Quality Cont.
Function is tested to uncover errors that indicate lack of conformance to customer
requirements
Structure is assessed to ensure that it properly delivers WebApp content
Usability is tested to ensure that each category of user is supported by the interface and
can learn and apply all required navigation.
Navigability is tested to ensure that all navigation syntax and semantics are exercised
to uncover any navigation errors
Ex., dead links, improper links, and erroneous links
Performance is tested under a variety of operating conditions, configurations and
loading
to ensure that the system is responsive to user interaction and handles extreme loading
Compatibility is tested by executing the WebApp in a variety of different host
configurations on both the client and server sides
Interoperability is tested to ensure that the WebApp properly interfaces with other
applications and/or databases
Security is tested by assessing potential vulnerabilities
5/22/2024
 Content Testing User Interface Testing
Errors in WebApp content can be Verification and validation of a WebApp
as trivial as minor typographical errors or
user interface occurs at three distinct
as significant as incorrect information, improper
organization, or violation of intellectual property
points
laws 1.During requirements analysis
Content testing attempts to uncover the interface model is reviewed to ensure that it
these and many other problems before conforms to stakeholder requirements
the user encounters them 2.During design
Content testing combines both the interface design model is reviewed to
reviews and the generation of executable ensure that generic quality criteria established for all
user interfaces have been achieved
test cases
3.During testing
Reviews are applied to uncover
the focus shifts to the execution of application-
semantic errors in content specific aspects of user interaction as they are
Executable testing is used to manifested by interface syntax and semantics.
uncover content errors that can be In addition, testing provides a
traced to dynamically derived content final assessment of usability
that is driven by data acquired from one or
more databases.
5/22/2024
 Component-Level Testing Navigation Testing
Component-level testing The job of navigation testing is to
(function testing), focuses on a set ensure that
of tests that attempt to uncover the mechanisms that allow the WebApp user to
errors in WebApp functions. travel through the WebApp are all functional
and,
Each W ebApp function is a softw are to validate that each Navigation Semantic Unit
component (implemented in one of a (NSU) can be achieved by the appropriate user
variety of programming languages) category
WebApp function can be tested using black- Navigation mechanisms should be
box (and in some cases, white-box)
techniques.
tested
are
Component-level test Navigation links,
cases are often Redirects,
driven by forms-level input. Bookmarks,
Once forms data are defined, the user selects a Frames and framesets,
button or other control mechanism to initiate Site maps,
execution.
Internal search engines.

5/22/2024
 Configuration Testing
Security Testing
Configuration variability and instability Security tests are designed to
are important factors that make WebApp probe
testing a challenge. vulnerabilities of the client-side
Hardware, operating system(s), environment,
browsers, storage capacity, network the network communications that occur as
data are passed from client to server and back
communication speeds, and a variety of other again, and
client-side factors are difficult to predict for the server-side environment.
each user.
One user’s impression of the WebApp and
the manner in which he/she interacts with security tester to uncover
it can differ significantly.
weaknesses
Configuration testing is to test a set of that can be exploited by those with the
probable client-side and server-side intent to do so.
configurations
to ensure that the user experience will be the same
on all of them and,
to isolate errors that may be specific to a particular
configuration
5/22/2024
 Performance Testing
Performance testing is used to uncover
performance problems thatcan resultfrom lack of server-side resources,
inappropriate netw ork bandwidth,
inadequate database capabiiltes,
faulty orweak operating systemcapabiiltes,
poorly designed W ebApp functionality,and
otherhardware or software issues thatcan lead to degraded client-serverperformance

5/22/2024