UNIT 5 - TESTING

TESTING

It is the process of evaluating a system or its component(s) with the intent to

find whether it satisfies the specified requirements or not.

According to IEEE standard, Testing is - Process of analyzing a software

item to detect the differences between existing and required conditions
(that is defects/errors/bugs) and to evaluate the features of the software
item.
What is Testing?
 Better definition: Testing is executing a program to find errors.
Why Test?
 Early testing saves money because fixing early errors costs less.
Who Should Test?
 Developers find it hard to spot their own mistakes.
 Testing by others (special testers) is better because they aim to find errors
objectively.
What to Test?
 Complete testing is not possible; focus on critical areas with high error chances.
 Use strategies to pick effective tests instead of testing randomly.
Key Terminologies
 Error/Mistake: A human fault like a typo or misunderstanding.
 Bug: Mistake in code.
 Fault: Error visible in diagrams or code.
 Failure: When the program doesn’t work as expected during execution.
Test Case and Test Suite
 Test Case: Includes input, expected output, and conditions to check program
behavior.
 Test Suite: A collection of multiple test cases.

VERIFICATION AND VALIDATION

Software Engineering CO 1
 Sr.No. Verification Validation
1 Verification addresses the concern: "Are you building Validation addresses the concern: "Are you
it right?" building the right thing?"
2 Ensures that the software system meets all the Ensures that the functionalities meet the
functionality. intended behavior.
3 Verification takes place first and includes the checking Validation occurs after verification and mainly
for documentation, code, etc. involves the checking of the overall product.
4 Done by developers. Done by testers.
5 It has static activities, as it includes collecting reviews, It has dynamic activities, as it includes
walkthroughs, and inspections to verify software. executing the software against the
requirements.
6 It is an objective process and no subjective decision It is a subjective process and involves
should be needed to verify software. subjective decisions on how well software
works.

TESTING AND DEBUGGING

Testing −

It involves identifying bug/error/defect in a software without correcting it.

Testing is performed in the testing phase.

Debugging −

It involves identifying, isolating, and fixing the problems/bugs.

Developers who code the software conduct debugging upon encountering an
error in the code.

It is a part of White Box Testing or Unit Testing.

It can be performed in the development phase while conducting Unit Testing

or in phases while fixing the reported bugs.

Software Engineering CO 2
 TYPES OF TESTING

The different types of testing that

may be used to test a software
during SDLC.

MANUAL TESTING

It includes testing a
software manually, i.e., without
using any automated tool or any
script.

In this type, the tester takes over the role

of an end-user and tests the software to
identify any unexpected behavior or
bug.

Testers use test plans, test cases, or test scenarios to test a software
to ensure the completeness of testing.

AUTOMATION TESTING

It is also known as Test Automation, is when the tester writes scripts and uses
another software to test the product.

This process involves automation of a manual process.

It is used to re-run the test scenarios that were performed manually, quickly,
and repeatedly

Apart from regression testing, automation testing is also used to test the
application from load, performance, and stress point of view.

It increases the test coverage, improves accuracy, and saves time and
money in comparison to manual testing.

What to Automate?

It is not possible to automate everything in a software.

The areas at which a user can make transactions such as the login form or
registration forms, any area where large number of users can access the
software simultaneously should be automated.Furthermore, all GUI items,
Software Engineering CO 3
 connections with databases, field validations, etc.can be efficiently tested by
automating the manual process.

Software Testing - Methods

There are different methods that can be used for software testing.

A. BLACK BOX TESTING

The technique of testing without having any knowledge of the interior

workings of the application is called black-box testing.

The tester has no knowledge of the system's internal structure and cannot
access the source code.

While performing a black-box test, a tester will interact with the system's
user interface by providing inputs and examining outputs without knowing
how and where the inputs are worked upon.

Software Engineering CO 4
 B. WHITE BOX TESTING

This is the detailed investigation of internal logic and structure of the code.

White-box testing is also called glass testing or open-box testing.

In order to perform white-box testing on an application, a tester needs to

know the internal workings of the code.

The tester needs to have a look inside the source code and find out which
unit/chunk of the code is behaving inappropriately

Advantages Disadvantages
Testers know the code, so testing the app is easier. Skilled testers are costly.
Helps in improving the code. Hard to find all errors, as some paths remain
unchecked.
Removes extra code, reducing hidden bugs. Needs special tools for testing and maintenance is
difficult.
Testers’ code knowledge ensures better testing
coverage.
C. GREY BOX TESTING

It is a technique to test the application with having a limited knowledge of the

internal workings of an application.
Unlike black-box testing, where the tester only tests the application's user
interface; in grey-box testing, the tester has access to design documents
and the database.

Having this knowledge, a tester can prepare better test data and test
scenarios while making a test plan.
COMPARISON BETWEEN BLACK, GREY AND WHITE BOX TESTING

Software Engineering CO 5
 Black Box Testing Gray Box Testing White Box Testing
Tester doesn’t know the code inside. Tester knows some parts of the Tester knows the full internal code.
code.
Also called closed-box or functional Also called translucent testing. Also called clear-box or structural
testing. testing.
Done by end-users, testers, and Done by end-users, testers, and Mostly done by testers and
developers. developers. developers.
Focuses on what the app should do. Uses diagrams like flowcharts. Focuses on how the app works
inside.
Simple and less time-consuming. Moderate time and effort needed. Most detailed and time-
consuming.
Not useful for testing algorithms. Not useful for testing algorithms. Best for testing algorithms.
Uses trial-and-error testing. Tests some internal data if available. Tests internal data thoroughly.ll

LEVELS OF TESTING
There are different levels during the process of testing.

Levels of testing include different methodologies that can be used while

conducting software testing.

The main levels of software testing are −

FUNCTIONAL TESTING

This is a type of black-box testing that is based on the specifications of the

software that is to be tested.

The application is tested by providing input and then the results are
examined that need to conform to the functionality it was intended for.

Step What it means

I Understand what the app is supposed to do.
II Create test data based on the app's requirements.
III Check what the app produces (output) using the test data.
IV Write test scenarios and run the tests.
V Compare the app's actual results with what is expected.

An effective testing practice will see the above steps applied to the testing
policies of every organization and hence it will make sure that the
organization maintains the strictest of standards when it comes to
softwarequality.

CLASSIFICATION OF FUNCTIONAL TESTING

A. UNIT TESTING
Software Engineering CO 6
 This type of testing is performed by developers before the setup is handed
over to the testing team to formally execute the test cases.

Unit testing is performed by the respective developers on the individual

units of source code assigned areas.

The developers use test data that is different from the test data of the quality
assurance team.

The goal of unit testing is to isolate each part of the program and show that
individual parts are correct in terms of requirements and functionality.

Limitations of Unit Testing

Testing cannot catch each and every bug in an application.

It is impossible to evaluate every execution path in every software

application.

The same is the case with unit testing.

There is a limit to the number of scenarios and test data that a developer can
use to verify a source code.

After having exhausted all the options, there is no choice but to stop unit
testing and merge the code segment with other units.

B. INTEGRATION TESTING

Integration testing is defined as the testing of combined parts of an

application to determine if they function correctly.

Integration testing can be done in two ways:

Bottom-up integration testing and Top-down integration testing.

In a comprehensive software development environment, bottom-up testing is
usually done first, followed by top-down testing.

The process concludes with multiple tests of the complete application,

preferably in scenarios designed to mimic actual situations

Software Engineering CO 7
 C. SYSTEM TESTING

System testing tests the system as a whole.

Once all the components are integrated, the application as a whole is tested
rigorously to see that it meets the specified Quality Standards.

This type of testing is performed by a specialized testing team.

System testing is important because of the following reasons −

System testing is the first step in the Software Development Life Cycle, where
the application is tested as a whole.

The application is tested thoroughly to verify that it meets the functional

and technical specifications.

The application is tested in an environment that is very close to the production

environment where the application will be deployed.

System testing enables us to test, verify, and validate both the business
requirements as well as the application architecture.

D. ACCEPTANCE TESTING

This is arguably the most important type of testing.

As it is conducted by the Quality Assurance Team who will gauge whether

the application meets the intended specifications and satisfies the clientʼs
requirement.

The QA team will have a set of pre-written scenarios and test cases that will
be used to test the application.

More ideas will be shared about the application and more tests can be
performed on it to gauge its accuracy and the reasons why the project was
initiated.
Software Engineering CO 8
 Acceptance tests are not only intended to point out simple spelling mistakes,
cosmetic errors, or interface gaps, but also to point out any bugs in the
application that will result in system crashes or major errors in the application.

By performing acceptance tests on an application, the testing team will reduce

how the application will perform in production. There are also legal and
contractual requirements for acceptance of the system

ALPHA TESTING

This test is the first stage of testing and will be performed amongst the teams
(developer and QA teams).

Unit testing, integration testing and system testing when combined

together is known as alpha testing.

During this phase, the following aspects will be tested in the application−

Spelling Mistakes

Broken Links

Cloudy Directions

The Application will be tested on machines with the lowest specification to test
loading times and any latency problems.

BETA TESTING

This test is performed after alpha testing has been successfully

performed.

In beta testing, a sample of the intended audience tests the application.

Beta testing is also known as pre-release testing.

Beta test versions of software are ideally distributed to a wide audience on the
Web, partly to give the program a "real-world" test and partly to provide a
preview of the next release.

In this phase, the audience will be testing the following −

Users will install, run the application and send their feedback to the project
team.

Typographical errors, confusing application flow, and even crashes.

Software Engineering CO 9
 Getting the feedback, the project team can fix the problems before releasing
the software to the actual users.

The more issues you fix that solve real user problems, the higher the quality of
your application will be.

Having a higher-quality application when you release it to the general public

will increase customer satisfaction

A. BOUNDARY VALUE ANALYSIS

Software Engineering CO 10
 B. ROBUSTNESS TESTING

Boundary value wale me hum

crititcal points pr check krte h (
leking bouondary ke andar)

Magar robustness wale me hum

boundardy wale test cases ke alawa
2 test cases boundary ke bahar ke
bhi add karte h

Software Engineering CO 11
 Inn me hum similar to boundary ki
tarah krte h bss jab hum ek ke
critical points lete h toh dussre ki
nominal value lene ki jarurat nhi
C. WORST CASE TESTING uske bhi hume critical lene h

D. EQUIVALNCE CLASS TESTING

Isme hume na kewal

boundary cases but
condition bhi likni hoti h ki
kab humara valid case
hoga

just like here we have to

check when our points
Software
form Engineering CO
a valid triangle 12
 E. DECISION TABLE TESTING

Hame har column ke lie test

cases generate krna padega
C1,c2,c3,c4,c5,c6 etc yehsare
cases ke lie kuch condn true h
kuch false unhe satisfy krne
wala target

Software Engineering CO 13
 - CAUSE EFFECT GRAPH

NON FUNCTIONAL TESTING

Non-functional testing involves testing a software from the requirements

which are non functional in nature but important such as performance,
security, user interface, etc.

Some of the important and commonly used non-functional testing types are
a. Performance Testing

It is mostly used to identify any bottlenecks or performance issues rather than

Software Engineering CO 14
 finding bugs in a software.

There are different causes that contribute in lowering the performance of

a software −

Network delay

Client-side processing

Database transaction processing

Load balancing between servers

Performance testing is considered as one of the important and mandatory

testing type in terms of the following aspects −

Speed (i.e. Response Time, data rendering and accessing)

Capacity

Stability

Scalability

Performance testing can be either qualitative or quantitative and can be

divided into different sub-types such as Load testing and Stress testing.

b. Load Testing

It is a process of testing the behavior of a software by applying maximum load

in terms of software accessing and manipulating large input data.

It can be done at both normal and peak load conditions.

This type of testing identifies the maximum capacity of software and its
behavior at peak time.

Most of the time, load testing is performed with the help of automated tools
such as Load Runner, AppLoader, IBM Rational Performance Tester,
Apache JMeter, Silk Performer, Visual Studio Load Test, etc.

Virtual users VUsers) are defined in the automated testing tool and the
script is executed to verify the load testing for the software.
The number of users can be increased or decreased concurrently or
incrementally based upon the requirements.

c. Stress Testing

Stress testing includes testing the behavior of a software under abnormal

Software Engineering CO 15
 conditions.

For example, it may include taking away some resources or applying a load
beyond the actual load limit.

The aim of stress testing is to test the software by applying the load to the
system and taking over the resources used by the software to identify
the breaking point.

This testing can be performed by testing different scenarios such as −

Shutdown or restart of network ports randomly

Turning the database on or off

Running different processes that consume resources such as CPU, memory,

d. Portability Testing

Portability testing includes testing a software with the aim to ensure its
reusability and that it can be moved from another software as well.

Following are the strategies that can be used for portability testing −

Transferring an installed software from one computer to another.

Building executable (.exe) to run the software on different platforms

Structural Software Testing (White Box Testing)

It is a type of software testing that uses the internal design of the software for
testing.

In other words the software testing which is performed by the team which
knows the development phase of the software, is known as structural testing.

Software Engineering CO 16
 It is related to the internal design and implementation of the software i.e. it
involves the development team members in the testing team.

It tests different aspects of the software according to its types.

Structural testing is just the opposite of behavioral testing.

There are 4 types of Structural Testing:

1. Mutation testing
It is used to check the quality of the test case that should fail the mutant code.

Mutation testing involves the development of new tests to be implemented on

the software for its testing process.

When we identify various errors, it implies that either the program is correct or
the test case is inefficient in locating the fault.

In the mutation testing, the developers make small modifications to the

previously accessible software tests and generate a mutant of the old
software test.

It used to cause an error in the program, which implies that the mutation
testing is performed to evaluate the test case's productivity.

2. Slice-based testing
It was initially created and established to keep the software.

The basic idea is to sort the complete code into small chunks and then
evaluate each portion carefully.

Software Engineering CO 17
 The slice-based testing is very beneficial for the maintenance of the software
along with fixing the application too.

Note: The developers can use these four types of structural testing as per their
requirements.

3. Control flow or Path testing

Basic model of Structural testing.

It is used to check the implementation order of commands or statements of

the code over a control structure.

Specific part of an extensive program is selected by the test engineer to set

the testing path.

Generally, the control flow testing technique is used in unit testing.

In this testing, the entire test is based on how the control is executed during
the code.

The complete information of all the software's features and logic is necessary
to execute the control flow testing.

In this testing method, the control flow graph of a program is designed to find
a set of linearly independent paths of execution.

In this method, Cyclomatic Complexity is used to determine the number of

linearly independent paths and then test cases are generated for each path.

To design test cases using this technique, four steps are followed :

Construct the Control Flow Graph

Identify the Independent Paths

Compute the Cyclomatic Complexity of the Graph

Design Test cases from Independent Paths

CONTROL FLOW GRAPH Directed graph which represents the control structure
of a program or module. V, E has V number of nodes/vertices and E number of
edges in it.

Software Engineering CO 18
 A control graph can also have :

Junction Node – a node with more than one arrow entering it.

Decision Node – a node with more than one arrow leaving it.

Region – area bounded by edges and nodes (area outside the graph is also
counted as a region.).

DD PATH GRAPH

Flow graph generation is the first step of path testing.

The second step is to draw a DD path graph from the flow graph.

The DD Path graph is known as decision to decision path graph. Here, we

concentrate only on decision nodes.

The nodes of flow graph, which are in a sequence are combined into a single
node.

DD Path graph is a directed graph in which nodes are sequences of

statements and edges represent control flow between nodes.

Software Engineering CO 19
Software Engineering CO 20
 INDEPENDENT PATHS

An independent path in the control flow graph is the one which introduces at
least one new edge that has not been traversed before the path is defined.

The cyclomatic complexity gives the number of independent paths present in

a flow graph.

Software Engineering CO 21
 NOTE CONTROL FLOW GRAPH IS NOT UNIQUE

Graph Matrices
Whenever graphs are used for testing, we are interested to find independent
paths.

Graph matrix is a square matrix with one row and one column for every node
in the graph.

Size of the matrix (i.e., the number of rows and columns) is equal to the
number of nodes in the flow graph.

There is a place to put every possible direct connection between any |node
and any other node.

If there are several links between two nodes, then the entry is a sum ; the ‘+ʼ
sign denotes parallel links.

Software Engineering CO 22
 Graph matrix is nothing but the tabular representation of a flow graph.

If we assign weight to each entry, the graph matrix can be

used for evaluating useful information required during testing.

Simplest weight is 1, if there is a connection and 0 if there is no connection.

Matrix with such weights is called a connection matrix.

Software Engineering CO 23
 The Kth power of matrix represents all paths of K links long.

4. Data flow testing

It is a group of testing approaches used to observe the control flow of
programs to discover the sequence of variables as per the series of events.

Software Engineering CO 24
 It implements a control flow graph and analysis the points where the codes
can change the data.

If we execute the data flow testing technique, the information is kept safe and
unchanged during the code's implementation.

Some Terminology

Structural Testing Tools

Software Engineering CO 25
 Structural testing also contains some open-source and commercial tools that
have their functionality.

Some of the most commonly used tools for structural testing are as follows:
Cucumber, JBehave , Cfix , JUnit

Software Engineering CO 26