Software Quality

Assurance & Testing

Lecture 8

Software Testing 1
 Software Testing
Software Testing is a method to check whether the actual software product
matches expected requirements and to ensure that the software product is
defect-free.

It involves the execution of software/system components using manual or

automated tools to evaluate one or more properties of interest.

The purpose of software testing is to identify errors, gaps or missing

requirements in contrast to actual requirements.
 Why Testing
• The purpose of software testing is to ensure that the software systems
would work as expected when they are used by their target customers and
users.
• The most natural way to show this fulfillment of expectations is to
demonstrate their operation through some “dry-runs” or controlled
experimentation in laboratory settings before the products are released or
delivered.
• In the case of software products, such controlled experimentation through
program execution is generally called testing.
 Two primary purpose

Testing fulfills two primary purposes:

• To demonstrate quality or proper behavior;

• To detect and fix problems.

 Why Testing is Important
• Software Testing is Important because if there are any bugs or errors in
the software, it can be identified early and can be solved before delivery
of the software product.

• Properly tested software product ensures reliability, security and high

performance which further results in time saving, cost effectiveness and
customer satisfaction.

• Testing is important because software bugs could be expensive or even

dangerous. Software bugs can potentially cause monetary and human
loss, and history is full of such examples.
 Major activities and the generic testing
process

• Test planning and preparation, which set the goals for testing, select an
overall testing strategy, and prepare specific test cases and the general
test procedure.

• Test execution and related activities, which also include related

observation and measurement of product behavior.

• Analysis and follow-up, which include result checking and analysis to

determine if a failure has been observed, and if so, follow-up activities are
initiated and monitored to ensure removal of the underlying causes, or
faults, that led to the observed failures in the first place.
Generic Testing Process
 Sub-activities in test planning and preparation
• Goal setting: The quality views and attributes have been decided by the
overall quality engineering process. What remains to be done is the specific
testing goals, such as reliability or coverage goals, to be used as the exit
criteria.

• Test case preparation: It includes constructing new test cases or

generating them automatically, selecting from existing ones for legacy
products, and organizing them in some systematic way for easy execution
and management.

• Test procedure preparation: It can be defined and followed to ensure

effective test execution, problem handling and resolution, and the overall
test process management.
 Basic questions about testing
• What artifacts are tested? The primary types of objects or software
artifacts to be tested are software programs or code written in different
programming languages. Program code is the focus of our testing effort
and related testing techniques and activities.
• What to test, and what kind of faults is found? Black-box (or functional)
testing verifies the correct handling of the external functions provided or
supported by the software, or whether the observed behavior conforms to
user expectations or product specifications. White-box (or structural)
testing verifies the correct implementation of internal units, structures,
and relations among them.
• When, or at what defect level, to stop testing? Most of the traditional
testing techniques and testing sub-phases use some coverage information
as the stopping criterion, with the implicit assumption that higher
coverage means higher quality or lower levels of defects.
 Basic questions about testing techniques

• What is the specific testing technique used?

• What is the underlying model used in a specific testing technique?

• Are techniques for testing in other domains applicable to software

testing?

• If multiple testing techniques are available, can they be combined or

integrated for better effectiveness or efficiency?
 Questions about test activities and management

• Who performs which specific activities?

• When can specific test activities be performed?

• What process is followed for these test activities?

• Is test automation possible? And if so, what kind of automated testing tools are available
and usable for specific applications?

• What artifacts are used to manage the testing process and related activities?

• What is the relationship between testing and various defect-related concepts?

• What is the general hardware/software/organizational environment for testing?

• What is the product type or market segment for the product under testing?
Software Testing Principles
Software Testing Principles
1. Testing shows presence of defects
• The goal of testing is to make the software fail. Sufficient testing reduces the
presence of defects. In case, testers are unable to find defects after repeated
regression testing does not mean that the software is bug free.
• Testing talks about presence of defects and don’t talk about absence of defects.
2. Exhaustive testing is impossible
• Exhaustive testing is a testing or quality assurance approach in which all
possible combinations of scenarios and use/test cases are used for testing.
• Suppose we have to test an input field which accepts age between 18 to 20. So
we do test the field using 18, 19, 20. In case, the same input field accepts the
range between 18 to 100 then we have to test using inputs such as 18, 19, …., 99,
100. It is a basic example, you may think that you could achieve it using
automation tool. Imagine the field accepts some billion values. It is impossible to
test all possible values due to release time constraints.
Software Testing Principles
3. Early testing
• Defects detected in early phases of SDLC are less expensive to fix. So conducting
early testing reduces the cost of fixing defects.
4. Defects Clustering
• Defects clustering in software testing means that a small module or functionality
contains most of the bugs or it has the most operational failures
• As per the Preto principle (80-20 rule), 80% of issues come from 20% of modules
and remaining 20% of issues from remaining 80% of modules. So we do emphasis
testing on the 20% of modules where we face 80% of bugs
5. Pesticide Paradox
• Process of repeating the same test cases again and again, eventually the same test
cases will no longer find new bugs. So to overcome this pesticide paradox, it is
necessary to review the test cases regularly and add or update them to find more
defects.
Software Testing Principles
6. Testing is context dependent
• Testing approach depends on context of the software we develop. We do
test the software differently in different contexts. For example, online
banking application requires a different approach of testing as compared to
e-commerce site.
7. Absence of errors - fallacy
• 99% of bug free software may still be unusable, if wrong requirements were
incorporated into the software and the software is not addressing the
business needs.
Software Testing Life Cycle (STLC)
 Requirement Analysis
• The first step of the manual testing procedure is requirement analysis. In this
phase, tester analyses requirement document of SDLC (Software Development Life
Cycle) to examine requirements stated by the client.
• After examining the requirements, the tester makes a test plan to check whether
the software is meeting the requirements or not.
 Test Plan Creation
• Test plan creation is the crucial phase of STLC where all the testing strategies are defined.
Tester determines the estimated effort and cost of the entire project. This phase takes
place after the successful completion of the Requirement Analysis Phase.
• Testing strategy and effort estimation documents provided by this phase. Test case
execution can be started after the successful completion of Test Plan Creation.
 Environment setup
• Setup of the test environment is an independent activity and can be started along
with Test Case Development. This is an essential part of the manual testing procedure as
without environment testing is not possible
• Environment setup requires a group of essential software and hardware to create a test
environment. The testing team is not involved in setting up the testing environment, its
senior developers who create it
 Test case Execution
• Test case Execution takes place after the successful completion of test planning. In this
phase, the testing team starts case development and execution activity. The testing team
writes down the detailed test cases, also prepares the test data if required. The prepared
test cases are reviewed by peer members of the team or Quality Assurance leader.
• RTM (Requirement Traceability Matrix) is also prepared in this phase. Requirement
Traceability Matrix is industry level format, used for tracking requirements. Each test case
is mapped with the requirement specification. Backward & forward traceability can be
done via RTM.
 Defect Logging
• Testers and developers evaluate the completion criteria of the software based on test
coverage, quality, time consumption, cost, and critical business objectives. This phase
determines the characteristics and drawbacks of the software. Test cases and bug reports
are analyzed in depth to detect the type of defect and its severity.
• Defect logging analysis mainly works to find out defect distribution depending upon
severity and types. If any defect is detected, then the software is returned to the
development team to fix the defect, then the software is re-tested on all aspects of the
testing.
• Once the test cycle is fully completed then test closure report, and test metrics are
prepared.
 Test Cycle Closure
• The test cycle closure report includes all the documentation related to
software design, development, testing results, and defect reports.
• This phase evaluates the strategy of development, testing procedure, possible
defects in order to use these practices in the future if there is a software with
the same specification.
END OF LECTURE