Software Testing

CS4153
PE-VII
By:
Dr. Akshay Jadhav
When do
defects
arise?

FOUNDATIONS OF SOFTWARE
TESTING
ISTQB CERTIFICATION
Dorothy Graham, Erik van Veenendaal,
Isabel Evans, Rex Black
Page 5
Re q u i r e m e n t 1 : T h i s r e q u i re me nt wa s c o r r e c t l y
u n d e r s to o d , d e s i g n e d, b u i l t , a n d t e s t e d, r e s u l ti ng i n
s o f t wa r e t h a t m e e t s b o t h f u n c t i o na l a n d n o n - f u n c ti o n a l
a t t r i but e s .

Re q u i r e m e n t 2 : E r r o r s o c c u r r e d d u r i ng c o d i n g , l e a di ng t o
d e f e c t s . T h e s e a r e t y p i c a l ly e a s y t o s p o t a n d c o r r e c t d u r i ng
testing because the product does not meet the design
s p e c i f i c a ti o n s .

Re q u i r e m e n t 3 : D e f e c ts w e r e i n t r o d uc e d i n t h e d e s i g n
p h a s e d u e t o m i s ta ke s b y t h e d e s i g n e r. T h e s e d e f e c t s a r e
h a r d e r t o d e t e c t d u r i ng t e s t i ng b e c a u s e t h e s o f t wa r e wa s
b u i l t a c c o r d i ng t o t h e f l a w e d d e s i g n, m a k i n g t h e m d i f f i c ul t
and costly to fix.

Re q u i r e m e n t 4 : T h e d e f e c t s o r i g i na te d d u r i n g t h e
r e q u i re me nts d e f i ni ti o n p h a s e . A l t h o u g h t h e s o f t wa r e
m e e t s t h e f l a w e d r e q u i re me n ts a n d d e s i g n , i t m a y b e
r e j e c t e d b y t h e u s e r o r c u s t o m e r. D e f e c ts f o u n d a t t h i s
s t a g e , p a r t i cul a rl y d u r i ng a c c e p ta nc e t e s t i n g o r l i ve u s e ,
a r e ve r y c o s t l y. S t u d i e s s h o w t h a t d e f e c t s f r o m
r e q u i re me nts a n d d e s i g n m a ke u p n e a r l y h a l f o f a l l d e f e c t s
i n s o f t wa r e p r o j e c t s .
Origin of
Defects
A fault (defect) model
can be describe d as a
link between the error
made (e.g., a missing
require me nt , a
misunderst ood design
element, a typographical
error), and the
fault /def e ct in the
sof tware .
 Origin of Defects
Requirements Issues: Defects often originate from unclear, incomplete, or misunderstood requirements. If the
requirements are not correctly captured or interpreted, the software may not meet the users' needs, leading to defects.

Design Flaws: Poor software design or architecture can introduce defects. This may occur due to a lack of
understanding of the system requirements, inappropriate design choices, or failure to consider non-functional
requirements such as performance and security.

Coding Errors: Mistakes made during the coding phase are a common source of defects. These can be due to
typographical errors, logical errors, or a misunderstanding of how a particular function or feature should be implemented.

Testing Gaps: Insufficient or ineffective testing can lead to defects going undetected until later stages of development or
even after deployment. This can happen if the testing process fails to cover all scenarios or if the test cases themselves
are flawed.
Environmental Factors: Differences in the development and production environments, such as hardware, operating
systems, or configurations, can cause defects. For example, software that works in a developer's environment may fail in
the production environment due to these differences.

Human Factors: Human error is a significant contributor to defects. This can stem from a lack of experience, fatigue,
miscommunication among team members, or simple oversight.
Myers approach to testing
Myers has a similar approach to testing. He describes the
successful test as one that reveals the presence of a
(hypothesized) defect. He compares the role of a tester to
that of a doctor who is in the process of construct ing a
diagnosis for an ill patient. The doctor develops hypotheses
about possible illnesses using her knowledge of possible
diseases, and the patient s‘ symptom s. Tests are made in
order to make the correct diagnosis . A successf ul test will
reveal the problem and the doctor can begin treatment .
Completing the analogy of doctor and ill patient, one could
view defective sof tware as the ill patient . Testers as
doctors need to have knowledge about possible defects
(illnesses) in order to develop defect hypotheses.
Myer’s hypotheses

• design test cases;

• design test procedures;

• assemble test sets;

• select the testing levels (unit, integration, etc.)appropriate for the tests;

• evaluate the results of the tests.

What is the cost
of defects?
Th e co s t o f d e fe ct s in s o f twa r e
te s ti ng r e f e r s to th e i n cr e a s ing
e xp e n s e a s s o ci a te d w i th
i d e n tifyin g a n d f i xi n g d e f e cts a t
va r i ou s s ta g e s o f th e s o f twa r e
d e ve lo pm en t l i f ecycle.
What is the cost of defects?

Cost of Fixing Defects : The cost of f inding and f ixing defects increase s signif icant ly as
the sof tware progresses through its development stages. Early detection, such as
during the requirement s or design phases, is much cheaper than f ixing defects found
later, like during acceptance testing or af ter implementat ion.

Early vs. Late Detecti on : If a defect is detected early, such as in the requirem ents or
design stages, it can be corrected with minimal expense. However, if a defect goes
unnoticed until later stages, it becomes much more costly to f ix because rework may be
needed across multiple stages, and previous testing may need to be redone.

Impact of Late Detecti on : Defects found late in the process may sometim es not be
f ixed due to the high cost. Additionally, even if sof tware meets its specif ications, it may
still be rejected by users if those specif ications were f lawed, leading to dissatisf action
or, in extreme cases, the need to de -install the system entirely.
T h e c o s t a s s o c i a t e d w i t h d e f e c t s i n s o f t w a r e c a n b e s i g n i f i c a n t a n d va r i e s d e p e n d i n g
on when the defect is detected and corrected. Costs increase exponentially as
d e f e c t s a r e f o u n d l a t e r i n t h e d e ve l o p m e n t c yc l e o r a f t e r t h e s o f t wa r e h a s b e e n
released.

The key costs include: Cost

• Development Phase Costs : If a defect is detected and fixed during the early
phases (such as during requirements gathering or design), the cost is relatively of
•
l o w. E a r l y d e f e c t d e t e c t i o n m a y i n vo l ve s i m p l e c h a n g e s t o d o c u m e n t s o r d e s i g n s .

Te s t i n g P h a s e C o s t s : D e f e c t s f o u n d d u r i n g t h e t e s t i n g p h a s e a r e m o r e e x p e n s i v e
Defect
to fix because they often require code changes, retesting, and regression testing.
The closer to release the defect is found, the higher the cost due to the potential
need for more extensive changes and testing.

• Po s t - Re l e a s e C o s t s : D e f e c t s t h a t a r e f o u n d a f t e r t h e s o f t w a r e h a s b e e n r e l e a s e d
to customers are the most expensive. These costs include customer support,
p a t c h e s o r u p d a t e s , p o t e n t i a l l e g a l l i a b i l i t i e s , a n d d a m a g e t o t h e c o m p a n y ’s
r e p u t a t i o n . F i x i n g d e f e c t s i n p r o d u c t i o n m a y a l s o i n vo l v e s i g n i f i c a n t d o w n t i m e
and disruption for users.

• Indirect Costs : These include the loss of customer trust, potential loss of
b u s i n e s s , a n d t h e i m p a c t o n t h e d e ve l o p m e n t t e a m ' s m o ra l e a n d p r o d u c t i v i t y.
Long-term, unresolved defects can also lead to increased technical debt.
Defect classes
Defects in sof tware can be
categorized into four major classes
based on their point of origin in the
sof tware development lifecycle:
Requirements /Specification Defects,
Design Defects, Coding Defects, and
Testing Defects. Each class reflects
specific phases where defects are
introduced and provides insight into
how testing can be most effectively
executed.
Requirements /Specification Defects

Functional Description Defects: These involve incorrect,

ambiguous, or incomplete descriptions of what the sof tware should
do.

Feature Defects: Related to missing, incorrect, or unnecessary

features that map to user requirements.

Feature Interaction Defects: Occur when features interact in

unintended ways, causing incorrect behavior.

Inter face Description Defects: Involves errors in describing how

sof tware interfaces with external systems, hardware, or users.
Design Defects

Algorithmic and Processing Defects: Result from incorrect

processing steps in algorithms or misuse of control logic.

Control, Logic, and Sequence Defects: These involve errors in the

logical f low of the sof tware, such as incorrect branching or
sequencing.

Data Defects: Occur when data structures are incorrectly designed,

leading to issues in data handling.

Module Inter face Description Defects: Errors in the interaction

between modules, of ten due to incorrect or inconsistent parameters.
Coding Defects

Algorithmic and Processing Defects: Include unchecked

conditions, incorrect operator usage, and other low -level coding
errors.

Control, Logic, and Sequence Defects: Errors in coding constructs

like loops, conditional statements, and logic operations.

Typographical Defects: Simple syntax errors usually caught by

compilers.

Initialization Defects: Occur when variables or data structures are

incorrectly or not initialized.
Coding Defects

Data-Flow Defects: Involve improper sequences in the usage of

data, such as using uninitialized variables.

Data Defects: Errors in implementing data structures, like incorrect

allocation of memor y or data types.

Module Inter face Defects : Involve issues with how modules

interact, such as incorrect parameter handling.

Code Documentation Defects: Errors in documentation that

mislead testers or cause incorrect tests to be created.

External Hardware, Sof tware Inter face Defects: Issues in

interacting with external systems, hardware, or other sof tware.
Testing Defects

Test Harness Defects: Defects in auxiliary code used to test the

sof tware, which can lead to incorrect test results.

Test Case Design and Test Procedure Defects: Errors in the

design of test cases or procedures, leading to incomplete or
incorrect testing.
 Defect
Prevention
Strategies
1 . Software Requirement Analysis

S o f t w a re r e q u i r em ents a n d d e s i g n b o t h a r e i m p o r t a nt a n d
s h o u l d b e a n a l y z e d i n a n e f f i c i en t w a y w i t h m o r e f o c u s .

R e q u ir em ents t h a t b a s i c a l l y d e s c r i b es f e a t u res an d
f u n c t i ona l i ti es o f t a r g e t p r o d u c t a n d a l s o c o n ve y s e x p e c ta t io ns
o r r e q u i re me nt o f u s e r s f r o m s o f t w a r e p r o d uc t.

I f r e q u i rem ent s a r e n o t u n d e r s too d w e l l b y t e s t e r a n d

d e ve l o p ers , t h e n t h e r e m i g h t b e c h a n c e o f o c c u r ri ng o f i s s u e o r
d e f e c t i n f u r t her p r o c e s s .
2.Review and Inspection : Review and inspection, both are
essential and integral part of software development. They are
considered as powerful tools that can be used to identify and
remove defects if present before their occurrence and impact on
production. Review and inspection come in different levels or stages
of defect prevention to meet different needs. They are used in all
software development and maintenance methods. There are two
types of review i.e., self -review and peer -review.
3.Defect Logging and Documentation : After successful analysis
and review, there should be records maintained about defects to
simply complete description of defect. This record can be further
used to have better understanding of defects. After getting
knowledge and understanding of defect, then only one can take
some effective and required measures and actions to resolve
particular defects so that defect cannot be carried further to next
phase.
4.Root Cause Analysis : Root because analysis is basically analysis
of main cause of defect. It simply analysis what triggered defect to
occur. After analyzing main cause of defect, one can find best way
to simply avoid occurrence of such types of defects next time.
5.Static Code Analysis: To find any problems in the source code
without running the program, use automated techniques for static
code analysis. Before the code is even compiled, these tools can
detect typical programming errors, coding standard violations and
other problems.
6.Pair programming: It involves two programmers sharing a
workstation, where one writes code while the other goes over each
line as it is written. This real -time communication between team
members promotes constant feedback and helps in the early
detection of mistakes.
7.Test-Driven Development (TDD): Prior to writing any code,
write automated tests. This method helps identify errors early in
the development process and guarantees that the code complies
with the requirements.

8.Training and Skill Development: Investing in continuous

training and skill development for team members is
recommended. Skilled developers are better able to follow best
practices and are less prone to make frequent mistakes.

9.Checklists: To make sure that crucial actions are not missed

during the various stages of development, use checklists.
Developers and reviewers can quickly confirm that established
standards are being followed by using checklists as a reference.
Defect Example
Defects:
1. Off-by-One Error:
Loop starts at i = 1, skipping the first element (pennies).
Fix: Start loop with i = 0.

2. Typo in Array Access:

Used coin_value[i] instead of coin_values[i].
Fix: Replace with coin_values[i].

3. Uninitialized Variable:
total_coin_value is not initialized, leading to potential garbage values.
Fix: Initialize it to 0.

4.Incorrect Cents Calculation:

Subtracting dollars from the total to calculate cents can introduce errors.
Fix: Use % 100 for correct cents calculation
Defect Repository
Defect monitoring should continue for each on-going
project. The distribution of defects will change as you
make changes in your processes. The defect data is
useful for test planning, a TMM level 2 maturity goal.
It helps you to select applicable testing techniques,
design (and reuse) the test cases you need, and
allocate the amount of resources you will need to
devote to detecting and removing these defects. This
in turn will allow you to estimate testing schedules
and costs.
A defect repository can help to support achievement
and continuous implementation of several TMM
maturity goals including controlling and monitoring of
test, software quality evaluation and control, test
measurement, and test process improvement.
Software Testing Strategies
Functional and Non-functional

Functional testing

This type of testing evaluates the software application against functional,

business, and customer requirements. It validates each functionality by utilizing
appropriate user inputs, verifying outcomes, and comparing them with expected
results.

Non-functional testing

Non-functional testing focuses on stability, performance, efficiency, portability,

and other areas that are not directly related to specific functionalities. It
encompasses testing features beyond functional requirements.
Aspect Functional Testing Non-functional Testing

Validates specific functionalities of Evaluates non-functional aspects like

Purpose
the software performance and stability

Checks against functional, business, Examines factors such as efficiency,

Focus
and customer requirements portability, reliability

Unit testing, integration testing, Performance testing, load testing,

Examples
system testing security testing

Assesses how well the software performs

Verifies if the software behaves as
Measures under different conditions and
expected under various conditions
environments
Software testing involves diverse methodologies to guarantee quality
and reliability. These encompass manual and automated approaches,
each with distinct advantages for defect detection and functionality
verification.
Aspect Manual testing Automated testing

Test execution Human testers execute test cases Test cases are executed using
process manually. automation tools and scripts.

Speed and Faster execution, especially for

Slower due to manual intervention.
efficiency repetitive tasks.

Test scripts can be reused across

Reusability Test cases may not be easily reusable.
multiple test cycles.

Well-suited for exploratory and ad-hoc Less effective for exploratory testing
Exploratory testing
testing. without human intuition.

Higher initial investment in tools and

Initial investment Lower initial setup and cost.
setup.

Unit testing, integration testing,

Types of Regression testing, load testing,
system testing, user acceptance
tests performed performance testing, stress testing
testing, exploratory testing
TestRail is a web-based test case
management tool. It is used by QA
engineers, developers, and team leads
to manage, track, and organize software
testing efforts. TestRail allows team
members to design test cases, organize
test suites, execute test runs, and track
their results, all from a modern and
easy-to-use web interface.
Software Testing Strategies

Software testing strategies are detailed plans that guide your approach to
testing and outline test objectives, scope, methodologies, and more. These
roadmaps assist in structured testing by specifying what, how, and when to
test. Test strategies furnish essential details for creating test documents.

Software testing strategies are integrated into every SDLC phase, enabling
early bug identification by guiding the review and verification of software
throughout the development lifecycle. Key roles of software testing
strategies throughout the SDLC include:
SDLC Phase Key Testing Strategies
•Requirement validation
Requirements analysis
•Risk analysis
•Comprehensive test planning
Planning
•Resource allocation
•Test case design
Design
•Traceability matrix
•Unit testing
Implementation
•Static analysis
•Integration testing
Integration
•Interface testing
•Functional testing
System •Non-functional testing (performance, security)
•Regression testing
•UAT planning
User acceptance (UAT)
•Beta testing
•Release readiness testing
Deployment
•Compatibility testing
•Maintenance testing
Post-deployment
•Performance monitoring

•Feedback loop for process improvement

Continuous improvement
•Continuous enhancement of test automation
Why different Software Testing Strategies?

Common reasons to use different software testing strategies include:

• Continuous Improvement: Allows flexibility in your testing approach based on new or

changed user requirements.

• Risk mitigation: Helps you mitigate unexpected challenges, like budget constraints, during
the software development process.

• Teamwork: Helps you develop better collaboration with different teams and leverages Agile
and DevOps work culture.

• Technological changes: Integrates new software technologies into your software development
process.

• Scalability: Allows you to scale the test process effectively by addressing agility.
• A meticulously planned sof tware testing strategy acts as the foundation
for your sof tware’s success, guiding you toward continuous improvement
and aligning seamlessly with user expectations.

• Finding the right balance between project dynamics is crucial for sof tware
testing strategies. The ability to adapt to evolving user requirements
ensures successful testing outcomes, delivering benefits to both your
development teams and end-users.

• Bottom line: A well-craf ted software testing strategy is instrumental for

achieving your testing goals, fostering continuous improvement, and
enhancing the overall user experience.
Verification and
Validation
Verification and Validation is the process of
investigating whether a sof tware system
satisfies specifications and standards and
fulfills the required purpose.
Barry Boehm described verification and
validation as the following:
Verification: Are we building the product
right?
Validation: Are we building the right product?
Verification

Verification is the process of checking that software achieves its

goal without any bugs. It is the process to ensure whether the
product that is developed is right or not. It verifies whether the
developed product fulfills the requirements that we have.
Verification is simply known as Static Testing.

1. Inspections 2. Reviews

3. Walkthroughs 4. Desk-checking
Validation

Validation is the process of checking whether the software

product is up to the mark or in other words product has high -
level requirements. It is the process of checking the validation of
the product i.e. it checks what we are developing is the right
product. it is a validation of actual and expected products.
Validation is simply known as Dynamic Testing.

1.Black Box Testing 2. White Box Testing

3.Unit Testing 4. Integration Testing