Software Testing
Software Testing
• Testing can show the presence of bugs, but not the absence
What is software Testing?
Error
An error refers to a human mistake or defect in the software code or
design. It is the root cause that leads to incorrect or unexpected behavior
in a system.
Examples:
Fault (Bug/Defect)
A fault is the manifestation/appearance of an error in the software. It is a
defect or bug in the program’s code or design that can potentially cause
the software to behave incorrectly.
Examples:
Key Point: Faults are introduced during coding or design and may not always be
detected immediately.
Testing Terminologies
Failure
A failure occurs when the system does not perform its intended function
as specified. It is the actual incorrect output or behavior of the software
observed during execution, caused by a fault.
Examples:
Key Point: Failures are observed during runtime when the system is in use.
Relationship Between Error, Fault, and Failure
White-box Black-box
methods methods
Methods
Strategies
Black-Box Testing
requirements Comparison
output
input events
Black-Box Testing
9999 100000
10000 50000 99999
Less than 10000 Between 10000 and 99999 More than 99999
Input values
Boundary Value Analysis
• For each range [R1, R2] listed in either the
input or output specifications, choose five R1 R2
cases:
– Values less than R1
– Values equal to R1
– Values greater than R1 but less than R2
– Values equal to R2
– Values greater than R2
Example: For an input range of 1 to 100:
Test with values like 0, 1, 55, 100, and 101.
Example of Black Box Testing
1. Performance Testing:
Objective: Assess how the system performs under various loads.
Performance testing assesses the application's performance under
specific conditions. It focuses on various aspects such as response time,
load time, and throughput rates under varying levels of user traffic.
Example: Checking if a website can handle 10,000 simultaneous users
without significant slowdown.
2. Load Testing:
Objective: Assess how the system performs under peak loads.
Example: Checking if a website can handle 10,000
simultaneous users without significant slowdown.
Non-Functional Testing
3. Stress Testing:
Objective: Stress testing pushes the software beyond its normal operational
capacity, often to a breaking point, to see how it handles extreme conditions. This
helps identify the application's upper limits and how it fails under stress.
Determine the system’s stability by testing it beyond its limits.
Example: Increasing the user load until the application crashes to identify the
breaking point.
4. Volume Testing:
Objective: To ensure that a software program or system can handle a
large volume of data.
Example: if the website is developed to handle traffic of 500 users,
volume testing will whether the site is able to handle 500 users or not.
Non-Functional Testing
5. Security Testing:
Objective: Security testing is critical in identifying vulnerabilities,
threats, and risks that could potentially lead to data loss,
breaches, or other security incidents. It ensures that the software
can protect data and maintain functionality as intended.
6. Usability Testing:
Objective: This focuses on the user's ease of using the application,
navigability, and overall user experience. Usability testing aims to
ensure that the software is intuitive and user-friendly.
Non-Functional Testing
7. Recovery Testing:
Objective: To ensure that a software program or system can be
recovered from a failure or data loss.
For example, when the application is running and the computer is
restarted, check the validity of the application’s integrity.
8. Usability Testing:
Objective: This focuses on the user's ease of using the application,
navigability, and overall user experience. Usability testing aims to
ensure that the software is intuitive and user-friendly.
Non-Functional Testing
9. Compatibility Testing
Objective: To ensure that a software program or system is
compatible with other software programs or systems.
For example, in this, the tester checks that the software is
compatible with other software, operating systems, etc.
AAAAAA
BBBBBB
CCCC
DDDDD
input
White-Box Testing
• Also known as Structural Testing, Clear-Box Testing, Open-Box Testing,
Logic-Driven Testing, transparent testing, or glass box testing)
• It is a software testing method where the internal structure, design, and
code of the software are tested.
• Using White-Box testing methods, the software engineer can derive test
cases that.
Guarantee that all independent paths within a module have been exercised at
least once.
Exercise all logical decisions on their true and false sides.
Execute all loops at their boundaries and within their operational bounds.
Exercise internal data structures to ensure their validity.
Knowledge of the internal code: The tester must understand the code
logic, data flow, and control flow.
Choose a test set T such that by executing program P for each test
case in T, each basic statement of P is executed at least once
Statement Coverage
if (x < y) { B0 (x < y)
x = 5 * y; Y N
x = x + 3;
} x = 5 * y B1 B2 y = 5
else x = x + 3
y = 5;
x = x+y;
x = x+y B3
assignAbsolute(int x)
B0 {
if (x < 0)
(x < 0)
x := -x;
true z := x;
B1 false
}
x := -x Test set {x = 1} does not execute
this edge, hence, it does not give
branch coverage
B2
z := x Test set {x = 1, x=2}gives both
statement and branch coverage
bool isEqual(int x, int y)
{ int max(int x, int y)
if (x = y) {
z := true; if (x > y)
else return x;
z := false; else
return z; return y;
} }
Generate test cases which will ensure statement coverage as well as branch
coverage
Statement vs. Branch Coverage
assignAbsolute(int x)
{ Consider this program segment, the test set
if (x < 0) T = {x = 1} will give statement coverage,
x := -x; however not branch coverage
z := x;
}
B0
Control Flow Graph: (x < 0)
true false
B1 Test set {x = 1} does not
x := -x execute this edge, hence, it
does not give branch coverage
B2
z := x
Path Coverage
Select a test set T such that by executing program P for each test case
d in T, all paths leading from the initial to the final node of P’s
control flow graph are traversed
areTheyPositive(int x, int y)
{
if (x >= 0)
print(“x is positive”);
else
Draw the Control flow chart for the given
print(“x is negative”); code
if (y >= 0)
print(“y is positive”);
else
print(“y is negative”);
}
Path Coverage
areTheyPositive(int x, int y) B0
{ (x >= 0)
if (x >= 0)
true false
print(“x is positive”);
else B1 B2
print(“x is negative”); print(“x is p”) print(“x is n”)
if (y >= 0)
print(“y is positive”); B3
else (y >= 0)
print(“y is negative”); true false
} B4 B5
Test set: print(“y is p”) print(“y is n”)
T2 = {(x = 12,y = 5), (x = 1,y = 35)}
gives both branch and statement B6
coverage but it does not give path coverage
return
Set of all execution paths: {(B0,B1,B3,B4,B6), (B0,B1,B3,B5,B6), (B0,B2,B3,B4,B6),
(B0,B2,B3,B5,B6)}
Test set T2 executes only paths: (B0,B1,B3,B5,B6) and (B0,B2,B3,B4,B6)
Path Coverage
B0
areTheyPositive(int x, int y)
(x >= 0)
{
if (x >= 0) true false
print(“x is positive”); B1 B2
else print(“x is p”) print(“x is n”)
print(“x is negative”);
if (y >= 0) B3
print(“y is positive”);
else (y >= 0)
print(“y is negative”); true false
} B4 B5
print(“y is p”) print(“y is n”)
Test set:
T1 = {(x=12,y=5), (x=1,y=35),
B6
(x=115,y=13),(x=91,y=2)}
return
gives both branch, statement and path
coverage
Condition Coverage
For complete Loop Coverage, you typically need to consider the following
scenarios:
A = 10
IF B > C THEN No. of Edges = E = 7
A=B No. of Nodes = N = 7
Cyclomatic Complexity = E – N + 2
ELSE =7 – 7 + 2 = 2
A=C
ENDIF
Print A
Print B
Print C
int module1 (int x, int y) { Start
while (x!=y){
if(x>y) F
x=x-y, x != y Return x
else y=y-x;
T
}
return x; T
x>y x=x-y
}
Stop
F
acceptance
requirements test
system
specification test
detailed integration
design test
implementation unit
code test
Unit Testing
• Unit Testing is a type of software testing where individual
components or units of the software are tested in isolation.
• The purpose of unit testing is to validate that each unit of the
software performs as expected.
• A "unit" is typically the smallest part of an application that can
be tested independently, such as a function, method, or class.
Unit Testing
• Involves testing a single isolated module
• Note that unit testing allows us to isolate the errors to a single module
– we know that if we find an error during unit testing it is in the module we are
testing
• Modules in a program are not isolated, they interact with each other. Possible
interactions:
– Calling procedures in other modules
– Receiving procedure calls from other modules
– Sharing variables
• For unit testing we need to isolate the module we want to test, we do this using two
things
– drivers and stubs
Drivers and Stubs
Module
Driver Stub
procedure Under Test procedure
call call
access to global
variables
• Driver and Stub should have the same interface as the modules they replace
• Driver and Stub should be simpler than the modules they replace
Drivers and Stubs
• Driver: A program that calls the interface procedures of the module being
tested and reports the results
• Stub: A program that has the same interface as a module that is being
used by the module being tested, but is simpler.
• The modules that may work properly and independently may not
work when they are integrated.
Bottom-Up Integration Testing: Starts testing from the lower-level modules and moves
upward. Uses drivers to simulate higher-level modules.
Beta Testing –
• The Beta test is conducted in the end-user’s environment.
• The developer is not present for the beta testing. The beta
testing is always in the real-world environment which is not
controlled by the developer.
• The end-user records the problems and reports it back to
the developer at intervals.
• Based on the results of the beta testing the software is made
ready for the final release to the intended customer base.
Test Plan
1. Introduction
• Objective: Purpose of the testing effort.
• Scope: Features or functionalities to be tested.
• Out of Scope: What will not be tested.
2. Test Items
• Specific items or components (modules, features, interfaces) that will be
tested.
3. Test Objectives
• Goals to achieve through testing, e.g., identifying defects, verifying
functionality, ensuring performance.
Key Components of a Test Plan
4. Testing Approach
• The testing methodology (e.g., manual or automated).
5. Entry and Exit Criteria
• Entry Criteria: Conditions that must be met before testing begins (e.g., code complete,
environment ready).
• Exit Criteria: Conditions to conclude testing (e.g., no critical defects, test cases
executed successfully).
6. Test Environment
• Details of hardware, software, network configurations, and tools required for testing.
7. Test Deliverables
• Outputs like test cases, test scripts, defect reports, and test summary reports.
Key Components of a Test Plan
8. Resource Allocation
• Test team members, roles, and responsibilities.
9. Schedule
• Timeline for testing activities, including milestones.
10. Risk Management
• Potential risks and their mitigation plans (e.g., lack of resources, delays in
environment setup).
11. Defect Management
• Process for reporting, tracking, and resolving defects.
12. Approval and Sign-off
• Criteria and stakeholders involved in approving the test plan.
Test Plan
• Test Data
– Data inputs required for the test case (e.g., username: user1, password: password123).
• Expected Result
– The outcome that should occur if the system functions correctly (e.g., "User is redirected to the
dashboard")
• Actual Result
• The observed outcome after executing the test case (e.g., "User is redirected to the dashboard")
• Pass/Fail Status
• Indicates whether the test case passed or failed based on a comparison of expected and actual
results.
• Priority
– The importance of the test case (e.g., High, Medium, Low).
1. Attachments
– Screenshots, logs, or any other evidence related to the test.
Example Testcase
Test Case ID TC001
Title Verify login with valid credentials
Objective Ensure successful login for valid users
Preconditions User account exists; Login page is accessible
Test Steps 1. Open the login page
2. Enter valid username and password
3. Click "Login" button
Test Data Username: test_user, Password: 12345
Expected Result User is redirected to the dashboard
Actual Result (To be filled after test execution)
Status Pass / Fail
Priority High
Environment OS: Windows 10, Browser: Chrome v114
Example Testcase
Test Test Input Expected Output Actual Output Pass/
Case ID Description Fail
TC01 Valid login Username: user1 Login successful;
credentials Password: Password123 redirects to
homepage
TC02 Invalid Username: invalidUser Error message:
username Password: password123 "Invalid credentials"
TC03 Invalid Username: user1 Error message:
password Password: wrongPass "Invalid credentials"
TC04 Empty Username: Error message:
username Password: "Fields cannot be
and empty"
password
TC05 Password Username: user1 Error message:
field case Password: password123 "Invalid credentials"
sensitivity
Test Suite