1.introduction To Test Case Design
1.introduction To Test Case Design
INTRODUCTION TO TEST
CASE DESIGN
WHAT IS SOFTWARE?
• Software is a set of instructions & data or programs used to operate computers and
execute specific tasks.
SOFTWARE TESTING
• Software testing is the process of evaluating and verifying that a software product
or application does what it is supposed to do.
• Verification is the process of checking that a software achieves its goal without any bugs. It is
the process to ensure whether the product that is developed is right or not. It verifies whether
the developed product.Verification is static testing.
Verification means Are we building the product right?
• Validation is the process of checking whether the software product is up to the mark . It is the
process of checking the validation of product i.e. it checks what we are developing is the right
product. It is validation of actual and expected product.Validation is the dynamic testing.
Validation means Are we building the right product?
BUG
• Definition
“A software bug is an error, flaw, failure or fault in a computer program or system that causes
it to produce an incorrect or unexpected result in operation or to behave in unintended ways”
• Classification of bug
1. Functional bugs :- Functional bugs are related to the functionality of a piece of software,
e.g. a button doesn't submit a form, the search doesn't react to the user input etc. Every time
you perform an action and the website/app doesn't respond as you expect, it might be a
functional issue.
• 2. Logical bug :- A logical bug disrupts the intended workflow of software and causes it
to behave incorrectly.These bugs can result in unexpected software behavior and even
sudden crashes. Logical bugs primarily take place due to poorly written code
or misinterpretation of business logic. Example of logical bugs include:
• Assigning a value to the wrong variable
• Dividing two numbers instead of adding them together resulting in unexpected output
3. Workflow Bugs
Workflow bugs are associated with the user journey (navigation) of a software application. Let’s
consider an example of a website where a user needs to fill up a form regarding their medical
history. After filling the form, the user has three options to choose from:
• Save
• Save and Exit
• Previous Page
From the available options, if the user clicks on “Save and Exit,” the user intends to save the
entered information and then exit. However, if clicking on the Save and Exit button leads to an
exit from the form without saving the information, it leads to a workflow bug.
4. Unit Level Bugs
• Once the initial modules of software are developed, developers perform unit testing to
ensure that the small batches of code are functioning as expected. Here’s where
developers encounter various bugs called unit bugs.
• For example, if a developer creates a single page form, a unit test will verify whether all
the input fields are accepting appropriate inputs and validate buttons for functionality. In
case a field doesn’t accept the appropriate characters or numbers, developers encounter
a unit-level bug.
5. System-Level Integration Bugs
• System-level integration bugs primarily pop up when two or more units of code written
by different developers fail to interact with each other.These bugs primarily occur due to
inconsistencies or incompatibility between two or more components. Such bugs are
difficult to track and fix as developers need to examine a larger chunk of code.They are
also time-consuming to replicate.
• Inappropriate interfacing between the application UI and the database is the common
example of system-level integration bugs.
6. Out of Bound Bugs
• These bugs occur when an end-user enters a value or a parameter outside the limits of
intended use — for example, entering a significantly larger or a smaller number or
entering an input value of an undefined data type.
IDENTIFICATION OF ERRORS AND BUGS IN THE
GIVEN APPLICATION
1.Quick Attacks
In the absence of formal requirements quality checking is performed using attacks.
For example, leave a couple of required fields blank, disrupt UI workflows, enter numbers
when users are supposed to enter words, exceed character limits, use disallowed
characters, enter too many incorrect passwords etc.
2. Set Goals for Software Quality
When a tester knows the standards to which software needs to be held, they will have a
better idea of what bugs to look for.
3.Test on Real Devices
• The tester’s primary job is to execute tests faster without compromising on the accuracy of results. That
is impossible to achieve without using real browsers and devices for testing.
4. Pay attention to the test environment
• Evaluation/ testing of test environment should also be done because flaws in the environment will cause
unnecessary and entirely avoidable delays in generating test results. It can also lead to the emergence of
bugs that aren’t actually caused by the software itself.
5.Use the Pareto Principle
It states that 80% of all bugs can be found in 20% of program modules.That means bugs are not distributed
equally throughout the software. In fact, they tend to cluster in certain sections of the code, especially the
serious defects.
RELATION BETWEEN ERROR, BUG AND FAILURE
• Bug is defined as, a mistake which is caused by a software developer while writing a software
code that causes it to produce an incorrect or unexpected result.
• A bug is the alternative name or informal name of defect which means that software or
application is not working as per the requirements.
• Error is the measure of deviation of the output given by a software developer while writing
software code.
• Fault is defined as the condition that leads to malfunctioning of software (caused due
to several reasons such as change in design, architecture or software code.
• The defect that causes an error in operation or a negative impact is called failure.
BUG REASONS
• 1. Wrong Coding
It means improper implementation.
• 2. Missing Coding
It is when the developer won't have developed the code only for that specific part.
• 3. Extra Coding
It means that the developer add some extra features in the system which is not needed as
per the requirements given by client. If we develop an additional functionality in the software
which is not needed in the requirement, it leads to unnecessary added efforts and it might also
happen that adding up that additional functionality will affect the other part of the software.
TEST CASE
• JMeter
• LoadRunner
• Selenium
• Appium
• TestProject
• Katalon
• CloudTest
• TestComplete AppliTools
TEST CASE DESIGN TECHNIQUES
1. Specification-based( Black-box)
▪ It uses the specification of the program as the point of reference for test data selection and adequacy. A
specification can be anything like a written document, collection of use cases, a set of models or a prototype.
▪ So black-box techniques are based on an analysis of the test basis documentation, including both functional and
non-functional aspects.
▪ They do not use any information regarding the internal structure of the component or system under test.
▪ Specification-based test case design techniques can be used to design test cases in a systematic format.
▪ These use external features of the software such as technical specifications, design, client's requirements and
more, to derive test cases.
▪ With this type of test case design techniques, testers can develop test cases that save testing time and allow full
test coverage.
SPECIFICATION BASED(BLACK BOX
TESTING )TECHNIQUES
• 1.Equivalence Partitioning
• 2.Boundary Value Analysis
• 3. Decision Table Testing
• 4. State Transition Testing
• 5. Use Case Technique
BLACK BOX TESTING TECHNIQUES
1.Equivalence Partitioning
• Testers can divide possible inputs into groups or “partitions”, and test only one example
input from each group.
• For example, if a system requires a user’s birth date and provides the same response for
all users under the age of 18, and a different response for users over 18, it is sufficient for
testers to check one birth date in the “under 18” group and one date in the “over 18”
group.
2.Boundary Value Analysis
• Testers can identify that a system has a special response around a specific boundary
value.
• For example, a specific field may accept only values between 0 and 99.Testers can focus
on the boundary values (-1, 0, 99 and 100), to see if the system is accepting and rejecting
inputs correctly.
3. Decision Table Testing
• Many systems provide outputs based on a set of conditions.Testers can then identify
“rules” which are a combination of conditions, identify the outcome of each rule, and
design a test case for each rule.
• For example, a health insurance company may provide different premium based on the
age of the insured person (under 40 or over 40) and whether they are a smoker or not.
This generates a decision table with four rules and up to four outcomes—below is an
example with three possible outcomes.
Condition Rule 1 Rule 2 Rule 3 Rule 4
s
OUTCOMES
1: High
premium
2: Medium
premium
3: Low
premium
4. State Transition Testing
• In some systems, significant responses are generated when the system transits from one state
to another. A common example is a login mechanism which allows users to authenticate, but
after a specific number of login attempts, transition to a different state, locking the account.
• If testers identify a state transition mechanism, they can design test cases that probe the
system when it transits states.
• For example, for a system that locks the account after five failed login attempts, a test case
can check what happens at the sixth login attempt.
5. Use Case Technique
• The use case is functional testing of the black box testing used to identify the test cases from the beginning to
the end of the system as per the usage of the system. By using this technique, the test team creates a test
scenario that can exercise the entire software based on the functionality of each function from start to end.
• It is a graphic demonstration of business needs, which describe how the end-user will cooperate with the
software or the application. The use cases provide us all the possible techniques of how the end-user uses the
application .
2. Structure-based (White-Box)
• Structural testing, also known as glass box testing or white box testing is an approach where
the tests are derived from the knowledge of the software's structure or internal
implementation.
• Structure based design techniques are based on the internal structure of the
software program and code. Developers go into minute details of the developed code and
test them one by one.
• 3. Experience-based
• Experience-based techniques are based on the tester's experience with testing, development,
similar applications, the same application in previous releases, and the domain itself
WHITE_BOXTESTING TECHNIQUES
• Often, a CASE (Computer Aided Software Engineering) tool keeps track of how many times
each statement has been executed.
1. input(int x, int y) {
2. sum=x+y;
3. if(sum>0)
4. printf(“this is positive result”)
5. else
6. printf(“this is negative result”)
7. }
TEST CASE 1.
• If x=6, y=2
• 1. input(int x, int y)
• 2. {
• 3. sum=x+y;
• 4. if(sum>0)
• 5. printf(“this is positive result”)
• 6. else
• 7. printf(“this is negative result”)
• 8. }
• Total no. of statements= 8.
• Number of executed statements= 6.
• = 6/8 x100
• =600/8
• = 75%
TEST CASE 2
• If x= -4 y= -3
• 1. input(int x, int y)
• 2.{
• 3. sum=x+y;
• 4. if(sum>0)
• 5. printf(“this is positive result”)
• 6. else
• 7. printf(“this is negative result”)
• 8. }
• Total no. of statements= 8.
• Number of executed statements= 7.
• = 7/8 x100
• = 700/8
• = 87.5%
EXAMPLE 2
• If x= 20, y=30
• 1. input ( int x, int y)
• 2.{
• 3. int z= (( x+ y) / 200) * 100;
• 4. if (z >50)
• 5. printf ( “pass”);
• 6. else
• 7. printf (“ fail”);
• 8. }
• Total no. of statements= 7.
• Number of executed statements= 6.
• = 7/8 x100
• = 700/8
• = 87.5%
TEST CASE 2
• If x=100, y=75
• 1. input ( int x, int y)
• 2.{
• 3. int z= (( x+ y) / 200) * 100;
• 4. if (z >50)
• 5. printf ( “pass”);
• 6. else
• 7. printf (“ fail”);
• 8. }
• Total no. of statements= 8.
• Number of executed statements= 6.
• = 6/8 x100
• = 600/8
• = 75%
EXAMPLE 3
• Decision coverage testing differs from branch coverage only when multiple conditions must
be evaluated to reach a decision.
• Branch coverage is also sometimes referred to as all edge coverage.
• Branch coverage requires sufficient test cases for each program decision or branch to be
executed so that each possible outcome occurs at least once.
• Like decision coverage, it also uses a control flow graph to calculate the number of branches.
• There are several methods to calculate Branch coverage, but pathfinding is the most common
meth.
• Sometimes, branch coverage technique can be used as the alternative of decision coverage as
both are closely related.
• Consider the a simple program to find the maximum of two numbers:
• int a = 2;
• int b = 5;
• if(a>b){
• cout<<"a is max";
• }else{
• cout<<"b is max";
• }
Branches
• The branches in the control flow graph above are A, B, C, D, E, F, and G.
Branch Coverage
• For maximum branch coverage, we take the path:
• Path1 = 1A-2B-3C-4D-5F-7
• Covered branches = A, B, C, D, F
• Uncovered branches = E, G
• The branch coverage of path1 will be:
• Branch coverage = (number of branches covered / total number of branches) x 100
• Branch coverage = (5 / 7) x 100
• Branch coverage = 71%
• Branches E and G are not covered in path1.To cover branches E and G, we take the path:
• Path2 = 1A-2B-3C-4E-6G-7
• Covered branches = A, B, C, E, G
• Uncovered branches = D, F
• The branch coverage of path2 will be:
• Branch coverage = (number of branches covered / total number of branches) x 100
• Branch coverage = (5 / 7) x 100
• Branch coverage = 71%
• For 100% branch coverage, we combine both paths, path1 and path2. The final branch testing of the
control flow graph above becomes:
• Path1 = 1A-2B-3C-4D-5F-7
• Path2 = 1A-2B-3C-4E-6G-7
• Example 1:
• 1. Read A
• 2. Read B
• 3 IF A+B>100 THEN
• 4. Print "Larger than 100
• 5. ENDIF
• 6 IF A+B>50 THEN
• 7 Print "Larger than 50
• 8 ENDIF
• The formula for finding branch coverage percentage is given below:
• Branch coverage =Number of executed branches X100
Total number of branches
• Branch Coverage will consider unconditional branch as well.
• Branch coverage for result Yes:
• Branch Coverage=5/8 X100
• =62.5%
• Branch coverage for result No:
• Branch Coverage=3/8 X100
• = 37.5%
EXAMPLE 2:
• 1. read a;
• 2. if (a >5)
• 3. a=a*3;
• 4. end if
• 5. print a;
• Branch coverage for result Yes:
• Branch Coverage=2/3 X100
• =67%
• Branch coverage for result No:
• Branch Coverage=1/3 X100
• = 33%
ADVANTAGES OF BRANCH COVERAGE
• 1. It allows to validate-all the branches in the code.
• 2. It helps to ensure that no branch leads to any abnormality of the program's operation.
• 3. It allows finding those areas which are not tested by other testing methods.
• 4. It allows finding a quantitative measure of code coverage.
• Disadvantages of Branch coverage:
• 1. It is costly.
• 2. It takes more time for performing this task.
LOOP COVERAGE TESTING
• Loop coverage testing criteria requires sufficient test cases for all program loops to be
executed for zero, one, two, and many iteration covering initialization, typical running and
termination (boundary) conditions.
• Loop testing is a type of white box testing technique to validate the loops. It is one of the
type of control structure testing.
• Loop testing is a test coverage criteria which checks whether loop body executed zero
times, exactly once or more than once.
The goals of loop coverage testing are given below:
• 1.To fix the infinite loop repetition problem.
• 2.To know the performance.
• 3.To identify the loop initialization problems.
• 4.To determine the uninitialized variables.
TYPES OF LOOP TESTING
}
while(condition 2)
{
(// statement(s));
}
4. Unstructured Loop Testing
• Testing applied on an unstructured loop is known as Unstructured Loop Testing.
• It is the combination of concatenated and nested loops.
• In other words, it is group of loops that are in no particular order.
• Example:
While
{
for()
{
}
while()
{
}
}
• Example 1: In order to have loop coverage, testers should exercise the tests given follows:
• Test 1: Design a test in which loop body should not execute at all (i.e.the zero iteration)
• Test 2: Design a test in which loop control variable should be negative .
• Test 3: Design a test in which loop iterates only once.
• Test 4: Design a test in which loop iterates twice.
• Test 5: Design a test in which loop iterates certain number of times, say n where:n<maximum
number of iterations possible.
• Test 6: Design a test in which loop iterates one less than the maximum number of iterations.
• Test 7: Design a test in which loop iterates the maximum number of iterations.
• Test 8: Design a test in which loop iterates one more than the maximum number
of iterations
• Example 2
• #include<stdio.h>
int main()
{
int i=1,n;
printf("Enter n:");
scanf("%d", &n);
while(i<n)
{
printf("%d\n",i);
i++;
}
return 0;
}
SOME TEST CASES WITH TEST DATA
• The path coverage-based testing strategy requires us to design test cases such that all
linearly independent paths in the program are executed at least once.
• A linearly independent path can be defined in terms of the Control Flow Graph (CFG of
a program).A CFG describes how the control flows through the program.
• In order to draw the control flow graph of a program, we need to first number all the
statements of a program.The different numbered statements serve as the nodes of the
control flow graph.
• An edge from one node to another node exists if the execution of the
statement representing the first node can result in the transfer of control to the other
node.
CFG FOR SEQUENCE, SELECTION AND
ITERATION
• Path
• A path through a program is a node and edge sequence from the starting to a terminal
node of the control flow graph of a program.
• writing test cases to cover all the paths of a typical program is impractical .
• For this reason, the path-coverage testing does not require coverage of all paths but only
coverage of linearly independent paths .
• Linearly Independent Path:
• A linearly independent path is any path through the program that introduces at least one
new edge that is not included in any other linearly independent paths.
• If a path has one new node compared to all other linearly independent paths, then the
path is also linearly independent.
• This is because any path having a new node automatically implies that it has a new edge.
• McCabe's Cyclomatic Complexity:
• Cyclomatic Complexity in Software Testing is a testing metric used for measuring the
complexity of a software program. It is a quantitative measure of independent path in the
source code of a software program.
• For more complicated programs it is not easy to determine the number of independent
paths of the program.
• McCabe's cyclomatic complexity defines an upper bound for the number of linearly
independent paths through a program.
• This metric was developed by Thomas J. McCabe in 1976 and it is based on a control
flow representation of the program. Control flow depicts a program as a graph which
consists of Nodes and Edges.
• There are three different ways to compute the Cyclomatic complexity.
• Method 1: Given a control flow graph G of a program, the Cyclomatic complexity V(G)
can be computed as,
V(G)= E-N+2
where N is the number of nodes of the control flow graph and E is the number of edges in
the control flow graph.
Method 2: An alternative way of computing the Cyclomatic complexity of a program from
an inspection of its control flow graph is as follows:
V(G) = Total number of bounded areas + 1.
In the program's control flow graph G, any region enclosed by nodes and edges can be
called as a bounded area.
EXAMPLE 1
• Read A
Read B
IF A+B > 50 THEN
Print "Large"
ENDIF
If A+B<50 THEN
Print "Small"
ENDIF
CFG for given example
• Cyclomatic complexity
• Using method 1 : V(G)= E-N+2
=8-7+2
=3
• Using method II : V(G) = Total number of bounded areas + 1
= 2+1
=3
Example 2
• Cyclomatic complexity
• Using method 1 : V(G)= E-N+2
• =8-7+2
• =3
• Using method II : V(G) = Total number of bounded areas + 1
• = 2+1
• =3
EXAMPLE 3
CFG for Example3
• Cyclomatic complexity
• Using method 1 : V(G)= E-N+2
• =7-6+2
• =3
• Using method II : V(G) = Total number of bounded areas + 1
• = 2+1
• =3
BUG TRACKING TOOLS
1.JIRA
• It is an open-source tool, which is used to help the customer, and the client to maintain
the track of the bugs.
• It is also used as a test management tool because, in this, we can easily link other test
case management tools .
• It supports various operating systems such as Windows, Linux, and Mac.
FEATURES OF THE BUGZILLA TOOL
• It is an open-source tool which is used to track the issues and as a web-based project
management tool.
• Redmine tool is written in Ruby programing language and also compatible with multiple
databases like MySQL, Microsoft SQL, and SQLite.
• While using the Redmine tool, users can also manage various projects and related
subprojects.
FEATURES OF REDMINE TOOL
• The backlog is widely used to manage the IT projects and track the bugs.
• It is mainly built for the development team for reporting the bugs with the complete
details of the issues, comments, updates and change of status.
• It is a project management software.
FEATURES OF BACKLOG TOOL
• It is very helpful in keeping track of all the tasks which need to be done and in which they should be
done.
• It helps in keeping the track of the time to manage the product releases. It is very helpful in avoiding the
last time glitches before the product release.
• It manages all the resources along with the names of the team members who are responsible for
completing which task.
• It helps in identifying the defects or issues or risks early in the development process so they can be
resolved quickly before time.
• It helps in avoiding bottlenecks throughout the project development process.
DESIGN ENTRY AND EXIT CRITERIA FOR TEST CASE
• Entry criterion is used to determine when a given test activity should start. It also
includes the beginning of a level of testing, when test design or when test execution is
ready to start.
• Examples for Entry Criterion:
• Verify if the Test environment is available and ready for use.
• Verify if test tools installed in the environment are ready for use.
• Verify if Testable code is available.
• Verify if Test Data is available and validated for correctness of Data.
EXIT CRITERIA
• Exit criteria address the issue of how to determine when testing has been completed.
The commonly considered exit criteria for terminating or concluding the process of testing are:
• Test case data can be managed in Excel.This requires an Excel spreadsheet which
arranges the test data in rows and columns.Test designers can use MS Excel to design
and build the tests.
• If we use MS Excel, it is best to keep everything in a single workbook and to include one
test conditions spreadsheet, one test data spreadsheet as many detailed test
spreadsheets are needed to adequately describe the environmental, pretest and post-
test activities that. are related to each test case.
STANDARD FORMAT OF TEST CASES TEMPLATE
• The industry standard, IEEE 829 template outline:
• Statuses visible in the Execution status column shows aggregate progress of Test Cases connected
with a given requirement.
• Displaying statuses in execution status column is based on the Test Case statuses.
• The Execution status is a status that indicates the overall state of tests of a requirement based on the
rule:
• Pass - requirement with all Test Cases on passed status.
• Fail - requirement with at least one Test Case on failed status
• Not Executed - requirement with Test Cases where all statuses are neither failed nor passed
• No Tests - requirement with no Test Cases at all
.
• At the bottom of the template is a summary section in which the tester indicates an
overall assessment of the test case.
• The tester should next note the specific system configuration used for the test.
• In the final part of the summary section, the tester should record his or her name
or initials , the date on which the test case was completed, the actual effort
expended in terms of person-hours, and the duration.
• The latter three pieces of information allow us to track progress.
• The common fields that are used in test case are explained below:
• 1. Test Case ID field is defined by what type of system we are testing. Each test case
should be represented by a unique ID.
• 2. Test Case Name field contains name of the feature we are testing. Requirement
number from the specifications and Requirement name as classified in client document.
• 3.Test Description field explains what type of feature we will test on which condition.
This description should detail what unit feature, or function is being tested or what is
being verified.
• 4. Steps To Execute field contains the steps to be executed on the system being tested
to get the expected results. Steps should be understandable and correct.They are
written and executed according to a sequence
• 5. Pre-conditions field must be fulfilled before the execution of the test case .Pre
conditions should be satisfied before the test case execution starts. List all the
preconditions in order to execute this test case successfully
• 6. Execution Steps field contains the steps to be performed on the system under test
to get the desired results. Steps must be defined clearly and must be accurate.They are
written and executed number wise
• 7. Expected Result field contains the desired outputs from the execution steps
performed. Results should be clearly defined for every step It specifies what the
specifications are and what we will get from a particular specification
• 8.Actual Result field has the real actual result after the performed execution steps on
the system under testing If the result matches with the expected result then we can
write as expected
• 9. Status field can state the test is Pass /Fail If the result is showing according to the
expected result, the test mark as pass and if not get the output according to the
expected, result mark as fail.We can use color to state the green color for Pass and red
color for Fail
• 10. Comment field column is for additional information for eg if status is set to "can not
be tested", then tester can give the reason in this column
Negative test cases
● Negative test cases are used to test the system for invalid inputs and unexpected
behaviors. They are designed to ensure that the system behaves as expected when given
invalid or unexpected inputs.
FEATURES OF A TESTING METHOD USED
• 1.High Probability of Detecting Errors:To detect maximum errors, the tester should
understand the software thoroughly and try to find the possible ways in which the
software can fail. For example, in a program to divide two numbers, the possible way in
which the program can fail is when 2 and 0 are given as inputs and 2 is to be divided by 0.
In this case, a set of tests should be developed that can demonstrate an error in the
division operator.
• 2. No Redundancy: Resources and testing time are limited in software development
process.Thus, it is not beneficial to develop several tests, which have the same intended
purpose. Every test should have a distinct purpose.
• 3. Choose the most Appropriate Test:There can be different tests that have the same
intent but due to certain limitations such as time and resource constraint, only few of
them are used. In such a case, the tests, which are likely to find more number of errors,
should be considered.
• 4. Moderate:A test is considered good if it is neither too simple, nor too complex. Many
tests can be combined to form one test case. However this can increase the complexity
and leave many errors undetected. Hence, all tests should be performed separately.