Software Testing

CONTENT
• Introduction
• What is Software Testing?
• Why do need to test software?
• What are the benefits of Software Testing?
• Terminologies
• Limitation of Testing
• Software Life –Cycle Model
• Types of Software Testing.
 INTRODUCTION
• We cannot imagine a day without using cell phones, logging on to the
internet, sending e-mails, watching television and so on.
• All these activities are dependent on software, and software is not reliable.
The world has seen many software failures and some were even fatal to
human life
• Software organizations are regularly getting failure reports of their products
and this number is increasing day by day. All of us would appreciate that it
is extremely disappointing for the developers to receive such failure reports.
The developers normally ask: how did these bugs escape unnoticed?
• It is a fact that software developers are experiencing failures in testing their
coded programs and such situations are becoming threats to modern
automated civilization.
 What is Software Testing?
• According to ANSI/IEEE 1059 standard – A process of analyzing a
software item to detect the differences between existing and required
conditions (i.e., defects) and to evaluate the features of the software
item.
• According to MYER – “Testing is the process of executing a program
with the intent of finding faults”.
• Software testing is a process, to evaluate the functionality of a
software application with an intent to find whether the developed
software met the specified requirements or not and to identify the
defects to ensure that the product is defect-free in order to produce a
quality product.
Example..
 Continued
We again consider the program ‘Minimum’ and concentrate on some typical and critical
situations as discussed below:
(i) A very short list (of inputs) with the size of 1, 2, or 3 elements.
(ii) An empty list i.e. of size 0.
(iii) A list where the minimum element is the first or last element.
(iv) A list where the minimum element is negative.
(v) A list where all elements are negative.
(vi) A list where some elements are real numbers.
(vii) A list where some elements are alphabetic characters.
(viii) A list with duplicate elements.
(ix) A list where one element has a value greater than the maximum permissible limit of an
integer.
Continued
 Continued
(i) The program has ignored the first and last values of the list

The program is not handling the first and last values of the list properly. If we see the
line numbers 22 and 23 of the program, we will identify the causes. There are two
faults. Line number 22 “i = 1;” should be changed to “i = 0;” in order to handle the
first
value of the list. Line number 23 “while (i<Number -1)” should be changed to “while
(i<=Number-1)” in order to handle the last value of the list.
(ii) The program proceeds without checking the size of the array
If we see line numbers 14 and 15 of the program, we will come to know that the
program
is not checking the size of the array / list before searching for the minimum value. A
list
cannot be of zero or negative size. If the user enters a negative or zero value of size or
value
greater than the size of the array, an appropriate message should be displayed.
(iii) Program has converted negative values to positive values
Line number 19 is converting all negative values to positive values. That is why the
program is not able to handle negative values. We should delete this line to remove
this fault.
Continued
Why do need to test software?

• As per the current trend, due to constant change and development in digitization, our lives are
improving in all areas. The way we work is also changed. For instance, we access our bank online,
we do shopping online, we order food online, and many more. We rely on software and systems.
What if these systems turn out to be defective. We all know that one small bug shows a huge
impact on business in terms of financial loss and goodwill.
• Testing is important because software bugs could be expensive or even dangerous.
• Software bugs can potentially cause monetary and human loss, and history is full of such
examples.
1. In April 2015, Bloomberg terminal in London crashed due to software glitch affected more than
300,000 traders on financial markets. It forced the government to postpone a 3bn pound debt
sale.
2. Nissan cars recalled over 1 million cars from the market due to software failure in the airbag
sensory detectors. There has been reported two accident due to this software failure.
 Continued
• Starbucks was forced to close about 60 percent of stores in the U.S and Canada due to software failure in its
POS system. At one point, the store served coffee for free as they were unable to process the transaction.
• Some of Amazon's third-party retailers saw their product price is reduced to 1p due to a software glitch. They
were left with heavy losses.
• Vulnerability in Windows 10. This bug enables users to escape from security sandboxes through a flaw in the
win32k system.
• In 2015 fighter plane F-35 fell victim to a software bug, making it unable to detect targets correctly.
• China Airlines Airbus A300 crashed due to a software bug on April 26, 1994, killing 264 innocents live
• In 1985, Canada's Therac-25 radiation therapy machine malfunctioned due to software bug and delivered
lethal radiation doses to patients, leaving 3 people dead and critically injuring 3 others.
• In April of 1999, a software bug caused the failure of a $1.2 billion military satellite launch, the costliest
accident in history
• In May of 1996, a software bug caused the bank accounts of 823 customers of a major U.S. bank to be
credited with 920 million US dollars.
Why do need to test software? Continued.
• Software testing is a very expensive and critical activity; but releasing the software without testing is
definitely more expensive and dangerous.
• No one would like to do it. It is like running a car without brakes. Hence testing is essential; but how much
testing is required? Do we have methods to measure it? Do we have techniques to quantify it? The answer is
not easy. All projects are different in nature and functionalities and a single yardstick may not be helpful in all
situations. It is a unique area with altogether different problems.
Why do need to test software? Continued
• What Should We Test? - Is it possible to test the program for all possible valid and invalid
inputs? The answer is always negative due to a large number of inputs.
• We consider a simple example where a program has two 8 bit integers as inputs. Total combinations of inputs
are 28 * 28.
• “How to choose a reasonable number of test cases out of a large pool of test cases?”
• What is the bottom line for testing? At least, we may wish to touch this bottom line, which may incorporate
the following:
(i) Execute every statement of the program at least once.
(ii) Execute all possible paths of the program at least once.
(iii) Execute every exit of the branch statement at least once.
Example
• 1. if (x > 0)
• 2. {
• 3. a = a + b;
• 4. }
• 5. if (y>10)
• 6. {
• 7. c=c+d;
• 8. }
The possible paths are: ACE, ABCE, ACDE and ABCDE. However, if we choose x = 9 and y = 15, all
statements are covered. Hence only one test case is sufficient for 100% statement coverage by traversing only
one path ABCDE. Therefore, 100% statement coverage may not be sufficient, even though that may be difficult
to achieve in real life programs.
 Objectives of testing
• To find any defects or bugs that may have been created when the software was
being developed
• To increase confidence in the quality of the software
• To prevent defects in the final product
• To ensure the end product meets customer requirements as well as the company
specifications
• To provide customers with a quality product and increase their confidence in the
company
 TERMINOLOGIES
• Program and Software :- The software is the superset of the program(s). It consists of one or many program(s), documentation
manuals and operating procedure manuals. These components are shown in Figure.

• The program is a combination of source code and object code.

TERMINOLOGIES( Continued)
• Verification and Validation :-
Verification: As per IEEE [IEEE01], “It is the process of evaluating the system or component to determine
whether the products of a given development phase satisfy the conditions imposed at the start of that phase.”
Validation: As per IEEE [IEEE01], “It is the process of evaluating a system or component during or at the end
of development process to determine whether it satisfies the specified requirements.”

Testing = Verification + Validation

Both are essential and complementary activities of software testing. If effective verification is carried out, it may
minimize the need of validation and more number of errors may be detected in the early phases of the software
development. Unfortunately, testing is primarily validation oriented.
Verification Vs. Validation
• Verification process includes checking of documents, design, code and program whereas Validation process
includes testing and validation of the actual product.
• Verification does not involve code execution while Validation involves code execution.
• Verification uses methods like reviews, walkthroughs, inspections and desk-checking whereas Validation uses
methods like black box testing, white box testing and non-functional testing.
• Verification checks whether the software confirms a specification whereas Validation checks whether the
software meets the requirements and expectations.
• Verification finds the bugs early in the development cycle whereas Validation finds the bugs that verification
can not catch.
• Verification process targets on software architecture, design, database, etc. while Validation process targets
the actual software product.
• Verification is done by the QA team while Validation is done by the involvement of testing team with QA
team.
• Verification process comes before validation whereas Validation process comes after verification.
TERMINOLOGIES( Continued)
• Fault, Error, Bug and Failure :-
Bug-When we make an error during coding, we call this a ‘bug’. Hence, error /mistake / defect in coding is
called a bug.
Fault-A fault is the representation of an error where representation is the mode of expression such as data flow
diagrams, ER diagrams, source code, use cases, etc. If fault is in the source code, we call it a bug.
Failure-A failure is the result of execution of a fault and is dynamic in nature. When the expected output does
not match with the observed output, we experience a failure.
• Test, Test Case and Test Suite:- Test and test case terms are synonyms and may be used interchangeably. A
test case consists of inputs given to the program and its expected outputs.
Test Suite- The set of test cases is called a test suite. We may have a test suite of all test cases, test suite of all
successful test cases and test suite of all unsuccessful test cases. Any combination of test cases will generate a
test suite. All test suites should be preserved as we preserve source code and other documents. They are equally
valuable and useful for the purpose of maintenance of the software. Sometimes test suite of unsuccessful test
cases gives very important information because these are the test cases which have made the program fail in the
past.
TERMINOLOGIES( Continued)
Deliverables and Milestones:-
• Deliverables:- Deliverables are generated during various phases of the software development. Theexamples
are source code, Software Requirements and Specification document (SRS), Software Design Document
(SDD), Installation guide, user reference manual, etc.
• Milestones:-The milestones are the events that are used to ascertain the status of the project. For instance,
finalization of SRS is a milestone; completion of SDD is another milestone. The milestones are essential for
monitoring and planning the progress of the software development
Alpha, Beta and Acceptance Testing:-
• Acceptance Testing: This term is used when the software is developed for a specific customer. The customer
is involved during acceptance testing. He/she may design adhoc test cases or well-planned test cases and
execute them to see the correctness of the software.
• Alpha Testing:- The alpha tests are conducted at the developer’s site by the customer. These tests are
conducted in a controlled environment and may start when the formal testing process is near completion.
• Beta Testing:- The beta tests are conducted by potential customers at their sites. Unlike alpha testing, the
developer is not present here. It is carried out in an uncontrolled real life environment by many potential
customers. Customers are expected to report failures, if any, to the company. These failure reports are studied
by the developers and appropriate changes are made in the software.
TERMINOLOGIES( Continued)
• Quality and Reliability:-
• Reliability:- Software reliability is one of the important factors of software quality. Other factors are
understandability, completeness, portability, consistency, maintainability, usability, efficiency, etc. These
quality factors are known as non-functional requirements for a software system. Software reliability is defined
as “the probability of failure free operation for a specified time in a specified environment” [ANSI91].
Although software reliability is defined as a probabilistic function and comes with the notion of time, it is not
a direct function of time. The software does not wear out like hardware during the software development life
cycle. There is no aging concept in software and it will change only when we intentionally change or upgrade
the software. Software quality determines how well the software is designed (quality of design), and how well
the software conforms to that design (quality of conformance).
• Quality :-Some software practitioners also feel that quality and reliability is the same thing. If we are testing a
program till it is stable, reliable and dependable, we are assuring a high quality product. Unfortunately, that is
not necessarily true. Reliability is just one part of quality. To produce a good quality product, a software tester
must verify and validate throughout the software development process.
TERMINOLOGIES( Continued)
Quality Assurance and Quality Control :-
• Quality Assurance:- The purpose of QA activity is to enforce standards and techniques to improve the
development process and prevent the previous faults from ever occurring. A good QA activity enforces good
software engineering practices which help to produce good quality software. The QA group monitors and
guides throughout the software development life cycle. This is a defect prevention technique and concentrates
on the process of the software development. Examples are reviews, audits, etc.
• Quality Control :- Quality control attempts to build a software system and test it thoroughly. If failures are
experienced, it removes the cause of failures and ensures the correctness of removal. It concentrates on
specific products rather than processes as in the case of QA. This is a defect detection and correction activity
which is usually done after the completion of the software development. An example is software testing at
various levels.
TERMINOLOGIES( Continued)
• Static Testing :- Static testing refers to testing activities without executing the source code. All verification
activities like inspections, walkthroughs, reviews, etc. come under this category of testing.
• Dynamic Testing:- This, if started in the early phases of the software development, gives good results at a
very reasonable cost. Dynamic testing refers to executing the source code and seeing how it performs with
specific inputs. All validation activities come in this category where execution of the program is essential.
• Debugging :- The purpose of testing is to find faults and find them as early as possible. When we find any
such fault, the process used to determine the cause of this fault and to remove it is known as. These are related
activities and are carried out sequentially.
 LIMITATIONS OF TESTING
• We want to test everything before giving the software to the customers. This ‘everything’ is very illusive and
has many meanings. What do we understand when we say ‘everything’? We may expect one, two or all of the
following when we refer to ‘everything’:
(i) Execute every statement of the program
(ii) Execute every true and false condition
(iii) Execute every condition of a decision node
(iv) Execute every possible path
(v) Execute the program with all valid inputs
(vi) Execute the program with all invalid inputs
‘Everything’ is impossible and we have to settle for ‘less than everything’ in real life situations.
• Errors in the Software Requirement and Specification Document
• Logical Bugs
• Difficult to Measure the Progress of Testing
THE V SHAPED SOFTWARE LIFE CYCLE MODEL
• The V shaped model is the modified form of the waterfall model with a special focus on testing activities. The
waterfall model allows us to start testing activities after the completion of the implementation phase. This was
popular when testing was primarily validation oriented. Now, there is a shift in testing activities from
validation to verification where we want to review / inspect every activity of the software development life
cycle. We want to involve the testing persons from the requirement analysis and specification phase itself.
They will review the SRS document and identify the weak areas, critical areas, ambiguous areas and
misrepresented areas. This will improve the quality of the SRS document and may further minimize the
errors.
• Relationship of Development and Testing Parts :- After the completion of implementation, we will have
the required test cases for every phase of testing. The only remaining work is to execute these test cases and
observe the responses of the outcome of the execution. This model brings the quality into the development of
our products. The encouragement of writing test cases and test plans in the earlier phases of the software
development life cycle is the real strength of this model. We require more resources to implement this model
as compared to the waterfall model. This model also suffers from many disadvantages of the waterfall model
like nonavailability of a working version of the product until late in the life cycle, difficulty in
accommodating any change, etc. This model has also limited applications in today’s interactive software
processes.
THE V SHAPED SOFTWARE LIFE CYCLE MODEL
• Graphical Representation :-
Types of Software Testing