Software Testing – 1

- Introduction
Lecture # 23

1
 Recap
• This is the first lecture after mid term
examination
• I hope you all have done wonderfully well
on your mid term exam

2
 Today’s Lecture
• Starting today, we’ll be starting a series of
lectures on software testing and the role of
software testing in software quality
assurance and quality control
• So, let’s begin

3
 Basics of Software Testing - 1
• Synonyms: checking, assessing,
examination
• Testing is one of the most important parts
of quality assurance and the most
performed QA activity

4
 Basics of Software Testing - 2
• Testing has been a basic form of defect
removal since software industry began
• For majority of software projects, it is the
only form of defect removal utilized

5
 Basics of Software Testing - 3
• The basic idea of testing involves the execution
of software and the observation of its behavior
or outcome
• If a failure is observed, the execution record is
analyzed to locate and fix the faults that caused
the failure
• Otherwise, we gain some confidence that the
software under testing is more likely to fulfill its
designated functions
6
 Why Software Testing? - 1
• The purpose of software testing is to ensure
that the software systems would work as
expected when they are used by their target
customers and users
• For software products, software testing
provides a mechanism to demonstrate their
operation through controlled execution

7
Following slide to be
insertedSoftware Testing

8
 Software Testing
• “The dynamic execution of software and
the comparison of results of that execution
against a set of known, pre-determined
criteria”
– Capers Jones

9
• This demonstration of proper behavior is a
primary purpose of software testing, which
can also be interpreted as providing
evidence of quality in the context of
software quality assurance, or as meeting
certain quality goals

10
• In comparison with the manufacturing
systems, problems in software products can
be corrected much more easily within the
development process to remove as many
defects from the product as possible
• Therefore, software testing has two primary
purposes

11
 Primary Purposes of Software
Testing

• To demonstrate quality or proper behavior

• To detect and fix problems

12
What Testing Shows?
errors
requirements conformance

performance

an indication
of quality

13
 Major Activities and Generic
Testing Process
• Test planning and preparation

• Test execution and related activities

• Analysis and follow-up

14
 Test Planning and Preparation
• Setting of goals for testing
• Selecting an overall testing strategy
• Preparing specific test cases
• Preparing general test procedure

15
 Test Execution
• Execution of software tests
• Observation and measurement of product
behavior

16
 Analysis and Follow-Up
• Includes result checking and analysis to
determine if a failure has been observed,
and if so, follow-up activities are initiated
and monitored to ensure removal of the
underlying causes, of faults, that led to the
observed failures in the first place

17
Generic Testing Process

18
• Due to the increasing size and complexity
of today’s software products, informal
testing without much planning and
preparation becomes inadequate
• Important functions, features, and related
software components and implementation
details could be easily overlook in such
informal testing
19
• Therefore, there is a strong need for
planned, monitored, managed and
optimized testing strategies based on
systematic considerations for quality,
formal models, and related techniques
• Test planning and preparation has these
sub-activities

20
 Sub-Activities in Test Planning
and Preparation
• Goal setting
– Reliability and coverage goals
• Test case preparation
– Constructing new test cases or generating automatically,
selecting them from existing ones for legacy products, and
organizing them in some systematic manner
• Test procedure preparation
– It is defined and followed to ensure effective test execution,
problem handling and resolution, and overall test process
management
21
 Testing as a Part of QA in
Overall Software Process - 1
• Testing and inspections often find different
kinds of problems, and may be more
effective under different circumstances
• Therefore, inspections and testing should be
viewed more as complementary QA
alternatives instead of competing ones

22
 Testing as a Part of QA in
Overall Software Process - 2
• Testing is an integral part of different
software development processes

23
 Testing and Defect Removal
Efficiency
• Most forms of testing are less than 30%
efficient in finding bugs
• Many forms of testing should be used in
combination with pre-test design and code
inspections to achieve high defect removal
efficiencies

24
 Testing and Defect Types
• Testing is much more effective in finding
errors of commission, or things that are
done wrong, than it is in finding errors of
omission or things that are left out by
accident

25
Questions about Testing
Basic, testing techniques, testing
activities and management

26
 Basic Questions about Testing
• Our basic questions about testing are related
to the objects being tested, perspectives and
views used in testing, and overall
management and organization of test
activities

27
 Basic Questions about Testing
• What artifacts are tested?
• What to test, and what kind of faults is
found?
• When, or at what defect level, to stop
testing?
– Coverage information, product reliability goals,

28
 Questions about Testing
Techniques
• Many different testing techniques can be
applied to perform testing in different sub-
phases, for different types of products, and
under different environments
• Various questions regarding these testing
techniques can help us get a better
understanding of many related issues

29
 Questions about Testing
Techniques - 1
• What is the specific testing technique used?
• What is the underlying model used in a
specific testing technique?
– There are two basic models: 1) based on simple
structures such as checklists and partitions, and
2) based on finite-state machines

30
 Questions about Testing
Techniques - 2
• Are techniques for testing in other domains
applicable to software testing?
– This defect seeding, mutation, immunization
used in physical domains and comes under
specialized testing in software
• If multiple testing techniques are available,
can they be combined or integrated for
better effectiveness or efficiency?
31
 Questions about Test Activities
and Management
• Various other questions can also be used to
help us analyze and classify different test
activities
• Some key questions are about initiators and
participants of these activities, organization
and management of specific activities
• (Eight questions)

32
 Questions about Test Activities
and Management - 1
• Who performs specific activities?
• When can specific test activities be
performed?
• What process is followed for these test
activities?
• Is test automation possible? And if so what
kind of automated testing tools are available
and usable for specific applications?
33
 Questions about Test Activities
and Management - 2
• What artifacts are used to manage the testing
process and related activities?
• What is the relationship between testing and
various defect-related concepts?
• What is the general hardware /software /
organizational environment for testing?
• What is the product type or market segment
for product under testing?
34
• Software test planners and testers need to
answer those question which are relevant
for their project
• In the series of lectures on software testing,
as a mechanism for quality control and
quality assurance, we will also address them

35
 References
• Software Quality Engineering: Testing,
Quality Assurance, and Quantifiable
Improvement by Jeff Tian, IEEE Computer
Society, 2005 (Chapter 6.1-6.2)

36