5/12/2019

Software Testing and Quality

Assurance
Week – 8

Agenda
 Test Planning
 Ingredients of a test plan

1
 5/12/2019

Overview

 Test Planning
 Test Plan Document
 Test Case Life Cycle
 Test Case Design
 System Test Execution

Ref: This lecture is based on Prof. S. Naik’s notes.

3

Test Plan Objectives and Motivation

 Objective: To get ready and organized for test execution

 Test plan documents

– Provide guidance for the execution management to support the
test project and release the necessary resources
– Provide a foundation for the system testing part of the overall
project
– Provide assurance of test coverage by creating a requirement
traceability matrix
– Outline an orderly schedule of events and test milestones to be
achieved
– Specify the personnel, financial, and facility resources required to
support the system testing part.

2
 5/12/2019

Test Plan Document Structure

 The structure of a Test Plan document usually is:
1. Introduction
2. Features to be tested and features not to be tested
3. Assumptions
4. Testing Approach (methodology)
5. Test Suite Structure
6. Item pass/ fail criteria
7. Test Environment
8. Test Execution Strategy
9. Suspension criteria and resumption requirements
10. Test deliverables
11. Other Issues:
– Environmental needs, Responsibilities, Staffing and, Training
needs, Schedule, Risks and contingencies, Approvals
5

Test Plan Document Contents

 For each major set of features:
– specify the major activities,
– techniques,
– and tools which will be used to test them.
 The specification should be in sufficient detail to permit
estimation and scheduling of the associated tasks.
 Specify techniques which will be used to assess:
– the comprehensiveness of testing and
– any additional completion criteria.
 Specify the techniques used to trace requirements.
 Identify any relevant constraints.

3
 5/12/2019

Test Case Life Cycle

 Main idea: test cases as products. Therefore test cases

have a life cycle.

Deleted Deprecated

Create Draft Review Released

Update

 Create: The create phase enters the following information:

– Test case ID
– Requirements IDs
– Title
– Originator group
– Creator
– Test category

 Draft: A test engineer enters the following information:

– Author of a test case
– Test objective
– Environment
– Test steps
– Clean up
– Pass/Fail criteria
– Candidate for automation
– Automation priority

4
 5/12/2019

 Review: Here, the creator is the owner

– The owner invites test engineers and developers to
review the test case
– Ensure that the test case is executable and Pass/Fail
criteria are clearly stated
– Changes may occur
– Once the test case is approved, it is moved to
“Released” state.

 Released
– The test case is ready for execution
– Here, the owner is the test organization
– Review the test case for re-usability
– If there is a need to update the test case, move it to
“update” state.

 Update
– Strengthen the test case as the system functionality or
environment changes
– By executing the test case 2/3 times, one may update the test
case to improve reliability
– One gets an idea about its automation potential
– A major update calls for a review of the test case.

 Deleted: If the test case is not a valid one.

 Deprecated: If a test case is obsolete, move it to this state.

A test case becomes obsolete for several reasons:
– System functionality has changed, but test cases have not been
properly maintained
– Test cases have not been designed with re-usability in mind
– Test cases have been carried forward due to carelessness; long
after their original justification has disappeared.

10

5
 5/12/2019

Creation of a Test Suite

Test suite: this is a set of test cases organized in a hierarchal
manner for a certain software release. Two characteristics of
a test suite:

 Test cases in a test suite relate to a certain release of the

system. This is important because not all functionalities
are supported in all the different versions of a system.

 Test cases are organized in a hierarchal manner for three

reasons:
– To obtain a balanced test suite
– To prioritize their execution
– To monitor the progress of system testing.

11

Test Suite Organization

Test Suite

Basic Functionality Robustness Performance ……

User …………… Admin Maint.

Local Long Dist. Intl. Pre-paid Credit Card

Local …………………….

12

6
 5/12/2019

State Transition Diagram of a Test Case Result

Execution is successful and the

Passed pass criteria is verified.

Test case successfully executed;

Failed Pass criteria not achieved.
Report the bug.
Untested
A bug prevents the execution of
Blocked
the test case. Report the bug.

Test case not applicable for this

Invalid release.

Getting Ready for Test Execution

 The “entry criteria” for the first cycle must be satisfied.

 The final item of the entry cycle is the test execution

working document. An outline of such a document is as
follows:
– Test engineers (names, availability, expertise, training)
– Test case allocation to engineers (based on expertise and interest)
– Test bed allocation (availability, number, distribution)
– Progress of test automation (track it)
– Projected execution rate (on a weekly basis)
– A strategy for the execution of failed test cases
– Development of new test cases
– Trial of system image (to get acquainted with the system, to verify
test beds)
– Schedule of defect review meetings.
14

7
 5/12/2019

Test Deliverables
 Test Plan Document
– Test Design Specification
– Test Case Specification
 Test Input Data and Output Data
 Test Procedure Specification
 Test Incident Report
– Test Log
– Test Summary Report
– Test Metrics, Coverage Analysis
 Program Quality Estimate
 Test Tool(s)

15

What Must be Included?

Goal
Introduction Scope

Test Spec Test Plan Unit Test

Integration Test
Validation Test
High-Order Test
Test Procedure

8
 5/12/2019

Tested By:
Test Type

Test Report Form Test Case Number

Test Case Name

Test Case Description

Item(s) to be tested
1
2
Specifications
Expected Output/Result
Input

Procedural Steps
1
2
3