Chap 2.

Software Review and Inspection

1. Introduction
2. Types of Software Reviews
3. Software Inspection
4. Inspection Checks
5. Inspection Metrics

1
1. Introduction
-Quality control activities determine whether a product conforms to
its requirements, specifications, or pertinent standards.
÷Nonconformance constitute defects.
-Performing quality control within the development process is more
efficient than looking for errors after the product has been completed.
-Software review is a verification method that is used to improve
work products before they are completed.
÷It consists of having people other than the author of a work product examine
that product to find defects and identify improvement opportunities.

-Software review is essential for uncovering a host of defects that

other quality control techniques such as testing or static analysis
are not adequate for or would miss. These include, for instance:
÷Product compliance with standards
÷Detecting logic errors in implementation
÷Spotting erroneous, ambiguous, or missing requirements 2
÷Judging clarity or maintainability of the code
2. Types of Software Reviews
-There are various types of review based on the degree of formality:
÷Buddy checking: consist of having a person other than the author informally
review a document or work. In general this doesn’t involve the use of checklists
to guide the process and as such is not repeatable.

÷Walkthrough: involve the author of an artifact presenting it to an audience of

their peers, and receiving comments and feedbacks. Usually, such process
involves limited documentation of the process and the issues uncovered, which
makes defect tracking difficult.

÷Review by circulation: consist of circulating an artifact among peers for

comments; operates like a walkthrough but without holding a meeting. Not holding
a meeting avoids potential arguments over issues, but also the benefits of discussion.

÷Inspection: formal and managed peer review process with the following
characteristics:
-The process is carried out by a review team with clearly defined roles.
-The process follows an unambiguous set of criteria for each type of artifacts.
-Specific data are collected during the process. 3
-The process is driven by quantitative goals such as process and quality improvements.
3. Software Inspection
-Software inspection was introduced by Michael Fagan in the 1970s,
as an elaborated process for systematic review of software artifacts.

-Since its introduction, inspection has been used on a wide variety of

software artifacts: code as well as all the other documents such as
specification, design, and test data.

-The main issues underlying software inspection:

÷Inspection can be very expensive because of the head counts (4-5 persons) and
time required;
÷Inspection outcomes depend on the experience and training of the inspectors;

-Still, inspection can be very effective when applied to a project

from start to finish.

4
-According to data collected from various studies, the benefits of
inspection include the following:
÷Net increase in productivity in the range of 30-100%;
÷Overall project time saving of 10-30%;
÷5 to 10 times reduction in test execution costs and time;
÷Reduction in maintenance costs of up to one order of magnitude;
÷Improvement in consequent product quality.
÷Minimal defect correction backlash at systems integration time.
-According to many previous studies inspection is more efficient than
testing, specifically when comparing inspection of code modules
with testing those modules.
÷Inspection finds most of the problems testing would find, and does so more
efficiently.

-But such enthusiasm should be tempered by the fact that some

subtle bugs may escape inspection, and could only be detected
through testing by executing the software code.
÷So, inspection and testing should be considered as complementary processes.
5
Inspection Participants and Procedure
Participants
-The following key participants are involved in a formal inspection
process:
÷Author (or Owner): the person who wrote the document or work product under
review; his role in the inspection consists solely of presenting his work, and helping
others understand it, but not of “defending” it.
÷Moderator: runs the inspection, by rigorously enforcing its purpose, which is to
discover deficiencies in the document being inspected.
÷Recorder: is in charge of recording possible deficiencies uncovered during the
inspection process, and along with the moderator, of preparing at the end of the
process, an inspection report, which will be used to fix the problems found.

÷Reader: Paraphrases the code or document at an inspection meeting.

÷Inspectors: are the ones who raise questions, suggest problems, and criticize
the document (but are not supposed to “attack” it). Inspectors may include people
from different expertise (e.g., quality assurance, marketing etc.), each bringing a
different viewpoint. 6
 Procedure
-All the participants are supposed to study the document in advance,
and identify issues to raise.
-Since often the same issues may come over and over, inspectors can
work from checklists of things to look for, lists to which they add as
their experience accumulates.
-Since code can be execution tested whereas other documents cannot,
any inspection program should start with the other documents.

-Formal inspections requires a minimum of four people and is best with

seven participants.
÷In small organizations, the personnel don’t exist to systematically carry it out.
÷In such case, if two engineers work well together, they can do useful inspection
with no moderator, and one of them acting as the recorder.

7
 Inspection Process

÷Planning: Identifies work product to be inspected, determines the size and composition
of the inspection team, and sets the inspection schedule.
÷Overview: Optional phase where team members who are unfamiliar with the work product
to be inspected receive orientation.
÷Preparation: Team members inspect the work individually looking for defects in the work
product with the guidance of relevant checklist.
-The majority of defects found in inspection processes are found in this stage (about 75%);
-The reviewer should record any issues found, and determine their severity (i.e., major or minor):
•major for a defect that will cause an observable product failure or departure from requirements;
•minor for a defect that will not cause a failure in execution of the product.
-At this stage, it is strictly required that reviewers work individually and do not attempt to find 8
solutions to defects found as this is an improper use of their time.
Inspection Process (ctd.)

÷Inspection Meeting: Inspection team members meet to discuss possible defects in the work
product. The moderator should ensure that all the issues raised by individual reviewers are
appropriately logged.
÷Rework: The work product is revised to conform to requirements and specifications.
÷Follow up: The rework is verified, final inspection data is collected and summarized, and the
inspection is officially closed. At this stage also, the moderator may calculate certain metrics
to assess the effectiveness of the inspection and recommend improvement to the inspection
process. 9
-Example of Individual Reviewer Log form (used during preparation stage)

10
 -Example of Log form (used during meeting phase)
Date: 14 March 2003
Team: DT1
Part/Level: WCMS-HLD-1-03 Cycle: 1
Moderator: M. Python Owner: M. Bean

Engineer Data

Name Defects Preparation Data Est.

Major Minor Size Time Rate Yield
(Size Units) (minutes) (Size Units/hr)
J.B. 8 2 15 pages 60 minutes 15 pages/hr
R. M. 9 4 15 pages 75 minutes 12 pages/hr
Totals: 67.5 minutes 13.5 pages/hr

Defect Data

No. Defect Description Engineers

(finding major defects)
J.B. R. M.
1.01 TBD indicates incomplete specification 1 1
UCD Missing use case for adding resources to courses 1 1
UCD Extra use case “Check Enrolment” 1
UCD Term course used inconsistently 1
SD 4.3 Message to list students incorrectly passed to 1 1
instance of Student
SD 4.10 Name does not match UCD 1 1
SD 4.11 Message to list courses incorrectly passed to 1 1
instance of Course
SD Message to add assessment item used inconsistently 1
4.14/4.15
SD4.23 Sequence diagram describes interaction not defined 1 1
in requirements – no corresponding use case
SD4.24 Sequence diagram describes interaction not defined 1 1
in requirements – no corresponding use case
SD4.25 Missing alternate for check weighting 1 1

Totals: 8 9
Unique Defects: 2 3
Inspection Summary Product Size: 15 Size Measure: pages
Total Defects for J.B: 8 Total Defects for R.M: 9 Common Defects: 6
Est. Total Defects: Total Number Found: Number Left:
Meeting Time: 30 min Total Insp. Hours: Overall Rate:
11
Major Defects/Total: Defects / Size Unit: Defects / Hour
Inspection Workload

-The amount of code which can be inspected in a given time depends

on the experience of the inspection team, the programming language
and the application domain.
-Measurement data collected at IBM, and confirmed by AT&T led
to the following observations:
1. About 500 source code statements per hour can be considered during the
overview stage.
2. During individual preparation, about 125 source code statements per hour can
be examined.
3. From 90 to 125 statements per hour can be inspected during the meeting.

-It was suggested that the maximum time spent on an inspection

should be about two hours as the efficiency of the defect detection
process falls off after that time.
÷Inspection should therefore be a frequent process, carried out on relatively small
software components, during program development (…1-2 hours prep + 1h 12 meeting)
 4. Inspection Checks
-The inspection process should always be driven by a checklist of
common programming errors.
÷This should be established by discussion with experienced staff and regularly
updated as more experience is gained of the inspection process.
÷Different checklist should be prepared for different programming languages.
÷Possible checks which can be made during the inspection include:

Fault class Inspection Check

Data Faults Are all program variables initialized before their values are used? Have all constant
been named? Is there any possibility of buffer overflow? Etc.
Control Faults For each conditional statement, is the condition correct? Is each loop certain to
terminate? Are compound statements correctly bracketed? Etc.
Input/output faults Are all input variables used? Are all output variables assigned a value before they are
output? Can unexpected inputs cause corruption? Etc.
Interface faults Do all function and method calls have the correct number of parameters? Do formal
and actual parameter types match? Are the parameter in right order? If components
access shared memory, do they have the same model of the shared memory structure?
Storage management faults If a linked structure is modified, have all links been correctly reassigned? If dynamic
storage is used, has space been allocated correctly? Is space explicitly de-allocated
after it is no longer required? Etc.
Exception management faults Have all possible error conditions been taken into account? 13
Sample Inspection Checklists
Requirements Inspection Checklist
1. Do requirements exhibit a clear distinction between functions and data?
2. Do requirements define all the information to be displayed to users?
3. Do requirements address system and user response to error conditions?
4. Is each requirement stated clearly, concisely, and unambiguously?
5. Is each requirement testable?
6. Are there ambiguous or implied requirements?
7. Are there conflicting requirements?
8. Are there areas not addressed in the Software Requirements Specification (SRS) that need to be?
9. Are performance requirements (such as response time, data storage requirements) stated?

Sample Design Inspection Checklist Questions

(Error Handling and Recovery)
1. Is there adequate error condition testing?
2. Are error conditions tested where the probability of an error is
high or results of an error would be fatal to the system?
3. Are return codes documented?
4. Are return messages understandable?
5. Does the program allow for successful error recovery...
- across module or process failures?
- across operating system failure?
- across interrupts?
- across hardware failures? 14
(Java) Code Inspection Checklist

1. Variable and Constant Declaration Defects (VC)

1. Are descriptive variable and constant names used in accord with naming conventions?
2. Are there variables with confusingly similar names?
3. Is every variable properly initialized?
4. Could any non-local variables be made local?
5. Are there literal constants that should be named constants?
6. Are there macros that should be constants?
7. Are there variables that should be constants?

2. Function Definition Defects (FD)

8. Are descriptive function names used in accord with naming conventions?
9. Is every function parameter value checked before being used?
10. For every function: Does it return the correct value at every function return point?

3. Class Definition Defects (CD)

11. Does each class have an appropriate constructor and destructor?
12. For each member of every class: Could access to the member be further restricted?
13. Do any derived classes have common members that should be in the base class?
14. Can the class inheritance hierarchy be simplified?

15
 5. Inspection Metrics
-The data collected during a software process are used to compute a set
of metrics that support evaluation and improvement of the process as
well as planning and tracking quality.
-The metrics computed during such process should be defined by the
requirements of your organization (typically in the quality manual).
÷The collection of data and calculation of metrics for no reason is a waste of time.

-Many different metrics can be calculated during an inspection

process including the following:
•The number of major and minor defects found
•The number of major defects found to total found. (If the proportion of minor defects to
major defects is too large a moderator may request that the reviewer repeat the review, focusing
on major defects, prior to commencing the logging meeting)

•The size of the artifact (pages, LOC, ...)

•The rate of review - the size of the reviewed artifact divided by time (normally expressed in
hours) (e.g., 15 pages /hour). 16
•The defect detection rate - the number of major defects found per review hour.
Total Number of Defects Found and Defects Density
-Total number of defects found is the sum of the total number of defects
found by each reviewer, minus the number of common defects found.
÷For instance, with 2 reviewers, the metric is computed by

Total Defects Found = A + B - C

Where A and B are the number found by reviewer A and B respectively and C is
the number found by both A and B.

-Defect density is the ratio of the number of defects found to the size
of the artifact. It is given by

Defect Density = Total Defects Found / Size

Where the size of the artifact is measured in number of pages, loc, or other size measure.

17
Example: compute inspection metrics from the following inspection log form
Date: 14 March 2003
Team: DT1
Part/Level: WCMS-HLD-1-03 Cycle: 1
Moderator: M. Python Owner: M. Bean

Engineer Data

Name Defects Preparation Data Est.

Major Minor Size Time Rate Yield
(Size Units) (minutes) (Size Units/hr)
J.B. 8 2 15 pages 60 minutes 15 pages/hr
R. M. 9 4 15 pages 75 minutes 12 pages/hr
Totals: 67.5 minutes 13.5 pages/hr

Defect Data

No. Defect Description Engineers

(finding major defects)
J.B. R. M.
1.01 TBD indicates incomplete specification 1 1
UCD Missing use case for adding resources to courses 1 1
UCD Extra use case “Check Enrolment” 1
UCD Term course used inconsistently 1
SD 4.3 Message to list students incorrectly passed to 1 1
instance of Student
SD 4.10 Name does not match UCD 1 1
SD 4.11 Message to list courses incorrectly passed to 1 1
instance of Course
SD Message to add assessment item used inconsistently 1
4.14/4.15
SD4.23 Sequence diagram describes interaction not defined 1 1
in requirements – no corresponding use case
SD4.24 Sequence diagram describes interaction not defined 1 1
in requirements – no corresponding use case
SD4.25 Missing alternate for check weighting 1 1

Totals: 8 9
Unique Defects: 2 3
Inspection Summary Product Size: 15 Size Measure: pages
Total Defects for J.B: 8 Total Defects for R.M: 9 Common Defects: 6
Est. Total Defects: Total Number Found: Number Left:
Meeting Time: 30 min Total Insp. Hours: Overall Rate:
18
Major Defects/Total: Defects / Size Unit: Defects / Hour
 Estimated Total Number of Defects

-The estimated total number of defects is the sum of the total number
of defects found and the estimated total number of defects remaining.
÷In order to estimate the total number of defects remaining in an artifact immediately
after inspection, we use an approach similar to the population sampling approach
used by biologists to estimate the population of a particular ecosystem.

Population Sampling Approach

-Suppose we have a fish farm and we want to estimate the total
number of fish we have. We could apply the capture-recapture method:
1. Capture a number of fish, tag them and release them (let this number be S1).
2. Allow time for the first sample population to redistribute.
3. Capture a second number of fish (let this number be S2).
4. Count the number of tagged fish in the second population (let this number be ST).
5. Calculate the proportion of tagged fish in the second population (let this number be T,
then T= ST/ S2).
6. We assume that T is representative of the proportion of tagged fish in the total
population (POP), so T∝POP= S1, or for our purposes POP= S1 /T.
19
-Using the population sampling approach to estimate the number
of defects remaining leads to the following steps:

Let the number of defects found by one reviewer be the tagged population (A).

1. Assume an even likelihood of finding all defects (even distribution,...)

2. Count the number of defects found by the second reviewer (B).
3. Count the number of defects found by the second reviewer that were also found by the
first (C the common defects).
4. Calculate the proportion of common defects in the second reviewers defects (T=C/B).
5. We assume that T is representative of the proportion of common defects in the total
number of defects (EstTot), so T ∝EstTot=A, or for our purposes
EstTot = A/T = (A ∝ B)/C.

-So assuming that defects are equally likely to be found in an artifact

and each reviewer is equally likely to find every defect, then:
Estimated Total Defects = (A ∝ B)/C

÷Note that in practice such assumptions are not always fulfilled, simply because some
defects are harder to find than others, and some reviewers are better than others.
20
Inspection Yield

-Inspection yield refers to the defect removal efficiency (DRE) of an

inspection process.

-The defect removal efficiency of a process is given by calculating

the percentage of total defects that were found by that process. So
the yield of an inspection is given by:

Yield = Total Defects Found / Estimated Total Defects ∝ 100%

21
Inspection Rate and Defect Detection Rate

-Requires computing the inspection time, which is the sum of each

reviewers review time plus the total person time spent in each meeting.
-The inspection rate is computed by:

Inspection rate = size / total inspection time

Where:
÷size stands for the size of the artifact in number of pages, loc, or other size measure.
÷total inspection time measured in hours

-The defect detection rate is computed by:

Defect finding efficiency = Total defects found / total inspection time

22
Calculating Inspection Metrics with more than two Reviewers

-If there are more than 2 reviewers, the same approach can be taken
to calculate inspection totals and yield by grouping reviewers into
two groups for calculation: group A and group B:

÷If there are 3 reviewers, it is often a good idea to choose the person who has the most
unique defects to be one group and the other two reviewers to be the other group. For
each group if any member of that group has found a defect, then count it for the group.

23
Example with 3 reviewers:
No. Defect Description Engineers (finding major defects)

R1 R2 R3 A (R2) B
1 1 1

1 1
1 1
1 1 1
1 1 1
1 1
1 1
1
1
1
1 1 1

Totals 7 9 7
Unique defects 1 2 0

Inspection Summary Product size: 10 Size measure: pages

Total defects for A: Total defects for B: Common defects: 24
Est. total defects: Total number found: Number left: