BITP 3253:

Validation& Verification
Week 5: Static Technique

Fakulti Teknologi Maklumat dan Komunikasi

 Overview

• Software Work Product and Static Techniques

• Static Vs Dynamic Testing
• Introduction of Reviews
• Reviews Types
• Reviews Process
• Reviews : Roles and Responsibilities

Fakulti Teknologi Maklumat dan Komunikasi

Software Work Product and Static
Techniques
• Reviews and tools
• Review range from informal to very formal
• Tools can perform some types of static tests
• Static techniques can be used for requirements and designs,
plus code, database schemas, documentation, tests..
• Models and prototypes
• A diagram of a complex system can often reveal design
problems that can hide in words
• An ugly diagram often means lots of bugs
• Test cases and data
• Test analysis and design based on requirements and design
specs is a form of structured review
• Test analysis and design often reveals problems

Fakulti Teknologi Maklumat dan Komunikasi

 Static Tools

• Static analysis
• Problematic wording: Spell/grammar checkers
• Dangerous programming: J-Test, Safer C, lint..
• Measurement: Complexity analysis
• System simulations
• General Purpose System Simulator
• Performance modeling/operations research tools
• Spreadsheets

Fakulti Teknologi Maklumat dan Komunikasi

 Static Vs Dynamic Testing
Similarities Differences
• Seek to identify defects • Each technique can find
• Work best when a broad different types of defects
cross sections of more effectively. (e.g. find
stakeholders are involved missing requirements is
more effective using
• Save the company
static technique)
money and time
• Static techniques find
defects rather than
failures

Fakulti Teknologi Maklumat dan Komunikasi

 Static Techniques

Static
Techniques

Review Static
Analysis

Fakulti Teknologi Maklumat dan Komunikasi

 Reviews
• Reviews are a way of testing software work products
(including code) and can be perform way before
dynamic testing
• A review could be done entirely as a manual activity,
but there is also tool support.
• The main manual activity is to examine a work product
and make comment about it.
• Any software work product can be reviewed. E.g.
requirement specifications, design specifications, test
specification, test cases, test scripts, user guides and
etc
• The objective is to find defects rather than failure

Fakulti Teknologi Maklumat dan Komunikasi

 Reviews: Costs and Benefits
• Costs
• Time required to perform reviews
• Effort required to gather and analyze metrics
• Process improvement
• Benefits
• Shorter schedule – due to early defect detection and correction
• Shorter testing periods and lower testing costs
• Development productivity improvements
• Improved quality of product (which reduces downstream
lifetime cost)
• Improved communications and collaborations
• Bottom line: Reviews of all kinds are proven, high-return
techniques for improving quality

Fakulti Teknologi Maklumat dan Komunikasi

 Reviews : What can be found?
• Reviews, static analysis and dynamic testing have the
same objective-identifying defects
• They are complementary-different techniques can find
different types of defects effectively and efficiently
• Typical defects found in reviews rather that dynamic
testing:-
• Deviations from standards
• Requirement defects
• Design defects
• Insufficient maintainability
• Incorrect interface specifications

Fakulti Teknologi Maklumat dan Komunikasi

 Types of Reviews : Informal

• Main purpose: Inexpensive way to get some

benefit
• No formal process (hallway chats, buddy tests,
pair programming), yet useful
Software Work Product and
• Cheap, popular
Static Techniques
• May vary in usefulness depending on the
reviewer
• Optionally maybe documented

Fakulti Teknologi Maklumat dan Komunikasi

Types of Reviews : Walkthrough

• Main purpose: Learning, gaining understanding,

defect finding
• Meeting led by author
• Scenarios, dry run, peer group
• Open-ended sessions
• Optionally, a pre-meeting reviewers, review
report, list of findings and scribe
• May vary in practice from quite informal to very
formal

Fakulti Teknologi Maklumat dan Komunikasi

 Types of Reviews : Technical
Reviews
• Main purpose: Discuss, make decisions, evaluate
alternatives, , find defects, solve technical problems
and check conformance to specifications and
standards
• Documented, defined defect process that includes
peers and technical experts; without managers
• Ideally led by trained moderator(not the author)
• Pre-meeting preparation
• Optionally the use of checklists, review report, list of
findings and management participants
• May vary In practice from quite informal to formal
• Fakulti Teknologi Maklumat dan Komunikasi
 Types of Reviews : Inspections
• Main purpose: Find defects
• Led by trained moderator (not the author)
• Formal process based on rules and checklists with entry
and exit criteria, which includes gathering defect removal
metrics
• Defined roles
• Formal follow-up process
• Deliverable: Inspection report, list of findings
• Optionally involve process improvement
When walkthroughs, technical reviews or inspections are
performed by a peer group (colleagues at the same
organizational level), the review may be called “Peer
Review”
Fakulti Teknologi Maklumat dan Komunikasi
 Consensus and Understanding
Long before any code exists, the
• Incompleteness and specification must be handed
to an outside testing group to
ambiguity can hide the be scrutinized for
real meaning of the completeness an clarity. As
[V.A] Vyssotsky [of Bell Lab’s
specifications Safeguard Project] says the
• Agreement and developers themselves cannot
do this:”They won’t tell you
uniform understanding they don’t understand it; they
of the specifications will happily invent their way
through the gaps and
obscurities”
-Fred Brooks
The Mythical Man-Mouth (1975)
Fakulti Teknologi Maklumat dan Komunikasi
 A Generic Review Process

1. Planning – Selecting of personnel, allocating

roles; training participants defining the entry and
exit criteria for more formal review types; and
selecting which parts of documents to look at.

2. Kick off – Distributing documents, explaining the

objectives, process and documents to the
participants; and checking entry criteria (for
more formal review types)

Fakulti Teknologi Maklumat dan Komunikasi

 A Generic Review Process
3. Individual Preparation – Work done by each
participants on their own before the review meeting,
noting potential defects, questions and comments
4. Review Meeting- Discussion or logging, with
documented results or minutes (for more formal
review types). The meeting participants may simply
note defects, make recommendations for handling the
defects, or make decisions about the defects
5. Rework/Repair– Fix defects found, typically done by
the author

These steps of the process repeat per each item reviewed.

Preparation is usually one to two hours alone. Meeting is one to
two hours together.
Fakulti Teknologi Maklumat dan Komunikasi
 A Generic Review Process

6. Follow-up – Checking the defects have been

addressed, gathering metrics and checking on
exit criteria (for more formal review process)

Yet, the details of the review process

depend on the specific review type
used on the project

Fakulti Teknologi Maklumat dan Komunikasi

 Reviews : Roles &
Responsibilities
Manager – Decides on the execution reviews,
plan(time & schedules), arrange resources and
training, determines if the review objectives have
been met
Moderator – Lead the review meetings, including
running the meeting, and follow up after the
meeting. The moderator may mediate between
various points
Author – The writer or person with chief
responsibility for the document(s) to be reviewed.
Describe, explain and answer questions on the
document(s)
Fakulti Teknologi Maklumat dan Komunikasi
 Reviews : Roles &
Responsibilities
Reviewers/Inspectors – Individuals with a specific
technical or business background who, after the
necessary preparation, identify and describe
findings(e.g. defects) in the product under review.
Reviewers should be chosen to represent different
perspectives and roles in the review process and
should take part in any review meetings.
Scribe – Document all the issues, problems and
open points the were identified during the meeting
** In some cases, one person may play multiple roles:-
• Authors sometimes act as moderators
• One of the reviewers can act as secretary
• The specifics are determined by the type of review
Fakulti Teknologi Maklumat dan Komunikasi
 Suggestions for Successful
Reviews
• Provide training • Insist on preparation(e.g
• Review the product, not by having people submit
the producer notes)
• Set and follow agenda • Develop a checklist for
and objectives each type of item that is
reviewed
• Limit debate
• Review the reviews
• Focusing on finding, not
fixing, problems • User the right techniques
• Take written notes • Ensure management
support
• Limit and carefully select
participants • Learn and get better!
Fakulti Teknologi Maklumat dan Komunikasi
 Common Requirements and
Design Bugs
• Ambiguities: What exactly does that mean?
• E.g. System shall allow user to read ISP email
• What is ISPs? What size e-mails? Attachments?
• Incompleteness: Okay, and then what?
• E.g. Upon three invalid passwords, system shall lock user
account..
• For how long? How to unlock? Who can unlock?
• Untestability: How can I check this item?
• E.g. System shall provide 100% availability
• No known test technique to demonstrate perfect availability
• Excessive dependencies, coupling and complexity
• Look for ugly design diagrams and confusing requirements
Fakulti Teknologi Maklumat dan Komunikasi
 IEEE 1028: Software Review
1. Overview
Standard
 Purpose, scope, conformance, organization, application
2. References
3. Definitions
4. Management Reviews
 Responsibilities, inputs/outputs, entry/exit criteria, procedures
5. Technical reviews
 Responsibilities, inputs/outputs, entry/exit criteria, procedures
6. Inspections
 Responsibilities, inputs/outputs, entry/exit criteria, procedures, data collection,
process improvement
7. Walkthroughs
 Responsibilities, inputs/outputs, entry/exit criteria, procedures, data collection, process
improvement
8. Audits
 Responsibilities, inputs/outputs, entry/exit criteria, procedures

Fakulti Teknologi Maklumat dan Komunikasi

 Static Analysis

• Static analysis is to find defects in software code

and software model
• It is to be performed without actually executing
the software being examined by using the tool
• Likewise the reviews, static analysis is to find
defects rather than failures

Fakulti Teknologi Maklumat dan Komunikasi

 Static Analysis Vs Dynamic
Testing
• Like dynamic testing, static analysis looks for defects in
software code and software models
• Unlike dynamic testing, static analysis is performed without
actually executing the system
• Static analysis involves analysis of the system or its
components by a tool, while dynamic testing does not
always involve tools
• Static analysis can find defects that are hard to find or
isolate in dynamic testing
• Examples includes maintainability issues, unsafe pointer
use.
• Isolation is easier because you find the bug, not the
symptom
Fakulti Teknologi Maklumat dan Komunikasi
 What Can We Analyze?
• Program code (e.g. control flow and data flow)
• Models of the program
• Generated output such as HTML and XML
• Requirements and design documents

Fakulti Teknologi Maklumat dan Komunikasi

 Benefits of Static Analysis
• Early and cheaper detection of bugs (before test
execution starts)
• Early warning about suspicious aspects of the code or
design, or where bug clusters might exist, due to
dangerous programming high complexity, etc
• Identification of defects not easily found by dynamic
testing
• Detecting dependencies and inconsistencies in
software models, such as links
• Improved maintainability of code and design
• Prevention of defects, if lessons are learned in
development
Fakulti Teknologi Maklumat dan Komunikasi
 Typical Static Analysis Bugs

• Referencing a variable with undefined value

• Inconsistent interface between modules and
components
• Variables that are never used
• Unreachable (dead) code
• Programming standard violations
• Security vulnerabilities
• Syntax violations of code and software models

Fakulti Teknologi Maklumat dan Komunikasi

 Using Static Analysis Tools

• Typical users are..

• Programmers, often during component and
integration testing
• Designers and system architects during design
• During initial introduction against an existing
system, static analysis tools may produce a large
number of warning messages
• Compilers do some static analysis, but many
sophisticated tools are available

Fakulti Teknologi Maklumat dan Komunikasi

Thank You

Fakulti Teknologi Maklumat dan Komunikasi