Software Testing Methodologies

Page |1

Unit I
Introduction
What is Testing?
Related terms : SQA, QC, Verification, Validation
Verification of functionality for conformation against given specifications
By execution of the software application

A Test
Passes:
Functionality OK.
Fails:
Application functionality NOK.
Bug/Defect/Fault: Deviation from expected functionality.
Its not always obvious.

Purpose of Testing
1. To Catch Bugs
Bugs are due to imperfect Communication among programmers
Specs, design, low level functionality
Statistics say: about 3 bugs / 100 statements
2. Productivity Related Reasons
Insufficient effort in QA => High Rejection Ratio =>
Higher Rework => Higher Net Costs
Statistics:
QA costs:
2% for consumer products
80% for critical software
Quality Productivity
3. Goals for testing
Primary goal of Testing: Bug Prevention
Bug prevented rework effort is saved
[bug reporting, debugging, correction,
retesting]
If it is not possible, Testing must reach its secondary goal of bud discovery.
Good test design & tests clear diagnosis easy bug correction
Test Design Thinking
From the specs, write test specs. First and then code.
Eliminates bugs at every stage of SDLC.
If this fails, testing is to detect the remaining bugs.
4. 5 Phases in testers thinking
Phase 0: says no difference between debugging & testing
Today, its a barrier to good testing & quality software.
Phase 1: says Testing is to show that the software works
A failed test shows software does not work, even if many tests pass.
Objective not achievable.
Phase 2: says Software does not work
One failed test proves that.
Tests are to be redesigned to test corrected software.
But we do not know when to stop testing.

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

Page |2

Phase 3: says Test for Risk Reduction

We apply principles of statistical quality control.
Our perception of the software quality changes when a test passes/fails.
Consequently, perception of product Risk reduces.
Release the product when the Risk is under a predetermined limit.
Phase 4: A state of mind regarding What testing can do & cannot do. What makes
software testable.
Applying this knowledge reduces amount of testing.
Testable software reduces effort
Testable software has less bugs than the code hard to test
Cumulative goal of all these phases:
Cumulative and complementary. One leads to the other.
Phase2 tests alone will not show software works
Use of statistical methods to test design to achieve good testing at acceptable risks.
Most testable software must be debugged, must work, must be hard to break.
5. Testing & Inspection
1. Inspection is also called static testing.
2. Methods and Purposes of testing and inspection are different, but the objective is to catch &
prevent different kinds of bugs.
3. To prevent and catch most of the bugs, we must
1. Review
2. Inspect &
3. Read the code
4. Do walkthroughs on the code

Some important points:

Test Design After testing & corrections, Redesign tests & test the redesigned tests
Bug Prevention
Mix of various approaches, depending on factors
culture, development environment, application, project size, history, language
Inspection Methods
Design Style
Static Analysis
Languages having strong syntax, path verification & other controls
Design methodologies & development environment
Its better to know:
Pesticide paradox
Complexity Barrier

Dichotomies
division into two especially mutually exclusive or contradictory groups or entities
the dichotomy between theory and practice
Let us look at six of them:
1. Testing & Debugging
2. Functional Vs Structural Testing
3. Designer vs Tester

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

Page |3

4. Modularity (Design) vs Efficiency

5. Programming in SMALL
Vs programming in BIG
6. Buyer
vs Builder
Testing Vs Debugging
1. Testing is to find bugs.
2. Debugging is to find the cause or misconception leading to the bug.
3. Their roles are confused to be the same. But, there are differences in goals, methods and
psychology applied to these
# Testing
Debugging
1 Starts with known conditions. Uses
Starts with possibly unknown initial
predefined procedure. Has predictable
conditions. End cannot be predicted.
outcomes.
2 Planned, Designed and Scheduled.
Procedures & Duration are not
constrained.
3 A demo of an error or apparent correctness.
A Deductive process.
4 Proves programmers success or failure.
It is programmers Vindication.
5 Should be predictable, dull, constrained, rigid There are intuitive leaps, conjectures,
& inhuman.
experimentation & freedom.
6 Much of testing can be without design
Impossible without a detailed design
knowledge.
knowledge.
7 Can be done by outsider to the development
Must be done by an insider (development
team.
team).
8 A theory establishes what testing can do or
There are only Rudimentary Results (on
cannot do.
how much can be done. Time, effort, how
etc. depends on human ability).
9 Test execution and design can be automated.
Debugging - Automation is a dream.

Functional Vs Structural Testing

Functional Testing: Treats a program as a black box. Outputs are verified for conformance to
specifications from users point of view.
Structural Testing: Looks at the implementation details: programming style, control method, source
language, database & coding details.
Interleaving of functional & Structural testing:
A good program is built in layers from outside.
Outside layer is pure system function from users point of view.
Each layer is a structure with its outer layer being its function.

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

Page |4

Examples:

Designer vs

For a given model of programs, Structural tests may be done first and later the Functional, Or
vice-versa. Choice depends on which seems to be the natural choice.
Both are useful, have limitations and target different kind of bugs. Functional tests can detect
all bugs in principle, but would take infinite amount of time. Structural tests are inherently
finite, but cannot detect all bugs.
The Art of Testing is how much allocation % for structural vs how much % for functional.
Tester
Completely separated in black box testing. Unit testing may be done by either.
Artistry of testing is to balance knowledge of design and its biases against ignorance &
inefficiencies.
Tests are more efficient if the designer, programmer & tester are independent in all of unit,
unit integration, component, component integration, system, formal system feature testing.
The extent to which test designer & programmer are separated or linked depends on testing
level and the context.

Programmer / Designer

Tester

Tests designed by designers are more

oriented towards structural testing and
are limited to its limitations.

With knowledge about internal test design, the

tester can eliminate useless tests, optimize & do
an efficient test design.

Likely to be biased.

Tests designed by independent testers are biasfree.

Tries to do the job in simplest &

cleanest way, trying to reduce the
complexity.

Tester needs to suspicious, uncompromising,

hostile and obsessed with destroying program.

Modularity (Design) vs Efficiency

1. system and test design can both be modular.
2. A module implies a size, an internal structure and an interface, Or, in other words.

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

Page |5

3. A module (well defined discrete component of a system) consists of internal complexity &
interface complexity and has a size.

Modularity

Efficiency

Smaller the component easier to

understand.

Implies more number of components & hence

more # of interfaces increase complexity &
reduce efficiency (=> more bugs likely)

Small components/modules are

repeatable independently with less
rework (to check if a bug is fixed).
Microscopic test cases need individual
setups with data, systems & the
software. Hence can have bugs.

Higher efficiency at module level, when a bug

occurs with small components.

Easier to design large modules &

smaller interfaces at a higher level.

Less complex & efficient. (Design may not be

enough to understand and implement. It may
have to be broken down to implementation
level.)

More # of test cases implies higher possibility

of bugs in test cases. Implies more rework and
hence less efficiency with microscopic test
cases

Programming in SMALL
Vs programming in BIG
Impact on the development environment due to the volume of customer requirements
#

Small

More efficiently done by informal, intuitive means

and lack of formality
if its done by 1 or 2 persons for small &
intelligent user population.
Done for e.g., for oneself, for ones office or for the
institute.

A large # of programmers & large # of

components.

Complete test coverage is easily done.

Acceptance level could be: Test coverage

of 100% for unit tests and for overall tests
80%.

B ig

Program size implies non-linear effects (on

complexity, bugs, effort, rework quality).

Buyer
Vs Builder (customer vs developer organization)
Buyer & Builder being the same (organization) clouds accountability.
Separate them to make the accountability clear, even if they are in the same organization.
The accountability increases motivation for quality.
The roles of all parties involved are:
Builder:
Designs for & is accountable to the Buyer.
Buyer:
Pays for the system.
Hopes to get profits from the services to the User.

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

Page |6

User:
Ultimate beneficiary of the system.
Interests are guarded by the Tester.
Tester:
Dedicated to the destruction of the s/w (builder)
Tests s/w in the interests of User/Operator.
Operator:
Lives with:

Mistakes of the Builder

Oversights of Tester

Murky specs of Buyer

Complaints of User

A Model for Testing

A model for testing - with a project environment - with tests at various levels.
(1) understand what a project is.
(2) look at the roles of the Testing models.
1. Project:
An Archetypical System (product) allows tests without complications (even for a large project).
Testing a one shot routine & very regularly used routine is different.
A model for project in a real world consists of the following 8 components:
1) Application: An online real-time system (with remote terminals) providing timely responses to
user requests (for services).
2) Staff: Manageable size of programming staff with specialists in systems design.
3) Schedule: project may take about 24 months from start to acceptance. 6 month maintenance
period.
4) Specifications: is good. documented. Undocumented ones are understood well in the team.
5) Acceptance test: Application is accepted after a formal acceptance test. At first its the
customers & then the software design teams responsibility.
6) Personnel: The technical staff comprises of : A combination of experienced professionals &
junior programmers (1 3 yrs) with varying degrees of knowledge of the application.
7) Standards:
Programming, test and interface standard (documented and followed).
A centralized standards data base is developed & administrated
8) Objectives: (of a project)
A system is expected to operate profitably for > 10 yrs
(after installation).
Similar systems with up to 75% code in common may be implemented in future.
9) Source: (for a new project) is a combination of
New Code
- up to 1/3rd
From a previous reliable system
- up to 1/3rd
Re-hosted from another language & O.S.
- up to 1/3rd
10) History: Typically:
Developers quit before his/her components are tested.
Excellent but poorly documented work.
Unexpected changes (major & minor) from customer may come in
Important milestones may slip, but the delivery date is met.
Problems in integration, with some hardware, redoing of some component etc..

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

Page |7

A model project is
A Well Run & Successful Project.
Combination of Glory and Catastrophe

2. Roles of Models for Testing

1) Overview:

Testing process starts with a program embedded in an environment.

Human nature of susceptibility to error leads to 3 models.

Create tests out of these models & execute

Results is expected Its okay

unexpected Revise tests and program. Revise bug model and program.
2) Environment: includes

All hardware & software (firmware, OS, linkage editor, loader, compiler, utilities,
libraries) required to make the program run.

Usually bugs do not result from the environment. (with established h/w & s/w)

But arise from our understanding of the environment.

3) Program:

Complicated to understand in detail.

Deal with a simplified overall view.

Focus on control structure ignoring processing & focus on processing ignoring

control structure.
If bugs not solved, modify the program model to include more facts, & if that fails,
modify the program.
4) Bugs: (bug model)

Categorize the bugs as initialization, call sequence, wrong variable etc..

An incorrect spec. may lead us to mistake for a program bug.

There are 9 Hypotheses regarding Bugs.

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

Page |8

Benign Bug Hypothesis:

1) The belief that the bugs are tame & logical.
2) Weak bugs are logical & are exposed by logical means.
3) Subtle bugs have no definable pattern.
Bug locality hypothesis:
4) Belief that bugs are localized.
5) Subtle bugs affect that component & external to it.
Control Dominance hypothesis:
6) Belief that most errors are in control structures, but data flow & data structure errors
are common too.
7) Subtle bugs are not detectable only thru control structure.
(subtle bugs => from violation of data structure boundaries & data-code separation)
Code/data Separation hypothesis:
Belief that the bugs respect code & data separation in HOL programming.
In real systems the distinction is blurred and hence such bugs exist.
Lingua Salvator Est hypothesis:
Belief that the language syntax & semantics eliminate most bugs.
But, such features may not eliminate Subtle Bugs.
Corrections Abide hypothesis:
Belief that a corrected bug remains corrected.
Subtle bugs may not.
For e.g.
A correction in a data structure DS due to a bug in the interface between modules A & B,
could impact module C using DS.
Silver Bullets hypothesis:
Belief that - language, design method, representation, environment etc. grant immunity
from bugs.
Not for subtle bugs.
Remember the pesticide paradox.
Sadism Suffices hypothesis:
Belief that a sadistic streak, low cunning & intuition (by independent testers) are
sufficient to extirpate most bugs.
Subtle & tough bugs are may not be - these need methodology & techniques.
Angelic Testers hypothesis:
Belief that testers are better at test design than programmers at code design.
5) Tests:

Formal procedures.
Input preparation, outcome prediction and observation, documentation of test,
execution & observation of outcome are subject to errors.
An unexpected test result may lead us to revise the test and test model.
6) Testing & Levels:
3 kinds of tests (with different objectives)
1) Unit & Component Testing
a. A unit is the smallest piece of software that can be compiled/assembled, linked, loaded
& put under the control of test harness / driver.
b. Unit testing - verifying the unit against the functional specs & also the implementation
against the design structure.

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

Page |9

c. Problems revealed are unit bugs.

d. Component is an integrated aggregate of one or more units (even entire system)
e. Component testing - verifying the component against functional specs and the
implemented structure against the design.
f. Problems revealed are component bugs.
2) Integration Testing:
a. Integration is a process of aggregation of components into larger components.
b. Verification of consistency of interactions in the combination of components.
c. Examples of integration testing are improper call or return sequences, inconsistent data
validation criteria & inconsistent handling of data objects.
d.
Integration testing
&
Testing Integrated Objects
are different

e. Sequence of Testing:
Unit/Component tests for A, B. Integration tests for A & B. Component testing for
(A,B) component
3) System Testing
System is a big component.
Concerns issues & behaviors that can be tested at the level of entire or major part of the
integrated system.
Includes testing for performance, security, accountability, configuration sensitivity,
start up & recovery
After understanding a Project, Testing Model, now lets see finally,
Role of the Model of testing :

Used for the testing process until system behavior is correct or until the model is insufficient
(for testing).
Unexpected results may force a revision of the model.
Art of testing consists of creating, selecting, exploring and revising models.
The model should be able to express the program.

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 10

Consequences of Bugs
Consequences:
(how bugs may affect users)
These range from mild to catastrophic on a 10 point scale.
Mild
Aesthetic bug such as misspelled output or mal-aligned print-out.
Moderate
Outputs are misleading or redundant impacting performance.
Annoying
Systems behavior is dehumanizing for e.g. names are truncated/modified arbitrarily, bills for
$0.0 are sent.
Till the bugs are fixed operators must use unnatural command sequences to get proper
response.
Disturbing
Legitimate transactions refused.
For e.g. ATM machine may malfunction with ATM card / credit card.
Serious
Losing track of transactions & transaction events. Hence accountability is lost.
Very serious
System does another transaction instead of requested e.g. Credit another account, convert withdrawals
to deposits.
Extreme
Frequent & Arbitrary - not sporadic & unusual.
Intolerable
Long term unrecoverable corruption of the Data base.
(not easily discovered and may lead to system down.)
Catastrophic
System fails and shuts down.
Infectious
Corrupts other systems, even when it may not fail.
Assignment of severity
Assign flexible & relative rather than absolute values to the bug (types).
Number of bugs and their severity are factors in determining the quality quantitatively.
Organizations design & use quantitative, quality metrics based on the above.
Parts are weighted depending on environment, application, culture, correction cost, current SDLC
phase & other factors.
Nightmares
Define the nightmares that could arise from bugs for the context of the organization/application.
Quantified nightmares help calculate importance of bugs.
That helps in making a decision on when to stop testing & release the product.
When to stop Testing
1. List all nightmares in terms of the symptoms & reactions of the user to their consequences.
2. Convert the consequences of into a cost. There could be rework cost. (but if the scope extends to the
public, there could be the cost of lawsuits, lost business, nuclear reactor meltdowns.)
3. Order these from the costliest to the cheapest. Discard those with which you can live with.
4. Based on experience, measured data, intuition, and published statistics postulate the kind of bugs
causing each symptom. This is called bug design process. A bug type can cause multiple symptoms.
5. Order the causative bugs by decreasing probability (judged by intuition, experience, statistics etc.).
Calculate the importance of a bug type as:

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 11

Importance of bug type j = C j k P j k

where,
all k
C j k = cost due to bug type j causing nightmare k
P j k = probability of bug type j causing nightmare k
( Cost due to all bug types = C jk P jk )
all k

all j

6. Rank the bug types in order of decreasing importance.

7. Design tests & QA inspection process with most effective against the most important bugs.
8. If a test is passed or when correction is done for a failed test, some nightmares disappear. As testing
progresses, revise the probabilities & nightmares list as well as the test strategy.
9. Stop testing when probability (importance & cost) proves to be inconsequential.
This procedure could be implemented formally in SDLC.
Important points to Note:
Designing a reasonable, finite # of tests with high probability of removing the nightmares.
Test suites wear out.
As programmers improve programming style, QA improves.
Hence, know and update test suites as required.

Taxonomy of Bugs
we had seen the:
1. Importance of Bugs - statistical quantification of impact
2. Consequences of Bugs - causes, nightmares, to stop testing
We will now see the:
3. Taxonomy of Bugs - along with some remedies
In order to be able to create an organizations own Bug Importance Model for the sake of controlling
associated costs
Why Taxonomy ?
To study the consequences, nightmares, probability, importance, impact and the methods of prevention
and correction.
Adopt known taxonomy to use it as a statistical framework on which your testing strategy is based.
6 main categories with sub-categories..
1. Requirements, Features, Functionality Bugs
24.3% bugs
2. Structural Bugs
25.2%
3. Data Bugs
22.3%
4. Coding Bugs
9.0%
5. Interface, Integration and System Bugs
10.7%
6. Testing & Test Design Bugs
2.8 %
1) Requirements, Features, Functionality Bugs
3 types of bugs : Requirement & Specs, Feature, & feature interaction bugs
I. Requirements & Specs.

Incompleteness, ambiguous or self-contradictory

Analysts assumptions not known to the designer

Some thing may miss when specs change

These are expensive: introduced early in SDLC and removed at the last
II. Feature Bugs

Specification problems create feature bugs

Wrong feature bug has design implications

Missing feature is easy to detect & correct

Gratuitous enhancements can accumulate bugs, if they increase complexity

Removing features may foster bugs

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

III.

P a g e | 12

Feature Interaction Bugs

Arise due to unpredictable interactions between feature groups or individual features.

The earlier removed the better as these are costly if detected at the end.

Examples: call forwarding & call waiting.

Federal, state & local tax laws.

No magic remedy. Explicitly state & test important combinations

Remedies

Use high level formal specification languages to eliminate human-to-human

communication
Its only a short term support & not a long term solution
Short-term Support:
Specification languages formalize requirements & so automatic test generation
is possible. Its cost-effective.
Long-term support:
Even with a great specification language, problem is not eliminated, but is
shifted to a higher level. Simple ambiguities & contradictions may only be
removed, leaving tougher bugs.

Testing Techniques
Functional test techniques - transaction flow testing, syntax testing, domain testing,
logic testing, and state testing - can eliminate requirements & specifications bugs.
2) Structural Bugs
we look at the 5 types, their causes and remedies.
I. Control & Sequence bugs
II. Logic Bugs
III. Processing bugs
IV. Initialization bugs
V.
Data flow bugs & anomalies
I.
Control & Sequence Bugs:
Paths left out, unreachable code, spaghetti code, and pachinko code.
Improper nesting of loops, Incorrect loop-termination or look-back, ill-conceived
switches.
Missing process steps, duplicated or unnecessary processing, rampaging GOTOs.
Novice programmers.
Old code (assembly language & Cobol)
Prevention and Control:
Theoretical treatment and,
Unit, structural, path, & functional testing.
II.
Logic Bugs
Misunderstanding of the semantics of the control structures & logic operators
Improper layout of cases, including impossible & ignoring necessary cases,
Using a look-alike operator, improper simplification, confusing Ex-OR with inclusive
OR.
Deeply nested conditional statements & using many logical operations in 1 stmt.
Prevention and Control:
Logic testing, careful checks, functional testing
III.
Processing Bugs
Arithmetic, algebraic, mathematical function evaluation, algorithm selection & general.
processing, data type conversion, ignoring overflow, improper use of relational
operators.

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

IV.

V.

P a g e | 13

Prevention
Caught in Unit Testing & have only localized effect
Domain testing methods
Initialization Bugs
Forgetting to initialize work space, registers, or data areas.
Wrong initial value of a loop control parameter.
Accepting a parameter without a validation check.
Initialize to wrong data type or format.
Very common.
Remedies (prevention & correction)
Programming tools, Explicit declaration & type checking in source language,
preprocessors.
Data flow test methods help design of tests and debugging.
Dataflow Bugs & Anomalies
Run into an un-initialized variable.
Not storing modified data.
Re-initialization without an intermediate use.
Detected mainly by execution (testing).
Remedies (prevention & correction)
Data flow testing methods & matrix based testing methods

3. Data Bugs
Depend on the types of data or the representation of data. There are 4 sub categories.
I. Generic Data Bugs
II. Dynamic Data Vs Static Data
III. Information, Parameter, and Control Bugs
IV. Contents, Structure & Attributes related Bugs
I. Generic Data Bugs

Due to data object specs., formats, # of objects & their initial values.

Common as much as in code, especially as the code migrates to data.

Data bug introduces an operative statement bug & is harder to find.

Generalized components with reusability when customized from a large parametric data to
specific installation.
Remedies (prevention & correction):
Using control tables in lieu of code facilitates software to handle many transaction types with
fewer data bugs. Control tables have a hidden programming language in the database.
Caution - theres no compiler for the hidden control language in data tables
II. Dynamic Data Vs Static Data
Dynamic Data Bugs
Transitory. Difficult to catch.

Static Data Bugs

Fixed in form & content.

Due to an error in a shared storage object initialization. Appear in source code or data base, directly or
indirectly
Due to unclean / leftover garbage in a shared resource. Software to produce object code creates a static data
table bugs possible
Examples
Examples
Generic & shared variable

Shared data structure

Telecom system software: generic parameters, a

generic large program & site adapter program to set
parameter values, build data declarations etc.
Postprocessor : to install software packages. Data is

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 14

initialized at run time with configuration handled by

tables.
Prevention
Prevention
Data validation, unit testing
Compile time processing
Source language features
III. Information, Parameter, and Control Bugs
Static or dynamic data can serve in any of the three forms. It is a matter of perspective.
What is information can be a data parameter or control data else where in a program.
Examples: name, hash code, function using these.
A variable in different contexts.
Information: dynamic, local to a single transaction or task.
Parameter: parameters passed to a call.
Control: data used in a control structure for a decision.
Bugs
Usually simple bugs and easy to catch.
When a subroutine (with good data validation code) is modified, forgetting to update the data
validation code, results in these bugs.
Preventive Measures (prevention & correction)
Proper Data validation code.
IV . Contents, Structure & Attributes related Bugs
Contents: are pure bit pattern & bugs are due to misinterpretation or corruption of it.
Structure: Size, shape & alignment of data object in memory. A structure may have
substructures.
Attributes: Semantics associated with the contents (e.g. integer, string, subroutine).
Bugs
Severity & subtlety increases from contents to attributes as they get less formal.
Structural bugs may be due to wrong declaration or when same contents are interpreted
by multiple structures differently (different mapping).
Attribute bugs are due to misinterpretation of data type, probably at an interface
Preventive Measures (prevention & correction)
Good source lang. documentation & coding style (incl. data dictionary).
Data structures be globally administered. Local data migrates to global.
Strongly typed languages prevent mixed manipulation of data.
In an assembly lang. program, use field-access macros & not directly accessing any
field.
4. Coding Bugs
Coding errors create other kinds of bugs.
Syntax errors are removed when compiler checks syntax.
Coding errors
typographical, misunderstanding of operators or statements or could be just arbitrary.
Documentation Bugs
Erroneous comments could lead to incorrect maintenance.
Testing techniques cannot eliminate documentation bugs.
Solution:
Inspections, QA, automated data dictionaries & specification systems.

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 15

5. Interface, Integration and Systems Bugs

There are 9 types of bugs of this type.
1. External Interfaces
2. Internal Interfaces
3. Hardware Architecture Bugs
4. Operating System Bugs
5. Software architecture bugs
6. Control & Sequence bugs
7. Resource management bugs
8. Integration bugs
9. System bugs
a) External Interfaces
Means to communicate with the world:
drivers, sensors, input terminals,
communication lines.
Primary design criterion should be - robustness.
Bugs: invalid timing, sequence assumptions related to external signals, misunderstanding external formats
and no robust coding.
Domain testing, syntax testing & state testing are suited to testing external interfaces.
b) Internal Interfaces
Must adapt to the external interface.
Have bugs similar to external interface
Bugs from improper
Protocol design, input-output formats, protection against corrupted data, subroutine call sequence, callparameters.
Remedies (prevention & correction):
Test methods of domain testing & syntax testing.
Good design & standards: good trade off between # of internal interfaces & complexity of the interface.
Good integration testing is to test all internal interfaces with external world.
c) Hardware Architecture Bugs:
A s/w programmer may not see the h/w layer / architecture.
S/w bugs originating from hardware architecture are due to misunderstanding of how h/w works.
Bugs are due to errors in:
Paging mechanism, address generation
I/O device instructions, device status code, device protocol
Expecting a device to respond too quickly, or to wait for too long for response, assuming a device
is initialized, interrupt handling, I/O device address
H/W simultaneity assumption, H/W race condition ignored, device data format error etc..
Remedies (prevention & correction):
o Good software programming & Testing.
o Centralization of H/W interface software.
o Nowadays hardware has special test modes & test instructions to test the H/W function.
o An elaborate H/W simulator may also be used.
d) Operating System Bugs:
Due to:
Misunderstanding of H/W architecture & interface by the O. S.
Not handling of all H/W issues by the O. S.
Bugs in O. S. itself and some corrections may leave quirks.
Bugs & limitations in O. S. may be buried some where in the documentation.
Remedies (prevention & correction):
Same as those for H/W bugs.

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 16

Use O. S. system interface specialists

Use explicit interface modules or macros for all O.S. calls.
The above may localize bugs and make testing simpler.
e) Software Architecture Bugs: (called Interactive)
The subroutines pass thru unit and integration tests without detection of these bugs. Depend on the
Load, when the system is stressed. These are the most difficult to find and correct.
Due to:
Assumption that there are no interrupts, Or, Failure to block or unblock an interrupt.
Assumption that code is re-entrant or not re-entrant.
Bypassing data interlocks, Or, Failure to open an interlock.
Assumption that a called routine is memory resident or not.
Assumption that the registers and the memory are initialized, Or, that their content did not
change.
Local setting of global parameters & Global setting of local parameters.
Remedies:
Good design for software architecture.
Test Techniques
All test techniques are useful in detecting these bugs, Stress tests in particular.
f) Control & Sequence Bugs:
Due to:
Ignored timing
Assumption that events occur in a specified sequence.
Starting a process before its prerequisites are met.
Waiting for an impossible combination of prerequisites.
Not recognizing when prerequisites are met.
Specifying wrong priority, Program state or processing level.
Missing, wrong, redundant, or superfluous process steps.
Remedies:
Good design.
highly structured sequence control - useful
Specialized internal sequence-control mechanisms such as an internal job control language useful.
Storage of Sequence steps & prerequisites in a table and interpretive processing by control
processor or dispatcher - easier to test & to correct bugs.
Test Techniques
Path testing as applied to Transaction Flow graphs is effective.
g) Resource Management Problems:
Resources: Internal: Memory buffers, queue blocks etc. External: discs etc.
Due to:
Wrong resource used (when several resources have similar structure or different kinds of resources
in the same pool).
Resource already in use, or deadlock
Resource not returned to the right pool, Failure to return a resource. Resource use forbidden to the
caller.
Remedies:
Design: keeping resource structure simple with fewest kinds of resources, fewest pools, and no
private resource mgmt.
Designing a complicated resource structure to handle all kinds of transactions to save memory is not
right.
Centralize management of all resource pools thru managers, subroutines, macros etc.
Test Techniques

Path testing, transaction flow testing, data-flow testing & stress testing.

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 17

h) Integration Bugs:
Are detected late in the SDLC and cause several components and hence are very costly.
Due to:
Inconsistencies or incompatibilities between components.
Error in a method used to directly or indirectly transfer data between components. Some
communication methods are: data structures, call sequences, registers, semaphores,
communication links, protocols etc..
Remedies:
Employ good integration strategies.
Test Techniques
Those aimed at interfaces, domain testing, syntax testing, and data flow testing when applied
across components.
i) System Bugs:
Infrequent, but are costly
Due to:
Bugs not ascribed to a particular component, but result from the totality of interactions among
many components such as:
programs, data, hardware, & the O.S.
Remedies:
Thorough testing at all levels and the test techniques mentioned below
Test Techniques
Transaction-flow testing.
All kinds of tests at all levels as well as integration tests - are useful.
6. Testing & Test Design Bugs
Bugs in Testing (scripts or process) are not software bugs.
Its difficult & takes time to identify if a bug is from the software or from the test script/procedure.
1) Bugs could be due to:
Tests require code that uses complicated scenarios & databases, to be executed.
Though an independent functional testing provides an un-biased point of view, this lack of bias may lead to
an incorrect interpretation of the specs.
Test Criteria
Testing process is correct, but the criterion for judging softwares response to tests is incorrect or
impossible.
If a criterion is quantitative (throughput or processing time), the measurement test can perturb the
actual value.
Remedies:
i. Test Debugging:
Testing & Debugging tests, test scripts etc. Simpler when tests have localized affect.
ii. Test Quality Assurance:
To monitor quality in independent testing and test design.
iii. Test Execution Automation:
Test execution bugs are eliminated by test execution automation tools & not using manual testing.
iv. Test Design Automation:
Test design is automated like automation of software development.
For a given productivity rate, It reduces bug count.

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 18

Unit II
Flow graphs and Path testing
Basics: Path Testing Definition
A family of structural test techniques based on judiciously selecting a set of test paths
through the
programs.
Goal: Pick enough paths to assure that every source statement is executed at least once.
It is a measure of thoroughness of code coverage(During unit testing: # stmts executed at least once /
total # stmts).
It is used most for unit testing on new software.
Its effectiveness reduces as the software size increases.
We use Path testing techniques indirectly.
Path testing concepts are used in and along with other testing techniques

Assumptions

Software takes a different path than intended due to some error.

Specifications are correct and achievable.
Processing bugs are only in control flow statements
Data definition & access are correct

Observations
Structured programming languages need less of path testing.
Assembly language, Cobol, Fortran, Basic & similar languages make path testing necessary.

Control Flow Graph

A simplified, abstract, and graphical representation of a programs control structure using process
blocks, decisions and junctions.

Process Block:
A sequence of program statements uninterrupted by decisions or junctions with a single entry and
single exit.
Junction:
A point in the program where control flow can merge (into a node of the graph)
Examples: target of GOTO, Jump, Continue
Decisions:
A program point at which the control flow can diverge (based on evaluation of a condition).

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 19

Examples: IF stmt.
Conditional branch and Jump instruction.
Case Statements:
A Multi-way branch or decision.
Examples: In assembly language: jump addresses table, Multiple GOTOs, Case/Switch
For test design, Case statement and decision are similar.
Control Flow Graph Vs Flow Charts
Control Flow Graph

Flow Chart

Compact representation of the program

Usually a multi-page description

Focuses on Inputs, Outputs, and the control flow into

and out of the block.

Focuses on the process steps inside

Inside details of a process block are not shown

Every part of the process block are drawn

Creation of Control Flow Graph from a program

One statement to one element translation to get a Classical Flow chart

Add additional labels as needed
Merge process steps
A process box is implied on every junction and decision
Remove External Labels
Represent the contents of elements by numbers.
We have now Nodes and Links

One statement to one element translation to get a Classical Flow chart

INPUT X, Y
Z := X + Y
V := X - Y
IF Z >= 0 GOTO SAM
JOE: Z := Z + V
SAM: Z := Z - V
FOR N = 0 TO V
Z := Z - 1
NEXT N
END

Merge process steps

INPUT X, Y
Z := X + Y
V := X - Y
IF Z >= 0 GOTO SAM
JOE: Z := Z + V
SAM: Z := Z - V
FOR N = 0 TO V
Z := Z - 1
NEXT N
END

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 20

A process box is implied on every junction and decision, Remove External Labels, Represent the
contents of elements by numbers.
INPUT X, Y
Z := X + Y
V := X - Y
IF Z >= 0 GOTO SAM
JOE: Z := Z + V
SAM: Z := Z - V
FOR N = 0 TO V
Z := Z - 1
NEXT N
END

Linked List Notation of a Control Flow Graph

Node
Processing, label, Decision
Next-Node
1
(BEGIN; INPUT X, Y; Z := X+Y ; V := X-Y) : 2
2
( Z >= 0 ? )
: 4 (TRUE)
: 3 (FALSE)
3
(JOE: Z := Z + V)
: 4
4
(SAM: Z := Z V; N := 0)
: 5
5
(LOOP; Z := Z -1)
: 6
6
(N = V ?)
: 7 (FALSE)
: END (TRUE)
7
(N := N + 1)
:5

Path Testing Concepts

1. Path is a sequence of statements starting at an entry, junction or decision and ending at another, or possibly
the same junction or decision or an exit point.
Link is a single process (block) in between two nodes.
Node is a junction or decision.
Segment is a sequence of links. A path consists of many segments.
Path segment is a succession of consecutive links that belongs to the same path. (3,4,5)
Length of a path is measured by # of links in the path or # of nodes traversed.
Name of a path is the set of the names of the nodes along the path. (1,2,3 4,5, 6)
(1,2,3,4, 5,6,7, 5,6,7, 5,6)
Path-Testing Path is an entry to exit path through a processing block.
2. Entry / Exit for a routines, process blocks and nodes.
Single entry and single exit routines are preferable.
Called well-formed routines.
Formal basis for testing exists.
Tools could generate test cases.
Multi-entry / Multi-exit routines: (ill-formed)
A Weak approach: Hence, convert it to single-entry / single-exit routine.
Integration issues:
Large # of inter-process interfaces. Creates problem in Integration.
More # test cases and also a formal treatment is more difficult.
Theoretical and tools based issues
A good formal basis does not exist.
Tools may fail to generate important test cases.

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 21

Convert a multi-entry / exit routine to a single entry / exit routine:

Use an entry parameter and a case statement at the entry => single-entry

Merge all exits to Single-exit point after setting one exit parameter to a value.

Test Strategy for Multi-entry / exit routines

Get rid of them.
Control those you cannot get rid of.
Convert to single entry / exit routines.
Do unit testing by treating each entry/exit combination as if it were a completely different routine.
Recognize that integration testing is heavier
Understand the strategies & assumptions in the automatic test generators and confirm that they do (or
do not) work for multi-entry/multi exit routines.
3. Fundamental Path Selection Criteria
A minimal set of paths to be able to do complete testing.
Each pass through a routine from entry to exit, as one traces through it, is a potential path.
The above includes the tracing of 1..n times tracing of an interactive block each separately.
Note: A bug could make a mandatory path not executable or could create new paths not related to processing.
Complete Path Testing prescriptions:
1. Exercise every path from entry to exit.
2. Exercise every statement or instruction at least once.
3. Exercise every branch and case statement in each direction, at least once.
Point 1 => point 2 and 3.
Point 2 & 3 are not the same
Point 1 is impractical.
For a structured language, Point 3 => Point 2
Path Testing Criteria :
1. Path Testing (P ):
Execute all possible control flow paths thru the program; but typically restricted to entry-exit paths.
Implies 100% path coverage. Impossible to achieve.
2. Statement Testing ( P1) :
Execute all statements in the program at least once under the some test.
100% statement coverage => 100% node coverage.
Denoted by C1
C1 is a minimum testing requirement in the IEEE unit test standard: ANSI 87B.

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 22

3. Branch Testing (P2) :

Execute enough tests to assure that every branch alternative has been exercised at least once under
some test.
Denoted by C2
Objective: 100% branch coverage and 100% Link coverage.
For well structured software, branch testing & coverage include statement coverage
Picking enough (the fewest) paths for achieving C1+C2

1.
2.
3.
4.

Does every decision have Y & N (C2)?

Are call cases of case statement marked (C2)?
Is every three way branch covered (C2)?
Paths
Decisions Process-link
Is every link covered at least once (C1)?
2
5 a b c d e
Make small changes in the path changing only
1 link or node at a time.
abdeg
Y
Y Y Y
Y Y
Revised path selection Rules
acdeg
No
Y Y
Y Y Y
1. Pick the simplest and functionally sensible
entry/exit path
abdefeg Y
N Y Y
Y Y
2. Pick additional paths as small variations from
acdefeg No
Y Y
Y Y Y
previous paths. (pick those with no loops, shorter
paths, simple and meaningful)
3. Pick additional paths but without an obvious functional meaning (only to achieve C1+C2 coverage).
4. Be comfortable with the chosen paths. play hunches, use intuition to achieve C1+C2
5. Dont follow rules slavishly except for coverage
4. Testing of Paths involving loops
Bugs in iterative statements apparently are not discovered by C1+C2.
But by testing at the boundaries of loop variable.
Types of Iterative statements
1. Single loop statement.
2. Nested loops.
3. Concatenated Loops.
4. Horrible Loops
Let us denote the Minimum # of iterations by nmin
the Maximum # of iterations by nmax
the value of loop control variable by V
the #of test cases by T
the # of iterations carried out by n
Later, we analyze the Loop-Testing times

g
Y
Y

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 23

1. Testing a Single Loop Statement (three cases)

Case 1. nmin = 0, nmax = N, no excluded values
1. Bypass the loop.
If you cant, there is a bug, nmin 0 or a wrong case.
2. Could the value of loop (control) variable V be negative?
could it appear to specify a ve n ?
3. Try one pass through the loop statement: n = 1
4. Try two passes through the loop statement: n = 2
To detect initialization data flow anomalies:
Variable defined & not used in the loop, or
Initialized in the loop & used outside the loop.
5. Try n = typical number of iterations : nmin < n < nmax
6. Try n = nmax -1
7. Try n = nmax
8. Try n = nmax + 1.
What prevents V (& n) from having this value?
What happens if it is forced?
Case 2. nmin = +ve, nmax = N, no excluded values
1. Try nmin - 1
Could the value of loop (control) variable V be < nmin?
What prevents that ?
2. Try nmin
3. Try nmin + 1
4. Once, unless covered by a previous test.
5. Twice, unless covered by a previous test.
4. Try n = typical number of iterations : nmin < n < nmax
5. Try n = nmax -1
6. Try n = nmax
7. Try n = nmax + 1.
What prevents V (& n) from having this value?
What happens if it is forced?
Case 3. Single loop with excluded values
1. Treat this as single loops with excluded values as two sets.
2. Example:
V = 1 to 20 excluding 7,8,9 and10
Test cases to attempt are for:
V = 0,1,2,4,6,7
and V = 10,11,15,19,20,21
(underlined cased are not supposed to work)
2. Testing a Nested Loop Statement
Multiplying # of tests for each nested loop => very large # of tests
A test selection technique:
1. Start at the inner-most loop. Set all outer-loops to Min iteration parameter values: Vmin.
2. Test the Vmin, Vmin + 1, typical V, Vmax - 1, Vmax for the inner-most loop. Hold the outer-loops to
Vmin. Expand tests are required for out-of-range & excluded values.
3. If you have done with outer most loop, Go To step 5. Else, move out one loop and do step 2 with all
other loops set to typical values.
4. Do the five cases for all loops in the nest simultaneously.
Assignment: check # test cases = 12 for nesting = 2, 16 for 3, 19 for 4.

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 24

Compromise on # test cases for processing time.

Expand tests for solving potential problems associated with initialization of variables and with
excluded combinations and ranges.
Apply Huangs twice thorough theorem to catch data initialization problems.
3. Testing Concatenated Loop Statements
Two loops are concatenated if its possible to reach one after exiting the other while still on the path
from entrance to exit.
If these are independent of each other, treat them as independent loops.
If their iteration values are inter-dependent & these are same path, treat these like a nested loop.
Processing times are additive.
4. Testing Horrible Loops
Avoid these.
Even after applying some techniques of paths, resulting test cases not definitive.
Too many test cases.
Thinking required to check end points etc. is unique for each program.
Jumps in & out of loops and intersecting loops etc, makes test case selection an ugly task.
Loop Testing Times
Longer testing time for all loops if all the extreme cases are to be tested.
Unreasonably long test execution times indicate bugs in the s/w or specs.
Case: Testing nested loops with combination of extreme values leads to long test times.
Show that its due to incorrect specs and fix the specs.
Prove that combined extreme cases cannot occur in the real world. Cut-off those tests.
Put in limits and checks to prevent the combined extreme cases.
Test with the extreme-value combinations, but use different numbers.
The test time problem is solved by rescaling the test limit values.
Can be achieved through a separate compile, by patching, by setting parameter values etc..
Effectiveness of Path Testing
Path testing (with mainly P1 & P2) catches ~65% of Unit Test Bugs ie., ~35% of all bugs.
More effective for unstructured than structured software.
Limitations
Path testing may not do expected coverage if bugs occur.
Path testing may not reveal totally wrong or missing functions.
Unit-level path testing may not catch interface errors among routines.
Data base and data flow errors may not be caught.
Unit-level path testing cannot reveal bugs in a routine due to another.
Not all initialization errors are caught by path testing.
Specification errors cannot be caught.
A lot of work
Creating flow graph, selecting paths for coverage, finding input data values to force these
paths, setting up loop cases & combinations.
Careful, systematic, test design will catch as many bugs as the act of testing.
Test design process at all levels at least as effective at catching bugs as is running the test designed by
that process.
More complicated path testing techniques than P1 & P2
Between P2 & P
Complicated & impractical
Weaker than P1 or P2.
For regression (incremental) testing, its cost-effective

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 25

Predicates, Predicate Expressions

Path
A sequence of process links (& nodes)
Predicate
The logical function evaluated at a decision : True or False.
(Binary , boolean)
Compound Predicate
Two or more predicates combined with AND, OR etc.
Path Predicate
Every path corresponds to a succession of True/False values for the predicates traversed on that
path.
A predicate associated with a path.
X > 0 is True
AND
W is either negative or equal to 122 is True
Multi-valued Logic / Multi-way branching
Predicate Interpretation
The symbolic substitution of operations along the path in order to express the predicate solely in terms
of the input vector is called predicate interpretation.
An input vector is a set of inputs to a routine arranged as a one dimensional array.
Example:
INPUT X, Y
INPUT X
ON X GOTO A, B, C
IF X < 0 THEN Y:= 2
A: Z := 7 @ GOTO H
ELSE Y := 1
B: Z := -7 @ GOTO H
IF X + Y*Y > 0 THEN
C: Z := 0 @ GOTO H
H: DO SOMETHING
K: IF X + Z > 0 GOTO GOOD ELSE GOTO BETTER
Predicate interpretation may or may not depend on the path.
IF -7 > 3 ..
Path predicates are the specific form of the predicates of the decisions along the selected path after
interpretation.
Process Dependency
An input variable is independent of the processing if its value does not change as a result of
processing.
An input variable is process dependent if its value changes as a result of processing.
A predicate is process dependent if its truth value can change as a result of processing.
A predicate is process independent if its truth value does not change as a result of processing.
Process dependence of a predicate doesnt follow from process dependence of variables
Examples:
X + Y = 10 Increment X & Decrement Y.
X is odd
Add an even # to X
If all the input variables (on which a predicate is based) are process independent, then predicate is
process independent
Correlation
Two input variables are correlated if every combination of their values cannot be specified
independently.
Variables whose values can be specified independently without restriction are uncorrelated.
A pair of predicates whose outcomes depend on one or more variables in common are correlated
predicates.
Every path through a routine is achievable only if all predicates in that routine are uncorrelated.
If a routine has a loop, then at least one decisions predicate must be process dependent. Otherwise,
there is an input value which the routine loops indefinitely.
Path Predicate Expression

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 26

Every selected path leads to an associated boolean expression, called the path predicate expression,
which characterizes the input values (if any) that will cause that path to be traversed.
Select an entry/exit path. Write down un-interpreted predicates for the decisions along the path. If
there are iterations, note also the value of loop-control variable for that pass. Converting these into
predicates that contain only input variables, we get a set of boolean expressions called path predicate
expression.
Example (inputs being numerical values):
If X 5 > 0 .OR. X 6 < 0 then
X1 + 3X2 + 17 >= 0
X3 = 17
X4 X1 >= 14 X2
A:
X5>0
E: X 6 < 0
B:
X1 + 3X2 + 17 >= 0
F: X1 + 3X2 + 17 >= 0
C:
X3 = 17
G: X3 = 17
D:
X4 X1 >= 14 X2
H: X4 X1 >= 14 X2
Converting into the predicate expression form:
ABCD+EBCD (A+E)BCD
If we take the alternative path for the expression: D then
(A + E ) B C D
Predicate Coverage:
Look at examples & possibility of bugs:
ABCD
A+B+C+D
Due to semantics of the evaluation of logic expressions in the languages, the entire expression
may not be always evaluated.
A bug may not be detected.
A wrong path may be taken if there is a bug.
Realize that on our achieving C2, the program could still hide some control flow bugs.
Predicate coverage:
If all possible combinations of truth values corresponding to selected path have been explored
under some test, we say predicate coverage has been achieved.
Stronger than branch coverage.
If all possible combinations of all predicates under all interpretations are covered, we have the
equivalent of total path testing.
Testing blindness
coming to the right path even thru a wrong decision (at a predicate). Due to the
interaction of some statements makes a buggy predicate work, and
the bug is not detected by the selected input values.
calculating wrong number of tests at a predicate
by ignoring the # of paths to arrive at it.

Cannot be detected by path testing and need other strategies

Assignment blinding: A buggy Predicate seems to work correctly as the specific value chosen in an
assignment statement works with both the correct & buggy predicate.
Correct
Buggy
X := 7
X := 7
IF Y > 0 THEN
IF X + Y > 0 THEN
(check for Y=1)
Equality blinding:
When the path selected by a prior predicate results in a value that works both for the correct &
buggy predicate.
Correct
Buggy
IF Y = 2 THEN
IF Y = 2 THEN ..
IF X + Y > 3 THEN
IF X > 1 THEN
(check for any X>1)

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 27

Self-blinding
When a buggy predicate is a multiple of the correct one and the result is indistinguishable
along that path.
Correct
Buggy
X := A
X := A
IF X - 1 > 0 THEN
IF X + A -2 > 0 THEN
(check for any X,A)

Achievable Paths
1. Objective is to select & test just enough paths to achieve a satisfactory notion of test completeness such as C1
+ C2.
2. Extract the programs control flow graph & select a set of tentative covering paths.
3. For a path in that set, interpret the predicates.
4. Trace the path through, multiplying the individual compound predicates to achieve a boolean expression.
Example: (A + BC) ( D + E)
5. Multiply & obtain sum-of-products form of the path predicate expression:
AD + AE + BCD + BCE
6. Each product term denotes a set of inequalities that, if solved, will yield an input vector that will drive the
routine along the selected path.
7. A set of input values for that path is found when any of the inequality sets is solved.
A solution found => path is achievable. Otherwise the path is unachievable.

Path Sensitization
Its the act of finding a set of solutions to the path predicate expression.
In practice, for a selected path finding the required input vector is not difficult. If there is difficulty, it may be
due to some bugs.
Heuristic procedures:
Choose an easily sensitizable path set, & pick hard-to-sensitize paths to achieve more coverage.
Identify all the variables that affect the decisions. For process dependent variables, express the nature of the
process dependency as an equation, function, or whatever is convenient and clear. For correlated variables,
express the logical, arithmetic, or functional relation defining the correlation.
1. Identify correlated predicates and document the nature of the correlation as for variables. If the same predicate
appears at more than one decision, the decisions are obviously correlated.
2. Start path selection with uncorrelated & independent predicates. If coverage is achieved, but the path had
dependent predicates, something is wrong.
3. If the coverage is not achieved yet with independent uncorrelated predicates, extend the path set by using
correlated predicates; preferably process independent (not needing interpretation)
4. If the coverage is not achieved, extend the path set by using dependent predicates (typically required to cover
loops), preferably uncorrelated.
5. Last, use correlated and dependent predicates.
6. For each of the path selected above, list the corresponding input variables. If the variable is independent, list
its value. For dependent variables, interpret the predicate ie., list the relation. For correlated variables, state the
nature of the correlation to other variables. Determine the mechanism (relation) to express the forbidden
combinations of variable values, if any.
7. Each selected path yields a set of inequalities, which must be simultaneously satisfied to force the path.

Examples for Path Sensitization

1.
2.
3.
4.

Simple Independent Uncorrelated Predicates

Independent Correlated Predicates
Dependent Predicates
Generic

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 28

1. Simple Independent Uncorrelated Predicates

4 predicates => 16 combinations

Set of possible paths = 8
Path
Predicate Values
abcdef
A
C
aghcimkf
A B C D
aglmjef
A B
D
abcdef
A
C
abcimjef
A
C D
abcimkf
A
C D
aghcdef
A B C
aglmkf
A B D
A Simple case of solving inequalities.
2. Correlated Independent Predicates

(obtained by the procedure for finding a covering set of paths)

Correlated paths => some paths are unachievable

ie., redundant paths.
ie., n decisions but # paths are < 2n
Due to practice of saving code which makes
the code very difficult to maintain.
Eliminate the correlated decisions.Reproduce common code.

If a chosen sensible path is not achievable,

- theres a bug.
- design can be simplified.
- get better understanding of correlated decisions

3. Dependent Predicates
Usually most of the processing does not affect the control flow.
Use computer simulation for sensitization in a simplified way.
Dependent predicates contain iterative loop statements usually.
For Loop statements:
Determine the value of loop control variable for a certain # of iterations, and then
work backward to determine the value of input variables (input vector).
4. The General Case
No simple procedure to solve for values of input vector for a selected path.
1. Select cases to provide coverage on the basis of functionally sensible paths.
Well structured routines allow easy sensitization.
Intractable paths may have a bug.
2. Tackle the path with the fewest decisions first. Select paths with least # of loops
3. Start at the end of the path and list the predicates while tracing the path in reverse.
Each predicate imposes restrictions on the subsequent (in reverse order) predicate.
4. Continue tracing along the path. Pick the broadest range of values for variables affected
and consistent with values that were so far determined.
5. Continue until the entrance & therefore have established a set of input conditions for the path.

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 29

If the solution is not found, path is not achievable, it could be a bug.

Alternately:
1. In the forward direction, list the decisions to be traversed.
For each decision list the broadest range of input values.
2. Pick a path & adjust all input values. These restricted values are used for next decision.
3. Continue. Some decisions may be dependent on and/or correlated with earlier ones.
4. The path is unachievable if the input values become contradictory, or, impossible.
If the path is achieved, try a new path for additional coverage.
Advantages & Disadvantages of the two approaches:
The forward method is usually less work.
you do not know where you are going as you are tracing the graph.

Path Instrumentation
Output of a test:
Results observed. But, there may not be any expected output for a test.
Outcome:
Any change or the lack of change at the output.
Expected Outcome:
Any expected change or the lack of change at the output (predicted as part of design).
Actual Outcome:
Observed outcome
Coincidental Correctness:
When expected & actual outcomes match,
Necessary conditions for test to pass are met.
Conditions met are probably not sufficient.
(the expected outcome may be achieved due to a wrong reason)

Path Instrumentation is what we have to do confirm that the outcome was achieved by the intended path.
Path Instrumentation Methods
1. General strategy:
1. Based on Interpretive tracing & use interpreting trace program.
2. A trace confirms the expected outcome is or isnt obtained along the intended path.
3. Computer trace may be too massive. Hand tracing may be simpler.
2. Traversal or Link Makers:
Simple and effective
1. Name every link.
2. Instrument the links so that the link is recorded when it is executed (during the test)
3. The succession of letters from a routines entry to exit corresponds to the pathname

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 30

Single Link Marker Instrumentation: An example

Sample path:
ijn
Single Link Marker Instrumentation: Not good enough

Problem:
Processing in the links may be chewed open by bugs. Possibly due to GOTO statements, control takes a
different path, yet resulting in the intended path again.
Double Link Marker Instrumentation:

The problem is solved.

Two link markers specify the path name and both the beginning & end of the link.
3. Link Counters Technique:
Less disruptive and less informative.
Increment a link counter each time a link is traversed. Path length could confirm the intended path.
For avoiding the same problem as with markers, use double link counters.
Expect an even count = double the length.
Now, put a link counter on every link.
(earlier it was only between decisions)

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 31

If there are no loops, the link counts are = 1.

Sum the link counts over a series of tests, say, a covering set. Confirm the total link counts with the
expected.
Using double link counters avoids the same & earlier mentioned problem.
Check list for the procedure:
Do begin-link counter values equal the end-link counter values?
Does the input-link count of every decision equal to the sum of the link counts of the output links from
that decision?
Do the sum of the input-link counts for a junction equal the output-link count for that junction?
Do the total counts match the values you predicted when you designed the covering test set?
This procedure and the checklist could solve the problem of Instrumentation.
Limitations
Instrumentation probe (marker, counter) may disturb the timing relations & hide racing condition
bugs.
Instrumentation probe (marker, counter) may not detect location dependent bugs.
If the presence or absence of probes modifies things (for example in the data base) in a faulty way,
then the probes hide the bug in the program.
Path Intrumentation Implementation
For Unit testing : Implementation may be provided by a comprehensive test tool.
For higher level testing or for testing an unsupported language:
Introduction of probes could introduce bugs.
Instrumentation is more important for higher level of program structure like transaction flow
At higher levels, the discrepancies in the structure are more possible & overhead of instrumentation
may be less.
For Languages supporting conditional assembly or compilation:
Probes are written in the source code & tagged into categories.
Counters & traversal markers can be implemented.
Can selectively activate the desired probes.
For language not supporting conditional assembly / compilation:
Use macros or function calls for each category of probes. This may have less bugs.
A general purpose routine may be written.
In general:
Plan instrumentation with probes in levels of increasing detail.

Implementation & Application of Path Testing

1. Integration, Coverage, and Paths in Called Components
1. Mainly used in Unit testing, especially new software.
2. In an Idealistic bottom-up integration test process integrating one component at a time. Use
stubs for lower level component (sub-routines), test interfaces and then replace stubs by real
subroutines.
3. In reality, integration proceeds in associated blocks of components. Stubs may be avoided. Need to
think about paths inside the subroutine.
To achieve C1 or C2 coverage:
Predicate interpretation may require us to treat a subroutine as an in-line-code.
Sensitization becomes more difficult.
Selected path may be unachievable as the called components processing may block it.

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 32

Weaknesses of Path testing:

It assumes that effective testing can be done one level at a time without bothering what
happens at lower levels.
predicate coverage problems & blinding.
2. Application of path testing to New Code
Do Path Tests for C1 + C2 coverage
Use the procedure similar to the idealistic bottom-up integration testing, using a mechanized test suite.
A path blocked or not achievable could mean a bug.
When a bug occurs the path may be blocked.
3. Application of path testing to Maintenance
Path testing is applied first to the modified component.
Use the procedure similar to the idealistic bottom-up integration testing, but without using stubs.
Select paths to achieve C2 over the changed code.
Newer and more effective strategies could emerge to provide coverage in maintenance phase.
4. Application of path testing to Rehosting
Path testing with C1 + C2 coverage is a powerful tool for rehosting old software.
Software is rehosted as its no more cost effective to support the application environment.
Use path testing in conjunction with automatic or semiautomatic structural test generators.

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 33

Unit III
Transaction Flow Testing
Transaction-flow
Transaction-flow represents a systems processing. Functional testing methods are applied for testing T-F.
Transaction-flow Graph
TFG represents a behavioral (functional) model of the program (system) used for functional testing by an
independent system tester.
Transaction
is a unit of work seen from systems user point of view.
consists of a sequence of operations performed by a system, persons or external devices.
is created (birth) due to an external act & up on its completion (closure), it remains in the form of
historical records.
Example: the sequence of steps in a transaction in an online information retrieval system
1. accept inputs
7. Accept Inputs
2. Validate inputs (Birth of tr.)
8. Validate inputs
3. Transmit ack. to the user
9. Process the request
4. Process input
10. Update file
5. Search file
11. Transmit output
6. Request direction from user
12. Record transaction in log & cleanup (Closure)
Users View of a transaction
: Single step
Systems view
: Sequence of many operations

User (terminal)

Terminal controller

CPU

Example:

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 34

Definitions
Transaction-flow Graph : a scenario between users & computer
Transaction-flow
: an internal sequence of events in processing a transaction
Uses of Transaction-flow
Specifying requirements of big, online and complicated systems.
Airline reservation systems, air-traffic control systems.
Loops are less as compared to CFG. Loops are used for user input error processing

Implementation of Transaction-Flow (in a system)

Implicit in the design of systems control structure & associated database.
No direct one-to-one correspondence between the processes and decisions of transaction-flow, and the
corresponding program component.
A transaction-flow is a path taken by the transaction through a succession of processing modules.
A transaction is represented by a token.
A transaction-flow graph is a pictorial representation of what happens to the tokens.

System Control Structure (architecture of the implementation) :

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 35

System control structure

System is controlled by a scheduler
A Transaction is created by filling in a Transaction Control Block (TCB) by user inputs and by placing that
token on input Q of Scheduler.
Scheduler examines and places it on appropriate process Q such as A. When A finishes with the Token, it
places the TCB back on the scheduler Q.
Scheduler routes it to the next process after examining the token :
1. It contains tables or code to route a token to the next process.
2. It may contain routing information only in tables.
3. Scheduler contains no code / data. Processing modules contain code for routing.
Transaction Processing System (simplified):
There are many Tr. & Tr-flows in the system.
Scheduler invokes processes A to E as well as disk & tape read & writes.
The order of execution depends on priority & other reasons.
Cyclic structure like in this example is common in process control & communication systems.
The criteria for implementation mechanism depends on performance and resource optimization.

A perspective of Transaction-Flow
Transaction-flow testing is a block box technique. (as we assumed nothing regarding computer, communications,
environment, O.S., transaction identity or structure or state.)
1. TFG is a kind of DFG.
TFG has tokens, & DFG has data objects with history of operations applied on them.
Many techniques of CFG apply to TFG & DFG
2. Decision nodes of TFG have exception exits to the central recovery process.
3. So, we ignore the effect of interrupts in a Transaction-flow.

Transaction Flows splitting & merging decisions

Splits of transactions (Births)
1. A decision point in TFG

2. Biosis

3. Mitosis

Mergers of transactions
1. Junction

2. Absorption

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 36

3. Conjugation

TFG splitting, merging Transactions

Multiple meanings now for decision and junction symbols in a TFG.
Simple TFG model is not enough to represent multi-processor systems & associated coordination systems.
Petrinet model uses operations for all the above. But Petrinets are applied to H/W, N/W protocol testing
but not Software.
Simplified TFG model
Add New Tr-Flows for Biosis, Mitosis, Absorption, Conjugation
Problems for programmer, designer & test designer.
Need to design specific tests possibility of bugs.

Transaction-flow Structure
Reasons for Unstructuredness
1. Processes involve Human Users
2. Part of Flow from External Systems
3. Errors, Failures, Malfunctions & Recovery Actions
4. Transaction Count, Complexity. Customer & Environment
5. New Transactions, & Modifications
6. Approximation to Reality
7. Attempt to Structure

Transaction - Flow Testing - Steps

First, Build / Obtain Transaction Flows
Represent Explicitly
Design details the Main Tr-Flows
Create From PDL
HIPO charts & Petrinet Representations
Objective Trace the transaction
1. Inspections, Reviews & Walkthroughs
Start From Preliminary Design
1. Conducting Walkthroughs
Discuss enough Transaction Types (98% Transactions)
User needs & Functional terms (Design independent)
Traceability to Requirements
2. Design Tests for C1 + C2 coverage
3. Additional Coverage
(> C1+C2)
Paths with loops, extreme values, domain boundaries
Weird cases, long & potentially troublesome Tr.
4. Design Test cases for Tr. Splits & mergers
5. Publish Selected Test Paths early
6. Buyers Acceptance functional & acceptance tests
2. Path Selection
Covering Set (C1+C2) of Functionally Sensible Tr.
Add Difficult Paths
Review with designers & implementers
Exposure of interface problems & duplicated processing
Very few Implementation bugs may remain
Transaction-flow Path Covering Set belongs in System Feature Tests.

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 37

3. Sensitization
Functionally Sensible Paths Simple
Error, Exception, External Protocol Interface Paths - Difficult
Testing Tr.Flows with External Interfaces
Use patches & break points, mistune, and break the rules,
4. Instrumentation
Link Counters are not Useful.
Need
Trace
Queues on which Tokens resided
Entries to & Exits from Dispatcher
A Running Log
Make Instrumentation as part of System Design
5. Test Data bases
Design & Maintenance of a Test Data base - Effort
Mistakes
Unawareness about design of a centrally administered test DB
Test DB design by Testers
Using one DB for all tests (need 4 to 5)
Need experienced System & Test Designers
6. Test Execution
Use Test Execution Automation
Have to do a large # of Tests for C1+C2 coverage

Data - Flow Testing - Basics

Anomaly
Unreasonable processing on data
Use of data object before it is defined
Defined data object is not used
Data Flow Testing (DFT) uses Control Flow Graph (CFG) to explore dataflow anomalies.
DFT Leads to testing strategies between P and P1 / P2
Definition
DFT is a family of test strategies based on selecting paths through the programs control flow in order to
explore the sequence of events related to the status of data objects.
Example
Pick enough paths to assure that every data item has been initialized prior to its use, or that all objects have
been used for something.

Motivation

Confidence in the program

Data dominated design.
Code migrates to data..
Source Code for Data Declarations
Data flow Machines vs Von Neumanns
Abstract M I M D
Language & compiler take care of parallel computations
Program Control flow with Von Neumanns paradigm
Given m, n, p, q, find e.
e = (m+n+p+q) * (m+n-p-q)
a := m + n
b := p + q
c := a + b
d := a - b

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 38

e := c * d
Multiple representations of control flow graphs possible.
Program Flow using Data Flow Machines paradigm
BEGIN
PAR DO
READ m, n, n, p, q
END PAR
PAR DO
a := m+n
b := p+q
END PAR
PAR DO
c := a+b
d := a-b
END PAR
PAR DO
e := c * d
END PAR
END
Control flow graph
Multiple representations
Data Flow Graph
A spec. for relations among the data objects. Covering DFG => Explore all relations under some test.
Assumptions
Problems in a control flow
Problems with data objects
Data Flow Graphs (DFG)
It is a graph with nodes & directed links
Test the Von Neumann way Convert to a CFG
Annotate : program actions (weights)
Data Object State & Usage
Program Actions (d, k, u):
Defined (created)
explicitly or implicitly
(d)
Killed (released)
directly or indirectly
(k)
Used
(u)
o In a calculation
(c)
o In a predicate
directly or indirectly (p)
Data Flow Anomalies
A Two letter sequence of Actions (d, k, u)
dd

harmless, suspicious

dk
du

:
:

probably a bug.
normal

kd
kk
ku

:
:
:

normal
harmless, but probably a bug
a bug

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

ud
uk
uu
Actions on
-d
-u
-k

P a g e | 39

:
normal.
Redefinition.
:
normal
:
normal
data objects
no action from START to this point
From this point till the EXIT
normal
anomaly
anomaly

k-

normal

u-

normal

possibly an anomaly

dpossibly anomalous
Data Flow Anomaly State graph
Data Object State
K, D, U, A
Processing Step
k, d, u
Data Flow Anomaly State graph
Object state
Unforgiving Data
flow state graph

Data Flow Anomaly State graph

Forgiving Data flow state graph

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 40

Data Flow State Graphs

Differ in processing of anomalies
Choice depends on Application, language, context
Static vs Dynamic Anomaly Detection
Static analysis of data flows
Dynamic analysis
Intermediate data values
Insufficiency of Static Analysis (for Data flow)
1. Validation of Dead Variables
2. Validation of pointers in Arrays
3. Validation of pointers for Records & pointers
4. Dynamic addresses for dynamic subroutine calls
5. Identifying False anomaly on an unachievable path
6. Recoverable anomalies & Alternate state graph
7. Concurrency, Interrupts, System Issues

Data Flow Model

Based on CFG
CFG annotated with program actions
link weights : dk, dp, du etc..
Not same as DFG
For each variable and data object
Procedure to Build
Entry & Exit nodes
Unique node identification
Weights on out link
Predicated nodes
Sequence of links
Join
Concatenate weights
The converse
Example:
an 1
Z = b + --------START
a - 1
INPUT a, b, n
Z := 0
IF a = 1 THEN Z := 1
GOTO DONE1
r := 1 c := 1
POWER:
c := c * a
r := r + 1
IF r <= n THEN GO TO POWER
Z := (c 1) / (a 1)
DONE1:
Z := b + Z
END

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 41

CFG annotated Data Flow Model for Z

CFG annotated Data Flow Model for c

CFG annotated Data Flow Model for r

CFG annotated Data Flow Model for b

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 42

CFG annotated Data Flow Model for n

CFG annotated Data Flow Model for a

Definitions
w.r.t. a variable or data object v
Assume all DF paths are achievable
1. Definition-clear path segment
no k, kd
Loop-free path segment
Simple path segment
du path from node i to k
definition-clear & simple
c
definition-clear & loop-free
p
DFT Strategies
All-du paths (ADUP)
All uses (AU) strategy
All p-uses/some c-uses and All c-uses/some p-uses
All Definitions Strategy
All p-uses, All c-uses Strategy
Purpose
Test Design, Develop Test Cases
1. All-du paths (ADUP)
Strongest DFT
Every du path for every variable for every definition to every use
2. All uses (AU) strategy
At least one definition clear path segment from every definition of every variable to every use of that
definition be exercised under some test.
At least one path segment from every definition to every use that can be reached from that definition.

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 43

3. All p-uses/some c-uses and All c-uses/some p-uses

APU + c
Stronger than P2
ACU + p
Weaker than P2
4. All Definitions Strategy (AD)
Cover every definition by at least one p or c
Weaker than ACU + p and APU + c
5. All-Predicate Uses,
All-Computational Uses Strategy
APU :
Include definition-free path for every definition of every variable from the definition to predicate use.
ACU :
Include for every definition of every variable include at least one definition-free path from the
definition to every computational use.
Ordering the strategies

Testing, Maintenance & Debugging in the Data Flow context

Slicing:
A static program slice is a part of a program defined wrt a variable v and a statement s; It is the set
of all statements that could affect the value of v at stmt s.
Stmt1
var v
stmt2
Stmt3
var v
Stmt4
var v
Stmt s var v
Dicing:
A program dice is a part of slice in which all stmts. which are known to be correct have been removed.
Obtained from slice by incorporating correctness information from testing / debugging.
Debugging:
Select a slice.
Narrow it to a dice.
Refine the dice till its one faulty stmt.
Dynamic Slicing:
Refinement of static slicing
Only achievable paths to the stmt s in question are included.

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 44

Slicing methods bring together testing, maintenance & debugging.

Application of DFT
Comparison Random Testing, P2, AU - by Ntafos
AU detects more bugs than
P2 with more test cases
RT with less # of test cases
Comparison of P2, AU - by Sneed
AU detects more bugs with 90% Data Coverage Requirement.
Comparison of # test cases for ACU, APU, AU & ADUP by Weyuker using ASSET testing system
Test Cases Normalized. t = a + b * d
d = # binary decisions
At most d+1 Test Cases for P2
loop-free
# Test Cases / Decision
ADUP > AU > APU > ACU > revised-APU
Comparison of # test cases for ACU, APU, AU & ADUP by Shimeall & Levenson
Test Cases Normalized. t = a + b * d
(d = # binary decisions)
At most d+1 Test Cases for P2
loop-free
# Test Cases / Decision

ADUP ~ APU*

AP ~ AC
DFT vs P1, P2
DFT is Effective
Effort for Covering Path Set ~ Same
DFT Tracks the Coverage of Variables
Test Design is similar
DFT

TOOLS
Cost-effective development
Commercial tools :
Can possibly do Better than Commercial Tools
Easier Integration into a Compiler
Efficient Testing

Questions from Previous Papers

1. How is data flow testing (DFT) helpful in fulfilling gaps in path testing?
2. Explain the data flow Graphs (DFG).
3. How can anomaly be detected? Explain different types of data flow anomalies and Data flow Anomaly
State Graphs.
4. Write applications of Data Flow Testing.
5. Name and explain Data flow testing strategies.

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 45

Unit - IV
Domain Testing
Domains and Paths:
Domain Testing
Views Programs as input data classifiers
Verifies if the classification is correct
Domain Testing Model

Two Views for Domain Model

Based on Specs
Functional Testing
Based on Implementation information
Structural Technique
Domain Testing needs
Variables
Simple combinations of two variables
Numbers from - to +
Input variables as numbers
Loop-free Programs
Structural Knowledge is not needed. Only Specs.
For each Input Case,
A Hypothetical path for Functional testing
An Actual path for Structural testing

Domain
A Single Connected Set of numbers
No arbitrary discrete sets
Defined by the boundaries
One or more boundaries
Specified by predicates
Bugs likely at the boundaries
One or more variables
Predicates
Interpretation
Structural Testing - CFG
Functional Testing - DFG
Specifies the Domain Boundary
n Predicates in Sequence 2n domains
Or, just two domains

A .AND. B .AND. C

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 46

Paths
At least One PATH thru the Program
More paths for disconnected domains
Points to remember
Domain for a loop-free program corresponds to a set of numbers defined over the input vector
For every domain, there is at least one path thru the routine, along which that domains processing is done
The set of interpreted predicates traversed on that path (ie., the paths predicate expression) defines the
domains boundaries.
Domain Closure
Boundary: Closed / Open

Domain Dimensionality
One dimension per variable
At least one predicate
Slices thru previously defined Domain
Slicing Boundary
N-spaces are cut by Hyperplanes
Bug Assumptions
Processing is OK. Domain definition may be wrong.
Boundaries are wrong.
Predicates are wrong.
Once input vector is set on the right path, its correctly processed.
More bugs causing domain errors
Double-zero representation
Ambiguous domains
Floating point zero check
Contradictory domains

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 47

Over-specified domains
X > 3 .AND. X < 2 .AND. Y > 3
Boundary errors
Boundary closure
Shifted
Boundary errors.
Tilted
Missing
Extra boundary
Closure Reversal

Faulty Logic
Simplification of compound predicates

Restrictions
1. Coincidental Correctness
DT cannot detect
Example
Representative outcome
Partition testing
Input Equivalence
2. Simple boundaries & Compound predicates

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 48

Compound predicates
Impact of .OR.
Concave, Disconnected
Adjacent domains with same function
Example
ABC + DEF
Eliminate compound predicates
3. Functional Homogeneity of Bugs
Functional form still Retained
a x + b y >= c
Bugs are only in a, b, c
4. Linear Vector Space
Linear boundary predicate, Interpreted
Simple relational operators
Conversion to linear vector space
2-d Polar co-ordinates
Polynomials
Problems with Non-linear Boundaries
5. Loop-free Software
Predicate for each iteration
Loop over the entire transaction
Definite loop

Nice Domains
Requirements
Bugs ill-defined domains
Before DT
Analyze specs
Make the Boundary Specs Consistent & Complete
Implemented Domains
Complete, Consistent & Process all inputs
Specified Domains
Incomplete, Inconsistent
Programmers / Designers Effort
Nice Domains
Linear, Complete, Systematic, Orthogonal, Consistently Closed, & Convex

Advantages
Ease of DT
Fewer Bug Occurrences

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 49

Boundaries are
1. Linear
2. Complete
3. Systematic
4. Orthogonal
1. Linear Boundaries
Interpreted linear inequalities
n-dim Hyperplane:
n+1 Points
n+1 + 1 Test Cases

5. Closure consistency
6. Convex
7. Simply connected

Non-Linear
Transform
2. Complete Boundaries
Span the total number space
(-, +)
One set of Tests
Incomplete
Reasons
Tests
3. Systematic Boundaries
Linear Inequalities differing by a constant
fj (X) kj
or,
fj (X) g (j, c)

g (j, c) = j + k * c

Parallel lines

Identical Sectors in a Circle

DT

Test a domain

Tests for other Domains

4. Orthogonal Boundaries
Two boundaries or, boundary sets
Parallel to axes
DT
Each Set Independently
# Tests O (n)
Orthogonal Boundaries
Tilted sets
transformation
Test cases: O(n)
Concentric circles with radial lines
Rectangular coordinates
Polar
r aj .AND. r < aj+1 .AND.

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 50

j .AND. < j+1

Non-Orthogonal Boundaries
Test Intersections O ( n2 ) + O (n)

5. Closure Consistency
A Simple pattern in all boundary closures
Example
Same relational operator for systematic boundaries

6. Convex Domain
Line joining any two points lies with in the domain
DT
n on-points & 1 off-pt
Concave
But, However, Except, Or in Specs
Handle with special care
7. Simply Connected Domain
In a single piece
2 complementary domains
D1: Convex
Programmer / Designer
Convex part First

D2: Concave, not-Connected

Ugly Domains
Generally
From Bad Specs
Programmer / Designer Simplifies => Ugly to Good
possibility of Introduction of more bugs ?!?
Causes
1.
2.
3.
4.

Non-linear Boundaries
Ambiguities & Contradictions
Simplifying the topology
Rectifying boundary closures

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 51

1. Non-linear Boundaries
Transform
2. Ambiguities
Holes in input vector space.
Missing boundary
Detected by Specification languages & tools.
Contradictions..
Overlapping of

3. Simplifying the Topology

Complexity !
Concavity, Holes, Disconnectedness
Smoothing out concavity

Filling in Holes

Joining the pieces

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 52

Correct:
Connect disconnected boundary segments
Extend boundaries to infinity
4. Rectifying Boundary Closures.
make closures in one direction for parallel boundaries with closures in both directions
Force a Bounding Hyperplane to belong to the Domain.

General DT Strategy
Select test points near the boundaries.
Define test strategy for each possible bug related to boundary
Test points for a domain useful to test its adjacent domain.
Run the tests. By post test analysis determine if any boundaries are faulty & if so how?
Run enough tests to verify every boundary of every domain
DT for Specific Domain Bugs
Generally,
Interior point
Exterior point
Epsilon neighborhood
Extreme point
On point

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

1.
2.
3.
4.
5.

P a g e | 53

1-d Domains
2-d Domains
Equality & inequality Predicates
Random Testing
Testing n-dimensional Domains

1. Testing 1-d Domains :

Bugs with open boundaries

Closure Bug

Shift left Bug

Shift Right Bug

Missing Boundary

Extra Boundary

2. Testing 2-d Domains

1. Closure bug
2. Boundary Shift : up / down
3. Tilted Boundary
4. Extra Boundary
5. Missing Boundary
Fig:
2 n : Domains share tests
3 n : no sharing of tests by domains

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 54

3. Equality & Inequality Predicates

An Equality predicate defines a line in 2-d

4. Random Testing
A Point in the center : verifies computation

5. Testing n-Dimensional Domains (strategy)

n-dimensions, p boundary segments
(n+1)*p test cases : n on points & 1 off point
Extreme pt shared : 2 * p points
Equalities over m-dimensions create a subspace of n-m dimensions
Orthogonal domains with consistent boundary closures, orthogonal to the axes & complete boundaries
Independent testing
Procedure for solving for values
Simple procedure. Need tools.
a. Identify input variables
b. Identify variables which appear in domain-defining predicates, such as control-flow predicates
c. Interpret all domain predicates in terms of input variables:
1. Transform non-linear to linear
2. Find data flow path
d. Predicate expression with p # predicates.
Find # domains : < 2 p
e. Solve inequalities for extreme points
f. Use extreme points to solve for nearby on points

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 55

Effectiveness
Cost effective
Bugs on boundaries, extreme points
Hardware logic testing tool intensive
Domains & Interface Testing
Domain
Range

Span compatibility

Interface Range/Domain Compatibility Testing

Test each var independently
Find an inverse function
Build a super domain
Domain Testing Questions from the previous years exams
1.
2.
3.
4.
5.

Explain Nice & Ugly Domains.

What is domain testing? Discuss applications of domain testing
Explain the domain boundary bugs for two dimensional domains
What is the purpose of Domain Testing? Give its schematic representation
Define the following concepts.
1. Domains
2. Domain closure
3. Domain dimensionality
4. Bug assumptions for Domain Testing
6. Explain simple domain boundaries & compound predicates.
7. Classify what can go wrong with boundaries, then define a test strategy for each case in
domain testing.

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 56

Unit V
Paths, Path products and Regular expressions
Path Products & expressions
Motivation:
Flow graph is an abstract representation of a program.
A question on a program can be mapped on to an equivalent question on an appropriate flow graph.
It will be a foundation for syntax testing & state testing
Path Expression
An algebraic representation of sets of paths in a flow graph.
Regular Expression
Path expressions converted by using arithmetic laws & weights into an algebraic function.
Annotate each link with a name.
The pathname as you traverse a path (segment) expressed as concatenation of the link names is the path
product.

Examples of path products between 1 & 4 are:

abd
a b c b d a b c b c b d .
Path Expression
Simply: Derive using path products.

Example
1. { a b d, a b c b d, a b c b c b d , .. }
abd + abcbd + abcbcbd + .
2.

{ abcd , abfhebcd , abfigebcd , abfijd }

abcd + abfhebcd + abfigebcd + abfijd

Path name for two successive path segments is the concatenation of their path products.
X = abc
Y = def
XY = abcdef
a X = aabc
X a = abca
XaX = abcaabc
X = ab + cd
Y = ef + gh
XY = abef + abgh + cdef + cdgh
Path Segments & Products
Loops
a1 = a
a2 = aa
a3 = aaa
an = aaaaa n times
1
X = abc
X = abc
X2 = (abc)2 = abcabc
Identity element
a0 = 1
X0 = 1
(path of length 0)
Path Product
Not Commutative:
XY YX in general
Associative
A ( BC ) = ( AB ) C = ABC
: Rule 1
Denotes a set of paths in parallel between two nodes.
Commutative

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 57

X + Y = Y + X
: Rule 2
Associative
(X+Y)+Z = X+(Y+Z) = X+Y+Z
: Rule 3
Distributive
A ( B + C) = A B + A C
: Rule 4
(A+B)C = AC+BC
Absorption
X + X = X
: Rule 5
X + any subset of X = X
X = a + bc + abcd
X + a = X + bc + abcd = X
Loop
An infinite set of parallel paths.

b* = b0 + b1 + b2 + b3 +
X* = X0 + X1 + X2 + X3 +
X+ = X1 + X2 + X3 +
X X* = X* X = X+
n
0
1
X = X + X + X2 + X3 + + Xn
More Rules
Xm + Xn = Xn
if n m
= Xm
if n < m
m n
m+n
X X
= X
Xn X* = X* Xn = X*
Xn X+ = X+ Xn = X+
X* X+ = X+ X* = X+
Identity Elements ..
1 : Path of Zero Length
1+1 = 1
1X
= X1 = X
1n
= 1n = 1 * = 1 + = 1
1+ + 1 = 1 * = 1
0 : empty set of paths
X + 0 = 0+X = X
X0
= 0X = 0
0*
= 1 + 0 + 02 + 03 + . . . = 1

a a* = a* a = a+

: Rule 6
: Rule 7
: Rule 8
: Rule 9
: Rule 10

: Rule 11
: Rule 12
: Rule 13
: Rule 14
: Rule 15
: Rule 16
: Rule 17

Reduction Procedure
To convert a flow graph into a path expression that denotes the set of all entry/exit paths.
Node by Node Reduction Procedure
Initialization Steps
1. Combine all serial links by multiplying their path expressions.
2. Combine all parallel links by adding their path expressions.
3. Remove all self-loops - replace with links of the form X*
4. Select a non-initial & non-final node.
Replace it with a set of equivalent links, whose path expressions correspond to all the ways you can
form a product of the set of in-links with the set of out-links of that node.
5. Combine any serial links by multiplying their path expressions.
6. Combine any parallel links by adding their path expressions.
7. Remove all the self-loops.

( as in step 1)
( as in step 2)
( as in step 3)

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 58

8. IF theres just one node between entry & exit nodes, path expression for the flow graph is the links
path expression.ELSE, return to step 4.
Path Expression for a Flow Graph
is not unique
depends on the order of node removal.

Reduction Procedure Example

Cross-Term Step (Step 4 of the algorithm)
Fundamental step.
Removes nodes one by one till theres one entry & one exit node.
Replace the node by path products of all in-links with all out-links and interconnecting its immediate
neighbors.
Processing of Loop Terms

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 59

Example

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 60

Flow Graph Path Expression :

a(bgif)*b(c+gkh)d {(ilhd)*imf(bgif)*b(c+gkh)d}* (ilhd)*e

Identities/Rules
(A+B)*

=
( A* + B* ) *
: I1
=
( A* B* )*
: I2
=
( A* B )* A*
: I3
=
( B* A )* B*
: I4
=
( A* B + A )*
: I5
=
( B* A + B )*
: I6
(A+B+C+...)*
=
( A* + B* + C* + . . . )*
: I7
=
( A* B* C* . . . )*
: I8
Derived by removing nodes in different orders & applying the series-parallel-loop rules.

Structured Flow Graphs

Reducible to a single link by successive application of the transformations shown below.

Structured flow graph transformations

Properties
No cross-term transformation.
No GOTOs.
No entry into or exit from the middle of a loop.
Some examples

Unstructured flow graphs/code:

K. Nuka Raju Associate Professor, DIET

Software Testing Methodologies

P a g e | 61

Questions from the previous years exams

1. Write short notes on: path products, path expressions, path sums, loops.
2. Define structured code. Explain lower path count Arithmetic.
3. What is the looping probability of a path expression. Write arithmetic rules. Explain with an example.
4. Write the steps involved in Node Reduction Procedure. Illustrate all the steps with the help of neatly labeled
diagrams.

K. Nuka Raju Associate Professor, DIET