Software Testing and Quality Assurance

Theory and Practice

Chapter 3
Unit Testing

1
 Outline of the Chapter
• Concept of Unit Testing
• Static Unit Testing
• Defect Prevention
• Dynamic Unit Testing
• Mutation Testing
• Debugging
• Unit Testing in eXtreme Programming
• Tools For Unit Testing

2
 Concept of Unit Testing
• Static Unit Testing
– Code is examined over all possible behaviors that might arise
during run time
– Code of each unit is validated against requirements of the unit
by reviewing the code
• Dynamic Unit Testing
– A program unit is actually executed and its outcomes are
observed
– One observe some representative program behavior, and reach
conclusion about the quality of the system
• Static unit testing is not an alternative to dynamic unit
testing
• Static and Dynamic analysis are complementary in nature
• In practice, partial dynamic unit testing is performed
concurrently with static unit testing
• It is recommended that static unit testing be performed
prior to the dynamic unit testing 3
 Static Unit Testing
• In static unit testing code is reviewed by applying techniques:
– Inspection: It is a step by step peer group review of a work product,
with each step checked against pre-determined criteria
– Walkthrough: It is review where the author leads the team through a
manual or simulated executed of the product using pre-defined
scenarios
• The idea here is to examine source code in detail in a systematic
manner
• The objective of code review is to review the code, and not to
evaluate the author of the code
• Code review must be planned and managed in a professional
manner
• The key to the success of code is to divide and conquer
– An examiner inspect small parts of the unit in isolation
• nothing is overlooked
• the correctness of all examined parts of the module implies the correctness
of the whole module

4
 Static Unit Testing (Code Review)
• Step 1: Readiness
– Criteria
• Completeness
• Minimal functionality
• Readability
• Complexity
• Requirements and design
documents
– Roles
• Moderator
• Author
• Presenter
• Record keeper
• Reviewers
• Observer
• Step 2: Preparation
– List of questions
– Potential Change Request (CR)
– Suggested improvement opportunities Figure 3.1: Steps in the code review proces
5
 Static Unit Testing (Code Review)
• Step 3: Examination A Change Request (CR) includes the
– The author makes a presentation following details:
– The presenter reads the code Give a brief description of the issue
– The record keeper documents the CR Assign a priority level (major or minor)
to a CR
– Moderator ensures the review is on
track Assign a person to follow it up
• Step 4: Re-work Set a deadline for addressing a CR
– Make the list of all the CRs
– Make a list of improvements
– Record the minutes meeting
– Author works on the CRs to fix the
issue
• Step 5: Validation
– CRs are independently validated
• Step 6: Exit
– A summary report of the meeting
minutes is distributes
6
 Static Unit Testing (Code Review)
The following metrics can be collected from a code review:

• The number of lines of code (LOC) reviewed per

hour
• The number of CRs generated per thousand lines of
code (KLOC)
• The number of CRs generated per hour
• The total number of hours spend on code review
process

7
 Static Unit Testing (Code Review)
• The code review methodology can be applicable to review other documents
• Five different types of system documents are generated by engineering department
– Requirement
– Functional Specification
– High-level Design
– Low-level Design
– code
• In addition installation, user, and trouble shooting guides are developed by technical
documentation group

Table 3.1: System documents

8
 Defect Prevention
• Build instrumentation code into the code
• Use standard control to detect possible occurrences of error
conditions
• Ensure that code exists for all return values
• Ensure that counter data fields and buffer
overflow/underflow are appropriately handled
• Provide error messages and help texts from a common
source
• Validate input data
• Use assertions to detect impossible conditions
• Leave assertions in the code.
• Fully document the assertions that appears to be unclear
• After every major computation reverse-compute the input(s)
from the results in the code itself
• Include a loop counter within each loop
9
 Dynamic Unit Testing
• The environment of a unit is emulated and tested in isolation
• The caller unit is known as test driver
– A test driver is a program that invokes the unit under test (UUT)
– It provides input data to unit under test and report the test result
• The emulation of the units called by the UUT are called stubs
– It is a dummy program
• The test driver and the stubs are together called scaffolding
• The low-level design document provides guidance for selection of input test data

Figure 3.2: Dynamic unit test

environment

10
 Concept of Stub and driver
• The caller unit is known as driver
• All emulations of units called by unit under
test is test stub
Test Driver: A test driver is a program that
invokes the unit under test. The unit under
test executes with input values received from
the driver and, upon termination, returns a
value to the driver.
• The test driver functions as the main unit in
the execution process. The driver not only
facilitates compilation, but also provides input
data to the unit under test in the expected
format.
• Stubs: A stub is a “dummy subprogram” that
replaces a unit that is called by the unit under
test. Stubs replace the units called by the unit
under test. A stub performs two tasks:
• First, it shows an evidence that the stub was,
in fact, called. Such evidence can be shown by
merely printing a message.
• Second, the stub returns a precomputed value
to the caller so that the unit under test can
continue its execution
 Dynamic Unit Testing
Selection of test data is broadly based on the following
techniques:
• Control flow testing
– Draw a control flow graph (CFG) from a program unit
– Select a few control flow testing criteria
– Identify a path in the CFG to satisfy the selection criteria
– Derive the path predicate expression from the selection paths
– By solving the path predicate expression for a path, one can generate the data
• Data flow testing
– Draw a data flow graph (DFG) from a program unit and then follow the
procedure described in control flow testing.
• Domain testing
– Domain errors are defined and then test data are selected to catch those
faults
• Functional program testing
– Input/output domains are defined to compute the input values that will cause
the unit to produce expected output values

14
 Mutation Testing
• Modify a program by introducing a single small change to
the code
• A modified program is called mutant
• A mutant is said to be killed when the execution of test case
cause it to fail. The mutant is considered to be dead
• A mutant is an equivalent to the given program if it always
produce the same output as the original program
• A mutant is called killable or stubborn, if the existing set of
test cases is insufficient to kill it
• A mutation score for a set of test cases is the percentage of
non-equivalent mutants killed by the test suite
• The test suite is said to be mutation-adequate if its mutation
score is 100%
15
 Mutation Testing
Mutation testing makes two major assumptions:

1. Competent Programmer hypothesis

– Programmers are generally competent and they do not
create random programs

2. Coupling effects
– Complex faults are coupled to simple faults in such a way
that a test suite detecting simple faults in a program will
detect most of the complex faults

16
 To calculate mutation score
To calculate mutation score for set of test
cases(T):
Mutation score is the percentage of nonequivalent mutants killed by test
data.

Mutation score= 100* D / (N-E)

Where D is dead mutants

N is total number of mutants
E is number of equivalent mutants
• If the estimated mutation adequacy
(according to previous slide) of test cases(T) is
not sufficiently high, then design a new test
case and add the new test case

Otherwise, if computed adequacy of test

cases(T) is more than appropriate threshold,
then accept T as good measure and stop
designing new test cases
 Debugging
• The process of determining the cause of a failure is
known as debugging
• It is a time consuming and error-prone process
• Debugging involves a combination of systematic
evaluation, intuition and a little bit of luck
• The purpose is to isolate and determine its specific
cause, given a symptom of a problem

19
There are three approaches to debugging

1. Brute force
2. Cause elimination
• Induction
• Deduction
3. Backtracking
 Unit Testing in eXtreme Programming
1. Pick a requirement, i.e., a story
2. Write a test case that will verify a small
part of the story and assign a fail verdict to
it
3. Write the code that implement particular
part of the story to pass the test
4. Execute all test
5. Rework on the code, and test the code until
all tests pass
6. Repeat step 2 to step 5 until the story is
fully implemented

Figure 3.3: Test-first process in

XP

21
 Unit Testing in eXtreme Programming
Three laws of Test Driven development (TDD)
• One may not write production code unless the first
failing unit test is written
• One may not write more of a unit test than is sufficient
to fail
• One may not write more production code than is
sufficient to make the failing unit test pass

Pair programming:
– In XP code is being developed by two programmers
working side by side
– One person develops the code tactically and the other one
inspects it methodically by keeping in mind the story they
are implementing
22
 JUnit – A Framework for Unit Testing
• JUnit: It is a framework for performing unit testing of Java
programs.
– Other frameworks: NUnit (C#), CPPUnit (C++), fUnit (Fortran)
• Intuitive steps to test a method in Java (Ex. Move() method
of PlanetClass)
– Create an object instance of PlanetClass. Call it Mars.
– Select values of all input parameters of Move().
– Compute the expected value to be returned by Move(). Let it be
y.
– Execute method Move() on Mars with the selected input values.
• Let Move() return a value called z.
– Compare the actual output (z) returned by Move() with the
expected value (y).
• If (z == y), Move() passes the test; otherwise it fails.  Report the
result.
• JUnit makes writing of test cases easier.  Next slide …
23
 JUnit – A Framework for Unit Testing
• JUnit provides a basic class called TestCase.
• The tester
– Extends the TestCase class for each test case. 10 extensions
for 10 test cases.
– Alternatively, extend TestCase to have 10 methods for 10 test
cases.
• The TestCase class provides methods to make assertions.
– assertTrue(Boolean condition)
– assertFalse(Boolean condition)
– assertEquals(Object expected, Object actual)
– assertEquals(int expected, int actual)
– assertEquals(double expected, double actual, double tolerance)
– assertSame(Object expected, Object actual)
– assertNull(Object testobject)
–…
• The tester can have her own assertions. 24
 Tools For Unit Testing
1.Code auditor
– This tool is used to check the quality of the software to
ensure that it meets some minimum coding standard
2. Bound checker
–This tool can check for accidental writes into the
instruction areas of memory, or to other memory
location outside the data storage area of the application
3.Documenters
–These tools read the source code and automatically
generate descriptions and caller/callee tree diagram or
data model from the source code
25
4. Interactive debuggers
–These tools assist software developers in
implementing different debugging techniques
Examples: Breakpoint and Omniscient debuggers
5.In-circuit emulators
–It provides a high-speed Ethernet connection
between a host debugger and a target
microprocessor, enabling developers to perform
source-level debugging
 Tools for Unit Testing
6. Memory leak detectors
– These tools test the allocation of memory to an application which request for memory
and fail to de-allocate memory
7. Static code (path) analyzer
– These tool identify paths to test based on the structure of code such as McCabe’s
cyclomatic complexity measure

Table 3.3: McCabe complexity

measure

27
8. Software inspection support
–Tools can help schedule group inspection
9. Test coverage analyzer
–These tools measure internal test coverage, often
expressed in terms of control structure of the test
object, and report the coverage metric
10. Test data generator
–These tools assist programmers in selecting test data
that cause program to behave in a desired manner
11. Test harness
–This class of tools support the execution of dynamic
unit tests 28
12. Performance monitors
–The timing characteristics of the software
components be monitored and evaluate by these
tools
13. Network analyzers
–These tools have the ability to analyze the traffic
and identify problem areas
14. Simulators and emulators
–These tools are used to replace the real software and
hardware that are not currently available. Both the
kinds of tools are used for training, safety, and economy
purpose
15. Traffic generators
–These produces streams of transactions or data packets.
16. Version control
–A version control system provides functionalities to
store a sequence of revisions of the software and
associated information files under development
30