1

Introduction
DR. HAGER HUSSEIN
 2
Learning outcomes

 On successful completion of the course students will be able to:

 1. Apply the concepts and theory related to software verification and
validation.
 2. Identify different testing techniques and design test plans, develop
test suites, and evaluate test suite coverage.
 3. Use testing frameworks and testing tools.
 3
What is the program doing: Specification

 Given any piece of code, Can you answer “Is the function correct?” ?.
 This question can’t be answered without knowing what the program is
supposed to be doing.
 One way to identify the behavior of the program is to analyze the code.
 Problems with that approach:
 Reading someone else’s code is a frustrating and an unreliable task.
 If we succeed in the attempt to understand “what’s going on,”, we still cannot
decide about the program’s correctness because by reading the code we can
only identify the actual program behavior, rather than the intended one.
 4
Specification

 A separate statement about the intended program’s behavior

has to be stated, this is referred to as the specification.
 It is supplied with narrative writing, charts, figures, and one or
more examples of the inputs and expected outputs.
 5
Example

 Specification: Given is an array A[1…n] of integers. The function

returns the length of a longest strictly increasing monotone
sequence in A (limseq, in short).
 Does this look clear?
 6
Is the specification statement clear?

 At first glance the above narrative looks clear enough. But, is it

really the case?
 What is meant by a “strictly increasing monotone sequence?”
 Apparently the originator of the problem must have assumed
that the reader (or the programmer who has to write the
program) is familiar with that concept.
 What is to be done when the programmer fails that expectation?
 7
Is the specification statement clear?
Cont.

 The best case scenario is to consult a textbook in which the

concept of monotonicity might be defined as “changing always
in the same direction or staying the same.”
 That would suggest that strict monotonicity must stand for “a
continuous change in the same direction.”
 Too often, unfortunately, the programmer will believe that the
problem is clear and will act upon that belief, producing an
incorrect program.
 8
Specification statement

 Trying to be optimistic, assume that the concept of strict

monotonicity has been clarified.
 Now, what is the “sequence in A” that is monotonic in that
sense?
 Are we talking about the elements in the array, or the indices to
those elements?
 9
Specification statement cont.

 For example, for N = 10 and A =[1, 2, 9, 4, 7, 3, 11, 8, 14, 6]

 In the former case one would expect maxl = 6, because of the
sequence 1, 2, 4, 7, 8, 14 or 1, 2, 4, 7, 11, 14 of values in A .
 In the later case, maxl = 6 would be due to the sequence 1, 2,
4, 5, 8, 9 or 1, 2, 4, 5, 7, 9 of the corresponding indices.
 Either sequence is strictly monotonic but that is due to the fact
that the elements in the array are distinct.
 Otherwise, when A can hold multiple occurrences of values, the
value sequence might not be strictly monotonic.
 10
Specification statement cont.

 Natural language narratives are ambiguous and imprecise,

although it is readable.
 Formal specifications that use the language of mathematics are
unambiguous, but unreadable.
 Although there is little doubt that the effort put into writing
formal specification is well spent, mastering the technique might
be prohibitive for many practitioners of the programming craft.
 11
Specification Requirements

 For functions or procedures, it is reasonable to require that the

specification provide the following items of data:
 IN parameters: those whose initial values may potentially be used in
a computation. The identifiers and types of the variables should be
specified.
 OUT parameters: those which are important outside the program on
its termination.
 External or global variables that the subprogram can read or modify
 A restriction on input values that identifies those values for which
the programming problem is defined. Such a restriction on the set of
all inputs to the program is referred to as the precondition of the
program. Any input that meets the precondition is valid .
 The desired relationships between the input and the output.
 12
Monotone Specification Function
 13
Monotone Specification Function
cont.
 14
How to make sure that the program is doing it right: Verification

 The specification of the program tells us WHAT has to be done,

while a program shows a particular way HOW it can be done.
 If the What part is consistent with the How part, then the
program is correct or consistent with the specification.
 There is always some specification for any program for which
the program is correct.
 This is the specification which states what the program is
actually doing, whether this is what we really want or not.
 15
Is that correct?

 if P is correct for S , it does not necessarily mean that P and S

are equivalent.
 16
Is that correct? Cont.

 First, P can actually accept more inputs than prescribed by the

precondition.
 For example, a search procedure whose precondition restricts
the inputs to sorted arrays may actually be implemented for
arbitrary arrays.
 Second, P can produce results that are more restricted than
those required by the postcondition.
 For example, a search procedure may return any position of the
element searched for to be returned, while the actual
implementation may return the smallest such a position.
 17
Verification vs Validation

 Verification is to establish whether the program is correct or not.

 Validation is usually understood as establishing whether the
specification really captures the user’s need.
 18
Monotone function verification

 The outcome of the verification can be either

(1) Positive (yes, the program is correct)
(2) Negative (no, it is not correct)
(3) Inconclusive (“don’t know” or “perhaps”).
 The first outcome can only be achieved through a successful
proof of correctness.
 The second outcome is due either to a successful proof of
incorrectness or to a successful testing, when a bug is detected.
 The third outcome is typical for inconclusive proof attempts, for
unsuccessful testing when no bug is detected.
 19
Monotone Function Verification cont.

 Let us consider testing for, if successful, it detects flaws both in

the implementation and the specification.
 According to the specification of the Monotone function, for n =
10 and A = [1, 2, 9, 4, 7, 3, 11, 8, 14, 6] the valid result is 6 and
this is exactly what the implementation returns. However, for
the input n = 3, A = [2, 2, 2], the expected valid result is 1,
while the implementation produces 3.
 Consequently, the function is incorrect for the specification: the
implementation mishandles arrays with multiple entries.
 20
Monotone Function Verification cont.

 Nevertheless, the problem originator might want to exclude

such arrays by strengthening the precondition by the following
restriction:
 “All elements in the array A between 1 and n are distinct.”
 Now, since the input [2, 2, 2] violates the above requirement,
the function’s output for it is undefined.
 This is because the specification allows any behavior for invalid
input, including nontermination. Therefore, one no longer can
claim that for A = [2, 2, 2] the function returns an incorrect
result 3. (A yet another example of that phenomenon is offered
when the value of n is 0: the function then returns a
meaningless value 1).
 21
Monotone Function Verification cont.

 Choosing then the more restricted specification of the problem

and assuming that the function has passed a collection of tests
(i.e., a test suite), is now the code correct for the specification?
 Formal verification is the only verification method which, if
successful, guarantees correctness of the program.
 22
Trying to Show That the Program is Incorrect: Testing

 Clearly, when a test “fails,” it succeeds in revealing an error.

 In contrast, when a test “passes” it simply fails to detect an
error, “Other bugs may exist”!
 23
Debugging and Static Analysis

 Trying to Locate the Cause of Incorrectness: Debugging

 What One Can Tell About The Program Without Executing It:
Static Analysis
 24
Grading Policy

Assessment Method
7th Week (30%) Exam(20) + Section Quiz1(5) + Section
Quiz2(5)

12th Week (20%) Project (20)

Coursework (10%) Assignments (5) + Section Quiz (5)

Final (40%) Examination (40)

 25

Thank You