ICT 3312

SOFTWARE VERIFICATION
AND VALIDATION

Oshadhi Munasinghe
[email protected]
WHAT IS SOFTWARE QUALITY?

Chapter 5
AFTER COMPLETING THIS CHAPTER,

YOU WILL BE ABLE TO:

• Define software, software quality and software quality assurance.

• Distinguish between software errors, software faults and
software failures.
• Identify the various causes of software errors.
• Explain the objectives of software quality assurance activities.
• Distinguish and explain the difference between software
quality assurance and quality control.
• Explain the relationship between software quality assurance
and software engineering.
OUTLINE
1. What is software?
2. Software errors, faults and failures
3. Classification of the causes of software errors
4. Software quality – definition
5. Software quality assurance – definition and objectives
1. Software quality assurance definitions
2. Software quality assurance vs. software quality
control
3. The objectives of SQA activities
6. Software quality assurance and software engineering
WHAT IS SOFTWARE?

The IEEE definition of software, which is almost identical to the ISO

definition, lists the following four components of software:
✓ Computer programs (the “code”)
✓ Procedures
✓ Documentation
✓ Data necessary for operating the software system
• Consider the following views, expressed by participants in a
human resources management conference, refer to the same
software package.

➢ “We’ve used the Simplex HR software in our Human Resources

Department for about three years and we have never had a
software failure”
➢ “I started to use Simplex HR two months ago; we had so many
failures that we are considering replacing the software package.”
➢ “We have been using the same software package for almost four
years. We were very satisfied through out the period until the last
few months, when we suddenly faced several server failures.
• Is it possible for such a variation in users’ experience
with failure to appear with the same software
package?

Yes
SOFTWARE ERRORS, FAULTS AND FAILURES

Software
Defects

Software Software Software

Errors Faults Failures
SOFTWARE ERRORS
Software errors are sections of the code that are partially or
totally incorrect as a result of a grammatical, logical or other
mistake made by a systems analyst, a programmer, or another
member of the software development team.
•An error can be a grammatical error in one or more of the
code lines,
•or a logical error in carrying out one or more of the client’s
requirements.
• An error may or may not be detected during the coding or
testing of the program before it is released to a customer.
 SOFTWARE FAULTS
Software faults are software errors that cause the incorrect functioning
of the software during a specific application.
•A software fault occurs as a result of an error that remains in the
executing program.
• However, not all software errors become software faults.
•Not all faults however are detected and the software may continue
executing without any obvious problems.
•There are cases where software faults go undetected for many years of
a programs existence
 SOFTWARE FAILURE
Software faults become software failures only when they are
“activated”, that is, when a user tries to apply the specific
software section that is faulty. Thus, the root of any software
failure is a software error.

•Results in a detectable problem; hence it is referred to as a

failure.
• Do all software faults end with software failures?
➢ Not necessarily: a software fault becomes a software failure only when it is
“activated”.
SOFTWARE ERRORS, FAULTS AND FAILURES
Developers and users have different views of the software
product regarding its internal defects.
➢While developers are interested in software errors and
faults, their elimination, and the ways to prevent their
generation,
➢ software users are worried about software failures.
CAUSES OF SOFTWARE ERRORS

1. Faulty requirements definition

2. Client–developer communication failures
3. Deliberate deviations from software requirements
4. Logical design errors
5. Coding errors
6. Non-compliance with documentation and coding instructions
7. Shortcomings of the testing process
8. Procedure errors
9. Documentation errors
1. FAULTY REQUIREMENTS DEFINITION

The faulty definition of requirements, usually prepared by the client, is

one of the main causes of software errors.
➢ Erroneous definition of requirements
➢ Absence of vital (necessary or essential) requirements
➢ Incomplete definition of requirements
➢Inclusion of unnecessary requirements, functions that are not
expected to be needed in the near future
2. CLIENT–DEVELOPER COMMUNICATION FAILURES

Misunderstandings resulting from defective client developer

communication are additional causes for errors that prevail in the early
stages of the development process.
➢ Misunderstanding of the client’s instructions
➢Misunderstanding of the client’s requirements changes presented to the
developer (in oral and written forms)
➢Misunderstanding of the client’s responses to the design problems
presented by the developer
➢ Lack of attention to client massages referring to requirements changes
3. DELIBERATE DEVIATIONS FROM SOFTWARE REQUIREMENTS

In several circumstances, developers may deliberately deviate from the documented

requirements.
• The developer reuses software modules taken from an earlier project without
sufficient analysis of the changes and adaptations needed to correctly fulfill the
new requirements.
• Due to time or budget pressures, the developers decides to omit part of the required
functions in an attempt to cope with these pressure.
• Developer initiated, unapproved improvements to the software, introduced without
client’s approval.
4. LOGICAL DESIGN ERRORS

Software errors can enter the system when the professional who design the system- system
architects, software engineers, analysts etc. formulate the requirements.
➢ Definitions that represent software requirements by mean of erroneous
algorithms.
➢ Process definitions that contain sequential errors
➢ Erroneous definition of boundary conditions
➢ Omission of required software system states. (real time
computerized apparatus)
➢Omission of definitions concerning reactions to illegal operation of the software
systems.
5. CODING ERRORS

A broad range of reasons cause programmers to make coding errors. These

include,
• misunderstanding the design documentation,
• linguistic errors in the programming languages,
• Errors in the applications of CASE and other development tools
• Errors in data selection
6. NON-COMPLIANCE WITH DOCUMENTATION AND CODING
INSTRUCTIONS

Almost every development unit has its own documentation and coding
standards that define the content, order and format of the documents, and the
code created by team members.
• If there is a “non –complying” team member,
➢It may be hard to understand by others (during collaboration and
replacement)
➢ It will reduce the efficiency of the design and code review teams
➢Maintenance teams required to contend with the “bugs” detected by users
and to change or add to the existing software will face difficulties when
trying to understand the software and its documentation.
 7. SHORTCOMINGS OF THE TESTING PROCESS

Shortcomings of the testing process affect the error rate by leaving a grater number of
errors undetected or uncorrected.
• Incomplete test plans leave untreated portions of the software or the application
functions and states of the systems
• Failures to document and report detected errors and faults
• Failures to promptly correct detected software faults as a result of inappropriate
indications of the reasons for the fault.
• Incomplete correction of detected errors due to negligence or time pressures.
8. PROCEDURE ERRORS

Procedures direct the user with respect to the activities required at each step of the
process.
They are of special importance in complex software systems where the processing is
conducted in several steps, each of which may feed a variety of types of data and allow for
examination of the intermediate results.
➢ E.g. : Consider the procedures related to a telecom billing application (post-paid)
o Extracting records from switches
o Mediation (extract/decode the relevant information)
o Feed data to the billing/rating app (done at the end of the
billing cycle, tariff may depend on type package, etc)
o Making invoices (loyalty points, debts, discounts, etc)
9. DOCUMENTATION ERRORS
• The documentation errors that trouble the development and maintenance
teams are errors in the design documents and in the coding documentation
integrated into the body of the software.(User manuals, On-line help)
▪ Omission of software functions in user documentation
▪ Errors in the explanations and instructions given to users, resulting in “dead end”
or incorrect applications
▪ Listing of non existing software functions, that is functions planned in the early
stages of development but later dropped, and functions that were active in
previous versions of the software but cancelled in the current version
▪ Reasons for documentation errors include human failure to understand earlier
documentation, negligence to complete documentation tasks and failure to
phrase correctly the required text.
CAUSES OF ERRORS

• It should be emphasized that all causes of error are human,

the work of systems analysts, programmers, software
testers, documentation experts, and even clients and their
representatives !!!!!
SOFTWARE QUALITY –DEFINITION

Two alternative definitions:

▪ “Quality means conformance to requirements” (Crosby, 1979).
▪ “(1) Quality consists of those product features which meet the needs of customers
and thereby provide product satisfaction.
▪ (2) Quality consists of freedom from deficiencies
PRESSMAN’S DEFINITION SUGGESTS THREE REQUIREMENTS FOR QUALITY
ASSURANCE THAT ARE TO BE MET BY THE DEVELOPER:
➢ Specific functional requirements, which refer mainly to the outputs
of the software system.
➢ The software quality standards mentioned in the contract.
➢ Good Software Engineering Practices(GSEP), reflecting state-of- the-art
professional practices, to be met by the developer even though not
explicitly mentioned in the contract.
SOFTWARE QUALITY ASSURANCE – DEFINITION AND OBJECTIVES
SQA –IEEE DEFINITION

• Plan and implement systematically.

➢SQA is based on planning and the application of a variety of
actions that are integrated into all the stages of the software
development process.
➢This is done in order to substantiate the client’s confidence that
the software product will meet all the technical requirements.

• Refer to the software development process.

• Refer to the specifications of the technical requirements.
SQA –EXPANDED DEFINITION

Adds to the IEEE definition.

➢ SQA should not be limited to the development process, but also to the
maintenance process.
➢ SQA actions should not be limited to the technical aspects of the
functional requirements, but should also include activities that deal
with scheduling and budget
SOFTWARE QUALITY ASSURANCE VS. SOFTWARE QUALITY
CONTROL

• Quality control is defined as “a set of activities designed to

evaluate the quality of a developed or manufactured
product”(IEEE,1991);
➢In other words, activities whose main objective is the with holding of
any product that does not qualify.
➢Accordingly, quality control inspection and other activities take place
as the development or manufacturing of the product is completed, yet
before the product is shipped to the client
SOFTWARE QUALITY ASSURANCE VS. SOFTWARE QUALITY CONTROL

• Quality assurance is to minimize the cost of guaranteeing quality by a

variety of activities performed throughout the development and
manufacturing processes/stages.
➢These activities prevent the causes of errors, and detect and correct
them early in the development process.
➢As a result, quality assurance activities substantially reduce the rate of
products that do not qualify for shipment and, at the same time,
reduce` the costs of guaranteeing quality in most cases
SOFTWARE QUALITY ASSURANCE VS. SOFTWARE QUALITY CONTROL

1) Quality control and quality assurance activities serve

different objectives.
2) Quality control activities are only a part of the total range
of quality assurance activities
 SOFTWARE QUALITY ASSURANCE VS. SOFTWARE QUALITY
CONTROL
Criteria Software Quality Assurance (SQA) Software Quality Control (SQC)
SQA is a set of activities for ensuring quality in SQC is a set of activities for
software engineering processes (that ultimately ensuring quality in software
Definition result in quality in software products). The products. The activities focus on
activities establish and evaluate the processes identifying defects in the actual
that produce products. products produced.
Focus Process focused Product focused
Orientation Prevention oriented Detection oriented
Breadth Organization wide Product/project specific
Relates to all products that will ever be created
Scope Relates to specific product
by a process
Reviews
* Process Definition and Implementation
Testing
Activities * Audits
* Training
THE OBJECTIVES OF SQA ACTIVITIES

The objectives of SQA activities refer to the functional, managerial and

economic aspects of software development and software maintenance.
These objectives are;
Software development (Process –oriented)
• Assuring and acceptable level of confidence that the software will
conform to functional technical requirements.
• Assuring an acceptable level of confidence that the software will
conform to managerial scheduling and budgetary requirements.
• Initiating and managing of activities for the improvement and
greater efficiency of software development and SQA activities.
THE OBJECTIVES OF SQA ACTIVITIES

• Software maintenance (Product –oriented)

➢Assuring and acceptable level of confidence that the software
maintenance activities will conform to functional technical
requirements.
➢Assuring with an acceptable level of confidence that the software
maintenance activities will conform to managerial scheduling and
budgetary requirements.
➢Initiating and managing of activities for the improve and increase
the efficiency of software maintenance and SQA activities
SOFTWARE QUALITY ASSURANCE AND SOFTWARE ENGINEERING

According to the IEEE(1991), software engineering is defined as follows:

(1)The application of a systematic, disciplined, quantifiable approach to the
development, operation and maintenance of software; that is, the application
of engineering to software.
(2) The study of approaches as in(1).

The characteristics of software engineering, especially the systematic,

disciplined and quantitative approach at its core, make the software
engineering environment a good infrastructure for achieving SQA objectives
 SOFTWARE QUALITY ASSURANCE AND SOFTWARE ENGINEERING

• The methodologies and tools that are applied by software engineering

determine, to a considerable extent, the level of quality to be expected
from the software process and the maintenance services.
➢ Therefore, it is desirable that when making decisions about software
s methodologies and tools, SQA considerations be added to the
d efficiency and economy considerations associated with software
engineering.
• It is commonly accepted that cooperation between software engineers and
the SQA team is the appropriate way to achieve efficient and economic
development and maintenance activities that, at the same time, assure the
quality of the product of these activities.
 SUMMARY AND QUESTIONS

• Define software, software quality and software quality assurance.

•Distinguish between software errors, software faults and software failures.
•Identify the various causes of software errors.
•Explain the objectives of software quality assurance activities.
•Distinguish and explain the difference between software quality
assurance and quality control.
•Explain the relationship between software quality assurance and software
engineering