Software Quality

Assurance & Testing

Lecture 1
 Text Book
Software Quality Engineering by Jeff Tian
 Quality Expectations

1. The software systems must do what they are supposed to do. In

other words, they must do the right things. (Validation)

2. They must perform these specific tasks correctly or satisfactorily. In

other words, they must do the things right. (Verification)
 Main tasks for software quality
engineering
• Quality planning;

• Execution of selected QA or software validation and verification

activities;

• Measurement and analysis to provide convincing evidence to

demonstrate software quality to all parties involved.
 An Initial Definition of (Software)
Quality
•Quality =
• ...meeting the customer’s requirements,
• ...at the agreed cost,
• ...within the agreed timescales.
•Quality = “Fitness for purpose”
•Quality = Customer satisfaction
 Standard Definitions of (Software)
Quality
• IEEE: Degree to which a system, component, or process meets (1) specified
requirements, and (2) customer or user needs or expectations

• ISO 8402: The totality of features and characteristics of a product or

service that bear on its ability to satisfy specified or implied needs
 Quality Problems in Software

• Pervasive use of software

• Growing reliance on software
• Large software systems
Software Failures: The Statistics

Cancelled
31%

Successful
16%

Challenged
53%
Software Failures: Cost Overruns

Under 20% 15.5%

21 - 50% 31.5%
51 - 100% 29.6%
101 - 200% 10.2%
201 - 400% 8.8%
Over 400% 4.4%

Average cost for “failed” projects is 189% of the original estimate, i.e.
almost double.
Software Failures: Time Overruns
Overrun Amount % cases
Under 20% 13.9%
21 - 50% 18.3%
51 - 100% 20.0%
101 - 200% 35.5%
201 - 400% 11.2%
Over 400% 1.1%

Average time for “failed” projects is 222% of original estimate, i.e.

more than double.
Testing, Quality assurance (QA), and Quality
engineering

Software Quality Engineering

Quality Assurance

Testing
 What is Quality
• Quality defines to any measurable characteristics such as correctness,
maintainability, portability, testability, usability, reliability, efficiency,
integrity, reusability, and interoperability.
 What is Quality

• Quality of Design: Quality of Design refers to the characteristics that

designers specify for an item. The grade of materials, tolerances, and
performance specifications that all contribute to the quality of design.
• Quality of conformance: Quality of conformance is the degree to
which the design specifications are followed during manufacturing.
Greater the degree of conformance, the higher is the level of quality
of conformance.
 Importance of Quality
• We would expect the quality to be a concern of all producers of goods and services.
However, the distinctive characteristics of software and in particular its intangibility and
complexity, make special demands.
• Increasing criticality of software: The final customer or user is naturally concerned about
the general quality of software, especially its reliability. This is increasing in the case as
organizations become more dependent on their computer systems and software is used
more and more in safety-critical areas. For example, to control aircraft.
• The intangibility of software: This makes it challenging to know that a particular task in a
project has been completed satisfactorily. The results of these tasks can be made tangible
by demanding that the developers produce 'deliverables' that can be examined for quality.
• Accumulating errors during software development: As computer system development is
made up of several steps where the output from one level is input to the next, the errors
in the earlier ?deliverables? will be added to those in the later stages leading to
accumulated determinable effects. In general the later in a project that an error is found,
the more expensive it will be to fix.
 What is Software Quality

• Software Functional Quality − It reflects how well it satisfies a given

design, based on the functional requirements or specifications.
• Software Structural Quality − It deals with the handling of non-
functional requirements that support the delivery of the functional
requirements, such as robustness or maintainability, and the degree
to which the software was produced correctly.
 Software Quality Assurance
• Software Quality Assurance − Software Quality Assurance (SQA) is a set of
activities to ensure the quality in software engineering processes that
ultimately result in quality software products. The activities establish and
evaluate the processes that produce products. It involves process-focused
action.
• Software quality assurance is a planned and systematic plan of all actions
necessary to provide adequate confidence that an item or product
conforms to establish technical requirements.
• A set of activities designed to calculate the process by which the products
are developed or manufactured.
• Software Quality Assurance (SQA) is simply a way to assure quality in the
software. It is the set of activities which ensure processes, procedures as
well as standards suitable for the project and implemented correctly.
 Software Quality Assurance

SQA Encompasses
• A quality management approach
• Effective Software engineering technology (methods and tools)
• Formal technical reviews that are tested throughout the software process
• A multitier testing strategy
• Control of software documentation and the changes made to it.
• A procedure to ensure compliances with software development standards
• Measuring and reporting mechanisms.
 Software Quality Assurance
Following activities are performed by an independent SQA group: 1
• Prepares an SQA plan for a project: The program is developed during project
planning and is reviewed by all stakeholders. The plan governs quality
assurance activities performed by the software engineering team and the
SQA group. The plan identifies calculation to be performed, audits and
reviews to be performed, standards that apply to the project, techniques for
error reporting and tracking, documents to be produced by the SQA team,
and amount of feedback provided to the software project team.
• Participates in the development of the project's software process
description: The software team selects a process for the work to be
performed. The SQA group reviews the process description for compliance
with organizational policy, internal software standards, externally imposed
standards (e.g. ISO-9001), and other parts of the software project plan.
 Software Quality Assurance
Following activities are performed by an independent SQA group: 2
• Reviews software engineering activities to verify compliance with the defined
software process: The SQA group identifies, reports, and tracks deviations from
the process and verifies that corrections have been made.
• Audits designated software work products to verify compliance with those
defined as a part of the software process: The SQA group reviews selected work
products, identifies, documents and tracks deviations, verify that corrections have
been made, and periodically reports the results of its work to the project manager.
• Ensures that deviations in software work and work products are documented
and handled according to a documented procedure: Deviations may be
encountered in the project method, process description, applicable standards, or
technical work products.
• Records any noncompliance and reports to senior management: Non-
compliance items are tracked until they are resolved.
 Software Quality Control

• Software Quality Control − Software Quality Control (SQC) is a set of

activities to ensure the quality in software products. These activities
focus on determining the defects in the actual products produced. It
involves product-focused action.
 Quality Assurance v/s Quality control
Quality Assurance Quality Control
Quality Assurance (QA) is the set of actions including Quality Control (QC) is described as the processes and
facilitation, training, measurement, and analysis needed to methods used to compare product quality to requirements
provide adequate confidence that processes are established and applicable standards, and the actions are taken when a
and continuously improved to produce products or services nonconformance is detected.
that conform to specifications and are fit for use.

QA is an activity that establishes and calculates the QC is an activity that demonstrates whether or not the
processes that produce the product. If there is no process, product produced met standards.
there is no role for QA.

QA helps establish process QC relates to a particular product or service

QA sets up a measurement program to evaluate processes QC verified whether particular attributes exist, or do not
exist, in a explicit product or service.

QA identifies weakness in processes and improves them QC identifies defects for the primary goals of correcting
errors.
Quality Assurance is a managerial tool. Quality Control is a corrective tool.

Verification is an example of QA. Validation is an example of QC.

 Views of Software Quality
In the transcendental view, quality is hard to define or describe in
abstract terms, but can be recognized if it is present. It is generally
associated with some intangible properties that delight users.
In the user view, quality is fitness for purpose or meeting user’s needs.
In the manufacturing view, quality means conformance to process
standards.
In the product view, the focus is on inherent characteristics in the
product itself in the hope that controlling these internal quality
indicators will improved external product behavior.
In the value-based view, quality is the customers’ willingness to pay for
a software.
 People’s roles and
responsibilities
• Different people would have different expectations based on their
roles and responsibilities
• We can divide the people into two broad groups:
• Consumers of software products or services
• customers, users
• can be extended to non-human or invisible users
• Producers of software products
• anyone involved in managing, developing, marketing and service
of software products
• can be extended to third party participants for add on or testing.
 Quality expectations on the
consumer side
• Basic quality expectation of a user - a software system performs useful functions
as it is specified
• Two basic elements to this expectation:
It performs the right functions as specified - fits the user’s needs
It performs these specified functions correctly - performs its functions
reliably
• Related concepts to these two basic elements:
Validation
Verification
• Other expectations:
• Usability, ease of installation, inter-operability and adaptability etc
 Quality expectations on the
producer side
• For software producers
• most fundamental quality question is to fulfill their contractual obligations
• For product and service managers:
• adherence to pre-selected software process and relevant standards,
• proper choice of software methodologies,
• languages, and good design,
• integrity of product components and reduce coupling
• Tools, may be closely related to quality
• For other people on the producer side:
• their different concerns may also produce different quality views and
expectations, for example: Usability and modifiability may be paramount for
people involved with software service
• maintainability for maintenance personnel
End of Lecture