3/21/2022

3 4 5 Standards
1 2 Life cycle
Infrastructure Management and
Overview components
components components Organizing
7
6 8 Test 9
Dynamic
Static tesing management Tools
testing

SQA Architecture

Slide 2

1
 3/21/2022

Learning objectives
 Explain the objectives of project progress control, software
quality metrics, costs of software quality measurements
 Explain the components of project progress control
 Classify software quality metrics
 Compare the classic model to the extended cost model of
software quality

Slide 3

References
 Galin (2004). Software Quality Assurance from theory to
implementation. Pearson Education Limited
 Ian Sommerville (2011). Software engineering. Ninth
Edition. Addison-Wesley

Slide 4

2
 3/21/2022

Câu hỏi hiểu bài

1. Phân biệt process metric và product metric.
2. Hiểu và giải thích được 3-5 công thức cụ thể của 2 loại độ
đo (process metric và product metric)
3. Hãy phân loại các chi phí liên quan đến chất lượng phần
mềm theo mô hình cổ điển. Cho ví dụ từng loại

Slide 5

1 2 3 4 5

Contents
6 7 8 9

 Project progress control

 Software quality metrics
 Software quality costs

Slide 6

3
 3/21/2022

Project progress control

 Objective
 immediate: early detection of irregular events
 long-term: initiation of corrective actions
 The main components
 risk management activities
 project schedule control
 project resource control
 project budget control

Slide 7

Project progress control

Gantt chart for FR–ATM service interworking test project Slide 8

4
 3/21/2022

Control of risk management activities

 Refers to the software development risk items identified in
the preproject stage, listed in contract review and project
plan documents, together with other risk items
 Systematic risk management activities required:
 periodic assessment about the state of the software risk
items
 based on this reports the project managers are expected to
intervene and help arrive at a solution in the more extreme
cases

Slide 9

Project schedule control

 Deals with the project’s compliance with its approved and
contracted timetables
 Based mainly on milestones in addition to periodic reports
 milestones set in contracts, especially dates for delivery,
receive special emphasis
 Control activities should be focused on critical delays
(which may effect final completion of the project)
 Management interventions:
 allocation of additional resources
 renegotiating the schedule with the customer

Slide 10

5
 3/21/2022

Project resource control

 Main control items:
 human resources
 special development and testing equipment (real-time
systems; firmware)
 Control is based on periodic reports of resources used
 True extent of derivations can only be assessed from the
point of view of the project progress
 Internal composition of the resource also counts
(percentage of senior staff involved, …)

Slide 11

Project budget control

 Based on comparison of actual with scheduled costs
 The main budget items to be controlled
 human resources
 development and testing facilities
 purchase of COTS software
 purchase of hardware
 payments to subcontractors
 How to control?
 based on the milestone reports and other periodic reports

Slide 12

6
 3/21/2022

Progress control of internal

projects and external participants
 Problem: In practice project control provides only a limited
view of the progress of internal software development and
an even more limited view on the progress made by
external project participants
 More significant efforts are required in order to achieve
acceptable levels of control for an external project
participant due to the more complex communication and
coordination
 Project progress control of external participants must focus
mainly on the project’s schedule and the risks identified in
planned project activities
Slide 13

Project progress control implementation

 Allocation of responsibilities for
 person or management unit for progress control
 frequency of progress reports required from each of the
unit levels and administrative level
 situations requiring the project leader to report immediately
to management
 situations requiring lower-level management to report
immediately to upper-level management
 Management audits of project progress
(1) how well progress reports are transmitted by project
leaders and by lower to upper-level management
(2) specific management control activities to be initiated
Slide 14

7
 3/21/2022

Computerized project progress control

 Required for non trivial projects
 Automation can reduce costs considerably
 Examples of services
 control of risk management activities
 project schedule control
 project resource control
 project budget control

Slide 15

1 2 3 4 5

Contents
6 7 8 9

 Project progress control

 Software quality metrics
 Software quality costs

Slide 16

8
 3/21/2022

Software quality metrics

 Definition and objectives
 Classification
 Process metrics
 Product metrics
 Implementation of software quality metrics
 Limitations of software metrics

Slide 17

Software quality metrics

 Definition (IEEE, 1993)
 A quantitative measure of the degree to which a system,
component, or process possesses a given attribute
 Main objectives
 to facilitate management control as well as planning and
execution of the appropriate managerial interventions
 to identify situations for development or maintenance
process improvement (preventive or corrective actions)

Slide 18

9
 3/21/2022

Classification of software quality metrics

 Classification by phases of software system
 process metrics – metrics related to the software
development process
 product metrics – metrics related to software maintenance
 Classification by subjects of measurements
 quality
 timetable
 effectiveness (of error removal and maintenance services)
 productivity

19

Software Volume/Errors Counted

 Software Volume Measures: use KLOC or Function Points
 KLOC – classic metric that measures the size of software by
thousands of code lines
 NFP (Number of Function Points) – a measure of the
development resources (human resources) required to
develop a program, based on the functionality specified for
the software system

 Errors Counted Measures: relate to the number of errors

or the weighted number of errors

Slide 20

10
 3/21/2022

Example: Error Counted Measures

Calculation of NCE Calculation of WCE
Error severity class Number of Errors Relative Weighted
Weight Errors
a b c D=bxc
low severity 42 1 42
medium severity 17 3 51
high severity 11 9 99
Total 70 --- 192

NCE 70 --- ---

WCE --- 192
Number of code errors (NCE) vs. weighted number of code errors (WCE)
Slide 21

Process metrics
Categories
1. Software process quality metrics
 error density metrics
 error severity metrics
2. Software process timetable metrics
3. Software process error removal effectiveness metrics
4. Software process productivity metrics

22

11
 3/21/2022

Process metrics
1. Quality metrics: Error density metrics
Code Name Calculation formula
NCE
CED Code Error Density CED = -----------
KLOC
NDE
DED Development Error Density DED = -----------
KLOC
WCE
WCED Weighted Code Error Density WCDE = ---------
KLOC
WDE
WDED Weighted Development Error Density WDED = ---------
KLOC
WCE
WCEF Weighted Code Errors per Function Point WCEF = ----------
NFP
Weighted Development Errors per WDE
WDEF Function Point WDEF = ----------
NFP
NCE = the number of code errors detected by code inspections and testing.
NDE = total number of development (design and code) errors detected in the development process.
WCE = weighted total code errors detected by code inspections and testing.
Slide 23
WDE = total weighted development (design and code) errors detected in development process.

Process metrics
1. Quality metrics: Error density metrics
 Example:

Calculation of CED Calculation of WCED

Measures and metrics (Weighted Code Error
(Code Error Density) Density)

NCE 70 --

WCE -- 192

KLOC 40 40

CED (NCE/KLOC) 1.75 --

WCED (WCE/KLOC) -- 4.8

Slide 24

12
 3/21/2022

Process metrics
1. Quality metrics: Error density metrics
 The concept of indicator
 A software development department may apply two
alternative metrics for calculation of code error density: CED
and WCED

 The unit has to determine indicators for unacceptable

software quality:
 CED > 2 and WCED > 4

25

Process metrics
1. Quality metrics: Error severity metrics
Code Name Calculation formula
ASCE Average Severity of Code WCE
ASCE = -----------
Errors NCE

ASDE Average Severity of WDE

ASDE = -----------
Development Errors NDE

NCE = the number of code errors detected by code inspections and testing
NDE = total number of development (design and code) errors detected in
the development process
WCE = weighted total code errors detected by code inspections and testing
WDE = total weighted development (design and code) errors detected in
development process

Slide 26

13
 3/21/2022

Process metrics
2. Timetable metrics
Code Name Calculation formula
MSOT
TTO Time Table Observance TTO = -----------
MS

Average Delay of Milestone TCDAM

ADMC Completion ADMC = -----------
MS

MSOT = milestones completed on time.

MS = total number of milestones.
TCDAM = total completion delays (days, weeks, etc.) for all milestones.

Slide 27

Process metrics
3. Error removal effectiveness metrics
Code Name Calculation formula
Development Errors Removal NDE
DERE Effectiveness
DERE = ----------------
NDE + NYF
Development Weighted WDE
DWERE Errors Removal Effectiveness DWERE = ------------------
WDE+WYF

NDE = total number of development (design and code) errors detected in

the development process.
WCE = weighted total code errors detected by code inspections and testing.
WDE = total weighted development (design and code) errors detected in
development process.
NYF = number software failures detected during a year of maintenance
service.
WYF = weighted number of software failures detected during a year of
maintenance service. Slide 28

14
 3/21/2022

Process metrics
4. Productivity metrics
Code Name Calculation formula
DevH
DevP Development Productivity DevP = ----------
KLOC
Function point DevH
FDevP Development Productivity FDevP = ----------
NFP
ReKLOC
CRe Code Reuse Cre = --------------
KLOC
ReDoc
DocRe Documentation Reuse DocRe = -----------
NDoc
DevH = total working hours invested in the development of the software
system.
ReKLOC = number of thousands of reused lines of code.
ReDoc = number of reused pages of documentation.
NDoc = number of pages of documentation. 29

Product metrics

 Refer to the software system’s operational phase

 Customer services are of two main types:
 Help desk services (HD)
 software support by instructing customers regarding the
method of application of the software and solution for
customer implementation problems
 HD metrics are based on all customer calls
 Corrective maintenance services
 correction of software failures identified by customers/users or
detected by the customer service team prior to their discovery
be the customer
 corrective maintenance metrics are based on failure reports

Slide 30

15
 3/21/2022

Product metrics
Categories
1. HD quality metrics:
 HD calls density metrics - measured by the number of calls
 HD calls severity metrics - the severity of the HD issues raised
 HD success metrics – the level of success in responding to HD calls
2. HD productivity and effectiveness metrics
3. Corrective maintenance quality metrics
 software system failures density metrics
 software system failures severity metrics
 failures of maintenance services metrics
 software system availability metrics
4. Corrective maintenance productivity and effectiveness
metrics
Slide 31

Product metrics
1. HD quality metrics: HD calls density metrics
Code Name Calculation Formula
NHYC
HDD HD calls density HDD = --------------
KLMC

WHYC
WHDD Weighted HD calls density WHYC = ------------
KLMC

Weighted HD calls per WHYC

WHDF function point WHDF = ------------
NMFP

NHYC = the number of HD calls during a year of service.

KLMC = thousands of lines of maintained software code.
WHYC = weighted HD calls received during one year of service.
NMFP = number of function points to be maintained.
Slide 32

16
 3/21/2022

Product metrics
1. HD quality metrics: Severity of HD calls metrics

Code Name Calculation Formula

WHYC
ASHC Average severity of HD calls ASHC = --------------
NHYC

WHYC = weighted HD calls received during one year of service.

NHYC = the number of HD calls during a year of service.

Slide 33

Product metrics
1. HD quality metrics: HD success metrics
 The capacity to solve problems raised by customer calls
within the time determined in the service contract
Code Name Calculation Formula
NHYOT
HDS HD service success HDS = --------------
NHYC

NHYNOT = number of yearly HD calls completed on time during one year of

service.
NHYC = the number of HD calls during a year of service.

Slide 34

17
 3/21/2022

Product metrics
2. HD productivity and effectiveness metrics
Code Name Calculation Formula
HDYH
HDP HD Productivity HDP= --------------
KLNC
HDYH
FHDP Function Point HD Productivity FHDP = ----------
NMFP
HDYH
HDE HD effectiveness HDE = --------------
NHYC

HDYH = total yearly working hours invested in HD servicing of the software

system.
KLMC = thousands of lines of maintained software code.
NMFP = number of function points to be maintained.
NHYC = the number of HD calls during a year of service.
Slide 35

Product metrics
3. Corrective maintenance quality metrics
 Software system failures density metrics
Code Name Calculation Formula
NYF
SSFD Software System Failure Density SSFD = --------------
KLMC
WYF
WSSFD
Weighted Software System Failure WFFFD = ---------
Density KLMC
WYF
WSSFF
Weighted Software System Failures WSSFF = ----------
per Function point NMFP
NYF = number of software failures detected during a year of maintenance service.
WYF = weighted number of yearly software failures detected during one year of
maintenance service.
NMFP = number of function points designated for the maintained software.
KLMC = thousands of lines of maintained software code.
Slide 36

18
 3/21/2022

Product metrics
3. Corrective maintenance quality metrics
 Software system failure severity metrics

Code Name Calculation Formula

Average Severity of WYF
ASSSF Software System Failures ASSSF = --------------
NYF

NYF = number of software failures detected during a year of maintenance

service.
WYF = weighted number of yearly software failures detected during one
year.

Slide 37

Product metrics
3. Corrective maintenance quality metrics
 Failures of maintenance services metrics

Code Name Calculation Formula

Maintenance Repeated RepYF
MRepF repair Failure metric - MRepF = --------------
NYF

RepYF = Number of repeated software failure calls (service failures).

NYF = number of software failures detected during a year of maintenance
service.

Slide 38

19
 3/21/2022

Product metrics
3. Corrective maintenance quality metrics
 Software system availability metrics
Code Name Calculation Formula
NYSerH - NYFH
FA Full Availability FA = -----------------------
NYSerH
NYSerH - NYVitFH
VitA Vital Availability VitA = -----------------------------
NYSerH
NYTFH
TUA Total Unavailability TUA = ------------
NYSerH
NYSerH = Number of hours software system is in service during one year.
NYFH = Number of hours where at least one function is unavailable (failed) during one year,
including total failure of the software system.
NYVitFH = Number of hours when at least one vital function is unavailable (failed) during
one year, including total failure of the software system.
NYTFH = Number of hours of total failure (all system functions failed) during one year.
NYFH ≥ NYVitFH ≥ NYTFH.
1 – TUA ≥ VitA ≥FA Slide 39

Product metrics
4. Software corrective maintenance productivity and
effectiveness metrics

Code Name Calculation Formula

Corrective Maintenance CMaiYH
CMaiP Productivity
CMaiP = ---------------
KLMC
Function point Corrective CMaiYH
FCMP Maintenance Productivity
FCMP = --------------
NMFP
Corrective Maintenance CMaiYH
CMaiE Effectiveness
CMaiE = ------------
NYF

CMaiYH = Total yearly working hours invested in the corrective maintenance of the
software system.
NYF = number of software failures detected during a year of maintenance service.
NMFP = number of function points designated for the maintained software.
KLMC = Thousands of lines of maintained software code.
Slide 40

20
 3/21/2022

1 2 3 4 5

Contents
6 7 8 9

 Project progress control

 Software quality metrics
 Software quality costs

Slide 41

Costs of software quality

 Objectives of cost of software quality metrics
 The classic model
 An extended model
 Application of a cost of software quality system
 Problems in the application of cost of software quality
metrics

Slide 42

21
 3/21/2022

Objectives of cost of software quality

 control organization-initiated costs to prevent and detect
software errors
 evaluation of the economic damages of software failures
as a basis for revising the SQA budget
 evaluation of plans to increase or decrease SQA activities
or to invest in a new or updated SQA infrastructure on the
basis of past economic performance

Slide 43

Classic model of cost of software quality

Prevention
costs
Costs of
control Appraisal
costs
Cost of
software
quality Internal
Costs of failure costs
failure of
control costs External
failure costs
Slide 44

22
 3/21/2022

Costs of control
Prevention costs
 Investments in development of new or improved SQA infrastructure
 procedures and work instructions
 support devices: templates, checklists etc
 so ware configura on management system
 software quality metrics
 Regular implementation of SQA preventive activities:
 instruction of new employees in SQA subjects
 certification of employees
 consultations on SQA issues to team leaders and others
 Control of the SQA system through performance of:
 internal quality reviews
 external quality audits by customers and SQA system certification
organizations
 management quality reviews

Slide 45

Costs of control
Appraisal costs
 Costs of reviews:
 formal design reviews (DRs)
 peer reviews (inspections and walkthroughs)
 expert reviews
 Costs of software testing:
 unit, integration and software system tests
 acceptance tests (carried out by customers)
 Costs of assuring quality of external participants
 subcontractors, suppliers of COTS software systems and
reusable software modules, customers
Slide 46

23
 3/21/2022

Costs of failure of control costs

Internal failure costs
 Represent the costs of error correction subsequent to
formal examinations of the software during its
development, prior to the system’s installation at the
customer’s site
 costs of redesign or design corrections subsequent to design
review and test findings
 costs of re-programming or correcting programs in response
to test findings
 costs of repeated design review and re-testing (regression
tests)

Slide 47

Costs of failure of control costs

External failure costs
 Entail the costs of correcting failures detected by
customers or maintenance teams after the software
system has been installed at customer sites
 Typical external failure costs (“overt” cost)
 resolution of customer complaints
 correction of software bugs
 correction of software failures after the warranty period
 damages paid to customers
 reimbursement of customer's purchase costs
 insurance against customer's claims
…

Slide 48

24
 3/21/2022

Costs of failure of control costs

External failure costs (cont’d)
 Typical examples of hidden external failure costs
 reduction of sales to customers that suffered from software
failures
 severe reduction of sales motivated by the firm's damaged
reputation
 increased investment in sales promotion to counter the
effects of past software failures

Slide 49

Galin’s extended model for cost of

software quality
Prevention costs

Costs of
Control Appraisal costs
costs
Managerial
Cost of preparations and
software control costs
quality
Internal failure
Costs of costs
Failure of External failure
control costs
costs
Managerial
failure costs
Slide 50

25
 3/21/2022

Galin’s extended model

Managerial preparation and control costs
 Costs of carrying out contract reviews
 Costs of preparing project plans, including quality plans
 Costs of periodic updating of project and quality plans
 Costs of performing regular progress control
 Costs of performing regular progress control of external
participants’ contributions to projects

Slide 51

Galin’s extended model

Managerial failure costs
 Unplanned costs for professional and other resources, resulting
from underestimation of the resources in the proposals stage
 Damages paid to customers as compensation for late project
completion, a result of the unrealistic schedule in the
Company’s proposal
 Damages paid to customers as compensation for late
completion of the project, a result of management’s failure to
recruit team members
 Domino effect: Damages to other projects planned to be
performed by the same teams involved in the delayed projects.
The domino effect may induce considerable hidden external
failure costs
Slide 52

26
 3/21/2022

Application of a cost of
software quality system
 Definition of a cost of software quality model and
specification of cost items
 Definition of the method of data collection for each cost
item
 Implementation of a cost of software quality system,
including thorough follow up
 Actions taken in response to the findings

Slide 53

Problems in the application of cost of

software quality metrics
 General problems
 inaccurate and/or incomplete identification and classification
of quality costs
 negligent reporting by team members
 biased reporting of software costs, especially of “censored”
internal and external costs
 biased recording of external failure costs - “camouflaged”
compensation of customers for failures

Slide 54

27
 3/21/2022

Slide 55

28