Session-12

Software Reliability and

Maintenance

Dr. Bharati Wukkadada

 contents
• reliability
• Reliability Metrics
• Reliability growth modelling
• Software reverse engineering
• Software maintenance costs
• Estimation of maintenance costs
Trustworthiness
or
dependability
 Reliability
• Reliability of a software specifies probability of
failure free operation for a given time duration
with respective to a software
• It is a dynamic characteristic / attribute in
nature where number of failures it encounter
 Software Reliability
• Reliability of S/w products denotes it’s
trustworthiness or dependability.

• It is defined as : “probability of the product

working correctly over a given period of time.”

• Reliability of the system improves, if the number

of defects in it are reduced
• the number of latent defects in the system.

• the perceived reliability of the product

• Reliability of the product also depends on the exact

location of the errors.

Can a poor user interface influence reliability? Yes. If the user is

unable to operate a service, it may be perceived as poor reliability
• The operational profile is a quantitative
characterization of how a system will be
used that shows how to increase
productivity and reliability and speed
development by allocating development
resources to function on the basis of use.
• LATENT DEFECT
• Testers, while ensuring the quality and performance of the product come across
various defects. Some are easy to identify while others are masked or hidden in
the product and require intensive measures to be uncovered. Among these
defects, there is an unusual one, that remains hidden until the circumstances
reveal it and it becomes a hindrance in the on-going process. This type of defect is
known as Latent Defect and is commonly observed in software testing. So, let us
understand what latent defect connotes to and the impact it has on the software
product, before as well as after the delivery.
• What is Latent Defect in Software Testing?
• One of the unusual defects found during software testing, Latent Defect is an error
that hasn’t occurred yet, and that can lead to failure whenever an accurate set of
conditions are not fulfilled. It is a systematic flaw that accompanies the software
during the production process and even passes the initial testing, including the
pre-production testing and extended use.
• Present in the system for a long time, latent defect is uncovered by the users in
the real-world environment, when a particular task is performed in the absence of
regular scenarios or due to an unusual or rare set of conditions.
• Latent Defect Example
• February has 28 days, except in a leap year. The system might is not able to
consider the leap year or an extra day in February, which results in a latent defect.
 PERCEIVED RELIABILTY

Frequently asked questions

1. Which three parts does user perceived reliability contribute to?
•A user interface that is easy to understand and operate,
•Lack of technical errors,
•No installation problems.

•Can a poor user interface influence reliability?

2. Can a poor user interface influence reliability?

Yes. If the user is unable to operate a service, it may be perceived as poor reliability though in the sense of technical
reliability will probably not.
3. How important is user perceived reliability for real-time person-to-person communication??

How important is user perceived reliability for real-time person-to-person communication??

It is important -- or very important. If the users cannot operate a system, they will not use it. For Real-time systems with
person-to-person communication, several results show it is essential.
4. How can user perceived reliability be improved?

How can user perceived reliability be improved?

it is here suggested 3 ways:
•Enhance user interface so it is easy to understand and operate – choose a system that can be operated
by an end-user, not only demonstrated. The user need to be able to use the service anytime, and that is
when the service is needed.
•Enhance technical reliability – this is mainly the service provider’s responsibility and can be revealed by
talking to other users. If their attitude to technical errors is not acceptable, do something with it. The
experience is that this will not solve by passive waiting.
•Be careful with installations – this is about installing, upgrading and remaining the installation. To install for
first time may result in new demands: Am I able of getting a public, fixed IP- address with my subscription and am
I able to traverse through the firewall? Upgrading the installation: This can lead to be asked question the installer
does not know answer to (opening certain Ports, identify microphone driver or license key received some time
earlier. Remaining the installation: This means that equipment used for the communication service (e. g.
microphone, camera) has been borrowed for other purposes and need to be found and reinstalled.
• It also depends upon how the product is used i.e. on
it’s execution profile.

– Executing only correctly implemented functions

reliability of the product will be high.

– If we select i/p data such that only those

function which contain errors are invoked, the
reliability of the system will be very low
• the reliability of s/w product is
observer-dependent & can not be determined
absolutely.

– E.g. Library Automation s/w

• Functions that library members use (issue
book, search book) are error free

• Function that librarian use (create member,

delete member) have many bugs.
Why software reliability is difficult to
measure
• Reliability improvement due to fixing a single
bug depends on where the bug is located

• Perceived reliability of a software product is

highly observer- dependent

• Reliability keep changing as the errors are

detected and fixed
Reliability Metrics
 Reliability metrics
• Reliability metrics are used to quantitatively expressed
the reliability of the software product. The option of
which metric is to be used depends upon the type of
system to which it applies & the requirements of the
application domain.
• According to ANSI, “Software Reliability is defined as
the probability of failure-free software operation for a
specified period of time in a specified environment”.
... Metrics used early can aid in detection and correction
of requirement faults that will lead to prevention of errors
later in the software life cycle.
 Reliability Metrics
• Metrics or techniques used to estimate the
quantitative reliability of s/w product are called
reliability metrics.

• By using this we can specify the level of reliability

required for a product can be specified in the SRS

• A good reliability metric should be

observer-independent, so that different people can
agree on degree of reliability that the system has.
(POFOD)

2 formulas
1. Rate of Occurrence of Failure (ROCOF)

– It measure the frequency of occurrence of

unexpected behavior (i.e. failure)
– It happens frequently

– Obtained by observing the behavior of s/w

product in operation over a specified time
interval &

– then calculating the total number of

failures during this interval.
2 . Mean Time to Failure (MTTF)
• It is the average time between two successive
failures, observed over a large number of
failures.[find out time of interval between 2
failures]

• Let the failure occur at the time instants

t1,t2,….,tn.
Then MTTF can be calculated as
∑ n ti+1 – ti
i=1
(n-1)

• Only run-time is considered in the time

measurements (Boot time etc. is not
considered)
3. Mean Time to Repair (MTTR)

– Once failure occurs, some time is

required to fix the error.

– MTTR measures the average time it

takes to track the errors causing the
failure & then to fix them.
4. Mean Time Between Failures (MTBF)

MTBF = MTTF + MTTR

– E.g. – if MTBF of 300 hrs. indicate that once a

failure occurs, the next failure is expected to
occur only after 300 hrs.
5. Probability of Failure on Demand (POFOD)

– It measure the likelihood of the system

failing when a service request is made.
– Considering here is “chances”
– Eg:Critical safety system

– E.g.- a POFOD of 0.001 means that 1 out

of every 1000 service request would
result in failure.
6. Availability

– It is the likelihood of the system made

available for use over a given period of time.

– This metric considers:

1. The no. of failures occurring during a time
interval &
2. The repair time of the system when the
failure occurs.

– This metric is used for systems like

telecommunication or operating system, which
are never supposed to be down
 Availability Management
Objective: ITIL Availability Management aims to
define, analyze, plan, measure and improve all
aspects of the availability of IT services.

Part of: Service Design

Process Owner: Availability Manager

 Don’t
forget
•Service availability is at the core of customer satisfaction and
business success:
poor service performance is defined as being
unavailable.
 ITIL Availability
Management sub-processes and
their process objectives:
• Design Services for Availability
• to fulfill the agreed availability levels.

• Availability Testing

• Availability Monitoring and Reporting

– comparing achieved vs. agreed availability
 • Availability Management Process

1. Reactive activities
2. Proactive activities
 How to measure
it ?
Availability: the ability of a service, component to perform its agreed
function when required.

(Agreed Service Time (AST) –

downtime) X 100
Availability (%) = %
-----------------------------------------------
--------Service Incidents (MTBSI)
Mean Time Between
or Mean Time BetweenAgreed Service
Failures Time (AST)
(MTBF):

Available time in hours

Reliability (MTBSI in hours) = ------------------------------------------
Number of breaks

Available time in HRS– Total downtime in HRS

Reliability (MTBF in hours) = -----------------------------------------------------
Number of breaks
 The Availability Management process depends heavily on
the measurement of service and component
achievements with regard to availability.

situation where a 24 x 7 service has been running for a period of 5,020 hours
with only two breaks, one of 6 hours and one of 14 hours, would give the
following figures:

Availability = (5,020–(6+14)) / 5,020 x 100 = 99.60%

Reliability (MTBSI) = 5,020 / 2 = 2,510 hours

Reliability (MTBF) = 5,020–(6+14) / 2 = 2,500 hours

Microsoft Online Services
IT Online Services availability

Service
level
agreement
 Classification of Failures
1. Transient
– Occur only for certain input values while
invoking a function of the system.

2. Permanent
– Occur for all input values while invoking a
function of a system.

3. Recoverable
– When recoverable failures occur, the system
recovers with or without operator intervention.
4. Unrecoverable
– The system may need to be restarted.

5. Cosmetic
– These failures cause minor irritations, &
do not lead to incorrect results

– E.g. – mouse button need to be clicked

twice to invoke a specific function
Reliability Growth Modeling
 Reliability growth Model
• A reliability growth model is a mathematical model of how
software reliability improves as errors are detected and
repaired.
• A reliability growth model can be used to predict when (or if
at all) a particular level of reliability is likely to be attained.
• Thus, reliability growth modeling can be used to determine
when to stop testing to attain a given reliability level.
• Although several different reliability growth model are
availbe , we will discuss few:

1. Jelinski-moranda model
2. Little wood and Verall’s model
3. Step function model
 Reliability Growth Modeling
• It is a mathematical model of how the s/w reliability
improves as errors are detected & repaired.

• This model tells us how the system reliability changes

over time during the testing process.

• It can be used to predict when a particular level of

reliability is likely to be attained

• It is used to determine when to stop testing to attain

a given reliability level.
 Jelinski & Moranda Model

• This is a simple step function model where the

reliability increases by a constant increment each
time a fault is discovered & repaired

• This model assumes that s/w repairs are always

correctly implemented so that the no. of s/w faults
& associated failures decreases in each new version
of the system

• It assumes that all errors contributes equally to

reliability growth
Rate of occurrence of failure
(ROCOF)
Rate of occurrence of failure (ROCOF)
It is the number of failures appearing in a
unit time interval. The number of
unexpected events over a specific time
of operation. ... A ROCOF of 0.02 mean
that two failures are likely to occur in
each 100 operational time unit steps. It is
also called the failure intensity metric
 Littlewood & Verall’s model
• This model allows for negative reliability growth to
reflect the fact that when a repair is carried out, it
may introduce additional errors.

• It also says that as faults are repaired, the average

improvement in reliability per repair decreases.

• This shows that each repair does not result in equal

amount of reliability improvement
• It is a constant growth model finding error
and repair and this is not feasible in real world
 Reliability Testing Tutorial: What
is, Methods, Tools, Example
• checks whether the software can perform a
failure-free operation for a specified period of
time in a specified environment.
 Reliability Testing can be
categorized into three segments

1.Modelling
2.Measurement
3.Improvement
 Factors Influencing Software
Reliability
1. The number of faults presents in the
software

2. The way users operate the system

 Why to do Reliability Testing

1. To find the structure of repeating failures.

2. To find the number of failures occurring is the
specified amount of time.
3. To discover the main cause of failure
4. To conduct Performance Testing of various
modules of software application after fixing
defect
 Types of reliability Testing

• Feature Testing
• Load Testing
• Regression Test
 How to do Reliability Testing

• Establish reliability goals

• Develop operational profile
• Plan and execute tests
• Use test results to drive decisions
• The key parameters involved in Reliability
Testing are:-
– Probability of failure-free operation
– Length of time of failure-free operation
– The environment in which it is executed
 Step 1) Modelling

• 1. Prediction Modelling
• 2. Estimation Modelling
Issues Prediction Models Estimation Models
Data Reference It uses historical data It uses current data from
the software development.
When used in It will be usually created It will be usually used at the
Development Cycle before the development or later stage of Software
testing phases. Development Life Cycle.
Time Frame It will predict the reliability It will predict the reliability
in the future. either for the present time
or in the future time.
 Step 2) Measurement

1. Product Metrics:-
Software size
Function point metric
complexity
Test coverage metric
2. Project Management Metrics
3. Process Metrics
4. Fault and Failure Metrics
 Step 3) Improvement

• Improvement completely depends upon the

problems occurred in the application or
system, or else the characteristics of the
software.
 Example Methods for Reliability
Testing
• There are mainly three approaches used for
Reliability Testing
1. Test-Retest Reliability
2. Parallel Forms Reliability
3. Decision Consistency

• DECISION CONSISTENCY
• After doing Test-Retest Reliability and
Parallel Form Reliability, we will get a
result of examinees either pass or fail.

• It is the reliability of this classification

decision that is estimated in decision
consistency reliability.
 5 aspects of quality in a business
context:
• Producing – providing something.

• Checking – confirming that something has been done correctly.

• Quality Control – controlling a process to ensure that the outcomes are

predictable.

• Quality Management – directing an organization so that it optimizes its

performance through analysis and improvement.

• Quality Assurance – obtaining confidence that a product or service

will be satisfactory. (Normally performed by a purchaser)
SOFTWARE REVERSE ENGG
✔
Reverse Engineering:
Reverse Engineering is also known as backward
engineering, is the process of forward
engineering in reverse.
✔ In this, the information are collected from the
given or exist application.
✔ It takes less time than forward engineering to
develop an application. In reverse engineering the
application are broken to extract knowledge or its
architecture.
Reverese engineering
REVERSE ENGG
✔ Program comprehension
✔ Analysis existing s/w with a view to understand its design
and specification
✔ A process which analyses a product t find out the design
aspects and its functions
✔ Builds a program db and generates information from this
REVERSE ENGG goals: REVERSE ENGG
✔ Cope up with complexity Activities:
✔ Recover lost information ✔ Understanding process
✔ Detect the effects ✔ Understanding data
✔ Synthesize higher ✔ Understanding user
abstraction interfaces
✔ Facilitate reuse
 Software
Maintenanc
e
 Software
Maintenanc
e
 Software maintenance
Software maintenance is widely accepted part of SDLC now a days. It stands
for all the modifications and updations done after the delivery of software
product. There are number of reasons, why modifications are required, some
of them are briefly mentioned below:

•Market Conditions - Policies, which changes over the time, such as taxation
and newly introduced constraints like, how to maintain bookkeeping, may
trigger need for modification.
•Client Requirements - Over the time, customer may ask for new features or
functions in the software.
•Host Modifications - If any of the hardware and/or platform (such as
operating system) of the target host changes, software changes are needed
to keep adaptability.
•Organization Changes - If there is any business level change at client end,
such as reduction of organization strength, acquiring another company,
organization venturing into new business, need to modify in the original
software may arise.
Definitio
ns
The act of keeping, or the expenditure
required to keep, an asset in condition to
perform efficiently the service for which it is
used.

Software Maintenance is the process of

modifying a software product after it has
been delivered to the customer. The main
purpose of software maintenance is to
modify and update software application after
delivery to correct faults and to improve
performance.
 TYPES OF MAINTENANCE
In a software lifetime, type of maintenance may vary based on its nature. It may be
just a routine maintenance tasks as some bug discovered by some user or it may
be a large event in itself based on maintenance size or nature. Following are some
types of maintenance based on their characteristics:
•Corrective Maintenance 21%- This includes modifications and updations done in
order to correct or fix problems, which are either discovered by user or concluded
by user error reports.
•Adaptive Maintenance 25% - This includes modifications and updations applied to
keep the software product up-to date and tuned to the ever changing world of
technology and business environment. [Porting and migration]
•Perfective Maintenance 50%- This includes modifications and updates done in
order to keep the software usable over long period of time. It includes new features,
new user requirements for refining the software and improve its reliability and
performance[ enhancement and scalbiility.]
•Preventive Maintenance 4% - This includes modifications and updations to
prevent future problems of the software. It aims to attend problems, which are not
significant at this moment but may cause serious issues in future.[ s/w re-engg]
Types of Maintenance
1.Corrective maintenance:
2.Adaptive maintenance
3.Preventive maintenance
4.Perfective maintenance
 Software Re-engineering

• When we need to update the software to keep it to the current

market, without impacting its functionality, it is called software
re-engineering. It is a thorough process where the design of
software is changed and programs are re-written.
• Legacy software cannot keep tuning with the latest technology
available in the market. As the hardware become obsolete, updating
of software becomes a headache. Even if software grows old with
time, its functionality does not.
• For example, initially Unix was developed in assembly language.
When language C came into existence, Unix was re-engineered in
C, because working in assembly language was difficult.
• Other than this, sometimes programmers notice that few parts of
software need more maintenance than others and they also need
re-engineering.

•
 Re-Engineering Process
•Decide what to re-engineer. Is
it whole software or a part of it?
•Perform Reverse Engineering,
in order to obtain specifications
of existing software.
•Restructure Program if
required. For example,
changing function-oriented
programs into object-oriented
programs.
•Re-structure data as required.
•Apply Forward
engineering concepts in order
to get re-engineered software.
 Cost of Maintenance
Reports suggest that the
cost of maintenance is high.
A study on estimating
software maintenance found
that the cost of maintenance
is as high as 67% of the cost
of entire software process
cycle.
On an average, the cost of
software maintenance is
more than 50% of all SDLC
phases. There are various
factors, which trigger
maintenance cost go high,
such as:
Real-world factors affecting Maintenance Cost
• The standard age of any software is considered up to 10 to 15 years.
• Older softwares, which were meant to work on slow machines with less
memory and storage capacity cannot keep themselves challenging against
newly coming enhanced softwares on modern hardware.
• As technology advances, it becomes costly to maintain old software.
• Most maintenance engineers are newbie and use trial and error method to
rectify problem.
• Often, changes made can easily hurt the original structure of the software,
making it hard for any subsequent changes.
• Changes are often left undocumented which may cause more conflicts in
future.
Software-end factors affecting Maintenance Cost
• Structure of Software Program
• Programming Language
• Dependence on external environment
• Staff reliability and availability
 Maintenance Activities
• IEEE provides a framework for sequential maintenance
process activities. It can be used in iterative manner and
can be extended so that customized items and
processes can be included.
• These activities go hand-in-hand with each of the following phase:
• Identification & Tracing - It involves activities pertaining to identification of requirement of
modification or maintenance. It is generated by user or system may itself report via logs or
error messages.Here, the maintenance type is classified also.
• Analysis - The modification is analyzed for its impact on the system including safety and
security implications. If probable impact is severe, alternative solution is looked for. A set of
required modifications is then materialized into requirement specifications. The cost of
modification/maintenance is analyzed and estimation is concluded.
• Design - New modules, which need to be replaced or modified, are designed against
requirement specifications set in the previous stage. Test cases are created for validation
and verification.
• Implementation - The new modules are coded with the help of structured design created in
the design step.Every programmer is expected to do unit testing in parallel.
• System Testing - Integration testing is done among newly created modules. Integration
testing is also carried out between new modules and the system. Finally the system is
tested as a whole, following regressive testing procedures.
• Acceptance Testing - After testing the system internally, it is tested for acceptance with the
help of users. If at this state, user complaints some issues they are addressed or noted to
address in next iteration.
• Delivery - After acceptance test, the system is deployed all over the organization either by
small update package or fresh installation of the system. The final testing takes place at
client end after the software is delivered.
• Training facility is provided if required, in addition to the hard copy of user manual.
• Maintenance management - Configuration management is an essential part of system
maintenance. It is aided with version control tools to control versions, semi-version or patch
management.
 Maintenance models-do it urself
• Quick-fix model
• Iterative enhancement model
• Reuse oriented model
• Boehm’s model
• Taute maintenance model
 Software Breakdown
Causes
• basic conditions neglected

• inadequate skills

• operating standards not

followed

• deterioration unchecked

• inherent design weakness

 Maintenance costs
Maintenance costs are usually greater than
development costs by a factor of 2 to 100.

•The costs arise from both technical and nontechnical

factors.
A deployed system is expensive to change
High cost of breaking an already working system
Maintenance costs increase over time and as the system
evolves.

Reasons:
Maintenance changes ,degrades the original system structure.
Aging software results in high support costs.
 Maintenance cost factors
Team stability
Maintenance costs are reduced if the same staff are involved with them for
some time.

Contractual responsibility
The developers of a system may have no contractual responsibility for
maintenance so there is no incentive to design for future change.

Staff skills
Maintenance staff are often inexperienced and have limited domain
knowledge.

Program age and structure

As programs age, their structure is degraded and they become harder to
understand and change.
 Strategies to reduce maintenance
costs:
- Correct slight defects in parts and jigs.
- Ensure basic equipment conditions are maintained
- Review basic operations
- Conduct physical analysis
- Adopt an analytical approach
 Maintenance
Problems
• Someone else's program.
• Developer not available.
• Proper documentation doesn't exist.
• Not designed for change.
• Maintenance activity not highly
regarded.
 Software Maintanance - Change Management

• Objective: Change Management

– aims to control the lifecycle of all Changes.
– to enable beneficial Changes to be made, with
minimum disruption to IT services.

• Part of: Service Transition

• Process Owner: Change Manager
Request for
change
Change
Management
process
• What is reliability
• Explain Reliability metrics
• What is Growth modelling
• Wha is software maintenance and estimation
of maintenance costs