Yekatit 12 Hospital

Medical College
Health Care Quality Management
Master’s Program
 Health System
Performance Analysis
By: Abiyou Kiflie (MD, MPH)
Country Director, Institute for Healthcare Improvement
Oct 14 – Nov 5, 2023
Knowing Each Other

Your name Back-ground Your job

education

Assign four individuals to tally and summarize the individual characteristics into a data
that gives meaning
 • Data management in rows • Graph management
Prerequisites: and columns • Changing graph type
manage data on • Appropriate format of • Adding labels, titles,
excel data/numbers values …
• Creating new columns • Formatting graphs
spreadsheet support by formulas • Axis scale management

• Arithmetic formulas • Utilizing dragging function

• Calculating indicators
(numerator,
denominator, multiplier) • Explore the function formulas

• Selecting a data that best fits • Advance your skills over time
your graphs
 Introduction to data in improvement - review

Understanding Variation
Course
Syllabus
Shewhart charts

Reference/textbook: Practical sessions on Shewhart chat

The Health Care Data Guide: learning from
data for improvement; Provost, Murray
 Methods and Expectations

Self learning process Attendance of in person + virtual class Laptop

Assignments and exercise completion Guiding tips to initiate self learning Laptop
Discussions Opportunity for Q&A SPC software
Readings X-Y labeler
Practice
Introduction to data in
Improvement

Review of your Quality Improvement Course

What is required for improvement?
• Healthcare professionals are expected to
improve the care they provide using their
medical knowledge
• Salesperson will do same improvement in Subject Matter Knowledge
sales
Improvement
• Deming proposed a body of knowledge
called “System of Profound Knowledge” Profound Knowledge
• The ability to make improvement is
enhanced by combining subject matter
knowledge and profound knowledge in
creative ways
Profound
Knowledge
Appreciation
for a system
• Profound - deep insight this knowledge
offers into how to make the changes that
will result in improvement in a variety of Building
settings Knowledge Psychology
of Change

• Profound knowledge is the interplay of

the theories of systems, variation,
knowledge and psychology Understanding
Variation

• The System of Profound Knowledge

provides a lens.
Data
• Data are documented
observations or the results of
performing a measurement
process.
 Physical
World
Observe
Data Behavioral
and/or Data
World
Measure
Perceptual
World
 1. Qualitative Data – nonnumeric data
Examples
• They have difficulty of answering the questions
• The patient is suffering from pain

Types of
Reasons to use qualitative date instead of quantitative:
Data • Quantitative data might be difficult or expensive to
obtain
• Information of interest is better obtained by observation

2. Quantitative Data – numeric data

• Usually, preferred data for learning
Traditional Data Typology
1. Nominal Data 3. Interval Data
• Non-numeric • Data measured along a scale
• Placed in distinct and separate • Each position within the scale is at equal
distance
categories
• It has no true zero value
• No order • Ex: personal experience scales, temperature
• Example: smoker / non-smoker, gender,
blood group
4. Ratio Data
• Numbers can be compared as multiples of
one another
2. Ordinal Data • Unlike interval data, ratio data has a true zero:
• Data suggest certain order zero is an absolute below which there are no
• Ranked in a meaningful way meaningful values
• The order matters but not the • Example: ht, wt, blood sugar, waiting time in
difference between values days; age
 1. Classification Data
• Attributes are recorded in one or two classes
• Go/no go, complete/incomplete, good/bad

2. Count
Data Types • Focuses on attributes that are unusual or undesirable
• Number of mistakes, number of accidents
(Improvement • If data is counted with a different intent like to know workload,
productivity, those counts are considered as continuous
Science ) because of their intent
3. Continuous
• Measured numerical value is recorded
• Measurement device or scale is required
• Time, money, physical properties
Data Classification in Science of
Improvement
Data

Qualitative Quantitative
Data Data

Classification
Rank data Rank data Continuous
Data Count data
(Ordinal data) (Ordinal data) data
(Nominal)
Operational Definitions
• An agreement as to how data will be collected to maintain data
collection consistency.
• It gives communicable meaning to concept (like error, waiting time,
appropriate care) by specifying how a concept is applied in a set of
circumstances.
Operational Definitions
Indicators
• Elements of an indicator
definition:
• Indicator name/definition
• Numerator
• Denominator
• Multiplier (if needed)
• Operational definition and
interpretation
• Disaggregation
• Source of data (data elements)
 Indicator
Definition in
Ethiopia
HMIS
Data use
• Data could be used for:
• Learning and improvement
• Judgement
• Research
• Examples of data driven decision make:
1. Allocating budget to address the most fatal disease
2. Formulating strategy for the number one cause of admission across the country
3. Firing a CEO of a hospital because of rampant deaths due to wrong allocation of
budget that leads to stockout of key supplies
4. Diabetic clinic start providing lab requests for next schedules because it is
found that FBS measurement was the cause for long waiting times
 Data for
Improvement,
Accountabilit
y and
Research
Vector of Measures
• Set of key organizational measures that are balanced across the
various dimensions and stakeholders of an organization
• Should reflect a balanced view of the purpose
• Needs to bring together information from all parts and perspectives
of the organization
• Select a relatively small number of measures that are pivotal to
understanding and improving the system
• Display the vector of measures appropriately to correctly interpret,
learn from, and react to variation for each measure
Group Work - 1
• Develop vector of measures for:
• Ministry of Health : Group 1, 4
• Woreda Health Office : Group 2, 5
• Hospital : Group 3

• Instruction:
• Link with purpose of each organizational level
• Limit the size of the measures – prioritize to the most
important
Individual Exercise - 1
A. Review the overall approach of Wk-2)
indicators design in Ethiopia 1. EPI dropout rate
Ministry of Health (prepare 1-2 2. Maternal mortality
power point slide)
3. Tuberculosis detection rate
B. Review the following indicator 4. Contraceptive acceptance rate
definition from Ethiopia HMIS and
5. ART retention rate
prepare description of what each
6. Average length of stay
indicator is intended to measure
(submit on a word document by 7. Proportion of model households
Individual Exercise 2
• Start mastering use of MS-excel sheet on the
following functions
• Manage rows and columns
• Calculate indicators
• Prepare various types of graphs
• Edit graph
• Add an excel ad-in to your MS-excel application
End of Day 1
Day 2:
- Use it to complete the three assignments and for self learning
- Arrange virtual call if need arises to upload the excel ad-in
Individual assignment
collection

Printouts of individual assignment will be collected at tea break time

 Seminar
Each group will present their work over 15 minutes

Assignment Presentations
Understanding Variation

Assignment Presentations
 W. Edwards Deming
(1900 - 1993)

Appreciation
for a system

Understanding Building
Psychology

Variation
Knowledge
of Change

Understanding
Variation

Walter Shewhart
(1891 – 1967)
 • From your statistic course:
Understanding
• Measures of central tendency
Variation
• Measures of variation
 • From your statistic course:
Understanding
• Measures of central tendency – Mean/average
Variation
• Measures of variation – Standard Deviation
 Clinic 1 Clinic 2 Clinic 3 Clinic 4
average average average average average average Average Average
wait time patient wait time patient wait time patient wait time patient
Week (minutes) rating (minutes) rating (minutes) rating (minutes) rating
1 40.0 3.98 40.0 3.43 40.0 4.27 32.0 4.71
2 32.0 4.53 32.0 3.93 32.0 4.62 32.0 5.12
3 52.0 4.21 52.0 3.63 52.0 1.63 32.0 4.15
4 36.0 3.60 36.0 3.62 36.0 4.45 32.0 3.58
5 44.0 3.84 44.0 3.37 44.0 4.10 32.0 3.77
6 56.0 3.02 56.0 3.95 56.0 3.58 32.0 4.48
7 24.0 4.38 24.0 4.94 24.0 4.96 32.0 5.38

Understanding 8
9
10
16.0
48.0
28.0
5.87
2.58
5.59
16.0
48.0
28.0
6.45
3.44
4.37
16.0
48.0
28.0
5.31
3.93
4.79
76.0
32.0
32.0
1.75
5.22
4.05

Variation 11
number of
weeks
20.0

11
5.16

11
20.0

11
5.63

11
20.0

11
5.14

11
32.0

11
4.56

11
overall
average 36.00 4.25 36.00 4.25 36.00 4.25 36.00 4.25
standard
deviation 13.27 1.02 13.27 1.02 13.27 1.02 13.27 1.02
correlation 0.82 0.82 0.82 0.82
regression
intercept/
slope 6.50 -0.0625 6.50 -0.0625 6.50 -0.0625 6.50 -0.0625
std error 0.62 0.62 0.62 0.62
p-value 0.0022 0.0022 0.0022 0.0022

Source: HCDG
 • How best we could visualize the data?
Understanding • Visual display of the data will give more
Variation information and make us more curious to
learning and improvement
 S catte rplot for Clinic 1 S catte rplot fo r Clinic 2
7 7

avg . patient s atis fac tio n (1-7)

avg . patient s atis factio n (1-7)

6 6

5 5

4 4

3 3

2 2

1 1

Understanding
0.0 20.0 40.0 60.0 80.0 0.0 20.0 40.0 60.0 80.0
ave rage w e e kly w aiting time (min) ave rage w e e kly w aiting time (min)

Variation 7
S catte rplot for Clinic 3
7
S c atte rplo t fo r Clinic 4

avg . patie nt s atis fac tio n (1-7)

avg . patient s atis fac tio n (1-7)
6 6

5 5

4 4

3 3

2 2

1 1
0.0 20.0 40.0 60.0 80.0 0.0 20.0 40.0 60.0 80.0
ave rage w e e kly w aiting time (min) ave rage w e e kly w aiting time (min)
 • Intended variation
• Purposeful, planned or considered variation
• Source of learning
• Un-intended variation
Understanding
• Not purposeful
Variation
• Professionals may not be aware of the variations
• Creates inefficiencies, waste , rework, errors,
ineffectiveness
• Reducing unintended variation results in improved
outcomes and lower costs
 • Common Cause Variation
• Inherent in the system
• Common causes, chance causes
• Stable process, process in statistical control
Understanding
Variation • Special Cause Variation
• Not part of the system all the time
• Arise because of specific circumstances
• Assignable cause
• Unstable process, process not in statistical control
 • Common Cause Variation
• Improvement requires redesigning the system

Understanding
Variation
• Special Cause Variation
• Improvement requires investigation on the cause
and taking appropriate action
 • MISTAKE 1: React to an outcome as if it came from a special cause, when
actually it came from common causes of variation.

• MISTAKE 2: Treat an outcome as if it came from common causes of variation,

when actually it came from a special cause.

Mistakes in an Take action on individual

Common Causes (all)
Mistake #1
Special Causes
Correct Decision
attempt to outcome (treat as a special
cause)
improve Treat outcome as part of Correct Decision Mistake #2
the system and work on
changing the system (treat
as common cause
variation)
 • Shewhart named it as control chart
• Can also be called as learning charts or system
performance charts
Shewhart • In improvement work, the most common use of
these charts is to learn about variation and to
Charts evaluate the impact of changes
• Is a statistical tool used to distinguish between
variation in a measure due to common causes and
variation due to special causes
 Data Points (each
dot is
3 sigma “subgroup”)
Y Axis: limits
Shewhart Scale
for the
data
Charts
3 sigma X Axis: Sequence of data-leave blank identifiers
limits
Shewhart
•.
Charts
Shewhart Charts
• Control limits:

• Straight line if the sub-group sizes are equal

• Varying limits if sub-group sizes are un-equal

• In some situations, control limits might be missing:

• Centerline of the chart is near the upper or lower

possible range of the measure

• Amount of data in each subgroup is too small to

detect special cause signals
 • Shewhart called the control limits "three-sigma" control
limits and gave a general formula to calculate the limits
for any statistic.
• Let S be the statistic to be charted, then
• the centerline: CL =

Shewhart • the upper control limit: UCL = + 3 *s

Charts • the lower control limit:

σs
LCL = - 3 *

• Where:
• is the expected value of the statistic

• σs is the standard error of the statistic (only common cause variation)

• Each type of Shewhart chart has its own specific

calculations based on this general formula.
 Rules to determine special cause
1. A single point outside the control limit
2. A run of 8 or more points in a row above or
Shewhart below the central line

Chart 3. Six consecutive points increasing or decreasing

4. Two out of three consecutive points near (outer
one-third of the chart) a control limit
5. Fifteen consecutive points close (inner one-third)
to the central line
Understanding
Variation
 Reading Assignment:

Understanding
• Revise interpretation rules of “Run Charts”
Variation and compare with interpretation rules of
“Control Charts”
End of Day 3
Day 3 afternoon and/or day 4:
- Practical sessions on Shewhart Charts
- Complete assignments on Shewhart Charts
Control Chart Selection

Reference: Data Guide Chapter 5

 Chart
Selection
Guide
 Attribute Data
Medical Record Reviews - Errors
Classification = 3/5 XXX
= 60% with errors X XX
x
X XXX

Count = 11errors per 5 records

2 0 8 1 0
= 2.2 errors per record
 Classification Count
Measure
Number of accidents per 1000 employee days
Number errors per 25 food trays
Percent of AMI patients who received aspirin within 24 hours of arrival in
ER
Percent of deaths per month
Number of surgical complications per 1000 surgeries performed
Proportion of pneumonia patients who get antibiotics appropriately at
time of admission
Number of falls per 1000 patient days
Number of medication errors per 10,000 doses dispensed
 Classification Count
Measure
Number of accidents per 1000 employee days
Number errors per 25 food trays
Percent of AMI patients who received aspirin within 24 hours of arrival in
ER
Percent of deaths per month
Number of surgical complications per 1000 surgeries performed
Proportion of pneumonia patients who get antibiotics appropriately at
time of admission
Number of falls per 1000 patient days
Number of medication errors per 10,000 doses dispensed
 Control Chart
Shewhart Selection
Chart Guide
Selection Guide
Type of Data 56

Count or Classification (Attributes Data) Continuous (Variables Data)

-Qualitative data such as # errors, #
nonconformities or # of items that passed or failed)
- Discrete: must be whole number when originally
collected (can’t be fraction or scaled data when
originally collected)
-This data is counted, not measured

Count (Nonconformities) Classification (Nonconforming

1,2,3,4, etc. Items) Either/Or, Pass/Fail, Each point on Each point on the X-bar
Yes/No the chart is a chart is the average of
single measure multiple measures

Nonconforming Units
Studies a system’s performance over
time by charting number or proportion
of nonconforming items a system
produces per subgroup. An entire item
either meets or fails to meet criteria
(entire unit is either/or,
conforming/nonconforming, pass/fail)
Tip: Can count both nonconforming
items and conforming units.
Numerator can’t possibly be greater
than denominator.

Equal Area of Unequal Area Unequal or Equal Subgroup Unequal Or

Opportunity of Subgroup Size Size of 1 Equal Subgroup
Opportunity (n=1) Size (n>1)

I Chart (also known

as X Chart)
X Bar and S
C Chart U Chart P Chart

Number of Nonconformities Percent Individual Average

Nonconformities Per Nonconforming Measurements And
Unit Standard
Deviation
 Control Chart Selection Guide
Type of Data 57

Count or Classification (Attributes Data) Continuous (Variables Data)

-Qualitative data such as # errors, #
nonconformities or # of items that passed or failed)
- Discrete: must be whole number when originally
collected (can’t be fraction or scaled data when
originally collected)
-This data is counted, not measured

Count (Nonconformities) Classification (Nonconforming

1,2,3,4, etc. Items) Either/Or, Pass/Fail, Each point on Each point on the X-bar
Yes/No the chart is a chart is the average of
single measure multiple measures

Nonconformities
Studies a system’s
performance over time by
charting the
nonconformities (errors,
occurrences, defects,
complications) the system
is producing per subgroup
Tip: Can count #
nonconformities but not #
of conformities. Numerator
can be greater than
denominator.

Equal Area of Unequal Area Unequal or Equal Subgroup Unequal Or

Opportunity of Subgroup Size Size of 1 Equal Subgroup
Opportunity (n=1) Size (n>1)

I Chart (also known

as X Chart)
X Bar and S
C Chart U Chart P Chart

Number of Nonconformities Percent Individual Average

Nonconformities Per Nonconforming Measurements And
Unit Standard
Deviation
Number of Data for Attribute Charts
• Ideal not to have too many 0’s
• If 25% of data points are 0 limits won’t be useful
• Ideal to have limits on both sides of center line
• For C charts:
• Ave C needs to be > 1.4 to avoid too many 0s
• Ave C needs to be >9 to have lower limit
Make a P chart
 Make
appropriate
chart

Create a phase at a point

which feels logical
Establishing and revising limits for Shewhart chart
Revisioning of limits should be done
when the existing limits no longer
useful

There are four circumstances when

the original limits should be
recalculated:
1. When trial limits have been calculated with
fewer than 20 to 30 subgroups
2. When initial Shewhart chart has special
cause
3. When improvement has been made to the
process
4. When the Shewhart chart remains unstable
 Selection of
appropriate
Shewhart chart