Open Access Research

Patient flow within UK emergency

BMJ Open: first published as 10.1136/bmjopen-2016-015007 on 9 May 2017. Downloaded from http://bmjopen.bmj.com/ on October 10, 2023 by guest. Protected by copyright.
departments: a systematic review of the
use of computer simulation modelling
methods
Syed Mohiuddin,1,2 John Busby,3 Jelena Savović,1,2 Alison Richards,1,2
Kate Northstone,1,2 William Hollingworth,1,2 Jenny L Donovan,1,2 Christos Vasilakis4

To cite: Mohiuddin S, ABSTRACT

Busby J, Savović J, et al. Strengths and limitations of this study
Objectives Overcrowding in the emergency department
Patient flow within UK
(ED) is common in the UK as in other countries worldwide. ►► We systematically reviewed the peer-reviewed
emergency departments:
Computer simulation is one approach used for understanding literature to investigate the contribution of various
a systematic review of
the use of computer the causes of ED overcrowding and assessing the likely computer simulation methods for analysis of patient
simulation modelling impact of changes to the delivery of emergency care. flow within emergency departments in the UK.
methods. BMJ Open However, little is known about the usefulness of computer ►► We searched eight bibliographic databases to
2017;7:e015007. doi:10.1136/ simulation for analysis of ED patient flow. We undertook identify the relevant studies. Further to the electronic
bmjopen-2016-015007 a systematic review to investigate the different computer search, we conducted backward and forward
simulation methods and their contribution for analysis of citation searches of all included studies.
►► Prepublication history and
additional material is available. patient flow within EDs in the UK. ►► We highlighted a number of methodological, data,
To view please visit the journal Methods We searched eight bibliographic databases stakeholder, implementation and reporting-related
(http://​dx.​doi.​org/ 10.1136/ (MEDLINE, EMBASE, COCHRANE, WEB OF SCIENCE, issues associated with current studies.
bmjopen-2016-015007) CINAHL, INSPEC, MATHSCINET and ACM DIGITAL LIBRARY) ►► We included studies that were conducted under the
from date of inception until 31 March 2016. Studies were jurisdiction of UK National Health Service only.
Received 2 November 2016 included if they used a computer simulation method
Revised 7 February 2017 ►► We were not aware of any formal assessment
to capture patient progression within the ED of an checklist to estimate quality scores, nonetheless
Accepted 15 March 2017
established UK National Health Service hospital. Studies we assessed the key components of methodological
were summarised in terms of simulation method, key quality of all included studies.
assumptions, input and output data, conclusions drawn
and implementation of results.
Results Twenty-one studies met the inclusion criteria. Of provides acute care for patients who attend
these, 19 used discrete event simulation and 2 used system
hospital without prior appointment. The
dynamics models. The purpose of many of these studies
EDs of most hospitals customarily operate
(n=16; 76%) centred on service redesign. Seven studies
(33%) provided no details about the ED being investigated. 24 hours a day, 7 days a week. Nevertheless,
Most studies (n=18; 86%) used specific hospital models of overcrowding in EDs is an increasing problem
ED patient flow. Overall, the reporting of underlying modelling in countries around the world, and especially
assumptions was poor. Nineteen studies (90%) considered so in the UK.1 2 ED overcrowding has been
patient waiting or throughput times as the key outcome shown to have many adverse consequences
measure. Twelve studies (57%) reported some involvement such as increased medical errors,3 decreased
of stakeholders in the simulation study. However, only three quality of care and subsequently poor patient
1
NIHR CLAHRC West, University studies (14%) reported on the implementation of changes
Hospitals Bristol NHS Foundation
outcomes,4 increased workload,1 frustration
supported by the simulation. among ED staff,4 5 ambulance diversions,6
Trust, Bristol, UK
Conclusions We found that computer simulation can
2
School of Social and increased patient dissatisfaction,5 prolonged
provide a means to pretest changes to ED care delivery
Community Medicine, University patient waiting times7 and increased cost of
before implementation in a safe and efficient manner.
of Bristol, Bristol, UK
3 However, the evidence base is small and poorly developed. care.8 Furthermore, some less severely ill
School of Medicine, Queen's
University Belfast, Belfast, UK There are some methodological, data, stakeholder, patients may leave without being seen by a
4
Centre for Healthcare implementation and reporting issues, which must be physician, only to return later with a more
Innovation & Improvement addressed by future studies. complicated condition.8
(CHI2), School of Management, In the UK, there is an enormous pressure
University of Bath, Bath, UK from public and government to alleviate
Correspondence to INTRODUCTION overcrowding and long waiting times expe-
Dr Syed Mohiuddin; An emergency department (ED), also known rienced in ED.1 The Department of Health
​syed.​mohiuddin@​bristol.​ac.​uk as accident & emergency department (A&E), set a target standard for acute hospitals in

 Mohiuddin S, et al. BMJ Open 2017;7:e015007. doi:10.1136/bmjopen-2016-015007 1

Open Access

the National Health Service (NHS) that at least 95% of and complex scenarios at the individual level to influence

BMJ Open: first published as 10.1136/bmjopen-2016-015007 on 9 May 2017. Downloaded from http://bmjopen.bmj.com/ on October 10, 2023 by guest. Protected by copyright.
patients attending an A&E department must be seen, the care pathway taken and the time between events, but
treated, admitted or discharged under 4 hours.7 This specialist analytical knowledge is required typically to
4-hour target standard was initially set at 98% in 2004, but achieve a greater flexibility.17
later reduced to 95% in 2010. Beyond target setting, it Another widely used simulation method is SD, which
has been argued that ED overcrowding can be improved is used to understand the behaviour of complex health-
by lean healthcare thinking with a focus on improving care systems over time through capturing aggregate
patient flow.1 9 (instead of individual) flows of patients. An SD model
Over recent decades, computer simulation and other is essentially a collection of stocks and flows between
modelling methods have been used to analyse ED patient them. Stocks are basic stores of quantities over time,
flow and resource capacity planning.10–14 In essence, a for example, number of patients with a disease or in a
computer simulation model is a simplified representa- particular part of a hospital department. Flows define
tion of reality used to aid the understanding of the key the movement of objects between different stocks over
relationships and dynamics in the care process, and to time. Unlike DES, SD does not lend itself readily to
evaluate the likely impact of changes before implementa- including random variables and thus input parameters
tion. Typically, a simulation model is based on the notion are given as simple rates in the majority of studies. As
that each simulated individual is tracked through the care such, SD is considered not the ideal method of choice
process; the population effect is then estimated from the for modelling a closely focused system that involves
sum of the individual effects.15 resource-constrained queuing networks, such as an ED.10
The precise way in which a simulation model works In a situation like this, DES should rather be the method
depends on the type of simulation method used. Gener- of choice to model high level of distinct detail.18 ABS is
ally, simulation models can be categorised as static or another method that has more recently been used in
dynamic, as stochastic or deterministic, and as discrete modelling the healthcare systems. As a new method in
time or continuous time.16 A static simulation represents this application area, ABS is often overlooked in favour
a process at a particular point in time, whereas a dynamic of using more established methods of DES and SD. The
simulation represents a process as it evolves over time. A usefulness and practicalities of ABS in modelling patient
simulation model in which at least one input parameter flow are not well understood.19
is a random variable is said to be stochastic, whereas a Increasing interest in this area is reflected in the
simulation model having no random variables is said to number of computer simulation studies of ED patient
be deterministic. A discrete time model is one in which flow and resource capacity planning that have been
the state variables change instantaneously at discrete published over recent decades. However, little is known
points in time. In contrast, a continuous time model is about the usefulness of different computer simulation
one in which the state variables change continuously methods for analysis of any changes to the delivery of
with respect to time. The advancement of computer emergency care. We, therefore, systematically investi-
technology has undoubtedly supported the use of more gated the peer-reviewed literature on the use of computer
sophisticated simulation methods for modelling health- simulation modelling of patient flow within EDs in the
care processes. Today, for example, computer simulation UK. Our specific objectives were as follows: (1) to inves-
is also capable of providing an insight into the workings tigate the contribution that computer simulation studies
of a system through visual animation. make to our understanding of the problem of ED over-
Various types of computer simulation exist, including crowding; (2) to identify the methodology used to
discrete event simulation (DES), system dynamics (SD) conduct patient flow simulation in terms of key assump-
and agent-based simulation (ABS). DES is a widely used tions, systems requirements, and input and output data;
method, and can replicate the behaviour of complex (3) to assess the usefulness of each simulation method
healthcare systems over time. A DES model is a network for service redesign and evaluating the likely impact of
of queues and activities (such as having a blood test, X-ray changes related to the delivery of emergency care; (4)
and treatment). One of the major advantages of using a to report on differences in conclusions about ED perfor-
DES model is its flexibility to model complex scenarios mance with different simulation modelling methods; and
at the individual level. Within a DES model, individuals (5) to identify studies that explicitly aimed to meet the
move from one activity to another in sequential order at prespecified needs of stakeholders.
a particular point in time. Typically, the individuals enter
a system and visit some of the activities (not necessarily
only once) before leaving the system. The variables that METHODS
govern the movement of modelled individuals (such as We conducted a systematic review of the peer-reviewed
arrival rate and duration of treatment) can be random literature to identify computer simulation studies of
and thus readily capture the variation that is inherent in patient flow within hospital EDs in the UK. This review
healthcare. As such, a DES model is considered partic- complies with the online supplementary PRISMA check-
ularly suitable for modelling queuing systems. This list (www.​prisma-​statement.​org). We produced a review
simulation method is able to incorporate life histories protocol (available from the corresponding author on

2 Mohiuddin S, et al. BMJ Open 2017;7:e015007. doi:10.1136/bmjopen-2016-015007

 Open Access

request) and set out the process to address our specific cases of discrepancy, we selected the studies for full-text

BMJ Open: first published as 10.1136/bmjopen-2016-015007 on 9 May 2017. Downloaded from http://bmjopen.bmj.com/ on October 10, 2023 by guest. Protected by copyright.
objectives. review by consensus.

Search strategy Appraisal of studies for inclusion

We retrieved relevant studies from the following An electronic questionnaire was designed to appraise
bibliographic databases: MEDLINE, EMBASE, the studies selected for full-text review. The question-
COCHRANE, WEB OF SCIENCE, CINAHL, INSPEC, naire included four key questions: (1) Is the study a full
MATHSCINET and ACM DIGITAL LIBRARY. We used a paper published in a peer-reviewed journal or conference
key review paper20 to select these databases, which were proceedings? (2) Is the study set within the UK NHS? (3)
searched from the date of their inception until 31 March Is the study conducted within the ED of an established
2016. A search strategy was designed to explore three hospital? (4) Does the study use a computer simulation
main domains of knowledge associated with the area of model of emergency patient flow? A study with positive
our interest: computer simulation, emergency care and responses on these four questions was then included in
patient flow. We included a wide range of search phrases, the final review. SM and JB completed this process inde-
both keywords and medical subject headings, such as pendently and resolved any discrepancies that arose by
‘computer simulation,’ ‘emergency department,’ ‘patient consensus. Further to the electronic search, SM and JB
care,’ ‘patient flow,’ ‘waiting time,’ ‘time to treatment’ reviewed the backward and forward citations of all studies
and ‘length of stay.’ included in the electronic search to identify other poten-
We first developed the search strategy for MEDLINE tially relevant studies.
since it is known to allow a rich taxonomy of subjects
Data extraction
and rubrics. We used a key review paper20 to inform the
An electronic data extraction form was created to retrieve
MEDLINE search strategy and made further refinements
information about a number of key aspects, including
using other relevant studies to improve sensitivity. Online
simulation methods, data sources, key assumptions, input
supplementary appendix 1 shows the MEDLINE search
and output data, conclusions drawn and benefits of simu-
strategy and results from 1946 to end of March 2016.
lation outputs in practice. SM and JB independently
We adapted the MEDLINE strategy to search the other
recorded, collated and extracted the necessary informa-
databases (available from the corresponding author on
tion. Any discrepancies were resolved by consensus.
request). We also conducted backward and forward cita-
tion searches of all included studies using Google Scholar.
RESULTS
Inclusion criteria We retrieved a total of 2436 references from the 8 data-
We identified studies as being eligible for inclusion if they: bases: 437 from MEDLINE; 460 from EMBASE; 14 from
(1) were published in peer-reviewed journals or confer- COCHRANE; 253 from WEB OF SCIENCE; 65 from
ence proceedings as full papers; (2) were conducted CINAHL; 1103 from INSPEC; 4 from MATHSCINET;
within the ED of an established UK NHS hospital and 100 from ACM DIGITAL LIBRARY. We removed 440
responsible for assessing and treating civilians in need duplicate references, and then assessed the remaining
of emergency care; (3) captured the progress of patients 1996 unique references by title and abstract screening.
through at least two activities of an ED care process; and At this stage, we selected 159 of the 1996 studies for full-
(4) used a computer simulation method such as DES, SD, text review. Nineteen of the 159 studies were included
ABS, hybrid simulation, Monte Carlo simulation, distrib- following full-text review. Two more studies were included
uted simulation or stochastic modelling. from the backward and forward citation searching of the
We excluded editorials, letters, commentaries, confer- 19 studies. A total of 21 studies10–14 21–36 were included
ence abstracts, notes and books. We also excluded in the final review. Four studies37–40 were excluded from
studies that used methods such as regression analysis, the final review because the models used by these studies
likelihood ratio test, time series analysis, generalised are identical to the models reported in other already
linear model, mathematical programming, optimisation included studies.13 30 33 36 Figure 1 shows a summary of the
methods, queuing theory, structural equation modelling, study selection process.
process mapping, problem structuring method or risk Table 1 summarises the included studies, outlining
analysis without combining it with a computer simula- publication type, simulation purpose, ED details and
tion method. patient flow description. The first study33 was published
in 2000 and the most recent study21 was published in
Selection of studies for full-text review 2013. The maximum number of studies (n=4) published
To identify the studies suitable for full-text review, two in any single year was in 2006 and 2011. Nine of the 21
authors (SM and JB) independently screened the titles included studies (43%) were published in conference
and abstracts of all the initially retrieved studies. The proceedings. The highest number of studies (n=7) was
individual responses from each reviewer were stored in a published in proceedings of the Winter Simulation
common database. At this stage, a study was excluded if Conference, the second highest (n=5) was in the Emer-
it was clearly irrelevant based on our inclusion criteria. In gency Medicine Journal, and the third highest (n=3)

Mohiuddin
 S, et al. BMJ Open 2017;7:e015007. doi:10.1136/bmjopen-2016-015007 3
Open Access

BMJ Open: first published as 10.1136/bmjopen-2016-015007 on 9 May 2017. Downloaded from http://bmjopen.bmj.com/ on October 10, 2023 by guest. Protected by copyright.
Figure 1 Flow chart of the study identification and inclusion process. ED, emergency department; NHS, National Health
Service.

was in the Journal of the Operational Research Society. studied), secondary (ie, collected in another setting) and
More than two-thirds of the studies (n=16; 76%) did not expert opinion. One study21 did not describe the source
provide the name of the hospital studied. All 21 studies of data for any of the model inputs. Three studies13 26 31
described the underlying purpose of simulation; and in described the source of patient arrival rates, but not the
many cases (n=16; 76%), this centred on service redesign. sources of activity duration, activity progression and use
Surprisingly, seven studies12 13 21 25 31 34 36 (33%) did not of resources. Eight studies (38%) stated explicitly that
provide any details about the ED being investigated, while they used expert opinion to populate some of the model
five studies10 21 23 26 30 (24%) did not provide patient flow inputs. The proportion of studies that used primary data
diagram. was reasonably high (table 3). In particular, 95% (n=20)
Table 2 provides summary of simulation methods, of the studies used primary data for patient arrival rates,
including simulation type, key assumptions and use of 67% (n=14) for activity duration, 62% (n=13) for activity
software. The types of simulation varied only between progression and 52% (n=11) for resource inputs.
two methods (DES and SD). DES modelling was used in The most common changes considered in the simula-
19 studies (90%), while SD was used in 2 studies (10%). tion studies were ED patient flow (eg, changes in the triage
All but one study25 either explicitly or implicitly justified system for arriving patients30) and resource capacity plan-
the choice of underlying modelling method used. The ning (eg, changes in the number of cubicles30). However,
majority of studies (n=18; 86%) used specific hospital one-third of the studies11 13 21 24 26 27 31 (n=7) did not provide
models of ED patient flow. The reporting of modelling any details about the changes considered by the simula-
assumptions was poor overall. For example, as many as 12 tion. The majority of studies (n=19; 81%) considered
studies10 12 13 21 23–25 29 31 32 34 35 (57%) did not provide any patient waiting times (ie, time from arrival to discharge,
details about simulation duration, warm-up period and admission or transfer) as the key outcome measure. In
run number. Only five studies11 22 27 30 33 (24%) specified particular, 11 studies10 11 13 21 22 25 27 29–31 34 considered
the number of simulation runs and three studies27 30 36 patient waiting times alone, 7 studies12 23 24 28 32 35 36 consid-
(14%) specified the simulation warm-up period. Simula- ered patient waiting times and resources used, and the
tion duration ranged from 24 hours27 33 to 52 weeks.26 30 other study33 considered patient waiting times, resources
Almost 50% (n=10) of the studies used Simul8 software used and elective cancellations. Two other outcome
(www.​simul8.​com) for running the model. Two studies24 34 measures considered were resources used26 and bed occu-
(10%) did not provide any details about the use of soft- pancy.14
ware. Only 12 studies (57%) reported some involvement
Table 3 provides detail of simulation inputs and outputs. of stakeholders in the simulation study, mainly when
The identified models were populated from three sources deciding the study questions or specifying the model
of data: primary (ie, collected within the hospital being structure. However, in the study conducted by Mould

4 Mohiuddin S, et al. BMJ Open 2017;7:e015007. doi:10.1136/bmjopen-2016-015007

 Open Access

BMJ Open: first published as 10.1136/bmjopen-2016-015007 on 9 May 2017. Downloaded from http://bmjopen.bmj.com/ on October 10, 2023 by guest. Protected by copyright.
Table 1 Detail of the included studies
Publication type Simulation Patient Flow
Study name (year) (name) Hospital name purpose ED Detail Description
Anagnostou et al21 (2013) Conference Unknown several Proof of concept No detail Textual; activity
proceedings hospitals in Greater list
(Winter Simulation London
Conference)
Au-Yeung et al22 (2006) Conference Unknown hospital in Service redesign More detail Flow chart;
proceedings North London textual
(Modelling and
Simulation)
Baboolal et al23 (2012) Journal article University Hospital of Service redesign More detail Textual
(Emergency Medicine Wales
Journal)
Bowers et al24 (2009) Journal article Unknown hospital in Service redesign Some detail Flow chart
(Journal of Fife, Scotland
Simulation)
Brailsford et al10 (2004) Journal article Nottingham City Service redesign More detail Textual
(Journal of Hospital and QMC in
Operational Research Nottingham
Society)
Coats and Michalis25 (2001) Journal article Royal London Service redesign No detail Flow chart
(Emergency Medicine Hospital in
Journal) Whitechapel, London
Codrington-Virtue et al26 Conference Unknown hospital Understand More detail Textual
(2006) proceedings capacity
(Computer-Based
Medical Systems)
Codrington-Virtue et al27 Conference Unknown hospital Proof of concept Some detail Flow chart;
(2011) proceedings textual
(Winter Simulation
Conference)
Coughlan et al28 (2011) Journal article Unknown district Service redesign Some detail Flow chart;
(Emergency Medicine general hospital in textual
Journal) West London
Davies29 (2007) Conference Unknown hospital Service redesign More detail Flow chart;
proceedings textual
(Winter Simulation
Conference)
Eatock et al11 (2011) Journal article Hillingdon Hospital in Service redesign More detail Flow chart;
(Journal of West London textual
Health Org. and
Management)
Fletcher et al12 (2007) Journal article Unknown hospitals Service redesign No detail Flow chart;
(Journal of (n=10) textual
Operational Research
Society)
Günal and Pidd13 (2009) Journal article Unknown hospital Understand No detail Flow chart;
(Emergency Medicine behaviour textual
Journal)
Günal and Pidd30 (2006) Conference Unknown hospital Service redesign Some detail Textual; activity
proceedings list
(Winter Simulation
Conference)
Continued

Mohiuddin
 S, et al. BMJ Open 2017;7:e015007. doi:10.1136/bmjopen-2016-015007 5
Open Access

BMJ Open: first published as 10.1136/bmjopen-2016-015007 on 9 May 2017. Downloaded from http://bmjopen.bmj.com/ on October 10, 2023 by guest. Protected by copyright.
Table 1 Continued
Publication type Simulation Patient Flow
Study name (year) (name) Hospital name purpose ED Detail Description
Hay et al31 (2006) Conference Unknown hospitals Understand No detail Flow chart;
proceedings (n=4) behaviour textual
(Winter Simulation
Conference)
Komashie and Mousavi32 Conference Unknown hospital in Service redesign More detail Flow chart
(2005) proceedings London
(Winter Simulation
Conference)
Lane et al33 (2000) Journal article Unknown teaching Service redesign; More detail Flow chart;
(Journal of hospital in London forecasting textual
Operational Research
Society)
Lattimer et al14 (2004) Journal article Nottingham City Service redesign; Some detail Flow chart
(Emergency Medicine Hospital and QMC in forecasting
Journal) Nottingham
Maull et al34 (2009) Journal article Unknown hospital Service redesign; No detail Flow chart
(The Service in South West of forecasting
Industries Journal) England
Meng and Spedding35 Conference Unknown hospital Service redesign More detail Flow chart;
(2008) proceedings textual
(Winter Simulation
Conference)
Mould et al36 (2013) Journal article Unknown hospital in Service redesign No detail Flow chart
(Health Systems) Fife, Scotland
ED, emergency department; QMC, Queen’s Medical Centre.

et al,36 stakeholders were involved in deciding the study impact of introducing a ‘see and treat’ strategy to reduce
questions, specifying the model structure and imple- patient waiting times in the ED. After implementation,
menting the model outputs. More than 80% (n=17) of the observed reduction in breaches of the 4-hour waiting
the studies carried out some form of validation, mainly time target closely mirrored the simulation model predic-
face and/or data-led validation. In face validation, project tions.
team members, potential users and other stakeholders We identified a broad range of challenges, including
subjectively compare model and real-life behaviours to oversimplified assumptions22 25 33 35 and model struc-
judge whether the model and its results are reasonable at ture,14 25 system complexity,11 14 30 31 34 poor data
‘face value.’41 Data-led validation involves the comparing quality,12 25 29 34 36 high expectations,24 short-timescale,33
of model output with ‘real world’ data and may also poor stakeholder engagement,12 limited specialist analyt-
include a sensitivity analysis to determine the effect of ical skills,36 model runtime,11 24 generalisability14 28 and
varying the model’s inputs on its output performance.42 impact of simulation36; six studies10 21 23 26 27 32 (29%) did
Table 4 describes simulation results, summarising not describe any emergent issues.
conclusions in terms of whether the changes considered
(eg, increase in staffing numbers) were supported by the
simulation, whether the changes supported were imple- DISCUSSION
mented in practice (eg, staffing increased), and barriers This review has shown that computer simulation has been
to conducting the simulation (eg, data issues) and imple- used to analyse ED patient flow and resource capacity
menting the changes supported (eg, poor clinician planning to the delivery of emergency care. The most
buy-in and credibility). Two-thirds of the studies (n=14; common types of computer simulation used were DES
67%) provided some discussion on the usefulness of (n=19; 90%) and SD (n=2; 10%). All but one study25
simulation for analysis of changes to the delivery of emer- provided either explicit or implicit justification for the
gency care (table 4): six studies supported the proposed choice of modelling method used. However, the use of
changes, one study opposed the proposed changes and computer simulation of patient flow within EDs in the
seven studies recommended differential changes. Only a UK does not appear to have increased in recent years as
small number of studies12 34 36 (n=3; 14%) reported that may have been expected. This could be a reflection of
the proposed changes supported by the simulations were the relatively limited availability of funding for research
implemented. For example, Maull et al34 estimated the in this area compared with funding for health technology

6 Mohiuddin S, et al. BMJ Open 2017;7:e015007. doi:10.1136/bmjopen-2016-015007


Table 2 Summary of simulation methods

Mohiuddin
Simulation Rationale for Simulation
Study name type simulation type Model type duration Warm-up period Simulation run Simulation software
21 *
Anagnostou et al DES Yes Specific Not reported Not reported Not reported Repast Simphony
Au-Yeung et al22 DES† Yes Specific Not reported Not reported 10 Written in Java
Baboolal et al23 DES Yes Specific Not reported Not reported Not reported Simul8
Bowers et al24 DES Yes Specific Not reported Not reported Not reported Not reported
10 ‡
Brailsford et al DES Yes Specific Not reported Not reported Not reported Simul8
Coats and Michalis25 DES No Specific Not reported Not reported Not reported Simul8
Codrington-Virtue et al26 DES Yes Specific 52 weeks Not reported Not reported Simul8
27
Codrington-Virtue et al DES Yes Specific 24 hours 24 hours 50 Simul8
Coughlan et al28 DES Yes Specific 3 weeks Not reported Not reported Simul8
Davies29 DES Yes Specific Not reported Not reported Not reported Simul8
Eatock et al11 DES Yes Specific 3 weeks Not reported 20 Simul8
12
Fletcher et al DES Yes Generic Not reported Not reported Not reported Simul8
13 §
Günal and Pidd DES Yes Generic Not reported Not reported Not reported Micro Saint Sharp
Günal and Pidd30 DES¶ Yes Generic 52 weeks 0 50 Micro Saint Sharp

S, et al. BMJ Open 2017;7:e015007. doi:10.1136/bmjopen-2016-015007

Hay et al31 DES Yes Specific Not reported Not reported Not reported Arena
32
Komashie and Mousavi DES Yes Specific Not reported Not reported Not reported Arena
Lane et al33 SD** Yes Specific 24 hours Not reported 6 iThink
14 ††
Lattimer et al SD Yes Specific 52 weeks Not reported Not reported Stella
Maull et al34 DES Yes Specific Not reported Not reported Not reported Not reported
35
Meng and Spedding DES Yes Specific Not reported Not reported Not reported MedModel
Mould et al36 DES‡‡ Yes Specific 3 months 24 hours Not reported Simul8

*The authors used an agent-based simulation approach to model the ambulance service, but modelled the ED through a DES. These two individual models were then
linked together to form a hybrid emergency services model.
†The authors used a Markovian queuing network, but computed the moments and densities of patient treatment time through a DES.
‡The authors used an SD model as part of a bigger picture, but modelled the ED through a DES.
§The authors used their ED model elsewhere37 to form a whole hospital DES model consisting of two other departments: inpatient and outpatient clinics.
¶The authors used their ED model elsewhere38 to form a whole hospital DES model consisting of three other components: inpatient bed management, waiting list
management and outpatient clinics.
**The authors used their ED model elsewhere39 to explore the issues that arise when involving healthcare professionals in the process of model building.
††The authors constructed the ED as a separate submodel which was not detailed in the paper. However, we believe this ED submodel14 is identical to the ED model
reported in another included study.10
‡‡The authors used their ED model elsewhere40 to illustrate the role of care pathways to the redesign of healthcare systems.
DES, discrete event simulation; ED, emergency department; SD, system dyamics.
Open Access

7
BMJ Open: first published as 10.1136/bmjopen-2016-015007 on 9 May 2017. Downloaded from http://bmjopen.bmj.com/ on October 10, 2023 by guest. Protected by copyright.
8
Table 3 Detail of simulation inputs and outputs
Source Source of Source of
of arrival activity activity Source of Changes Outcomes Stakeholder
Study name rates duration progression resources use considered considered Validation input
Anagnostou et al21 Not Not described Not described Not described None Waiting times None None
Open Access

described
Au-Yeung et al22 Primary Primary; Primary; Primary ED patient flow Waiting times Data led Model
expert opinion expert opinion specification
Baboolal et al23 Primary Primary; Primary Primary Resources Waiting times*; Face Model
expert opinion resources Dark world model specification
Bowers et al24 Primary Primary Primary Not described None Resources* Data led Model
Face specification
Brailsford et al10 Primary Secondary Not described Primary ED patient flow; Waiting times Data led Study question;
arrival rates Face model
specification
Coats and Michalis25 Primary Primary Not described Not described Shift patterns Waiting times* Data led None
26
Codrington-Virtue et al Primary Not described Not described Not described None Resources None None
Codrington-Virtue et al27 Primary Primary; Primary Primary; None Waiting times Data led None
expert opinion expert opinion
Coughlan et al28 Primary Not described Primary Primary Resources Waiting times*; Data led None
resources
Davies29 Primary Primary Primary Primary ED patient flow Waiting times* None None
11 *
Eatock et al Primary Primary; Primary; Primary None Waiting times Data led None
expert opinion expert opinion
Fletcher et al12 Primary Secondary; Primary Primary ED patient flow; Waiting times*; Data led Study question;
expert opinion resources; resources Face model
demand specification
Günal and Pidd13 Primary Not described Not described Not described None Waiting times* None None
30 *
Günal and Pidd Primary Primary Primary Not described ED patient flow; Waiting times Data led Model
resources specification
Hay et al31 Primary Not described Not described Not described None Waiting times* Data led None
32
Komashie and Mousavi Primary Primary; Primary; Primary; ED structure; Waiting times; Data led Study question;
expert opinion expert opinion expert opinion resources resources Face model
specification
Lane et al33 Primary Primary; Expert opinion Primary Resources; Waiting times; Data led Study question;
expert opinion demand resources; Face model
elective specification
cancellations
Continued

Mohiuddin S, et al. BMJ Open 2017;7:e015007. doi:10.1136/bmjopen-2016-015007

BMJ Open: first published as 10.1136/bmjopen-2016-015007 on 9 May 2017. Downloaded from http://bmjopen.bmj.com/ on October 10, 2023 by guest. Protected by copyright.
 Open Access

assessment. There is also a limited number of research

BMJ Open: first published as 10.1136/bmjopen-2016-015007 on 9 May 2017. Downloaded from http://bmjopen.bmj.com/ on October 10, 2023 by guest. Protected by copyright.
groups with the analytical skills required to develop tech-

Study question;

Study question;

Study question;
implementation

implementation
specification;
nically complex simulation models for the analysis of
Stakeholder

specification

specification
service redesign.
Identified studies varied in the style and quality of

Result
model

model

model

result
input
reporting; but assumptions used in the analyses were not
always transparently reported. The opaque reporting of
key assumptions prevents decision makers from appraising
the quality of evidence from simulation experiments.
Although there is a set of guidelines for researchers of
Validation

DES to follow when building models,43 this has not been

Data led

Data led

Data led

Data led
widely adopted yet. Most of the studies (n=19; 90%)
Face

Face
considered patient waiting or throughput times as the
main outcome measure. This is perhaps unsurprising
since waiting time has been shown to be a key determinant
Bed occupancy

Waiting times*;
Waiting times;
Waiting times*

of patient satisfaction and has been strongly prioritised

considered
Outcomes

through the 4-hour targets.2 Some studies13 21 26 31 did not

resources

resources

provide enough information on how input parameters

were selected and synthesised. A handful of studies used
expert opinion to populate some of the model inputs,
but none explicitly justified the reason for using expert
admission rates

ED patient flow

opinion. It is important to have transparent criteria for

ED structure;

ED structure;
considered

using expert opinion since it can overestimate or under-

Resources
resources
Changes

estimate the model inputs. There are several methods for

eliciting expert opinion as discussed by Grigore et al.44
Most models were intended to capture specific aspects
of the emergency care process, but some authors have
resources use

argued that understanding of patient flow requires

Not described

Not described

Not described

study of the entire care process.45 Conversely, others

Source of

argue that it is sufficient to focus on the specific needs

Primary

of the care process rather than modelling a large and

complicated care process.27 Most of the studies (n=18;
86%) used specific hospital models of ED patient flow.
*These studies used 4-hour target breach as part of their waiting time considerations.
Not described

Interestingly, there seemed to be no standard hospital

progression

model of patient flow of emergency care process. One

Source of

generic model was developed by the Department of

Primary

Primary

Primary
activity

Health in 2007 for use across all EDs.12 This generic

approach allows hospitals to benefit from simulation
methodology with minimal costs and technical exper-
expert opinion

tise, but there are challenges of using a generic national

model for specific local use due to the local context of
Source of

duration

Primary;
Primary

Primary

Primary
activity

each NHS hospital including differences in physical

space, the demographics of local patient populations,
and so on.
Just over half of the studies in our review reported
of arrival

some involvement of stakeholders in the simulation

Primary

Primary

Primary

Primary
Source

study. Involving stakeholders is important since it helps

rates

to understand the problem better,8 46 assess the simu-

lation outputs fully47 and translate simulation outputs
into policy.15 Very few studies reported clear summaries
ED, emergency department.

of whether the changes considered were supported by

Meng and Spedding35
Table 3 Continued

the simulation and of whether the changes supported

were implemented. Some studies drew attention to
Lattimer et al14

a number of challenges particularly associated with

Mould et al36
Study name

Maull et al34

simulation conduct and implementation. Brailsford48

provided a helpful discussion on how to overcome the
barriers such as methodological suitability, data crisis
and stakeholder issues.

Mohiuddin
 S, et al. BMJ Open 2017;7:e015007. doi:10.1136/bmjopen-2016-015007 9
10
Table 4 Summary of simulation results
Reported
the changes
Study name Conclusions Conclusions detail implemented? Result implementation Barriers
21
Anagnostou et al None None No NA None
Open Access

Au-Yeung et al22 Supported Prioritisation of treatment for patient No NA Simplified assumptions

the changes with minor problems over major
considered problems could lead to improved
outcome
Baboolal et al23 Supported A change in staffing levels could lead to No NA None
the changes substantial cost savings and reduce the
considered 4-hour breaches
Bowers et al24 None None No NA Model runtime; high expectancy
Brailsford et al10 Opposed Streaming of patients by triage No NA None
the changes category was not an efficient use of
considered clinical resources
Coats and Michalis25 Supported Shift pattern that best matches patient No NA Simplified model structure and
the changes arrivals would give shorter waiting assumptions; poor data quality
considered times
Codrington-Virtue et al26 None None No NA None
27
Codrington-Virtue et al None None No NA None
Coughlan et al28 Proposed Adding an emergency nurse No NA Generalisability
differential practitioner would not reduce the
changes waiting times. Resource reallocation
would improve throughput times
Davies29 Supported The separation of see and treat would No NA Poor data quality
the changes be beneficial
considered
Eatock et al11 None None No NA System complexity; model
runtime
Fletcher et al12 Proposed Deflecting demand away from A&E Yes Unknown as other interventions Poor data quality; poor
differential would lead to improvement around were introduced in parallel stakeholder engagement
changes waiting for beds, specialists and
assessment processes
Günal and Pidd13 None None No NA Explaining the causes of change
in performance
Günal and Pidd30 Proposed More senior doctors, less X-ray No NA Modelling multitasking behaviour
differential requisitions and more cubicles would of staff
changes reduce waiting times
Hay et al31 None None No NA System complexity
Continued

Mohiuddin S, et al. BMJ Open 2017;7:e015007. doi:10.1136/bmjopen-2016-015007

BMJ Open: first published as 10.1136/bmjopen-2016-015007 on 9 May 2017. Downloaded from http://bmjopen.bmj.com/ on October 10, 2023 by guest. Protected by copyright.

Mohiuddin
Table 4 Continued
Reported
the changes
Study name Conclusions Conclusions detail implemented? Result implementation Barriers
32
Komashie and Mousavi Proposed Adding a nurse or doctor to minors No NA None
differential would reduce the waiting times by
changes 28%. Increasing the cubicles/beds
would make smaller change
Lane et al33 Proposed Changing bed numbers led to no No NA Short timescale; simplified
differential noticeable change in waiting times assumptions
changes but a substantial difference to elective
cancellations
Lattimer et al14 Proposed System would not be able to cope No NA Simplified model structure;
differential with increasing demand from scenario system complexity;
changes 1*, but scenarios 2†, 3‡ and 4§ could generalisability
improve this
Maull et al34 Supported See and treat reduced the 4-hour Yes Marked reduction in no. of Poor data availability and quality;
the changes breaches from 13.2% to 3.4% breaches from 13.2% to 1.4%. system complexity

S, et al. BMJ Open 2017;7:e015007. doi:10.1136/bmjopen-2016-015007

considered No. of patients waiting less than
1 hour increased from 12% to
23%. No. of patients with major
problems waiting between 3 and
4 hours increased
Meng and Spedding35 Proposed Reduced times to see a consultant No NA Simplified assumptions
differential would reduce the waiting times. Access
changes to 24-hour X-ray would reduce the
waiting times too
Mould et al36 Supported A new staff roster would reduce the Yes Mean time for minor problems Poor data quality; limited
the changes waiting times dropped from 100 to 94 min, for analytical skills; impact of
considered major problems it dropped from simulation
200 to 195 min. Mean time for
minor problems fell by 16 min
after adjusting other factors
*Five-year model run assuming 4% year-on-year growth in emergency admissions and 3% year-on-year growth in general practitioner (GP) referral for planned admissions.
†Impact of increase in demand for front door services.
‡Reducing emergency admissions of patients with respiratory or coronary problems, ill-defined conditions and over 65 years.
§Effects of earlier discharge of patients admitted as emergencies and subsequently discharged to nursing or residential homes.
A&E, accident & emergency department; NA, not applicable.
Open Access

11
BMJ Open: first published as 10.1136/bmjopen-2016-015007 on 9 May 2017. Downloaded from http://bmjopen.bmj.com/ on October 10, 2023 by guest. Protected by copyright.
Open Access

Only three studies12 34 36 (14%) reported on the imple- score. For example, in the field of randomised controlled

BMJ Open: first published as 10.1136/bmjopen-2016-015007 on 9 May 2017. Downloaded from http://bmjopen.bmj.com/ on October 10, 2023 by guest. Protected by copyright.
mentation of the changes supported by the simulation trials (RCTs), there is evidence that the use of quality
outputs. This may show that the impact of computer simu- scores and scales, especially of those with a numerical
lation modelling within the field of UK’s emergency care summary, is problematic and meaningless.55 The current
has been limited, though we do not know if any changes best practice for assessment of validity of RCTs requires
were implemented at a later date. We also do not know assessing individual components of trial design, conduct
if any changes implemented led to any improvements and analysis (eg, Cochrane risk of bias tool). We adopted
in the process or outcomes of ED care. The systematic a similar approach, whereby assessing the key method-
use of simulation modelling is not yet part of healthcare, ological components of all included studies.
whereas its use in other sectors like in manufacturing Third, we neither verified whether any of the hospitals
or airline industry is an integral part of the actual deci- implemented the findings found from simulation exper-
sion-making process.49 Why is simulation yet to make the iments, nor do we know if any changes implemented led
same impact in healthcare as in other industries? Lack to any improvements. Typically, there is little opportunity
of stakeholders’ engagement has been argued as one of to assess the impact of the simulation since publication
the main reasons for this.8 46 48 49 To this end, Harper and emerges before the work is fully implemented in many
Pitt46 discussed the basic components of successful imple- healthcare studies.56 Finally, we did not include Google
mentation of simulation methods in healthcare. Absence Scholar in the database search list since it has a number of
of lucid guidelines about how to use simulation methods issues with its indexing and citation algorithm, although
effectively in healthcare has been argued as another it is known to provide increased access to non peer-re-
reason.50 However, more recently in 2012, the ISPOR- viewed publications.57 Anecdotal evidence suggests that
SMDM Modeling Good Research Practices Task Force-4 NHS hospitals have used simulation modelling (and
laid out a set of guidelines about how to use DES method other methods) to improve patient flow through the ED.
effectively in healthcare.43 In line with a few others,8 46 48 49 However, our review will not capture all of this work as it is
we also argue that if simulation is to make sustained impact not all reported in peer-reviewed academic publications.
in healthcare, the clinicians and decision makers must We used a key review paper20 to select a wide range of
cooperate across physical and organisational boundaries databases covering the comprehensive sources of litera-
and come to understand how seemingly small changes in ture in computer science, operations management and
design of processes can improve patient care. healthcare fields.
We systematically searched eight bibliographic databases
to identify the included studies; however, our study has
some limitations. First, we focused on the use of computer CONCLUSIONS
simulation methods in the context of patient flow within We found that computer simulation can provide a means
EDs under the jurisdiction of UK NHS only. Improving to pretest the likely impact of changes to the delivery of
emergency care is a research priority for UK NHS.1 In this emergency care before implementation in a safe and effi-
review, we examined the current literature that analysed cient manner. In particular, it is used to identify the key
ED patient flow within the context of UK, and discussed relationships and bottlenecks in the process of ED care,
how simulation can be better used as a tool to address this test ‘what-if’ scenarios for service redesign, determine
problem. It would be interesting to compare the identi- levels of uncertainty, provide visualisations and forecast
fied methods with other jurisdictions across Europe, in future performance. However, the evidence base is small
the USA and Australasia, but this was beyond the scope of and poorly developed, with many methodological and
this study. Besides, comparing studies from different juris- practical issues, including lack of awareness regarding
dictions and reaching consensus would be challenging system complexity, lack of good quality data, lack of
since healthcare delivery is different in the UK. Neverthe- persistent engagement of stakeholders in the modelling
less, computer simulation has been used to analyse and process, lack of in-house analytical skills and lack of an
design ED overcrowding in other countries. In particular, implementation plan. Furthermore, the level of detail of
DES models have been used to identify optimal ED flow reporting of the computer simulation methods differed in
patterns,51 forecast ED overcrowding52 53 and evaluate the style and quality of reporting; and in some instances,
staffing levels and changes in ED bed capacity.54 Fletcher key aspects of the assumptions underpinning the analyses
et al12 cited a number of other international ED models were not always reported explicitly and transparently.
which have different designs to English ED. This review is a useful source providing direction on
Second, we were not aware of any formal assessment why simulation needs to be better used as a tool for
checklist to estimate quality scores of the identified analysis of ED patient flow. Future studies should justify
studies. The set of guidelines reported by the ISPOR- the choice of simulation modelling method explicitly,
SMDM is not a quality assessment checklist for reviewers.43 avoid making selective use of the available data, engage
It is rather a set of recommended best practices for stakeholders in the modelling process and keep them
modelling teams to consider and embrace when building on board continually, be transparent in the reporting
DES models. Furthermore, there is a good rationale for of simulation inputs and outputs, and report on the
a component-based approach, instead of using a quality implementation of changes supported by the findings of

12 Mohiuddin S, et al. BMJ Open 2017;7:e015007. doi:10.1136/bmjopen-2016-015007

 Open Access

simulation experiments. We recommend the adoption of 13. Günal MM, Pidd M. Understanding target-driven action in

BMJ Open: first published as 10.1136/bmjopen-2016-015007 on 9 May 2017. Downloaded from http://bmjopen.bmj.com/ on October 10, 2023 by guest. Protected by copyright.
emergency department performance using simulation. Emerg Med J
reporting guidelines43 by academic journals and confer- 2009;26:724–7.
ence proceedings, and more persistent exploitation of 14. Lattimer V, Brailsford S, Turnbull J, et al. Reviewing emergency care
innovative models of engagement and knowledge mobil- systems I: insights from system dynamics modelling. Emerg Med J
2004;21:685–91.
isation between academics and healthcare professionals 15. Sobolev B, Kuramoto L. Policy analysis using patient flow
such as the Researchers in Residence.58 Further research simulations: conceptual framework and study design. Clin Invest
Med 2005;28:359–63.
is necessary to assess the quality of computer simulation 16. Pitt M, Monks T, Crowe S, et al. Systems modelling and simulation in
models of ED patient flow across different countries and health service design, delivery and decision making. BMJ Qual Saf
to establish the extent to which the simulation outputs 2016;25:38–45.
17. Davies R, Roderick P, Raftery J. The evaluation of disease
have been translated into policy. prevention and treatment using simulation models. Eur J Oper Res
2003;150:53–66.
Contributors SM and JB independently screened and appraised the relevant 18. Brailsford S, Hilton N. A comparison of discrete event simulation and
studies, and extracted the data from the included studies. CV, SM, JS, KN, JB system dynamics for modelling health care systems, 2001.
and AR designed the search strategy. AR, an information specialist, conducted 19. Hutzschenreuter A, Bosman P, Blonk-Altena I, et al, 2008. Agent-
the electronic search. SM drafted the manuscript in conjunction with CV. JD, WH based patient admission scheduling in hospitals. proceedings of
and CV revised the draft critically for intellectual content, while the other authors the 7th international joint conference on autonomous agents and
multiagent systems: industrial track. International Foundation for
commented on the draft. All authors read and approved the submitted manuscript.
Autonomous Agents and Multiagent Systems
Funding All the authors apart from CV are supported by the National Institute for 20. Sobolev BG, Sanchez V, Vasilakis C. Systematic review of the use of
Health Research (NIHR) Collaboration for Leadership in Applied Health Research computer simulation modeling of patient flow in surgical care. J Med
and Care West (CLAHRC West) at University Hospitals Bristol National Health Service Syst 2011;35:1–16.
(NHS) Foundation Trust. The views expressed are those of the authors and not 21. Anagnostou A, Nouman A, Taylor SJE. Distributed hybrid Agent-
Based discrete event emergency medical services simulation. Wint
necessarily those of the NHS, the NIHR or the Department of Health.
Simul C Proc 2013:1625–36.
Competing interests None declared. 22. Au-Yeung SWM, Harrison PG, Knottenbelt WJ. A queueing network
model of patient flow in an accident and emergency department.
Provenance and peer review Not commissioned; externally peer reviewed. Modelling and Simulation 2006;2006:60–7.
Data sharing statement No additional data available. 23. Baboolal K, Griffiths JD, Knight VA, et al. How efficient can
an emergency unit be? A perfect world model. Emerg Med J
Open Access This is an Open Access article distributed in accordance with the 2012;29:972–7.
terms of the Creative Commons Attribution (CC BY 4.0) license, which permits 24. Bowers J, Ghattas M, Mould G. Success and failure in the
others to distribute, remix, adapt and build upon this work, for commercial use, simulation of an accident and emergency department. Simulation J
provided the original work is properly cited. See: http://​creativecommons.​org/​ 2009;3:171–8.
licenses/​by/​4.​0/ 25. Coats TJ, Michalis S. Mathematical modelling of patients flow
through an accident and emergency department. Emerg Med J
© Article author(s) (or their employer(s) unless otherwise stated in the text of the 2001;18:190–2.
article) 2017. All rights reserved. No commercial use is permitted unless otherwise 26. Codrington-Virtue A, Chaussalet T, Millard P, et al. A system
expressly granted. for patient management based discrete-event simulation and
hierarchical clustering. Comp Med Sy 2006:800–4.
27. Codrington-Virtue A, Kelly J, Chaussalet T, 2011. Using simplified
Discrete-Event simulation models for health care applications.
Proceedings of the 2011 Winter Simulation Conference (WSC) 1154–65
REFERENCES 28. Coughlan J, Eatock J, Patel N. Simulating the use of re-prioritisation
1. Banerjee A, Mbamalu D, Hinchley G. The impact of process re- as a wait-reduction strategy in an emergency department. Emerg
engineering on patient throughput in emergency departments in the Med J 2011;28:1013–8.
UK. Int J Emerg Med 2008;1:189–92. 29. Davies R, Winter Simulation Conference, “See and Treat” or “See” and
2. Hemaya SA, Locker TE. How accurate are predicted waiting times, “Treat” in an emergency department. 2007
determined upon a patient's arrival in the Emergency Department? 30. Günal MM, Pidd M Proceedings of the 2006 Winter Simulation
Emerg Med J 2012;29:316–8. Conference,Understanding accident and emergency department
3. Trzeciak S, Rivers EP. Emergency department overcrowding in the performance using simulation.2006 ;446–52.
United States: an emerging threat to patient safety and public health. 31. Hay A, Valentin E, Bijlsma R Proceedings of the 2006 Winter Simulation
Emerg Med J 2003;20:402–5. Conference, Modeling emergency care in hospitals: a paradox-the
4. Miró O, Antonio MT, Jiménez S, et al. Decreased health care quality patient should not drive the process. 2006;IEEE.
associated with emergency department overcrowding. Eur J Emerg 32. Komashie A, Mousavi A Winter Simulation Conference, Modeling
Med 1999;6:105–7. emergency departments using discrete event simulation
5. Trout A, Magnusson AR, Hedges JR. Patient satisfaction techniques. proceedings of the 37th conference on winter
investigations and the emergency department: what does the simulation. 2005
literature say? Acad Emerg Med 2000;7:695–709. 33. Lane DC, Monefeldt C, Rosenhead JV. Looking in the wrong place
6. Fatovich DM, Hirsch RL. Entry overload, emergency department for healthcare improvements: a system dynamics study of an
overcrowding, and ambulance bypass. Emerg Med J 2003;20:406–9. accident and emergency department. J Oper Res Soc 2000.
7. Blunt I. Focus on: a&e attendances Why are patients waiting longer? 34. Maull RS, Smart P, Harris A, et al. An evaluation of ‘fast track’in A&E:
Quality Watch, The Health Foundation and Nuffield Trust. 2014 a discrete event simulation approach. The Service Industries Journal
(31 August 2016). 2016 http://www.​nuffieldtrust.​org.​uk/​sites/​files/​ 2009;29:923–41.
nuffield/​publication/​140724_​focus_​on_​ae_​attendances.​pdf​. 35. Meng LY, Spedding T Winter Simulation Conference. Modelling
8. Carrus B, Corbett S. Khandelwal D. A hospital-wide strategy for patient arrivals when simulating an Accident and Emergency Unit.
fixing emergency-department overcrowding. McKinsey Quarterly 2008 1509–15
2010:1–12. 36. Mould G, Bowers J, Dewar C, et al. Assessing the impact of systems
9. de Koning H, Verver JP, van den Heuvel J, et al. Lean six sigma in modeling in the redesign of an Emergency Department. Health
healthcare. J Healthc Qual 2006;28:4–11. Systems 2013;2:3–10.
10. Brailsford S, Lattimer V, Tarnaras P, et al. Emergency and on-demand 37. Günal MM, Pidd M, 2007. Interconnected DES models of emergency,
health care: modelling a large complex system. J Oper Res Soc outpatient, and inpatient departments of a hospital.Winter Simulation
2004;55:34–42. Conference
11. Eatock J, Clarke M, Picton C, et al. Meeting the four-hour deadline in 38. Günal MM, Pidd M. DGHPSIM: generic simulation of hospital
an A&e department. J Health Organ Manag 2011;25:606–24. performance. TOACMS 2011;21:23.
12. Fletcher A, Halsall D, Huxham S, et al. The DH Accident and 39. Lane DC, Monefeldt C, Husemann E. Client involvement in simulation
Emergency Department model: a national generic model used locally. model building: hints and insights from a case study in a London
J Oper Res Soc 2007;58:1554–62. hospital. Health Care Manag Sci 2003;6:105–16.

Mohiuddin
 S, et al. BMJ Open 2017;7:e015007. doi:10.1136/bmjopen-2016-015007 13
 Open Access

40. Mould G, Bowers J. Roles of pathway-based models and their 50. Jun GT, Ward J, Morris Z, et al. Health care process modelling: which

BMJ Open: first published as 10.1136/bmjopen-2016-015007 on 9 May 2017. Downloaded from http://bmjopen.bmj.com/ on October 10, 2023 by guest. Protected by copyright.
contribution to the redesign of health-care systems. Int J Care Coord method when? Int J Qual Health Care 2009;21:214–24.
2011;15:90–7. 51. Hung GR, Whitehouse SR, O'Neill C, et al. Computer modeling of
41. Validation BO. Verification, and testing techniques throughout patient flow in a pediatric emergency department using discrete
the life cycle of a simulation study. Annals of operations research event simulation. Pediatr Emerg Care 2007;23:5–10.
1994;53:121–73. 52. Connelly LG, Bair AE. Discrete event simulation of emergency
42. Robinson S. Simulation: the practice of model development and use. department activity: a platform for system-level operations research.
Hoboken, NJ: John Wiley & Sons, 2004. Acad Emerg Med 2004;11:1177–85.
43. Karnon J, Stahl J, Brennan A, et al. Modeling using discrete event 53. Hoot NR, LeBlanc LJ, Jones I, et al. Forecasting emergency
simulation: a report of the ISPOR-SMDM Modeling Good Research department crowding: a discrete event simulation. Ann Emerg Med
Practices Task Force-4. Value Health 2012;15:821–7. 2008;52:116–25.
44. Grigore B, Peters J, Hyde C, et al. A comparison of two methods for 54. Khare RK, Powell ES, Reinhardt G, et al. Adding more beds to
expert elicitation in health technology assessments. BMC Med Res the emergency department or reducing admitted patient boarding
Methodol 2016;16:85.
times: which has a more significant influence on emergency
45. Haraden C, Resar R. Patient flow in hospitals: understanding and
department congestion? Annals of emergency medicine
controlling it better. Front Health Serv Manage 2004;20:3–15.
2009;53:575–85.
46. Harper PR, Pitt M. On the challenges of healthcare modelling and
a proposed project life cycle for successful implementation. J Oper 55. Jüni P, Witschi A, Bloch R, et al. The hazards of scoring the quality of
Res Soc 2004;55:657–61. clinical trials for meta-analysis. JAMA 1999;282:1054–60.
47. Eldabi T, Irani Z, Paul RJ. A proposed approach for modelling health- 56. Fone D, Hollinghurst S, Temple M, et al. Systematic review of the use
care systems for understanding. J Manag Med 2002;16:170–87. and value of computer simulation modelling in population health and
48. Brailsford S. Overcoming the barriers to implementation of health care delivery. J Public Health Med 2003;25:325–35.
operations research simulation models in healthcare. Clin Invest Med 57. Shultz M. Comparing test searches in PubMed and Google Scholar.
2005;28:312–5. J Med Libr Assoc 2007;95:442.
49. Eldabi T, Paul R, Young T. Simulation modelling in healthcare: 58. Marshall M, Pagel C, French C, et al. Moving improvement research
reviewing legacies and investigating futures. J Oper Res Soc closer to practice: the Researcher-in-Residence model. BMJ Qual
2007;58:262–70. Saf 2014;23:801–5.

14 Mohiuddin S, et al. BMJ Open 2017;7:e015007. doi:10.1136/bmjopen-2016-015007