Fernandez et al.

Health Research Policy and Systems (2015) 13:66

DOI 10.1186/s12961-015-0057-0

REVIEW Open Access

Evidence-based medicine: is it a bridge too

far?
Ana Fernandez1*, Joachim Sturmberg2, Sue Lukersmith3, Rosamond Madden3, Ghazal Torkfar4, Ruth Colagiuri4
and Luis Salvador-Carulla5

Abstract
Aims: This paper aims to describe the contextual factors that gave rise to evidence-based medicine (EBM), as well
as its controversies and limitations in the current health context. Our analysis utilizes two frameworks: (1) a complex
adaptive view of health that sees both health and healthcare as non-linear phenomena emerging from their
different components; and (2) the unified approach to the philosophy of science that provides a new background
for understanding the differences between the phases of discovery, corroboration, and implementation in science.
Results: The need for standardization, the development of clinical epidemiology, concerns about the economic
sustainability of health systems and increasing numbers of clinical trials, together with the increase in the
computer’s ability to handle large amounts of data, have paved the way for the development of the EBM
movement. It was quickly adopted on the basis of authoritative knowledge rather than evidence of its own
capacity to improve the efficiency and equity of health systems. The main problem with the EBM approach is the
restricted and simplistic approach to scientific knowledge, which prioritizes internal validity as the major quality of
the studies to be included in clinical guidelines. As a corollary, the preferred method for generating evidence is the
explanatory randomized controlled trial. This method can be useful in the phase of discovery but is inadequate in
the field of implementation, which needs to incorporate additional information including expert knowledge,
patients’ values and the context.
Conclusion: EBM needs to move forward and perceive health and healthcare as a complex interaction, i.e. an
interconnected, non-linear phenomenon that may be better analysed using a variety of complexity science
techniques.
Keywords: Complexity of knowledge, Evidence-based medicine, Evidence-based practice, External validity, Framing,
Generalizability, Internal validity, Randomized controlled trial

Background available external clinical evidence from systematic

Over the past 20 years or more, the concept of research” [1]. Muir Gray regarded this definition as too
evidence-based medicine (EBM) has increasingly been doctor-centric and expanded it to emphasize the im-
accepted as the gold standard for decision making in portance of the patient perspective and proposed that,
medical/health practice and policy. “…evidence based clinical practice is an approach to
EBM provides a standard procedure for using evi- decision making in which the clinician uses the best
dence in clinical decision making. It is framed as “…the scientific evidence available, in consultation with the
conscientious, explicit and judicious use of best evidence patient, to decide upon the option which suits the
in making decisions about the care of individual [sic] patient best.” [2]. In their respective papers, both
patients. The practice of evidence-based medicine means Sacket and Gray described the stages of EBM decision
integrating individual clinical experience with the best making as (1) assessment and synthesis of external
evidence using clinical epidemiology, systematic
* Correspondence: [email protected] search and meta-analysis, and other techniques such
1
Brain and Mind Centre, Faculty of Health Sciences, The University of Sydney,
94 Mallett Street, Camperdown, NSW 2050, Australia
as cost analysis and modelling; (2) use of probabilistic
Full list of author information is available at the end of the article reasoning, taking into account, clinical expertise, and
© 2015 Fernandez et al. Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0
International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and
reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to
the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver
(http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.
Fernandez et al. Health Research Policy and Systems (2015) 13:66 Page 2 of 9

patients’ values and preferences. Remarkably, this broad contributed to the rise of EBM, whilst in the second, we
but sensitive approach to rational clinical decision making discuss why this movement evolved so rapidly and was
was actually followed when applied to guideline develop- so broadly accepted. The third describes a ‘restricted’ ap-
ment, but reduced the evidence in a skewed manner. Only proach to EBM and its use in designing standard
evidence from explanatory randomized controlled trials methods for developing practice guidelines, and finally,
(RCTs) was admitted as ‘reliable evidence’. we comment upon the current challenges faced by the
Whilst the value of EBM has been staunchly defended EBM movement in the context of systems thinking and
by its proponents, it has been widely criticized by many implementation sciences.
disciplines including clinical practice [3–8], epistemology
[9–14], health sociology [15, 16], and implementation Where does EBM come from?
science [17]. Moreover, in recent years, previously sup- There were three factors at the beginning of the 20th
portive EBM researchers argue for a ‘renaissance’ of the century that predated the development of EBM, namely
movement that follows and applies their original broad (1) the transformation of hospitals in the USA, from a
principles and multidisciplinary values, specially regard- shelter for the sick, to prestigious organizations, where
ing the components of EBM related to shared decisions medical care was based on scientific principles [29]; (2)
with patients and to expert judgment, built of evidence the reform of medical education [30], and (3) the birth of
and experience [18, 19]. The main argument is that, in clinical epidemiology [31]. The transformation of hospitals
spite of its benefits, EBM could have also had important was accompanied by a process of standardization of
negative consequences for healthcare delivery, policy healthcare provision through guidelines, which was also
and financing. Examples of this include (1) failing to closely related to the efforts of the American Medical
manage complexity, the individual’s needs, and the per- Association to establish its position as the reference ac-
son’s context and issues such as multi-morbidity; (2) the creditation body in medicine [29]. Standardization in-
quantity of research studies and the variable quality, which cluded the regulation of the medical profession, which
has become impossible to manage and in some cases lack ensured surgeons were well trained; the development of
clinical significance; and (3) the medicalization of life, procedural standards in hospitals, which reduced variabil-
namely creating new diseases for non-specific complaints ity and improved quality; and inclusion, for the first time,
and the use of the evidence-based ‘quality markers’ to of the patient record file, allowing hospital managers
widely promote drugs and medical devices [20–22]. to control what the physicians were doing [32]. As
This paper contributes to the descriptive rational recon- Timmermans and Berg suggested [29] the use of stan-
struction of EBM by analysing its historical development dards and guidelines, together with the emerging sci-
and controversies [23], as well as its limitations in the entific knowledge and technologies enabled the growth
current healthcare context. We approach this analysis of professional autonomy. However, standards and
from a complex adaptive systems science perspective with guidelines also became major triggers for the decline
its focus on the relational interactions of health and in clinical autonomy by the late 20th century [29].
healthcare variables [24] and the unified approach to the The subject of clinical epidemiology was progressively
philosophy of science as suggested by Schurz [23]. A com- introduced into medical programs based on the Enlight-
plex adaptive view of health as a balanced state between enment idea that progress was achievable through ob-
the person’s physical, social, emotional and cognitive jectivity and rationality, so medicine has to be a science,
experiences and its consequences for shaping complex not an art [29]. In 1968, McMaster University (Canada)
adaptive healthcare and healthcare systems as highly was the first to offer an integrative ‘problem-based learn-
responsive to the person’s unique needs as well as a ing’ curriculum, combining the studies of basic sciences,
complex adaptive understanding of medical knowledge clinical epidemiology and clinical medicine resulting
have been described in detail elsewhere [25–27]. The from clinical problems [31, 33]. The ‘father’ of EBM, David
unified approach to the philosophy of science provides a Sackett, directed this department. The publication of a
systematization of the basic assumptions of scientific series of recommendations by the Canadian Task Force
knowledge and revises the role of values in science. It on Periodic Health Examination that was led by Sackett
provides a new framework for understanding the dif- [34, 35] in 1979, underscored the rationale for using in-
ferences between the phases of discovery, corrobor- sights from clinical epidemiology to inform clinical prac-
ation and implementation in science. Its importance tice. The findings supported recommendations to
for defining new areas of scientific knowledge and abandon routine annual check-ups in favour of selective
the role of different logic inferences in each phase approaches based on the patient’s age and sex. It was the
have been reviewed elsewhere [28]. first time that recommendations were made according to
The present paper is structured in four sections: in the the ‘levels of evidence’ and exclusively based on ‘grading
first, we review the origins, principles and actors who study designs’, i.e. RCTs provide good evidence (level I),
Fernandez et al. Health Research Policy and Systems (2015) 13:66 Page 3 of 9

cohort studies and case–control studies provide fair evi- major principles [42]: (1) appropriateness, which relies
dence (level II), and expert opinion arising from clinical on standards and guidelines; (2) routine outcome assess-
experience provides poor evidence (level III). Unsurpris- ment based on routine and systematic measures of pa-
ingly the same basis for grading treatment recommenda- tients’ functioning and wellbeing, along with disease-
tions was applied from level A: to apply the intervention specific clinical outcomes at appropriate time intervals;
to level E: not to apply the intervention. (3) the link to data mining to pool clinical and outcome
A prerequisite for the widespread adoption of EBM re- data on a massive scale; and (4) a focus on dissemination
quired clinicians to be more critical when appraising the and impact analysis to take into account the segment of
scientific literature. In 1981, Sackett et al. [33] published the database most appropriate to the concerns of each
a series of articles in the Canadian Medical Association decision maker. OM differs from EBM in its emphasis
Journal that explained the criteria for assessing the in- on ‘real data’ in contrast with EBM’s ‘experimental data’,
ternal validity of study designs as RCTs providing the while both OM and EBM aimed to empower clinicians
gold standard for treatment, cohort studies for diagnosis, to improve their clinical decision making capacity
and case–control studies for etiology or harm. However, through the new tools on offer. This contrasts markedly
as Zimmerman indicated [31], this simplification was with the view of managerialism, or neo-liberal ap-
one of the most important weaknesses of EBM. Indeed, proaches, where the power of decision making is shifted
the major resistance to EBM relates to the specification from clinicians to managers and auditors [41, 43].
of the knowledge base of medicine as something The need for standardization, the development of clin-
rational/technical/linear/predictable rather than contin- ical epidemiology, and concerns about the economic
gent/experiential/non-linear/unpredictable [16, 26, 36]. sustainability of health systems together with the in-
Managing the vast amount of research literature be- creased capacity of computers to handle large amounts
came possible with wider availability of computers, in of data paved the way to the development of the EBM
particular the personal computer on the doctor’s desktop movement, officially founded in 1991 [33]. The publica-
[37]. This enabled Iain Chalmers, director of the tion of “Evidence-based medicine. A new approach to
National Perinatal Epidemiology Unit in Oxford (United teaching the practice of medicine” by the Evidence-Based
Kingdom) in the mid 1980s, to establish an electronic Working Group in JAMA [44] rapidly spread the concept
database of perinatal trials which made this information and principles of EBM allowing the Evidence-Based
readily accessible to clinicians [38, 39]. The concepts Working Group to pronounce EBM to be a ‘new para-
and creation of electronic databases and increasing com- digm’. It would change the ‘old way’ of ‘solely’ practicing
puting power facilitated the democratisation of know- subjectivity-based medicine predicated on intuition, clin-
ledge management, something previously confined to ical experience and pathophysiological rationale with an
only a few experts [40]. Some years later, The Cochrane objective approach based on ‘scientific’ evidence. Whilst
Collaboration emerged as an organisation that systemat- they advocated the addition of evidence as a key con-
ically combed, reviewed and synthesised the vast amount sideration, they also clearly rejected the role of ex-
of research literature to make it accessible to the perts with ‘authoritative opinions’ as guiding clinical
clinician at the time of the patient consultation. decision making.
Another contextual factor to explain the development
of the EBM was the rising concern about the sustainabil- Why was EBM so widely accepted?
ity of health systems during the 1970s. This concern re- From 1992 until September 2015, the PubMed database
sulted in the emergence of new disciplines, such as revealed over 20,000 papers with ‘evidence-based’ in
health economics, that influenced the development of their title. Evidence-based practice guidelines are the
major approaches to healthcare reform such as manager- norm for the majority of official agencies and profes-
ialism [41] and outcomes management [42], in addition sional organizations, and EBM approaches are at the
to EBM. These three approaches focused on the ‘specific’ core of today’s scientific thinking. The RCT is regarded
to achieve measurable objectives; continuous evaluation as the fundamental research response underpinning the
of performance against defined objectives, outputs and ‘perceived new paradigm’ of EBM for healthcare [45],
standards; and rationing of resources by effectiveness and these ideas have now expanded far beyond the realm
criteria to make the work of physicians more transparent of medicine (consider, for instance, the debate if Conser-
through control and surveillance. vation Science needs to include RCTs in the same way
Closely related to the development of Health Mainten- medicine does: http://blog.nature.org/science/2013/08/
ance Organizations was Outcomes Management (OM) 15/debate-randomized-control-trials-in-conservation/).
in the United States, which adopted the principles of As Pope suggested [16], EBM evolved as a social
quality improvement to facilitate physicians’ autonomy movement that started with agitation (i.e. we need to
and control of their clinical practice. OM follows four change the current paradigm based on experience). It
Fernandez et al. Health Research Policy and Systems (2015) 13:66 Page 4 of 9

was crystalized by the shared experience of the group at and Oxford University, respectively. Gordon Guyatt, as
McMaster University and the development of an endur- co-founder of the Medical Reform Group, a Canadian
ing sense of purpose, disseminated in a series of position medical group composed of young doctors and nurses
papers, declarations, and guidelines published in influen- based in Toronto [31], notably agitated against the prac-
tial medical journals by key opinion leaders in clinical tice of medicine guided by senior doctors’ opinions. Per-
epidemiology. So, ironically, the adoption of EBM by the sonal experience gained under extreme conditions shaped
scientific community was not based on evidence but on the views of Archie Cochrane, a doctor and prisoner of
authoritative knowledge, precisely the type of approach war, and Iain Chalmers, a doctor in Gaza; they realized
EBM was meant to replace, a point recently acknowl- that in many cases new expensive treatments were no
edged by one of its key proponents, Sackett himself [46]. better than older ones [50]. From any point of view, the
leading professionals were clearly well motivated but in
Use of authoritative knowledge in EBM practice their recommendations resulted in an over-
We can identify three factors related to authoritative simplified approach to ‘the clinical care of patients’.
knowledge that could have played a major role in the A related reputational effect is gained from the
success of EBM: reputation, the Matthew effect, and the Matthew effect [51] – raising the credibility of a
invisible college. viewpoint and an author group by excessive cross-
The first ever paper on EBM written by an almost an- citation amongst its proponents,b a practice utilized
onymous EBM Working Group a appeared in JAMA and by scientists since the 17th century. As a result, a
provided the movement with instant grounds of credibil- group is highly likely to gain influence and power to set
ity. Publishing under the authorship of a working group future research, practice and policy agendas (through
raised its status to that of an authoritative consensus grants, publications, conference presentations, etc.), made
paper. However, as Zimmerman suggested [31], the easier by the current state of publication policies and qual-
EBM Working Group used a language closer to a polit- ity assessment procedures [52].
ical manifesto, calling for far-reaching changes in the The extraordinary ability of the major EBM players to
practice of medicine, in the process creating an ‘enter- promote, implement and expand collaborative groups
prise of scientific objectivity’. This working group, to- and networking resulted in what is known as an ‘invis-
gether with Drummond Rennie, deputy editor at JAMA, ible college’ [51]. The invisible college consists of a
remained the main advocates of the EBM movement for group of scientists or professionals who may live in sep-
the first critical years: out of 22 articles on EBM published arate locations but attend the same conferences, publish
in the first 3 years, 12 were published by JAMA, reflecting in the same journals, and invite each other to give key-
Rennie’s and JAMA’s remarkable commitment to the new note lectures to share the same ideas. An invisible col-
approach [31, 46]. This new movement was not only sus- lege emerged from the collaboration between the groups
tained by JAMA, it also found the British Medical Journal at McMaster University and the Cochrane Collaboration.
to be its keen European supporter [1, 47]. One could argue that the Cochrane Collaboration over
Within 3 years, the movement was threatened by an time has morphed into a form of ‘visible college’. Indeed,
equally prestigious journal, The Lancet, which took a the Cochrane Collaboration’s initiative of a series of
critical position of EBM. In 1995, an anonymous editor- small workshops started an international social network
ial stating that although “The Lancet applauds practice of EBM supporters.
based on the best available evidence – bringing critically
appraised news of such advances to the attention of From a broad model to a narrow version of EBM
clinicians is part of what peer-reviewed medical journals The historical and philosophical basis for EBM started
do – but we deplore attempts to foist evidence-based with a broad health system’s perspective. In the 1930s,
medicine on the profession as a discipline in itself” [48]. the then medical student, Cochrane, demanded on a
The Lancet has since remained one of the most critical protest placard that “All effective treatment must be free”
journals about the EBM movement. For instance, in [53, p. 1]. This call was about demonstrating a cost/
2005, it published a paper entitled “External validity of benefit perspective, predicated on measuring “the effect
randomized controlled trials: To whom do the results of of a particular medical action in altering the natural his-
this trial apply?” criticizing the hierarchy of evidence as tory [sic] of a particular disease for the better” [53, p. 2].
its focus is internal validity, neglecting the critical issue Cochrane argued that the RCT would remove bias and
of external validity/generalizability of those results [49]. subjective opinion from managing disease, and indeed
The reputations of the EBM movement’s key propo- RCTs demonstrated important but limited gains in under-
nents and authors were well established. David Sackett standing therapeutic interventions. He clearly distinguished
and Iain Chalmers were renowned clinical epidemiologists between ‘effectiveness’ and ‘efficiency’ and observed that,
and worked in highly regarded institutions – McMaster while the RCT as a scientific method could demonstrate
Fernandez et al. Health Research Policy and Systems (2015) 13:66 Page 5 of 9

‘effectiveness’ in the trial population, this would not equate ‘Grading’ knowledge
to greater ‘efficiency’ in healthcare, i.e. the same outcomes The initial problems with translating evidence-based
would generally not be achieved in routine practice due to guidelines into practice were attributed to the difficulties
the “complexities within the health system” [53, p. 2]. In in properly rating the supporting body of research.
addition, Cochrane was much more interested in the as- Hence, a detailed analysis of the grading of RCTs was sug-
pects of care crather than cure, alluding to the often gested [59]. The Grades of Recommendation, Assessment,
neglected concern of ‘equality’ within the health system. As Development, and Evaluation (GRADE) working group
he stated: “In particular I believe that cure is rare while the analyzed six grading systems used by 51 organizations and
need for care is widespread [sic], and that the pursuit of found, to its surprise, poor reliability in the assessment of
cure at all costs may restrict the supply of care, but the bias the quality of studies. None of the systems were usable for
has at least been declared” [53, p. 7]. all user groups (professionals, patients and policymakers).
The beginnings of the EBM approach were clearly This study probably was the first major criticism of EBM
focused on understanding the complexities of the ‘work- by one of its leading groups, concluding that the systems
ings of the healthcare system’ and its relationship to for grading levels and strength of evidence had important
making the ‘best possible decision’s for the care of shortcomings for proposing clinical recommendations
patients’. However, these complexities have rapidly been [60]. The same group later recognized that the early
reduced to a narrow focus on standardised and typically systems of grading, which focused almost exclusively on
single disease management guidelines. randomized trials, were inadequate. The group realized
that observational studies had features that could both de-
Managing scientific knowledge for practice and the crease or increase the quality of the supporting body of re-
guideline development movement search used for recommendations, and that high quality
One of the main objectives of EBM is to make large observational studies could contribute to better clinical
amounts of scientific knowledge more accessible, and decision making [61]. The factors considered when rating
developing clinical guidelines with recommendations to evidence include the overall quality of studies, the coher-
support clinical decisions seemed the obvious way to ence of studies, uncertainties about the balance of benefits
proceed. versus harms, and uncertainties in values and opportunity
Although clinical guidelines are useful they are also costs – reconnecting with Cochrane’s much broader
limiting if, for instance, they only draw on one source of vision.
information (i.e. the explanatory RCT). These guidelines Recognition of the need to separate assessment of
will also restrain the freedom of professionals to use quality of evidence from the strength of recommenda-
other sources of knowledge in their clinical decision tions was a major step forward – “high quality evidence
making, like knowing patients’ preferences and clinical doesn’t necessarily imply strong recommendations, and
experiences [54]. Evidence-based guidelines for a specific strong recommendations can arise from low quality
area of practice are typically seen by clinicians as the evidence” [62]. However, a new question arose: who
penultimate and authoritative practice pathway, rein- should make these recommendations? Paradoxically,
forced by adverse litigation and clinical review commit- recommendations in clinical guidelines developed by
tee outcomes [29]. Consequently, many practitioners see ‘EBM experts’ use the opinion of their group of experts
clinical guidelines as the main threat to adapting clinical to work through and agree on the wording of their rec-
decisions to individual patients’ needs and contexts, i.e. ommendations, grade the body of evidence, and utilize
interfering with their necessary clinical autonomy. In- their values and background, i.e. apply their ‘prior expert
deed, EBM supporters like JR Hampton, 32 years ago, knowledge’. However, how the information, expert opin-
asked for the death of clinical freedom as they saw ‘clinical ion and contextual factors are balanced in these deliber-
judgment’ as the major obstacle to advancing medicine ations often remains unclear.
[55]; only recently they realized that clinical autonomy is In recognition of the need for systematic and explicit
needed so practitioners can use their ‘expert knowledge’ approaches in the development and grading of recom-
in the best interest of their patients [56]. mendations, guideline development standards now require
The preoccupation with the quality of the studies used a summary of findings [63, 64] or evidence statements
to develop clinical guidelines most likely explains the [65] for each recommendation and these are included in
transformation of the broader EBM framework into its the guideline’s technical report.
narrower RCT-driven form. The difficulties in translat-
ing the recommendations contained in EBM guidelines Various ways of knowing
into practice and policy and the consecutive process of The EBM community is starting to respond to criticism of
revision of the reductionist EBM approach to guidelines limiting evidence and is attempting to incorporate and
has been reviewed by others [49, 57, 58]. value other sources of knowledge, such as observational
Fernandez et al. Health Research Policy and Systems (2015) 13:66 Page 6 of 9

studies and the findings of qualitative research. Neverthe- controlled (RCTs) – often fail to be translatable into
less, the fundamental flaws inherent in the grading sys- practice because the research context does not reflect
tems remain such as the assumption that: study designs real world clinical practice/reality [67, 68].
and quality can be arranged in a systematic but simplistic The EBM and guideline communities have also recog-
linear structure, when, in reality, the use of different and nized the limited implementation of guidelines jeopardized
highly relevant information such as patient preferences, by their current static and unfriendly structure [18, 54, 69].
and applicability to local or practice contexts would lead New proposals, such as the development of dynamic wiki-
to completely different grading systems [66]; and that based clinical guidelines, might eventually resolve this prob-
studies graded according to epidemiological principles will lem and enable the participation of all stakeholders (e.g. pa-
diminish the risk of bias. This criterion may make sense in tients, clinicians and decision makers), in a collaborative
the discovery of new knowledge but is not the key consid- effort that may result in greater transparency and accept-
eration if the main aim is to implement research into local ability [70, 71].
practice. The leap from discovery/corroboration to implementa-
It is important to consider that scientific knowledge is tion was partly reflected in the criticism made in The
divided in three major areas: discovery, corroboration Lancet in 2005 [49] and in the position of other EBM ex-
and implementation, and that the information of one perts such as ER Epstein, who developed the disease
domain cannot be directly applied to the others [23, 28]. management approach, superseded by Wagner’s
The value-neutrality principle that guides discovery and chronic/integrated care model [71]. Even though he ad-
corroboration requires further – and complex – clarifi- hered to EBM, Epstein’s vision of health knowledge was
cation in the implementation phase. According to clearly beyond RCTs and much closer to Paul Ellwood’s
Schurz [23], the value-neutrality requirement “implies OM approach. He considered EBM as one of several
that the scientist separates her scientific knowledge from tools to improve quality of care: “The new paradigm is
fundamental value assumptions which she assumes in population-based risk and disease assessment, systems of
means-end inferences”. Means-end inferences and abduc- disease prevention and health promotion, community-
tion are used in EBM guideline development without an based intervention and provider contacts within a frame-
adequate formalization of their contribution to the con- work of automated information, evidence-based medicine,
struction of the guideline recommendations [28]. The and defined protocols of care, with explicit collection of
roots of this philosophical debate are far beyond the outcomes information” [72]. Epstein and Sherwood, al-
scope of this paper and, unfortunately, the philosophy of though subscribing to the gold-standard of RCTs, mention
science principles necessary to support the underpin- the difficulty of using them as the main source of informa-
nings of EBM have as yet not been properly explored. tion in outcome management/implementation and men-
In the fields of discovery and corroboration the im- tion ‘prospective effectiveness trials’ as the alternative to
portant criterion is internal validity (observed variation RCTs [72, 73].
can be interpreted as a causal relationship, therefore, the
study design needs to guarantee that the risk of bias is
low). In the field of implementation, the important cri- The limitation of RCTs to assess real world outcomes
terion is the degree of external validity of the results ap- A more fundamental question would be, can real world
plicability to the local context and acceptability of the outcomes be achieved/evaluated with randomized con-
intervention/s to the patient. External validity is import- trolled trials? In short, the answer is no, if we only use
ant as it means that the results can be generalized to dif- explanatory randomized trials as preferred by its propo-
ferent persons, settings and times. There is an inverse nents. However, pragmatic controlled trials that, by def-
relationship between internal and external validity. If the inition, are conducted under usual conditions offering
final purpose of EBM was to improve the health of real practitioners considerable freedom in deciding how to
people in real settings, external validity should be em- apply the intervention to be tested, are not obtrusive (i.e.
phasized and strengthened. It is not only crucial to know there is no special effort to improve compliance by pa-
if a treatment is effective in controlled situations (i.e. in- tients or practitioners), and use administrative databases
ternal validity), but also that it is going to be effective in for the detection of outcomes, can offer a valid alterna-
the real world (i.e. external validity). While the grading tive. While explanatory RCTs will be linked to discovery
systems for developing clinical guidelines used by EBM and corroboration and will aspire to removing variability,
are systematic and reliable, they often prioritize internal pragmatic controlled trials (even including randomization)
validity and therefore are not ‘fit for purpose’. The em- fit in the area of implementation and embrace variability
phasis on internal validity has contributed to the failure as the norm [74, 75]. They take into account the local con-
of EBM, as recommendations – being based on experi- text and are mostly valued when driven by theory and
mental designs where variables and confounders are complemented by other sources of knowledge [76].
Fernandez et al. Health Research Policy and Systems (2015) 13:66 Page 7 of 9

In conclusion: the challenges facing EBM evidence for clinical decision-making. In this sense, EBM
Most likely, EBM grew too fast to effectively incorporate needs to go beyond the sole use of the RCT and acknow-
its original propositions: evidence, expert knowledge, ledge that scientific knowledge is multidimensional and
and patients’ preferences [1]. The reliance of EBM on cannot be arranged in only one hierarchical system.
the RCT was useful for acute (mostly single disease) Knowledge coming from studies using different meth-
conditions treated with simple interventions, but this ap- odological approaches is complementary [28]. Hence, to
proach is not suitable in the current epidemiological have a complete picture, information coming from ex-
context characterized by chronicity and multimorbidity planatory RCTs has to be complemented and contrasted
in complex health systems. In particular, EBM has with information coming from pragmatic RCTs evaluat-
largely disregarded the importance of social determi- ing effectiveness in routine practice. This implies some
nants of health and local context – hence the nicknames loss of ‘internal validity’ and an increase in the uncer-
‘cookbook approach’ or ‘MacDonaldization’ of medicine tainty of the results, but ‘gains in representativeness’.
[29, 77]) – and its real impact on the ‘effectiveness’ and The most important challenge facing the EBM-
‘efficiency’ of healthcare on the ‘equality’ of needed movement is the provision of a detailed description of
healthcare services. its methods for scientific reasoning. This requires an
As an a priori, evidence is context sensitive, and there- analysis of its taxonomic principles, including formal
fore to some extent tacit [78], and both global and local definitions of ‘scientific knowledge’, ‘evidence’, and ‘deci-
evidence need to be combined in the development of us- sion making’ in health, as well as the different types of
able recommendations for clinical decision making [79]. logic inferences used in the scientific reasoning process
Local evidence includes the presence of modifying fac- [28]. As others have highlighted, we believe that this aca-
tors in the specific settings, magnitude of needs (preva- demic exercise is crucial to clarify the confusion between
lence, baseline risk or status), patient values, costs (to ‘good’ evidence [84] and scientific ‘truth’. Apart from
the patient and the system), and the availability of re- systems thinking [85, 86], healthcare researchers, clini-
sources in the system [80]. This local evidence needs to cians and policymakers could benefit from greater know-
be combined with ‘expert knowledge’, which should be ledge of the philosophy of science to design and
differentiated from ‘expert opinion’ and valued in a dif- interpret research, and their use in guiding decision
ferent way. By ‘expert knowledge’ we mean the implicit making processes – beyond the classical experimental/
knowledge that professionals have that helps them to deductive approach favoured by the EBM movement
better understand the local conditions. It is based on [18, 23, 28, 87].
data (their accumulated experiences) and thus different It should also be highlighted that health systems re-
to simple opinions or feelings about something [81, 82]. search involves different disciplines (including social
There is on-going debate of the relevance of ‘colloquial ones) with different perspectives, epistemologies, and
evidence’ in the development of guidelines [83]. This re- ways of conceptualizing and conducting research. Health
flects a worrying lack of a basic understanding by systems research, as intimated by Cochrane, is broader
authors and reviewers of the fundamentals of scientific than identifying ‘clinical effectiveness’ – ‘efficiency’ and
knowledge and the differences between expert know- ‘equality’ are equally important considerations for
ledge and evidence. achieving successful implementation of health system
There is an imperative to explore and then learn from improvement; therefore, all stakeholders’ fundamental
other disciplines on how to use research evidence and value assumptions should be explicit.
incorporate it with local context and expert knowledge
to achieve best possible patient outcomes. For example, Ethics
in other areas of science, e.g. conservation science and Ethics approval was not required as this manuscript is a
artificial intelligence, expert knowledge is routinely in- narrative review of published papers.
corporated in the analysis. Expert-Based Collaborative
Analysis is a systematic procedure to incorporate expert Endnotes
a
knowledge into data analysis; such an approach has been They signed as a group but in a footnote they
proven to be useful when dealing with complex issues included their names.
b
and can be seen as a powerful tool in the current health It refers to the fact that even though the majority of pa-
context characterized by an increase in the number of pa- pers have little impact on the scientific community (the
tients with multiple conditions, resulting from heteroge- hath not), a small number of them attracts the attention
neous genomic/pathophysiological pathways and diverse of the community and receive a lot of citations (the hath
personal needs [81]. abundance).
In future, the inclusion of ‘expert knowledge’ in the c
As an aside, Cochrane’s observations working as a
analysis of research data might produce more usable doctor in a prisoners of war camp provided him with
Fernandez et al. Health Research Policy and Systems (2015) 13:66 Page 8 of 9

the empirical evidence of the great benefits of care, its 12. De Vreese L. Evidence-based medicine and progress in the medical
scientific basis has been untangled by the study of sciences. J Eval Clin Pract. 2011;17(5):852–6.
13. Sehon SR, Stanley DE. A philosophical analysis of the evidence-based
psychoneuroimmunology. medicine debate. BMC Health Serv Res. 2003;3(1):14.
14. Holmes D, Murray SJ, Perron A, Rail G. Deconstructing the evidence-based
Abbreviations discourse in health sciences: truth, power and fascism. Int J Evid Based
EBM: Evidence-based medicine; OM: Outcomes management; Healthc. 2006;4(3):180–6.
RCTs: Randomized controlled trials. 15. De Vries R, Lemmens T. The social and cultural shaping of medical
evidence: case studies from pharmaceutical research and obstetric science.
Competing interests Soc Sci Med. 2006;62(11):2694–706.
The authors declare that they have no competing interests. 16. Pope C. Resisting evidence: the study of evidence-based medicine as a
contemporary social movement. Health. 2003;7(3):267–82.
Authors’ contributions 17. Porter S, O’Halloran P. The use and limitation of realistic evaluation as a tool
This review paper is the result of the discussions made by the for evidence-based practice: a critical realist perspective. Nurs Inq.
multidisciplinary expert group “Dialogues On Complexity And Health 2012;19(1):18–28.
Systems” (DOCAHS), to advance the understanding of complexity issues in 18. Greenhalgh T, Howick J, Maskrey N, Evidence Based Medicine Renaissance
health sciences, particularly in public health research held at the Menzies Group. Evidence based medicine: a movement in crisis? BMJ.
Centre for Health Policy, University of Sydney, in 2013 and 2014. AF was the 2014;348:g3725.
rapporteur of the sessions, and, together with LSC drafted the paper. AF, JS, 19. Greenhalgh T. Why do we always end up here? Evidence-based medicine’s
RM, SL, GT, RC and LSC reviewed the literature and contributed intellectually conceptual cul-de-sacs and some off-road alternative routes. Int J
to the debate and reviewed the different drafts. LSC guided the discussion. Prosthodont. 2013;26(1):11–5.
All authors approved the final version of this manuscript. 20. Moynihan R, Heath I, Henry D. Selling sickness: the pharmaceutical industry
and disease mongering. Br Med J. 2002;324:886–91.
Acknowledgements 21. Gøtzsche P. Deadly medicines and organised crime: how big pharma has
Other participants at some of the sessions of the DOCAHS were Abdolvahab corrupted healthcare. Oxford: Radcliffe; 2013.
Baghbanian, Di O’Halloran and Andrew Wilson. AF and LSC also thank the 22. Ioannidis J. Contradicted and initially stronger effects in highly cited clinical
students of the Unit of Study HSBH3005 “Evidence-based Healthcare”, Faculty research. JAMA. 2005;294(2):218–28.
of Health Sciences, University of Sydney, 2013, for their critical contribution 23. Schurz G. Philosophy of science. A unified approach. New York: Routledge; 2014.
to this debate. 24. Sturmberg JP, Martin CM. Handbook of systems and complexity in health.
New York: Springer; 2013.
Author details 25. Sturmberg JP. The personal nature of health. J Eval Clin Pract.
1 2009;15(4):766–9.
Brain and Mind Centre, Faculty of Health Sciences, The University of Sydney,
94 Mallett Street, Camperdown, NSW 2050, Australia. 2Discipline of General 26. Sturmberg JP, Martin CM. Knowing–in medicine. J Eval Clin Pract.
Practice, School of Medicine and Public Health, The University of Newcastle, 2008;14(5):767–70.
Newcastle, Australia. 3Centre for Disability Research and Policy, Faculty of 27. Sturmberg JP, O’Halloran DM, Martin CM. People at the centre of complex
Health Sciences, The University of Sydney, Sydney, Australia. 4Menzies Centre adaptive health systems reform. Med J Aust. 2010;193(8):474–8.
for Health Policy, School of Public Health, Sydney Medical School, The 28. Salvador-Carulla L, Fernandez A, Madden R, Lukersmith S, Colagiuri R,
University of Sydney, Sydney, Australia. 5Centre for Disability Research and Torkfar G, et al. Framing of scientific knowledge as a new category of health
Policy-Brain and Mind Centre, Faculty of Health Sciences, The University of care research. J Eval Clin Pract. 2014;20(6):1045–55.
Sydney, Sydney, Australia. 29. Timmermans S, Berg M. The gold standard: the challenge of evidence-based
medicine and standardization in health care. Philadelphia: Temple University
Received: 7 January 2015 Accepted: 29 October 2015 Press; 2003.
30. Flexner A. Medical Education in the United States and Canada. A Report to
the Carnegie Foundation for the Advancement of Teaching. New York:
References Carnegie Foundation; 1910.
1. Sackett D, Rosenberg W, Gray J, Haynes R, Richardson W. Evidence based 31. Zimerman AL. Evidence-based medicine: a short history of a modern
medicine: what it is and what it isn’t. Br Med J. 1996;312(7023):71–2. medical movement. Virtual Mentor. 2013;15(1):71–6.
2. Muir JG. Evidence-based healthcare. London: Churchill Livingston; 1996. 32. Rosenberg CE. The tyranny of diagnosis: specific entities and individual
3. Carrhill R. Welcome to the Brave-New-World of evidence based medicine. experience. Milbank Q. 2002;80(2):237–60.
Soc Sci Med. 1995;41(11):1467–8. 33. Sur RL, Dahm P. History of evidence-based medicine. Indian J Urol.
4. Miles A, Loughlin M. Continuing the evidence-based health care debate in 2011;27(4):487–9.
2006. The progress and price of EBM. J Eval Clin Pract. 2006;12(4):385–98. 34. The periodic health examination. Canadian task force on the periodic health
5. Wilson K. Evidence-based medicine. The good, the bad and the ugly. examination. Can Med Assoc J. 1979;121(9):1193–254.
A clinician’s perspective. J Eval Clin Pract. 2010;16(2):398–400. 35. Bell N, Connor Gorber S, Tonelli M, Pottie K, Singh H, Joffres M, et al. From
6. Sturmberg JP. EBM: a narrow and obsessive methodology that fails to meet ABCs to GRADE: Canadian Task Force on Preventive Health Care’s new
the knowledge needs of a complex adaptive clinical world: a commentary rating system for clinical practice guidelines. Can Fam Physician.
on Djulbegovic, B., Guyatt, G. H. & Ashcroft, R. E. (2009) Cancer Control, 16, 2013;59(12):1282–9.
158–168. J Eval Clin Pract. 2009;15(6):917–23. 36. Malterud K. The social construction of clinical knowledge - the context of
7. Silva SA, Wyer PC. Where is the wisdom? II – Evidence-based medicine and culture and discourse. Commentary on Tonelli (2006), Integrating evidence
the epistemological crisis in clinical medicine. Exposition and commentary into clinical practice: an alternative to evidence-based approaches. J Eval
on Djulbegovic, B., Guyatt, G. H. & Ashcroft, R. E. (2009) Cancer Control, 16, Clin Pract 12, 248–256. J Eval Clin Pract. 2006;12(3):292–5.
158–168. J Eval Clin Pract. 2009;15(6):899–906. 37. Bakken S. An informatics infrastructure is essential for evidence-based
8. Miles A. Evidence-based medicine: requiescat in pace? A commentary on practice. J Am Med Inform Assoc. 2001;8(3):199–201.
Djulbegovic, B., Guyatt, G. H. & Ashcroft, R. E. (2009) Cancer Control, 16, 38. Chalmers I, Hetherington J, Newdick M, Mutch L, Grant A, Enkin M, et al.
158–168. J Eval Clin Pract. 2009;15(6):924–9. The Oxford Database of Perinatal Trials: developing a register of published
9. Newton W. Rationalism and empiricism in modern medicine. Law Contemp reports of controlled trials. Control Clin Trials. 1986;7(4):306–24.
Probl. 2001;64(4):299–316. 39. Chalmers I. A register of controlled trials in perinatal medicine. WHO Chron.
10. Harari E. Whose evidence? Lessons from the philosophy of science and the 1986;40(2):61–5.
epistemology of medicine. Aust N Z J Psychiatry. 2001;35(6):724–30. 40. Georgiou A. Data, information and knowledge: the health informatics
11. Wyer PC, Silva SA. Where is the wisdom? I–a conceptual history of model and its role in evidence-based medicine. J Eval Clin Pract.
evidence-based medicine. J Eval Clin Pract. 2009;15(6):891–8. 2002;8(2):127–30.
Fernandez et al. Health Research Policy and Systems (2015) 13:66 Page 9 of 9

41. Beck M, Sara M. Quality management and managerialism in healthcare. multiple stakeholders including patients in the promotion of best practices
A critical historical survey. New York: Palgrave Macmillan Publishing; 2014. in Trauma care: the WikiTrauma interrupted time series protocol. JMIR Res
42. Ellwood PM. Outcomes management. N Engl J Med. 1988;318(23):1549–56. Protoc. 2015;4(1):e21.
43. Davies B. Death to critique and dissent? The policies and practices of new 71. den Breejen EM, Nelen WL, Knijnenburg JM, Burgers JS, Hermens RP, Kremer
managerialism and of evidence-based practice. Gend Educ. 2003;15(1):91–103. JA. Feasibility of a wiki as a participatory tool for patients in clinical
44. Evidence-Based Medicine Working Group. Evidence-based medicine. guideline development. J Med Internet Res. 2012;14(5):e138.
A new approach to teaching the practice of medicine. JAMA. 72. Epstein RS, Sherwood LM. From outcomes research to disease management:
1992;268(17):2420–5. a guide for the perplexed. Ann Intern Med. 1996;124(9):832–7.
45. Montori VM, Guyatt GH. Progress in evidence-based medicine. JAMA. 73. Wagner EH, Austin BT, Davis C, Hindmarsh M, Schaefer J, Bonomi A.
2008;300(15):1814–6. Improving chronic illness care: translating evidence into action. Health Aff
46. Smith R, Rennie D. Evidence-based medicine—an oral history. JAMA. (Millwood). 2001;20(6):64–78.
2014;311(4):365–7. 74. Relton C, Torgerson D, O’Cathain A, Nicholl J. Rethinking pragmatic
47. Ham C, Hunter DJ, Robinson R. Evidence based policymaking. BMJ. randomised controlled trials: introducing the “cohort multiple randomised
1995;310(6972):71–2. controlled trial” design. BMJ. 2010;340:c1066.
48. Evidence-based medicine, in its place. Lancet. 1995;346(8978):785. 75. Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG,
49. Rothwell PM. Treating individuals 1 - external validity of randomised et al. A pragmatic-explanatory continuum indicator summary (PRECIS): a
controlled trials: “To whom do the results of this trial apply?”. Lancet. tool to help trial designers. J Clin Epidemiol. 2009;62(5):464–75.
2005;365(9453):82–93. 76. Marchal B, Westhorp G, Wong G, Van Belle S, Greenhalgh T, Kegels G, et al.
50. Hill GB. Archie Cochrane and his legacy – an internal challenge to Realist RCTs of complex interventions - an oxymoron. Soc Sci Med.
physicians’ autonomy? Commentary. J Clin Epidemiol. 2000;53(12):1189–92. 2013;94:124–8.
51. Blashfield RK. Feighner et al., invisible colleges, and the Matthew effect. 77. Timmermans S, Mauck A. The promises and pitfalls of evidence-based
Schizophr Bull. 1982;8(1):1–12. medicine. Health Aff. 2005;24(1):18–28.
52. Pendlebury DA. The use and misuse of journal metrics and other citation 78. Polanyi M. Personal knowledge. Towards a post-critical philosophy. London:
indicators. Arch Immunol Ther Exp. 2009;57(1):1–11. Routledge; 1958.
53. Cochrane A. Effectiveness and efficiency: random reflections on health 79. Remme JH, Adam T, Becerra-Posada F, D’Arcangues C, Devlin M, Gardner C,
services. London: The Nuffield Provincial Hospitals Trust; 1972. et al. Defining research to improve health systems. PLoS Med.
54. Zuiderent-Jerak T, Forland F, Macbeth F. Guidelines should reflect all 2010;7(11):e1001000.
knowledge, not just clinical trials. BMJ. 2012;345:e6702. 80. Oxman AD, Schunemann HJ, Fretheim A. Improving the use of research
55. Hampton JR. The end of clinical freedom. Br Med J (Clin Res Ed). evidence in guideline development: 7. Deciding what evidence to include.
1983;287(6401):1237–8. Health Res Policy Syst. 2006;4:19.
56. Hampton J. Commentary: the need for clinical freedom. Int J Epidemiol. 81. Gibert K, Garcia-Alonso C, Salvador-Carulla L. Integrating clinicians,
2011;40(4):849–52. knowledge and data: expert-based cooperative analysis in healthcare
57. Naylor CD. Grey zones of clinical practice: some limits to evidence-based decision support. Health Res Policy Syst. 2010;8:28.
medicine. Lancet. 1995;345(8953):840–2. 82. Martin TG, Burgman MA, Fidler F, Kuhnert PM, Low-Choy S, McBride M, et al.
58. Ansari MT, Tsertsvadze A, Moher D. Grading quality of evidence and Eliciting expert knowledge in conservation science. Conserv Biol.
strength of recommendations: a perspective. PLoS Med. 2009;6(9), 2012;26(1):29–38.
e1000151. 83. Sharma T, Choudhury M, Kaur B, Naidoo B, Garner S, Littlejohns P, et al.
59. Moher D, Cook DJ, Jadad AR, Tugwell P, Moher M, Jones A, et al. Assessing Evidence informed decision making: the use of “colloquial evidence” at
the quality of reports of randomised trials: implications for the conduct of nice. Int J Technol Assess Health Care. 2015;31(3):138–46.
meta-analyses. Health Technol Assess. 1999;3(12):i–iv. 1–98. 84. McCaughey D, Bruning NS. Rationality versus reality: the challenges of
evidence-based decision making for health policy makers. Implement Sci.
60. Atkins D, Eccles M, Flottorp S, Guyatt GH, Henry D, Hill S, et al. Systems for
2010;5:39.
grading the quality of evidence and the strength of recommendations - I:
85. Sturmberg JP, Martin CM, Katerndahl DA. Systems and complexity thinking
critical appraisal of existing approaches The GRADE Working Group. BMC
in the general practice literature: an integrative, historical narrative review.
Health Serv Res. 2004;4:38.
Ann Fam Med. 2014;12(1):66–74.
61. Brozek JL, Akl EA, Alonso-Coello P, Lang D, Jaeschke R, Williams JW, et al.
86. Peters DH. The application of systems thinking in health: why use systems
Grading quality of evidence and strength of recommendations in clinical
thinking? Health Res Policy Syst. 2014;12:51.
practice guidelines. Allergy. 2009;64(5):669–77.
87. Sturmberg JP, O’Halloran DM, Martin CM. Health care reform – the need for
62. Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al.
a complex adaptive systems approach. In: Sturmberg JP, Martin CM, editors.
GRADE: an emerging consensus on rating quality of evidence and strength
Handbook of systems and complexity in health. New York: Springer; 2013.
of recommendations. 2008;336:924.
p. 827–53.
63. Cluzeau F, Burgers J, Brouwers M, Grol R, Makela M, Littlejohns P, et al.
Development and validation of an international appraisal instrument for
assessing the quality of clinical practice guidelines: the AGREE project.
Qual Saf Health Care. 2003;12(1):18–23.
64. Makarski J, Brouwers MC. The AGREE Enterprise: a decade of advancing
clinical practice guidelines. Implement Sci. 2014;9:103.
65. National Health and Medical Research Council. NHMRC levels of evidence
and grades for recommendations for developers of guidelines. Sydney: Submit your next manuscript to BioMed Central
NHMRC; 2009. and take full advantage of:
66. Green LW, Glasgow RE. Evaluating the relevance, generalization, and
applicability of research: issues in external validation and translation
• Convenient online submission
methodology. Eval Health Prof. 2006;29(1):126–53.
67. Metge CJ. What comes after producing the evidence? The importance of • Thorough peer review
external validity to translating science to practice. Clin Ther. 2011;33(5):578–80. • No space constraints or color figure charges
68. Steckler A, McLeroy KR. The importance of external validity. Am J Public
Health. 2008;98(1):9–10. • Immediate publication on acceptance
69. Francke AL, Smit MC, de Veer AJ, Mistiaen P. Factors influencing the • Inclusion in PubMed, CAS, Scopus and Google Scholar
implementation of clinical guidelines for health care professionals: • Research which is freely available for redistribution
a systematic meta-review. BMC Med Inform Decis Mak. 2008;8:38.
70. Archambault PM, Turgeon AF, Witteman HO, Lauzier F, Moore L,
Lamontagne F, et al. Implementation and evaluation of a Wiki involving Submit your manuscript at
www.biomedcentral.com/submit