Teaching and Learning in Medicine

An International Journal

ISSN: 1040-1334 (Print) 1532-8015 (Online) Journal homepage: http://www.tandfonline.com/loi/htlm20

Systems Thinking and Systems-Based Practice

Across the Health Professions: An Inquiry Into
Definitions, Teaching Practices, and Assessment

Margaret M. Plack, Ellen F. Goldman, Andrea R. Scott, Christine Pintz, Debra

Herrmann, Kathleen Kline, Tracey Thompson & Shelley B. Brundage

To cite this article: Margaret M. Plack, Ellen F. Goldman, Andrea R. Scott, Christine Pintz,
Debra Herrmann, Kathleen Kline, Tracey Thompson & Shelley B. Brundage (2017): Systems
Thinking and Systems-Based Practice Across the Health Professions: An Inquiry Into
Definitions, Teaching Practices, and Assessment, Teaching and Learning in Medicine, DOI:
10.1080/10401334.2017.1398654

To link to this article: https://doi.org/10.1080/10401334.2017.1398654

Published online: 28 Dec 2017.

Submit your article to this journal

View related articles

View Crossmark data

Full Terms & Conditions of access and use can be found at

http://www.tandfonline.com/action/journalInformation?journalCode=htlm20

Download by: [Australian National University] Date: 29 December 2017, At: 08:49
 TEACHING AND LEARNING IN MEDICINE
https://doi.org/10.1080/10401334.2017.1398654

GROUNDWORK

Systems Thinking and Systems-Based Practice Across the Health Professions: An

Inquiry Into Definitions, Teaching Practices, and Assessment
Margaret M. Plack a, Ellen F. Goldman b,c, Andrea R. Scott b, Christine Pintzd, Debra Herrmanne,
Kathleen Kline f, Tracey Thompson f, and Shelley B. Brundage g
a
Department of Physical Therapy and Health Care Sciences, School of Medicine and Health Sciences, George Washington University,
Washington, DC, USA; bDepartment of Human and Organizational Learning, Graduate School of Education and Human Development,
George Washington University, Washington, DC, USA; cOffice of Faculty Affairs and Professional Development, George Washington University,
Washington, DC, USA; dSchool of Nursing, George Washington University, Washington, DC, USA; eDepartment of Physician Assistant Studies,
George Washington University, Washington, DC, USA; fOffice of Medical Education, School of Medicine and Health Sciences, George Washington
University, Washington DC, USA; gDepartment Speech, Language, and Hearing Sciences, George Washington University, Washington, DC, USA
Downloaded by [Australian National University] at 08:49 29 December 2017

ABSTRACT KEYWORDS
Phenomenon: Systems thinking is the cornerstone of systems-based practice (SBP) and a core systems thinking; systems-
competency in medicine and health sciences. Literature regarding how to teach or apply systems based practice; medicine and
thinking in practice is limited. This study aimed to understand how educators in medicine, physical health sciences education;
therapy, physician assistant, nursing, and speech-language pathology education programs teach teaching; assessment
and assess systems thinking and SBP. Approach: Twenty-six educators from seven different degree
programs across the five professions were interviewed and program descriptions and relevant
course syllabi were reviewed. Qualitative analysis was iterative and incorporated inductive and
deductive methods as well as a constant comparison of units of data to identify patterns and
themes. Findings: Six themes were identified: 1) participants described systems thinking as ranging
across four major levels of healthcare (i.e., patient, care team, organization, and external
environment); 2) participants associated systems thinking with a wide range of activities across the
curriculum including quality improvement, Inter-professional education (IPE), error mitigation, and
advocacy; 3) the need for healthcare professionals to understand systems thinking was primarily
externally driven; 4) participants perceived that learning systems thinking occurred mainly
informally and experientially rather than through formal didactic instruction; 5) participants
characterized systems thinking content as interspersed across the curriculum and described a
variety of strategies for teaching and assessing it; 6) participants indicated a structured framework
and inter-professional approach may enhance teaching and assessment of systems thinking.
Insights: Systems thinking means different things to different health professionals. Teaching and
assessing systems thinking across the health professions will require further training and practice.
Tools, techniques, taxonomies and expertise outside of healthcare may be used to enhance the
teaching, assessment, and application of systems thinking and SBP to clinical practice; however,
these would need to be adapted and refined for use in healthcare.

Introduction
each discussed these concepts differently. These differen-
Systems-based practice (SBP) was identified as a core ces indicated a need to understand how each of the pro-
competency in graduate medical education as early as fessions defined, taught, and assessed systems thinking
1998 with the initiation of the Outcomes Project.1 Since and SBP better. In this article we provide an overview of
then it has been challenging for the medical profession SBP and its link to systems thinking; discuss some of the
to fully operationalize this concept. Recently, several challenges associated with teaching and assessing sys-
members of our research team involved in implementing tems thinking; and present the findings of our study of
interprofessional education (IPE) activities across medi- how faculty across the medicine and health professions
cine, nursing, physical therapy, physician assistants, and at our university define, teach, and assess systems think-
speech-language pathology programs at our university ing. We conclude with recommendations for enhancing
noticed that each profession was discussing SBP and its the teaching and assessment of these concepts in medi-
overarching construct of systems thinking; however, cine and health sciences from pre- to postlicensure.

CONTACT Margaret M. Plack [email protected] The George Washington University School of Medicine and Health Sciences, Department of Physical
Therapy and Health Care Sciences, 2000 Pennsylvania Avenue, NW Suite 218, Washington, DC, 20036, USA.
© 2017 Taylor & Francis Group, LLC
 2 M. M. PLACK ET AL.

Systems-Based practice drivers of output, and anticipating the impact of external

forces.27,28
SBP is a core competency for medical and health science
Medical and health science educators have recognized
professionals alike2–10 and is essential for maximizing the
the value of systems thinking in understanding disease
quality and safety of patient care.11 Common requirements
processes29 and examining medical errors.30 In 2005, the
across the professions include the ability to provide patient-
Institute of Medicine and the National Academy of
centered care, work effectively in different delivery settings,
Engineering issued a joint report recommending systems
coordinate care, consider costs and risks versus benefits,
thinking be widely applied to improve healthcare deliv-
advocate for quality, and participate in error detection and
ery.31 Soon after, systems thinking was recognized as the
prevention.12–14 To effectively implement and evaluate SBP
cornerstone of SBP for delivering safe, high-quality
in healthcare requires a broad understanding of what con-
patient care.11 More recently, systems thinking has been
stitutes a “system,” coupled with an understanding of SBP
identified as a requirement for teaching and transform-
and its linkage to systems thinking.11
ing the health sector, moving from disease to prevention,
A system is a set of interacting elements15 that are
and population health.32,33 Despite acknowledging its
interdependent and work together to perform the
value, literature regarding how to integrate systems
required functions needed to achieve a system’s
Downloaded by [Australian National University] at 08:49 29 December 2017

thinking into medical and health sciences education is

purpose.11 Systems have certain conditions that define
quite scarce and focuses more on directives to employ
their behavior.16,17 For example, each part affects the
systems thinking than the models, tools, and techniques
whole. Each part is necessary but insufficient for achiev-
needed.11
ing system aims and the effect of any part on the system
as a whole depends on the behavior of at least one other
part.17 Learning to see interrelationships rather than lin- Teaching and assessing systems thinking and SBP
ear cause-and-effect chains and recognizing that change
To date, although some have offered teaching tips, there
is a process are essential for understanding systems.18
is no consensus on best practices for teaching systems
In healthcare, the importance of understanding
thinking in medicine or the health professions.34,35 Fun-
systems as interrelated parts of a whole cannot be
damental questions, such as how and when to teach sys-
overstated.19,20 Literature identifies four major levels of
tems thinking remain unanswered, although work has
systems in healthcare delivery: the patient, the care team
begun in both areas. In medicine, nursing, and the health
(including health care professionals, family members,
sciences, some have suggested that systems thinking
and others), the organizations in which care is delivered,
requires both classroom and experiential learning,
and the environment (regulatory, market, and pol-
incorporating reflective practice and authentic experien-
icy).21,22 All interact with one another. Effective SBP is
ces.35–37 The question of when to teach systems thinking
sensitive to all levels and interactions11 and the applica-
has been addressed by a number of researchers. As a
tion of systems thinking is key to understanding interre-
higher order thinking skill, it has been suggested that
lationships shaping the behavior of systems.18,23,24 As
instruction in systems thinking begin early in elementary
Colbert et al. suggested, systems thinking is “the missing
grades38 and include identifying, describing, and analyz-
foundational construct underlying SBP behavioral
ing systems; appreciating behaviors of complex systems;
expectations.”25(p179)
and applying systems approaches and methodologies to
real-world issues.39 In medicine and nursing, advocates
believe that instruction should begin at the premedical
Systems thinking
level or prelicensure level and continue through resi-
Systems thinking is the discipline of seeing all elements dency and even into practice through continuing profes-
in a given environment as interrelating. It considers the sional education.34,40 Unfortunately, what adds to the
impact of actions in one part of the system on other parts challenge is the lack of faculty trained in systems think-
of the system and views change as a process rather than a ing available to teach and apprentice mentees in medi-
snapshot in time.18 It is a body of knowledge, theory, and cine, nursing, and the health sciences.25,40
techniques developed through study, coaching, and Assessing systems thinking is equally challenging.
experience, and it provides a framework for seeing inter- Assessment requires measuring the impact of instruction
relationships among elements, patterns of change, and on the student’s systems thinking abilities, which neces-
structures underlying complex situations.26 Key skills sitates agreement on what concepts are essential and
associated with systems thinking include challenging how achievement can be measured.25,28 In the health
mental models, identifying structural relationships, pre- professions in particular, assessment tools for systems
dicting changes if processes continue, determining thinking are rare.41,42 However, assessment models exist
 TEACHING AND LEARNING IN MEDICINE 3

outside of the health professions that may be of potential a basic interpretive study using semistructured inter-
use in health professions education. For example, based views supplemented by program information accessed
on their work around systems thinking interventions on the schools’ websites, as well as course material pro-
and assessment, Hopper and Stave43 developed a taxon- vided by the participants.
omy of systems thinking levels and assessment measures
grounded in general systems theory. In developing
Selection and description of participants
this taxonomy, they identified key components of sys-
tems thinking, developed benchmarks, and proposed We recruited a purposive sample.47,48 To obtain perspec-
tests to measure to assess outcomes at each level. The tives from across the health professions, research mem-
taxonomy includes seven levels of systems thinking, bers nominated faculty from their work units who taught
from lowest to highest: (a) recognizing interconnections, or were most familiar with the concepts of systems
(b) identifying feedback, (c) understanding dynamic thinking or SBP in their professions. We interviewed 26
behavior, (d) differentiating types of variables and flows, faculty members from the following professional educa-
(e) using conceptual models, (f) creating simulation tion programs: doctor of physical therapy (PT; n D 6),
models, and (g) testing policies. Assessment methods graduate medical education (GME; n D 3), graduate
Downloaded by [Australian National University] at 08:49 29 December 2017

range from lists and descriptions to story writing, com- nursing education (GN; n D 2), physician assistant stud-
puter modeling, and policy design. Few studies have ies (PA; n D 4), speech and language pathology (SLP;
addressed higher levels of the taxonomy.43,44 n D 4), undergraduate medical education (UME; n D 5),
Although systems thinking is the cornerstone of SBP, and undergraduate nursing education (UN; n D 2). The
our limited ability to articulate how systems thinking is programs varied in duration from 2 to 4 years with credit
taught, implemented, and evaluated in medicine and requirements ranging from 42 to 135 hours (excluding
health sciences often restricts its value to patient quality medicine), depending on degree and specialty. All pro-
and safety, effective teamwork, and the improvement of grams included a mix of didactic coursework and clinical
health systems design.11 The challenges we face in teach- experiences, though the proportion of each varied.
ing and assessing systems thinking and SBP also limit The participants were a mix of female (18) and male
the potential for cross-disciplinary collaboration and (eight) seasoned healthcare professionals and educators
potentially the pace at which research is translated across (assistant, associate, and full professors, program direc-
professions and from bench to bedside. tors, clinic directors) from a variety of backgrounds with
10 to 45 years of experience.They taught a variety of
didactic, clinical, and research courses to undergraduate
Problem and purpose
and graduate students, some of which included content
As researchers from across the health professions, we related to systems thinking or SBP. Prior experiences
recognized the need to articulate more clearly, how sys- included course work in public health, engagement in
tems thinking is defined, taught, and assessed. Having a advocacy and national policy development, work in
common understanding would enable us to develop a managed care environments and nonprofit health sys-
more effective model of training that recognizes the tems, and experience on regulatory and licensure boards.
diversity of thought across the health professions and Participants have served on a variety of SBP committees
fosters inclusion through complementary practices that and are involved in didactic and/or clinical curriculum;
support high-quality interprofessional systems thinking some have held administrative and leadership roles
and SBP. within their departments.
The aims of this study were to
1. identify how the various health professions define,
Procedure
teach, and assess systems thinking and SBP;
2. articulate the gaps and challenges in current teach- The George Washington University Institutional Review
ing and assessment practices; and Board approved this study as exempt (IRB #041608).
3. recommend improvements in teaching and assess- Participation was voluntary and unrelated to any evalua-
ing systems-based thinking and SBP. tion; we offered no incentives and obtained informed
consent verbally from all participants prior to data
collection.
Methods
We used a 60-minute semistructured interview proto-
As we were interested in understanding how various col (see the appendix) to interview participants in homo-
health professions teach and assess systems thinking and geneous groups of two to three by profession, except for
SBP, we chose a qualitative approach.45,46 We completed two faculty members interviewed individually due to
 4 M. M. PLACK ET AL.

their time constraints. We used small groups to maxi- We paired up and assigned each pair two additional
mize knowledge capture on how each educational pro- interviews to code using the agreed-upon codes. We met
gram defined, taught, and assessed systems thinking. again to review the coded data, discuss discrepancies,
Given our experiences and literature review, which noted and once again refine and reach consensus on codes. As
a lack of a shared understanding around systems think- the analysis continued we identified instances of congru-
ing concepts among professions, at the end of each inter- ence with existing frameworks found in the literature
view we provided participants with a common language (e.g., four levels of the healthcare system), which were
using the Hopper and Stave43 taxonomy to reflect on its explored further deductively. Pairing team members and
potential relevance and usefulness. We invited partici- having them code transcripts of participants not in their
pants to take part in the study via e-mail. Participants own professional fields and discussions at team meetings
received the interview protocol prior to the interview. In helped mitigate any potential biases or preconceptions
addition, we asked participants to provide materials that individual team members may have brought to the
from courses that addressed systems thinking and SBP. analysis process. Team members created profiles sum-
Participants provided course syllabi, course learning marizing program information in each category and
objectives, project and assignment descriptions, grading returned them to participants for member checks. Two
Downloaded by [Australian National University] at 08:49 29 December 2017

rubrics, evaluation tools, and other relevant materials. researchers (ARS, KK) reviewed the supplemental mate-
Our eight-member research team represented a variety rials provided using the same iterative process. Materials
of disciplines at the George Washington University and included didactic and clinical course syllabi, learning
was inclusive of each of the professional education pro- activities, and assessments from across the professions.
grams under study. The team comprised one faculty mem- We analyzed materials for evidence of teaching and
ber (EFG) who has a dual appointment in the Graduate assessment of systems thinking and SBP.
School of Education and Human Development (GSEHD) Throughout the process, we continually compared
and the School of Medicine and Health Sciences (SMHS), data across programs to identify themes and negative
two from SMHS (DH, MMP), one from the School of cases.49 Refinement of themes was ongoing, and the pro-
Nursing (CP), one from the Columbian College of Arts cess continued until no new codes or themes were identi-
and Sciences (SBB), a doctoral candidate from GSEHD fied (i.e., saturation was reached) and we unanimously
(ARS), and two professional staff members from SMHS confirmed the accuracy of the findings.
(KK, TT). Faculty researchers have had 10 to 21 years of We used the following strategies to maximize credibil-
experience teaching in the professions, all have been ity and trustworthiness of this study: triangulation of
engaged in IPE, and all were familiar with the concepts of researchers and sources of data,45,50,51 purposive sam-
systems thinking as they relate to their own professions. pling for diverse perspectives,47,48 rich descriptions to
Two staff researchers have had responsibilities for curricu- support emergent themes,45,49,51 member checks,45 code
lum development and have participated on a SBP curricu- checking and peer debriefing to ensure accuracy of inter-
lum task force. Two team members also had coursework pretation and presentation of findings,45 and use of dev-
in systems thinking as part of their MBA degree programs il’s advocate and negative case discussion in building
and coconducted an extensive literature review on the consensus around findings.49 Finally, we made methods
topic, which we used to inform the research team further. transparent to enable readers to judge the credibility and
Seven of eight members conducted interviews, and to miti- transferability of the findings and conclusions.45,46
gate potential bias we matched interviewers with interview-
ees not from their own department. We audiotaped each
Results
interview and had each professionally transcribed verbatim
with identifying information removed. Analysis of the 17 interviews with 26 participants along
with materials from 55 courses resulted in six themes
detailing how faculty from across the five health profes-
Analysis
sions define, teach, and assess systems thinking and SBP.
We used an iterative approach to the qualitative data
analysis including both inductive and deductive meth-
Theme 1: Participants described systems thinking
ods. We compared units of data to one another and to
as ranging across four major levels of healthcare
the literature to identify patterns and themes.46 We
(i.e., Patient, Care Team, Organization, and External
began inductive analysis by having all researchers ana-
Environment)
lyze the same transcript, examining it for clusters of
meaning to develop initial codes. The team came Analysis of concepts and phrases used by participants to
together to refine the initial codes and reach consensus. describe systems thinking revealed conceptualizations
 TEACHING AND LEARNING IN MEDICINE 5

Table 1. Participants’ recognition of systems thinking at different levels of the healthcare delivery system.
Level Illustrative Quotes

Patient  We use a lot of frameworks to help conceptualize the patient, the environment that the patient’s living in (PT).
 Understanding of each body system and the interactions among the various body systems (PA).
 We start small. What are you going to see in your client? What are some of the symptoms you’re going to see? What do we do to treat the
disease?—so the pathophysiology and then treatment (UN).
Care team  Patient-centered would be the closest because it would assume that person exists in a context within systems but that are unique to each
person. It would probably include everything from family context, work environment (SLP).
 Clear communication with the physician amongst all the team members nursing, the physician, the whole care team, how—where am I
within that care team when I’m going to see that patient? (PT).
 How as a clinician we are practicing in the model of our entire medical community with all the other people who are working and
contributing to take care of patient care there’s an attending residents nurses techs secretaries and they integrate with the ICU and with the
floor and with orthopedics and all these different levels that are coming together to take care of that one patient (GME).
Organization  …. Systems thinking is a very comprehensive approach to looking at a particular solution not just from what the particular problem is, but
broadening it out to see what has an impact on trying to find solutions (UN).
 Systems thinking is looking at the macro view looking at the dynamics of your organization in relation to other organizations (GN).
 To be looking at the complex relationships and associations between multiple stakeholders in an organization or in an institution as well as
to look at how one drives improvement within that system (PT).
Environment  When they say systems thinking they’re thinking healthcare system and payment models [people] who are more interested in the
or society underserved patients and the low health literate patients. We’re thinking all the people who could potentially touch my patient and help.
Downloaded by [Australian National University] at 08:49 29 December 2017

People who are more interested in the management piece are more thinking of throughput and how can we officially be working as a
system. It seems to really change based on the scenario of the people who are discussing it (GME).
 Health disparities social determinants, how the social factors are key parts of people’s healthcare and health and where does that role play
in the system that you’re working in as a clinician? (PA)
 We talk about [systems thinking as] how can you affect change and how can you think beyond these kind of constraints that you have (PT).

Note: GME D graduate medical education; GN D graduate nursing; ICU D intensive care unit; NICU D neonatal intensive care unit; PA D physician assistant; PT D
physical therapy; SLP D speech-language pathology; UME D undergraduate medical education; UN D undergraduate nursing.

at all four levels of the healthcare delivery system21 (see  Recognize the basic structure and function of the
Table 1). Overall, there was no consistent language used U.S. healthcare and public health systems (environ-
to describe systems thinking and no consensus on mental level, UME).
the definition of systems thinking across or within
professions.
Theme 2: Participants associated systems thinking
Course materials confirmed course objectives
with a wide range of activities across the curriculum
addressed concepts related to all four levels of the
including quality improvement, IPE, Error Mitigation,
healthcare delivery system. Although all professions
and Advocacy
addressed the patient level, the degree to which each
profession addressed the other levels varied. For Participants often associated systems thinking with qual-
example, the PA curriculum primarily addressed the ity improvement and medical error mitigation, working
care team level, UN/GN addressed the organization in teams and interprofessional education, and advocacy,
and environment levels, UME primarily addressed the as the following quotes illustrate: “It’s improved out-
care team and environment levels with limited cover- comes … improving the quality care we give” (UN); “the
age of the organization level, SLP addressed all levels concept of quality improvement and M&M [morbidity
in one course with limited coverage in other courses, and mortality]” (GME); it’s related to “healthcare reform
and PT addressed all levels as a thread across the cur- terminology, which does include things about quality,
riculum. Examples of learning objectives included the quality initiatives, quality improvement, and systems
following: that are within quality improvements or methods within
 Understand how stuttering may influence the lives quality improvements” (PT); “You have to understand
of persons who stutter and their families (patient where errors can occur and how to problem solve those
level, SLP). errors” (UME); “Important concept in systems thinking
 Participate efficiently, collaboratively, and profes- is that idea: How do you bring groups together and
sionally with other healthcare providers and team develop a new and perhaps improved system?” (PT);
members within the context of inpatient surgical “Working in an interdisciplinary team … [you need to]
care (care team level, PA). know who to refer to and when to refer” (SLP); and
 Define the origin and purpose of quality in health- “How do you make a change to a system? … You have
care in relation to enhancing the work environment to be the squeaky wheel, you have to advocate for those
to reduce errors and improve results (organization changes, and so advocacy falls under that systems-based
level, UN). practice” (UME).
 6 M. M. PLACK ET AL.

Course materials confirmed the presence of learning I think academia really drives these things. … Our job is
objectives associated with these same concepts: to be reflective, to be collaborative, to be all-inclusive,
“Design action plans for quality improvement that and to make sure that our professions are paying atten-
tion to that. (SLP)
include barrier analysis, the establishment of appropri-
ate high-impact performance goals and monitoring
processes” (quality improvement, UN), “Use knowl-
Theme 4: Participants perceived that learning
edge of the roles and perspectives of different health-
systems thinking occurred primarily informally
care professionals to work collaboratively in a
and experientially, rather than through formal
respectful inter-professional team” (IPE; UME), and
didactic instruction
“Identify mechanisms to advocate for changes in laws,
regulations, standards, and guidelines that affect physi- Participants indicated students learn systems thinking
cal therapist practice” (advocacy; PT). mainly while observing, being immersed in, or interact-
ing with elements of systems, either during clinical expe-
riences or after graduation. Exemplary quotes included
Theme 3: The need for healthcare professionals the following:
to understand systems thinking was primarily
Downloaded by [Australian National University] at 08:49 29 December 2017

externally driven Examples come to mind when the student really starts to
notice how they can’t just order any laboratory diagnos-
Most participants described the drivers of systems think- tic test that they want because maybe the patient … does
ing as being external to the organization, such as accredi- not have insurance and cannot afford it and then they
tation requirements; pressures for cost containment, realize: Well, ultimately it’s not going to affect how they
manage that patient, so what is the value of it? … We
quality care, and reduction of medical errors; health care
can say it in class but until they actually have a name
reform; and consumers. Both UME and GME pointed to and a face of that patient that it’s directly affecting, then
the Accreditation Council for Graduate Medical Educa- I think that’s the ah-ha [sic] moment. (PA)
tion (ACGME): “The two biggest drivers are the
ACGME competencies … [and the] clinical learning A lot of what we do learn is trial and error: We learn
environment review [visits]” (GME) and “The ACGME from our attendings … from the nurses … from our
adopted these competencies; everyone sort of fell in line” patients … from all these people around us, and then
you start to figure out what the best practices are and
(UME).
especially when they align with who you want to be as
Nursing focused on healthcare reform, cost contain- you envision yourself as a clinician. … It is a process.
ment, and quality care: “Healthcare reform … changed (GME)
payments to outcomes … incentivizes all the players to
look at systems” (GN) and Several participants suggested that students initially
need to focus on clinical practice and learning the skills
that desire to reduce error but also to be more mindful of associated with it. Once they have grasped those skills,
… cost containment … not paying for readmissions if they are ready to learn about systems, which some con-
they happen within 30 days because instructions weren’t
given or things were missed … so the insurance compa-
tended is not until the later part of their program or even
nies, especially Medicare and Medicaid, won’t pay for it. at the graduate level.
(UN)
If we started doing it earlier, they might not be ready for
it, because just like the undergrads when they don’t real-
Still other comments related to patients and consum-
ize what is and isn’t out of scope of practice … in grad
ers themselves as drivers: school, finding out where those boundaries are is impor-
I think the consumer is really driving a lot of this too tant, and so if we blurred the lines too early it might be
because they want to go to the hospital that has the low- more problematic. (SLP)
est C-section rate and they want to go to the hospital
that has the lowest surgical site infection rate, … that’s
also a big driver. (GME) Theme 5: Participants characterized systems
thinking content as interspersed across
Although most described the drivers as being external,
the curriculum and described a variety of strategies
two participants also described internal drivers:
for teaching and assessing systems thinking
The electronic medical record was designed to organize
a system … a lot of the changes I’ve seen in sort of how Although some programs embed specific models such as
our operation runs were because of the electronic medi- the International Classification of Functioning, Disability
cal record but not necessarily because of the technology, and Health (ICF) or quality improvement models
because of the way the systems would change. (UME) throughout their curricula, systems thinking frameworks
 TEACHING AND LEARNING IN MEDICINE 7

were limited. Most participants described systems think- papers. PA syllabi indicated limited evidence of assess-
ing as present in their didactic or clinical experiences ment other than use of the final preceptor evaluation
across the professions; however, it is not always system- form. Most graduate nursing courses had assignments
atically or intentionally integrated as these participants that required students to analyze systems issues using
noted: “While our objectives were based on the ACGME interventions, case studies, and presentations. In SLP,
core competencies, of which one of them was systems- although systems thinking was identified in two course
based practice, there was no structured curriculum for syllabi, specific assessments of systems thinking were not
systems specifically” (UME); “I think it goes throughout evident. UME syllabi revealed evidence of the assess-
the curriculum because you want to encourage them to ments using multiple-choice exams (body systems),
just develop it as they’re moving through the program” reflection papers, projects, and subjective evaluations;
(UN); and “Systems are talked about in most of the clas- however, assessment of systems thinking was limited in
ses; I would be surprised if they weren’t. I’m not sure it’s the clinical courses.
the same system; I’m not sure that we actively go out and
say … [Let me] tell you about systems thinking in voice
Theme 6: Participants indicated a structured
class. … I don’t think I do that, that systematically”
framework and interprofessional approach may
Downloaded by [Australian National University] at 08:49 29 December 2017

(SLP). It was also noted that inclusion of systems think-

enhance teaching and assessment of systems
ing concepts is often dependent upon particular instruc-
thinking
tors as this person suggests: “It might be faculty member
dependent on whether or not I think that is a fundamen- At the end of each interview, when shown the published
tal part of treatment” (SLP). Faculty also indicated that framework for teaching and assessing systems thinking
this lack of cohesion and intentional instruction leaves by Hopper and Stave,43 most participants could better
students with gaps, as this individual noted: “They will articulate how systems thinking was already being taught
have a lot of the building blocks but not many of the con- or addressed in their curriculum. Participants voiced
nections” (PA). familiarity with some systems thinking tools (e.g., con-
Participants discussed and course materials confirmed cept maps, flowcharts, and root cause analysis), and all
the use of teaching strategies such as case scenarios, proj- believed the framework could enhance the teaching and
ects, group discussions, reflection papers, oral presenta- assessment of systems thinking—although some noted,
tions, role-play, and to a lesser degree concept maps, to be effective, the taxonomy would need to be less theo-
flowcharts, feedback loops, and root cause analyses. For retical and more applicable to clinical settings. Some
example, participants from both nursing and GME indi- were concerned also that it was too broad and suggested
cated they used root cause analysis, PT used reflection a change to make it more user friendly for educators and
papers, SLP and PT used concept maps, PA used per- ultimately learners.
spective taking, and UME used flow charts.
I like the idea of the taxonomy … in trying to break it
Some participants described the assessment of sys- into the parts. … Even as I think about it in a curricu-
tems thinking as subjective, observational, informal, and lum, this allows for instance to say by the time someone
challenging to assess, as these two participants suggest: graduates. … I want them to be able to recognize and
maybe identify [interconnections and feedback], but I
There’s a lot of “subjective” feedback that happens … don’t want them to be … testing policies. … I like that
that’s how we assess it, and it’s a case-by-case basis it’s broken down; it’s been thought through at that level.
where we’re teaching it. It’s not a formalized curriculum, I like the listing of the tests; that’s very helpful. (PA)
but it’s more like how to use the resources in real time.
(GME)
To make this applicable to our curriculum, we would
have to tweak it to push it a little bit back toward health-
It’s all observer based too. … That’s the only way they care systems so we’re so focused on the hospital, the
are being assessed on—based on how they’re performing clinic, the patient. (UME)
and what they’ve used, so it’s pretty subjective. Outside
of that, I don’t think there is any other assessment. …
The terminology is quite foreign. … I think it’s an inter-
[It’s] a hard-to-assess area. (UME)
esting context, a way of structuring issues. … I’m not
sure I would be able to, at first glance, differentiate each
A review of course materials found that systems
of these levels, but that’s not to say I don’t think there
thinking was assessed through several means including couldn’t be some value in having a conceptual frame-
projects, case studies, exams, presentations, reflection work that cuts across clinical and academic areas. (SLP)
papers, and the use of various evaluation forms for
assessment. PT primarily assessed systems thinking with Absolutely! This [framework] is really—it gives a struc-
projects, case studies, concept maps, and reflection ture for the students to look at this whole concept. …
 8 M. M. PLACK ET AL.

They’re standing on that scaffolding and they’re building activities and skills rather than as a metacognitive pro-
that towards their broader project. (GN) cess to be learned and honed over time. However, this
type of thinking is unlike some in the literature such as
O’Brien et al.,37 who advocated for early authentic clini-
Discussion
cal experiences, or Whitehead and Scherer40 and Salter,
In this study, we aimed to identify how educators across Phillips, and Dolansky,34 who suggested that systems
the five professions represented in the study (medicine, thinking be introduced early and revisited frequently
nursing, physical therapy, physician assistant, and from undergraduate education through residency train-
speech-language pathology) defined, taught, and assessed ing and beyond. Colbert et al.25 also advocated for the
systems thinking and SBP. We identified six themes in identification of a series of milestones along the contin-
which participants confirmed the lack of a shared defini- uum from novice to expert to guide the instructional
tion and noted that the drivers were external to acade- process.
mia. Participants also described a variety of learning It was evident that systems thinking is present in the
activities associated with these concepts and discussed curricula of all five professions; however, the approach to
various teaching and assessment strategies but believed teaching and assessing systems thinking was not always
Downloaded by [Australian National University] at 08:49 29 December 2017

that learning occurred primarily informally and experi- intentional or systematic. Learning objectives, assign-
entially. Finally, all welcomed a structured framework ments, projects, and papers addressing concepts of sys-
and perceived its use with an interprofessional approach tems thinking across all four levels were present in the
could enhance current teaching and assessment of sys- documents provided, but it was not consistent or com-
tems thinking. prehensive and at times depended upon the instructor of
Although participants did not use consistent language a given course.
to describe or define systems thinking, they voiced an Although quality care in the various health profes-
understanding of different levels of systems as outlined sions demands competency in systems thinking and/or
by Reid et al.21 They articulated how all levels interact SBP and it is evident that many participants attempted
with one another in the delivery of healthcare services to integrate activities requiring systems thinking into
and noted how changes in one part of the system can their curricula, there remains no clear consensus on defi-
affect another. Despite being driven by external forces, nitions or best practices in teaching or assessing these
all participants embraced systems thinking as central to concepts across the professions education.11,25 The litera-
improving healthcare delivery.11,31 In describing how ture offers some explanations. Many educators in the
systems thinking was taught, it was most often associated health professions are expert clinicians, and as our par-
with activities (e.g., quality improvement, IPE, medical ticipants noted, they learned systems thinking informally
error mitigation, and advocacy) rather than as a distinct and on the job. Many were not trained in the metacogni-
process or way of thinking. Some participants seemed to tive processes or analytical tools underlying systems
assume that to effectively perform activities such as IPE thinking, making it challenging for them to break down
or quality improvement, they must be using systems systems thinking into the requisite knowledge and skills
thinking rather than identifying systems thinking as the needed for instruction.25,27,28 Lacking this background
metacognitive process underlying these activities that limits their effectiveness in training and in properly
needs to be taught. assessing learners.40 Finally, although other professions
Norman36 suggested that teaching systems thinking in like business, engineering, computer science, and even
health sciences requires classroom and experiential divinity have various models, tools, and techniques to
learning, as well as reflective practice. Most of our partic- teach systems thinking, these are generally lacking in the
ipants believed that students learn systems thinking health professions.11 Without these models, tools, and
informally in the classroom and clinic and often do not techniques, as well as experienced educators who know
have the opportunity to truly apply their skills until after how to use them, it can be very challenging to teach sys-
graduation when immersed in the system to solve prob- tems thinking.
lems. Some participants believed that learning systems Systems thinking and SBP are essential to maximizing
thinking is a developmental process, suggesting that stu- the quality and safety of patient care.11,25 The lack of
dents be introduced to the concepts of system thinking training, knowledge of tools, and a framework for teach-
later in the curriculum, noting they must first learn about ing and assessing systems thinking noted in our study is
their own clinical practice and the scope of practice of problematic. Our findings indicate that, with few excep-
other practitioners before they can begin to integrate sys- tions, health professions educators at our institution
tems thinking fully. This makes sense given that many of could use assistance in helping their learners develop
our participants associated systems thinking with these essential skills, and our institution is not alone.25,37
 TEACHING AND LEARNING IN MEDICINE 9

When shown the published framework for teaching and To be effective, all of this must start with faculty
assessing systems thinking,43 participants indicated that development. Further, for systems thinking and SBP to
it would be useful but were uncertain how to operation- permeate the healthcare system and for our learners to
alize it, and some suggested that it be modified to be develop the “habit of mind” essential for effective sys-
more user friendly for the healthcare educator. tems thinking, education must move beyond faculty to
Based on the results of our study and a review of the include continuing education for licensed healthcare
most current literature regarding the teaching and providers as well.34 Institutions should design and offer
assessment of systems thinking, we offer several recom- professional development activities to formally train fac-
mendations for future consideration and exploration. ulty and licensed healthcare providers in the metacogni-
Our first recommendation is to develop a consensus defi- tive processes and analytical tools underlying systems
nition of systems thinking across the health professions. thinking, as well as provide strategies for teaching and
SBP is how systems thinking is applied in medicine; assessing outcomes. In doing so, we agree with
however, although systems thinking is used across pro- Whitehead and Scherer40 and recommend that they
fessions, SBP is not. Developing a shared mental model look beyond healthcare to capitalize on the work of
and common language would serve to enhance interpro- nonmedical experts in areas steeped in systems thinking
Downloaded by [Australian National University] at 08:49 29 December 2017

fessional collaboration. We could accomplish this such as engineering and business. Faculty interviewed
through a joint task force with members from across the were receptive to the Hopper and Stave43 taxonomy
health professions. presented and suggested it would help them organize
In developing that shared model, we advocate their understanding of systems thinking and SBP; how-
emphasizing systems thinking as a metacognitive pro- ever, they noted it would need to be adapted to make it
cess. We should focus on systems thinking as a higher more relevant for the health professions. Coupling the
order thinking process using the reflective process to expertise of nonmedical faculty with health professions
monitor our thinking and processing of information52 faculty could guide the development of curricula around
rather than solely a set of skills or specific competencies systems thinking in the health professions. Curricula
to be learned or implemented. As with critical thinking should include the same models, tools, and strategies
and reflective practice, this can be challenging; however, needed to learn the requisite metacognitive skills of sys-
removing the focus from specific activities such as IPE tems thinking. Curricula should adopt a longitudinal
or quality improvement projects to the development of approach that adds complexity as learners move from
thinking skills would be the first step in broadening fac- early didactic work in the classroom to their clinical
ulty perspectives on how and when to teach systems training phases. Identifying benchmarks for achieve-
thinking. ment along the continuum from novice to expert would
If our goal is for learners to develop systems thinking also help the educator in teaching and assessing systems
as a “habit of mind,” it would necessitate early introduc- thinking.25
tion of basic concepts, which faculty can use to scaffold We note some limitations to this study. This is a qual-
learning.53–55 From there, faculty can provide progres- itative study, which limits but does not preclude
sively more challenging situations to develop the com- generalization.57 The value of qualitative research is in
plex thinking skills required of a systems thinker.35 On a the thick, rich description it provides. The quotes pro-
basic level, early in the curricula, exploration of various vide a deeper context enabling the reader to judge trans-
health conditions using tools such as the ICF,56 devel- ferability or the applicability of the results to their own
oped by the World Health Organization, can help learn- circumstances.58 To help the reader we provide a trans-
ers begin to conceptualize systems and utilize systems parency in methods and analysis, direct quotes for con-
thinking. Using the four levels of the healthcare system text and evidence of each theme identified, and multiple
(i.e., patient, team, organization, and environment) can strategies to enhance credibility and trustworthiness and
then provide a scaffold for learners to further their con- minimize potential bias throughout the process. We
ceptualization of systems thinking.17 Introducing these interviewed a limited number of faculty; however, to
concepts early in curricula will enable faculty to build on minimize this limitation, we selected a diverse group of
their learners’ prior knowledge by layering on challenges participants with a breadth of clinical and academic
across the four levels with increasing complexity. Early experiences as well as a familiarity with professional
authentic experiences, as O’Brien et al.37 suggested, will standards and competencies for the various health pro-
enable learners to apply their thinking skills to real-world fessions studied. Finally, this study took place in a single
problems and using cases and simulations requiring institution in a large metropolitan area; however, it does
interprofessional collaboration will further build linkages provide multiple perspectives and represents a breadth
and shared mental models across professions. of health professions education programs.
 10 M. M. PLACK ET AL.

Conclusions 2018-19_Functions-and-Structure_2017-08-02.docx.
Updated March 2017. Accessed October 21, 2017.
Participants articulated how they routinely used systems 7. Association of American Medical Colleges. AAMC core
thinking in the clinical setting but acknowledged the chal- entrustable professional activities for entering residency:
lenge of translating expert practice in general, and this Curriculum developers’ guide. https://members.aamc.org/
complex process more specifically, to teaching and assess- eweb/upload/core%20EPA%20Curriculum%20Dev%20
Guide.pdf. Updated 2014. Accessed May 11, 2017.
ment. Applying systems thinking across the health profes-
8. Commission on Accreditation in Physical Therapy Educa-
sions becomes more complex because it means different tion. CAPTE accreditation handbook: PT standards and
things to different professionals and involves different lev- required elements. http://www.capteonline.org/Accredita
els of the healthcare system, each of which is complex in tionHandbook. Updated 2016. Accessed May 11, 2017.
its own right. Effectively applying systems thinking across 9. Accreditation council on graduate medical education.
the health professions will require further training and Common program requirements. http://www.acgme.org/
Portals/0/PFAssets/ProgramRequirements/CPRs_2017-
practice. Capitalizing on existing tools, techniques, and 07-01.pdf Accessed October 21, 2017.
taxonomies from across the health professions as well as 10. American Academy of Physician Assistants. Competencies
outside healthcare may enhance teaching and assessment for the physician assistant profession. https://prodcmsstora
of systems thinking and SBP and ultimately clinical prac- gesa.blob.core.windows.net/uploads/files/PACompeten
Downloaded by [Australian National University] at 08:49 29 December 2017

tice, although these would need to be adapted and refined cies.pdf. Updated 2012. Accessed October 21, 2017.
11. Johnson JK, Miller SH, Horowitz SD. Systems-based prac-
for use in healthcare. Further training needs to occur at all
tice: Improving safety and quality of patient care by recog-
levels from student to clinician. nizing and improving the systems in which we work.
https://www.ahrq.gov/downloads/pub/advances2/vol2/
Advances-Johnson_90.pdf Updated 2008. Accessed Octo-
Funding ber 18, 2017.
The School of Medicine and Health Sciences Office of Faculty 12. Institute of Medicine. To err is human: Building a safer
Affairs and Professional Development at the George health system. Washington, DC: National Academies
Washington University partially funded this study. Press; 2000.
13. Institute of Medicine. Crossing the quality chasm: A new
health system for the 21st century. Washington, DC:
National Academies Press; 2001.
ORCID
14. Institute of Medicine. Health professions education: A
Margaret M. Plack http://orcid.org/0000-0001-9929-9708 bridge to quality. Washington, DC: National Academies
Ellen F. Goldman http://orcid.org/0000-0003-2560-4399 Press; 2003.
Andrea R. Scott http://orcid.org/0000-0002-4034-0259 15. Bertalanffy LV. General systems theory: Foundations,
Kathleen Kline http://orcid.org/0000-0003-3078-470X development, applications. New York: George Braziller;
Tracey Thompson http://orcid.org/0000-0002-1840-2345 1968.
Shelley B. Brundage http://orcid.org/0000-0002-7281-0948 16. Ackoff R. Redesigning the future: A systems approach to
societal problems. New York: John Wiley & Sons; 1974.
17. Ackoff R. The democratic corporation: A radical prescrip-
References tion for recreating corporate america and rediscovering suc-
cess. New York: Oxford University Press; 1994.
1. Swing S. The ACGME outcome project: Retrospective 18. Senge P. The fifth discipline. New York: Currency Double-
and prospective. Medical Teacher. 2007;29:648–654. day; 2006.
doi:10.1080/01421590701392903. PMID: 18236251. 19. Batalden P, Mohr J. Building knowledge of health care as a
2. American Association of Colleges of Nursing. The essen- system. Quality Management in Health Care. 1997;5:1–12.
tials of baccalaureate education for professional nursing doi:10.1097/00019514-199705030-00001. PMID: 10168367.
practice. Washington, DC: AACN; 2008. 20. Bowe C, Armstrong A. Assessment for systems learn-
3. American Association of Colleges of Nursing. The essen- ing: A holistic assessment framework to support deci-
tials of master’s education in nursing. Washington, DC: sion making across the medical education continuum.
AACN; 2011. Academic Medicine. 2017;92(5):585–592–592.
4. American Association of Colleges of Nursing. The essen- doi:10.1097/ACM.0000000000001321.
tials of doctoral education for advanced nursing practice. 21. Reid P, Compton W, Grossman J, Fanjiang G. Building a
Washington, DC: AACN; 2006. better delivery system: A new engineering/health care part-
5. American Speech-Language-Hearing Association. Guide- nership. Washington, DC: National Academies Press; 2005.
lines for the clinical doctorate in speech-language pathology. 22. Ferlie EB, Shortell, SM. Improving the quality of health care
http://www.asha.org/policy/GL2015-00341/. Updated in the United Kingdom and the United States: A framework
2015. Accessed May 11, 2017. for change. The Milbank Quarterly. 2001;79(2):281–315.
6. Liaison Committee on Medical Education. Functions and doi:10.1111/1468-0009.00206. PMID: 11439467.
structures of a medical school: Standards for accreditation 23. Phillips J, Stalter A, Dolansky M, Lopez G. Fostering
of medical education programs leading to the M.D. degree. future leadership in quality and safety in health care
http://lcme.org/wp-content/uploads/filebase/standards/ through systems thinking. Journal of Professional Nursing.
 TEACHING AND LEARNING IN MEDICINE 11

2016;32(1):15–24. doi:10.1016/j.profnurs.2015.06.003. healthcare. Presented at: 7th Annual Systems Conference;

PMID: 26802587. April 15–18, 2013; Orlando, FL.
24. Reineck C. Create a learning organization. Nursing Man- 41. Moore S, Dolansky M, Palmieri P, Singh M, Alemi F.
agement. 2002;33(10):42–3. doi:10.1097/00006247- Developing a measure of system thinking: A key compo-
200210000-00019. PMID: 12395066. nent in the advancement of the science of QI. Presented at
25. Colbert C, Ogden P, Ownby A, Bowe C. Systems-based International Health Forum on Quality and Safety in
practice in graduate medical education: Systems thinking Healthcare; April 23, 2010; Nice, France.
as the missing foundational construct. Teaching and 42. Moore S, Dolansky M, Singh M, Palmieri P, Alemi F. The
Learning in Medicine. 2011;23(2):179–185. doi:10.1080/ systems thinking scale, 2011. https://case.edu/nursing/
10401334.2011.561758. PMID: 21516607. media/nursing/pdf-dox/STS_Manual.pdf. Accessed May
26. Three Sigma I. A systems thinking primer. http://www.three 11, 2017.
sigma.com/systems_primer.htm. Accessed May 11, 2017. 43. Hopper M, Stave K. Assessing the effectiveness of systems
27. Richmond B. Systems thinking: Critical thinking skills for thinking interventions in the classroom. Presented at: 26th
the 1990s and beyond. System Dynamics Review. International Conference of the System Dynamics Society;
1993;9:113–33. doi:10.1002/sdr.4260090203. July 20–24, 2008; Athens, Greece.
28. Sweeney L, Sterman J. Bathtub dynamics: Initial results of 44. Stave K, Hopper M. What constitutes systems thinking? A
a systems thinking inventory. System Dynamics Review. proposed taxonomy. Presented at: 25th International Con-
2000;16:249–86. doi:10.1002/sdr.198. ference of the System Dynamics Society; July 29–August 2,
Downloaded by [Australian National University] at 08:49 29 December 2017

29. Nutley S, Davies H. Developing organizational learning in 2007; Boston, MA.

the NHS. Medical Education. 2001;35:35–42. doi:10.1046/ 45. Creswell J. Qualitative inquiry and research design: Choos-
j.1365-2923.2001.00834.x. PMID: 11123593. ing among five approaches 2nd ed. Thousand Oaks, CA:
30. Bierema L. Systems thinking: A new lens for old problems. Sage; 2007.
Journal of Continuing Education in the Health Professions. 46. Merriam S, and Associates. Qualitative research in prac-
2003;23(Suppl 1):S27–33. doi:10.1002/chp.1340230407. tice: Examples for discussion and analysis. San Francisco,
PMID: 14666831. CA: Jossey-Bass; 2002.
31. Trbovich P. Five ways to incorporate systems thinking into 47. Barbour R. Checklists for improving rigour in qualita-
healthcare organizations. Horizons. 2014;Fall:31–6. tive research: A case of the tail wagging the dog? Brit-
32. Sandhu G, Garcha I, Sleeth J, Yeates K, Walker G. AIDER: ish Medical Journal. 2001;322:1115–7. doi:10.1136/
A model for social accountability in medical education bmj.322.7294.1115. pmid: 11337448.
and practice. Medical Teacher. 2013;35:e1403–8. 48. Kuzel A. Sampling in qualitative inquiry. In: Crabtree B,
doi:10.3109/0142159X.2013.770134. PMID: 23444886. Miller W, eds. Doing qualitative research. London: Sage;
33. Pfeiffer J, Beschta J, Hohl S, Gloyd S, Hagopian A, Wasser- 1992:31–44.
heit J. Competency-based curricula to transform global 49. Miles M, Huberman A. Qualitative data analysis: An
health: Redesign with the end in mind. Academic Medi- expanded sourcebook. 2nd ed. Newbury Park, CA: Sage;
cine. 2013;88:131–6. doi:10.1097/ACM.0b013e318276bdf4. 1994.
PMID: 23165274. 50. Denzin N, Lincoln Y, eds. Handbook of qualitative
34. Stalter A, Phillips J, Dolansky M. QSEN institute RN-BSN research. Thousand Oaks: Sage; 1994.
task force: White paper on recommendation for systems- 51. Guba E, Lincoln Y. Fourth generation evaluation. New-
based practice competency. Journal Nursing Care Quality. bury Park, CA: Sage; 1989.
2017;32:(4):354–8. doi:10.1097/NCQ.0000000000000262. 52. Atkinson HL, Nixon-Cave K. A tool for clinical reasoning
35. Phillips J, Stalter A. Integrating systems thinking into and reflection using the International Classification of
nursing education. The Journal of Continuing Education Functioning, Disability and Health (ICF) framework and
in Nursing. 2016;47(9):395–397. doi:10.3928/00220124- patient management model. Phys Ther. 2011;91(3):416–
20160817-05. PMID: 27580505. 430. doi:10.2522/ptj.20090226. pmid: 21273627.
36. Norman C. Teaching systems thinking and complexity 53. Puntambekar S, H€ ubscher R. Tools for scaffolding stu-
theory in health sciences. Journal of Evaluation in Clinical dents in a complex learning environment: What have we
Practice. 2013;19:1087–9. doi:10.1111/jep.12065. pmid: gained and what have we missed? Educational Psycholo-
23829410. gist. 2005;40(1):1–12. doi:10.1207/s15326985ep4001_1.
37. O’Brien B, Bachhuber M, Teherani A, Iker T, Batt J, O’Sul- 54. Larkin M. Using scaffolded instruction to optimize learning.
livan P. Systems-oriented workplace learning experiences Arlington, VA: ERIC Clearinghouse on Disabilities and
for early learners: Three models. Academic Medicine. Gifted Education; 2002.
2017;92(5):684–693. doi:10.1097/ 55. Northern Illinois University, Faculty Development and
ACM.0000000000001243. pmid: 27254010. Instructional Design Center. Instructional scaffolding to
38. Dawidowicz P. The person on the street’s understanding improve learning. http://www.niu.edu/facdev/resources/
of systems thinking. Systems Research and Behavioral Sci- guide/strategies/instructional_scaffolding_to_improve_
ence. 2012;29(1):2–13. doi:10.1002/sres.1094. learning.pdf. Accessed November 2, 2016.
39. Kay J, Foster J. About teaching systems thinking. Pre- 56. World Health Organization. International Classification of
sented at: HKK conference; june 14–15, 1999; University Functioning, Disability and Health: ICF. http://www.who.
of Waterloo, Ontario int/classifications/icf/en/. Accessed August 29, 2017.
40. Whitehead N, Scherer W. Moving from systematic treat- 57. Polit D, Beck C. Generalization in quantitative and quali-
ment to a systemic approach: A path for sustainable U.S. tative research: Myths and strategies. International Journal
 12 M. M. PLACK ET AL.

of Nursing Studies. 2010;47:1451–1458. doi:10.1016/j.  If it is a thread, probe on where and how it starts,
ijnurstu.2010.06.004. pmid: 20598692. how it is integrated throughout, and how it is
58. O’Brien B, Harris I, Beckman T, Reed D, Cook D. Stand- assessed.
ards for reporting qualitative research: A synthesis of rec-
ommendations. Academic Medicine. 2014;89(9):1245–
 What are some strategies you use to teach this?
1251. doi:10.1097/ACM.0000000000000388. pmid:  Has any of this changed over time? What was the
24979285. impetus for change?
 What, if any, changes would you like to see?
4. What does “systems thinking” mean to your
profession?
Appendix Semistructured interview guide  How is it discussed?
 What terminology is being used? Is there stan-
Systems Thinking and Systems-Based Practice
dard language used to describe it and/or assess
Across the Health Professions: An Inquiry into
it?
Teaching Practices & Assessment Interview
 Are there particular models being discussed or
Guide
advocated for in your profession? (e.g., ICF)
Downloaded by [Australian National University] at 08:49 29 December 2017

IRB #:041608  What are some of the drivers of the current

Participant ID:__________________ discussion?
Participant’s Level/Administrative Role: __________  If it is not being discussed, why do you think
The purpose of this interview is to explore how your there is no discussion?
students learn to appreciate and integrate into practice 5. What, if anything, should GW do to enhance the
the systems within which each patient exists (i.e., systems teaching of systems thinking and systems-based
thinking and systems-based practice). There are no right practice across the health professions?
or wrong answers; we are interested in gaining a full 6. Based on our review of the literature, the require-
understanding of what is taught, how and why, and how ments on this page (“Requirements”) are what
it is assessed, as well as any concerns you have and areas have been suggested as concepts that should be
you would like to see improved. included in health care curricular on systems based
1. Please start by sharing your teaching background: practice. What are your thoughts on these
 What courses/topics do you usually teach? requirements?
 Have these changed at all over time? Probes:
 Is there anything else about your background we  Does your curriculum include these? If not, have
should know as it pertains to the purpose of this you considered including it and not done so—
study? why?
2. When we refer to “systems thinking,” what does  Do you think it is important to include—why?
that mean to you as a practitioner? What, if any issues do you anticipate in includ-
 Can you give me some examples from your ing these topics?
practice? 7. The following has been suggested as a taxonomy of
 Can you tell me how systems thinking is assessed systems thinking levels and assessments (Systems
in practice? Thinking Levels and Assessment) (show Hopper
 How did you learn to think from a systems and Stave Taxonomy). Please provide your
perspective? thoughts on how useful this might be to your
3. When we refer to “systems thinking,” what does curriculum.
that mean in your current curriculum? 8. Please add any other comments you would like.

 Where is this taught within your curriculum? Requirements
 Does your curriculum include specific objectives 1. Define the system:
related to systems thinking or systems-based  Small micro-office, clinic
practice? Can you give me some examples? (if  Large micro-ambulatory, hospital, ltc, home care
examples are shared, ask for materials)  Small macro-local/regional community, ACO,
 How is it assessed? PH
 Can you give me some examples of how this is  Large macro-health care policy, Medicare, US
taught in your curriculum? health care system, socioeconomic environment
 If it is a course, probe on how it is taught and 2. Analyze and diagnose system:
assessed, and ask for a syllabus and any teaching  Tools to describe the characteristics and func-
materials they are willing to share. tioning of the system:
 TEACHING AND LEARNING IN MEDICINE 13

 Observation, process flowcharts, stages, concept  Short-term, long-term unintended consequences

maps, etc.  Validation
 Tools to analyze the system:  Planning change

 Interdependencies, root cause analysis, struc- Requirements were a synthesis of concepts we identi-
tures, simulations, feedback loops fied in our literature review

 Tools to uncover thinking: Hopper M, Stave K. Assessing the effectiveness of
Perspective taking, surfacing assumptions. Collabo- systems thinking interventions in the classroom. Pre-
rative inquire sented at: 26th International Conference of the Sys-
3. Identify and test improvements: tem Dynamics Society; July 20–24, 2008; Athens,
 Leverage points and actions Greece.
Downloaded by [Australian National University] at 08:49 29 December 2017