Perspective

Building a Framework of Entrustable Professional

Activities, Supported by Competencies and
Milestones, to Bridge the Educational Continuum
Carol Carraccio, MD, MA, Robert Englander, MD, MPH, Joseph Gilhooly, MD,
Richard Mink, MD, MACM, Dena Hofkosh, MD, MEd, Michael A. Barone, MD,
and Eric S. Holmboe, MD

Abstract
The transition to competency-based that define a developmental trajectory the process for identifying EPAs for the
medical education (CBME) and for individual competencies. EPAs specialty and subspecialties of pediatrics.
adoption of the foundational domains are observable and measureable The method of integrating EPAs with
of competence by the Accreditation units of work that can be mapped to competencies and milestones through
Council for Graduate Medical Education, competencies and milestones critical to a mapping process is discussed, and
Association of American Medical Colleges performing them safely and effectively. an example is provided. The authors
(AAMC), and American Board of Medical illustrate the alignment of the AAMC’s
Specialties’ certification and maintenance The pediatrics community integrated Core EPAs for Entering Residency with
of certification (MOC) programs provided the two frameworks to create a the general pediatrics EPAs and, in turn,
an unprecedented opportunity for potential pathway of learning and the alignment of the latter with the
the pediatrics community to create assessment across the continuum from subspecialty EPAs, thus helping build
a model of learning and assessment undergraduate medical education the bridge between UME and GME.
across the continuum. Two frameworks (UME) to graduate medical education The authors propose how assessment
for assessment in CBME have been (GME) and from GME to practice. The in GME, based on EPAs and milestones,
promoted: (1) entrustable professional authors briefly describe the evolution can guide MOC to complete the bridge
activities (EPAs) and (2) milestones of the Pediatrics Milestone Project and across the education continuum.

The adoption of the six core and the building blocks, respectively, and feedback are essential for further
competencies of the Accreditation to realize the implementation of this professional development.
Council for Graduate Medical continuum.
Education (ACGME) and the American Parallel to and complementing the work of
Board of Medical Specialties (ABMS) EPAs, first introduced by ten Cate6 in EPAs, the ACGME and member boards of
as the foundation for outcomes-based the Netherlands in 2005, are receiving the ABMS partnered to initiate the process
undergraduate medical education increasing attention internationally as a that developed the milestones, with each
(UME) by the Association of American framework for meaningful assessment specialty creating a shared mental model
Medical Colleges (AAMC), for graduate of physician competence at both the of performance levels or milestones for
medical education (GME) by the UME and GME levels.7–11 As measureable competencies within the specialty.5
ACGME, and for board certification units of observable work, EPAs describe
as well as maintenance of certification important routine activities of a given The pediatrics community has integrated
(MOC) by the ABMS has provided an specialist or subspecialist that require the two frameworks of EPAs and
unprecedented opportunity to create integration of competencies for safe and milestones.14 This integration provides a
a seamless continuum of learning and effective performance. In the aggregate, pathway that leads to assessment across
assessment in medicine.1–3 Parallel they define the specialty or subspecialty.4 the educational continuum from UME
advancements in assessment through The qualifier of “entrustable” aligns with to GME and from GME to practice. We
entrustable professional activities the ultimate ability of an individual to wrote this article to (1) briefly describe
(EPAs)4 and educational milestones, the perform the professional activity safely the evolution of the Pediatrics Milestone
latter effort initiated by the ACGME and effectively without supervision and Project and the identification of EPAs
and the ABMS,5 provide the foundation has resulted in the proposal of rating for the specialty and subspecialties of
scales with levels of supervision leading to pediatrics, (2) discuss the value added to
readiness for unsupervised practice.8,12,13 assessment by integrating the EPAs
Please see the end of this article for information
about the authors. Specialties have developed EPAs for with competencies and milestones,
residents, and the AAMC has developed (3) illustrate the bridge between UME
Correspondence should be addressed to Carol
Carraccio, American Board of Pediatrics, 111 Silver Core EPAs for Entering Residency.7,9–11 and GME built upon alignment of
Cedar Ct., Chapel Hill, NC 27514; telephone: (919) The Core EPAs for Entering Residency Core EPAs for Entering Residency with
929-0461; e-mail: [email protected]. set the expectation that trainees should pediatrics EPAs, and (4) propose how
be able to perform 13 EPAs on Day 1 of assessment in GME based on EPAs and
Acad Med. XXXX;XX:00–00.
First published online residency without direct supervision, but milestones can guide MOC to complete
doi: 10.1097/ACM.0000000000001141 experts agree that ongoing observation the bridge across the educational

Academic Medicine, Vol. XX, No. X / XX XXXX 1

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
 Perspective

continuum, not only for pediatrics but the potential list of EPAs for the general The combination of the top-down and
also for other specialties. pediatrician. Through an online survey, bottom-up approach to evaluating
we vetted the initial list of EPAs with EPA performance is ultimately the
most powerful. EPAs provide the
members of the Association of Pediatric
The Unfolding of the Pediatrics clinical context for the assessment of
Program Directors. In response to their competencies, which uses a “panoramic”
Milestone Project feedback, we revised the list and then lens for assessing learners who must
In 2009, the ACGME and the American sent a second survey asking for additional integrate competencies to deliver care,
Board of Pediatrics (ABP) invited the refinements. We also communicated with and their milestones, which provide a
pediatrics community to initiate the other members of our community asking “zoom” lens for assessing the learner at a
development of the milestones for for their input. The final first iterations of more granular level.17
pediatrics. Creation of the guiding the 17 EPAs defining the desired outcomes
principles, process, and outcome of this for the pediatrics residency-to-practice Pangaro and ten Cate18 suggest
three-year endeavor has previously been transition appear in the middle columns of conceptual frameworks for assessment
described.15 The full pediatrics milestone Charts 1 and 2, with 6 in Chart 1 and the that can be categorized as analytic,
document is published as a supplement remaining 11 in Chart 2. synthetic, or developmental. The ACGME
to Academic Pediatrics.16 competencies would be considered an
As conceived by the pediatrics analytic framework, breaking down
In brief, the pediatrics milestones are community, the EPA and competency/ core domains into further detailed
narrative descriptions of behaviors milestone frameworks should be descriptions of competencies. Milestones
across the developmental continuum integrated, not only in pediatrics but for are an example of a developmental
beginning with a novice learner (early other specialties as well: framework, representing a logical
medical student) and progressing along
the continuum to advanced beginner,
then to competent learner, proficient
learner, and, finally, expert learner Chart 1
Examples of EPAs That Directly Align Across the Continuum of Pediatrics Education
(after years of deliberate practice). The
and Traininga
competencies (e.g., “gather essential and
accurate information about the patient”) The AAMC’s Core EPAs Pediatric residency EPAs
and thus their narratives are context for Entering Residency7 for entering practiceb Pediatric fellowship EPAs
independent and therefore do not define Give or receive a patient Facilitate handovers to Facilitate handovers to another
the complexity of the patient encounter handover to transition care another health care provider, health care provider, either within
or the clinical environment in which the responsibility. either within or across or across settings.
settings.
skills will be demonstrated. Reporting
of individual performance levels or Recognize a patient requiring Resuscitate, initiate Diagnose, initially manage, and
urgent or emergent care stabilization of the patient, refer children with advanced
milestones for 21 of the 48 competencies, and initiate evaluation and and then triage to align care or end-stage heart failure and/
within the six competency domains, on a management. with severity of illness. or pulmonary hypertension to
semiannual basis began in 2014 as part of experts for medical therapy,
the data collection for the ACGME’s Next ECMO, ventricular assist device,
and/or cardiac transplantation.c
Accreditation System.5*
Collaborate as a member of Lead an interprofessional Lead an interprofessional health
an interprofessional team. health care team. care team.
Identifying General Pediatrics Lead within the profession.
EPAs Obtain informed consent for Demonstrate competence Demonstrate competence
As the first iteration of the milestones was tests and/or procedures. in performing the common in performing the common
Perform general procedures procedures of the general procedures of the pediatric
being developed, a body of literature on pediatrician. pulmonary subspecialist.c
of a physician.
EPAs was emerging. As early as 2005,6 ten
Cate began to write about the construct Identify system failures and Apply public health principles Apply public health principles and
contribute to a culture of and quality improvement quality improvement methods
of EPAs with a seminal article4 published safety and improvement. methods to improve care to improve care and safety for
in 2007 that spoke to the implementation and safety for populations, populations, communities, and
of their use for assessment in obstetrics– communities, and systems. systems.
gynecology in the Netherlands. By focusing Recommend and interpret Provide recommended Provide preventive health care
on care delivery, which requires the common diagnostic and pediatric health screening. that includes conditions specific
integration of multiple competencies, EPAs screening tests. to the adolescent and young
adult population.c
align what clinicians assess with what they
do in the authentic environment, adding Abbreviations: EPA indicates an entrustable professional activity; AAMC, Association of American Medical
meaning and value to assessment. Some Colleges; ECMO, extracorporeal membrane oxygenation.
a
The alignments shown in this chart and Chart 2 help build the bridge between undergraduate and graduate
of us (C.C., R.E., J.G.) began to identify medical education. The authors propose how assessment in GME, based on EPAs and milestones, can guide
both certification and maintenance of certification programs to complete the bridge across the medical
*In this article, we are using the proposed education continuum.
b
international terms1 domain of competence (which This chart shows 6 of the general pediatrics EPAs in the middle column. Chart 2 shows the other 11 in its
in the United States is typically either competency middle column. See https://www.abp.org/entrustable-professional-activities-epas for more about the pediatrics
or core competency) and competency (which in the EPAs.
c
United States is typically subcompetency). Subspecialty-specific EPA. Common subspecialty EPAs are shown in italics.

2 Academic Medicine, Vol. XX, No. X / XX XXXX

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
 Perspective

Chart 2
Examples of EPAs That Are Sequenced Across the Continuum of Pediatrics Education,
With Subsequent EPAs Building on Preceding EPAsa

The AAMC’s Core EPAs Pediatric residency EPAs for entering

for Entering Residency7 practiceb Pediatric fellowship EPAs
Gather a history and Provide a medical home for well children of all ages. Manage healthy patients with pediatric infectious diseases.c
perform a physical Provide a medical home for patients with complex, Provide a medical home for patients with hematologic,
examination. chronic, or special health care needs. oncologic, or stem cell transplant needs.c
Prioritize a differential Recognize, provide initial management, and refer Provide care to patients in the neonatal intensive care unit
diagnosis following a patients presenting with surgical problems. with surgical problems in collaboration with pediatric and
clinical encounter. subspecialty surgeons.c
Enter and discuss orders Assess and manage patients with common Recognize and longitudinally manage behavioral variations,
and prescriptions. behavior/mental health problems. problems, and disorders in typically developing children and
Document a clinical children with developmental disabilities.c
encounter in the patient Provide consultation to other health care providers Provide for and obtain consultation from other health care
record. caring for children. providers caring for children.
Provide an oral presentation
Refer patients who require consultation.
of a clinical encounter.
Manage patients with acute, common diagnoses in Manage patients with acute complex respiratory disease in an
an ambulatory, emergency, or inpatient setting. ambulatory, emergency, or inpatient setting.c
Care for the well newborn. Manage neonates with acute, common single-system diseases in
an inpatient setting.c
Facilitate the transition from pediatric to adult care. Facilitate the transition of patients with endocrine disorders
from pediatric to adult health care.c
Form clinical questions and Manage information from a variety of sources for Engage in scholarly activities through the discovery, application,
retrieve evidence to advance both learning and application to patient care. and dissemination of new knowledge.
patient care.
Contribute to the fiscally sound and ethical Contribute to the fiscally sound and ethical management of a
management of a practice (e.g., coding billing, practice (e.g., coding billing, scheduling, and record keeping).
scheduling, and record keeping).
Abbreviations: EPA indicates an entrustable professional activity; AAMC, Association of American Medical
Colleges.
a
The alignments shown in this chart and Chart 1 help build the bridge between undergraduate and graduate
medical education. The authors propose how assessment in GME, based on EPAs and milestones, can guide
both certification and maintenance of certification programs to complete the bridge across the medical
education continuum.
b
This chart shows 11 of the general pediatrics EPAs in the middle column. Chart 1 shows the other 6 in its middle
column. See https://www.abp.org/entrustable-professional-activities-epas for more about the pediatrics EPAs.
c
Subspecialty-specific EPA. Common subspecialty EPAs are shown in italics.

progression of steps over time. Finally, for a given level of performance (novice mental model of performance could then
EPAs represent a synthetic framework to expert) across the competencies in be applied to learner assessment, with
that combines multiple domains of the map. For example, in Chart 3, the faculty choosing the vignette that most
competence. The framework we propose behaviors of the novice are described in closely aligns with the behaviors of the
embraces the above models in that it is the second column, advanced beginner trainees that they have directly observed
a framework of EPAs integrated with in the third, competent learner in the over some period of time to assess
competencies and milestones. fourth, proficient learner in the fifth, and performance level.
expert in the sixth column.
Once identified, EPAs are judiciously Learners in the workplace will likely not
mapped to those competencies and Our model suggests that reading down demonstrate all the described behaviors
milestones critical to making an any column provides the requisite of one column or performance level but,
entrustment decision for that specific behaviors of a learner at a specific rather, some of the behaviors from two
EPA.19 Through this mapping process level of performance for all of the or possibly three performance levels. The
we create, for pediatrics, a matrix of the critical competencies required for an shared mental models painted by the
critical competencies for a given EPA entrustment decision on that EPA. For behaviors in each column are meant to be
and their respective milestones (see example, integrating the behaviors anchors that frame a discussion between
Chart 3). The first column of the matrix in the third column of Chart 3 into a the learner and a mentor or the learner
(on the left) lists the competencies that clinical vignette paints a picture of an and a clinical competency committee
are critical for making an entrustment advanced beginner performing the EPA. around which behaviors have been
decision for the given EPA. Each row This same process would apply to each demonstrated and which behaviors are
represents the progression of milestones performance level, the end result being a desired. The information gleaned about
for the given competency. Columns two picture of a learner at each performance a learner’s performance level using this
through six represent all of the milestones level carrying out the EPA.14 This shared framework will be combined with many

Academic Medicine, Vol. XX, No. X / XX XXXX 3

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
 Perspective

Chart 3
The “Handover EPA”a Mapped to Competencies and Their Milestones That Are
Critical to Making an Entrustment Decision, With Examples of Behaviors Associated
With the Milestone for One of the Performance Levelsb

Competencies critical for

making an entrustment Milestone Milestone Milestone Milestone Milestone
decision for this EPA level 1 level 2 level 3 level 4 level 5
Patient care. Organize and Organizes the simultaneous care of a few patients
prioritize responsibilities to with efficiency; occasionally prioritizes patient care
provide care that is safe, responsibilities to proactively anticipate future needs; each
effective, and efficient. additional patient or interruption in work leads to notable
decreases in efficiency and ability to effectively prioritize;
permanent breaks in task with interruptions are less
common, but prolonged breaks in task are still common.
Patient care. Provide transfer Uses a standard template for the information provided
of care that ensures seamless during the handoff. Unable to deviate from that template
transitions. to adapt to more complex situations. May have errors
of omission or commission, particularly when clinical
information is not synthesized. Neither anticipates nor
attends to the needs of the receiver of information.
Interpersonal and Begins to understand the purpose of the communication
communication skills. and at times adjusts length to context, as appropriate.
Communicate with physicians However, will often still err on the side of inclusion of
and other health professionals. excess details.
Interpersonal and Documentation often contains all appropriate data
communication skills. sections, although some information may be missing from
Maintain comprehensive, some sections or presented in a sequence that confuses
timely, and legible medical the reader (evolution of symptoms is not documented
records. chronologically). Documentation may be overly lengthy
and detailed. It may contain erroneous information
carried forward from review of the past medical record.
However, the practitioner at this stage begins to go
beyond documentation of specific encounters and may
update the patient-specific databases (e.g., problem
list and diabetes care flowsheet) where applicable.
Documentation is often in the medical record in a timely
manner, but may need subsequent amendment to be
considered complete. Handwritten documentation is
usually legible, timed, dated, and signed.
Practice-based learning and Dependent on external sources of feedback for
improvement. improvement; beginning to acknowledge other points of
Incorporate formative view, but reinterprets feedback in a way that serves his
evaluation feedback into daily or her own need for praise or consequence-avoidance
practice. rather than informing a personal quest for improvement;
little to no behavioral change occurs in response to
feedback (e.g., listens to feedback but takes away only
those messages he or she wants to hear).
Practice-based learning Demonstrates a willingness to try new technology for
and improvement. Use patient care assignments or learning. Able to identify
information technology to and use several available databases, search engines, or
optimize learning and care other appropriate tools, resulting in a manageable volume
delivery. of information, most of which is relevant to the clinical
question. Basic use of an EHR is improving, as evidenced
by greater efficacy and efficiency in performing needed
tasks. Beginning to identify shortcuts to getting to the
right information quickly, such as use of filters. Also
beginning to avoid shortcuts that lead one astray of the
correct information or perpetuate incorrect information in
the electronic health record.
a
A general pediatrics and common subspecialty EPA: “Facilitate handovers to another healthcare provider either
within or across settings.”
b
This chart illustrates the mapping of the pediatrics “handover EPAs” to the competencies (shown in the left-
hand column) and their associated milestones for five performance levels (shown in columns 2 through 6).
The chart includes descriptions of behaviors for the milestone for level 2 to illustrate how the milestones for
the five performance levels are associated with their competencies. Levels 1 to 5 represent progression along a
developmental trajectory from novice to advanced beginner, competent learner, proficient learner, and expert
learner. The authors caution that labels of the levels (e.g., “advanced beginner”) be avoided, to keep the learner
and the assessor focused on behaviors rather than labels. The authors propose how assessment in GME, based
on EPAs and milestones, can guide both certification and maintenance of certification programs to complete the
bridge across the medical education continuum.

4 Academic Medicine, Vol. XX, No. X / XX XXXX

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
 Perspective

other data points that experts will use in and illustrating their relationships to to build the bridge that connects EPAs
making a balanced judgment regarding each other. The subspecialists then for medical students with general
entrustment. reviewed the general pediatrics EPAs, pediatrics EPAs for all residents, which
adopting five that spanned the generalist in turn connects to the pediatrics EPAs
Regehr et al20 provide some support for to subspecialist role and that applied to for fellowships. Review of the lists of
this assessment strategy. Creating video all subspecialties. Through a consensus EPAs illustrates the natural linkages that
vignettes of each of the five performance process, the group identified two represent either a direct alignment (e.g.,
levels provides an additional tool for additional common subspecialty EPAs as a student, collaborating as a member
faculty development in the assessment of (see Charts 2 and 3), one on leading of an interprofessional team; as a resident
EPAs and milestones.21 Using EPAs, with within the profession and a second on or fellow, leading an interprofessional
their dimension of entrustment, allows scholarly activity. With regard to the latter health care team—see Chart 1) or
a formal decision to trust a learner to EPA, the ABP requires the completion of where student EPAs serve as building
safely perform the professional activity scholarly work during fellowship in order blocks for residency EPAs, the latter
without direct supervision at the UME-to- for the learner to sit for the subspecialty serving as building blocks for fellowship
GME transition and without supervision certification examination, which provides EPAs (e.g., as a student, entering and
at the GME-to-practice transition, thus the rationale for the identification of an discussing orders and prescriptions; as
bringing the required level of supervision EPA on scholarly activity. The second a resident or fellow, managing patients
as another data point into the assessment day of the workshop concluded with a with common acute illnesses in a variety
equation.13 Being mindful that context discussion of the approach to identifying of settings—see Chart 2). This bridge
matters, verification of a learner’s subspecialty-specific EPAs to maximize across the continuum of learning and
readiness to practice at a given level of consistency across subspecialties. assessment is reminiscent of what Harden
supervision through ongoing observation and Stamper24 highlighted as the value
and feedback is warranted at transition The participants left with two challenges: of the “spiral curriculum,” where the
points until readiness for unsupervised (1) to lead the effort to identify the “competence of students increases with
practice is truly achieved. Even then, subspecialty-specific EPAs for their each visit until the final overall objectives
significant changes in context may require community by developing a title, short are achieved.”
self-assessment, peer observation, and description, and functions (list of tasks
coaching until readiness for unsupervised that one needs to do to perform the
practice is regained. EPA) as well as to judiciously map each The Missing Link: Guiding MOC
EPA to its critical competencies23; and With GME EPAs
As the development of the general (2) with the help of CoPS, to obtain On the basis of performance in residency
pediatrics milestones was being and incorporate feedback from their and fellowship, one can envision specific
completed, the ACGME required that communities into the work product. EPAs and competencies forming the
each of the subspecialties within a given Each subspecialty community identified foundation for improvement during
specialty develop subspecialty-specific between three and six subspecialty- the first cycle of MOC and beyond. As a
milestones. The pediatrics milestones specific EPAs to add to the seven prerequisite, our community must decide
were unique in that they spanned the common subspecialty EPAs. Leadership the performance level that equates with
continuum from early medical students from the ABP worked with each entrustment for each EPA, for residents
to practicing pediatricians, thus including subspecialty community to ensure a as well as fellows, and whether a level
performance levels applicable for fellows. consistent approach. of supervision short of readiness for
Embedding the competencies in clinical unsupervised practice for any of the EPAs
contexts or EPAs relevant to each of the will be acceptable for transition from
The Bridge Across the
14 pediatrics subspecialties with ABP residency or fellowship into practice.
UME-to-GME Continuum
certification was the next and best step
for our community. The ACGME agreed Shortly after work began on the The ABP just initiated research with our
that this plan represented a reasoned pediatrics EPAs, the AAMC constituted community to address these questions
approach, and the ABP took the lead in a multidisciplinary drafting panel to for pediatrics learners. For example, in
this effort. identify the Core EPAs for Entering a case where a learner is entrusted with
Residency. A panel comprising 16 of the 17 EPAs, with the exception
approximately 100 thought leaders across being the EPA to “manage information
Identifying EPAs for the the spectrum of medical education from a variety of sources for both
Pediatrics Subspecialties provided feedback to the drafting panel, learning and application to patient care,”
In March 2013, the ABP, with the help of resulting in an iterative process of would we accept a level of performance
the Council of Pediatric Subspecialties creation, reaction, and revision. The final that equates with a requisite period of
(CoPS), convened a workshop for document details 13 EPAs that medical indirect supervision or guidance in a
education thought leaders from each of students should be entrusted to perform new practice by a peer/colleague who is
the ABP-certified pediatrics subspecialties without direct supervision on Day 1 of more experienced in this professional
to identify EPAs common to the residency training.7 activity? Or consider this scenario: A
subspecialties.22 The two-day workshop resident is entrusted to perform all 17
began with faculty development around With the creation of the Core EPAs EPAs at the completion of training and
defining and distinguishing pediatrics for Entering Residency, the pediatrics enters a general pediatrics practice that
EPAs, competencies, and milestones community now had the infrastructure cares for a large number of adolescent

Academic Medicine, Vol. XX, No. X / XX XXXX 5

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
 Perspective

patients, many of whom are struggling and GME with the desired outcomes Third, the scaffolding that supports the
with either an anxiety disorder or for practicing physicians. For example, bridges we are building is constructed of
depression. While the new practicing in the UME setting it is essential to sound educational theory.20,31–33
physician had demonstrated the requisite lay the foundation for handovers by
competencies to be entrusted during introducing a template, which includes We did not create our list of EPAs with
residency to perform without supervision situation awareness, illness severity, broad input from practitioners or input
the EPA “Assess and manage patients action planning, and contingency from patients; however, practicing general
with common behavior/mental health planning as well as the communication pediatricians and pediatric subspecialists
problems,” the more limited patient skills to effectively engage as either a created the original lists. These lists
volume and experience during training sender or a receiver of the handover were subsequently vetted among our
did not allow for the development of (e.g., skill at readbacks). These various communities of practice, which
proficiency. The new practicing physician foundational skills as well as content include general pediatricians and the 14
realizes that patients will benefit if knowledge must be advanced during subspecialties for which ABP certification
he or she continues to move along GME, building on and reinforcing is offered, and we revised the lists on the
the developmental trajectory toward what came before to provide deeper basis of their feedback. However, we need
proficiency and expertise. Seen in this learning and ultimately readiness to “test” the lists, which we are currently
light, EPAs may focus reflection on for entrustment. In addition, faculty doing, with studies at both the residency
practice that helps individuals identify development for assessment can focus and fellowship levels in pediatrics. Of
gaps and seek learning activities to fill on experience with a given set of tools note, the AAMC is engaging a number
those gaps. used across the continuum, developing of medical schools in piloting the Core
expertise among faculty raters. EPAs for Entering Residency.34 Testing
Of note, entrustment for unsupervised in practice will come by having trainees
practice during training does not carry EPAs from GME into practice for
Next Steps
equate with expertise. In both of these the first cycle of MOC as a way of moving
examples, the opportunity to address Although the road ahead is long, for the them further along the trajectory from
identified gaps during the transition first time it is continuous. As the work competent/proficient to expert. Plans for
from residency to practice presents of the pediatrics community indicates, testing in practice are also on the horizon.
an opportunity to design a cycle of bridges are available to connect us at the
MOC that enhances the professional transition points from UME to GME and Dr. David Leach, a former executive
development of the pediatrician—or from GME to fellowship and practice for director of the ACGME, a visionary who
any other physician—toward expertise. any specialty that has developed EPAs. As saw the road ahead before we could even
Investment in improvement through our response to those who will invariably begin to imagine it, said of assessment
continuous professional development question whether the road ahead will that it is
and MOC should be used to facilitate a lead to a physician workforce that
process of structured learning throughout delivers better quality of care and who are dependent on an integrated version of
a physician’s career. When gaps are reluctant to follow a path without proof, the competencies, whereas measurement
identified during training, learning we offer three thoughts: relies on a speciated version of the
competencies. The paradox cannot
activities can be initiated as part of an be resolved easily. The more the
individualized curriculum and continued First, although we will not have proof competencies are specified, the less
into MOC. In addition, there may be new that our proposed road leads to relevant to the whole they become.35
EPAs identified as a physician’s career desired patient care outcomes until
evolves or changes in scope. According to implementation and study, we have little The integration of the EPAs, compe-
ten Cate et al,25 “entrustment decisions evidence supporting the current model tencies, and milestones across the
should … have an expiration date if no or despite its entrenchment, and we have educational continuum provides the
too little practice has occurred.” Further clear evidence that this discontinuous road map to resolving this paradox.
work in this area holds promise of path leads us away from, rather than
ensuring that MOC remains meaningful closer to, desired outcomes.27,28 Acknowledgments: The authors wish to thank the
throughout a physician’s career. I-PASS Entrustable Professional Activity (EPA)
Working Group, I-PASS Institute, for review and
Second, program directors’ expectations
input on the handover EPA.
of what new residents should be able to
The Critical Contributions of the do are not aligned with residents’ actual Funding/Support: The authors received partial
Continuum to Competency-Based skills.29 First-year residents also report support from the American Board of Pediatrics
Education performing some professional activities Foundation.
One of the current concerns about with less supervision than clinical
Other disclosures: E.S. Holmboe receives
trainees is that they may not be well supervisors say they expect, particularly royalties from Mosby-Elsevier for a textbook on
prepared for real-world practice. The at night.30 Misalignment between actual assessment.
EPAs for residents and fellows are based skills and expected skills or between an
on what one expects in practice, and expected level of supervision and an Ethical approval: Reported as not applicable.
thus a backward-visioning process is actual level of supervision puts a patient’s Previous presentations: Some content was
used to set expectations and outcomes safety at risk and puts the learner in presented at the American Board of Pediatrics
for GME.26 Thus, the continuum enables a position of making a potentially Future of Testing Invitational Conference,
the alignment of the work of UME preventable medical error. Durham, North Carolina, May 15 and 16, 2015.

6 Academic Medicine, Vol. XX, No. X / XX XXXX

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.
 Perspective

C. Carraccio is vice president of competency-based mededportal.org/icollaborative/resource/887. performance and entrustment decisions

assessment, American Board of Pediatrics, Chapel Accessed January 15, 2016. [published online August 11, 2015]. Acad
Hill, North Carolina. 8 Chen HC, van den Broek WE, ten Cate O. Med. doi:10.1097/ACM.0000000000000853.
The case for use of entrustable professional 22 Council of Pediatric Subspecialties.
R. Englander was senior director of competency- activities in undergraduate medical Entrustable professional activities. http://
based learning and assessment, Association of education. Acad Med. 2015;90:431–436. pedsubs.org/issues/EPAs.cfm. Accessed
American Medical Colleges, Washington, DC, at the 9 Caverzagie KJ, Cooney TG, Hemmer PA, January 15, 2016.
time this article was written.
Berkowitz L. The development of entrustable 23 Ten Cate O. AM last page: What entrustable
J. Gilhooly is adjunct professor of pediatrics, Oregon professional activities for internal medicine professional activities add to a competency-
Health & Science University, Portland, Oregon. residency training: A report from the based curriculum. Acad Med. 2014;89:691.
Education Redesign Committee of the 24 Harden RM, Stamper N. What is a spiral
R. Mink is chief, Division of Pediatric Critical Alliance for Academic Internal Medicine. curriculum? Med Teach. 1999;21:141–143.
Care Medicine, and director, pediatric critical care Acad Med. 2015;90:479–484.
fellowship, Harbor–UCLA Medical Center, Torrance,
25 Ten Cate O, Chen HC, Hoff RG, Peters
10 Hauer KE, Kohlwes J, Cornett P, et al. H, Bok H, van der Schaaf M. Curriculum
California, and professor of pediatrics, David Geffen
Identifying entrustable professional activities development for the workplace using
School of Medicine at UCLA, Los Angeles, California.
in internal medicine training. J Grad Med entrustable professional activities
D. Hofkosh is professor of pediatrics and associate Educ. 2013;5:54–59. (EPAs): AMEE guide no. 99. Med Teach.
dean for faculty affairs, University of Pittsburgh 11 Shaughnessy AF, Sparks J, Cohen-Osher 2015;37:983–1002.
School of Medicine, Pittsburgh, Pennsylvania, and M, Goodell KH, Sawin GL, Gravel J Jr. 26 Frenk J, Chen L, Bhutta ZA, et al.
president, Association of Pediatric Program Directors, Entrustable professional activities in family Health professionals for a new century:
McLean, Virginia. medicine. J Grad Med Educ. 2013;5:112–118. Transforming education to strengthen health
12 Kennedy TJ, Lingard L, Baker GR, Kitchen L, systems in an interdependent world. Lancet.
M.A. Barone is associate professor of pediatrics
Regehr G. Clinical oversight: Conceptualizing 2010;376:1923–1958.
and director of medical student education, Johns
Hopkins University School of Medicine, Baltimore,
the relationship between supervision and 27 Di Francesco L, Pistoria MJ, Auerbach AD,
Maryland. safety. J Gen Intern Med. 2007;22:1080–1085. Nardino RJ, Holmboe ES. Internal medicine
13 Ten Cate O. Nuts and bolts of entrustable training in the inpatient setting. A review of
E.S. Holmboe is senior vice president of milestone professional activities. J Grad Med Educ. published educational interventions. J Gen
development and evaluation, Accreditation Council 2013;5:157–158.
for Graduate Medical Education, Chicago, Illinois.
Intern Med. 2005;20:1173–1180.
14 Carraccio C, Englander R, Holmboe E, Kogan
28 Kohn LT, Corrigan JM, Donaldson MS, eds.
J. Driving care quality: Aligning trainee
To Err Is Human—Building a Safer Health
assessment and supervision through practical
References application of entrustable professional
Care System. Washingon, DC: National
Academies Press; 1999.
1 Englander R, Cameron T, Ballard AJ, Dodge J, activities, competencies and milestones. Acad
29 Raymond MR, Mee J, King A, Haist SA,
Bull J, Aschenbrener CA. Toward a common Med. 2016;91:199–203.
taxonomy of competency domains for the Winward ML. What new residents do during
15 Hicks PJ, Schumacher DJ, Benson BJ, et
health professions and competencies for al. The pediatrics milestones: Conceptual their initial months of training. Acad Med.
physicians. Acad Med. 2013;88:1088–1094. framework, guiding principles, and 2011;86(10 suppl):S59–S62.
2 Accreditation Council for Graduate approach to development. J Grad Med Educ. 30 Touchie C, De Champlain A, Pugh D,
Medical Education. ACGME common 2010;2:410–418. Downing S, Bordage G. Supervising
program requirements. http://www. 16 Carraccio C, Gusic M, Hicks P, eds. The incoming first-year residents: Faculty
acgme.org/acgmeweb/Portals/0/PFAssets/ pediatrics milestone project. Acad Pediatr. expectations versus residents’ experiences.
ProgramRequirements/CPRs2013.pdf. 2014;14(2 suppl):S13–S97. Med Educ. 2014;48:921–929.
Revised July 1, 2013. Accessed January 15, 17 Englander R, Carraccio C. From theory to 31 Gingerich A, Regehr G, Eva KW. Rater-based
2016. practice: Making entrustable professional assessments as social judgments: Rethinking
3 American Board of Medical Specialties. activities come to life in the context of the etiology of rater errors. Acad Med.
Based on core competencies. http://www. milestones. Acad Med. 2014;89:1321–1323. 2011;86(10 suppl):S1–S7.
abms.org/board-certification/a-trusted- 18 Pangaro L, ten Cate O. Frameworks for 32 Crossley J, Johnson G, Booth J, Wade W.
credential/based-on-core-competencies/. learner assessment in medicine: AMEE guide Good questions, good answers: Construct
Accessed January 15, 2016. no. 78. Med Teach. 2013;35:e1197–e1210. alignment improves the performance of
4 ten Cate O, Scheele F. Competency-based 19 Jones MD Jr, Rosenberg AA, Gilhooly JT, workplace-based assessment scales. Med
postgraduate training: Can we bridge the gap Carraccio CL. Perspective: Competencies, Educ. 2011;45:560–569.
between theory and clinical practice? Acad outcomes, and controversy—linking 33 Pangaro LN. Two cheers for milestones.
Med. 2007;82:542–547. professional activities to competencies to J Grad Med Educ. 2015;7:4–6.
5 Nasca TJ, Philibert I, Brigham T, Flynn improve resident education and practice. 34 Association of American Medical Colleges.
TC. The next GME accreditation system— Acad Med. 2011;86:161–165. Medical schools to test guidelines for
rationale and benefits. N Engl J Med. 20 Regehr G, Ginsburg S, Herold J, Hatala R, preparing medical students for residency
2012;366:1051–1056. Eva K, Oulanova O. Using “standardized training. https://www.aamc.org/newsroom/
6 ten Cate O. Entrustability of professional narratives” to explore new ways to represent newsreleases/403960/09122014.html.
activities and competency-based training. faculty opinions of resident performance. Accessed January 15, 2016.
Med Educ. 2005;39:1176–1177. Acad Med. 2012;87:419–427. 35 Ginsburg S, McIlroy J, Oulanova O, Eva
7 Association of American Medical Colleges. 21 Calaman S, Hepps JH, Bismilla Z, et al. K, Regehr G. Toward authentic clinical
Core entrustable professional activites Creation of standard-setting videos to evaluation: Pitfalls in the pursuit of
for entering residency (updated). www. support faculty observations of learner competency. Acad Med. 2010;85:780–786.

Academic Medicine, Vol. XX, No. X / XX XXXX 7

Copyright © by the Association of American Medical Colleges. Unauthorized reproduction of this article is prohibited.