Does Active Learning Work?

A Review
of the Research

MICHAEL PRINCE ble to provide some generally accepted definitions and to highlight
Department of Chemical Engineering distinctions in how common terms are used.
Bucknell University Active learning is generally defined as any instructional method
that engages students in the learning process. In short, active learning
requires students to do meaningful learning activities and think about
ABSTRACT what they are doing [1]. While this definition could include tradi-
tional activities such as homework, in practice active learning refers to
This study examines the evidence for the effectiveness of active activities that are introduced into the classroom. The core elements of
learning. It defines the common forms of active learning most active learning are student activity and engagement in the learning
relevant for engineering faculty and critically examines the core process. Active learning is often contrasted to the traditional lecture
element of each method. It is found that there is broad but where students passively receive information from the instructor.
uneven support for the core elements of active, collaborative, Collaborative learning can refer to any instructional method in
cooperative and problem-based learning. which students work together in small groups toward a common goal
[2]. As such, collaborative learning can be viewed as encompassing all
Keywords: active, collaborative, cooperative, problem-based learning group-based instructional methods, including cooperative learning
[3–7]. In contrast, some authors distinguish between collaborative
and cooperative learning as having distinct historical developments
I. INTRODUCTION and different philosophical roots [8–10]. In either interpretation, the
core element of collaborative learning is the emphasis on student in-
Active learning has received considerable attention over the teractions rather than on learning as a solitary activity.
past several years. Often presented or perceived as a radical change Cooperative learning can be defined as a structured form of group
from traditional instruction, the topic frequently polarizes faculty. work where students pursue common goals while being assessed in-
Active learning has attracted strong advocates among faculty look- dividually [3, 11]. The most common model of cooperative learn-
ing for alternatives to traditional teaching methods, while skeptical ing found in the engineering literature is that of Johnson, Johnson
faculty regard active learning as another in a long line of educa- and Smith [12, 13]. This model incorporates five specific tenets,
tional fads. which are individual accountability, mutual interdependence, face-
For many faculty there remain questions about what active to-face promotive interaction, appropriate practice of interpersonal
learning is and how it differs from traditional engineering educa- skills, and regular self-assessment of team functioning. While dif-
tion, since this is already “active” through homework assignments ferent cooperative learning models exist [14, 15], the core element
and laboratories. Adding to the confusion, engineering faculty do held in common is a focus on cooperative incentives rather than
not always understand how the common forms of active learning competition to promote learning.
differ from each other and most engineering faculty are not inclined Problem-based learning (PBL) is an instructional method where rele-
to comb the educational literature for answers. vant problems are introduced at the beginning of the instruction cycle
This study addresses each of these issues. First, it defines active and used to provide the context and motivation for the learning that
learning and distinguishes the different types of active learning follows. It is always active and usually (but not necessarily) collaborative
most frequently discussed in the engineering literature. A core ele- or cooperative using the above definitions. PBL typically involves sig-
ment is identified for each of these separate methods in order to dif- nificant amounts of self-directed learning on the part of the students.
ferentiate between them, as well as to aid in the subsequent analysis
of their effectiveness. Second, the study provides an overview of rel-
evant cautions for the reader trying to draw quick conclusions on III. COMMON PROBLEMS INTERPRETING THE
the effectiveness of active learning from the educational literature. LITERATURE ON ACTIVE LEARNING
Finally, it assists engineering faculty by summarizing some of the
most relevant literature in the field of active learning. Before examining the literature to analyze the effectiveness of
each approach, it is worth highlighting common problems that en-
gineering faculty should appreciate before attempting to draw con-
II. DEFINITIONS clusions from the literature.

It is not possible to provide universally accepted definitions for A. Problems Defining What Is Being Studied
all of the vocabulary of active learning since different authors in the Confusion can result from reading the literature on the effec-
field have interpreted some terms differently. However, it is possi- tiveness of any instructional method unless the reader and author

July 2004 Journal of Engineering Education 223

take care to specify precisely what is being examined. For example, ability to solve open-ended problems or engage in life-long learn-
there are many different approaches that go under the name of ing, since PBL typically provides practice in both skills. However,
problem-based learning [16]. These distinct approaches to PBL problem solving and life-long learning are difficult to measure. As a
can have as many differences as they have elements in common, result, data are less frequently available for these outcomes than for
making interpretation of the literature difficult. In PBL, for exam- standard measures of academic achievement such as test scores.
ple, students typically work in small teams to solve problems in a This makes it difficult to know whether the potential of PBL to
self-directed fashion. Looking at a number of meta-analyses [17], promote these outcomes is achieved in practice.
Norman and Schmidt [18] point out that having students work in Even when data on higher-level outcomes are available, it is easy
small teams has a positive effect on academic achievement while to misinterpret reported results. Consider a study by Qin et al. [21]
self-directed learning has a slight negative effect on academic that reports that cooperation promotes higher quality individual
achievement. If PBL includes both of these elements and one asks if problem solving than does competition. The result stems from the
PBL works for promoting academic achievement, the answer seems finding that individuals in cooperative groups produced better solu-
to be that parts of it do and parts of it do not. Since different appli- tions to problems than individuals working in competitive environ-
cations of PBL will emphasize different components, the literature ments. While the finding might provide strong support for cooper-
results on the overall effectiveness of PBL are bound to be confus- ative learning, it is important to understand what the study does not
ing unless one takes care to specify what is being examined. This is specifically demonstrate. It does not necessarily follow from these
even truer of the more broadly defined approaches of active or col- results that students in cooperative environments developed
laborative learning, which encompass very distinct practices. stronger, more permanent and more transferable problem solving
Note that this point sheds a different light on some of the avail- skills. Faculty citing the reference to prove that cooperative learning
able meta-analyses that are naturally attractive to a reader hoping results in individuals becoming generically better problem solvers
for a quick overview of the field. In looking for a general sense of would be over-interpreting the results.
whether an approach like problem-based learning works, nothing A separate problem determining what works is deciding when
seems as attractive as a meta-analysis that brings together the results an improvement is significant. Proponents of active learning some-
of several studies and quantitatively examines the impact of the ap- times cite improvements without mentioning that the magnitude of
proach. While this has value, there are pitfalls. Aggregating the re- the improvement is small [22]. This is particularly misleading when
sults of several studies on the effectiveness of PBL can be mislead- extra effort or resources are required to produce an improvement.
ing if the forms of PBL vary significantly in each of the individual Quantifying the impact of an intervention is often done using effect
studies included in the meta-analysis. sizes, which are defined to be the difference in the means of a sub-
To minimize this problem, the analysis presented in Section IV ject and control population divided by the pooled standard devia-
of this paper focuses on the specific core elements of a given instruc- tion of the populations. An improvement with an effect size of 1.0
tional method. For example, as discussed in Section II, the core ele- would mean that the test population outperformed the control
ment of collaborative learning is working in groups rather than group by one standard deviation. Albanese [23] cites the benefits of
working individually. Similarly, the core element of cooperative using effect sizes and points out that Cohen [24] arbitrarily labeled
learning is cooperation rather than competition. These distinctions effect sizes of 0.2, 0.5 and 0.8 as small, medium and large, respec-
can be examined without ambiguity. Furthermore, focusing on the tively. Colliver [22] used this fact and other arguments to suggest
core element of active learning methods allows a broad field to be that effect sizes should be at least 0.8 before they be considered sig-
treated concisely. nificant. However, this suggestion would discount almost every
available finding since effect sizes of 0.8 are rare for any intervention
B. Problems Measuring “What Works” and require truly impressive gains [23]. The effect sizes of 0.5 or
Just as every instructional method consists of more than one ele- higher reported in Section IV of this paper are higher than those
ment, it also affects more than one learning outcome [18]. When found for most instructional interventions. Indeed, several decades
asking whether active learning “works,” the broad range of out- of research indicated that standard measures of academic achieve-
comes should be considered such as measures of factual knowledge, ment were not particularly sensitive to any change in instructional
relevant skills and student attitudes, and pragmatic items as student approach [25]. Therefore, reported improvements in academic
retention in academic programs. However, solid data on how an in- achievement should not be dismissed lightly.
structional method impacts all of these learning outcomes is often Note that while effect sizes are a common measure of the mag-
not available, making comprehensive assessment difficult. In addi- nitude of an improvement, absolute rather than relative values are
tion, where data on multiple learning outcomes exists it can include sometimes more telling. There can be an important difference be-
mixed results. For example, some studies on problem-based learn- tween results that are statistically significant and those that are sig-
ing with medical students [19, 20] suggest that clinical performance nificant in absolute terms. For this reason, it is often best to find
is slightly enhanced while performance on standardized exams de- both statistical and absolute measures of the magnitude of a report-
clines slightly. In cases like this, whether an approach works is a ed improvement before deciding whether it is significant.
matter of interpretation and both proponents and detractors can As a final cautionary note for interpreting reported results, some
comfortably hold different views. readers dismiss reported improvements from nontraditional in-
Another significant problem with assessment is that many rele- structional methods because they attribute them to the Hawthorne
vant learning outcomes are simply difficult to measure. This is par- effect whereby the subjects knowingly react positively to any novel
ticularly true for some of the higher level learning outcomes that are intervention regardless of its merit. The Hawthorne effect is gener-
targeted by active learning methods. For example, PBL might nat- ally discredited, although it retains a strong hold on the popular
urally attract instructors interested in developing their students’ imagination [26].

224 Journal of Engineering Education July 2004

C. Summary Despite this, the empirical support for active learning is exten-
There are pitfalls for engineering faculty hoping to pick up an ar- sive. However, the variety of instructional methods labeled as active
ticle or two to see if active learning works. In particular, readers learning muddles the issue. Given differences in the approaches la-
must clarify what is being studied and how the authors measure and beled as active learning, it is not always clear what is being promoted
interpret what “works.” The former is complicated by the wide by broad claims supporting the adoption of active learning. Perhaps
range of methods that fall under the name of active learning, but it is best, as some proponents claim, to think of active learning as an
can be simplified by focusing on core elements of common active approach rather than a method [31] and to recognize that different
learning methods. Assessing “what works” requires looking at a methods are best assessed separately.
broad range of learning outcomes, interpreting data carefully, quan- This assessment is done in the following sections, which look at
tifying the magnitude of any reported improvement and having the empirical support for active, collaborative, cooperative and prob-
some idea of what constitutes a “significant” improvement. This last lem-based learning. As previously discussed, the critical elements of
will always be a matter of interpretation, although it is helpful to each approach are singled out rather than examining the effective-
look at both statistical measures such as effect sizes and absolute val- ness of every possible implementation scheme for each of these dis-
ues for reported learning gains. tinct methods. The benefits of this general approach are twofold.
No matter how data is presented, faculty adopting instructional First, it allows the reader to examine questions that are both funda-
practices with the expectation of seeing results similar to those re- mental and pragmatic, such as whether introducing activity into the
ported in the literature should be aware of the practical limitations lecture or putting students into groups, is effective. Second, focusing
of educational studies. Educational studies tell us what worked, on on the core element eliminates the need to examine the effectiveness
average, for the populations examined and learning theories suggest of every instructional technique that falls under a given broad catego-
why this might be so. However, claiming that faculty who adopt a ry, which would be impractical within the scope of a single paper.
specific method will see similar results in their own classrooms is Readers looking for literature on a number of specific active learning
simply not possible. Even if faculty master the new instructional methods are referred to additional references [1, 6, 32].
method, they can not control all other variables that affect learning.
The value of the results presented in Section IV of the paper is that A. Active Learning
they provide information to help teachers “go with the odds.” The We have defined the core elements of active learning to be intro-
more extensive the data supporting an intervention, the more a ducing activities into the traditional lecture and promoting student
teacher’s students resemble the test population and the bigger the engagement. Both elements are examined below, with an emphasis
reported gains, the better the odds are that the method will work for on empirical support for their effectiveness.
a given instructor.
Notwithstanding all of these problems, engineering faculty 1) Introducing student activity into the traditional lecture: On
should be strongly encouraged to look at the literature on active the simplest level, active learning is introducing student activity into
learning. Some of the evidence for active learning is compelling and the traditional lecture. One example of this is for the lecturer to
should stimulate faculty to think about teaching and learning in pause periodically and have students clarify their notes with a part-
nontraditional ways. ner. This can be done two or three times during an hour-long class.
Because this pause procedure is so simple, it provides a baseline to
study whether short, informal student activities can improve the ef-
IV. THE EVIDENCE FOR ACTIVE LEARNING fectiveness of lectures.
Ruhl et al. [33] show some significant results of adopting this
Bonwell and Eison [1] summarize the literature on active learn- pause procedure. In a study involving 72 students over two courses
ing and conclude that it leads to better student attitudes and im- in each of two semesters, the researchers examined the effect of in-
provements in students’ thinking and writing. They also cite evi- terrupting a 45-minute lecture three times with two-minute breaks
dence from McKeachie that discussion, one form of active learning, during which students worked in pairs to clarify their notes. In par-
surpasses traditional lectures for retention of material, motivating allel with this approach, they taught a separate group using a
students for further study and developing thinking skills. Felder straight lecture and then tested short and long-term retention of
et al. [27] include active learning on their recommendations for lecture material. Short-term retention was assessed by a free-recall
teaching methods that work, noting among other things that active exercise where students wrote down everything they could remem-
learning is one of Chickering and Gamson’s “Seven Principles for ber in three minutes after each lecture and results were scored by the
Good Practice” [28]. number of correct facts recorded. Short-term recall with the pause
However, not all of this support for active learning is compelling. procedure averaged 108 correct facts compared to 80 correct facts
McKeachie himself admits that the measured improvements of dis- recalled in classes with straight lecture. Long-term retention was as-
cussion over lecture are small [29]. In addition, Chickering and sessed with a 65 question multiple-choice exam given one and a half
Gamson do not provide hard evidence to support active learning as weeks after the last of five lectures used in the study. Test scores
one of their principles. Even studies addressing the research base for were 89.4 with the pause procedure compared to 80.9 without
Chickering and Gamson’s principles come across as thin with re- pause for one class, and 80.4 with the pause procedure compared to
spect to empirical support for active learning. For example, Scorcelli 72.6 with no pause in the other class. Further support for the effec-
[30], in a study aimed at presenting the research base for Chicker- tiveness of pauses during the lecture is provided by Di Vesta [34].
ing and Gamson’s seven principles, states that, “We simply do not Many proponents of active learning suggest that the effectiveness
have much data confirming beneficial effects of other (not coopera- of this approach has to do with student attention span during lecture.
tive or social) kinds of active learning.” Wankat [35] cites numerous studies that suggest that student

July 2004 Journal of Engineering Education 225

 reports that student involvement is one of the most important pre-
dictors of success in college. Hake [39] examined pre- and post-test
data for over 6,000 students in introductory physics courses and
found significantly improved performance for students in classes
with substantial use of interactive-engagement methods. Test
scores measuring conceptual understanding were roughly twice as
high in classes promoting engagement than in traditional courses.
Statistically, this was an improvement of two standard deviations
above that of traditional courses. Other results supporting the effec-
tiveness of active-engagement methods are reported by Redish et al.
[40] and Laws et al. [41]. Redish et al. show that the improved
learning gains are due to the nature of active engagement and not to
Figure 1. Active-engagement vs. traditional instruction for im- extra time spent on a given topic. Figure 1, taken from Laws et al.,
proving students’ conceptual understanding of basic physics concepts shows that active engagement methods surpass traditional instruc-
(taken from Laws et al., 1999) tion for improving conceptual understanding of basic physics con-
cepts. The differences are quite significant. Taken together, the
studies of Hake et al., Redish et al. and Laws et al. provide consider-
attention span during lecture is roughly fifteen minutes. After that, able support for active engagement methods, particularly for ad-
Hartley and Davies [36] found that the number of students paying dressing students’ fundamental misconceptions. The importance of
attention begins to drop dramatically with a resulting loss in reten- addressing student misconceptions has recently been recognized as
tion of lecture material. The same authors found that immediately an essential element of effective teaching [42].
after the lecture students remembered 70 percent of information In summary, considerable support exists for the core elements of
presented in first ten minutes of the lecture and 20 percent of infor- active learning. Introducing activity into lectures can significantly
mation presented in last ten minutes. Breaking up the lecture might improve recall of information while extensive evidence supports the
work because students’ minds start to wander and activities provide benefits of student engagement.
the opportunity to start fresh again, keeping students engaged.
B. Collaborative Learning
2) Promoting Student Engagement: Simply introducing activ- The central element of collaborative learning is collaborative vs.
ity into the classroom fails to capture an important component of individual work and the analysis therefore focuses on how collabora-
active learning. The type of activity, for example, influences how tion influences learning outcomes. The results of existing meta-stud-
much classroom material is retained [34]. In “Understanding by ies on this question are consistent. In a review of 90 years of research,
Design” [37], the authors emphasize that good activities develop Johnson, Johnson and Smith found that cooperation improved learn-
deep understanding of the important ideas to be learned. To do ing outcomes relative to individual work across the board [12]. Simi-
this, the activities must be designed around important learning out- lar results were found in an updated study by the same authors [13]
comes and promote thoughtful engagement on the part of the stu- that looked at 168 studies between 1924 and 1997. Springer et al.
dent. The activity used by Ruhl, for example, encourages students [43] found similar results looking at 37 studies of students in science,
to think about what they are learning. Adopting instructional prac- mathematics, engineering and technology. Reported results for each
tices that engage students in the learning process is the defining fea- of these studies are shown in Table 1, using effect sizes to show the
ture of active learning. impact of collaboration on a range of learning outcomes.
The importance of student engagement is widely accepted and What do these results mean in real terms instead of effect sizes,
there is considerable evidence to support the effectiveness of student which are sometimes difficult to interpret? With respect to academic
engagement on a broad range of learning outcomes. Astin [38] achievement, the lowest of the three studies cited would move a

Table 1. Collaborative vs. individualistic learning: Reported effect size of the improvement in different learning outcomes.

226 Journal of Engineering Education July 2004

 Table 2. Collaborative vs. competitive learning: Reported effect size of the improvement in different learning outcomes.

student from the 50th to the 70th percentile on an exam. In absolute effective teamwork and interpersonal skills. For engineering faculty,
terms, this change is consistent with raising a student’s grade from the need to develop these skills in their students is reflected by the
75 to 81, given classical assumptions about grade distributions.* ABET engineering criteria. Employers frequently identify team
With respect to retention, the results suggest that collaboration re- skills as a critical gap in the preparation of engineering students.
duces attrition in technical programs by 22 percent, a significant Since practice is a precondition of learning any skill, it is difficult to
finding when technical programs are struggling to attract and retain argue that individual work in traditional classes does anything to
students. Furthermore, some evidence suggests that collaboration is develop team skills.
particularly effective for improving retention of traditionally under- Whether cooperative learning effectively develops interpersonal
represented groups [44, 45]. skills is another question. Part of the difficulty in answering that
A related question of practical interest is whether the benefits of question stems from how one defines and measures team skills.
group work improve with frequency. Springer et al. looked specifical- Still, there is reason to think that cooperative learning is effective in
ly at the effect of incorporating small, medium and large amounts of this area. Johnson et al. [12, 13] recommend explicitly training stu-
group work on achievement and found the positive effect sizes associ- dents in the skills needed to be effective team members when using
ated with low, medium and high amount of time in groups to be 0.52, cooperative learning groups. It is reasonable to assume that the op-
0.73 and 0.53, respectively. That is, the highest benefit was found for portunity to practice interpersonal skills coupled with explicit in-
medium time in groups. In contrast, more time spent in groups did structions in these skills is more effective than traditional instruction
produce the highest effect on promoting positive student attitudes, that emphasizes individual learning and generally has no explicit in-
with low, medium and high amount of time in groups having effect struction in teamwork. There is also empirical evidence to support
sizes of 0.37, 0.26, and 0.77, respectively. Springer et al. note that the this conclusion. Johnson and Johnson report that social skills tend
attitudinal results were based on a relatively small number of studies. to increase more within cooperative rather than competitive or indi-
In summary, a number of meta-analyses support the premise vidual situations [46]. Terenzini et al. [47] show that students re-
that collaboration “works” for promoting a broad range of student port increased team skills as a result of cooperative learning. In addi-
learning outcomes. In particular, collaboration enhances academic tion, Panitz [48] cites a number of benefits of cooperative learning
achievement, student attitudes, and student retention. The magni- for developing the interpersonal skills required for effective team-
tude, consistency and relevance of these results strongly suggest that work.
engineering faculty promote student collaboration in their courses. In summary, there is broad empirical support for the central
premise of cooperative learning, that cooperation is more effective
C. Cooperative Learning than competition for promoting a range of positive learning out-
At its core, cooperative learning is based on the premise that co- comes. These results include enhanced academic achievement and a
operation is more effective than competition among students for number of attitudinal outcomes. In addition, cooperative learning
producing positive learning outcomes. This is examined in Table 2. provides a natural environment in which to enhance interpersonal
The reported results are consistently positive. Indeed, looking at skills and there are rational arguments and evidence to show the ef-
high quality studies with good internal validity, the already large ef- fectiveness of cooperation in this regard.
fect size of 0.67 shown in Table 2 for academic achievement in-
creases to 0.88. In real terms, this would increase a student’s exam D. Problem-Based Learning
score from 75 to 85 in the “classic” example cited previously, though As mentioned in Section II of this paper, the first step of deter-
of course this specific result is dependent on the assumed grade dis- mining whether an educational approach works is clarifying exactly
tribution. As seen in Table 2, cooperation also promotes interper- what the approach is. Unfortunately, while there is agreement on
sonal relationships, improves social support and fosters self-esteem. the general definition of PBL, implementation varies widely.
Another issue of interest to engineering faculty is that coopera- Woods et al. [16], for example, discuss several variations of PBL.
tive learning provides a natural environment in which to promote
“Once a problem has been posed, different instructional methods may be
used to facilitate the subsequent learning process: lecturing, instructor-
*Calculated using an effect size of 0.5, a mean of 75 and a normalized grade dis-
tribution where the top 10 percent of students receive a 90 or higher (an A) and the facilitated discussion, guided decision making, or cooperative learning. As
bottom 10 percent receive a 60 or lower (an F). part of the problem-solving process, student groups can be assigned to

July 2004 Journal of Engineering Education 227

 complete any of the learning tasks listed above, either in or out of class. In the use, textbook reading, class attendance and studying for meaning
latter case, three approaches may be adopted to help the groups stay on track rather than simple recall [19, 20, 53, 54].
and to monitor their progress: (1) give the groups written feedback after each We have already discussed the problems with meta-studies that
task; (2) assign a tutor or teaching assistant to each group, or (3) create fully compare non-uniform and inconsistently defined educational inter-
autonomous, self-assessed “tutorless” groups.” ventions. Such studies are easily prone to factors that obscure re-
sults. The approach for handling this difficulty with active, collabo-
The large variation in PBL practices makes the analysis of its ef- rative and cooperative learning was to identify the central element
fectiveness more complex. Many studies comparing PBL to tradi- of the approach and to focus on this rather than on implementation
tional programs are simply not talking about the same thing. For methods. That is more difficult to do with PBL since it is not clear
meta-studies of PBL to show any significant effect compared to tra- that one or two core elements exist. PBL is active, engages students
ditional programs, the signal from the common elements of PBL and is generally collaborative, all of which are supported by our pre-
would have to be greater than the noise produced by differences in vious analysis. It is also inductive, generally self-directed, and often
the implementation of both PBL and the traditional curricula. includes explicit training in necessary skills. Can one or two ele-
Given the huge variation in PBL practices, not to mention differ- ments be identified as common or decisive?
ences in traditional programs, readers should not be surprised if no Norman and Schmidt [18] provide one way around the difficul-
consistent results emerge from meta-studies that group together ty by identifying several components of PBL in order to show how
different PBL methods. they impact learning outcomes. Their results are shown in Table 3,
Despite this, there is at least one generally accepted finding that taken directly from Norman and Schmidt using the summary of
emerges from the literature, which is that PBL produces positive meta-studies provided by Lipsey and Wilson [17]. The measured
student attitudes. Vernon and Blake [19] looking at 35 studies from learning outcome for all educational studies cited by Lipsey and
1970 to 1992 for medical programs found that PBL produced a sig- Wilson was academic achievement.
nificant effective size (0.55) for improved student attitudes and Norman and Schmidt present this table to illustrate how differ-
opinions about their programs. Albanese and Mitchell [20] similar- ent elements of PBL have different effects on learning outcomes.
ly found that students and faculty generally prefer the PBL ap- However, the substantive findings of Table 3 are also worth high-
proach. Norman and Schmidt [18] argue “PBL does provide a lighting for faculty interested in adopting PBL because there seems
more challenging, motivating and enjoyable approach to education. to be considerable agreement on what works and does not work in
That may be a sufficient raison d’etre, providing the cost of the im- PBL.
plementation is not too great.” Note that these and most of the re- Looking first at the negative effects, there is a significant nega-
sults reported in this section come from studies of medical students, tive effect size using PBL with non-expert tutors. This finding is
for whom PBL has been widely used. While PBL has been used in consistent with some of the literature on helping students make the
undergraduate engineering programs [49, 50] there is very little transition from novice to expert problem solvers. Research compar-
data available for its effectiveness with this population of students. ing experts to novices in a given field has demonstrated that becom-
Beyond producing positive student attitudes, the effects of PBL ing an expert is not just a matter of “good thinking” [42]. Instead,
are less generally accepted, though other supporting data do exist. research has demonstrated the necessity for experts to have both a
Vernon and Blake [19], for example, present evidence that there is a deep and broad foundation of factual knowledge in their fields. The
statistically significant improvement of PBL on students’ clinical same appears to be true for tutors in PBL.
performance with an effect size of 0.28. However, Colliver [22] There is also a small negative effect associated with both self-
points out that this is influenced strongly by one outlying study with paced and self-directed learning. This result is consistent with the
a positive effect size of 2.11, which skews the data. There is also evi- findings of Albanese and Mitchell [20] on the effect of PBL on test
dence that PBL improves the long-term retention of knowledge results. In seven out of ten cases they found that students in PBL
compared to traditional instruction [51–53]. Evidence also suggests programs scored lower than students in traditional programs on
that PBL promotes better study habits among students. As one tests of basic science. However, in three out of ten cases, PBL
might expect from an approach that requires more independence students actually scored higher. Albanese and Mitchell note that
from students, PBL has frequently been shown to increase library these three PBL programs were more “directive” than others,

Table 3. Effect sizes for academic achievement associated with various aspects of problem-based learning.

228 Journal of Engineering Education July 2004

indicating that this element might be responsible for the superior challenge traditional assumptions about engineering education and
exam performance for students in those programs. Therefore, facul- these are most worth highlighting.
ty might be advised to be cautious about the amount of self-direc- For example, students will remember more content if brief activi-
tion required by students in PBL, at least with regard to promoting ties are introduced to the lecture. Contrast this to the prevalent con-
academic achievement as measured by traditional exams. tent tyranny that encourages faculty to push through as much materi-
Looking at what seems to work, there are significant positive ef- al as possible in a given session. Similarly, the support for collaborative
fect sizes associated with placing students in small groups and using and cooperative learning calls into question the traditional assump-
cooperative learning structures. This is consistent with much of the tions that individual work and competition best promote achieve-
literature cited previously in support of cooperative learning. While ment. The best available evidence suggests that faculty should struc-
PBL and cooperative learning are distinct approaches, there is a ture their courses to promote collaborative and cooperative
natural synergy that instructors should consider exploiting. That is, environments. The entire course need not be team-based, as seen by
real problems of the sort used in PBL require teams to solve effec- the evidence in Springer et al. [43], nor must individual responsibility
tively. At the same time, the challenge provided by realistic prob- be absent, as seen by the emphasis on individual accountability in co-
lems can provide some of the mutual interdependence that is one of operative learning. Nevertheless, extensive and credible evidence sug-
the five tenets of cooperative learning. gests that faculty consider a nontraditional model for promoting aca-
Table 3 also shows that positive results come from instruction in demic achievement and positive student attitudes.
problem solving. This is consistent with much of the advice given Problem-based learning presents the most difficult method to
by proponents of problem-based learning [55]. While practice is analyze because it includes a variety of practices and lacks a domi-
crucial for mastering skills such as problem solving, greater gains are nant core element to facilitate analysis. Rather, different implemen-
realized through explicit instruction of problem solving skills. How- tations of PBL emphasize different elements, some more effective
ever, traditional engineering courses do not generally teach problem for promoting academic achievement than others. Based on the lit-
solving skills explicitly. Table 3 suggests that faculty using PBL erature, faculty adopting PBL are unlikely to see improvements in
consider doing just that. student test scores, but are likely to positively influence student atti-
In conclusion, PBL is difficult to analyze because there is not tudes and study habits. Studies also suggest that students will retain
one or two core elements that can be clearly identified with student information longer and perhaps develop enhanced critical thinking
learning outcomes. Perhaps the closest candidates for core elements and problem-solving skills, especially if PBL is coupled with explicit
would be inductive or discovery learning. These have been shown instruction in these skills.
by meta-studies to have only weakly positive effects on student aca- Teaching cannot be reduced to formulaic methods and active
demic achievement [56, 57] as measured by exams. This might ex- learning is not the cure for all educational problems. However, there
plain why PBL similarly shows no improvement on student test is broad support for the elements of active learning most commonly
scores, the most common measure of academic achievement. discussed in the educational literature and analyzed here. Some of
However, while no evidence proves that PBL enhances academ- the findings are surprising and deserve special attention. Engineer-
ic achievement as measured by exams, there is evidence to suggest ing faculty should be aware of these different instructional methods
that PBL “works” for achieving other important learning outcomes. and make an effort to have their teaching informed by the literature
Studies suggest that PBL develops more positive student attitudes, on “what works.”
fosters a deeper approach to learning and helps students retain
knowledge longer than traditional instruction. Further, just as co-
operative learning provides a natural environment to promote inter- ACKNOWLEDGMENTS
personal skills, PBL provides a natural environment for developing
problem-solving and life-long learning skills. Indeed, some evi- The author would like to thank Richard Felder for his thoughtful
dence shows that PBL develops enhanced problem-solving skills in critique of this work and for many similar pieces of advice over the
medical students and that these skills can be improved further by past several years. The National Science Foundation through Project
coupling PBL with explicit instruction in problem solving. Similar- Catalyst (NSF9972758) provided financial support for this project.
ly, supporting arguments can be made about PBL and the impor-
tant ABET engineering outcome of life-long learning. Since self-
directed learning and meta-cognition are common to both PBL REFERENCES
and life-long learning, a logical connection exists between these de-
sired learning outcomes and PBL instruction, something often not [1] Bonwell, C.C., and J. A. Eison, “Active Learning: Creating Ex-
true when trying to promote life-long learning through traditional citement in the Classroom,” ASHEERIC Higher Education Report No.
teaching methods. 1, George Washington University, Washington, DC , 1991.
[2] Online Collaborative Learning in Higher Education, http://clp.
cqu.edu.au/glossary.htm, accessed 12/3/2003.
IV. CONCLUSIONS [3] Millis, B., and P. Cottell, Jr., “Cooperative Learning for Higher
Education Faculty,” American Council on Education, ORYX Press, 1998.
Although the results vary in strength, this study has found sup- [4] Smith, B., and J. MacGregor, “What is Collaborative Learning?,”
port for all forms of active learning examined. Some of the findings, in Goodsell, A., M. Mahler, V. Tinto, B.L.Smith, and J. MacGreger,
such as the benefits of student engagement, are unlikely to be con- (Eds), Collaborative Learning: A Sourcebook for Higher Education (pp. 9–22).
troversial although the magnitude of improvements resulting from University Park, PA: National Center on Postsecondary Teaching, Learn-
active-engagement methods may come as a surprise. Other findings ing and Assessment, 1992.

July 2004 Journal of Engineering Education 229

 [5] Cusea, J., “Collaborative & Cooperative Learning in Higher Edu- [26] Bracey, G., “Tips for Readers of Research: Beware the ‘Classic
cation: A Proposed Taxonomy,” Cooperative Learning and College Teaching, Study’,” Phi Delta Kappan, Vol. 83, No. 8, April 2002, p. 642.
Vol. 2, No. 2, 2–4, 1992. [27] Felder, R., D. Woods, J. Stice, and A. Rugarcia, “The Future of
[6] Bean, J., “Engaging Ideas: The Professor’s Guide to Integrating Engineering Education: II.Teaching Methods that Work,” Chemical Engi-
Writing, Critical Thinking, and Active Learning in the Classroom,” Josey- neering Education, Vol. 34, No. 1, 2000, pp. 26–39.
Bass Publishers: San Francisco, 1996. [28] Chickering, A., and Z. Gamson, “Seven Principles for Good
[7] Felder, R., Brent, R., and Stice, J., “National Effective Teaching Practice,” AAHE Bulletin, Vol. 39, ED 282 491, March 1987, pp. 3–7.
Institute: Workshop Materials,” 2002 American Society for Engineering [29] McKeachie, W., “Research on College Teaching,” Educational
Education Annual Conference, Montreal, Quebec, Canada, 2002. Perspectives, Vol. 11, No. 2, May 1972, pp. 3–10.
[8] Bruffee, K., “Sharing Our Toys: Cooperative Learning Versus [30] Sorcinelli, M., “Research Findings on the Seven Principles,” in
Collaborative Learning,” Change, January/February, 1995, p. 12. A.W. Chickering and Z.F. Gamson, eds., Applying the Seven Principles for
[9] Panitz, T., “Collaborative Versus Cooperative Learning-A Com- Good Practice in Undergraduate Education, New Directions in Teaching and
parison of the Two Concepts Which Will Help Us Understand the Un- Learning, #47, San Francisco: Jossey-Bass, 1991.
derlying Nature of Interactive Learning,” http://capecod.net/~tpanitz/ [31] http://trc.ucdavis.edu/trc/active/defini.html, accessed
tedsarticles/coopdefinition.htm, accessed 12/2/2003. 12/3/03.
[10] http://www.wcer.wisc.edu/nise/CL1/CL/question/TQ13. [32] MacGregor, J., Cooper, J., Smith, K., and Robinson, P. (Eds.)
htm, accessed 12/3/2003. “Strategies for Energizing Large Classes: From Small Groups to Learning
[11] Feden, P., and R. Vogel, Methods of Teaching: Applying Cognitive Communities,” New Directions for Teaching and Learning, Vol. 81, Jossey-
Science to Promote Student Learning, McGraw Hill Higher Education, 2003. Bass, 2000.
[12] Johnson, D., R., Johnson, and K. Smith, Active Learning: Cooper- [33] Ruhl, K., C. Hughes, and P. Schloss, “Using the Pause Procedure
ation in the College Classroom, 2nd ed., Interaction Book Co., Edina, MN, to Enhance Lecture Recall,” Teacher Education and Special Education, Vol.
1998. 10, Winter 1987, pp. 14–18.
[13] Johnson, D., R., Johnson, and K. Smith, “Cooperative Learning [34] Di Vesta, F., and D. Smith, “The Pausing Principle: Increasing
Returns to College: What Evidence is There That it Works?,” Change, the Efficiency of Memory for Ongoing Events,” Contemporary Educational
Vol. 30, No. 4. July/Aug., 1998, p. 26–35. Psychology, Vol. 4, 1979.
[14] Stahl, R., “The Essential Elements of Cooperative Learning in [35] Wankat, P., The Effective Efficient Professor: Teaching, Scholarship
the Classroom,” ERIC Digest ED370881, 1994, available online at and Service, Allyn and Bacon: Boston, MA, 2002.
http://www.ericfacility.net/ericdigests/ed370881.html. [36] Hartley, J., and Davies, I., “Note Taking: A Critical Review,”
[15] Slavin, R., “Cooperative Learning. Research on Teaching Mono- Programmed Learning and Educational Technology, Vol. 15, 1978,
graph Series,” ERIC Digest ED242707, 1983. pp. 207–224.
[16] Woods, D., R Felder, A. Rugarcia, and J. Stice, “The Future of [37] Wiggins, G., and J. McTighe, “Understanding by Design,”
Engineering Education. III. Developing Critical Skills,” Chemical Engi- Merrill Education/ASCD College Textbook Series, ASCD, Alexandria,
neering Education, Vol. 34, No. 2, 2000, pp. 108–117. Virginia, 1998.
[17] Lipsey, M.W., and Wilson, D.B., “The Efficacy of Psychological, [38] Astin, A., What Matters in College?; Four Critical Years Revisited,
Educational and Behavioral Treatment: Confirmation from Meta- Josey-Bass: San Francisco, CA, 1993.
Analysis,” American Psychology, Vol. 12, 1993, p. 1181–1209. [39] Hake, R., “Interactive-Engagement vs. Traditional Methods: A
[18] Norman, G., and H. Schmidt, “Effectiveness of Problem-Based Six-Thousand-Student Survey of Mechanics Test Data for Introductory
Learning Curricula: Theory, Practice and Paper Darts,” Medical Education, Physics Courses,” American Journal of Physics, Vol. 66, No. 1, 1998, p. 64.
Vol. 34, 2000, pp. 721–728. [40] Redish, E., J. Saul, and R. Steinberg, “On the Effectiveness of
[19] Vernon, D., and R. Blake, “Does Problem-Based Learning Active-Engagement Microcomputer-Based Laboratories,” American
Work? A Meta-Analysis of Evaluative Research,” Academic Medicine, Vol. Journal of Physics, Vol. 65, No. 1, 1997, p. 45.
68, No. 7, July 1993. [41] Laws, P., D. Sokoloff, and R. Thornton, “Promoting Active
[20] Albanese, M. and S. Mitchell, “Problem-Based Learning: A Re- Learning Using the Results of Physics Education Research,” UniServe
view of Literature on Its Outcomes and Implementation Issues,” Academic Science News, Vol. 13, July 1999.
Medicine, Vol. 68, No. 1, January 1993. [42] Bransford, J., A. Brown, and R. Cocking, (Commission on Be-
[21] Qin, Z., Johnson, D., and Johnson, R., “Cooperative Versus havioral and Social Science and Education, National Research Council),
Competitive Efforts and Problem Solving,” Review of Educational Research, “How People Learn: Body, Mind, Experience and School,” National
Vol. 65, No. 2, Summer 1995, p. 129. Academy Press, Washington D.C., 2000. Available online at
[22] Colliver, J., “Effectiveness of Problem-Based Learning Curricu- http://www.nap.edu/html/howpeople1/.
la,” Academic Medicine, Vol. 75, 2000, p. 259. [43] Springer, L., M. Stanne, and S. Donovan, “Effects of Small-
[23] Albanese, M., “Problem-Based Learning: Why Curricula Are Group Learning on Undergraduates in Science, Mathematics, Engineer-
Likely to Show Little Effect on Knowledge and Clinical Skills,” Medical ing and Technology: A Meta-Analysis,” Review of Educational Research,
Education, Vol. 34, No. 9, 2000, p. 729. Vol. 69, No. 1, 1999, pp. 21–52.
[24] Cohen, J., Statistical Power Analysis for Behavioral Sciences, Revised [44] Fredericksen, E., “Minority Students and the Learning Commu-
Edition, Englewood Cliffs, New Jersey: Erlbaum; 1977. nity Experience: A Cluster Experiment,” U.S.: Texas 1998-0400,
[25] Dubin R., and T. Taveggia, “The Teaching -Learning Paradox. ED423533, 1998.
A Comparative Analysis of College Teaching Methods,” University of [45] Berry, Jr., L., “Collaborative Learning: A Program for Improving
Oregon, USA: Center for the Advanced Study of Educational Administra- the Retention of Minority Students, U.S.: Virginia, 1991-00-00,
tion, 1968. ED384323, 1991.

230 Journal of Engineering Education July 2004

 [46] Johnson, D., and R. Johnson, Cooperation and Competition, Theory [55] Woods, Donald R., http://www.chemeng.mcmaster.ca/pbl/
and Research, Edina, MN: Interaction Book Company, 1989. pbl.htm, accessed 12/3/03
[47] Terenzini, P., Cabrera, A., Colbeck, C., Parente, J., and Bjorklund [56] El-Nemr, M., “A Meta-Analysis of the Outcomes of Teaching
S., “Collaborative Learning vs. Lecture/Discussion: Students’ Reported Biology as Inquiry,” (Doctoral Dissertation, University of Colorado, 1979),
Learning Gains,” Journal of Engineering Education, Vol. 90, No. 1, 2001. Dissertation Abstracts International, 40, 5812A. (University Microfilms
[48] Panitz, T., “The Case for Student Centered Instruction via International No 80-11274), 1980.
Collaborative Learning Paradigms,” U.S. Massachusetts, 1999-12-00, ED [57] Lott, G., “The Effect of Inquiry Teaching and Advance Organiz-
448444, 1999. ers upon Student Outcomes in Science Education,” Journal of Research in
[49] http://www.udel.edu/pbl/others.html#undergrad, accessed Science Teaching, Vol. 20, 1983, p. 437.
1/16/04.
[50] http://pbl.cqu.edu.au/content/online_resources.htm,
accessed 1/16/04. AUTHOR’S BIOGRAPHY
[51] Norman, G., and H. Schmidt, “The Psychological Basis of Prob-
lem-Based Learning: A Review of Evidence,” Academic Medicine, Vol. 67, Dr. Michael Prince is a professor in the Department of Chemi-
1993, pp. 557–565. cal Engineering at Bucknell University, where he has been since re-
[52] Martensen, D., H. Eriksson, and M. Ingleman-Sundberg, “Med- ceiving his Ph.D. from the University of California at Berkeley in
ical Chemistry: Evaluation of Active and Problem-Oriented Teaching 1989. He is the author of several education-related papers for engi-
Methods,” Medical Education, Vol. 19, 1985, p. 34. neering faculty and gives faculty development workshops on active
[53] Gallagher, S., “Problem-Based Learning: Where did it comes learning. He is currently participating in Project Catalyst, an NSF-
from, what does it do and where is it going?,” Journal for Education of the funded initiative to help faculty re-envision their role in the learning
Gifted, Vol. 20, No. 4, Summer 1997, pp. 332–362. process.
[54] Major, C., and B. Palmer, “Assessing the Effectiveness of Prob- Address: Department of Chemical Engineering, Bucknell Uni-
lem-Based Learning in Higher Education: Lessons from the Literature,” versity, Lewisburg, PA 17837; telephone: 570-577-1781; e-mail:
Academic Exchange Quarterly, Vol. 5, No. 1, 2001, p. 4. [email protected]

July 2004 Journal of Engineering Education 231