science | technology | engineering | math

WHITE PAPER

Does the Use of Technology Improve Learning?

The Answer Lies in Design
by Atsusi Hirumi, Ph.D.
Associate Professor
University of Central Florida

As an Associate Professor of Instructional Design and Technology, Im often asked if certain

technologies enhance learning or to compare the relative effectiveness of two or more
technologies. Do [educational] video games increase student achievement and/or
motivation? Is Web-based distance education better than conventional face-to-face (f2f)
classroom instruction? How about virtual worlds and mobile devices? While I understand the
interest in such questions, particularly from administrators who may be responsible for
making high-cost purchasing decisions, I argue that asking whether a particular technology
is effective or if one technology is better than another for promoting learning is no longer
meaningful. Decades of media comparison studies have shown that the technology used to
deliver instruction has relatively little to no direct effect on student learning or motivation. To
be more precise, the design of the instruction accounts for more variance in how and why
people learn than the technology used to deliver the instruction. As such, I encourage
educators and educational researchers to focus on determining how to better integrate the
use of a given technology to facilitate learning, rather than asking if it works or if one is more
effective than another.
Educators and educational researchers have considered the effects of mediated instruction for
over a hundred years. In 1910, George Kliene published the first catalog of instructional films
for public schools and in 1913, Thomas Edison predicted that books would soon become
obsolete because, It is possible to teach every branch of human knowledge with the
motion picture. In 1920s, The National Academy of Visual Instruction was formed and
Dorris published one of the first textbooks on visual instruction, advocating the use of
seeing experiences such as pictures, models, exhibits, charts, maps and graphs to enrich
education. In the 1930s and 40s, the radio received considerable attention (e.g., Woelfel &
Tyler, 1945) and advances in sound and audio recording technology shifted focus from
visual to audiovisual instruction. In 1947, the Department of Visual Instruction (DVI) becomes
the Department of Audiovisual Instruction (DAVI) and McGraw Hill Book Company
manufactured its first textfilms, designed to supplement textbooks by expanding the visual
treatment of the subject matter. The 1950s saw the growth of television and a number of
studies compared televisions versus traditional classrooms as a means for delivering
Ins t r uct ional Technology

1/8

science | technology | engineering | math

education. In 1952, Alabama established the first statewide educational television network,
and in 1956, one the largest single experiments on the use of closed-circuit instructional
television was conducted in Hagertown, Maryland. Inspired by Hagertowns early success,
the Ford Foundation then began a nationwide study on the use of Interactive Television that
included over 200,000 students in over 800 elementary and secondary schools.
Although early studies on the use of instructional media did yield some interesting results
(e.g., younger students are more effectively taught by television than are their older peers;
simplified line drawings may be more effective than high quality photographs), for the most
part, researchers found that: (a) learning can take place through the use of audio, still
images, or motion pictures as well as one-way or two-way, interactive educational television
as the sole means of transmitting information; and (b) when compared with "traditional"
classroom techniques, audio recordings, still images, motion pictures and (either one or twoway) educational television programs of the same content produce no significant differences
in learning.
With the advent of the personal computer, computer-based tutorials, simulations and simple
drill and practice games became popular in the 1970s and 80s. Like earlier forms of
instructional media, computer-based instruction (CBI) and computer-based training (CBT)
were compared to traditional, face-to-face (f2f) classroom instruction. For example, Jamison,
Suppes and Wells (1974) conducted a comprehensive review of media comparison research
and found no significant differences in achievement were typically reported between the use
of computers, televisions and radios compared to traditional classroom instruction. While
some studies reported advantages for media, others reported greater achievement with
traditional instruction. They concluded, When highly stringent controls are imposed on a
study, the nature of the controls tends to force the methods of presentation into such similar
formats that one can only expect no significant differences which were found (p. 38).
Concentrating on studies comparing CBI and CBT to traditional classroom instruction, Kulik
and Kulik (1991, 1987) and Roblyer (1989) used meta-analytic techniques to synthesize and
analyze findings and to better estimate effect size. Although significant differences in final
exam scores were found in several cases, closer examination revealed that most of the large
effect sizes attributed to the use of computers were actually due to poorly designed studies
and other confounding factors (Clark, 1983). For instance, when studies involving only one
teacher in the planning and delivery of the instruction were considered, the positive effect of
media tended to disappear. The evidence in these meta-analyses.is that it is the method
of instruction rather than the choice of medium that leads directly and powerfully to
learning (Clark & Sugrue, 1994, p. 353). Such findings led Clark (1983) to coin the
popular phrase that, "[...] media are mere vehicles that deliver instruction but do not
influence student achievement any more than the truck that delivers our groceries causes
changes in our nutrition" (p. 445).
Reviews of research examining the relative effectiveness of educational video games have lead
to similar conclusions. For example, ONeil, Wainess, and Baker (2005) located over 4,000
articles about computer games published in peer-reviewed journals and found only 19
Ins t r uct ional Technology

2/8

science | technology | engineering | math

studies that included qualitative or quantitative data about learning or motivation. In

addition, their analysis of the learning and transfer measures used in all 19 studies
concluded, positive findings regarding the educational benefits of gamescan be
attributed to instructional design and not to games per se. Alsomany studies claiming
positive outcomes appear to be making unsupported claims for the media. (p. 461-462).
Similarly, in her dissertation research, Kebritchi (2008) noted that the majority of research
reviews on the effectiveness of computer games yielded either mixed results (Hays, 2005;
Mitchell & Savill-Smith, 2004; Randel, Morris, Wetzel, & Whitehill, 1992) or found no
positive effects (Emes, 1997; Harris, 2001). As with other forms of technology, in some
cases, the use of games resulted in slightly higher levels of achievement and/or motivation
compared to traditional classroom instruction while in other cases they did not. In addition,
Kebritichi, Hirumi and Bai (2010) found that what teachers did before and after game play
was as important, if not more so, to learning than the game itself, further substantiating the
importance of design.
As Internet access has become more pervasive, an increasing number of organizations are
offering web-based training and distance education programs. Many, in turn, have
questioned the relative effectiveness of on-line programs to traditional classroom instruction.
In one of the most extensive efforts to synthesize such research, Bernard, Abrami, Lou, and
Borokhovski (2004) analyzed 688 studies comparing classroom and distance learning
offerings of the same course and found no differences in either learning or motivation.
Focusing primarily on adults in business and college settings, Sitzmann, Kraiger, Stewart, and
Wisher (2006) reviewed 96 studies and found the same result. Apparently, web and
classroom-based instruction yield similar amounts of learning and are equally motivating.
While research indicates otherwise, Seels (2011) suggests that practice provides considerable
evidence that technology does influence learning by: (a) addressing students and teachers
media preferences; (b) interfacing content and learners; (c) creating a context for
communications; (d) offering alternative ways to produce, store, present, and distribute
information; (e) enhancing motivation, attention, perception and retention; and (f) providing
models, symbolic representations, interactivity, cueing, and vicarious experiences. Obviously,
different technologies offer different capabilities. It would be difficult for teachers to interact
with students from all over the world with a stand-alone computer and no Internet access.
Although media may simply be vehicles for delivering instruction, you wouldnt want to
transport sushi in a van with no refrigeration. But, the capabilities investigated so far (e.g.,
motion, sound, simulation, connectivity) are not exclusive to specific technologies. Each
capability may be provided by other technologies and no capability has been found to
produce unique cognitive effects (Clark & Sugrue, 1991).
So, how will the use of virtual, mixed and augmented realities compare to web-based and
traditional classroom instruction? Can social networking sites, such as Facebook, Second Life
and Bebo enhance learning? What about the advances in mobile technology and the
continued development of Web 2.0 technologies? With the advent of new and emerging
technology, a fresh crop of studies will inevitably appear either demonstrating the
Ins t r uct ional Technology

3/8

science | technology | engineering | math

instructional effectiveness of the technology or comparing the new with an older technology.
Each new technology seems to attract its own set of advocates who ask research questions
that seem similar to past media studies and claim their technology improves learning and/or
motivation. However, as Ive argued throughout this paper, its how the technology is used
and integrated with instruction that makes the difference, not the technology used to deliver
or others facilitate the instruction.
For many of us, the classroom was pretty much the same as when our parents went to
school. The teacher stood in front, acting as the epistemological center of authority, and
lectured to the groups of students who were expected to start and complete the same
curriculum at the same pace and at the same time. Parents and other members of the
community contributed to our learning, but typically, in an uncoordinated manner (as
depicted on the left side of Figure 1). Even with the infusion of personal computer in the
1970s, 80s and 90s, schools and more importantly, the educational experience remained
predominately teacher-centered until the late 1990s and early 2000s when the Internet and
high speed networks became more prevalent. The World Wide Web gave students direct
access to the knowledge base and enabled teachers to become learning facilitators, helping
students to access, interpret, organize, synthesize and apply information, rather than
directing students to memorize and regurgitate information for an upcoming test. Advances
in information technology also facilitated communications among students and between
family and community members as well as experts around the world, further breaking down
the barriers of traditional classroom walls and promoting the development of studentcentered learning environments (as depicted on the right side of Figure 1).

Ins t r uct ional Technology

4/8

science | technology | engineering | math

Figure 1. A comparison of teacher centered and student centered learning environments

(from Hirumi, 497-537.)

Now, Web 2.0 technologies, such as YouTube, wikis, and blogs, allow students to readily
create and share user-generated content, providing tools to further the concept of studentcentered learning. However, rather than questioning if Web 2.0 or other emerging
technologies promote learning or if one technology is better than another, we should
investigate how to best integrate these tools to facilitate student learning and motivation.
Current research and literature provide some initial insights. For instance, Shelly, Gunter and
Gunter (2010) suggest challenging students to establish community-based blogs to discuss
issues important to local government or to communicate with and assist senior citizens. Other
recommendations include, but are not limited to tasking students to produce and share
short digital video clips using YouTube to demonstrate their knowledge of a topic, and
having students create wikis to build communication and teamwork skills (Shelly, Gunter &
Gunter, 2010). In my opinion, questioning what educators can do before, during and after
the use of technology, and testing alternative strategies to facilitate learning are the keys to
improvement.
Over the past 100 years, literally thousands of studies have shown benefits from the use of
technology, while others report higher achievement with classroom instruction, and even
more result in no significant difference. I, like others, believe that such mixed results reveal
that the design of instruction makes the difference in learning, not the medium used to
deliver the instruction. The question is no longer if technology enhances learning, but rather
how do we improve our use of technology to enhance learning? So, the next time youre
Ins t r uct ional Technology

5/8

science | technology | engineering | math

asked if mixed, mobile, game, audio, video, virtual, distance learning, social media, Web,
Web 2.0 or other emerging technologies enhance learning, or if one is better than another,
you can say, the answer lies in design.

References
Bernard, R., Abrami, P. L., Lou, Y. & Borokhovski, E. (2004). How does distance education
compare with classroom instruction? A meta-analysis of the empirical literature. Review of
Educational Research, 74, 379439.
Clark, R. E. (1983). Reconsidering Research on Learning from Media, Review of Educational
Research, 53, 445-59.
Clark, R. E. & Sugrue, B. M. (1994). Research on Instructional Media, 1978-1988. In G.
Anglin (Ed.). Instructional Technology: Past, Present and Future (2nd Ed.) (pp. 348-364).
Englewood, CO: Libraries Unlimited.
Clark, R. E. & Sugrue, B. M. (1991). Anglin, G. J.. ed. Instructional technology: past,
present, and future (pp.327343). Englewood, Colorado: Libraries unlimited.
Clark, R. E., Yates, K., Early, S. & Moulton, K. (2010). An Analysis of the Failure of Electronic
Media and Discovery-based learning: Evidence for the performance benefits of Guided
Training Methods. In Silber, K. H. & Foshay, R. (Eds.). Handbook of Training and Improving
Workplace Performance, Volume I: Instructional Design and Training Delivery (pp. 263297). New York: John Wiley and Sons.
Emes, C. E. (1997). Is Mr Pac Man eating our children? A review of the impact of video
games on children. Canadian Journal of Psychiatry, 42(4), 409414.
Harris, J. (2001).The effects of computer games on young children A review of the
research. RDS Occasional Paper No. 72. London: Research, Development and Statistics
Directorate, Communications Development Unit, Home Office.
Hays, R.T. (2005). The effectiveness of instructional games: A literature review and discussion.
Naval Air Warfare Center Training System Division (No. 2005-004). Retrieved October 7,
2007 from
http://stinet.dtic.mil/oai/oai?&verb=getRecord&metadataPrefix=html&identifier=ADA441935.
Hirumi, A. (2002). Student-centered, technology-rich, learning environments (SCenTRLE):
Operationalizing constructivist approaches to teaching and learning. Journal for Technology
and Teacher Education, 10(4), 497-537.

Ins t r uct ional Technology

6/8

science | technology | engineering | math

Hoban, C. F., & Van Ormer, E. B. (1979). Instructional film research, 1918 to 1950. North
Stratford, NH: Ayer Company.
Jamison, D., Suppes, P., & Wells, S. (1974). The effectiveness of alternative instructional
media: A survey. Review of Educational Research, 44, 1-69.
Kebritchi, M. (2008). The effects of computer game on mathematics achievement and class
motivation: An Experimental Study. Unpublished doctoral dissertation, University of Central
Florida, Orlando, FL.
Kebritchi, M., & Hirumi, A., Bai, H. (2010). The effects of modern mathematics computer
games on mathematics achievement and class motivation, Computers & Education, 55(2),
427-443.
Kulik, C. C., & Kulik, J. A. (1991). Effectiveness of Computer-Based Instruction: An Updated
Analysis. Computers in Human Behavior, 7, 7594.
Kulik, J. A., & Kulik, C. C. (1987). Computer-Based Instruction: What 200 Evaluations Say.
Paper presented at the Annual Convention of the Association for Educational
Communications and Technology, Atlanta, GA.
Mitchell, A., & Savill-Smith, C. (2004). The use of computer games for learning.
Retrieved July 23, 2007 from
http://www.m-learning.org/archive/docs/Theuseofcomputerandvideogameforlearning.pdf.
ONeil, H. F., Wainess, R., & Baker, E. (2005). Classification of learning outcomes; Evidence
from the games literature. The Curriculum Journal, 16(4), 455-474.
Roblyer, M. D. (1989). The Impact of Microcomputer-Based Instruction on Teaching and
Learning: A Review of Recent Research. Syracuse, NY: ERIC Clearinghouse on Information
Resources.
Randel, J. M., Morris, B. A., Wetzel, C. D., & Whitehill, B. V. (1992). The effectiveness of
games for educational purposes: a review of recent research. Simulation and Gaming, 23(3),
261276.
Seels, B. (2011). From Dale to Delivery Systems: The Problem of Media Selection Theory. In G.
Anglin (Ed.). Instructional Technology, Past, Present and Future (3rd Ed.) (pp. 55-68). Santa
Barbara, CA: Libraries Unlimited.
Shelly, G. B., Gunter, G. A., & Gunter, R. E. (2010). Integrating Technology and Digital
Media in the Classroom (6 th Ed.). Boston, MA: Course Technology.
Sitzmann, T., Kraiger, K., Stewart, D., & Wisher, R. (2006). The comparative effectiveness of
web-based and classroom instruction: A meta-analysis. Personnel Psychology, 59, 623-664.
Ins t r uct ional Technology

7/8

science | technology | engineering | math

Woelfel, N. and Tyler, I. K. (1945). Radio and the school, a guidebook for teachers and
administrators. Tarrytown-on-Hudson, N.Y.: World Book.

Ins t r uct ional Technology

8/8