Chapter 4

Research Says…
Overview
• What are the results of studies about the effectiveness of games for
learning?
• Do all studies of the effectiveness of games for learning yield the
same results?
• What are the limitations of the research in defining the effectiveness
of games for learning?
• What game mechanics or attributes have been found to be
particularly effective for learning or particularly ineffective?
Introduction
• Review research works describing 1) the EFFECTIVENESS of games
and 2) the EFFECTIVENESS of specific game elements
1) Meta-Analysis Research Studies: a meta-analysis is a study of findings based
on the aggregations of finding from many other studies. The results were
found through the study of studies on the effectiveness of games.

2) Elements of Games: examines individual research studies that examine

specific elements of games such as reward structures, use of avatars, and
player’s perspective and how they relate to elements of games.
Game Research
• Theoretical underpinnings
• Opinion
• Wishful thinking
Randel’s Meta-Analysis
• Conducted a review of games and simulations for learning in 1992
• Examined 68 studies spanning a time from of 28 years up until 1991
• Covered social sciences, math, language arts, logic, physics, and
biology, Not corporate related or business games or simulations
• 38 (56%) no difference
• 22 (32%) favoring simulation/games in student performance
• 5 (7%) favored simulation/games but the controls were questionable
• 3 (5%) in favor of more traditional instruction
Wolfe’s Meta-Analysis
• Only computer-based business games
• 7 studies conducted between 1966 and 1988
• Treatment and control group
• Criteria
• Comparing game use with at least one other instructional approach
• Having predefined, objectively measured instructional objectives
• Having objectively measured learning outcomes.
• ?Every study examined indicated that the game-based approach
produced significant knowledge-level increases over the conventional
case-based teaching methods
Hays’ Meta-Analysis
Conclusions:
1.The empirical research on the effectiveness of instructional games is fragmented.
2.This does not tell us whether to use a game for our specific instructional task.
3.There is no evidence to indicate that games are the preferred instructional
method in all situations.
4.Instructional games should be embedded in instructional programs that include
debriefing and feedback so the learners understand what happened in the game
and how these events support the instructional objectives.
5.Instructional support to help learners understand how to use the game increases
the instructional effectiveness of the gaming experience by allowing learner to
focus on the instructional information rather than the requirements of the game.
Hays’ Meta-Analysis Cont.
• Some other interesting findings:
• people can learn from games
• Will only be effective if it is designed to meet specific instructional objectives
and used as it was intended.
• If the game is not designed to directly support specific instructional objectives
related to actual job requirements, much of the learning may be irrelevant.
• Many program managers and game developer do not appreciate the
importance of instructional design.
• Games are not a panacea.
Hays’ Meta-Analysis Cont.
• Hays’ suggestions:
• The decision to use a game should be based on a detailed analysis of the learning
requirements and an analysis of the tradeoffs among alternate instructional
approaches.
• Program managers and procurement personnel should insist that game developers
clearly demonstrate how the design of a game will provide interactive experiences
that support properly designed instructional objectives.
• Instructors should view instructional games as adjuncts and aids to help support
instructional objectives. Learners should be provided with debriefing and feedback
that clearly explains how their experiences with the game help them meet these
instructional objectives.
• Instructor-less approaches must include all “instructor functions”. These include
performance evaluation, debriefing, and feedback.
Vogel’s Meta-Analysis
• This study was conducted by a team of researchers at the University of
Central Florida.

• 32/248 sufficient quality studies were used for final analysis.

• The team found strong, positive effect sizes of interactive simulations and
games in comparison to traditional teaching methods in two areas:
cognitive gains and attitude.

• However, a working or operational definition of “cognitive gains” or

“attitude” was not provided in the reported study.
Ke’s Qualitative Meta-Analysis
• Conducted a review of 89 research articles.
• The effects of computer-based games on learning are positive. (52%
of the studies examined)
• But! Instructional support features are a necessary part of
instructional computer games and when support is present the
studies indicate significant results.
• Instructional computer games seem to facilitate motivation across
different learner groups and learning situations.
Sitzmann’s Meta-Analysis
• In 1997, DOD developed department-wide strategy called the Advanced
Distributed Learning (ADL)
• 55 research reports were examined, 6476 trainees
• Trainees’ gain higher confidence (20% higher)
• Trainees participating in simulation game learning experiences have higher
declarative knowledge (11% higher), procedural knowledge (14% higher)
and retention of training material (9% higher)
• Simulation games don’t have to be entertaining to be educational.
• Trainees learn more from simulation games that actively engage trainees ,
when they can utilize the simulation game as many times as desired
Elements of Games
• Reward Structures
• Player Motivation
• Avatars
• Player Perspective
Reward Structures
• The value, or size of an anticipated reward influences the
motivational signal sent to the brain only within the context of the
reward system.
• The uncertainty of an outcome influences the brain’s response to
reward.
• Gaming uncertainty can transform the emotional experience of
learning improving engagement and, more importantly, improving
encoding and later recall
Player Motivation
• Engagement-contingent rewards
• Completion-contingent rewards
• Performance-contingent rewards
• The study concluded that when organizations opt for the use of
reward to control behavior, the rewards are likely to be accompanied
by greater surveillance, evaluation, and competition, all of which have
also been found to undermine intrinsic motivation.
• Are grades intrinsic or extrinsic?
• Can we design a motivation, which is initially extrinsic, but become to
intrinsic over time.
Avatar
• Change a person’s real-life perceptions
• Watching an avatar that looks like you performing an activity
influences you to perform a similar or same activity in the future.
• Watching an avatar that resembles oneself changing in some way
impacts one’s future decisions.
• Behavioral changes occurring in a virtual environment can transfer to
physical environment.
• Better to have one “expert” and another “motivational” avatar in a
learning environment
Player Perspective
• Research found that a person is more likely to adjust his or her self-
concept to match a desired behavior if that behavior is imagined from
a third-person observer’s perspective
• Game can provide this