GAMES AND

GAMIFICATION
LITERATURE REVIEW

CENTER FOR
ADVANCED
HINDSIGHT

Duke University
Center for Advanced Hindsight
334 Blackwell St,
Durham, NC 27701
Advanced-hindsight.com/
1
 This literature review was created by the global team at the Center for Advanced
Hindsight:

Jan Willem Lindemans (contacting author: [email protected]) with Nina

Bartmann, Judson Bonick, Samantha Garland, Riya Gilja, Will Hopper, Ting Jiang,
Rebecca Kelley, Katya Kuzi, Ciara Lutz, Alex Moog, Andrew Schneider and Alec Wall.

We acknowledge and appreciate the generous gift from Joep Lange Institute that has
been crucial in making this work possible.

Interested in collaborating with the Global team, contact: Ting Jiang, Principal, Global
Health and Development: [email protected]

2
INTRODUCTION: SOME CONCEPTS 3

GAMIFICATION OF LEARNING 6

EDUCATIONAL GAMES 6

GAMIFICATION OF CONTINUOUS LEARNING 8

WHEN EXACTLY DOES IT WORK? 9

GAMIFICATION OF OTHER ACTIVITIES 13

GAMIFICATION OFTEN IMPROVES BEHAVIORS 13

THE EXTENT OF GAMIFICATION 19

LIMITS TO GAMIFICATION 21

INDIVIDUAL DIFFERENCES 25

EXPERIMENTAL GAMES 28

REFLECTION 30

REFERENCES 31

3
 INTRODUCTION:
SOME CONCEPTS
Interventions in health and personal finance increasingly make
use of games and gamifications. By now there is a substantial
literature on the impact of games and gamification. Before
discussing that literature, let us take a moment to review what a
game is, and what gamification is. This somewhat tedious exercise
is necessary, because gamification is a buzzword used in many
different ways, some of which have little to do with games.

game
/geym/
According to the Oxford Dictionary, a game in the broad sense is “an activity
that one engages in for amusement or fun”. In that broad sense, “game” is
synonymous with “play”. More specifically, the Oxford Dictionary defines a game
as “a form of competitive activity or sport played according to rules”.

In the same spirit, the historical Oxford English Dictionary defines a game
as “an activity or diversion of the nature of or having the form of a contest or
competition, governed by rules of play, according to which victory or success
may be achieved through skill, strength, or good luck”.

From this, we derive a three-part working definition:

A game is (1) an activity engaged in for fun (play), (2) governed by rules, (3) in which
player(s) try to win. Let’s discuss these three characteristics.

• First, the game is fun to play, and people play it because it’s fun. People desire
to win the game, and the excitement caused by the uncertainty of the outcome
is pleasurable. Winning is fun, and losing can be a bit frustrating, but the
excitement about the possibility of losing is part of the fun.
- Many games also tap into certain emotions that in real life are negative:
feeling nervous, anxious, scared, dizzy, etc. In real life, we don’t like these
emotions. But in games, like in movies, when we know that there is no
possible harm, we can enjoy these emotions. For instance, in real life, the
nervousness we feel when seeing a clock tick down towards a deadline can
be extremely unpleasant. In games, it is fun.
• Second, there is a clear set of rules of the game, and a clear structure. There are
rules about what is possible and impossible in the game. Behaviors in the game
often have clear consequences - clearer than in real life.
• Third, there are rules about the goal of the game - players can win the game or
lose it (binary) in a certain way, or at least perform better or worse (continuous) in
the game. Players may win something tangible, like money or some other prize,
or something intangible like a title or honor.

4
OTHER TYPICAL CHARACTERISTICS OF GAMES. In addition to these three central
ingredients, here are some other characteristics that are often present in games.

• Games are often a simplified simulation of a real event or situation. Many games
have a story behind them, but games are normally not real themselves. Instead,
they have little real-life consequences, and hence they are just play.

• Games usually have a specific location (e.g., a game board, a field, etc.), and a
limited duration of play.

• Players may compete with other players, or just play “against the game” - on their
own, or collaborating with other players.

• Many games require some skill, be it physical or mental, and playing the game
makes you more skilled at it - sometimes causing excellent players to be in a very
pleasant state of flow.

• Many games also have an element of chance, and it is uncertain who is going to
win. If it is obvious too soon who will win or lose, the rest of the game is no fun.

• Many games are made for children, but even the games made for adults often
still make use of some imagery that appeals to children (like cartoon figures,
fantasy, bright colors, etc.). Many games also make use of humor to make it more
fun.

A game is an activity that has a substantial number of the above typical characteristics,
although it is hard to say which characteristics really need to be present for the
activity to be a game. Note that all these characteristics are possible candidates for
experimentation and manipulation. We will see later that there is surprisingly little
research that surgically manipulates individual characteristics in games.

gamification
/gey-muh-fi-key-shuh n/
The Oxford Dictionary defines gamification as “the application of typical
elements of game playing (e.g. point scoring, competition with others, rules of
play) to other areas of activity, typically as an online marketing technique to
encourage engagement with a product or service”.

SOFT GAMIFICATION VERSUS HARD GAMIFICATION. Note that this definition does
not say that gamification is the transformation of an activity into a full-fledged
game. It says gamification is the application of some game-like characteristics
to an activity, like points or competition. It is making an activity a bit more like
a game. And this is indeed how gamification has been most widely conceived
and implemented. We will call the application of some game-like elements to
an activity soft gamification. By contrast, we call the transformation of an activity
into a full-fledged game hard gamification. Hard gamification is much rarer than
soft gamification, but it exists. Educational games are a good example of hard
gamifications of learning. And perhaps more suprisingly, sports are a great example
of hard gamifications of exercise. Soccer is gamified running. Hard gamification is
very difficult, because it is difficult to turn a tedious activity into something people
really engage in purely because they enjoy it. When friends get together Sunday

5
 afternoon and the question is raised what game they want to play, no-one will ever
suggest to collect points for a rewards programs and to try to achieve Gold status.
Rewards programs are soft gamifications, and no-one ever really engages in it purely
because it is fun. You couldn’t sell it in a game store. But people do play soccer
because it is fun—not just to exercise. And children do play Little Professor because
it is fun—at least, some children do. Hard gamification is hard, but its benefits can be
immense. When hard gamification works, more research should be geared towards it.

Flatla et al. (2011) provide an instructive process for how to gamify a task. The
process involves first identifying the core task or tasks that participants need to
accomplish, and then carefully selecting game elements that can be added.

Building on past literature, the researchers identify four basic elements of games.

1. The first is that it is challenging: games provide challenging goal elements tied to
rewards, e.g., collecting items, or shooting objects.
2. Secondly, games have a theme: they include vicarious aesthetic representation
and unifying aesthetic elements, e.g., cloaking the context or player
representation in a fantasy or with evocative mental imagery, such as alternate
worlds or player avatars.
3. Third, games provide elements of reward to serve as behavior reinforcement, e.g.,
simple visual representations or pleasant sounds, like in the game Candy Crush.
4. Lastly, games show us when we have made progress through specific units and
markers, e.g., levels, worlds, quests, and achievement markers, such as badges
and score information.

In what follows, we first discuss the gamification of learning, followed by a

discussion of gamification of other activities, like exercise, saving, etc.

6
GAMIFICATION
OF LEARNING
EDUCATIONAL GAMES
Games have been widely used to teach knowledge and/or skills that will be useful
in the real world, after the game is played. Such games are commonly called
educational games. We distinguish between one-time educational games (this
section) and other types of gamification of continuous learning (next section).

GAMES CAN IMPROVE KNOWLEDGE AND

BEHAVIORS. Educational games can have
an impact on health-related behaviors. For
instance, computer games can be an effective
way to teach valuable safety tips to disabled
children. Coles, Strickland, Padgett, and
Bellmoff (2007) presented 32 children aged
4-10 who had been diagnosed with fetal alcohol
syndrome (FAS) or partial FAS with a game to
teach them either fire or street safety skills. The Research shows
game took place in a virtual world in which an that educational
animated dog demonstrated the correct actions games can increase
in the relevant settings (crossing the street, or
knowledge and
escaping a house fire). Children then had the
chance to navigate through these situations improve behaviors,
on their own, and the dog positively reinforced compared to
the children’s behavior when they chose the more traditional
correct action, or told them to try again if they
education.
chose incorrectly. Children were first pre-tested
on their knowledge of fire and street safety and
post-tested both immediately after they had
achieved “mastery,” (i.e., were able to navigate
through the correct actions in the game without the help of the avatar) and one
week later. The children were also asked to demonstrate the correct sequence of
both behaviors in real life.

Immediately after playing the street safety game, those who had played the
fire safety game demonstrated significantly increased knowledge of fire safety,
and those who had played the street safety game demonstrated significantly
increased knowledge of street safety. Knowledge of fire safety one week later was
still significantly higher compared to fire safety knowledge in the initial pretest,
but knowledge of street safety was only marginally improved. This knowledge
also translated into actual performance, as the vast majority of children correctly
performed 3-4 out of 4 of the desired behaviors related to fire safety both
immediately following game mastery (87.5%) and one week later (81.3%). Most
children also correctly performed 3-4 of the desired behaviors related to street

7
 safety immediately following the game (81.3%) and one week later (75.4%). This
gamified intervention thus had somewhat lasting effects.

Games have also improved language learning. Calvo-Ferrer (2017) examined the
effect of using computer games as stand-alone learning tools on learning gains
and found that game increased vocabulary acquisition significantly more than
a stand-alone booklet containing the same information. Fifty-nine English as a
Second Language students of Spanish nationality (ages 19-20) participated in
the study. These students were divided into two groups - one group who played
an educational game called The Conference Interpreter, and one group which
received a booklet with the same information contained in the game, but without
the rewards (such as digital ribbons and level-ups), score tables, and power-ups
of the game. The experiment was carried out over five days. On the first day, the
students were given pre-test questionnaires and assessments, as well as assigned
to the different treatment groups. In the next three days, the students individually
used either the computer game or booklet in 2-hour sessions. On the fifth day, the
students were given post-test questionnaires and assessments.

An analysis on the assessment scores of the students post-intervention found that

students who utilized the computer game scored significantly higher (M=35.56 out
of 63) than those given the booklet (M=31.63 out of 63). The author concluded that
The Conference Interpreter game helped students gain vocabulary more effectively
than information alone, and that this type of intervention could be a powerful tool
since it does not require any instructional support.

Figure 1. The Conference Interpreter

8
GAMES CAN REDUCE COGNITIVE BIASES. Morewedge et al. (2015) shows that it
is possible to create computer games of debiasing training to reduce cognitive
biases. In both experiments, participants were either assigned to watch a 30-minute
instructional video, or to play a 30-minute debiasing video game. The instructional
video defined heuristics, described what each bias was, and showed vignettes
of actors demonstrating each bias, while the video game task was to simply find
a missing neighbor. Within the video game, participants were presented with
situations in which they might commit each cognitive bias. At the end of each
level, the game described each bias in detail and had participants rate how biased
they were during the game. Participants were immediately given feedback about
their level of bias. In experiment 1, the researchers sought to decrease the effects
of a) bias blind spot, b) confirmation bias, and c) fundamental attribution error.
In experiment 2, the researchers were trying to decrease the effects of anchoring,
projection, and representativeness heuristics.

In both experiments, the authors found reductions in cognitive biases that are
critical to intelligence analysis. Both produced medium to large debiasing effects
immediately (31.94% reduction in the video game condition; 18.60% reduction
in the instructional game condition) that persisted even 2 months later (23.57%
reduction vs. 19.20% reduction).

GAMIFICATION OF CONTINUOUS LEARNING

GAMIFICATION OF LEARNING. As with one-time educational games, the gamification
of continuous learning may effectively increase learning. Nevin et al. (2014) conducted
a study on gamification-based software for learning medical information on medical
residents that showed high levels of acceptance, use of software, and retention of
knowledge. Note that this study did not have a control with non-gamified learning, so
we cannot conclude that the gamification is what worked.

Students used the software to complete questions individually, and in a group, via
anonymous usernames. Three rounds of questions on a variety of topics were picked
by the faculty. Round 1 was open to 128 students at the University of Alabama
Birmingham only. For round 2, 24 University of Alabama Huntsville residents were
also invited, and round 3 was open to all students. Round 3 consisted of 3 daily
questions – 2 new ones and one repeat from round 1 or 2. Points were given for
correct answers, continued daily participation and streaks of correct answers. The
leaderboard was updated continuously, and participants could see the scores of
other participants whose scores were just above and below their own.

Focus groups conducted after round 1 determined that the leaderboard was the
most important motivator, with people or teams at the bottom of the leaderboard
experiencing less motivation as the competition progressed. The number of
participants decreased after each round, yet there was a statistically significant
increase in use of the program after weekly competition status emails were sent.
28.8% of questions were answered on weekends between 17:00 and 08:00 (outside
normal teaching hours). Players were correct 11.9% more frequently when a
question was repeated.

The limitations were that internal medicine was the only speciality tested and only
residents who volunteered to participate were included.

9
 WHEN EXACTLY DOES IT WORK?
LEARNING INTRINSIC TO THE GAME. Making educational elements intrinsic to an
educational game improves learning. Habgood and Ainsworth (2011) found that
children improved their mathematical abilities more when given an educational
game when the math education in the game was intrinsic to the game itself. The
authors define intrinsically integrated games as (a) delivering learning material
through the parts of the game that are most fun to play, riding on the back of
the flow experience produced by the game, and not interrupting or diminishing
its impact, and (b) embedding the learning material within the structure of
the gaming world and the player’s interactions with it, providing an external
representation of the learning content that is explored through the core mechanics
of the gameplay.

The authors created a video game called “Zombie Division” specifically for this
study. This game involves a fantasy context where the main character battles
skeletons with symbols on their torso, where the symbols indicated what kinds of
attacks would be effective in slaying them. There are three versions of this game,
representing the three conditions of the study. First was the intrinsic condition,
where the symbols on the skeletons were numbers and the attacks to slay them
had corresponding number values. A skeleton could only be slain by an attack that
could divide into the number on their torso (a #9 skeleton could only be slain by
a #3 attack). The skeletons in the extrinsic and control conditions had symbols on
their chest corresponding to the attacks that would slay them, but the attacks were
non-numeric such as a sword or shield, thereby having the same exact gameplay
in all conditions but hiding the mathematical aspects in the extrinsic and control
conditions. In the extrinsic condition, the mathematical content was delivered in
a quiz at the end of each level of the game. In the control condition, there was no
mathematical content in the game at all. All 58 of the children in the study also
received a teacher-led reflection session about mathematics after the children had
become familiar with the game, but had not spent a large amount of time with it.
Participants completed the mathematics outcome test as a pretest 10 days before
the study started, as a post-test after the children had spent 100 minutes with the
game, and as a delayed test given after the children’s final playing session.

Figure 2. (from Habgood & Ainsworth, 2011): “Explanations of mathematical learning

content in the intrinsic (left) and extrinsic (right) versions of Zombie Division.
Comparable features are labeled”

10
Children who played the intrinsic version of the game not only maintained their
mathematical abilities, but improved further on the following delayed test (50% to
60%). The extrinsic and control children, on the other hand, only maintained their
previous levels of mathematical skill (at 39%, and 36% respectively).

Figure 3. (from Habgood & Ainsworth, 2011): “Mean Percentage Score by Time and
Game Condition”

GAMES NEED TO BE ALIGNED WITH LEARNING GOALS. de-Marcos et al. (2016)

compared several game-like approaches to learning: full educational games,
gamification only, social networking only, and social gamification. They found that
while all approaches significantly impacted learning, social gamification was the
most effective. However, depending on the type or the time of the assessment,
some of the approaches performed significantly worse than the control group.

The authors recruited 379 undergraduate students attending a course on basic

computer literacy (the MS Office applications Word, Excel, Powerpoint and Access),
who were divided into five conditions, which were given the following treatments: a)
the control condition, in which participants received standard blended learning, b)
the educational game condition, in which participants played the Microsoft game
Ribbonhero (this ready-made game not directly aligned with the course’s learning
objectives, teaches Office functions by helping an avatar to solve challenges), c)
the gamification condition, in which learning objectives on the online learning
platform were described as challenges, trophies were awarded, and a leaderboard
kept track of people’s performance, d) the social condition, in which students could
communicate on a social network by posting blogs, answering Q&A’s, or posting
links to resources as well as liking and friending each other, or e) the gamified social
condition. The gamified social condition was the same as the social condition,
but with gamification rewards for social interactions, such as points for answering
questions or participating in peer grading, and a leaderboard; points could also be
used to buy items such as +5% on the final mark or avatar improvements.

11
 In all conditions, including the control, participants had one lecture every two weeks
and had to study independently in between. There was a pre-test before treatments
began, an assignment at the end of every two-week module testing practical
knowledge of the module, and one final assignment at the end of the course testing
conceptual knowledge of all four modules.

In the practical assignments, all treatment conditions outperformed the control

group. In the final conceptual exam, however, only the gamified social treatment
group achieves results similar to the control group, with all other groups performing
worse (mean grades: control 74.77, educational game 66.30, gamification 59.50,
social 61.24, social gamification 72.00). The researchers hypothesize that this may
be due to “shallow gamification,” particularly for a learning game that is not aligned
with learning objectives, and to gamification leading to an increase in the number
of interactions with the learning material, but not in their quality. Unfortunately,
their research design does not allow a conclusion on whether the type or the time
of assessment are responsible for the worse performance of most approaches on
the final exam.

For practical gamification applications, this should serve as a warning not to have
the gamification elements overshadow the actual learning content—in other words,
they should motivate students to engage with the learning content, rather than
distract them from it.

DOES GAMIFICATION WORK BETTER FOR PRACTICAL SKILLS THAN FOR

THEORETICAL KNOWLEDGE? Dominguez et al. (2013) show that students with
the opportunity to complete a gamified online university course performed
better on practical tasks compared to students who completed a non-gamified
version, although there was no difference in final grades. The results also show that
gamification has the potential to increase student motivation.

The researchers studied student motivation toward completing optional exercises

using the online course “Qualification for users of ICT”, which was a 15-week
course in the 2011/2012 spring semester that teaches various Information and
Communications Technology (ICT) tools. Students could sign up for the same course
on either of two campuses. One group of 80 students was randomly assigned to
complete the regular, non-gamified online course, which offers downloadable
optional exercises to prepare for the final exam, whereas the other group of 131
students completed the gamified course, in which the optional exercises to prepare
for the final exam were available online in a gamified version. The video-game-like
exercises had three unique design features. First, students could complete two
challenges per topic, and within each challenge, they could complete four tasks
that were each rewarded with a trophy (copper, silver, gold and platinum). Second,
by successfully completing tasks, students were awarded medals. Students knew
that some medals would be awarded upon completion of a known task, whereas
some medals were awarded by surprise when a special task was completed. Third,
the authors included an element of competition by introducing a leaderboard
that ranks players according to their number of medals earned. Students in the
treatment group, however, were free to choose whether to use the gamified version,
the paper version or a combined version for their optional exercises. Fifty-eight
students decided to use the gamified version of the course.

12
Looking at the four core modules of the course, students in the treatment group
performed better in practical exercises about spreadsheets (11.24 points more on
a 100 point scale), software presentation (25.27 points more) and databases (29.4
points more). However, there was no difference in students’ performance on word
processing and in students’ final grade for the course.

13
 GAMIFICATION OF
OTHER ACTIVITIES
GAMIFICATION OFTEN IMPROVES
BEHAVIORS
GAMIFICATION OF WALKING (STEPS GOALS). In an
intervention aimed at increasing physical activity, Patel
et al. (2017) found that participants in the gamified
study arm achieved their step goals for a significantly
Studies have shown greater proportion of days as compared to the non-
gamified control arm. This was also true in the months
that gamifying after the intervention. All 200 participants had at least
unpleasant activities, one family member participating with them in the
like exercising and study and used either a smartphone app or a wearable
saving, can make device (e.g., Fitbit) to track step counts. During a two-
week run-up period, participants established a baseline
people participate in step count and then chose a step goal increase of 33%,
these activities more. 40%, 50% or any goal of at least 1,000 steps greater
than baseline. All participants received daily feedback
via text and/or email that informed them whether they
had achieved their step goal on the previous day.

Participants in the gamified study arm signed an online pledge committing to

try their best to achieve their daily step goal and were also entered into a 12-week
gamified intervention with their family. Each Monday, the family received 70 points
(10 points for each day of the week) and each day of the week a family member was
selected at random to represent the family’s step count. If that member achieved
his/her step goal, the family kept its 10 points. If not, the family lost 10 points for that
day. All families started out at a bronze level and had the opportunity to advance
up a level each week (bronze, silver, gold, and platinum). They advanced up a level if
they maintained 50 points or more by the end of the week. If the family was at the
gold or platinum level by the end of the intervention period, they received a coffee
mug branded with the study logo. Lastly, all family members had five lifelines that
they could use on days when it was not feasible to achieve their steps goal.

The mean proportion of days that participants in the gamified study arm achieved
their steps goal during the 12-week intervention period was 0.53 as compared to
0.32 in the control arm. During the 12-week follow-up period where steps were
counted after the end of the intervention, this trend continued. During the fol-
low-up period, the mean proportion of days that participants in the gamified study
arm achieved their steps goal was 0.44 as compared to 0.33 in the control arm.
These results indicate that gamified elements have the potential to not only in-
crease physical activity in the short-term but can also create sustainable habits of
increased physical activity.

14
GAMIFICATION OF OTHER HEALTH BEHAVIORS. Adding a gamification element
to blood glucose monitoring has also helped diabetic children keep their blood
glucose levels down. Kumar, Wentzel, Mikkelsen, Pentland, and Laffel (2004)
conducted a 4-week randomized clinical trial with 40 children aged 8-18 with type
1 diabetes (one participant had type 2) who were regularly treated with insulin
and already regularly monitoring blood glucose levels. Each participant received a
PDA, as well as diabetes software and equipment. Participants were encouraged to
check their blood glucose at least 4 times a day, and to upload their blood glucose,
carbohydrate intake, and insulin dose data into the PDA. This data could be sent
wirelessly to the researchers. Half (19) of the children were also given a game in
which they could see the first 3 blood glucose levels they recorded that day, and
were instructed to guess what the 4th reading would be. They were awarded points
for simply playing the game, and even more points if their blood glucose prediction
was accurate. There was no element of competition, as the participants played the
game against themselves.

The treatment group sent in their data significantly more often (1,662 values) com-
pared to the control group (1,471 values, p < .001). During the experiment, the treat-
ment group also experienced high blood sugar significantly less often (318 times)
than the control group (377 times, p < .001). The children in the treatment group also
exhibited greater knowledge of diabetes at the conclusion of the experiment com-
pared to the beginning (p < .005). This difference was marginally significant in the
control group (p < .09), but the researchers did not directly compare the two groups’
diabetes knowledge.

GAMIFICATION OF PERSONAL FINANCE. In this paper, Liu et al. (2017) describe the
design process and prototype for a savings app that involves gamification as well
as other behavioral interventions. After review, an unstated claim from Liu et al.
(2017) seems to be that to maximize success of the product, multiple interventions
need to be leveraged. The product leveraged interventions across the target user
journey of planning, saving, and keeping. Both of the gamified features in the app
were found in the savings section: (a) Personal savings challenges + gamification: a
feature allowing the user to set personal saving challenges against him/herself (or
a computer). Based on data analytics, the app automatically suggested potential
challenges and amounts to the users and sent prompts on their progress, and(b)
Social Influence + gamification: a platform to compete with peers or family
members on the pre-selected savings challenges (like a FitBit for savings).

The team tested the prototype through 50 behavioral interviews in the field, but the
paper includes a design for a much more robust RCT to test specific aspects of the
app. The paper outlines technical risks, business risks, and behavioral risks to the
product, as well as a potential future business model.

15
 Figure 4. Personal Finance Interventions (Liu et al., 2017)

Figure 5. User Response to Product Features (Liu et al., 2011)

Figure 6. Liu et al.’s (2017) Intervention

16
Figure 7. Liu et al. (2017) gamification image 1

Figure 8. Liu et al. (2017) gamification image 2

GAMIFICATION OF SIMPLE REPETITIVE TASKS. Adding simple gamified elements

can increase the motivation to complete even the most tedious tasks, such as
calibration tasks that require hundreds of repetitive trials. Flatla et al. (2011)
modified three different calibration tasks that are intended to train and inform
computer-interface models of human perception. One example of these types
of tasks is to provide participants many different combinations of colors and
the participant determines if the two colors are different or the same (see figure
9 below). The ability to differentiate color can vary between individuals due to
factors like age, fatigue, and environmental lighting. A task such as this is used to
determine the best color choices for information visualization. The researchers
redesigned each calibration task with several gamified elements (see figure 10).

In comparing the two versions with 36 university students (12 per task), the research-
ers found two main results. First, using a 5-point likert scale, participants reported
that the gamified tasks were significantly more enjoyable (averaging 4.0 for the
first task) and fun (averaging 4.0) than the traditional tasks (averaging 2.8 and 2.9,
respectively). Second, the gamified tasks accrued significantly different calibration

17
 data than the traditional task though this does not mean that gamified data is
necessarily less accurate. In fact, the researchers argue that the gamified calibration
data might be relatively more accurate given that the gamified version was more
enjoyable and thus participants may have been trying harder.

Figure 9. Color differentiation calibration task. Users respond to presentations of

colored circles with ‘same’ or ‘different’ response. Approximately 1,920 presentations
are used to calibrate the model.

Figure 10 Gamified version of a color differentiation task.

In one gamified version of the color differentiation task, users move a spaceship
horizontally across 20 fixed columns and shoot at dots when they are visible. Users
try to prevent the dots from falling to the bottom. By altering the background and
target colors, the “same” or “different” core task is effectively replicated, but the task
becomes implicit to users (i.e. their task is to “shoot” when visible, and not report
difference). The game also provides the user a progress bar, “worlds” and “levels” that
represent different sets of trials, an accuracy score, a high score, and new “items,”
such as new missile representation and additional target explosion sound effects,
for attaining high accuracy.

18
GAMIFICATION IN BUSINESS. Robson et al. (2015) discuss how gamification can aid
customer and employee engagement, and delineate between four different types of
customers and employees who act as ‘players’ in gamified experiences. They include
four different illustrative examples of how gamification has been used successfully
and unsuccessfully by different companies and conclude by presenting five lessons
for managers who wish to utilize gamification. Those lessons are: a) understand your
players before deciding on gamification mechanics, b) timing of rewards is key, c)
add new levels, tasks, or players as needed, d) managers must act as referees, and e)
use gamification mechanics to keep track of the score.

The article then concludes with some remarks on the dearth of research around
gamification and areas for possible future research, such as the ethics of consent
and implicit employee consent required for using gamification to engage and
motivate employees.

Figure 11. Robson et al. (2015) player types

Figure 12. Robson et al. (2015) gamification mechanics

19
 THE EXTENT OF GAMIFICATION
MORE GAMIFICATION IS BETTER. Cechanowicz et al. (2013) showed that the effect
of different gamification elements can be additive. Adding more elements can lead
to significantly more participation by users, yet specific elements may also influence
data quality. In their experiment, they divided roughly 600 participants of an online
marketing and branding survey into three groups: a plain survey, a partially gamified
condition adding interactive elements, and a fully gamified condition further
adding rewards, challenges, progress elements, etc. Each of the groups received
questions of the same three types: image identification, where they guessed the
company behind an advertising image, slogan matching, where they connected
companies to their respective slogans, and a memory quiz, in which they wrote
free-form responses to questions testing their recollection of certain elements after
watching a five second excerpt of an ad.

Figure 13. (from Cechanowicz et al., 2013) “Full Game (top row). Partial Game
(middle), and Survey (bottom) versions of the Image Identification (left column),
Slogan Matching (centre), and Five-Second Quiz (right) question types.”

20
Participants were free to end the survey after any question, making the number
of questions answered a meaningful metric for their motivation. On average,
participants in the fully-gamified condition answered about twice as many
questions as those in the plain survey group (21 vs. 12, respectively), with the
partially gamified condition falling somewhere in between. Notably, the researchers
found no differences in the number of questions answered, nor in the impact of
gamification by gender, age group, or between gamers and non-gamers.

Answer quality was similar between the conditions, but for two of the tasks, specific
gamification elements affected the quality of responses: for image identification,
the fully gamified condition performed better than others. This is likely because
they were shown the correct answer after giving wrong answers and thus benefitted
from a training effect the other conditions didn’t have. For the free-form answers
to the memory quiz, the researchers used answer length as a proxy for quality and
found that answers in the fully gamified condition – the only one that included a
timer – were slightly, but significantly shorter (by 1.8 characters) than those in the
plain survey condition. As with other studies, this shows that gamification can be a
useful motivator, but its potential effects on qualitative outcomes have to be taken
into account in the design stage.

FRAMING AN ACTIVITY AS A GAME IS SOMETIMES ENOUGH. Although the study

above found that more gamification is better in the right context, some research
fails to find a difference between merely framing an activity as a game and truly
gamifying that activity. Even just framing a serious activity as a game without
including actual gamification mechanics increases motivation. Lieberoth (2014)
found that using game-like materials was just as good for interest and enjoyment of
a collegiate feedback activity as adding game mechanics. Student volunteers from
third year psychology classes in college were assigned to one of three conditions.
Both the framing and full game conditions involved a game board and pawns, that
represented players, to move around the board. The full game condition additionally
included a competitive component that was not present in the framing condition.
The control participants were given sheets of paper leading them through the
same core task as the framing and full game participants. Subjects were tasked
with providing feedback on the psychology department. In the framing condition,
each player moved one space on the game board at a time and discussed prompts
related to the evaluation of the department. After this, all of the players rated the
feedback of each player with a rating from 1 to 5 stars. In the full game condition,
players moved their pawns based on how good of a rating they were given for their
feedback, adding competition to the game. Participants in the control were given
the same evaluation items to discuss, but simply went down the list of prompts
on the paper. All groups received 30 minutes of required time, and 20 minutes of
elective time for the task.

Participants in the framing and full game conditions did not report significantly
different interest/enjoyment (averaging 3.24, and 3.36 out of 5 respectively).
Participants in these game conditions reported significantly higher interest/
enjoyment than the controls (averaging 2.60) on a 5 point scale. However,
participants in the control condition addressed significantly more (averaging 9.80)
discussion items than either of the framing or full game conditions (averaging 5.50,
and 7.66 respectively). Adding a game frame to a task may have crowded out output
motivation for the task.

21
 LIMITS TO GAMIFICATION
GAMIFICATION MAY INCREASE QUANTITY BUT NOT QUALITY. Some research
suggests that gamification may increase productivity only in a purely quantitative
sense, but to increase quality, adding meaning works better. Both adding the
gamified element of points and meaningfully framing the task increase motivation
to complete the task, albeit in different ways. Mekler, Bruhlmann, Opwis, and Tuch
(2013) found that people perceived a painting tagging task as more valuable when
they were told that their tags would help improve computerized affective imagery
categorization (i.e., helping computers label the emotional content of art), and that
people gave more tags when they received arbitrary points for completing the task.
The researcher had subjects provide tags (words that are perceived as related to the
artwork) to 15 abstract paintings, and manipulated whether they received arbitrary
points and/or whether the task was framed in a meaningful way. After completing
the tag task, participants completed the Intrinsic Motivation Inventory (IMI), a
measure of intrinsic motivation to complete a task.

Participants who had been told that their input would be used to improve
computerized affective imagery categorization to advance science perceived the
task as significantly more valuable and marginally more important, regardless
of points, compared to those who were not told this. Giving points for each tag
entered resulted in significantly more tags, while framing did not significantly
impact the number of tags. Points did not affect the quality of tags, but meaningful
framing generated significantly better tag quality. Because the task asked for tags
describing the mood and emotional content of the paintings, there was a significant
impact on the amount of affective tags (related to emotion) when a meaningful
frame was introduced. Also, intrinsic motivation was significantly increased by all
conditions against the control. It appears that giving people arbitrary points to
complete a task will motivate them to complete that task, but not with the same
caliber of work as when they are told that their work is meaningful. Giving people
immediate feedback (in terms of points) will increase the quantity of the behavior
without regard to quality.

GAMIFICATION MAY DECREASE INTRINSIC MOTIVATION. Despite the popularity

of gamification, Hanus et al. (2014) argue that overly emphasizing rewards may
backfire. They recruited 80 university students in two classes being taught at
the same time and, in one of the classes, added badges and a leaderboard as
gamification elements. Besides this addition, the course was identical for the two
classes.

Figure 14. Sample badge

22
Figure 15. The leaderboard

Students’ motivation (together with satisfaction, effort, feeling of empowerment

and degree of social comparison) was assessed by surveys administered every
four weeks for the duration of the course. In addition, learning outcomes were
measured through an exam at the midpoint and at the end of the course. The
researchers found that students’ intrinsic motivation, having been comparable
in the first assessment, was significantly lower for the gamified group during the
second assessment (4.57 vs. 4.92 on a 7-point scale) and, to a lesser extent, the third
assessment. This lower intrinsic motivation led to worse performance on the final
exam for the gamified condition, although the researchers note that there was
no direct effect of the condition on the final exam grade once they controlled for
intrinsic motivation.

Besides its direct implications for the use of gamification to increase motivation,
a further possible interpretation of this and related research is that gamification
works well where it is the desired outcome that is gamified (e.g., filling in your Linke-
dIn profile), but should be used with caution in contexts in which the gamified act
is only a means to achieve a desired outcome (e.g., increasing attendance to achieve
better learning outcomes).

SOME RESEARCH FAILS TO FIND AN EFFECT OF GAMIFICATION. Zuckerman

and Gal-Oz (2014) found that adding virtual rewards and social comparison to a
mobile activity tracker resulted in no significant change in physical activity. In a
first observational study, the authors found that merely introducing a mobile app
that tracked walking time, enabled goal setting, and provided real-time feedback
resulted in a significant increase in walking time. Thirty participants installed an
accelerometer-based mobile app on their phones that tracked their activity for
an initial three-day period (baselines measure) when the app was not active and
a subsequent two week period when the app was active. When active, the app
automatically set a goal for users that reflected a 10% increase over their baseline
and gave real-time feedback on goal progression in the form of total walking
time and a progress bar showing percent complete. Users also received pop-up
messages congratulating them on meeting their goal if they met their daily goal.
The authors compared the mean walking time achieved in the baseline measure
with the mean achieved during the two weeks when the app was active and found
that participants significantly increased their walking time from 20.07 minutes
at baseline to 30.24 minutes during the intervention. However, since there was no
control group, the authors could not be sure about the causal role of the app.

23
 In a follow-up study, building on the same mobile application discussed above, 59
participants were randomly assigned to use one of three versions of the app for ten
days. Participants either received the quantified version of the app described above
or received the points version where they were awarded points for each minute
walked or the leaderboard version where they could see a real-time leaderboard
ranking all users based on their accumulated points. Although no differences were
found with regard to the impact of the three versions on how much people walked,
the authors did find that the the number of times people accessed the quantified
and points version of the app had a positive impact on the amount of walking they
accomplished. The authors concluded that goal-setting and real-time feedback
can facilitate increased activity and that the addition of virtual rewards and social
comparison might not provide any additional benefit.

Figure 16.

Other studies also fail to find an effect of gamification. In a field experiment

examining the online behavior of users of a peer-to-peer trading service (similar
to Craigslist), Hamari (2013) found that gamification largely did not improve users’
engagement with the online service. In three intervention groups, users were
exposed to badges that they had earned, or that they could earn, as well as badges
that other users had earned. In a control group, users were not exposed to any
badges. The study collected the data of 3,234 users over a year and a half period.
Results showed no significant differences in users’ engagement with the service
between the four groups. However, the researcher did find that the number of times
that a user viewed her own list of earned and unearned badges (see image 2 below)
had a positive correlation with her engagement with the service. The number of
times that a user viewed other users’ badges (see image 1 below) also was correlated
with engagement with the service, but the effect size was small. In other words,
the more often someone looked at their badges, the more engaged with the
service they were. But, gamification by itself did not work as a motivator for more
engagement.

24
This research suggests possible limitations to gamification. The author argues that
the lack of overall effects could be explained by several factors. First, there could
have been low goal commitment, or relatively low determination of users to reach
the goal of attaining badges. Second, relatedly, the author argues that for utilitarian
activities such as service trading (e.g. Craigslist), users might exercise more cognitive
involvement than affective involvement, and therefore gamified (hedonic) features
might be largely ignored. However, the significant effect from exposure to earned
and unearned badges suggests that providing users specific and actionable steps
to take is a promising way to encourage increased engagement. Of course, the
challenge then becomes getting users to engage with the badges that are intended
to engage them with the service.

Figure 17. Sharetribe Badge Page

Figure 18. Images of intervention

25
 INDIVIDUAL DIFFERENCES
GENDER. Koivisto and Hamari (2014) examined how the effects of gamification
differ by demographic attributes and found that the perceived benefits of exercise
games vary depending on the age and gender of the participant, as well as how
long the participant has been engaged with the game. The authors administered
online surveys to 195 users of Fitocracy, an online exercise gamification service. A
link to the survey was posted in forums and in group pages within the service, so
only those registered with the service were able to take part. The authors noted that
these people tended to have above average technology-awareness, as well as higher
overall health consciousness (which correlates often with higher-than-average
education). The survey consisted of constructs for facilitating factors and social,
hedonic, and utilitarian benefits. Continued exercise intentions were also measured
as a behavioral outcome.

Gender differences were found for almost all measured benefits, except for the
utilitarian aspect. Women seemed to have perceived the social benefits of the
platform as greater than how the men perceived them, implying that women value
the social aspects of gamification more than men. Finally, women were found to be
more motivated to increase exercising by using the platform compared to men.

AGE. Younger people typically play more, and there is weak evidence that
gamification works better on them. Webster and Martocchio (1993) show that young
employees receiving microcomputer software training sessions labelled as “play”
show higher motivation to learn and score higher on learning outcome measures
compared to older employees. However, there is no difference in learning outcome
measures between young and older employees when the software training session
is labelled as “work”.

Half of the 68 clerical and administrative employees of a public university that

registered to participate in this software training course were randomly assigned to
the training session labelled as “play”, whereas the other half were assigned to the
training session labelled as “work”. Participants in each of the two training sessions
received the same training content. The only difference between the one-hour
sessions was whether the instructor labelled the session as “play” or “work” in his
introduction. All participants were asked to provide demographic and background
information on computer experience and training expectations, among others. After
their respective training sessions, participants also completed an objective test of
software knowledge as well as a questionnaire on motivation to learn.

Examining learning outcomes, Webster and Martocchio (1993) did not find any age-
related difference in microcomputer software training outcomes, nor did they find
differences in learning outcomes between sessions labelled as “play” or “work”. The
researchers did, however, find an interaction effect between training condition and
age. Employees under the age of 40 who received microcomputer training labelled
as “play” scored higher on learning outcome measures compared to employees
above the age of 40. The researchers suspect that this result may be due to the
increased work experience of older employees, which makes tasks at work less
ambiguous for them. But another interpretation is possible: Maybe young people
like to play more, so that it acts as an incentive for them. A limitation of this study
is that it is about merely framing an activity as a game, rather than really gamifying
the activity.

26
In Koivisto and Hamari (2014)’s survey on Fitocracy users (see earlier), age affected
few of the benefits of gamification in the Fitocracy platform directly. Only ease
of use decreased with age. But the authors also found an interesting interaction
effect. They found that the perceived usefulness, enjoyment, and playfulness of
the gamified platform decreased with time of use. This suggests that gamification
may be a novelty that can help with initiation, but not as much with retention. The
authors also found an interaction between age and time using the service, showing
that the novelty effects are more pronounced the younger the user is. The authors
state that “this finding is consistent with the general belief that younger people,
while being more susceptible to playful interactions, might also get bored more
quickly than more mature users.” An interaction between age, length of use, and
value of networking was also found. Specifically, the authors believe that the older
a participant was, and the longer that the participant was on the gamified site, the
more valuable they found their game social network.

In contrast with Webster and Martocchio (1993)’s finding, Cechanowicz et al. (2013)
found no differences for age group. gender, age group, as well as between gamers
and non-gamers.

COMPETITIVENESS. Adding competitive elements to a motivational exergame

seems to have little effect on actual exercise, but may decrease self-efficacy and
enjoyment in participants who do not consider themselves to be competitive. Song,
Kim, Tenzek, and Lee (2010) assigned 72 undergraduate students at a university in
the Midwest to one of four conditions. Participants first completed a measure of
individual competitiveness (including items like “Competition is fun”), and were
labeled as either low or high in the trait of competitiveness based on a median
split. Next, participants were either assigned to play a the Hula Hoop Wii Fit game
with or without competitive aspects in a future lab session. In the competition
condition, participants were competing with three other participants (really
confederates) to win a $20 gift card, while in the non-competition condition, one
of the four participants would ostensibly be randomly chosen to receive the gift
card. Participants were required to play the game for 10 minutes. They could also
continue playing for up to 8 additional minutes after the required 10 minutes
was over. This, along with a self-report measure in which participants provided
seven adjectives to describe the game, was used to measure participants’ intrinsic
motivation. Participant mood, self-efficacy, expected competence, and heart rate
were also assessed. All scales were reported from 1-10.

The authors did not find any difference between the competitive and
noncompetitive games in terms of actual exercise. Nor did they find an
interaction effect on exercise, between the competitiveness of the game and the
competitiveness of the players. But the authors did find some other interesting
results on the competitiveness of players. Participants low in competitiveness
played for over a minute longer than those high in competitiveness (4.45 min vs.
3.26 min), but only when the game was non-competitive. There was no effect of
the competition level on behavioral intrinsic motivation for highly competitive
participants, however. Participants who were low in competitiveness experienced
significantly lower positive mood following the competitive game (7.71) compared
to the non-competitive game (8.5). Again, no effect was found for highly
competitive participants. Participants low in competitiveness reported lower self-
efficacy following the competitive game (6.13) compared to the non-competitive

27
 game (7.36). Heart rate (which the authors used as a proxy for physical exertion)
increased significantly in response to the competitive game, regardless of whether
participants considered themselves to be highly competitive. Highly competitive
participants’ heart rate was higher in the non-competitive game relative to less
competitive participants.

In line with the above study, Staiano, Abraham and Calvert (2013) found that a
cooperative game led adolescents to lose more weight than the control group,
but they did not find this for the competitive version. More generally, Staiano,
Abraham and Calvert (2013) found that exergames (video games that require motor
movement), especially cooperative ones, can be an effective tool for weight loss
amongst adolescents.

Fifty-four overweight and obese African-American participants aged 15-19 took part
in a 20-week program. They were assigned to competitive exergame, co-operative
exergame or control groups. The exergame consisted of 30-60 minutes of daily
exercise using the Wii Active game. Competing adolescents were encouraged to
earn the most points and expend the most calories compared to their opponent,
whereas co-operative adolescents were encouraged to earn points and expend
calories as a team. Data on weight, self-efficacy, self-esteem and peer support was
collected at baseline, 10 weeks and 20 weeks after the intervention began.

The cooperative group lost significantly more weight (1.65 kg vs. 0 kg) and had
increased self-efficacy compared to the control group. The competitive condition
was not significantly different from the cooperative or control groups in terms of
weight loss (.04 kg) or self-efficacy. One major limitation was small sample size, with
nearly half of the participants dropping out by 20 weeks.

Smart interventions: The above research finds individual differences in the

effectiveness of games and gamification. For instance, competitiveness and gender
may matter. However, we did not find any literature on smart interventions that give
the right games to the right people.

28
EXPERIMENTAL GAMES
In the studies discussed above, games and simulations are used as interventions or
manipulations. Researchers in such studies are interested in the effect of (a certain
version of) a game, compared to a control. Other studies make use of games and
simulations not as a manipulation but as a lab environment. The real world is a
messy place. In games and simulations, one can more cleanly measure aspects of
behavior. Games and simulations have been widely used as lab environments, with
interesting results.

The real world is a messy place. In games and

simulations, one can more cleanly measure
aspects of behavior.

GAMES AS LAB ENVIRONMENTS. For instance, Shah, Mullainathan and Shafir

(2012) used a variety of game-style experiments to test the effects of scarcity on
attentional focus on spending decisions. They found that having less makes people
focus more on things directly related to scarcity but less on other things. The first
experiment began with participants playing Wheel of Fortune and experimenters
varying the number of guesses that each participant had in order to make “rich”
and a “poor” conditions. The rich participants had 20 guesses per round while the
poor participants had 6 per round. Subsequently, participants completed a Dots-
Mixed task designed to measure attention and cognitive control. In all experiments,
unused guesses could be saved for future rounds. Rich participants had on average
53 correct responses on the Dots-Mixed task, while poor participants had on average
only 45 correct responses. The authors hypothesize that poor participants were
depleted because of greater engagement in the previous game.

Another Shah et al. experiment consisted of an Angry-Birds style game where

participants fired shots from a slingshot. Shah et al. made rich and poor conditions
again by manipulating the number of shots. The poor earned on average 2.3 points
per shot, while the rich earned on average only 1.7 points. The rich also spent
less time aiming their shots. The authors also manipulated whether or not the
participant could borrow more shots for the present round, with an interest rate
of 100%. The interest payment was subtracted from the total number of shots for
the game, so interest was effectively paid in the last round. While rich participants
borrowed only 2% of their budget, poor participants borrowed 24% of their budget.
Moreover, they overborrowed to such an extent that poor players who were not
given the opportunity to borrow performed better.

Shah and coauthors run several other experiments in which they used games to
learn about attention shifts in the poor. In a third experiment, participants played

29
 Family Feud and the amount of time the rich and poor groups had to guess was
manipulated. A control group could not borrow time from future rounds, another
could borrow with no interest, and the last could borrow with 100% interest. In
another iteration of this experiment, the effects of borrowing were felt before the
end of the game by reducing payment in the next round by the amount borrowed
divided by the number of remaining rounds. There were two borrowing conditions:
one without borrowing and one with 100% interest. Lastly, another experiment
made previews of the next round’s topic available to some participants in order to
test the amount of attention paid to the present and future between the rich and
poor conditions.

SIMULATIONS AS LAB ENVIRONMENTS. Like games, simulations allow researchers

to cleanly measure behavior. For instance, Webley, Levine and Lewis (1991) used
a quite realistic simulation of saving and spending decisions. The authors had
children participate in a “play economy”. Children tried to save tokens they received
to buy a very nice toy, but they were tempted to spend it immediately on sweets or
entertaining activities. The experiment was staged over six days, with each session
lasting about 40 minutes. The child was given 30 tokens at the start, and a 10 token
allowance on each of the six days, for a total of 90 possible tokens. In order to buy
the toy, the children needed to save 70 tokens. The child would spend 10 minutes in
each of the four available rooms; one was the home base, containing the bank, the
toy store, and a lounge area. The other three rooms provided temptation to spend.
They had options for different activities that cost different amounts: each room had
a 10-token option, a varying option, and a free option. The child could deposit their
money into the bank at any time. The authors found that older children showed a
better understanding of savings and saved more.

30
REFLECTION

ARE GAMIFICATIONS REALLY GAMES? In the introduction,

we listed out typical characteristics of a game. Although there
is not one definition of what a game is, this survey of typical
characteristics of a game makes one wonder whether examples
of gamification (e.g., Actify steps challenges) really make a full-
fledged game out of a certain behavior. Usually there are just
some game-like aspects added to the behavior. For instance,
the Actify steps challenges are arguably fun to play and have
clear rules, but they lack the circumscribed time and place of a
typical game (e.g., 90 minutes on a soccer field), and the humor
and lightheartedness of one. Of course gamifications are also
different from typical games in the sense that they have real,
rather than fictitious, consequences. Still, there is no reason why
gamifications could not be more game-like.

SPORTS ARE GAMIFICATIONS OF EXERCISE. In that sense,

sports are the ultimate gamifications of exercise. People do
not typically consider sports to be gamifications, but there is
no reason why they are not. Races (10K, marathon, etc.) are
obviously gamified running. But soccer too is gamified running.
Football is gamified bodybuilding. Etc. On the other hand, some
people may feel that the circumscribed nature of time in this
context is what distinguishes a game from gamification. A daily
steps challenge of 10K steps is a gamification, while playing
a soccer game is a game. Is there research on whether sports
make people exercise more compared to non-sports exercise?
For instance, could you make people run more by having them
play a game of soccer compared to asking them to run?

Any thoughts or comments on this vision, feel

free to drop Jan Willem a note: [email protected]

31
 REFERENCES
Calvo-Ferrer, J. R. (2017). Educational games as stand-alone learning tools and their
motivational effect on L2 vocabulary acquisition and perceived learning gains. British
Journal of Education Technology, 48, 264–278.

Cechanowicz, J., Gutwin, C., Brownell, B., Goodfellow, L. (2013). Effects of gamification on
participation and data quality in a real-world market research domain. Proceedings of
the First International Conference on Gameful Design, Research, and Applications
(Gamification ‘13), 58-65. http://dx.doi.org/10.1145/2583008.2583016

Coles, C. D., Strickland, D. C., Padgett, L., & Bellmoff, L. (2007). Games that “work”: Using
computer games to teach alcohol-affected children about fire and street safety.
Research in Developmental Disabilities, 28(5), 518-530.

de-Marcos, L., Garcia-Lopez, E., Garcia-Cabot, A. (2016) On the effectiveness of game-like

and social approaches in learning: Comparing educational gaming, gamification
& social networking. Computers & Education, 95, 99-113. https://doi.org/10.1016/j.
compedu.2015.12.008

Domínguez, A., Saenz-de-Navarrete, J., Luis de-Marcos, L., Fernández-Sanz, L., Pagés, C.,
Martínez-Herráiz, J. (2013). Gamifying learning experiences: Practical implications
and outcomes. Computers & Education, 63, 380–392. http://dx.doi.org/10.1016/j.
compedu.2012.12.020

Flatla, D. R., Gutwin, C., Nacke, L. E., Bateman, S., & Mandryk, R. L. (2011). Calibration
games:making calibration tasks enjoyable by adding motivating game elements.
Proceedings of the 24th Annual ACM Symposium on User Interface Software and
Technology - UIST’11, 403–412. https://doi.org/10.1145/2047196.2047248

Fogel, V. A., Miltenberger, R. G., Graves, R., & Koehler, S. (2010). The effects of exergaming on
physical activity among inactive children in a physical education classroom. Journal of
Applied Behavior Analysis, 43(4), 591-600.

Habgood J., Ainsworth S. (2011). Motivating children to learn effectively: exploring the value
of intrinsic integration in education games. Journal of the Learning Sciences, 20, 169-
206. http://shura.shu.ac.uk/3556/1/Habgood_Ainsworth_final.pdf

Hamari, J. (2013). Transforming homo economicus into homo ludens: A field experiment
on gamification in a utilitarian peer-to-peer trading service. Electronic Commerce
Research and Applications, 12(4), 236–245. https://doi.org/10.1016/j.elerap.2013.01.004

Hill, J., Jia, Y., Defazio, J., (2016). Theory and Research into Practice: Using Self-Determination
Theory to Analyze Gamification and Motivational Affordances in Serious Games for
Health Education. Poster session presented at IUPUI Research Day 2016, Indianapolis,
Indiana. http://hdl.handle.net/1805/10107

Hanus, M. D., Fox, J. (2014) Assessing the effects of gamification in the classroom: A
longitudinal study on intrinsic motivation, social comparison, satisfaction, effort, and
academic performance. Computers & Education, 80, 152-161. https://doi.org/10.1016/j.
compedu.2014.08.019.

Koivisto, J., & Hamari, J. (2014). Demographic differences in perceived benefits from
gamification. Computers in Human Behavior, 35, 179-188.

Kumar, V. S., Wentzell, K. J., Mikkelsen, T., Pentland, A., Laffel, L. M. (2004). The DAILY (Daily
Automated Intensive Log for Youth) trial: A wireless, portable system to improve
adherence and glycemic control in youth with diabetes. Diabetes Technology &
Therapeutics, 6(4), 445-453.

32
Liu, M., Maza, A., Milkowska, M., Perry, E., Rafiq, S., Teti, K.. (2017). SAVEty Net: How BIT
Ventures Can Help UK Customers Build Emergency Savings Funds. IFC UK: Behavioral
Insights. http://cpl.hks.harvard.edu/files/cpl/files/team_9_-_rainy_day_savings_-_final.pdf

Mekler E., Brulmann, F., Tuch, A., Opwis K., (2015). Towards understanding the effects of
individual gamification elements on intrinsic motivation and performance. Center for
Cognitive Psychology and Methodology, University of Basel, Switzerland.
https://doi.org/10.1016/j.chb.2015.08.048.

Mekler E., Bruhlmann F., Opwis K., Tuch A. (2013). Dissassembling Gamification: The Effects of
Points and Meaning on User Motivation and Performance. CHI Extended Abstracts.

Morewedge, C. K., Yoon, H., Scopelliti, I., Symborski, C. W., Korris, J. H. & Kassam, K. (2015).
Debiasing Decisions. Improved Decision Making With A Single Training Intervention.
Policy Insights from the Behavioral and Brain Sciences, 2(1), pp. 129-140. doi:
10.1177/2372732215600886

Nevin, C., Westfall, A., Rodriguez, J., Dempsey, D., Cherrington, A., Roy, B., … Willig, J. (2014).
Gamification as a tool for enhancing graduate medical education. Postgraduate
Medical Journal, 90(1070), pp.685-693. doi: 10.1136/postgradmedj-2013-132486

Patel, M. S., Benjamin, E. J., Volpp, K. G., Fox, C. S., Small, D. S., Massaro, J. M., … Murabito, J. M. (2017).
Effect of a Game-Based Intervention Designed to Enhance Social Incentives to Increase
Physical Activity Among Families: The BE FIT Randomized Clinical Trial. JAMA Internal
Medicine, 177(11), 1586-1593. https://doi.org/10.1001/jamainternmed.2017.3458

Robson, K., Plangger, K., Kietzmann, J., McCarthy, I., Pitt, L. (2016). Game on: Engaging
customers and employees through gamification. Business Horizons, 59(1), 29-36.
https://doi.org/10.1016/j.bushor.2015.08.002

Shah, A. K., Mullainathan, S., & Shafir, E. (2012). Some Consequences of Having Too Little.
Science. http://doi.org/10.1126/science.1222426

Staiano, A., Abraham, A. and Calvert, S. (2013). Adolescent exergame play for weight loss and
psychosocial improvement: A controlled physical activity intervention. Obesity, 21(3),
pp.598-601.

Tong X., Gromala D., Shaw C.D., Choo A. (2016) A Field Study: Evaluating Gamification
Approaches for Promoting Physical Activity with Motivational Models of Behavior
Changes. In: Kurosu M. (eds) Human-Computer Interaction. Novel User Experiences. HCI
2016. Lecture Notes in Computer Science, vol 9733. Springer, Cham.

Webley, P., Levine, M., & Lewis, A. (1991). A study in economic psychology: Children’s
saving in a play economy, Human Relations, 44(2), 127-146. https://doi.
org/10.1177/001872679104400202

Webster, J., Martocchio, J.J. (1993). Turning work into play: Implications for microcomputer
software training. Journal of Management 19 (1), 127-146.
https://doi.org/10.1016/0149-2063(93)90049-S

Zuckerman, O., & Gal-Oz, A. (2014). Deconstructing gamification: evaluating the effectiveness
of continuous measurement, virtual rewards, and social comparison for promoting
physical activity. Personal and Ubiquitous Computing, 18(7), 1705–1719. https://doi.
org/10.1007/s00779-014-0783-2

33