INFLUENCE OF GAMIFICATION ON STUDENT

ENGAGEMENT IN ONLINE DISCUSSIONS USING

SELF-DETERMINATION THEORY
Rico Putra Pradana, Universitas Indonesia
Ave Adriana Pinem, Universitas Indonesia
Putu Wuri Handayani, Universitas Indonesia

ABSTRACT
This study uses self-determination theory to examine the effect of gamification on the students’
behavioral, emotional, and cognitive engagement in online discussion forums by providing more instruc-
tor badges than automatic badges and using a quasi-experimental one-group pretest–posttest design.
Behavioral engagement was measured using the number of posts and replies; emotional engagement
was measured using perceived relatedness; and cogitative engagement was measured by the quality of
the post/reply’s content. Sixty participants used the developed online discussion website and held two
3-day discussion sessions (non-gamified and gamified) with different topics. The data were collected using
user logs (number of posts and replies), questionnaire responses (perceived relatedness), and assessment
rubrics (scores on the post/reply content), analyzed using the one-way repeated measures analysis of vari-
ance. The results revealed that gamification affected emotional engagement, which was influenced by the
“like” feature, which made it easier for participants to appreciate others’ activities. However, this feature
also decreased the number of replies (behavioral engagement) because the students perceived “liking”
as an easier way to appreciate other posts/replies instead of writing a reply. The number of posts also
decreased since participants’ motivation in the second session tended to focus on the quality of the content
caused by the badge list page, which guides the expected best content from the participant. This study can
provide guidance to universities in implementing gamification in LMS.
Keywords: gamification, online learning, higher education, e-learning
INTRODUCTION system (LMS) for various learning activities,
In response to the Covid-19 pandemic, the including online discussion forums.
Indonesian Ministry of Education and Culture Although the e-learning platform has various
issued directives for remote teaching and learn- benefits, its impact would be suboptimal if students
ing activities (Ministry of Research and Higher had low interest in using it. Studies conducted
Education, 2020). The use of e-learning is expected prior to the Covid-19 pandemic state that students
to meet these needs. Several Indonesian universi- are less involved in online discussion activities and
ties are already using e-learning in their learning rarely access learning material in discussion media
processes. XYZ University is one of the universi- (Ding et al., 2017; Hasan et al., 2019). The problem
ties that has implemented a learning management of low student engagement in online discussions

JOURNAL OF EDUCATORS ONLINE

has been challenging for the effective use of an (gamification) period are significantly higher than
LMS (Ding et al., 2017). Therefore, we observed pre-implementation (no gamification).
the utilization of the LMS at XYZ University in Instructor badges have been demonstrated
24 online classes. The results indicate that from to influence autonomous motivation, a type of
an average of eight discussion forums available motivation that encourages someone to behave
in each class—attended by approximately 205 according to their needs, which is stronger than
students—there was only one post made by one automatic badges (Ding et al., 2017). The previ-
student per class, with an average of 12 replies per ous studies had some limitations; however, Hasan
post. This low number of students encouraged us et al. (2019) discussed behavioral engagement and
to research student involvement in the online dis- did not focus on online discussion forums. Hamari
cussion forum. Student engagement consists of (2017) and Sitra et al. (2017) discussed the appli-
three components: behavioral, emotional, and cog- cation of gamification in the form of badges using
nitive engagement (Ding et al., 2017; Nisiotis & experimental methods but not in the context of
Kleanthous, 2019). Gamification is defined as the online discussions. Furthermore, Hamari (2017)
use of game elements in non-game environments. suggests conducting quasi-experimental research
Zeybek and Saygı (2023) describe that gamifica- using a one-group pretest–posttest design to
tion can increase learning motivation, engagement, identify differences in behavior caused by gamifi-
and achievement in an online educational environ- cation. Therefore, this paper’s study examines the
ment and state that it is used most commonly in effect of gamification on all components of student
higher education, specifically at the undergraduate engagement (behavioral, emotional, and cogni-
level (Zeybek & Saygı, 2023). tive engagement) in online discussion forums by
Most studies have focused on the design of providing more instructor badges than automatic
gamification in certain learning environments badges and using a one-group pretest–posttest
and its impact on learning. Grabner-Hagen and design. The results are expected to provide answers
Kingsley (2023) analyzed the mechanics within related to online discussion design with gamifica-
gamification design in a blended learning envi- tion, which can improve student engagement.
ronment—face-to-face instruction and an LMS.
Sanchez et al. (2019) analyzed the impact of gami- LITERATURE REVIEW
fied quizzes on student learning. Legaki et al. E-Learning and Asynchronous Online Discussion
(2020) analyzed the impact of gamification using Clark and Mayer (2016) defined e-learning as
points, levels, challenges, and leaderboards on “instruction delivered via digital devices (such
learning statistics education. In a systematic review as desktop computers, laptops, tablets, or smart-
of gamification trends for young learners, Behl et phones) that are intended to support learning.”
al. (2022) state that universities should identify Nowadays, students can access their teaching
the best gamification techniques and software materials anytime and anywhere through the inter-
for e-learning. Oliveira et al. (2023) reported that net (Tamm, 2019). Asynchronous learning can thus
universities should design tailored, gamified edu- be used to carry out learning activities through
cational environments that shape students’ needs e-learning (Epignosis, 2014; Ghirardini, 2011).
and preferences. Asynchronous online discussion allows stu-
Gamification can increase student involve- dents to ask questions or discuss anytime and
ment in online discussions. It enhances cognitive anywhere by creating a post on a discussion forum
and behavioral engagement in online discussions or commenting on a post (Abawajy & Kim, 2011).
(Ding, 2019). By contrast, Ding et al. (2017) show According to Ding (2019), three main components
that gamification only significantly positively of asynchronous online discussion are a sense of
affects partial emotional engagement (perceived community, participation, and cognitive thinking.
relatedness). Hasan et al. (2019) demonstrate that The sense of community explains the interactions/
the application of gamification in online discussion relationships between students in discussions that
media positively influences behavioral engagement can encourage the students to create more posts
in discussions. Hamari (2017) indicates that all user (Ding, 2019). Active participation in online discus-
activities recorded during the post-implementation sions can be realized through reading and creating

JOURNAL OF EDUCATORS ONLINE

posts in the discussion (Ding, 2019). Cognitive theory divides motivation into two types: intrinsic
thinking is the ability to think at a higher level and extrinsic (Ryan & Deci, 2000). Intrinsic moti-
(higher-order thinking), such as analysis, synthesis, vation is in play when an action or activity that is
and evaluation, which can be found in discussion conducted for the satisfaction that can be felt sim-
activities (Oh et al., 2018). ply by doing these activities, whereas extrinsic
motivation involves performing an action or activ-
STUDENT ENGAGEMENT
ity to obtain things beyond the satisfaction of doing
According to Nisiotis and Kleanthous (2019),
these activities (Ryan & Deci, 2000). According
student engagement is a commitment or effort put
to Ryan and Deci (2000), only intrinsic motivation
forth by students to be involved in or participate in
can improve one’s creativity and learning out-
learning activities. Student engagement comprises
comes. They also explain that giving gifts, which
three components: behavioral, emotional, and
is a form of extrinsic motivation, can reduce intrin-
cognitive engagement (Ding et al., 2017; Nisiotis
sic motivation in various fields. Therefore, we used
& Kleanthous, 2019). Behavioral engagement is a
SDT as a reference to increase the participants’
form of student learning behavior, such as actively
intrinsic motivation in the discussion.
participating in learning activities, including con-
Human basic psychological needs for auton-
tributing to discussion activities (Ding et al., 2017).
omy, competence, and relatedness are at the core of
Emotional engagement is the psychological or emo-
SDT (Legault, 2017). The need for autonomy is the
tional reaction to the posts made by peers/teachers,
need for a sense of freedom in determining actions
such as feeling attracted, bored, happy, and sad
based on self-consideration (Legault, 2017; Sailer
(Ding et al., 2017; Fredricks et al., 2004). Cognitive
et al., 2017), which relates to behavioral engage-
engagement is a form of attention, willingness, and
ment, where the participants determine how they
effort to understand complex concepts, master dif-
will participate in discussion activities. The need
ficult skills, and use higher-order thinking skills
for relatedness is the need to build relationships
(Ding et al., 2017; Fredricks et al., 2004; Sedláček
and a sense of concern for others (Legault, 2017),
& Šeďova, 2020).
and relates to emotional engagement in which
GAMIFICATION psychological or emotional reactions to friends or
Deterding et al. (2011) defined gamification as teachers in class can arise in discussion activities.
“a process of using game elements in a non-game The need for competence is the need for a sense
context to motivate and engage users.” The use of competence and effectiveness when interacting
of game elements is expected to make non-game with the environment (Legault, 2017; Sailer et al.,
activities feel similar to those of playing a game 2017), which relates to cognitive engagement in
(Sailer et al., 2017). Gamification can make non- which the participants’ attention, will, and efforts
game activities fun, encourage social interaction are mobilized for understanding complex concepts,
in the learning community (Ding et al., 2017), and mastering difficult abilities, and using higher-order
may increase motivation and performance in per- thinking skills in discussion activities. Self-
forming an activity (Sailer et al., 2017). Various determination theory can be used to determine the
game elements can be used in implementing gami- game elements that will be used in applying gami-
fication in an activity, including badges, points, fication to an activity (Wee & Choong, 2019).
leaderboards, progress bars, reaction systems,
HYPOTHESIS DEVELOPMENT
levels, meaningful stories, avatars, teammates,
This study evaluates the effect of gamification
and challenges.
on components: the number of posts and replies
SELF-DETERMINATION THEORY (behavioral engagement), perceived relatedness
Self-determination theory (SDT) is the scores (emotional engagement), and the grades/
most popular theory for analyzing gamifica- scores for the post and reply (cognitive engage-
tion (Kirchner-Krath et al., 2021). It is “a theory ment). Behavioral engagement in online discussion
of motivation and personality that discusses how activities can be measured through the active par-
individuals interact and depend on the social envi- ticipation of students in discussions (Ding et al.,
ronment” (Legault, 2017). Self-determination 2017), for example, through the number of posts

JOURNAL OF EDUCATORS ONLINE

and replies made by them (Ding, 2019; Hamari, Quantitative data were analyzed using one-way
2017). Emotional engagement can be assessed repeated measures analysis of variance (ANOVA),
through the variable perceived relatedness, which and qualitative data were analyzed using a general
is measured using a questionnaire adapted from inductive analysis.
the Intrinsic Motivation Inventory (Ding, 2019).
RESEARCH INSTRUMENT
Cognitive engagement in online discussion activi-
ties can be measured using the grade obtained Discussion Forum
based on the quality of the post/reply content We created a website prototype as an experi-
during discussions (Ding, 2019). Gamification sig- mental tool, the design of which was based on the
nificantly affects the number of posts and replies LMS in the XYZ University to reduce the partici-
made in the discussion (Barata et al., 2013; Hamari, pants’ adaptation burden. This prototype had two
2017), perceived relatedness (Ding et al., 2017), and versions: non-gamified and gamified. Both ver-
the grades obtained in the discussion (Barata et al., sions have the same main functionality, but the
2013). On the basis of this explanation, this study gamified version has two additional components:
proposes four hypotheses: badges and likes (the reaction system). These two
H1. Gamification affects the number of elements were chosen because they can fulfill three
posts made in online discussions. basic human psychological needs in SDT (Ding,
2019; Sailer et al., 2017; Saputro et al., 2017). This
H2. Gamification affects the number of forum allowed participants to read, create, modify,
replies made in online discussions. delete, and reply to posts and replies in the forum;
H3. Gamification affects the notifications that notify participants of new posts/
perceived relatedness felt in online replies; and basic functionalities such as logging
discussion activities. in, logging out, account registration, and viewing
H4. Gamification affects the grades obtained personal profiles. Table 1 summarizes the partici-
by the online discussion participants. pants’ demographic characteristics.
RESEARCH METHOD Table 1.
Research Design Participants’ Demographic Characteristics
This study used a mixed-methods design,
where quantitative data were obtained through logs Variable Number Percentage
of participant activity in discussion forums and
15–20
questionnaires, and qualitative data were obtained years
19 31.7%
through interviews. We recruited 60 participants Age
20–25
through volunteer sampling of all active XYZ 41 68.3%
years
University students in the even semesters of 2019–
2020. The participants attended a general course Male 18 30%
Gender
for character-building lectures and topics of dis- Female 42 70%
cussion that related to knowledge for that course.
Have used e-learning
A one-group pretest–posttest design was used Yes 52 86.7%
other than the LMS
in this study: the participants held two 3-day Not yet 8 13.3%
of the university
discussion sessions in a discussion forum with
different topics. The first session did not have gam-
ification (pretest), and the second did (posttest). At Assessment Rubric
the end of each discussion session, the participants We measured the grades obtained by the stu-
were asked to fill out a questionnaire containing dents based on the quality of the post content and
closed questions to measure perceived relatedness their replies during online discussion activities.
and open questions to investigate their experi- A scoring rubric was used as a guide for assign-
ences during the discussion. After the collected ing a score for each post and reply (Appendix C),
data was processed, interviews were conducted which was adapted from Brown (2014) regard-
with six participants to deepen the research results. ing the application of active learning in online

JOURNAL OF EDUCATORS ONLINE

teacher education lectures. This rubric was also Figure 1.
used to design criteria for collecting badge skills Badges List Page
(Brown, 2014).
Questionnaires and Interviews
The participants completed the questionnaire at
the end of the first and second discussion sessions
(Days 3 and 6, respectively). This questionnaire
comprised two parts. The first part included a
5-point Likert scale (1 = strongly disagree to 5 =
strongly agree) to measure the variable perceived
relatedness (emotional engagement) adapted from
the Intrinsic Motivation Inventory. The second
part involved open-ended questions to evaluate
the participants’ attitudes toward online discussion
activities in each LMS version adapted from Ding Figure 2.
(2019). Furthermore, several interview questions Profile Page in Gamified Version
were developed based on the results of the analy-
sis of the collected experimental data. Appendix A
describes the interview instruments and Appendix
B presents the questionnaire instruments.
GAMIFICATION DESIGN
Gamification Components
The second discussion forum (gamified ver-
sion) applies two gamification components: badges
and likes. Using the “likes” feature, the partici-
pants can like posts/replies in the discussion forum
and investigate the number of likes they have
given and the number of likes their posts/replies
have received. Furthermore, the participants could
obtain badges when they give/receive likes and
create posts/replies whose content meets certain
Figure 3.
criteria. A list of badges and their respective acqui- Likes Feature
sition criteria was also provided in the gamified
version. The participants could see a list of badges
that they have successfully collected on their pro-
file page, which could only be seen by themselves.
The participants received a notification every
time they successfully received a badge or like.
Examples of pages containing a list of badges, a
profile page, and a “likes” feature are provided in
Figures 1, 2, and 3. performing basic activities, such as creating posts
Two types of badges were used in this study: and replies and giving/receiving likes. Skill badges
participatory and skill. Participatory badges help were awarded if a skill was successfully mastered
participants recognize what activities they can by the participants (Ding, 2019). The gamified
do in the gamified version, one of which is the version provided 14 skill badges (Figure 5) with
badge system, so that participants are motivated badge collection criteria that refer to an assessment
to be involved in discussion activities (Ding et al., rubric adapted from Brown’s (2014) research on
2017). The gamified version provided four partici- the application of active learning in online teacher
patory badges (Figure 4), which were awarded on education lectures.

JOURNAL OF EDUCATORS ONLINE

Figure 4. not obliged to be fulfilled (Laerd Statistics, 2018).
Participatory Badges Furthermore, the sphericity assumption test can be
ignored because this study only has two treatment
conditions: pretest (non-gamification) and posttest
(gamification) (Field, 2009).
ANOVA
A one-way repeated measures ANOVA was
used to assess the effect of gamification as an
Figure 5. independent variable on each dependent variable,
Skill Badges
including the number of posts and replies made by
participants, the grades obtained by participants,
and the perceived relatedness scores that were felt
during online discussion activities. The results
revealed that two dependent variables have a sig-
nificant difference in scores: the reply variable (F
(1, 59) = 6.868, p < 0.05) and the perceived related-
ness (F (1, 59) = 13.088, p < 0.05). However, the
increase in the mean value occurred only in the
perceived relatedness variable, whereas the reply
variable decreased. ANOVA results, descriptive
statistics, and hypothesis test results are presented
in Tables 2, 3, and 4, respectively.
RESULT
Table 2.
Pre-Analysis ANOVA
Perceived relatedness was measured using a
Likert scale questionnaire. The questionnaire’s
validity was evaluated using Pearson’s product Variable Sig. (p value)
moment test (Santosa et al., 2017) and reliabil- Number of posts 0.350 (> 0.05)
ity using Cronbach’s alpha (Field, 2009; Raharjo,
2019). Both tests were conducted using IBM SPSS Number of replies 0.011 (< 0.05)
v25. The questionnaire was considered valid with Grade 0.111 (> 0.05)
Pearson’s r value of the score for each item with a
positive total score and p < 0.05 (Raharjo, 2019). Perceived relatedness 0.001 (< 0.05)
Furthermore, the questionnaire was considered
reliable because the non-gamification and gamifi- Table 3.
cation questionnaires had Cronbach’s alpha values​​ Descriptive Statistics
of 0.814 and 0.763 (> 0.6) (Field, 2009).
This study used one-way repeated measures Non-Gamification
ANOVA to process the quantitative data. Its three Gamification (N = 60)
Variable (N = 60)
basic assumptions are the independence of obser-
M SD M SD
vations, normality, and sphericity (Laerd Statistics,
2018). The assumption of the independence of Number of posts 0.22 0.454 0.28 0.585
observations is fulfilled because the study sample Number of replies 5.02 4.459 3.87 2.777
was obtained randomly through the volunteer sam-
pling method so that the score of each individual Grade 293.50 253.623 254.83 182.687
in the sample was independent. This study had the Perceived
same number of samples for the pretest and post- 16.7083 1.26405 17.1542 1.12567
relatedness
test conditions (60 people), and the sample size
was >30 so that the assumption of normality is *Note: M = mean, SD = standard deviation

JOURNAL OF EDUCATORS ONLINE

Table 4. information related to the achievement obtained
Hypothesis Testing by their peers impaired their frequency of posting.
The tendency for student behavior to be less active
Hypothesis Result in class and willing to start discussions can also
Gamification affects the number of
affect the number of posts.
H1
posts made in online discussions.
Rejected The results also reveal that gamification sig-
nificantly decreased the number of replies (H2
Gamification affects the number of
H2 Accepted accepted). In line with Ding (2019), the interview
replies made in online discussions.
results reveal that the “like” feature made it easier
Gamification affects perceived relatedness for participants to appreciate other participants,
H3 Accepted
that is felt in online discussion activities. thereby decreasing the need for replying to a post
Gamification affects the grades obtained to appreciate other participants. In addition, the
H4 Rejected
by the online discussion participants. participants’ motivation in the second discussion
session was predominantly to write good-quality
content than to post as much as possible. This may
GENERAL INDUCTIVE ANALYSIS be due to the badge list page, which can be used as
Interviews were conducted to investigate the a guide in designing the post/reply content she/he
reasons behind the rejection of hypotheses H1 and wants to create (Groening & Binnewies, 2019).
H4 and the decrease in the mean value in the num- However, when participants tried to write
ber of replies. These qualitative data were analyzed quality content to obtain a badge but failed to
using a general inductive analysis. This approach obtain it, they experienced a sense of despair, thus
broadly consists of four stages: preparing data, diminishing the motivation to actively discuss it.
understanding data, categorizing, and refining the Furthermore, the participants no longer felt an
identified categories (Thomas, 2006). Appendix obligation to post replies because they had already
C summarizes the results of the analysis of the done so in the first session.
interviews answers. Consistent with Ding et al. (2017), our results
DISCUSSION AND IMPLICATIONS show that gamification significantly increased the
Our results indicate that gamification did not perceived relatedness scores in the second discus-
significantly affect the number of posts made in sion session (H3 accepted). Ding (2019) stated that
the discussion (H1 is rejected). This is in line with the like feature can indeed support the fulfillment
Dicheva et al. (2015), who reported that gamifica- of the need for relatedness. Consistently, our inter-
tion did not affect teaching and learning activities. view results reveal that participants felt it easier
In addition, the participants were not required to to appreciate other participants through the like
post in either discussion session. Furthermore, feature. This appreciation is a form of caring for
the interview results (Appendix A) show that others, which is an encouragement from fulfilling
the participants were not aware of the benefits of the need for relatedness (Legault, 2017). In addi-
grades and badges, so the new motivation in the tion, analysis of the open-ended answers revealed
second discussion session that was generated by that more participants felt they could share ideas/
these two things disappeared. The participants opinions — both the same and different opinions
may find it helpful to have clear information about — in the second session of the discussion than in
these benefits so that they know how to partici- the first session.
pate in the discussion and ultimately increase their Finally, the results reveal that gamification
involvement (Ding et al., 2017). In addition, the had no significant effect on the grades obtained by
participants could not see the profiles of other par- online discussion participants (H4 rejected). The
ticipants on the gamified version to measure their rejection of H4 may be because H1 was not proven.
abilities compared with others. The participants This happens because the total score that partici-
tended to compare the badges collected, trying pants get during the discussion is proportional
to collect the same badges as most participants to the number of posts made in the discussion.
(Hamari, 2017). Thus, even though the partici- Consistently, Barata et al. (2013) state that the num-
pant understood the assessment rubric, the lack of ber of posts and the grades are strongly correlated.

JOURNAL OF EDUCATORS ONLINE

Furthermore, the analysis of the open-ended perceived relatedness scores felt by participants
answers indicates that the topic of discussion in the in the discussion. This may occur because of the
second session was less interesting. The instructor like feature, which makes it easier for participants
should first determine which topics the participants to appreciate the activities of other participants.
would be interested in, because the reason for the Conversely, behavioral engagement was measured
lack of interesting discussion topics based on the using the number of replies made by participants
interview results is the participants’ lack of under- in the discussion, which decreased significantly,
standing of the topics raised in the second session. likely because the participants preferred to use
IMPLICATIONS the like feature over replying to posts to appreci-
This research has several important implica- ate other participants. In addition, the participants’
tions for discussion instructors, managers of online motivation in participating in the second discus-
discussion forums, and future studies. This study sion session tended to focus on the quality of the
fails to prove that gamification can significantly reply content, which might be caused by the badge
affect the number of posts made in the discussion, list page, which can be used as a guide in design-
which is likely due to the inappropriate design of ing the post/reply content they want to make.
the gamification component (Ding, 2019). Dicheva Our data also indicate that gamification did not
et al. (2015) suggest considering students’ abili- significantly affect behavioral engagement (num-
ties and motivation when choosing and designing ber of posts) or cognitive engagement (grade).
badges to be used in the learning process. The average number of posts made in the second
The participants could not look at the profiles discussion session increased, whereas the average
of other participants to compare their achievements grade decreased. Gamification design is one of the
with those of others. According to the theory of factors that might cause this to happen. In addition,
social comparison, social proof theory, and social the participants were not obliged to create posts
validation, the comparison of student achievement
in both sessions, which may have also caused the
can encourage participants to be more active in
lack of a significant effect on these two variables.
discussion (Hamari, 2017). Our results also indi-
cate that the participants experienced a feeling The participants also did not know the usefulness/
of despair when they failed to obtain the badges benefits of grades and badges for him, so the new
they had been chasing. In addition, the like feature motivation in the second discussion session that
made participants prefer to give likes as a form should have been raised by these two things disap-
of appreciation for the activeness of other partic- peared. Finally, the lack of functionality to view
ipants, rather than expressing it via a reply, thus the profiles of other participants and compare their
decreasing the number of replies. achievements may underline these findings.
The instructors should first comprehensively We have several suggestions for future studies
assess the ability and motivation of students when to also adopt the theory of social comparison, social
designing a gamification component that will be proof theory, and social validation in designing
applied in online discussion activities. A func- functionality in a system that allows participants to
tionality that allows students to compare their compare their own achievements with other partic-
achievements with others should also be provided. ipants. The provision of the like feature also needs
Furthermore, badges can have negative effects, attention, because this feature can positively or
such as feelings of despair when the efforts to negatively affect participant involvement in online
obtain them lead to failure. Therefore, attention discussions. Future studies should explore strate-
must be paid to the various components of gamifi- gies to overcome the negative effects caused by
cation that will be applied to the forum, along with the gamification component. Next, when design-
their positive and negative influences. ing a gamification component that will be applied
CONCLUSION in online discussion activities, instructors/manag-
This study proves that the application of gam- ers must comprehensively assess the abilities and
ification in online discussion activities increases motivations of students who will be participating
emotional engagement, measured using the in the online discussion activities.

JOURNAL OF EDUCATORS ONLINE

 References Groening, C., & Binnewies, C. (2019). “Achievement unlocked!”—
The impact of digital achievements as a gamification element
Abawajy, J., & Kim, T. -H. (2011). Engaging and effective on motivation and performance. Computers in Human
asynchronous online discussion forums. Communications Behavior, 151–166. https://doi.org/10.1016/j.chb.2019.02.026
in Computer and Information Science, 695–705. https://doi. Hamari, J. (2017). Do badges increase user activity? A field
org/10.1007/978-3-642-27207-3_75 experiment on the effects of gamification. Computers in
Aldiab, A., Chowdhury, H., Kootsookos, A., & Alam, F. (2016). Human Behavior, 71, 469–478. https://doi.org/10.1016/j.
Prospect of e-learning in higher education sectors of Saudi chb.2015.03.036
Arabia: A review. Energy Procedia, 574–580. https://doi. Hasan, H. F., Nat, M., & Vanduhe, V. Z. (2019). Gamified
org/10.1016/j.egypro.2017.03.187 collaborative environment in Moodle. IEEE Access, 7, 89833–
Barata, G., Gama, S., Jorge, J., & Goncalves, D. (2013). Improving 89844. https://doi.org/10.1109/ACCESS.2019.2926622
participation and learning with gamification. ACM International Holmes, K. (2005). Analysis of asynchronous online discussion
Conference Proceeding Series, pp.10–17. https://doi. using the SOLO taxonomy. Australian Journal of Educational
org/10.1145/2583008.2583010 and Developmental Psychology, 5, 117–127.
Behl, A., Jayawardena, N., Pereira, V., Islam, N., Del Giudice, M., Laerd Statistics. (2018). Understanding repeated-measured
& Choudrie, J. (2022). Gamification and e-learning for young ANOVA. Retrieved from https://statistics.laerd.com/statistical-
learners: A systematic literature review, bibliometric analysis, guides/repeated-measures-anova-statistical-guide.php
and future studies agenda. Technological Forecasting and
Kirchner-Krath, J., Schürmann, L., & von Korzfleisch, H. F. (2021).
Social Change, 176, 121445.
Revealing the theoretical basis of gamification: A systematic
Brown, A. L. (2014). Implementing active learning in an online review and analysis of theory in research on gamification,
teacher education course. American Journal of Distance serious games and game-based learning. Computers in
Education, 28(3), 170–182. https://doi.org/10.1080/08923647. Human Behavior, 125, 106963. https://doi.org/10.1016/j.
2014.924695 chb.2021.106963
Dicheva, D., Dichev, C., Agre, G., & Angelova, G. (2015). Legaki, N. -Z., Xi, N., Hamari, J., Karpouzis, K., & Assimakopoulos,
Gamification in education: A systematic mapping study. V. (2020). The effect of challenge-based gamification on
Educational Technology & Society, 18(3), 75–88. learning: An experiment in the context of statistics education.
Ding, L. (2019). Applying gamifications to asynchronous online International Journal of Human-Computer Studies, 144,
discussions: A mixed methods study. Computers in Human 102496. https://doi.org/10.1016/j.ijhcs.2020.10249
Behavior, 91, 1–11. https://doi.org/10.1016/j.chb.2018.09.022 Legault, L. (2017). Encyclopedia of Personality and Individual
Ding, L., Kim, C., & Orey, M. (2017). Studies of student Differences. Springer, Cham. https://doi.org/10.1007/978-3-
engagement in gamified online discussions. Computers 319-28099-8
and Education, 115, 126–142. https://doi.org/10.1016/j. Mekler, E. D., Brühlmann, F., Tuch, A. N., & Opwis, K. (2017).
compedu.2017.06.016 Towards understanding the effects of individual gamification
Epignosis. (2014). E-learning Concepts, Trends, Applications. elements on intrinsic motivation and performance. Computers
Field, A. (2009). Discovering Statistics Using SPSS. London: in Human Behavior, 71, 525–534. https://doi.org/10.1016/j.
SAGE Publications Ltd. chb.2015.08.048
Fredricks, J. A., Blumenfeld, P. C., & Paris, A. H. (2004). School Ministry of Research and Higher Education. (2020). PDDikti—
engagement: Potential of the concept, state of the evidence. Pangkalan Data Pendidikan Tinggi. Retrieved June 16, 2020,
Review of Educational Research, 74(1), 59–109. https://doi. from https://forlap.ristekdikti.go.id
org/10.3102/00346543074001059 Nisiotis, L., & Kleanthous, S. (2019). The relationship between
Ghirardini, B. (2011). E-learning methodologies: A guide for students’ engagement and the development of transactive
designing and developing e-learning courses (2nd ed.). Food memory systems in MuvE: An experience report. Annual
and Agriculture Organization of the United Nations (FAO). Conference on Innovation and Technology in Computer
Retrieved from http://www.fao.org/3/i2516e/i2516e.pdf Science Education, pp. 71–77.
Grabner-Hagen, M. M., & Kingsley, T. (2023). From badges to boss Oh, E. G., Huang, W. -H., Hedayati Mehdiabadi, A., & Ju, J. B.
challenges: Gamification through need-supporting scaffolded (2018). Facilitating critical thinking in asynchronous online
design to instruct and motivate elementary learners. discussion: Comparison between peer and instructor-
Computers and Education Open, 4, 100131. redirection. Journal of Computing in Higher Education 30(6),

JOURNAL OF EDUCATORS ONLINE

 489–509. https://doi.org/0.1007/s12528-018-9180-6 ACKNOWLEDGEMENT
Oliveira, W., Hamari, J., Shi, L., Toda, A. M., Rodrigues, L., We want to convey our gratitude to the
Palomino, P. T., & Isotani, S. (2023). Tailored gamification in University of Indonesia for the Grant Hibah PUTI
education: A literature review and future agenda. Education Hibah Publikasi Terindeks Internasional/PUTI
and Information Technologies, 28, 373–406. Pasca Sarjana titled “Model Tata Kelola Teknologi
Raharjo, S. (2019). How to perform Product Moment Validity Test Informasi Sebagai Katalisator Good Governance
with SPSS. SPSS Indonesia [website]. Retrieved July 2, 2020, Rumah Sakit Di Indonesia Untuk Mendukung Satu
from https://www.spssindonesia.com/2014/01/uji-validitas- Data Indonesia” year 2024.
product-momenspss.html
Ryan, R. M., & Deci, E. L. (2000). Self-determination theory and
the facilitation of intrinsic motivation, social development, and
well-being. American Psychologist, 55(1), 68–78. https://doi.
org/10.1037/0003-066X.55.1.68
Sailer, M., Hense, J. U., Mayr, S. K., & Mandl, H. (2017). How
gamification motivates: An experimental study of the effects
of specific game design elements on psychological need
satisfaction. Computers in Human Behavior, 69, 371–380.
https://doi.org/10.1016/j.chb.2016.12.033
Sanchez, D. R., Langer, M., & Kaur, R. (2019). Gamification in
the classroom: Examining the impact of gamified quizzes on
student learning. Computers & Education, 144, 103666.
Saputro, R. E., Salam, S. B., & Zakaria, M. H. (2017). A review
of intrinsic motivation elements in gamified online learning.
Journal of Theoretical and Applied Information Technology,
95(19), 4934–4948.
Sedláček, M., & Šeďova, K. (2020). Are student engagement
and peer relationships connected to student participation in
classroom talk? Learning, Culture, and Social Interaction, 26,
100411.
Sitra, O., Katsigiannakis, V., Karagiannidis, C., & Mavropoulou, S.
(2017). The effect of badges on the engagement of students
with special educational needs: A case study. Education
and Information Technologies, 22(6), 3037–3046. https://doi.
org/10.1007/s10639-016-9550-5
Tamm, S. (2019, December 21). What is E-Learning? E-Student.
org [website]. Retrieved June 22, 2020, from https://e-student.
org/what-is-e-learning
Thomas, D. R. (2006). A general inductive approach for analyzing
qualitative evaluation data. American Journal of Evaluation,
27(2), 237–246. https://doi.org/10.1177/1098214005283748
Wee, S. -C., & Choong, W. -W. (2019). Gamification: Predicting the
effectiveness of variety game design elements to intrinsically
motivate users’ energy conservation behaviour. Journal of
Environmental Management, 97–106. https://doi.org/10.1016/j.
jenvman.2018.11.127
Zeybek, N., & Saygı, E. (2023). Gamification in education:
Why, where, when, and how?—A systematic
review. Games and Culture, 19(2), 1–28. https://doi.
org/10.1177/1555412023115862

JOURNAL OF EDUCATORS ONLINE

 APPENDIX A. INTERVIEW INSTRUMENTS
1. What do you like or enjoy about online discussions?
2. Which aspects of the gamification component influenced your participation in this online discussion?
Please explain each gamification component used.

APPENDIX B. QUESTIONNAIRE INSTRUMENTS

3. Non-Gamified Questions
Strongly Disagree Neutral Agree Strongly
Questions
Disagree (1) (2) (3) (4) Agree (5)
In discussion forums, I feel that I do not have a close relationship with the
participants who have responded with replies to my posts/replies.
I really doubt that in the future me and the participants who have responded
with replies to my posts/replies will become friends in everyday life.
I feel that I can really trust the participants who have responded with replies to my
posts/replies.
I would like to have the opportunity to interact more often with the participants who
have responded with replies to my posts/replies.
In the future, I really prefer not to interact with the participants who have responded by
replying to my posts/replies.
I don’t feel I can fully trust the participants who have responded with replies to my
posts/replies.
It is possible that I and the participants who have responded with replies to my posts/
replies can become friends in everyday life if we often interact in discussion forums.
In discussion forums, I feel that I have a close relationship with the participants who
have responded with replies to my posts/replies.

4. Gamified Questions
Strongly Disagree Neutral Agree Strongly
Questions
Disagree (1) (2) (3) (4) Agree (5)
In the discussion forum, I feel that I do not have a close relationship with the
participants who have responded with replies or likes to my posts/replies.
I really doubt that in the future I and the participants who have responded with
replies or likes to my posts/replies will become friends in everyday life.
I feel that I can really trust the participants who have responded with replies and likes
to my posts/replies.
I would like to have the opportunity to interact more often with the participants
who have responded with replies or likes to my posts/replies.
In the future, I really prefer not to interact with the participants who
have responded with replies or likes to my posts/replies.
I don’t feel like I can fully trust the participants who have responded with replies or likes
to my posts/replies.
It is possible that I and the participants who have responded with replies or likes to my
posts/replies can become friends in everyday life if we frequently interact in discussion
forums.
In discussion forums, I feel that I have a close relationship with the participants
who have responded with replies and likes to my posts/replies.

JOURNAL OF EDUCATORS ONLINE

 APPENDIX C. ASSESSMENT RUBRIC

Category Score Post Criteria Reply Criteria

0 Not creating posts. -

Creating a post that is good enough but not with in-depth

Create Post 10 thought and preparation so that the issues raised do not -
meet all aspects of the big topic of the discussion forum.

Create a post that is well designed so that it meets all

20 -
aspects of the big topic of the discussion forum.

Post a reply that is not a further response

0 -
to a post/reply made by another user.

Make Reply Post replies that are only in the form of agree/disagree
10 -
statements so as not to enrich the discussion.
Post replies by analyzing/responding to other
20 -
users’ posts/replies to widen the discussion.
Create posts that raise issues that are irrelevant Post replies that are irrelevant to the
0
to the big topic of the discussion forum. issues raised in related posts.
Create posts that raise issues that are relevant to the big Post replies that are relevant to the issues
10 topic of the discussion forum but do not provide additional raised in related posts but do not add significant
Content meaningful (substantive) information for the discussion. (substantive) information to the discussion.
Contribution
Create posts that contain correct information Post replies that contain correct information
and are relevant to the big topic of the discussion and are relevant to the issues raised in related
20
forum and provide significant (substantive) posts and provide additional information that is
additional information for the discussion. significant (substantive) for the discussion.

Create posts without including references in Post a reply without including references in the
0 the form of writing or personal experience that form of writing or personal experience that
underlies the information conveyed in the post. underlies the information conveyed in the reply.

Reference and Create a post by including personal experience Post a reply by including personal experience
Support Evidence 10 as the basis for the information but excluding as the basis of the information but excluding
references from a reading or research article. references from a reading or research article.

Create a post by including references from a reading, Post a reply by including references from a reading,
20
research literature, or personal experience. research literature, or personal experience.

Create long and disorganized posts that may contain Create long and disorganized replies that may contain
0
many errors in word choice, grammar, or spelling. many errors in wording, grammar, or spelling.

Create posts with the delivery of information that Post replies by conveying information that is
Clarity and is friendly, polite, and helps readers understand friendly, polite, and helps arguments that are
Writing 10
the points of information you want to convey, but already in the replies of other users, but there
Mechanism there are still some mistakes in writing them. are still some mistakes in writing them.
Create posts that are clear and concise and presented Post clear and concise replies and present them
20 in a delivery style that is easy to understand and in a delivery style that is easy to understand and
free from grammatical and spelling errors. free from grammatical and spelling errors.

JOURNAL OF EDUCATORS ONLINE

 APPENDIX D. SUMMARY OF INTERVIEW

Condition Description Example of the Response

“My desire to post on the second session was greater
Not knowing the benefit of grades and
because of the ratings. However, because I did not know the
Grade and badge benefits badges eliminates the motivation that
usefulness of grades and badges, the new motivation that
should arise in the gamification session.
emerged in the second discussion session disappeared.”

“When I tried to create a reply post according to the criteria

Participants felt hopeless when they failed but didn’t get a badge, I felt hopeless and lost motivation
Failed to obtain the badge
to obtain the badge they were targeting. to create a post/reply again to catch up with the badges.
Finally, after giving up, I stopped posting/replies.”

Badges motivated the initiation of

“I feel that I have to try to create posts with full explanations so that
Motivation to obtain started discussion in the second session that
the content meets the criteria so that I can get a certain badge.”
tends to lead to quality, not quantity.

Badges motivated the initiation of “In the second version, more people have made replies,
Motivation to reply discussion in the second session that so I am more selective which posts to comment
tends to lead to quality, not quantity. on with good and appropriate comments.”

Participant perceived that the obligation

Perceived obligation to reply/respond in the second session “I felt that the obligation to reply/respond again
to give response disappeared because they already disappeared because I have already responded.”
responded in the first session.

Participant could not see other users’ “I can’t see other people’s profiles to compare
Comparing achievements profiles to compare achievements badges and grades so I can’t measure myself and my
and measure one’s abilities. own achievements compared with others.”

“My motivation for posting replies is to appreciate other

It is easier to respect other participants participants who have also replied to my posts/replies. However,
Respect fellow participants
through the like feature than writing a reply. because the second version has a like feature, the way I appreciate
them is by making the reply change to just giving likes.”

The topic of the second session’s discussion

“Feeling less interested in the topics/issues raised
Discussion topic was not suitable because it was not interesting
in the second session of the discussion forum.”
and was not understood by the participants.

JOURNAL OF EDUCATORS ONLINE