Informatics in Education, 2022, Vol. 21, No.

1, 179–200 179
© 2022 Vilnius University, ETH Zürich
DOI: 10.15388/infedu.2022.07

A Strategy of Learning Computational Thinking

through Game Based in Virtual Reality: Systematic
Review and Conceptual Framework

Sukirman SUKIRMAN1,2, Laili Farhana Md IBHARIM1*,

Che Soh SAID1, Budi MURTIYASA2
1
Faculty of Art, Computing, and Creative Industry, Universiti Pendidikan Sultan Idris,
Perak, Malaysia
2
Faculty of Teacher Training and Education, Universitas Muhammadiyah Surakarta, Indonesia
e-mail: [email protected], [email protected], [email protected], Budi.Murtiya-
[email protected]

Received: February 2021

Abstract. Research trends on computational thinking (CT) and its learning strategies are show-
ing an increase. The strategies are varying, for example is using games to provide enjoyment,
engagement, and experience. To improve the high level of immersion and presence of game
objects, learning strategies through games can be improved by virtual reality (VR) technology
and its application. However, a systematic review that specifically discusses game based in VR
(GBiVR) settings is lacking. This paper reports previous studies systematically about the strate-
gies used to learn CT through games and VR applications. 15 papers were selected through Pre-
ferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines. As the
result, this study proposes a conceptual framework for designing a strategy to learn CT through
GBiVR settings. The framework consists of critical aspects of variables that can be considered
in the learning environment like game elements, VR features, and CT skills. All the aspects are
discussed below.
Keywords: systematic review, conceptual framework, computational thinking, game-based learn-
ing, virtual reality, prisma.

1. Introduction

Research trends on computational thinking (CT) are showing an increase in recent

decades (Ilic et al., 2018; Tang et al., 2020). It can be seen from the international
research publications about CT or relates are improved in two periods (2006–2012

* Corresponding author
180 S. Sukirman et al.

and 2013–2018) from 3798 to 7175 or escalated narrowly double, whereas it is only
indexed by Web of Science (WoS) (Tang et al., 2020). The researches also showed
that subjects and participants involved are not only higher education degrees, but
also secondary level and even primary students, for example, Fagerlund et al. (2020)
even conducted a research to assess CT skills of 4th grade students through Scratch
programming projects. One of the reasons why this research is growing because CT
offers a set of strategies to resolve complex problems by applying computer sciences’
reasoning processes (Hooshyar et al., 2021). In the computer science (CS) area itself,
Zhang & Nouri (2019) stated that CT is considered as the core amongst prevalent
topics such as computing, programming, coding, and problem-solving as illustrated
in Fig. 1. They added that programming and coding are often interchangeable in collo-
quial use. Coding can be pointed as the writing of computer programming code, while
programming itself is more than just coding since involves complex tasks such as
understanding a problem, designing, coding, and maintenance. The larger concept is
computing which covers CT, coding, programming, and computing. Hence, students
who mastered CT are potentially to resolve problems encountered in programming
scope or related it as well. As we know, programming lies behind all digital technol-
ogy, software, and systems around us today. However, CT is not merely about CS,
but it is a combination of thinking skills that are crucial for handling complex prob-
lems, such as theoretical (mathematical) thinking, engineering thinking, and scientific
thinking (Chou, 2019).
CT is described as thought processes involved in formulating problems and their
solutions that are represented in an effective form to be carried out by an information-
processing agent (Cuny et al., 2010). CT is believed as one competence that should be
expertized by all educational levels of 21st-century students as it promotes a way of
thinking inspired by CS styles in solving problems (Nouri et al., 2020; Zhang & Nouri,
2019). Even, Wing (2006) argued that CT is not only for CS students, but it is a funda-
mental skill that should be understood by everyone outside CS learners, like reading,
writing, and arithmetic. By learning CT, students may benefit from the principles, con-
cepts, and approaches commonly applied to computer science.
Many countries have started or currently in the process to introduce CT and CS
or elements thereof in their official national curricula, for example, England, Finland,
Sweden, Portugal, Malaysia, and many others (Nouri et al., 2020; Saad, 2020). They

Fig. 1. Relationship among prevalent topics.

 A Strategy of Learning Computational Thinking through Game Based in ... 181

realize that CT is important for everybody since we live in this digital era and deal with
technology based on computers. In Indonesia, the ministry of education and culture
(Kemdikbud) issued a regulation to apply CS in the middle (junior and high) schools as
a standalone subject with the name “Informatika”, where the basic core of the subject is
CT (Kemdikbud, 2019). In this subject, CT is placed in the bottom layer among the foun-
dation of knowledge area, computing practice, and information communication technol-
ogy (ICT). However, CT actually has been introduced in many countries through the
Bebras challenge, an international competition that aims to promote CT among students
of all levels since several years ago (Bavera et al., 2020). More than 50 countries are
currently participating in this challenge which is the involved participants do not require
to have prior knowledge of programming or CS. It means that CT has internationally ac-
knowledged and should be adopted to prepare a better young generation. As it is known,
today’s society is massively computerized in all spheres.
Along with the advance of CS and its applications today, many studies reported that
integrate CT into school curricula would benefit students learning processes, both cog-
nitive and non-cognitive aspects (Hooshyar et al., 2021; Malva et al., 2020; Román-
González et al., 2017). Lot of tools and devices used for teaching CT are becoming more
diverse, from visual block-based programming like Scratch, Blockly, Kodu, Construct 2,
and App Invertor (Zhang & Nouri, 2019), as far as augmented reality (AR) (Cleto et al.,
2020) and virtual reality (VR) application (Chen et al., 2020). Additionally, strategies
adopted for learning CT are also varying, for example, project-based learning, problem-
based learning, cooperative learning, and game-based learning (GBL) (Hsu et al., 2018).
The majority goal is to improve the learning processes to be fun for all students, provide
enjoyment, engagement, and getting more experiences.
One popular setting employed to teach CT that allows learners more enjoy and free to
demonstrate their own goals is GBL (Hsu et al., 2018). This strategy offers a high degree
of autonomy and lets learners make multiple decisions on how to execute and how the ap-
proach that used to tackle the problems which encountered in the environment (Nietfeld,
2020). Actually, the approaches to acquire CT with GBL can be designed in two settings,
(1) learning through designing a game and (2) learning through gameplay (Turchi et al.,
2019). Learning through designing a game entails students to design and develop a game
that contains goals and rules so that can be played by others. The activities and instruc-
tions are followed by students and they learn from the challenging tasks assigned. When
learning to design and develop a game, learners may study CT such as problem-solving,
decomposition, abstraction, and pattern recognition (Cetin, 2016; Turchi et al., 2019).
Meanwhile, learning through gameplay refers to learners acquire CT concepts by play-
ing a game and accomplish the tasks provided in the games. For example, Penguin Go,
a video game designed with Blockly to facilitate students of middle schools for learning
CT (Zhao & Shute, 2019). In this game, students may learn an algorithm concept by
developing a sequence of steps and explicitly put them into a program.
To enhance users’ experience in a high level of immersion and presence, the learn-
ing strategy through gameplay settings in a game can be designed through VR features
(Radianti et al., 2020). It is because VR provides a sense of “being” in the task envi-
ronment instead so that makes users feel actually “there”. VR is an artificial object and
182 S. Sukirman et al.

its environment generated by computer technologies to simulate real-world artifacts

and provide a new digital environment that makes users feel like present in the new
world (Berntsen et al., 2016). Applying a game based in virtual reality (GBiVR) for
training or learning means incorporating learning material contents into the game-
play to enhance playability, interactivity, playfulness, presence, and immersion so that
makes users more enjoy while studying. Compared to just a VR learning environment
(VRLE) only, GBiVR offers more challenges and enjoyment that encourage learn-
ing motivation (Giannakos, 2013). Additionally, compared to traditional classroom
instruction, Shi et al. (2019) stated that GBiVR has two promising characteristics, they
are 1) active learning driven by the experience of challenge, engagement, and playful-
ness; and 2) representation of situated knowledge which provides facilities knowledge
acquisition.
Due to the increasing scholarly attention to CT learning, there have been several
systematic reviews of tools or strategies employed to teach or learn CT, such as visual
block-based programming Scratch (Zhang & Nouri, 2019), robot (Yang et al., 2020),
toys and kit (Yu & Roque, 2019) or even without a computer and its application named
as “unplugged” (Huang & Looi, 2020). However, a systematic review that specifically
discusses GBiVR as a tool and strategy for learning CT is lacking or even has not
been conducted. Therefore, this paper aims to investigate systematically from previ-
ous studies about the strategy that can be used for learning CT through GBiVR. The
main research question (RQ) of this study is “What are the essential factors that should
be considered in developing a strategy to learn CT through GBiVR?”. Based on the
review, we propose a conceptual framework of learning CT through GBiVR that poten-
tially can be used to guide the development and implementation of the strategy to learn
CT through GBiVR.

2. Methodology

This systematic review employs Preferred Reporting Items for Systematic Reviews and
Meta-Analyses (PRISMA) guidelines that consist of four steps, namely identification,
screening, eligibility, and included as illustrated in Fig. 2 (Liberati et al., 2009). This
strategy was chosen due to the direction allows us to reduce potential bias, produce
a good quality of the reports and meta-analysis, and may be adopted to all types of sys-
tematic reviews, not limited to clinical trials (Salvador-Ullauri et al., 2020). Addition-
ally, PRISMA provides checklist items that lead us to analyze and synthesize the data
collected systematically.

2.1. Search Strategies and Database Sources

Determining appropriate keywords and selecting databases are the most crucial steps
in a systematic review. The results may affect to the answer of research questions.
The keywords used in this systematic review are “computational thinking”, “game”
 A Strategy of Learning Computational Thinking through Game Based in ... 183

Fig. 2. PRISMA flowchart employed in this study.

and “virtual reality”. The searching strategies were performed by the combination
of keywords separately and sequentially utilizing the feature of “search within re-
sults”, namely “computational thinking” followed by “game” and then “virtual real-
ity”. The searching was performed to the database sources that provide a facility of
“search within results”, namely Scopus, Web of Science (WoS), and IEEE Xplore. Ad-
ditionally, the searching was also conducted to ACM digital library, Science Direct,
Springer Link, and Wiley Online Library in a slightly different way due to the facility
of “search within results” is not provided. However, the keywords and combinations
used were kept in the same, utilizing the Boolean operator “AND” and quotation
mark. The final search was carried out in the last week of November 2020 which
yielded 301 documents. The detailed result obtained by the search protocol that has
been defined previously is provided in Table 1. It can be seen that the most found
documents are recorded in the Scopus database followed by Springer Link, they are
109 and 98 respectively. However, the records still need to be assessed to find out the
requirements of inclusion criteria.
184 S. Sukirman et al.

Table 1
Detail results obtained by the search protocol

Database sources Documents identified

Scopus 109
WoS 6
IEEE Xplore 13
ACM Digital Library 48
Science Direct 16
Springer Link 98
Wiley Online Library 11
Total 301

2.2. Eligibility Criteria

Before screening processes, the duplicate records generated by the different databases
should be removed first, and they remained 265 documents. The records excluded from
the screening processes were 108 and remained 157 documents. The record was elimi-
nated when it met the exclusion criteria, and it will be selected when it met the inclusion
criteria as seen in Table 2. The conference name was excluded because it is just the name
of event and there was no content related to the RQ. Irrelevant documents refer to the
records that do not contain both CT and VR, so that although the record contain CT but
does not contain VR or the opposite, it was excluded. The documents also must be writ-
ten in English, if not in English, they will be excluded. The record was also excluded if it
was a research that has not been accomplished or ongoing research, so that it was lacking
result that cannot be analyzed further. The documents that contain the word “augmented
reality” were also excluded due to its characteristic is different from VR, start from the
definition until the device used. Based on the criteria provided in Table 2, we excluded
145 documents as ineligible criteria and remained 12 records as eligible criteria.
After finished all the steps within PRISMA guidelines, we included 3 additional
papers that are considered eligible but escaped from the search processes that recom-
mended by the other authors. It was carried out by a snowballing approach, a strategy to
retrieve relevant articles based on target papers’ references list or paper citing (Wohlin,
2014). The criteria of inclusion and exclusion were also same as the ones applied in

Table 2
Exclusion and inclusion criteria

Exclusion criteria Inclusion criteria

Name of conference events Non-review articles

Irrelevant documents Relevant studies
Non English documents Written in English
Ongoing research publications Completed research
Contain word “augmented reality” Appropriate to the research question
 A Strategy of Learning Computational Thinking through Game Based in ... 185

the PRISMA framework. Initially, some authors recommended 5 additional articles to

be included, but after scrutinized and analyzed the contents, we decided excluded 2 ar-
ticles. We excluded them because the articles were not containing CT contents despite
them containing VR and learning outcomes. The aforementioned processes through
PRISMA guidelines are diagrammatically illustrated in Fig. 2. The final result of the
papers that we identified as being relevant to this systematic review is 15 documents.
The selected documents have been through rigorous screening based on determined
criteria written in Table 2.

2.3. Quality Assessment of the Documents

The next step is assessing the quality of selected papers. To assess the quality, we ad-
opted an assessment strategy proposed by Feng et al. (2018) and Connolly et al. (2012).
The assessment of the selected documents of this study was involving in two raters using
the questions below:
i. How relevant is the focus of the study to address the research question of this
review?
ii. How appropriate are the methods and analysis used to answer the research ques-
tion of this review?
iii. How appropriate is the research design for addressing the research question?
A score of the weight of evidence (WoE) that is used to assess the quality of pa-
pers is between 1 and 3, which means 1 is low quality, 2 is medium, and 3 is a high-

Fig. 3. Weight of Evidence (WoE) scores calculated from 15 selected papers.

186 S. Sukirman et al.

quality article. Based on the 3 provided questions above, an article has a score between
3 and 9. Each paper is assessed by two raters from an internal author and an external
author who has the same background research in VR or CT. The scores were then
calculated to obtain the mean of the final score to represent the quality of all reviewed
articles in this study. The mean score of 15 papers is 6.73, while the standard deviation
is 1.668. Fig. 3 illustrates the histogram that describes the frequency score of 15 papers
assessed by two raters. It can be seen that the curve illustrated in the histogram is left-
skewed which the mean score is closer to the left. Therefore, it shows that the quality
of selected articles quite good.

2.4. Data Extraction and Analysis

Final selected papers were extracted by reading each full-text manuscript and coding it
with a scheme in an Excel file format. Therefore, the findings were systematically well
organized to ease the analysis. The coding scheme was compiled based on the formu-
lated research questions and the available detailed information in the papers. The main
scheme was arranged through the following fragments:
●● Author information consists of the authors’ last name and publication year.
●● The title of the reviewed manuscript.
●● Objectives of the study.
●● Methodology, research design, or strategies adopted in the research.
●● Obtained results and conclusions based on the findings and discussions.
●● Elements or factors considered in the research.
●● Research variables including independent and dependent variables.
●● Strategies and instruments to collect data.
●● Challenges, limitations, and future works.

3. Results

This section provides the findings based on 15 papers that have been selected, coded,
and analyzed systematically. It consists of 10 conference proceedings papers, and 5 oth-
ers are articles published in journals as provided in Table 3. The articles were published
by reputable publishers such as the Institute of Electrical and Electronics Engineers
(IEEE), Association for Computing Machinery (ACM), Springer International Publish-
ing, Routledge – Taylor & Francis, and John Wiley & Sons, Ltd. They are known as
a good quality publisher in many spheres, mainly computer science and related. The
majority of the articles are published in 2020, namely 7 articles, followed by 2019 which
consists of 5 articles, and 3 others are published in 2018, 2017, and 2016. It means that
the articles are fresh and up to date since the publications are recently issued. The num-
ber of publications is also showing an increase, which means that the trend about the
strategy of learning CT and its related through game based in VR settings is positively
growing over the year.
 A Strategy of Learning Computational Thinking through Game Based in ... 187

Table 3
Selected papers based on inclusion criteria

No. Authors & Year Types Publisher

1 (Parmar et al., 2016) CP IEEE

2 (Lin et al., 2017) CP ACM
3 (A Dengel, 2018) CP IEEE
4 (Banic & Gamboa, 2019) CP IEEE
5 (Berns et al., 2019) CP ACM
6 (Kao, 2019) CP ACM
7 (Lai et al., 2019) JA Routledge
8 (Turchi et al., 2019) JA Springer
9 (Andreas Dengel, 2020) CP ACM
10 (Chen et al., 2020) JA Frontiers
11 (Jin et al., 2020) CP ACM
12 (Leonard et al., 2020) JA Routledge
13 (Nguyen et al., 2020) CP Springer
14 (Pearl et al., 2020) CP ACM
15 (Segura et al., 2020) JA John Wiley & Sons, Ltd

CP: conference proceeding

JA: journal article

3.1. Learning Objectives and Pedagogical Types

Based on the 15 selected papers for review, it was identified that the main objectives
of the studies can be classified to be four categories, namely fostering CT, introducing
programming concepts, teaching CS concepts, and leveraging CT concepts in courses
for problem-solving. All of the studies were set up in the VR environment either in
game-based learning (GBL) settings or non-GBL settings. Fig. 4 illustrates the total
number of each setting based on the objective categories classified in Fig.1. It can be

Fig. 4. Main objectives of the studies.

188 S. Sukirman et al.

seen that teaching the concepts of CS is the most widely conducted which a total of
5 articles, while leveraging CT concepts for problem-solving is the lowest one which
only 2 papers.
More deeply related to CT, it can be grouped to become two pedagogical types,
namely (1) CT acts as a learning outcome, and (2) CT acts as a strategy of learning.
Table 4 shows the objectives classification based on the pedagogical types. CT acts as
a strategy of learning means utilizing or incorporating CT concepts for an approach to
tackle the problem as the research conducted by Lai et al. (2019) and Chen et al. (2020).
Lai et al. (2019) integrate the concept of CT education into a course on plot image-based
VR. In that research, students were allowed to demonstrate the CT skills to resolve prob-
lems that designed in the plot such as broke the plot down into problem decomposition,
problem exploration, and program algorithm. Meanwhile, Chen et al. (2020) combine
head-mounted VR and CT experiments to drive students to create ideas for real disaster
relief scenarios. In their research, students were allowed to think about different script
situations to find a suitable maker design for the project.
CT acts as a learning outcome means that CT is set up for the goal of learning. The
example is the research that conducted by Turchi et al. (2019), Jin et al. (2020), Leonard
et al. (2020), and Pearl et al. (2020). Turchi et al. (2019) conducted a research to im-
prove CT skills emphasizing two different elements provided in GBL settings, namely
playfulness, and collaboration. They investigate whether the CT skills can be enhanced
through gameplay in a GBL setting or vice versa. The result stated that the overall re-
sponse was showing positive feedback despite the participants needed some practices at
the beginning. Slightly different from Turchi et al. (2019), Leonard et al. (2020) taught
CT through the creating of dance performances for virtual characters in a VR environ-

Table 4
Classification of objectives based on pedagogical types

Objectives Including Pedagogical Authors

types

Leverages CT Integrating CT concepts into plot planning, Strategy (Lai et al., 2019), (Chen et al.,
concepts for leveraging CT concepts to create ideas for 2020)
problem-solving disaster scenarios
Teaching CS Facilitates learning CS, enhance the under- Outcome (Parmar et al., 2016), (A Den-
concepts standing of CS, teaching CS itself, making gel, 2018), (Banic & Gamboa,
CS more tangible, presenting CS concepts 2019), (Berns et al., 2019),
(Andreas Dengel, 2020)
Introducing Introduce programming concepts, Aid users Outcome (Lin et al., 2017), (Kao, 2019),
Programming learning (Java) programming, propose (Nguyen et al., 2020), (Segura
concepts a programming environment based on VR, et al., 2020)
teaching the basic programming concepts
Fostering CT Fostering CT through collaborative game- Outcome (Turchi et al., 2019), (Jin et al.,
based learning, to boost children’s creativity 2020), (Leonard et al., 2020),
and CT skill, teaching CT, to lower the thre- (Pearl et al., 2020)
shold the construction of computational
algorithms
 A Strategy of Learning Computational Thinking through Game Based in ... 189

ment. Students were explained that dance choreography and computer programming
have similar compositional processes. Therefore, the students were set to engage in sev-
eral choreographic and programming tasks. From this setting, the students can learn
several computational concepts like repetition as loop, theme-and-variations as Boolean
logic, variations as variables, and unison in dance as parallelism. The research showed
that students’ CT abilities were improved.
However, the majority of the studies sought to make the learning of CT and prevalent
topics to be more fun and more provided new experiences through VR technology. With
the new settings in the VR environment, students can learn more engage and enjoy the
activities. Students also can improve their knowledge, abilities, and change their per-
ceptions about who computer scientists are and what they actually do (Leonard et al.,
2020). Additionally, students are also motivated and unleash their ability to think more
creatively and differently to solve a problem (Chen et al., 2020).

3.2. Study Designs and Settings

From the 15 selected papers, experimental research is the type most employed as illus-
trated in Fig. 5, namely 33% or 5 articles (Banic & Gamboa, 2019; Chen et al., 2020;
Lai et al., 2019; Lin et al., 2017; Segura et al., 2020), followed by exploratory 27% or 4
articles (Andreas Dengel, 2020; Leonard et al., 2020; Parmar et al., 2016; Turchi et al.,
2019), and pilot testing 20% or 3 articles (Berns et al., 2019; Jin et al., 2020; Nguyen
et al., 2020). Meanwhile, 13% or 2 articles (A Dengel, 2018; Pearl et al., 2020) are un-
clearly explained and 7% or 1 article (Kao, 2019) employed research and development
(R&D) by recruiting experts through an online portal. In the experimental research,
Segura et al. (2020) conducted three times experiment with different purposes of each,
namely (1) to evaluate the general performance of the system and its user experience, (2)
to assess the acceptance of users, and (3) to evaluate whether the designed system is bet-
ter or the opposite which was set up through the experimental group and control group.

Fig. 5. Study designs that employed in the researches.

190 S. Sukirman et al.

Almost all research in the experimental type is designing the setting in two groups,
intervention or experimental group and control group. The difference is the approach
used, for example, Banic & Gamboa (2019) employed visual design problem-based
learning by allowing students to create 3D sculptures compared to traditional strategy,
while Lin et al. (2017) allowed both groups to use the developed system with different
tasks. Lai et al. (2019) and Chen et al. (2020) almost had the same design, grouping the
classes into two groups with the same treatment at the beginning to make sure that all
groups had the same basic knowledge, then made different interventions in the middle
of learning processes to identify the differences between the intervention group and
natural group.
In the exploratory type, Parmar et al. (2016) and Leonard et al. (2020) almost used
the same design. They allowed students to create dance choreography with a CT con-
cepts approach. The differences are Leonard et al. (2020) needed a longer time and
more participants involved than Parmar et al. (2016). Meanwhile, both Turchi et al.
(2019) and Andreas Dengel (2020) divided the participants into several groups, eight
and three respectively. The participants were allowed to play the game designed by both
researchers, then they explored and observed the participants’ attitude toward overall the
application system. Additionally, they also asked participants to complete questionnaires
provided by the researchers.

3.3. Variables and Learning Impacts

Table 5 shows the classification of variables that are generally concerned in the research
based on the 15 reviewed papers. We categorized them into four types such as game

Table 5
Classification of the research variables

Game Elements VR Features CT Concepts Learning impacts

Playability Telepresence Decomposition Creativity

Engagement Social presence Pattern recognition Self-directed learning/ autonomy
Excitement Metaphorical representation Abstraction Independently
Interesting Presence Algorithm design/thinking Persistence/Maintain
Enjoyment Immersion Answer set programming Learning experience
Customization Embodied interaction Finite state machine Learning interest
Playfulness Social interaction Problem-solving Learning motivation
Collaborative Visibility of abstract Recursively thinking Learning outcomes
Interactivity Visual aspects Heuristic Knowledge, & comprehension
Visual layout Data representation Practical motivation
Debugging (evaluating The mental model of computer
solutions) scientists
Sequence Common sense
Loop Creative thinking
Variable Systemic reasoning
 A Strategy of Learning Computational Thinking through Game Based in ... 191

elements, VR features, CT concepts, and learning impacts. The elements of the game
consist of playability, engagement, excitement, interesting, enjoyment, customization,
playfulness, collaborative, interactivity, and visual layout. Meanwhile, elements of VR
are telepresence, social presence, metaphorical representation, presence, immersion,
embodied interaction, social interaction, and visibility of the abstract objects. CT el-
ements consist of decomposition, pattern recognition, abstraction, algorithm design/
thinking, answer set programming, finite state machine, problem-solving, recursively
thinking, heuristic, data representation, debugging (evaluating solutions), sequence,
loop, and variable. In the meanwhile, learning impacts comprise creativity, self-directed
learning/ autonomy, independently, persistence/maintain, learning experience, learning
interest, learning motivation, learning outcomes, knowledge & comprehension, practi-
cal motivation, mental model of computer scientists, common sense, creative thinking,
and systemic reasoning. Based on these variables and the categories, we constructed
a conceptual framework that is discussed in the next separated section.
The first variable in the game elements is playability, which refers to how easy or
how long duration of a game can be played by users. Playability is closely related to
others variables in the game elements. According to Paavilainen (2020), the playabil-
ity of a game is defined as the design quality of the game, formed by its functional-
ity, usability, and gameplay. However, he argued that games with good playability are
not necessarily fun, sometimes games with poor playability may provide an enjoyable
experience. The main objective of playability is to provide enjoyment and playful-
ness. Hence, it can improve the excitement and interest to play the games. As the re-
search conducted by Parmar et al. (2016) and Banic & Gamboa (2019), they are more
concerned about the enhancement of users’ engagement prefer than playability. Their
strategies were to design enjoyable tasks with an interesting visual layout and provide
a virtual character with the immersive embodied interaction supported by VR technol-
ogy. Hence, utilizing game elements in the VR settings can improve user experiences
and playability as well.
In the VR environment, Parmar et al. (2016) measured telepresence by designing
various questions that lead to the sense of “being there”, for example, “Did you feel like
you were inside and surrounded by the environment?”. If we analyzed, it is similar to
the term “presence” or “virtual presence” as stated by Lombard & Ditton (1997) and
Selzer et al. (2019). All three different terms have the same meaning; it is a concept
of being perceptually present in the virtual environment generated by VR. However,
Parmar et al. (2016) slightly distinguish it with the term “social presence” since they
used virtual character for the research, but the concept is still the same as “presence”.
On the other hand, Leonard et al. (2020) investigated other VR features that showed
distinct relationships namely social, embodied interactions and their expressed learning
engagement in computational practices. They also stated that embodied experiences
obtained in a VR environment allowed students to engage in a variety of computational
practices.
Related to CT, it can be viewed from two designs as explained in section 3.1, CT as
the learning outcome and CT as the strategy of learning. Turchi et al. (2019) designed
CT as the learning outcome through collaborative game-based learning. The CT con-
192 S. Sukirman et al.

cepts considered in their research are decomposition, abstraction, algorithmic thinking,

problem-solving, data representation, and debugging. Problem-solving is described as
understanding what the goal of the problem is and finding a solution to solve it, while
algorithmic thinking explained as a way to find a solution through a set of steps. In
line with Turchi et al. (2019), Leonard et al. (2020) also designed CT as the learning
outcome. CT concepts adopted in their research are sequence, loop, and variables. To
measure the concept of sequence, students were asked to construct a dance movement
of the virtual character and investigate how they accomplish it. Leonard et al. (2020)
believed that the concept of a sequence is similar to dance choreography that has itera-
tion. In the meanwhile, CT designed as the strategy of learning is applied by Lai et al.
(2019) and Chen et al. (2020), namely problem decomposition, program algorithm,
pattern recognition, and abstraction. Lai et al. (2019) integrated the concept of CT
into a course on plot image-based VR. They stated that the used approaches are able to
make students more familiar with scenarios, improve their learning interest and overall
academic performance. Almost similar to Lai et al. (2019), Chen et al. (2020) employed
the CT concept to allow students to think about the different script situations in a sce-
nario of emergency disaster training. They stated that it is able to strengthen students’
practical learning motivation and programming skills as well.
Based on the reviewed papers, the researchers stated that their design of studies has
several impacts on learning. As the research conducted by Banic & Gamboa (2019)
said that self-directed learning of students in the concepts of computer programming
was improved. Additionally, based on the observations, Banic & Gamboa (2019) stated
that students’ creativity to find out solutions was increased as well and students inde-
pendently sought to learn more computer programming concepts on their own. Even,
Leonard et al. (2020) claimed that their interventions were able to change students’
computational perspectives and mental model of who computer scientists. The majority
revealed that their research may improve students’ learning experiences, motivation,
common sense, creative thinking, and systematic reasoning (Berns et al., 2019; Lai
et al., 2019; Segura et al., 2020).

3.4. Software and Hardware Devices

Fig. 6 (a) illustrates the VR devices used in the 15 selected papers. Mostly, the papers
did not clearly mention what the devices used in the researches, 57% or more than
half. They only mentioned head-mounted display (HMD) devices. Oculus, both Rift
or Quest types, are the most used in the papers with a total of 33% or five articles.
Meanwhile, only 14% or two articles used HTC Vive to generated VR environments.
The papers that utilized Oculus Rift in the researches are Parmar et al. (2016), Lin
et al. (2017), Leonard et al. (2020), and Pearl et al. (2020), while Jin et al. (2020) used
Oculus Quest type. The reason why Jin et al. (2020) chose this platform since it pro-
vides an all-in-one VR gaming system that may track users’ motion without cables so
that support immersive learning experiences. Andreas Dengel (2020) and Segura et al.
(2020) used HTC Vive for their researches to construct the VR application system.
 A Strategy of Learning Computational Thinking through Game Based in ... 193

(a) (b)

Fig. 6. (a) VR devices and (b) Software used to develop VR application.

Segura et al. (2020) selected HTC Vive because it is free and provides the functional
plugin for other software like Unity 3D, so that may help developers to build user
interfaces.
Unity 3D software is the type most used to build VR application systems as illustrat-
ed in Fig. 6(b), 53% or 8 papers. This software is selected for several reasons as stated
by Segura et al. (2020), (1) provides the required tools to create interactive 3D environ-
ments, (2) the documentation for application programming interface (API) is complete
and well-documented, and (3) the software can be implemented for many different
platforms like PCs, mobile devices, web-based, and even consoles. Meanwhile, 40%
or 6 papers did not mention what the type of software used in the researches is. One
paper or 7% used A-frame, web-based software to develop the VR application system,
namely Berns et al. (2019). They chose A-frame due to the framework allows the devel-
opers to plug into an entity so that may add appearance, behavior, or functionality.

4. Discussion

Based on the articles reviewed above, we summarized several factors that can be con-
sidered to develop and implement the learning of CT skills through game based in VR
settings. We proposed a conceptual framework to implement a strategy of learning CT
skills through GBiVR as depicted in Fig. 7. Generally, it can be grouped into 3 parts,
they are game elements, VR features, and CT concepts or CT skills. The Game ele-
ments and VR features can be encapsulated to be VRGACT (virtual reality game appli-
cation for CT). The Game elements and VR features are positioned as the independent
variables that intervene in CT skills as the dependent variables. We only selected sev-
eral aspects of game elements, VR features, and CT concepts that should be considered
in the proposed conceptual framework. This selection is based on the identification
194 S. Sukirman et al.

Fig. 7. Conceptual framework to implement the learning of CT skills through GBiVR.

and analysis conducted in the previous sections. The selected aspects in the proposed
conceptual framework are playability and interactivity gained from game elements,
while presence and immersion are the elements proposed from the VR environment.
Meanwhile, enjoyment is variable generated by the combination of both game elements
and VR features collectively. By playing game in the VRGACT world, users or students
are expected to be more enjoy the designed environment and its gameplay. Finally, CT
skills that we selected as the goal of learning are composed of decomposition, pattern
recognition, abstraction, and algorithm design which in line with the research designed
by Chen et al. (2020).
The first element is playability, which refers to the ability of VRGACT to provide
an environment that allows users to play and learn at the same time as discussed in
the section 3.3. It is in line with the research conducted by Turchi et al. (2019) that
the VR-based game application should be playable and learnable to support learning
to play and playing to learn. Playability is an important factor that should be consid-
ered while design a game since it is the core of the game and determine its quality
(Paavilainen, 2020). Playability also describes the degree to which a game is fun to
play and usable, with an emphasis on the interaction style and plot quality of the game
(Usability First, 2020). The appropriate value of playability will give users obtaining
more positive experience in assimilating the educational and playful concepts underly-
ing the dynamic game.
Another important element in the conceptual framework is interactivity, which re-
lates to the ability of VRGACT to respond and give appropriate feedback to users while
playing. Hence, it can engage the users as the research carried out by Leonard et al.
(2020). Interactivity establishes a two-way communication process between users or
players as the learners and the features provided by the game apps. The communication
will construct learning activities that involve the taught material contents and the avail-
able properties of the apps. In the GBiVR, interactivity can be designed in both physical
and virtual forms, so that improve users’ involvement and participation. The physical
 A Strategy of Learning Computational Thinking through Game Based in ... 195

form can be designed as the embodied interaction, while the virtual form is the effect
generated by the VRGACT system.
Elements of VR we proposed on the conceptual framework are presence and im-
mersion. The presence or virtual presence is a concept of being perceptually present in
a virtual environment so deeply as “if the medium were not there” (Selzer et al., 2019).
The feature of presence makes the objects designed virtually become more real so that
allows users to apply embodied interaction between users and the virtual objects as the
research carried out by Leonard et al. (2020). By this feature, Leonard et al. (2020)
found distinct relationships between students’ social, embodied interactions, and learn-
ing engagement. Meanwhile, immersion in VR pertains to users’ perception of being
physically present in a non-physical world like a digital environment. This perception
can be generated by visual displays, sounds, and other stimuli surrounding the users
that influence the perception. Berns et al. (2019) stated in their researches that visual
aspects or displays may improve overall learning experiences so that students enjoy
to see the results produced by the VR game application system. The combination of
game elements and VR features finally should make users enjoy to play and learn in
the environment created by the application. When students enjoy the settings, they will
automatically complete all the tasks provided by the system. Further, the factor of en-
joyment will make users more engaged and self-directed learning as argued by Banic
& Gamboa (2019). That is why we selected enjoyment as the element that should be
considered in the conceptual framework.
We propose CT skills as the dependent variables in the conceptual framework,
which consists of four components, they are decomposition, pattern recognition, ab-
straction, and algorithm design. Actually, the definition of CT is varying due to the
consensus of it has not been reached, but the majority of researchers do not dispute the
definition (Selby & Woollard, 2013). Chen et al. (2020) applied four CT components
aforementioned to experiment with their research. Students were allowed to think about
different script situations by CT concepts and discuss to find a suitable maker of the
project to resolve disaster relief scenarios. In this research, students have a chance to
demonstrate CT skills. The result shows that the practical motivation of the partici-
pants was strengthened. Decomposition refers to the ability to break a complex problem
down into simpler parts so that it is easier to be handled. Among the simple problems
as the result of decomposition are often found patterns. The ability to find out similari-
ties or patterns among the decomposed problems in CT is called pattern recognition or
sometimes called generalization. Abstraction pertains to the skill to remove or filtering
out the characteristics of patterns that are not needed. Then, to execute and finish the
recognized pattern problems are needed algorithm design or a set of step-by-step in-
structions to lead and tackle the problems.
Based on these reviews, the research conducted through this proposed conceptual
framework can be evaluated by several approaches. For example, Chen et al. (2020)
employed the technology acceptance model (TAM) to evaluate the students’ experience
of using VR game applications and the learning effectiveness after the experiments
with CT concepts. On the other hand, Segura et al. (2020) used surveys to evaluate the
user experience, general performance, and effectiveness of the developed application
196 S. Sukirman et al.

system. The surveys were developed by the team to find out the result directly from the
participants. Hence, it can be stated that the evaluation strategy can be carried out by
many approaches based on the need of the aspects that will be known.

5. Conclusion

According to the results and discussions obtained from the 15 reviewed papers, we pro-
posed a conceptual framework to design an approach for learning CT through GBiVR
settings. We categorized it into three main parts, they are game elements, VR features,
and CT skills. The CT skills can be design in two ways, as learning outcomes and as
the strategy of learning. In this conceptual framework, we more emphasize that CT is
designed as the learning outcome. The study can be designed in experimental research
or exploratory with two groups, intervention, and control group, to find out the differ-
ences. To make more engagement, the development of VRGACT application system
should consider the elements of game and VR by optimizing the software and hardware
function. Hence, students can be more engaged and enjoy the learning environment.
This study contributes to the future design of the GBiVR framework to conduct a re-
search related to the topics of CT, GBL, VR technology and its application. However,
the articles that can be reviewed in this study are limited since the research on learning
CT through GBiVR is still rarely conducted or perhaps currently in progress. This con-
ceptual framework is constructed based on the previous studies. Therefore, experiments
to students as the real participants are necessary to confirm the proposed framework and
find out the impacts and limitations.

Acknowledgments

Thanks to Rector of Universitas Muhammadiyah Surakarta (UMS) and the Dean of

Faculty of Teacher Training and Education UMS for supporting this study. The study is
fully funded by Universitas Muhammadiyah Surakarta. This study is a part Doctor of
Philosophy program in Information Technology Education, Faculty of Art, Computing,
and Creative Industry, Universiti Pendidikan Sultan Idris, Malaysia.

References

Banic, A., Gamboa, R. (2019). Visual Design Problem-based Learning in a Virtual Environment Improves
Computational Thinking and Programming Knowledge. 2019 IEEE Conference on Virtual Reality and 3D
User Interfaces (VR), 1588–1593. https://doi.org/10.1109/VR.2019.8798013
Bavera, F., Quintero, T., Daniele, M., Buffarini, F. (2020). Computational Thinking Skills in Primary Teach-
ers: Evaluation Using Bebras BT – Computer Science – CACIC 2019 (P. Pesado & M. Arroyo (eds.); pp.
405–415). Springer International Publishing.
Berns, C., Chin, G., Savitz, J., Kiesling, J., Martin, F. (2019). MYR: A Web-Based Platform for Teaching
Coding Using VR. Proceedings of the 50th ACM Technical Symposium on Computer Science Education,
77–83. https://doi.org/10.1145/3287324.3287482
 A Strategy of Learning Computational Thinking through Game Based in ... 197

Cetin, I. (2016). Preservice Teachers’Introduction to Computing: Exploring Utilization of Scratch. Journal of Ed-
ucational Computing Research, 54(7), 997–1021. https://doi.org/10.1177/0735633116642774
Chen, S.-Y., Lai, Y.-H., Lin, Y.-S. (2020). Research on Head-Mounted Virtual Reality and Computational
Thinking Experiments to Improve the Learning Effect of AIoT Maker Course: Case of Earthquake Relief
Scenes. Frontiers in Psychology, 11, 1164. https://doi.org/10.3389/fpsyg.2020.01164
Chou, P.-N. (2019). Using ScratchJr to Foster Young Children’s Computational Thinking Competence: A Case
Study in a Third-Grade Computer Class. Journal of Educational Computing Research, 0735633119872908.
https://doi.org/10.1177/0735633119872908
Cleto, B., Sylla, C., Ferreira, L., Moura, J.M. (2020). CodeCubes: Coding with Augmented Reality. In R.
Queirós, F. Portela, M. Pinto, & A. Simões (Eds.), First International Computer Programming Education
Conference (ICPEC 2020) (Vol. 81, pp. 7:1–7:9). Schloss Dagstuhl – Leibniz-Zentrum für Informatik.
https://doi.org/10.4230/OASIcs.ICPEC.2020.7
Connolly, T.M., Boyle, E.A., MacArthur, E., Hainey, T., Boyle, J.M. (2012). A systematic literature review
of empirical evidence on computer games and serious games. Computers & Education, 59(2), 661–686.
https://doi.org/https://doi.org/10.1016/j.compedu.2012.03.004
Cuny, J., Snyder, L., Wing, J.M. (2010). Demystifying computational thinking for non-computer scientists.
Unpublished Manuscript in Progress, Referenced in
http://Www.Cs.Cmu.Edu/CompThink/Resources/TheLinkWing.Pdf
Dengel, A. (2018). Seeking the Treasures of Theoretical Computer Science Education: Towards Educational
Virtual Reality for the Visualization of Finite State Machines. 2018 IEEE International Conference on
Teaching, Assessment, and Learning for Engineering (TALE), 1107–1112.
https://doi.org/10.1109/TALE.2018.8615288
Dengel, A. (2020). How Important is Immersion for Learning in Computer Science Replugged Games? Pro-
ceedings of the 51st ACM Technical Symposium on Computer Science Education, 1165–1171.
https://doi.org/10.1145/3328778.3366837
Fagerlund, J., Häkkinen, P., Vesisenaho, M., Viiri, J. (2020). Assessing 4th Grade Students’ Computational
Thinking through Scratch Programming Projects. Informatics in Education, 19(4), 611–640.
https://doi.org/10.15388/infedu.2020.27
Feng, Z., González, V.A., Amor, R., Lovreglio, R., Cabrera-Guerrero, G. (2018). Immersive virtual reality
serious games for evacuation training and research: A systematic literature review. Computers & Educa-
tion, 127, 252–266. https://doi.org/https://doi.org/10.1016/j.compedu.2018.09.002
Hooshyar, D., Malva, L., Yang, Y., Pedaste, M., Wang, M., Lim, H. (2021). An adaptive educational computer
game: Effects on students’ knowledge and learning attitude in computational thinking. Computers in Hu-
man Behavior, 114, 106575. https://doi.org/https://doi.org/10.1016/j.chb.2020.106575
Hsu, T.-C., Chang, S.-C., Hung, Y.-T. (2018). How to learn and how to teach computational thinking: Sugges-
tions based on a review of the literature. Computers & Education, 126, 296–310.
https://doi.org/https://doi.org/10.1016/j.compedu.2018.07.004
Huang, W., Looi, C.-K. (2020). A critical review of literature on “unplugged” pedagogies in K-12 computer
science and computational thinking education. Computer Science Education, 1–29.
https://doi.org/10.1080/08993408.2020.1789411
Ilic, U., Haseski, H.İ., Tugtekin, U. (2018). Publication trends over 10 years of computational thinking re-
search. Contemporary Educational Technology, 9(2), 131–153.
https://doi.org/10.30935/cet.414798
Jin, Q., Liu, Y., Yuan, Y., Yarosh, L., Rosenberg, E.S. (2020). VWorld: An Immersive VR System for Learn-
ing Programming. Proceedings of the 2020 ACM Interaction Design and Children Conference: Extended
Abstracts, 235–240. https://doi.org/10.1145/3397617.3397843
Kao, D. (2019). JavaStrike: A Java Programming Engine Embedded in Virtual Worlds. Proceedings of the
14th International Conference on the Foundations of Digital Games.
https://doi.org/10.1145/3337722.3341828
Kemdikbud, D. (2019). Kesiapan Sekolah Menengah Pertama dan Sekolah Menengah Atas dalam Menerap-
kan Informatika sebagai Mata Pelajaran pada Tahun 2019/2020.
Lai, Y.-H., Chen, S.-Y., Lai, C.-F., Chang, Y.-C., Su, Y.-S. (2019). Study on enhancing AIoT computational
thinking skills by plot image-based VR. Interactive Learning Environments, 1–14.
https://doi.org/10.1080/10494820.2019.1580750
Leonard, A.E., Daily, S.B., Jörg, S., Babu, S.V. (2020). Coding moves: Design and research of teaching com-
putational thinking through dance choreography and virtual interactions. Journal of Research on Technol-
ogy in Education, 1–19. https://doi.org/10.1080/15391523.2020.1760754
198 S. Sukirman et al.

Liberati, A., Altman, D.G., Tetzlaff, J., Mulrow, C., Gøtzsche, P.C., Ioannidis, J.P.A., Clarke, M., Devereaux,
P. J., Kleijnen, J., Moher, D. (2009). The PRISMA statement for reporting systematic reviews and meta-
analyses of studies that evaluate healthcare interventions: explanation and elaboration. BMJ, 339, b2700.
https://doi.org/10.1136/bmj.b2700
Lin, L., Parmar, D., Babu, S.V, Leonard, A.E., Daily, S.B., Jörg, S. (2017). How Character Customization
Affects Learning in Computational Thinking. Proceedings of the ACM Symposium on Applied Perception.
https://doi.org/10.1145/3119881.3119884
Lombard, M., Ditton, T. (1997). At the Heart of It All: The Concept of Presence. Journal of Computer-Medi-
ated Communication, 3(2). https://doi.org/10.1111/j.1083-6101.1997.tb00072.x
Malva, L., Hooshyar, D., Yang, Y., Pedaste, M. (2020). Engaging Estonian primary school children in compu-
tational thinking through adaptive educational games: A qualitative study. 2020 IEEE 20th International
Conference on Advanced Learning Technologies (ICALT), 188–190.
https://doi.org/10.1109/ICALT49669.2020.00061
Nguyen, V.T., Zhang, Y., Jung, K., Xing, W., Dang, T. (2020). VRASP: A Virtual Reality Environment for
Learning Answer Set Programming BT – Practical Aspects of Declarative Languages (E. Komendants-
kaya & Y. A. Liu (eds.); pp. 82–91). Springer International Publishing.
Nietfeld, J. L. (2020). Predicting transfer from a game-based learning environment. Computers & Education,
146, 103780. https://doi.org/https://doi.org/10.1016/j.compedu.2019.103780
Nouri, J., Zhang, L., Mannila, L., Norén, E. (2020). Development of computational thinking, digital com-
petence and 21st century skills when learning programming in K-9. Education Inquiry, 11(1), 1–17.
https://doi.org/10.1080/20004508.2019.1627844
Paavilainen, J. (2020). Defining Playability of Games: Functionality, Usability, and Gameplay. Proceedings
of the 23rd International Conference on Academic Mindtrek, 55–64.
https://doi.org/10.1145/3377290.3377309
Parmar, D., Isaac, J., Babu, S.V, D’Souza, N., Leonard, A.E., Jörg, S., Gundersen, K., Daily, S. B. (2016).
Programming moves: Design and evaluation of applying embodied interaction in virtual environments
to enhance computational thinking in middle school students. 2016 IEEE Virtual Reality (VR), 131–140.
https://doi.org/10.1109/VR.2016.7504696
Pearl, H., Arrants, S., Swanson, H., Trninic, D. (2020). The AL Goldberg Machine: A Virtual Environment
for Engaging Learners in Algorithmic Practices. Proceedings of the 2020 ACM Interaction Design and
Children Conference: Extended Abstracts, 211–216. https://doi.org/10.1145/3397617.3397845
Radianti, J., Majchrzak, T.A., Fromm, J., Wohlgenannt, I. (2020). A systematic review of immersive virtual
reality applications for higher education: Design elements, lessons learned, and research agenda. Comput-
ers & Education, 147, 103778.
https://doi.org/https://doi.org/10.1016/j.compedu.2019.103778
Román-González, M., Pérez-González, J.-C., Jiménez-Fernández, C. (2017). Which cognitive abilities un-
derlie computational thinking? Criterion validity of the Computational Thinking Test. Computers in Hu-
man Behavior, 72, 678–691. https://doi.org/https://doi.org/10.1016/j.chb.2016.08.047
Saad, A. (2020). Students’ Computational Thinking Skill through Cooperative Learning Based on Hands-on,
Inquiry-based, and Student-centric Learning Approaches. Universal Journal of Educational Research,
8(1), 290–296.
Salvador-Ullauri, L., Acosta-Vargas, P., Luján-Mora, S. (2020). Web-Based Serious Games and Accessibil-
ity: A Systematic Literature Review. In Applied Sciences (Vol. 10, Issue 21).
https://doi.org/10.3390/app10217859
Segura, R.J., del Pino, F.J., Ogáyar, C.J., Rueda, A.J. (2020). VR-OCKS: A virtual reality game for learning
the basic concepts of programming. Computer Applications in Engineering Education, 28(1), 31–41.
https://doi.org/https://doi.org/10.1002/cae.22172
Selby, C., Woollard, J. (2013). Computational thinking: the developing definition. University of Southampton
(E-prints). https://eprints.soton.ac.uk/356481/
Selzer, M.N., Gazcon, N.F., Larrea, M.L. (2019). Effects of virtual presence and learning outcome using low-
end virtual reality systems. Displays, 59, 9–15.
https://doi.org/https://doi.org/10.1016/j.displa.2019.04.002
Shi, A., Wang, Y., Ding, N. (2019). The effect of game–based immersive virtual reality learning environment
on learning outcomes: designing an intrinsic integrated educational game for pre–class learning. Interac-
tive Learning Environments, 1–14. https://doi.org/10.1080/10494820.2019.1681467
Tang, K.-Y., Chou, T.-L., Tsai, C.-C. (2020). A Content Analysis of Computational Thinking Research: An
 A Strategy of Learning Computational Thinking through Game Based in ... 199

International Publication Trends and Research Typology. The Asia-Pacific Education Researcher, 29(1),
9–19. https://doi.org/10.1007/s40299-019-00442-8
Turchi, T., Fogli, D., Malizia, A. (2019). Fostering computational thinking through collaborative game-based
learning. Multimedia Tools and Applications, 78(10), 13649–13673.
https://doi.org/10.1007/s11042-019-7229-9
Usability First. (2020). Playability (Glossary). Foraker Labs.
https://www.usabilityfirst.com/glossary/playability/
Wing, J.M. (2006). Computational Thinking. Commun. ACM, 49(3), 33–35.
https://doi.org/10.1145/1118178.1118215
Wohlin, C. (2014). Guidelines for Snowballing in Systematic Literature Studies and a Replication in Soft-
ware Engineering. Proceedings of the 18th International Conference on Evaluation and Assessment in
Software Engineering. https://doi.org/10.1145/2601248.2601268
Yang, K., Liu, X., Chen, G. (2020). The Influence of Robots on Students? Computational Thinking: A Litera-
ture Review. International Journal of Information and Education Technology, 10(8).
Yu, J., Roque, R. (2019). A review of computational toys and kits for young children. International Journal
of Child-Computer Interaction, 21, 17–36.
https://doi.org/https://doi.org/10.1016/j.ijcci.2019.04.001
Zhang, L., Nouri, J. (2019). A systematic review of learning computational thinking through Scratch in K-9.
Computers & Education, 141, 103607.
https://doi.org/https://doi.org/10.1016/j.compedu.2019.103607
Zhao, W., Shute, V.J. (2019). Can playing a video game foster computational thinking skills? Computers & Ed-
ucation, 141, 103633. https://doi.org/https://doi.org/10.1016/j.compedu.2019.103633

S. Sukirman is a staff of Informatics Engineering Department, Faculty of Teacher

Training and Education, Universitas Muhammadiyah Surakarta (UMS), Indonesia. He
takes Doctor of Philosophy program in Information Technology Education, Faculty of
Art, Computing, and Creative Industry, Universiti Pendidikan Sultan Idris (UPSI), Ma-
laysia. His research interests are computational thinking, game-based learning, virtual
and augmented (mixed) reality for educational purposes.
L.F.M. Ibharim is a senior lecturer at the Computing Department, Faculty of Art,
Computing and Creative Industry, Universiti Pendidikan Sultan Idris (UPSI). She has
graduated with her Ph.D. in Multimedia in Education. She has been teaching game de-
sign and development courses from 2011 until the present. Various agencies are often
invited her as a consultant and trainee for teaching and learning courses, especially
for gamification and game-based learning. Her interests are any research projects or
practices in Game Design and Development, Game-Based Learning, Gamification,
Human-Computer Interaction, Usability Engineering, and Multimedia in Education,
specifically designing technologies and content creation.
200 S. Sukirman et al.

C.S. Said is a senior lecturer of the Deparment of Computing, Faculty of Art, Com-
puting and Creative Industry, Universiti Pendidikan Sultan Idris. He holds master of
science in computer science from Universiti Putra Malaysia, and Ph.D degree in in-
structional technology from Universiti Sains Malaysia. His research interest includ-
ing study the impact of virtual environment, gaming and multimedia technologies in
education.
B. Murtiyasa is a Professor of Mathematics Education, Universitas Muhammadiyah
Surakarta, Indonesia. Research interests in the field of ICT in education include the
use of technology to improve the quality of learning and its management. Some of the
research topics that have been carried out include the development of smart classes,
ICT-based multimedia development, and mobile learning. Currently doing research on
gamification in mathematics learning. He is also active in providing training for young
lecturers in the field of learning technology.