This article has been accepted for inclusion in a future issue of this journal.

Content is final as presented, with the exception of pagination.

IEEE TRANSACTIONS ON EDUCATION 1

Computational Thinking and User

Interfaces: A Systematic Review
Sara Rijo-García , Eduardo Segredo , Member, IEEE, and Coromoto León

Abstract—Contribution: This document presents a systematic

bibliographic review that demonstrates the need to conduct
research on how the user experience impacts the development
of computational thinking.
Background: In the field of computer science, computational
thinking is defined as a method that enhances problem-solving
skills, system design, and human behavior understanding. Over
the last few decades, several tools have been proposed for the
development of computational thinking skills; however, there is
no area of study that evaluates the implications or the impact
that these types of platforms have on users belonging to any
knowledge area.
Research Question: Do user interfaces influence the develop-
ment of computational thinking skills? Fig. 1. Computational thinking skills.
Methodology: To address this issue, a systematic review of the
literature was conducted using the preferred reporting items for
systematic reviews and meta-analyses (PRISMA) methodology
for analyzing and evaluating scientific publications. thinking, algorithmic thinking, design thinking, computational
Findings: The results show that despite the dearth of litera- thinking and others [1]—each one with cognitive processes
ture on the subject, the specific design of a user interface has a that are not mutually exclusive and that can be used to solve
significant impact on the development of computational thinking. a given problem.
Bearing the above in mind, it is necessary to conduct research
that delves more deeply into the effects caused by the technolo- Specifically, computational thinking shares certain similari-
gies that are used to develop computational thinking, this being ties with other models of thinking, since it can be regarded as a
a line of research that is worthy of consideration. technique that includes the skills involved in problem solving,
Index Terms—Computational thinking, human–computer systems design and understanding human behavior [2].
interaction, preferred reporting items for systematic reviews and Based on these characteristics, Aho [3] proposed com-
meta-analyses (PRISMA), survey, systematic review, usability, putational thinking as a research method to generate new,
user experience, user interface, visual programming. underlying computing models that are suitable for formulat-
ing problems. Elsewhere, Alan Perlis promoted the idea that
programming could serve as a mental tool to understand and
solve any problem at the university level [4]; while Papert, a
I. I NTRODUCTION pioneer of constructionist learning, introduced the use of tech-
HINKING is a cognitive process that relies on learning nology in teaching children, arguing that programming could
T and previous experience, and is responsible for generating
ideas and concepts or understanding situations. This process
be taken as an active learning process based on understand-
ing a problem in a practical way, providing students with a
entails a set of skills that human beings develop throughout way of thinking based on their own learning [5]. But it was
their lives and that can be promoted by applying different really at the beginning of this century that computational think-
models of thought. In the context of computer science, there ing came into its own. Wing [2] stressed the importance of
are different ways of thinking—mathematical thinking, critical developing computational thinking skills by focusing on the
abilities of students, and not only on mastering certain pro-
Manuscript received April 20, 2021; revised December 5, 2021 and gramming languages. Later, Zapata-Ros [6], building on the
February 28, 2022; accepted March 10, 2022. This work was supported in concepts of Papert and Wing, consolidated the idea that using
part by the Cabildo de Tenerife. Fundación General de la Universidad de
La Laguna through the “Piens@ Computacion@ULLmente” project (REF learning environments from the earliest educational stages fos-
21120050). Programa educativo para el fomento del pensamiento computa- ters the skills involved in computational thinking. It may thus
cional a través de la realizacón de actividades que permitan su desarrollo y be said that computational thinking is divided into six skills
su inclusión en el currículo” (Educational program to encourage computa-
tional thinking by engaging in activities that allow for its development and (see Fig. 1) as follows.
its inclusion in the curriculum). (Corresponding author: Sara Rijo-García.) 1) Decomposition: Divide the problem into smaller
The authors are with the Departamento de Ingeniería Informática y de problems.
Sistemas, Universidad de La Laguna, 38200 San Cristóbal de La Laguna,
Spain (e-mail: [email protected]; [email protected]; [email protected]). 2) Pattern Recognition: Find similarities inside or outside
Digital Object Identifier 10.1109/TE.2022.3159765 the problem situation.
This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

2 IEEE TRANSACTIONS ON EDUCATION

TABLE I
3) Abstraction: Simplify a complex problem by focusing S ELECTED K EYWORDS
only on the relevant details.
4) Algorithm: Define the steps in order needed to solve the
problem.
5) Coding: Develop a program in a language that the
computer understands based on the algorithm.
6) Debugging: Correct errors or improve the program.
Since then, multiple tools have been developed to promote
and enhance computational thinking skills and algorithmic
thinking. These tools offer a series of options that allow the
user to carry out a plethora of actions to solve a given problem;
as a result, this could at first lead to a steep learning curve
due to the complexity of the interface. Consequently, having an
adequately designed user interface allows the student to focus
on the information and the task to be completed in order to
satisfy their objectives [7], while a confusing and inefficient
Thus, the main objective of this document is to con-
design makes it difficult for people to do work and leads to
duct a systematic analysis of the literature to see if there
more mistakes, causing frustration and stress for the user [8].
are articles that show how the different user interfaces
Because of this, user interfaces should be user friendly, easy to
of these tools influence the development of computational
understand, and they should satisfy the needs of users while
thinking skills. The authors’ intention is to identify the
making their tasks easier [7]. Usability [9], along with user
results that exist in the field of study that combines com-
experience, is the key point when designing and developing
puter science, human–computer interaction and education, in
accessible interfaces.
order to determine the advantages and shortcomings in the
The user interface is the means by which people can com-
design and development of platforms in the field of com-
municate and interact with a computer system. There are
putational thinking, and establish a line of research in this
different categories of user interfaces; however, depending
field.
on how the user interacts with the system, a distinction can
be made between command line interface, graphical user
interface, natural user interface, voice user interface, tangible III. M ETHODOLOGY
user interface, and others [10].
To ascertain this state of the art, a systematic review was
The design of the user interface must make the interaction
conducted that relied on the “preferred reporting items for
between the user and system efficient and effective, which
systematic reviews and meta-analyses” (PRISMA) [16] method-
is why the design should be developed using quality stan-
ology for analyzing and evaluating scientific publications. The
dards so as to optimize usability and the user experience. It
first step was to establish the research question in order to
is thus highly recommended to follow user-centered design
frame the field of study. The goal is to study the impact of
perspectives when developing interfaces [11], [12].
user interfaces and how they affect the experience of users who
For these reasons, it is important that environments for
employ technology to develop the skills that are provided by
learning programming exhibit a friendly interface that pro-
computational thinking. This allowed us to posit the following
motes aspects of computational thinking in students in early
question:
stages of education [13] since, as noted in the previous para-
graph, if the interface requires a high cognitive load [14], users Do user interfaces influence the development of
could have a negative experience with the activity, which could computational thinking skills?
affect student learning [15]. Table I defines the search terms, as well as their possible
This article seeks to present a systematic study of the synonyms and abbreviations. Once the set of keywords was
literature in order to undertake research aimed at determin- determined, the databases that were best suited to this study
ing how the user experience, as it relates to the interfaces were selected, which were as follows.
used, impacts the development of the skills that computational 1) Association for Computing Machinery (ACM) Digital
thinking provides. Library (https://dl.acm.org/).
2) Digital Bibliography and Library Project (DBLP)
Computer Science Bibliography (https://dblp.org/).
II. O BJECTIVE 3) Institute of Electrical and Electronics Engineers (IEEE)
Over the last few decades, a large amount of research has Xplore (https://ieeexplore.ieee.org/Xplore/home.jsp).
been conducted aimed primarily at defining computational 4) ScienceDirect (https://www.sciencedirect.com/).
thinking, developing different tools to promote these skills, and 5) Scopus (https://www.scopus.com/home.uri).
describing methodologies to apply in curricula at various edu- 6) Web of Science (https://www.webofscience.com/wos/
cational stages. However, there are still large empirical gaps alldb/basic-search).
involving the usability and efficiency of the platforms used to The next step was to define the query to carry out in the
develop and promote computational thinking skills. different databases
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

RIJO-GARCÍA et al.: COMPUTATIONAL THINKING AND USER INTERFACES: A SYSTEMATIC REVIEW 3

Fig. 2. Flow diagram of the PRISMA methodology.

("Models of thinking" OR "Computational e) Papers that may describe an experiment or study

Thinking") AND ("User Interface" OR "UI" with proven results.
OR "User Experience" OR "UX") AND ("Visual f) Papers whose subject is related to computational
Programming Languages" OR "VPL" OR "STEM"
OR "Education" OR "Human Computer thinking and user interfaces.
Interaction" OR "HCI" OR *Thinking).
Finally, exclusion and inclusion criteria were established to A. Search, Screening, and Selection Process Carried Out in
determine the quality of the articles to be studied. the Different Stages
1) Exclusion Criteria: After performing different searches in the databases men-
a) Duplicate papers. tioned above, a total of 1069 records were obtained, of which
b) Papers that do not match the search terms. 450 were duplicate studies. This resulted in a total of 619 stud-
c) Papers whose subject refers to other aspects of ies after eliminating duplicates. During the screening phase,
design. 407 papers were excluded based on the title and keywords,
d) Papers that focus purely on aspects of robotics. and 119 based on the abstract. In the eligibility phase, of the
e) Papers written in a language other than English or remaining 93 studies, a total of 71 studies that did not satisfy
Spanish. the inclusion criteria were excluded. In the end, 22 papers
f) Papers consisting solely of an abstract. were found that were relevant to this study (see Fig. 2).
2) Inclusion Criteria:
a) No time restriction is applied.
b) Papers written in Spanish or English. IV. R ESULTS
c) Peer-reviewed papers. Once the screening phase was completed, 22 papers were
d) Papers from journals. selected in order to conduct a more in-depth analysis. The
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

4 IEEE TRANSACTIONS ON EDUCATION

so as to encourage aspects of computational thinking. In this

case, an environment is deemed to be appealing and interesting
based on its user interface. If said interface is not intuitive or
eye-catching, the environment may not be viewed in the same
light, which could detract from the positive aspects related to
the user experience on the platform.
App Inventor for Android (Report From a Summer
Camp [17]): This article details the experience of using App
Inventor in a summer camp with high school students. The
author provides a detailed description of the design of these
camps, including the process of selecting and configuring an
Android device. Furthermore, an evaluation process was con-
ducted with the students through surveys to determine the
Fig. 3. Graph showing the selected papers grouped by year of publication.
effects of this type of tool compared to other, less intangi-
ble, tools. In addition to the experience with the platform,
the interesting thing about this article is that it presents the
advantages of this tool compared to others, as well as the
improvements that the platform would need to increase its
usability and improve the user experience.
Elsewhere, Roy highlighted as advantages of App Inventor
its fusion of the best tangible and intangible interfaces, it com-
Fig. 4. Word cloud of the categories into which the papers analyzed have plements the learning experience of users who have already
been classified. used Scratch or Alice, and for computer science students, it can
serve as a basis for the later study of programming languages
such as Java.
resulting study period ranged from 2010 to 2021, and can thus Improvements include social interaction and sharing
be regarded as involving novel and current areas of research. between users, developing an easily accessible media library
It should be noted that the number of papers on the subject like Scratch, improving the design of the user interface with
has been increasing over the years, and most of the articles a single window so that elements are always visible, and
selected for the study are from recent years (see Fig. 3). improving the usability of the implementation.
The papers selected are those that best adapt to the research Interfaces for Thinkers (Computer Input Capabilities That
question that the authors are interested in solving, and they can Support Inferential Reasoning [14]): This article considers
be classified into various categories, which are not mutually the implications of designing effective computational think-
exclusive, and where the same article could be included in ing tools. One of the fundamental goals of this research is
several categories (see Fig. 4). to determine if inferential thinking deteriorates or improves
The results of this analysis are presented in the sections that depending on the cognitive load of the interface used.
follow. To achieve an optimal design of the interfaces of educa-
Machines for Thinking [13]: The article focuses on how tional tools, one where learning prevails, it is essential for
learning environments for programming should be appealing said interface to have a low cognitive load. To this end,
and interesting in order to promote aspects of computational Sharon Oviatt, the author of this research, did a comparative
thinking in students in the early stages of education. study contrasting the inference that results from using different
It lays out a background by taking a tour of the first learning interfaces.
tools for programming, which later transformed into environ- This study showed that inferential precision is affected by
ments designed, especially, for young students. The precursor interfaces that facilitate greater distractions, since the user
was provided by Papert and the Logo language. cannot focus on solving the problem at hand.
The authors believe that this type of platform incorpo- Currently, the use of digital tools is widespread in all areas
rates both an intellectual relationship with the learning that of society. It is thus of great importance to understand how the
takes place, and a social relationship with the users, where design of these tools impacts certain aspects of human cogni-
each system is capable of presenting a theory for teaching tion. The long-term goal of this research is to design computer
programming. interfaces that act as thinking tools to improve learning.
These types of tools offer great advantages due to the ubiq- To conclude, the authors note that user interfaces that are
uitousness of the computers present in many of the classrooms not designed with the cognitive load they place on the student
at a large number of schools compared to other, higher cost in mind make the user unable to focus on the solution of
alternatives, as well as due to the communities that have been the problem at hand, thus potentially frustrating the learning
created to share the educational experiences of students and process.
teachers alike. TUI, GUI, HUI (Is a Bimodal Interface Truly Worth the
With the above in mind, it could be said that environments Sum of Its Parts? [18]): This article conducts a comparative
for learning programming should be appealing and interesting study of three types of interfaces dedicated to the development
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

RIJO-GARCÍA et al.: COMPUTATIONAL THINKING AND USER INTERFACES: A SYSTEMATIC REVIEW 5

of computational thinking: 1) tangible interfaces; 2) hybrid apply specific measures to adapt the improvements to the user
interfaces; and 3) graphic interfaces. The study was carried interface.
out on a sample of boys and girls between five and six years Measuring the Usability and Capability of App Inventor to
old. It yielded promising results for tangible interfaces, as Create Mobile Applications [21]: The goal of this study is
they help to enhance computational thinking skills, although to evaluate the usability of App Inventor to develop applica-
graphic interfaces obtained higher scores in the evaluation of tions for mobile devices. To this end, the environment must be
repetition structures. intuitive, usable and functional so that basic users can create
The statistics on the results show that the group of tan- simple applications, and higher level users can develop more
gible interfaces had a higher proportion of correct answers advanced applications.
in the majority of programming tasks, whereas the graphical In order to carry out this research, the applications from a
interfaces obtained the highest score when evaluating repeating sample of 5228 resources were categorized and filtered. The
structures. findings indicate that the interface is limited by the learning
As a final finding, the strategies used by students to solve resources, and that there is thus a relationship between the
problems are closely related to the learning interface used, usability of the interfaces and the tutorials proposed to carry
meaning it is essential to conduct research in this field in order out the activity.
to determine how these technologies impact computational Floors and Flexibility (Designing a Programming
thinking learning. Environment for 4th-6th Grade Classrooms [22]): Several
Students’ Experiences From the Use of MIT App Inventor studies focused on Human–Computer Interaction show the
in Classroom [19]: In this article, Perdikuri described how differences between how children interact compared to adults.
the App Inventor works and its use as a means of developing It is these differences that must therefore be kept in mind
computational skills among secondary school students. when designing visual tools for programming, since the
In student evaluations of their experience with the tool, development interface plays a fundamental role, as it aims to
more than 50% of the students said that App Inventor pro- promote change in the students’ roles as they transition from
vides a fairly usable development environment. In noting the being mere users to developers. As a result, this paper argues
significant aspects of this tool, students deemed it important for the need to have a tool that can be adapted to different
to develop mobile applications that are simple and accessible educational levels, making the user focus on the project to be
to everyone, a Web-based platform and a user-friendly visual developed without becoming lost learning the interface.
environment for the development of user interfaces. The stu- To do this, the authors of this article developed an environ-
dents also noted the greater complexity of the block editor ment that can reveal sections of the interface as the student
view. progresses in their learning, so as not to cognitively saturate
Despite having used a small study sample, and in the the student. As an alternative, the environment can be made
absence of studies involving larger groups, the author was able completely visible so that users can develop their creativity by
to state that App Inventor is a good tool for developing com- engaging in unscripted projects.
putational thinking skills since, due to the visual nature of In developing this platform, the authors established the
the platform, it lets users focus more on solving the prob- following design principles.
lems and less on the language syntax, which minimizes the 1) Support multiple task types.
programming learning process. 2) Require age-appropriate content.
As a line of future work, the author mentions conducting a 3) Include an age-appropriate interface.
comparative study of the App Inventor and other programming 4) Aid the project developers.
environments, which are used to introduce programming to These design principles rely on existing visual programming
high school students. languages based on blocks, as well as on research during the
In this article, the author addresses the need to use visual pilot implementation phase. Likewise, the authors identified
programming environments with an interface that helps users inconsistencies in the designs of programming environments,
develop computer and computational thinking skills, by mak- such as Scratch, ScratchJr, and Blockly. This led them to
ing it possible for users to focus on solving the problem in carry out a comparison applying the design principles outlined
question, and not just on the code. above, and which they also used to develop their own environ-
Entry (Visual Programming to Enhance Children’s ment based on the strengths of each of the interfaces studied.
Computational Thinking [20]): This article introduces Entry, With the above in mind, the design criteria established by the
a visual programming platform based on HTML5 that authors yielded an improved user experience and encouraged
encourages the development of computational thinking in computational thinking, specifically in fourth- and sixth-grade
learners. students.
It features a simple and intuitive block-based interface. In the future, the authors plan to conduct further stud-
Through this friendly environment, it allows the user to engage ies, including the benefits provided by a visual development
in problem-based learning that spans the basic principles environment and how learning in these environments affects
of programming. Furthermore, since it is Web-based, it is educators, since many teachers do not usually have previous
adaptive and can be easily used on mobile devices. experience teaching computational thinking.
In future research, the authors propose conducting qual- This article clearly shows the need to have programming
itative surveys to study the usability of the interface and environments that are tailored to cognitive skills, since many
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

6 IEEE TRANSACTIONS ON EDUCATION

block-based tools have features in their user interfaces that of tangible interfaces versus graphics on the performance of
are too complex and not suited to the development of students computational thinking skills in boys and girls between the
ranging in age from nine to eleven. ages of five and seven.
Using Computational Thinking Patterns to Scaffold The researchers focused specifically on ScratchJr as a graph-
Program Design in Introductory Programming Course [23]: ical interface, and the KIBO Robot as the tangible interface.
Chang, the author of this research, has created a system that The results showed significant differences between the two
monitors student development in a block-based programming types of interface; however, for this sample, the group that
environment. This system covers the deficiencies present in worked with the KIBO robot obtained better scores than the
certain user interfaces by suggesting to the student a snippet group that worked with ScratchJr, especially in the sequenc-
of code that can help them, such that the user can progress in ing and debugging activities. This may be due to the nature of
their learning without obsessing over the multitude of options the interfaces proposed. Since the KIBO Robot is a tangible
that these interfaces offer. interface, it allows children to use their own hands to under-
This could be one possible solution in terms of the devel- stand how the actions associated with the blocks are translated
opment of computational thinking by relying on block-based into the robot’s movements in a physical space. By contrast,
programming environments, since it would optimize the user ScratchJr, which is a graphical interface shown on a screen,
interface while also reducing its cognitive load. could pose some additional complexities or increase the level
Can Students Design Software? The Answer Is More of distractions, especially when one is unfamiliar with the
Complex Than You Think [24]: The author of this article development environment.
defined the design as a series of software elements, as well as In conclusion, this study provides evidence on the impact
the interaction between them. Hu argued that software design that both interface types have on student learning, which shows
should be an iterative process that improves the final design that the type of interface presented to the student influences, in
of a product. one way or another, the experiences of the children and, thus,
Therefore, a quality design must be able to adapt to changes on the development of skills associated with computational
in the initial requirements, as well as to other possible mod- thinking. Because of this, as the authors of the research assert,
ifications. This is why practitioners using agile software it is important to implement technologies that are tailored to
development methodologies may believe that design is not the different educational stages so as to promote the proper
only highly iterative, but emergent. Then, only coding, test development of computational thinking skills.
execution and code refactoring reveal the correct functioning Exploring Factors Influencing the Acceptance of Visual
of a good design. Programming Environment Among Boys and Girls in Primary
In conclusion, the author points out that in order to obtain Schools [26]: This study adapts the “Technology Acceptance
stable, adaptable and durable designs over time, the teaching Model” (TAM), by Davis, Bagozzi and Warshaw, to study the
of design must be improved, such that any student finishing effect that visual programming environments have on primary
a computer science program should be able to design optimal school students. The model is based on four principles: per-
software. ceived utility, perceived ease of use, attitude and behavioral
Employing Retention of Flow to Improve Online intention.
Tutorials [15]: It is imperative that designers and developers The results of the research show that students tended to
create optimal platforms to promote learning in computer perceive the utility of the environment rather than the ease
science, since if users have a negative experience with the of use. It also determined that outside assistance is necessary
activity, it could have effects that are counterproductive to for the perceived ease of use. These results thus indicate that
the specified objective. In order to identify those parts where there is a link between external assistance and the ease of use
users normally leave the activity, methods are needed to perceived by students.
identify these points so they can be subsequently improved. The Computational Puzzle Design Framework (A Design
This article maintains that the development of platforms has Guide for Games Teaching Computational Thinking [27]):
to adhere to design norms, such as, for example, the clarity After a systematic investigation, the authors of this study con-
and simplicity of the interface, in order to allow for a proper clude that there is nothing in the literature on how to design
user–interface interaction in order to minimize user error. This and develop games to teach computational thinking.
thus promotes the learning of computer science since, if users This article tries to address the unknown that arises when
have negative experiences with the activity in question, effects developing this type of platform in an efficient and effective
contrary to the defined objective could result. It is because of way, and presents a design framework for game development,
this that in this case, the researchers created a tool that can which the authors of this research put into practice by applying
be used to measure the flow retention of students. This allows said framework to redesign a game that focuses on learning
identifying the cognitive loads in order to improve the design computational thinking.
of the user interface. Keeping the above in mind, using this As a result, a common framework should be established
type of tool could lead to qualitative improvements in both the for the design and development of games that teaches com-
user interface and user experience, enhancing computational putational thinking and that also allows the researcher to
thinking. measure the effect that these interface have on its develop-
The Impact of User Interface on Young Children’s ment, and determine the objective improvements that could be
Computational Thinking [25]: This article studies the impact made.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

RIJO-GARCÍA et al.: COMPUTATIONAL THINKING AND USER INTERFACES: A SYSTEMATIC REVIEW 7

As future work, the researchers propose testing this design According to the surveys conducted during the study, the
and development framework. game is usable, the design of the game is appealing, the col-
Fostering Computational Thinking Through Collaborative ors are representative and the game provides guided learning.
Game-Based Learning [28]: This study tries to determine if These are important characteristics in the design and develop-
the design of a game system can be used to help teach com- ment of user interfaces. As for the user experience, parameters,
putational thinking in a way that is fun and entertaining. To such as confidence, challenge, satisfaction, engagement, and
argue this hypothesis, the researchers in this study developed relevance were evaluated.
a tangible programming environment using mobile devices. On the other hand, that students are expected to easily
The most significant difference with other environments is grasp the basic concepts of computational thinking as they
that the system developed had a greater degree of abstraction play with the BlocklyScript platform. However, due to the
compared to other environments, such as Scratch. The experi- current COVID-19 pandemic, new teaching models have been
ment yielded positive results, suggesting future research where born and as future research, the authors propose improving the
certain improvements will be applied. design and features of BlocklyScript so that it can be used in
Computational Thinking With the Web Crowd Using nonface-to-face education.
CodeMapper [29]: In this article, a new programming plat- Finally, there is evidence that the user interface does affect
form, called CodeMapper, is presented. It can build com- the outcome of these abilities, since this platform was designed
putational logic in separate modules and aggregate them to taking into consideration user-centered design norms. As a
create complex modules, so that students can focus first on result, the authors were able to develop a platform with
programming logic and then on code development. an appealing design that focuses on usability while also
CodeMapper offers a visual interface that allows students promoting computational thinking skills.
to stay in a conceptual realm, similar to block-based program- Pixasso (A Development Stage-Based Learning Application
ming environments, and also to write software in a language, for Children [32]): In this article, Nandan et al. presented
such as Java, C++, or Python. a tool to teach programming by coloring the pixels of an
The platform is developed in ASP.NET Core 2.0 and React image. In terms of the user interface and user experience,
due to their flexibility, as it allows for incremental design. The this application was developed employing the user-centered
result is a dynamic user experience and interaction. design paradigm (in this case, child-centered design), which
In conclusion, the authors emphasize that CodeMapper is aims to improve the proposed skills in computational thinking
part of an overall system called MindReader, but can stand and encourage computer science education at an early age.
alone as a smart Web application with great potential for The Pixasso application was designed to be adaptive and
teaching computational thinking. to let students select the difficulty level for the various skills.
A Natural User Interface Implementation for an Interactive To validate the design of the application, the authors relied
Learning Environment [30]: This article shows a develop- on the Developmentally Situated Design (DSD) card, adding
ment environment called Create and Play based on natural cards applicable to the computational thinking use case, i.e.,
user interfaces. The TAM was used to study user impressions. problem solving, attention and instruction. They also applied
In addition, the Interface Style was included as an external the Touchscreen Interaction Design Recommendations for
variable of the research because, according to previous stud- Children framework to properly develop the application.
ies, the use of a given system can be affected by the style of As is apparent, this article provides a clear example of
the interface used, thus influencing the users’ attitude when how the user interfaces of tools for learning programming can
using the platform. affect the cognitive development of the user when attempting
After conducting an empirical study, the authors show that to solve a problem. In this case, the researchers employed a
the type of interface influences the ease of use of the tech- user-centered design that was validated using DSD cards. By
nology and is related to the usefulness and enjoyment of the doing so, the authors were able to identify potential problems
platform. Therefore, the authors claim that the use of natural in the tool and apply solutions to improve their design.
user interfaces in computational thinking provides additional Enhancing Computational Thinking Capability of Preschool
motivation for secondary school students. Children by Game-based Smart Toys [33]: This study proposes
BlocklyScript (Design and Pilot Evaluation of an RPG a system based on tangible user interfaces, using Arduino
Platform Game for Cultivating Computational Thinking Skills as the basis for assembling robot cars and colored cards for
to Young Students [31]): Karakasis and Xinogalos, the developing applications. The aim of the study is to investigate
authors of this article, present a new educational game called different teaching approaches for learning computational logic
BlocklyScript, which aims to teach basic programming con- and programming concepts.
cepts, algorithm design and error correction, skills that are The article builds on previous research that showed that
intrinsic to computational thinking. there are no major differences in learning usage between
To provide an improved gaming experience for the graphical and tangible interfaces; however, this study is based
user, the authors relied on design norms focused on on the hypothesis that users will engage more actively with a
usability, multimodality and entertainment. This translates tangible interface.
into an implementation of a gaming environment that is The learning difficulties were reduced and learner interest
easy to use and navigate and is compatible with various was increased thanks to facilitated entertainment scenarios
browsers. and user-friendly interfaces, promoting the improvement of
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

8 IEEE TRANSACTIONS ON EDUCATION

computational thinking skills. As a result, the cognitive load

associated with learning is lower, as students quickly become
accustomed to the interactive interface.
Comparing TUIs and GUIs for Primary School
Programming [34]: This study is part of broader research
that aims to analyze how different user interfaces can
support the development of computer skills in primary school
students, which is why the authors of this article focus on
comparing the use of graphical user interfaces and tangible
user interfaces.
The results suggest that graphical user interfaces can be
more effective for teaching the basic concepts of program-
ming, while tangible user interfaces can be more effective
for teaching computer science to younger students. Thus,
Fig. 5. Graph showing the categories into which the papers analyzed are
both interfaces can be beneficial depending on the educational classified.
objective.
In this article, the authors consider the difference that
exists between tangible and graphical user interfaces when designing and developing tools that encourage computational
assimilating programming concepts, with tangible interfaces thinking and are suited to different educational levels, and to
being more popular among younger children, and graphical studying the impact that these platforms have in terms of the
interfaces being easier for older children to use. cognitive load that students can tolerate.
The results also show that the users of graphical interfaces This work shows that the type of interface used in platforms
exhibited significantly greater improvements than the users of that promote computational thinking affects student learning,
tangible interfaces in terms of learning. which makes it important to ascertain what types of interfaces
As a result, the type of interface used when teaching pro- are best suited to each educational stage. This makes it pos-
gramming can influence the development of computational sible to determine how these tools will be presented to the
thinking skills. user.
EEG-Based Cognitive Load Assessment in MATLAB GUI The authors’ review of the literature yielded several types
and Impact on Learning System [35]: This research focuses of interfaces: natural, tangible, graphic, and hybrid. All of
on reducing the cognitive load for computational thinking- them, depending on the degree of use, enhance computational
based learning. The aim is to build a graphical user interface thinking skills to some extent [14], [18], [25], [30], [34].
for MATLAB software in order to instantly measure cogni- It is also important to consider the cognitive load imposed
tive load based on the electroencephalograms of individuals. by these tools, since the user learning process could be
The user interface is easy to use, since it does not require frustrated, resulting in a negative experience and a counter-
programming knowledge in order to use it, and its objective productive outcome [14], [15], [23], [33], [35].
is to determine the cognitive load experienced by the user As noted by Jiang et al., since there is no design framework
when solving a given problem. According to the authors, this for developing this type of tool, it is difficult to establish a
research could be applied in the field of education to determine pattern that can be used to bridge or mitigate this gap [27].
the learning status of students. Therefore, to provide a better user experience, it is neces-
sary to follow certain design guidelines focusing on usability
and multimodality [31], this being an iterative process that
V. D ISCUSSION improves the final design of the product [24]. A proper eval-
Most of the results studied have been presented at inter- uation of a software prototype is a fundamental part of the
national conferences and congresses, and can be categorized development process, as it provides useful information on
into three areas: 1) Computer Science; 2) Human–Computer usability and identifies potential problems so that the software
Interaction; and 3) Education. The field of study of this can be improved later [36].
research is a combination of these three areas. In terms of the design of graphical user interfaces, the
In turn, this article presents a classification of the papers developers of computational thinking platforms should use
based on their content and the area of study that is considered a user-centered approach in order to achieve tools that are
herein, which were grouped into: visual programming envi- intuitive, usable and appealing [13], [22], [31], [32]. As
ronment, user interface, user experience, usability, cognitive per the ISO 9241-210:2019 standard on the “Ergonomics of
load, comparison of interfaces, type of interface, design, and human–system interaction—Part 210: Human-centered design
monitoring systems. The classification is shown in Fig. 5. for interactive systems,” six design principles are defined as
Despite the long history of the study of computational think- follows.
ing, in reality, the existing literature on the usability and 1) The design is based upon an explicit understand-
effectiveness of learning tools for programming is truly scarce. ing of users, tasks, and environments. It is important
Based on the studies analyzed, the authors of this article deter- to know the target audience for which the tool is
mined that there is a significant gap both when it comes to intended, the context, the needs and goals of the user,
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

RIJO-GARCÍA et al.: COMPUTATIONAL THINKING AND USER INTERFACES: A SYSTEMATIC REVIEW 9

as well as the tasks the user will carry out on the is an analysis of the state of the art involving the intersection
interface. between the areas of Computer Science, Human–Computer
2) Users are involved throughout the design and develop- Interaction, and Education, the goal of which is to identify
ment. This point suggests making the user a participant the scope of the research, trends and any existing gaps.
in the interface design process and learning their point The results showed that despite the relative dearth of lit-
of view, thus making it possible to design interfaces that erature on the relationship between user interfaces and the
are better suited to the user’s needs. development of computational thinking skills, it is safe to say
3) The design is driven and refined by user-centered eval- that user interfaces do affect the development of these skills.
uation. The user’s viewpoint is taken into account such There is also the need to conduct research that further ana-
that the user evaluates all the versions, from the start of lyzes the effect caused by the technologies employed in the
the development phase until the final version. development of these interfaces. This is thus a worthy area of
4) The process is iterative. It is not a linear process; rather, endeavor for future research.
the design is evaluated along the various phases. Over the course of this study, a series of more specific
5) The design addresses the whole user experience. Just as questions emerged as follows.
usability is an implicit part of this process, so must the 1) How do the interfaces of visual programming languages
user experience be. affect the development of computational thinking skills?
6) The design team includes multidisciplinary skills and 2) How does the design of a user interface influence the
perspectives. This is a process that should involve every development of computational thinking skills?
team member, not just designers or developers. 3) How does the user experience involving visual pro-
Various papers analyzed present a series of solutions that gramming tools affect the development of computational
can address some of the problems exhibited by the current thinking?
tools in terms of developing computational thinking. 4) What are the differences between the interfaces of visual
These alternatives include the implementation of technology programming languages?
adapted to the different educational stages [22], [25], systems 5) What improvements can be applied to visual program-
that measure the retention flow of students on the platform ming tools or platforms to make their designs more
and that allow measuring the cognitive loads that are imposed inclusive and accessible?
on students [15], and assistive technologies that are activated 6) Is it possible to improve the design of a user interface
when the user cannot find a solution to the problem [23]. by applying other thought models?
In conclusion, and in response to the hypothesis proposed The answer to these questions will provide a starting point
in this research, it may be deduced that user interfaces indeed for future research.
affect the development of computational thinking skills. A
more exhaustive research process would be beneficial in order
to ascertain the impact that these technologies have on the R EFERENCES
development of intrinsic computational thinking skills. As ana- [1] C. Frauenberger and P. Purgathofer, “Ways of thinking in informatics,”
lyzed in previous studies, there is a close relationship between Commun. ACM, vol. 62, no. 7, pp. 58–64, Jun. 2019. [Online]. Available:
solving a given problem and the learning interface used for https://dl.acm.org/doi/10.1145/3329674
[2] J. M. Wing, “Computational thinking,” Commun. ACM, vol. 49, no. 3,
this [13], [18], [21], [33], [37]. p. 33, Mar. 2006. [Online]. Available: http://portal.acm.org/citation.cfm?
This work is the first step, and involved a study to sys- doid=1118178.1118215
tematically map the literature to see if there are any papers [3] A. V. Aho, “Computation and computational thinking,”
Comput. J., vol. 55, no. 7, pp. 832–835, Jul. 2012.
that conclude that user interfaces influence the development of [Online]. Available: https://academic.oup.com/comjnl/article-
computational thinking skills. As the results show, there are lookup/doi/10.1093/comjnl/bxs074
few studies in this field, meaning more work is required in this [4] M. Tedre and P. J. Denning, “The long quest for computational think-
ing,” in Proc. 16th Koli Calling Int. Conf. Comput. Educ. Res. Koli
area of research to determine the extent to which each type of Calling ’16, Koli, Finland, 2016, pp. 120–129. [Online]. Available:
user interface is best suited to learning in the various stages http://dl.acm.org/citation.cfm?doid=2999541.2999542
of education, and to study the usability and user experience [5] I. Harel and S. Papert, “Software design as a learning environment,”
of these interfaces, the goal being to develop programming Interact. Learn. Environ., vol. 1, no. 1, pp. 1–32, 1990. [Online].
Available: https://doi.org/10.1080/1049482900010102
environments that are tailored to the user’s cognitive skills. [6] M. Zapata-Ros, “Pensamiento computacional: Una nueva alfabetización
digital,” Revista de Educación a Distancia, vol. 2015, no. 46, Oct. 2015.
[Online]. Available: https://revistas.um.es/red/article/view/240321
VI. C ONCLUSION [7] M. Ritter and C. Winterbottom, UX for the Web: Build Websites for User
Experience and Usability. Birmingham, U.K.: Packt Publ., Sep. 2017.
Recent years have seen a great wave of research on com- [8] D. Stone, C. Jarrett, M. Woodroffe, and S. Minocha, User Interface
putational thinking and its effect in the field of education. Design and Evaluation, 1st ed. Amsterdam, Netherlands: Elsevier,
2005. [Online]. Available: https://www.elsevier.com/books/user-
However, there are no mapping studies that consider the impli- interface-design-and-evaluation/stone/978–0-12–088436-0
cations of the impact caused by the various tools that have been [9] NSAI Stand, Dublin, Ireland, ISO 9241–11:2018(en).
developed to promote computational thinking skills. This arti- “Ergonomics of Human-System Interaction–Part 11: Usability:
cle presents a systematic review that relies on PRISMA as the Definitions and Concepts.” 2018. [Online]. Available:
https://www.iso.org/obp/ui/#iso:std:iso:9241:-11:ed-2:v1:en
analytical methodology. Twenty-two documents out of a total [10] F. J. M. Lopez, Instalación y Actualización De Sistemas Operativos
of 1069 were identified and filtered for further study. The result –UF0852. Madrid, Spain: Editorial Paraninfo, 2017.
 This article has been accepted for inclusion in a future issue of this journal. Content is final as presented, with the exception of pagination.

10 IEEE TRANSACTIONS ON EDUCATION

[11] S. Yadav and P. Chakraborty, “Human-computer interaction [29] P. Vanvorce and H. M. Jamil, “Computational thinking with the
as an important aspect of software: A tutorial,” in Proc. Web crowd using codemapper,” in Proc. 34th ACM/SIGAPP Symp.
IEEE Int. Conf. Comput. Power Commun. Technol. (GUCON), Appl. Comput., New York, NY, USA, 2019, pp. 2532–2534. [Online].
2020, pp. 40–44, doi: 10.1109/GUCON48875.2020.9231155. Available: https://doi.org/10.1145/3297280.3298913
[12] M. Rauterberg, “How to measure and to quantify usability attributes of [30] M. L. Barron-Estrada, R. Zatarain-Cabada, and B. A. Cardenas-Sainz,
man-machine interfaces,” in Proc. 5th IEEE Int. Workshop Robot Human “A natural user interface implementation for an interactive learning envi-
Commun. RO-MAN, 1996, pp. 262–267. ronment,” in Proc. IEEE 20th Int. Conf. Adv. Learn. Technol. (ICALT),
[13] S. Fincher and I. Utting, “Machines for Thinking,” ACM Trans. 2020, pp. 341–343.
Comput. Educ., vol. 10, no. 4, pp. 1–7, Nov. 2010. [Online]. Available: [31] C. Karakasis and S. Xinogalos, “Blocklyscript: Design and pilot evalu-
https://dl.acm.org/doi/10.1145/1868358.1868360 ation of an RPG platform game for cultivating computational thinking
[14] S. L. Oviatt, “Interfaces for thinkers: Computer input capabilities that skills to young students,” Informat. Educ., vol. 19, no. 4, pp. 641–668,
support inferential reasoning,” in Proc. 15th ACM Int. Conf. Multimodal Dec. 2020.
Interact. ICMI, Sydney, NSW, Australia, 2013, pp. 221–228. [Online]. [32] V. Nandan, A. Spittlemeister, and F. Brubacher, “Pixasso: A develop-
Available: http://dl.acm.org/citation.cfm?doid=2522848.2522849 ment stage-based learning application for children,” in Proc. 7th ACM
[15] A. R. Basawapatna and A. Repenning, “Employing retention of flow Conf. Learn. Scale, New York, NY, USA, 2020, pp. 361–364. [Online].
to improve Online tutorials,” in Proc. ACM SIGCSE Tech. Symp. Available: https://doi.org/10.1145/3386527.3406747
Comput. Sci. Educ., Seattle, WA, USA, Mar. 2017, pp. 63–68. [Online]. [33] S.-Y. Lin, S.-Y. Chien, C.-L. Hsiao, C.-H. Hsia, and K.-M. Chao,
Available: https://dl.acm.org/doi/10.1145/3017680.3017799 “Enhancing computational thinking capability of preschool chil-
[16] D. Moher, A. Liberati, J. Tetzlaff, D. G. Altman, and The dren by game-based smart toys,” Electron. Commer. Res. Appl.,
PRISMA Group, “Preferred reporting items for systematic vol. 44, Nov./Dec. 2020. Art. no. 101011. [Online]. Available:
reviews and meta-analyses: The PRISMA statement,” PLoS Med., https://www.sciencedirect.com/science/article/pii/S1567422320300880
vol. 6, no. 7, Jul. 2009, Art. no. e1000097. [Online]. Available: [34] A. Almjally, K. Howland, and J. Good, “Comparing TUIs and GUIs
https://pubmed.ncbi.nlm.nih.gov/19621072 for primary school programming,” in Proc. 51st ACM Tech. Symp.
[17] K. Roy, “App inventor for android: Report from a summer Comput. Sci. Educ., New York, NY, USA, 2020, pp. 521–527. [Online].
camp,” in Proc. 43rd ACM Tech. Symp. Comput. Sci. Educ., Available: https://doi.org/10.1145/3328778.3366851
New York, NY, USA, 2012, pp. 283–288. [Online]. Available: [35] S. Ahmed, M. A. A. Walid, and M. Islam, “EEG-based cognitive load
https://doi.org/10.1145/2157136.2157222 assessment in matlab GUI and impact on learning system,” in Proc. 2nd
[18] A. Strawhacker, A. Sullivan, and M. U. Bers, “TUI, GUI, Int. Conf. Adv. Inf. Commun. Technol. (ICAICT), 2020, pp. 484–487.
HUI: Is a bimodal interface truly worth the sum of its parts?” [36] A. Zitek, M. Poppe, M. Stelzhammer, S. Muhar, and B. Bredeweg,
in Proc. 12th Int. Conf. Interact. Design Children - IDC, “Evaluating the effects of a new qualitative simulation software
New York, NY, USA, 2013, pp. 309–312. [Online]. Available: (Dynalearn) on learning behavior, factual and causal understanding,” in
http://dl.acm.org/citation.cfm?doid=2485760.2485825 Artificial Intelligence in Education (Lecture Notes in Computer Science),
G. Biswas, S. Bull, J. Kay, and A. Mitrovic, Eds. Berlin, Germany:
[19] K. Perdikuri, “Students’ experiences from the use of MIT
Springer, 2011, pp. 594–596, doi: 10.1007/978-3-642-21869-9_112
app inventor in classroom,” in Proc. 18th Panhellenic Conf.
[37] A. C. Calderon and T. Crick, “Using interface design to develop com-
Informat., New York, NY, USA, 2014, p. 1–6. [Online]. Available:
putational thinking skills,” in Proc. Workshop Primary Sec. Comput.
https://doi.org/10.1145/2645791.2645835
Educ. (WiPSCE), New York, NY, USA, 2015, pp. 127–129. [Online].
[20] A. Han, J. Kim, and K. Wohn, “Entry: visual programming to
Available: https://doi.org/10.1145/2818314.2818333
enhance children’s computational thinking,” in Proc. ACM Int.
Joint Conf. Pervasive Ubiquitous Comput. Symp. Wearable Comput.
(UbiComp), Osaka, Japan, 2015, pp. 73–76. [Online]. Available:
http://dl.acm.org/citation.cfm?doid=2800835.2800871 Sara Rijo-García received the bachelor’s degree in computer science and
[21] B. Xie, I. Shabir, and H. Abelson, “Measuring the usabil- the master’s degree in teacher training for secondary education, baccalaureate,
ity and capability of app inventor to create mobile appli- vocational training, and foreign language teaching from the Universidad de La
cations,” in Proc. 3rd Int. Workshop Program. Mobile Touch Laguna, San Cristóbal de La Laguna, Spain, in 2014 and 2015, respectively,
(PROMOTO), Pittsburgh, PA, USA, 2015, pp. 1–8. [Online]. Available: where she is currently pursuing the Ph.D. degree in industrial, computer and
http://dl.acm.org/citation.cfm?doid=2824823.2824824 environmental engineering.
[22] C. Hill, H. A. Dwyer, T. Martinez, D. Harlow, and D. Franklin, She has worked with the Technology Transfer Office, Universidad de La
“Floors and flexibility: Designing a programming environment for Laguna for three years.
4th-6th grade classrooms,” in Proc. 46th ACM Tech. Symp. Comput. Sci.
Educ. (SIGCSE), Kansas City, MO, USA, 2015, pp. 546–551. [Online].
Available: http://dl.acm.org/citation.cfm?doid=2676723.2677275
[23] C.-K. Chang, “Using computational thinking patterns to scaf- Eduardo Segredo (Member, IEEE) received the B.S., M.S., and Ph.D. degrees
fold program design in introductory programming course,” in in computer science from the Universidad de La Laguna, San Cristóbal de La
Proc. 5th IIAI Int. Congr. Adv. Appl. Informat. (IIAI-AAI), Laguna, Spain, in 2006, 2008, and 2014, respectively.
Kumamoto, Japan, Jul. 2016, pp. 397–400. [Online]. Available: He is currently a Lecturer with the Departamento de Ingeniería Informática
http://ieeexplore.ieee.org/document/7557641/ y de Sistemas, Universidad de La Laguna. He has authored or coau-
[24] C. Hu, “Can students design software? The answer is more complex than thored over 50 technical papers and book chapters, including more than
you think,” in Proc. Assoc. Comput. Mach., Feb. 2016, pp. 199–204. 15 journal papers. His publications currently report over 400 citations in
[25] A. Pugnali, A. Sullivan, and M. U. Bers, “The Impact of User Interface Google Scholar with an H-index of 13. His current research interests
on Young Children’s Computational Thinking,” J. Inf. Tech. Educ. include single/multi/many-objective optimization, evolutionary algorithms,
Innov. Pract., vol. 16, no. 1, pp. 171–193, 2017. [Online]. Available: metaheuristics, machine learning, and computational thinking.
https://www.informingscience.org/Publications/3768 Dr. Segredo currently serves for the editorial board of multiple international
conferences.
[26] G. Cheng, “Exploring factors influencing the acceptance of
visual programming environment among boys and girls in pri-
mary schools,” Comput. Human Behav., vol. 92, pp. 361–372,
Mar. 2019. [Online]. Available: https://linkinghub.elsevier.com/retrieve/ Coromoto León received the M.S. degree in mathematics and the Ph.D.
pii/S0747563218305788 degree in computer science from the Universidad de La Laguna, San Cristóbal
[27] X. Jiang, C. Harteveld, X. Huang, and A. Y. H. Fung, “The compu- de La Laguna, Spain, in 1990 and 1996, respectively.
tational puzzle design framework: a design guide for games teaching She currently teaches programming languages and paradigms in graduate
computational thinking,” in Proc. 14th Int. Conf. Found. Digit. Games, and master’s programs, with the Departamento de Ingeniería Informática y de
San Luis Obispo, CA, USA, Aug. 2019, pp. 1–11. [Online]. Available: Sistemas, Universidad de La Laguna. She has led dozens of doctoral courses
https://dl.acm.org/doi/10.1145/3337722.3337768 and multiple seminars and has directed several research projects and contracts
[28] T. Turchi, D. Fogli, and A. Malizia, “Fostering computational think- for innovation and industry transfer. She has published more than 30 journal
ing through collaborative game-based learning,” Multimedia Tools Appl., papers and refereed conferences. Her research interests include programming
vol. 78, no. 10, pp. 13649–13673, May 2019. [Online]. Available: languages, algorithmic techniques, optimization, parallel programming, and
http://link.springer.com/10.1007/s11042–019-7229–9 computational thinking.