Received: 31 December 2019 Accepted: 1 May 2020

DOI: 10.1111/jcal.12454

ARTICLE

To solve or to observe? The case of problem-solving

interactivity within child learning games

Tereza Tetourová1 | Tereza Hannemann2 | Ondřej Javora3 | Kristina Volná4 |

Vít Šisler5 | Cyril Brom2

1
Faculty of Education, Charles University,
Prague, Czech Republic Abstract
2
Faculty of Mathematics and Physics, Charles We explored whether problem-solving interactivity within an instructional game fos-
University, Prague, Czech Republic
ters learning for children aged 8–10 years. Participants (N = 139) studied a biological
3
Faculty of Mathematics and Physics & Faculty
of Arts, Charles University, Prague, Czech topic either through a game-based learning environment (in which they solved
Republic assigned problems by interacting with a plant model) or from a standard learning
4
New Media Dramaturgy Center, Czech
environment (in which they observed how the problems were solved in an animation)
Television, Prague, Czech Republic
5
Faculty of Arts, Charles University, Prague,
(i.e., between-subject design). The treatments were equal with regard to learning con-
Czech Republic tent and guidance. No between-group differences in learning outcomes were

Correspondence
detected (comprehension: d = 0.16; transfer: d = −0.01). Self-rating of enjoyment
Cyril Brom, Faculty of Mathematics and tended to be higher in the game group (d = 0.32), and when the children could
Physics, Charles University, Malostranske
Namesti 25, Prague, 11800, Czech Republic.
choose between the treatments in a free-choice period, they strongly preferred the
Email: [email protected] interactive game (87.5%). The results suggest that both the interactive and the non-

Funding information
interactive treatments are useful, but their applicability may differ depending on the
Charles University, Grant/Award Numbers: learning context (e.g., school vs. home).
PRIMUS/HUM/03, Progress Q15; European
Regional Development Fund Project, Grant/
KEYWORDS
Award Number:
CZ.02.1.01/0.0/0.0/16_019/0000734; animation, game-based learning, interactivity, learning outcomes, problem-solving
Grantová Agentura České Republiky, Grant/
Award Number: 19-02532S

Peer Review
The peer review history for this article is
available at https://publons.com/publon/10.
1111/jcal.12454.

1 | I N T RO DU CT I O N that solving the problems within the game could be more motivating,
increasing cognitive activation. In addition, it could, in and of itself,
Does problem-solving interactivity within a game-based learning envi- foster meaningful processing of the instructional message, helping
ronment enhance intrinsic motivation and learning outcomes for chil- children to build better mental representations of the target phenom-
dren? Suppose 9-year-olds play a simulation game. In the game, they ena. These two benefits might improve deep learning. On the other
have to acquire mental models of scientific concepts; say, of photo- hand, one can speculate that solving the problems (rather than just
synthesis and water transport in plants. As part of the game-play, they seeing how they are being solved) may be cognitively too demanding
solve assigned problems (after having received information delivery for children, overloading their limited mental resources.
on the topic). Now, suppose a designer creates an animation that The case of problem-solving interactivity within game-like learning
demonstrates how to solve the same problems (using the in-game environments, especially with regard to children, is an important one. On
content). Which of these two approaches, a game or an animation, will the empirical level, despite several decades of game-based learning
be instructionally superior? On the one hand, one can hypothesize research (see., e.g., Abdul Jabbar & Felicia, 2015; Boyle et al., 2016;

J Comput Assist Learn. 2020;36:981–996. wileyonlinelibrary.com/journal/jcal © 2020 John Wiley & Sons Ltd 981
 13652729, 2020, 6, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12454 by Bbw Hochschule, Wiley Online Library on [06/04/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
982 TETOUROVÁ ET AL.

Mayer, 2014, Wouters, van Nimwegen, van Oostendorp, & van der tutoring systems). In this context, interactivity can be generally viewed
Spek, 2013), it is still poorly understood how design elements of learning as the possibility for the learner to manipulate certain elements of the
games, including problem-solving interactivity, influence learning pro- instructional system (cf. Scheiter, 2014; Weber, Behr, & DeMartino,
cesses (see, for example, Clark, Tanner-Smith, & Killingsworth, 2016; 2014). This may also include manipulation of the learning content and
Mayer, 2014; Wouters & van Oostendorp, 2017). In general, few exper- the way it is presented.
imental, game-based learning studies have been conducted with primary To understand problem-solving interactivity, its relation to other
school children (see Hainey, Connolly, Boyle, Wilson, & Razak, 2016; interactivity incarnations, and related empirical work, we need to
Hussein, Ow, Cheong, Thong, & Ebrahim, 2019 for reviews). Specifically, break down the general definition described above. We will do this in
only a handful of studies have examined the instructional effectiveness two steps (Figure 1) and we will consider interactivity from the educa-
of problem-solving interactivity (Moreno & Mayer, 2005; Ritterfeld, tional rather than technological point of view, i.e., we will focus on
Shen, Wang, Nocera, & Wong, 2009; Sawyer, Smith, Rowe, Azevedo, & what forms of learning activities interactivity enables (if any). The first
Lester, 2017; see also Song et al., 2014; Wang, Vaughn, & Liu, 2011). step involves asking the following question: Does interactivity relate
They provided mixed evidence of effectiveness and none of them were to learning content or the instructional system's contextual features?
conducted with primary school children. Problem-solving interactivity belongs to the former strand. The latter
Three points are crucial from the theoretical perspective. First, scientific one includes, for example, the possibility to customize a background
phenomena can be simulated conveniently within games, making this theme or visual features for the learner's avatar in a gamified tutoring
medium especially suitable for mental models acquisition (this type of learn- system (Snow, Allen, Jacovina, & McNamara, 2015). It is out of the
ing is included in the present scope). Second, learning games explicitly target present scope.
affective-motivational states during learning (e.g., Brom, Šisler, Slussareff, The second step concerns learning-content interactivity only.
Selmbacherová, & Hlávka, 2016; Ninaus et al., 2019). Third, children differ Within learning sessions that address mental models acquisition,
from older learners in many developmental factors, such as by having more learners can typically participate in several, conceptually different
limited mental resources (Gathercole, Pickering, Ambridge, & Wearing, modes. The main ones include (1) exposition, (2) problem-solving and
2004). Predictions of contemporary learning theories with regard to (3) elaboration (cf., Mayes, 2015, pp. 362–363).
problem-solving interactivity have so far been primarily examined by means
of studies on learning from worked examples (vs. solving problems), but out- 1 Exposition mode: Within this mode, information is delivered to
side media targeting affective-motivational factors of learning experiences, learners. Their primary task is to create initial mental models of the
within the context of cognitive skills acquisition (rather than mental models to-be-learned content in their working memory.
acquisition), and with higher education learners (see, for example, Renkl & 2 Problem-solving mode: Within this mode, learners mentally work
Atkinson, 2010; Renkl, 2017; but see also van Gog, Kester, & Paas, 2011; on the emerging mental model by interfacing with assigned tasks,
Coppens, Hoogerheide, Snippe, Flunger, & van Gog, 2019). further refining the model and integrating it with their prior
From a practical perspective, it is important to know whether problem- knowledge.
solving interactivity within game-like environments creates added value (and 3 Elaboration mode: Within this mode, learners typically receive cor-
if so, what forms should the interactivity take). The reason is that non- rective feedback and/or are made to reflect on their emerging men-
interactive learning environments are cheaper to develop and easier to tal models. This is done to find weak spots in the mental models
implement in schools (cf. Brom, Levčík, Buchtová, & Klement, 2015). and to facilitate their re-organization (if needed) and further inte-
The problem-solving interactivity hypothesis is that children will gration with prior knowledge.
learn better and will be more motivated when they solve problems
within a simulation game environment than just seeing how the prob- Depending on the learning environment, these three modes may
lems are solved within an animation. Despite this hypothesis' intuitive represent distinct phases, or they may be inseparably intertwined
appeal, it is questionable whether it holds empirically (cf. Clark & and/or repeated during the learning session. A mode or two may also
Feldon, 2014). The present study provides an experimental test of be absent. The point is that within each of these modes, interactivity
this hypothesis, using a large child sample. means something different (Table 1). We are now concerned with the
problem-solving mode only, other forms of interactivity are listed in
Data S1: Appendix A.
2 | S T U D Y B A C K GR O U N D Within the problem-solving mode, learners may be given the pos-
sibility to solve tasks themselves, fully or partially. We call this prob-
2.1 | Understanding problem-solving interactivity lem-solving interactivity. For example, problem-solving interactivity
and instructional modes within a botany game (Moreno & Mayer, 2005) refers to learners
being asked to design a virtual plant (from various types of roots,
This work focuses on how learners acquire mental models of complex stems and leaves) so that it survives in specific environmental condi-
processes or systems (Mayer, 2009) by means of studying in a single- tions (e.g., heavy rainfall and strong wind).
learner, computerized instructional environment, particularly simula- Problem-solving interactivity is a graded phenomenon. On the
tion games (other examples include slides, animations or intelligent one end of the spectrum, the user can find learning environments
 13652729, 2020, 6, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12454 by Bbw Hochschule, Wiley Online Library on [06/04/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
TETOUROVÁ ET AL. 983

F I G U R E 1 Examples of
interactivity types: a classification tree

TABLE 1 Examples of interactivity forms

Mode Interactivity type Definition

(1) Exposition Information-gathering Learners fully or partially control what information chunks will be
presented to them and when and how.
(2) Problem-solving Task-selection interactivity Learners have some level of control over what tasks to solve.
Problem-solving interactivity Learners fully or partly solve assigned or self-selected tasks, that
is, they must choose from at least two options for how to
proceed with a task.
(3) Elaboration Elaboration interactivity Learners influence (directly or indirectly) how the elaboration phase
proceeds by interfacing with the instructional environment.

Note: This study's focus is given in boldface.

offering multiple unconstrained options that can be freely explored. 2009). They must select relevant information from learning materials,
On the other end, the number of choices is limited, and the options organize it within working memory and integrate it with prior knowl-
are supported by guiding prompts. As long as there are at least two edge. Problem-solving interactivity is supposed to support organizing
ways the learner can proceed to the problem's solution, the condition and integrating, because it helps activate relevant prior knowledge
can be viewed as interactive. and prompts learners to operate cognitively on the new mental model
being built (Moreno & Mayer, 2005). In addition, problem-solving
interactivity increases provision of choice, thus enhancing the percep-
2.2 | Problem-solving interactivity in general: tion of control. Within self-determination theory (Deci & Ryan, 1985;
Theoretical background cf. Vansteenkiste, Sierens, Soenens, Luyckx, & Lens, 2009; Patall,
Cooper, & Robinson, 2008), this implies fostering intrinsic motivation.
Contemporary learning theories make conflicting predictions with Especially in games, increased provision of choice is typically con-
regard to the instructional effectiveness of problem-solving interactiv- nected to the game mechanics, further increasing intrinsic motivation.
ity. On a positive note, cognitive theory of multimedia learning posits According to the cognitive-affective theory of learning from media
(apart from other things) that in order to acquire a new mental model, (which is an expansion of the cognitive theory of multimedia learning;
learners must actively engage in several cognitive processes (Mayer, Moreno, 2005), elevated motivation (and related affective-
 13652729, 2020, 6, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12454 by Bbw Hochschule, Wiley Online Library on [06/04/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
984 TETOUROVÁ ET AL.

motivational states) should boost learners' cognitive activity, further interactivity (i.e., yes/no), with the exception of Sawyer et al. (2017)
enhancing learning. Therefore, problem-solving interactivity can be (interactivity: high/low/no). With one exception (Ritterfeld et al.,
theorized to have (at least) two positive effects on learning: a direct 2009), the studies did not use the yoked design (i.e., assigning partici-
cognitive one (by supporting organizing and integrating) and an indi- pants in the control condition to the game replays of participants from
rect motivational one (through increasing provision of choice). the experimental condition). Instead, they showed the correct solu-
On a negative note, cognitive theories warn us that learners' men- tions or how the problem is being solved by someone else in the con-
tal resources are limited. For instance, within cognitive load theory trol condition.
(Sweller, Ayres, & Kalyuga, 2011; see also Renkl & Atkinson, 2010), The studies provided a pattern of mixed results, ranging from
problem-solving interactivity can be viewed as too demanding, espe- (a) negative with large effect sizes to (b) neutral to (c) somewhat posi-
cially when the interactivity is unguided (cf. Alfieri, Brooks, Aldrich, & tive. This pattern is consistent with the following interpretation (and
Tenenbaum, 2011; Kirschner, Sweller, & Clark, 2006). Limited mental also the theoretical standpoints from the previous section).
resources can be strained even further when learners do not possess
relevant prior knowledge and/or when they are not familiar with the 1 The findings are negative with large effect sizes only in one case:
delivery medium. For example, if mental resources are depleted in the When conditions differ not only due to interactivity, but also
course of learners' struggling with the learning environment's control because of a confounding variable negatively influencing the inter-
interface, or during ineffective mental search for a solution to an active condition. This is the case of the study by Moreno and May-
overly difficult problem, or because the possibility of interaction leads er (2005, Exp. 2a). This study includes an elaboration mode
to learning-irrelevant thoughts, not enough mental resources will reflection. However, this reflection in the interactive condition is
remain for meaningful learning to occur. Therefore, problem-solving different from, and inferior to, the reflection used in the non-
interactivity can be theorized to have (at least) one negative effect on interactive condition.
learning: overloading limited mental resources. 2 The findings tend to be neutral (or negative with small effect sizes)
The theories agree on two things: (a) that problem-solving inter- in two cases. The first one involves situations when interactivity is
activity, in general, should be guided (to manage limited mental supplemented by a properly implemented elaboration phase
resources) and (b) that it may be more instructionally beneficial for (i.e., when the elaboration mode can be theorized to target the
advanced learners as opposed to beginners (cf. so-called expertise- organizing and integrating processes similar to the interactivity, the
reversal effect; Kalyuga, 2007; see also Chen, Retnowati, & Kalyuga, positive effects of elaboration and interactivity may not be addi-
2019; Reisslein, Atkinson, Seeling, & Reisslein, 2006; Renkl & tive) (Moreno & Mayer, 2005, Exp. 3a). The second one is when
Atkinson, 2010). Aside from that, the conflicting predictions can only the interactivity condition clearly increases difficulty (Song et al.,
be reconciled empirically. In regard to game-like environments, empiri- 2014), possibly causing cognitive overload. The detrimental effect
cal evidence regarding problem-solving interactivity is scarce with of higher difficulty is also the probable culprit behind the difference
respect to higher education learners. It is even more limited with between low vs. high conditions in Sawyer et al.'s study (2017). In
regard to children (as detailed in the next section). Results from higher that project, the high condition allowed for greater freedom in
education studies may not generalize to child audiences, because chil- information gathering (possibly increasing the difficulty of the high
dren's mental capacity is lower compared with older learners condition).
(e.g., Gathercole et al., 2004). This fact warrants the present study. 3 In other cases, the findings tend to be somewhat positive. The
positive effect in Ritterfeld's study (2009) could even have been
reduced due to the yoked design. Technically, yoked designs
2.3 | Problem-solving interactivity in game-like enable a perfect control group (control participants see exactly
environments: Empirical background what the experimental participants saw). However, the video-
replays given to participants in the control condition obscure the
We are aware of six experiments that compared learning outcomes reasons behind the choices made by the yoked learners who
for individual learners studying either from a game-based (or a similar played the game (cf. Weber et al., 2014). There is related evi-
type of) learning environment, featuring problem-solving interactivity, dence from vicarious learning literature that yoked video-replays
or from complementary environments, lacking problem-solving inter- are inferior to demonstrating expert solutions (Craig, Sullins, Wit-
activity (Table 2). All these experiments studied higher education herspoon, & Gholson, 2006).
learners. As far as we can tell, they all featured some form of exposi-
tory mode combined with a problem-solving mode. Only Moreno and All in all, there is the following tentative evidence as concerns
Mayer (2005) reported the use of a sophisticated elaboration mode. higher education learners: problem-solving interactivity within game-
The studies typically implemented a program feedback questionnaire, like learning environments, provided it does not exhaust too many
but only one reported affective-motivational outcomes (Ritterfeld mental resources, creates added value as concerns learning out-
et al., 2009: gained interest in the topic: null results). Between-group comes; or at least it does not worsen them. The question of present
differences in time-on-task were rarely reported and could present a interest is—what are its effects in the case of primary school
possible confounding element. The studies used one level of children?
 TABLE 2 Problem-solving interactivity studies

Medium in the Freedom in problem-solving

Paper Experiment/condition Design experimental group interactivitya Level of elaboration Learning outcomesb
Moreno and Exp. 2; (a) groups with 2 × 2 (reflection yes/no × Problem-solving game Constrained (selection from High (self-reflection + Large negative - medium
TETOUROVÁ ET AL.

Mayer (2005) problematic reflection interactivity yes/no) environment pre-defined options) feedback) negative
after problem-solvingg
Exp. 2; (b) groups without Problem-solving game Low (feedback) Negligible - large positive
reflection after environment
problem-solving
Exp. 3; (a) groups with 2 × 2 (reflection yes/no × Problem-solving game Constrained (selection from High (self-reflection + Negligible
proper reflection after interactivity yes/no) + 1c environment pre-defined options) feedback)
problem-solving
Exp. 3; (b) groups without Problem-solving game Low (feedback) Negligible - small positive
reflection after environment
problem-solving
Ritterfeld Partial 2 × 3d (interactivity 3D, narrative, game-based Probably less constrained Unclear Negligible - medium
et al. (2009) yes/no × 3 levels of learning environment (unspecific interaction in a positive
multimodality) 3D world)
Wang et al. (2011) 4 groups (static, animation, Learning environment with simple Constrained (the possibility Probably absent Negligible - medium
interactive, with quiz-based elements posing tasks to change certain positive
feedback)e statistical inputs)
Song et al. (2014) 2 × 2f (information-gathering 3D learning environment Unconstrained (selecting a Low (feedback) Small negative
interactivity yes/no × posing tasks location within a 3D model
problem-solving of the brain without much
interactivity yes/no) assistance)
Sawyer et al. (2017) High interactivity vs. 1 + 1 + 1 (no [replay] vs. low 3D, narrative, game-based learning Probably less constrained Unclear Small positive
replay vs. high interactivity)h environment (unrestricted) (unspecific interaction in a
Low interactivity vs. 3D world)
replay 3D, narrative, game-based learning Probably less constrained Unclear Large positive
environment (information (unspecific interaction in a
collected and tasks solved in a 3D world)
specific order)

Notes: See Supplementary Material for the full version of this table.
a
Greater freedom in choices, that is, few constraints, typically imply higher difficulty.
b
Cohen's d (small 0.2; medium 0.5; large 0.8). Positive outcome means that interactivity was superior to non-interactivity.
c
The group with reflection and interactivity in Exp. 3 (Moreno & Mayer, 2005) was shown the correct solution and prompted to reflect on the correct solution (like the control group). The experiment also
included a fifth condition, which replicated the problematic reflection from Exp. 2 (that group is not analyzed in this table).
d
In this table, interactive game vs. game replay is contrasted.
e
In this table, only interactive vs. animation groups are contrasted. The interactive group also had access to all non-interactive materials.
f
In this table, the results are collapsed across two levels of information-gathering interactivity.
g
The group with reflection and interactivity in Exp. 2 (Moreno & Mayer, 2005) was prompted to reflect on its own solutions, which could either be right or wrong. This is problematic, because the control condi-
tion reflected on a correct solution given by the system.
h
The difference between low vs. high interactivity conditions lies primarily in information-gathering and task-selection interactivity. The difference between the no-interactivity condition and other conditions
985

lies also in problem-solving interactivity. The high group most likely received cognitively more demanding materials compared with the low group.

13652729, 2020, 6, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12454 by Bbw Hochschule, Wiley Online Library on [06/04/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
 13652729, 2020, 6, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12454 by Bbw Hochschule, Wiley Online Library on [06/04/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
986 TETOUROVÁ ET AL.

3 | THI S STU DY during the experiment, and that they could ask for water when thirsty.
After the experiment, the children were escorted away by their par-
This study aims at expanding knowledge about the effects of ents, legal representatives or persons designated by parents/legal rep-
problem-solving interactivity within computerized learning environ- resentatives. Voice recordings of the children were encrypted
ments, in the case of primary school children (8–10 years of age). Chil- immediately after the experimental session. Overall, the experiment
dren differ from higher education learners in many developmental did not pose a greater risk to children than any ordinary interaction
factors. Thus, the existing findings, which tend to support the with a computer.
problem-solving interactivity hypothesis as concerns learning
outcomes, may not generalize to the former audience.
As the first step toward achieving this end, children in this study 4.2 | Participants
learn about photosynthesis and water transport, either from a simula-
tion game-based learning environment or from a complementary Participants included 139 Czech children from third or fourth grade
non-interactive environment. Then, their learning outcomes and moti- (aged 8–10 years, M = 8.78 SD = 0.67). They were recruited via
vation variables are examined. The interactive environment does not calls made by Czech TV's children's channel (i.e., a mainstream,
enable free exploration, it is structured and guided (i.e., to avoid public television service for Czech children aged 4–12 years). The
depletion of mental resources). It also includes non-interactive exposi- children came from large cities with over 100,000 residents
tory phases, alternated with problem-solving phases. Its non- (n = 66) and from smaller cities and rural towns with less than
interactive counterpart presents the same expository phases, 100,000 residents (n = 71). An additional 17 children were excluded
alternated with animated, expert solutions to the assigned problems from the data analysis due to technical issues, illness or speech
(i.e., to avoid issues with yoked design). The non-interactive version is disorders. Children were randomly assigned either to the interac-
as much a content- and guidance-equivalent to the interactive version tive game condition (n = 69) or the non-interactive animation
as possible. (n = 70) condition (see Table 3 for average age and gender distribu-
Our primary research questions address whether children will tion). Random assignment was balanced based on children's gender
learn better (R1) and be more motivated (R2) when they solve prob- and age. The children received a LEGO set (worth 20 EUR) and
lems within the simulation game-based environment themselves, university and Czech TV promo merchandise for their participation.
i.e., compared with just seeing how the problems are solved within an Sample size was determined based on a priori power calculations
animation. The interactivity hypothesis is that they will learn better in G*Power 3 (Faul, Erdfelder, Lang, & Buchner, 2007). We
and be more motivated. needed 64 participants per group to detect the medium effect size
We also explore (R3) whether students' “ability” will moderate (Cohen's d = 0.5) using t-tests (for α = .05 and 1-β = .80).
these effects, based on the expertise-reversal idea (Kalyuga, 2007).
That is, we explore whether learning outcome differences between
the interactive and the non-interactive conditions will favour the 4.3 | Materials: Interventions
interactive game more with regard to the more “able” children versus
less as concerns the less “able” ones. “Ability” is operationalized in 4.3.1 | Game
two ways: as prior topic knowledge and prior game experience.
The interactive intervention was a learning environment we devel-
oped previously for conducting research studies with our age group
4 | METHOD (Javora et al., 2019). In this environment, children acquired the
notions of photosynthesis and water transport in plants. The environ-
4.1 | Ethical approval ment consisted of six instructional blocks with increasing difficulty
that segmented the lesson (each was 2–4 min. long). These blocks
This study has been approved by the Ethics Review Board of the Insti- were called “chapters” in both conditions. Five blocks were used in
tute of Psychology of the Czech Academy of Sciences. Participant the intervention and the sixth one in the free-choice period, as
data were anonymized by assigning the children numbers. For the detailed below. Each block consisted of two segments presented in
sake of protecting participant anonymity, data are not openly pro- the following order: (a) information delivery and (b) solving problems
vided to third parties (see also “Data availability” statement). Parents within a simulation mini-game (Figure 2).
and children were informed before the experiment that they could Information was delivered through self-paced, narrated slides
stop the experiment anytime without any consequences for them and (Figure 3). The slides introduced the learning content, game controls
ask that their data be discarded, as long as data of individual partici- and goals for the subsequent gaming segment (e.g., “to grow three
pants could be identified (due to the anonymisation). The experiment new leaves” or “to accumulate enough energy to grow a first flower”).
was designed such that children do not look at the screen for more The gaming part concerned “building” a plant, that is, gradually
than 20–25 min. The intervention included only age-appropriate building new stem parts, leaves, roots and flowers (Figure 4). The stem
material. Children were informed that they could ask for a break could hold new leaves; the leaves captured energy from the sun; and
 13652729, 2020, 6, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12454 by Bbw Hochschule, Wiley Online Library on [06/04/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
TETOUROVÁ ET AL. 987

TABLE 3 Control variables, including

Game Animation
t-test and chi-squared test results
Mean SD Mean SD t df p d
Domain interest 3.32 1.27 3.19 1.52 0.57 137 .572 0.10
Names prior 1.05 0.74 1.04 0.74 0.09 137 .931 0.01
Concepts prior 0.93 0.69 1.05 0.76 −0.91 137 .367 −0.15
Comprehen. 1.10 1.38 1.26 1.33 −0.70 137 .485 −0.12
prior
Time-on-task 16.45 2.62 13.60 1.54 7.73 132 <.001 1.33
Age 8.80 0.70 8.77 0.64 0.23 137 .822 0.04
Class 3.49 0.50 3.54 0.50 −0.59 137 .588 −0.10
Game exper. 2.07 0.88 1.88 0.71 1.41 136 .162 0.24
Boys Girls Boys Girls χ 2
Df p
Gender 41 28 39 31 0.195 1 .658
Cities Other Cities Other χ2 Df p
a
Residence 33 35 33 36 0.007 1 .934
a
Two participants did not provide residence.

FIGURE 2 Structure of the interventions

the roots absorbed water from the soil. Having a flower bloom was Children controlled the game via buttons (Figure 4, on the left;
the game's ultimate goal. e.g., there was a button for adding a leaf) and clicking on the plant
How was problem-solving interactivity realized within the game? (e.g., where the leaf should be added). During the game play, the chil-
To achieve the goal in each block, the participant engaged in solving dren were guided by narrated hints (2–4 per each game segment),
the following problems: such as “the flower is withering, build two or three roots”. These hints
were activated by a research administrator, who followed a protocol
• accumulating energy by adding leaves (from the first block that consisted of triggering rules (e.g., “the leaves of the flower are
onward); getting brown”). The protocol was composed such that hints with
• making strategic decisions like whether to “invest” accumulated essential information (such as the one above) were always triggered. If
energy into building new stem parts, leaves, roots (from the second a child struggled with controlling the game or got lost, a hint lacking
block onward) or flowers (from the fourth block onward); essential information but referring to the game user interface or how
• managing the plant's internal water resources by building new to proceed, was triggered (e.g., “There is no room on the plant for
roots (from the second block onward) or regulating water transpi- building another leaf; use ‘scissors button’ to remove an existing leaf
ration through opening/closing stomata on the leaves (from the first”; or “What about trying to build some leaves now?”).
third block onward); The intervention was conceived with instructional design princi-
• making strategic decisions whether to react to weather changes and ples (Mayer, 2014) in mind. Most notably, it followed a segmenting
thus influence the speed of photosynthesis and transpiration by open- principle (segmenting the intervention into blocks), a pre-training prin-
ing/closing stomata on the leaves (from the fourth block onward). ciple (providing key information before each problem-solving seg-
ment; having increasing difficulty in game levels), a modality principle
Solving these problems was designed to help children improve (using pictures with spoken words rather than printed words), a signal-
their emerging mental models of photosynthesis and water transport ing principle (visually highlighting key information) and a coherence
in plants and integrate these models with prior knowledge. As partici- principle (excluding extraneous material). The game also followed the
pants proceeded through the blocks, they had to solve an increasing self-testing principle (Fiorella & Mayer, 2015), which included self-
number of problems at the same time.1 testing parts: after the second, the fourth and the fifth blocks. In each
 13652729, 2020, 6, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12454 by Bbw Hochschule, Wiley Online Library on [06/04/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
988 TETOUROVÁ ET AL.

F I G U R E 3 Screenshot from the

information delivery segment (texts in the
Czech language) [Color figure can be
viewed at wileyonlinelibrary.com]

F I G U R E 4 Screenshots from the problem-solving segments: four different weather conditions. The difference between screenshots from the
animation (top) and the game (bottom) lies in the absence of two small buttons in the upper right corner of the images (i.e., sound on/off, full
screen). In addition, the control buttons on the left side of the screenshots were clickable only in the game [Color figure can be viewed at
wileyonlinelibrary.com]

of these parts, learners answered three yes/no questions for testing with animations (Figure 4). These animations were expert replays of
knowledge gained from the previous two blocks (e.g., “Photosynthesis the respective game segments: they demonstrated how to achieve
means capturing energy from the sun—yes/no?”). After each question, the assigned goals (i.e., by engaging in solving the problems listed
the learner received corrective feedback. above). During these replays, the same hints with essential informa-
tion were activated (i.e., as in the game), but there were no hints as
concerns controlling the game or advising children when they got lost.
4.3.2 | Animation Each segment was divided into several subsegments. The partici-
pant proceeded through the subsegments by pressing the “Next” but-
The non-interactive intervention consisted of the same six blocks as ton. The “Replay” button enabled replaying the prior subsegment, but
the game intervention; however, the gaming segments were replaced children rarely used this feature. These buttons appeared on the
 13652729, 2020, 6, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12454 by Bbw Hochschule, Wiley Online Library on [06/04/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
TETOUROVÁ ET AL. 989

screen when the subsegment ended. The lengths of the replay seg- their presentation: children were asked about the reasons for their
ments were about 1–1.5 min when shown continuously (i.e., partici- choices. Two evaluators graded the answers based on a pre-defined
pants interfaced with these segments a bit longer because of the rubric (this rubric is further detailed in Javora et al., 2019) with
‘Next and ‘Replay’ buttons). These lengths were based on how long, acceptable agreements (names: r = .95; possible scale 0–3; concepts:
on average, it took the children in our previous experiment to play r = .91; possible scale 0–9). The averages are used in the subsequent
these segments (Javora, Hannemann, Stárková, Volná, & Brom, 2019). analysis.
After the second, fourth and fifth blocks, the child received the
same questions as in the game.
4.4.3 | Game experience

4.4 | Materials: Measures Game experience was examined by means of a simple question-
naire composed of two questions with five-point scale. We asked
We checked whether the groups were balanced in the following the participants the following questions: “Do you like playing PC
variables: domain interest in plants, prior domain knowledge, prior games?” and “How often do you play PC games?”
comprehension and game experience. Intervention outcomes were
measured using cognitive variables (comprehension, near transfer)
and affective-motivational variables (learning enjoyment, evaluation 4.4.4 | Comprehension
of the game and of the animation, free choice-motivation and pref-
erence for learning). All of these variables are described in this sec- Comprehension (i.e., understanding of target concepts) was mea-
tion. The variables are the same or similar to the ones we used in sured before and after the intervention using a drawing test. This
our previous study (Javora et al., 2019), where they are described test was made by us and had been used previously (Javora et al.,
further in detail. 2019). Participants were first given the following instructions:
“Imagine you are drawing a picture for a botany textbook for your
classmates. The image should show how photosynthesis works.
4.4.1 | Domain interest in plants Draw or write in the following image what happens during photo-
synthesis in a way that your classmates can easily understand. You
This variable was examined by means of a structured interview. We can also draw info bubbles, arrows and other symbols.” [emphasis
asked about children's interest in plants in (a) schools (e.g., “do you like in the original]. Afterwards, participants were given a sheet of
learning about plants?”), (b) home environments (e.g., “do you watch paper with the picture as shown in Figure 5. Participants could earn
films about plants?”), and (c) outdoors (e.g., “do you like observing 1 point for each of seven correct key ideas or 0.5 points for each of
plants outdoors?”). The answers were coded by two coders. For the five correct, less-important ideas (and then either 0.25 or 0.5 points
first area, up to four points could be assigned based on a rubric. For for a partially correct solution). The possible scale was 0–9. The
each of the other two areas, up to three points could be awarded tests were evaluated by two independent raters based on a rubric
(i.e., possible scale 0–10; inter-coder agreement: r = .86; Cronbach's (between-rater agreement: pre-comprehension: r = .98; post-com-
α = .58). Low α suggests that the variable may not reflect a unitary prehension: r = .97).
construct. However, this is a control variable, so we kept it for the
pre-analysis. The averages of the raters' scores are used in the subse-
quent analysis. 4.4.5 | Near transfer

Near transfer (i.e., the ability to use learned concepts in new, but
4.4.2 | Prior domain knowledge related, situations) was tested with seven oral questions (e.g., “If
the plant's stomata are constantly closed what will happen? Say
We measured two variables: knowledge of domain names and domain everything that comes to mind.”). This test was also made by us
concepts. We used a structured interview with two cueing questions. and had also been used previously (Javora et al., 2019). Due to
Participants were first shown the following multiple-choice questions time constraints and to avoid cueing of what should be remem-
and asked to select the correct answers (“Which words relate to bered, transfer was assessed only after the treatment. Each ques-
plants?—(a) photosynthesis; (b) transpiration; (c) hydrogen; (d) carbon tion had 2–6 correct solutions. Participants could receive up to
dioxide”; “Why do plants have leaves?—(a) they help them withstand 1 point for each of them (and then 0.25, 0.5 or 0.75 points for a
the wind; (b) they breathe in oxygen and create carbon dioxide; partially correct solution; the possible scale: 0–24). Participants
(c) they capture energy from the sun; (d) they help plants ‘absorb’ could use their own words to express the ideas. The answers were
water from their roots; (e) they capture mineral substances from the evaluated by two independent coders based on a rubric (between-
air”; multiple answers were correct). These multiple-choice questions coder agreement: r = .90). The average values are used in the sub-
were not graded. The important part was the interview that followed sequent analysis.
 13652729, 2020, 6, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12454 by Bbw Hochschule, Wiley Online Library on [06/04/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
990 TETOUROVÁ ET AL.

Thereafter, they interacted with the sixth block from the selected ver-
sion for the remainder of the period (about 3–5 min.).

4.4.9 | Learning preference

We also asked the participants “what version they would prefer to

use when preparing for a school exam”.

4.5 | Procedure

Participants were tested one by one in a laboratory.2 The session

lasted for about an hour. After a brief introduction to the experiment
for the child and one of its parents (but without disclosing the experi-
ment's purpose), the parent signed the informed consent form and left
the lab. Participants were familiarized with the smiley scale and inter-
viewed about domain interest. Afterwards, they completed the draw-
ing pre-test (i.e., comprehension). Next, we inquired after their prior
knowledge. Then, the participants started to interact with the learning
environment based on their condition (i.e., game or animation). The
game was never referred to as a ‘game’; instead, it was called the
“controllable version”. The animation was referred to as the “video
version”.
After the fifth block was completed, participants had to rate their
learning enjoyment. Thereafter, they were given the drawing post-test
FIGURE 5 Drawing test—empty sheet
(for the second time) and the oral transfer test.
The children were then shown the problem-solving part of the
4.4.6 | Learning enjoyment fifth block of the learning materials' alternative version (i.e., the game
group saw the animation and vice versa). Next, the children evaluated
It was assessed by two questions printed on a sheet of paper along both versions.
with a six-point smiley scale (“I enjoyed learning how a plant works”; Afterwards, the free-choice period started. Participants were
“I enjoyed learning from the slides”; r = .50). Children checked one informed that they still had about 5 min before the end of the session.
smiley for each question. The average of scores is used in the subse- They were asked whether they would prefer to continue with the
quent analysis. game or the animation during that time. Participants selected their
version of choice and interfaced with it (i.e., with the sixth block) until
the end of the session. At the very end, they were asked which ver-
4.4.7 | Evaluation of the game and the animation sion they would prefer to use when preparing for an exam. Then, they
were interviewed briefly, thanked and given a reward.
After participants were introduced the second version of the interven-
tion, they evaluated both versions. They were asked “How did you
like these learning materials?” Participants were given two six-point 4.6 | Data analysis
smiley scales at the same time. They used one for rating the game and
the other for rating the animation. Tests were computed in jamovi version 1.0 (The jamovi Project,
2019). To check whether the groups were balanced with respect to
gender and residence (urban vs. rural areas), we used a chi-squared
4.4.8 | Free-choice motivation test. To test whether portions of the children who selected a specific
version of the intervention during the free-choice period differed
In order to measure motivation to interact with the game versus the from the baseline 50–50% proportion, we used a binomial test. To
animation, we examined children's behaviour during the free-choice assess RQ3, that is, whether game experience and prior knowledge
period—as is typically done in intrinsic motivation research (e.g., Deci, (prior concepts, prior names) moderate the effect of interactivity on
Koestner, & Ryan, 1999). First, children could choose which version comprehension and transfer, we used ANCOVAs with the treatment
they wanted to interact with (see Section 4.5 for further details). factor (game vs. animation) and one of the three possible moderators
 13652729, 2020, 6, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12454 by Bbw Hochschule, Wiley Online Library on [06/04/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
TETOUROVÁ ET AL. 991

as a covariate. Potential significant interaction would be indicative of learning from either version. Those learning from the game enjoyed
a moderation effect. the experience slightly more compared with those who learned from
Otherwise, between-group differences were primarily analysed the animation. However, this effect was only of borderline
through t-tests. In general, data were not normally distributed, so we significance.
also checked between-group differences by means of Mann–Whitney When children contrasted both versions (i.e., after they were
U tests. In regard to time-on-task, the groups were not balanced with introduced to the second intervention version), they clearly pre-
respect to it. So, we also used ANCOVAs with time-on-task as a ferred the game (evaluation game variable: M = 1.19, SD = 0.41)
covariate. Because the t-test, the U test and ANCOVA results were over the animation (evaluation animation variable: M = 1.89,
similar, and because t-tests are robust with regard to normality viola- SD = 0.87), t(135) = −8.84, p < .001, d = 0.76. Note that means in
tions (e.g., Rasch, Teuscher, & Guiard, 2007), we report t-test results the previous sentence are reported across both groups, that is, no
as our main analytical output. Therefore, effect sizes are expressed in matter whether the child was assigned to the game or the anima-
terms of Cohen's d and classified into small (d 0.2), medium (d 0.5) tion (a group breakdown can be found in Table 4). Children clearly
and large (d 0.8) based on Cohen (1988). preferred the game in the free-choice period (Table 5). Children
Children with partially missing data were excluded only from ana- also preferred the game when asked what version they would have
lyses related to the missing data. chosen for learning, although their preferences were more balanced
in this last case and biased toward the version they interacted with,
χ 2(137) = 8.49, p = .004 (Table 6).
5 | RESULTS

5.1 | Descriptive results and control variables 5.4 | Research question 3: Expertise-reversal
effect
The descriptive results for all control variables are presented in
Table 3 and for all dependent variables in Table 4. As is apparent from Using a series of six ANCOVAs with a treatment factor and one of
the data, the groups were balanced in terms of all control variables— additional covariates (prior concepts, prior names or game experi-
except for time-on-task. However, this variable correlated only weakly ence), we explored whether the levels of each covariate moderated
with all dependent variables (Data S2: Table B1). So, its influence on the treatment's effect on learning outcomes (comprehension post,
between-group differences in dependent variables was negligible. transfer). For example, participants with higher game experience could
have higher transfer with the game, whereas participants with lower
game experience could have higher transfer with the animation. How-
5.2 | Research question 1: Learning outcomes ever, none of the six treatment×covariate interactions were signifi-
cant (ps > .088).
As is evident from Table 4, children working with the interactive ver-
sion did not learn better (or worse) compared with children learning
from the non-interactive version. We, therefore, report null results 6 | DI SCU SSION
concerning both comprehension and transfer.
This study examined a problem-solving interactivity hypothesis within
a game-based learning environment and in the case of 8- to 10-year-
5.3 | Research question 2: Motivational outcomes olds. The hypothesis states that problem-solving interactivity has
positive effects on learning outcomes and affective-motivational
Children rated enjoyment before they were introduced to the second variables. In this study, we found no support for the hypothesis as
intervention version. As it is evident from Table 4, children enjoyed concerns learning outcomes and partial support as concerns

TABLE 4 Dependent variables

Game Animation
including t-test results
Mean SD Mean SD t df p d
Comprehension gain 1.56 1.37 1.34 1.42 0.92 137 .361 0.16
Comprehension post 2.66 1.52 2.61 1.69 0.20 137 .838 0.03
Transfer 8.95 2.68 8.97 2.59 −0.05 137 .957 −0.01
Enjoyment 1.22 0.38 1.38 0.57 −1.88 137 .063a −0.32
Evaluation gameb 1.20 0.40 1.18 0.42 0.33 134 .738 0.06
Evaluation animationb 1.99 0.96 1.79 0.75 1.31 134 .192 0.23
a
The result of enjoyment using Mann–Whitney U test: U = 2,143, p = .176.
b
Lower scores mean higher enjoyment/better evaluation.
 13652729, 2020, 6, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12454 by Bbw Hochschule, Wiley Online Library on [06/04/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
992 TETOUROVÁ ET AL.

TABLE 5 Free-choice and binominal test results 6.2 | Interpretation of the results
Game Animation
p On a theoretical level, the findings can be interpreted as follows. First,
Free-choice 119 87.5% 17 12.5% <.001
to justify the view that the game was more motivating than the ani-
Test choice 84 61.3% 53 38.7% .010
mation, but the difference was not a large one, it is important to real-
ize two things: A) The more reliable motivational variable is behaviour
during the free-choice period (showing large effects favouring the
TABLE 6 Test choice results broken down by experimental interactivity) rather than learning enjoyment assessed using the smiley
condition scales (showing only a small trend). The reason is that the latter mea-
Choice sure has ceiling effect problems: Children's answers are frequently
skewed towards positive values (cf. Hannemann et al., 2018). B)
Group Game Animation Total
Learning from the animation most likely triggered a reasonable level
Game 50 18 68
of motivation-just a bit lower than the game. The reason is that had
Animation 34 35 69
interfacing with the animation been a boring experience, this would
have been detected by the learning enjoyment variable.
Second, the game could be slightly more mentally demanding
affective-motivational variables. The interactive intervention ver- compared with the animation, or it could even cause a mild cognitive
sion neither improved, nor hampered, learning outcomes (compre- overload; especially because children's mental resources are more lim-
hension, transfer). When children were shown both the interactive ited than those of adults (cf. Gathercole et al., 2004). However, these
and the non-interactive versions, they strongly preferred the inter- additional mental demands, if present, could not have been large. Oth-
active one (i.e., this was a within-subject comparison). However, erwise, we would have seen detrimental effects from the game on
the interactive version only tended to enhance learning enjoyment learning outcomes (instead, we saw null results).
(i.e., before the children were introduced to the second version; as To summarize, we think that learning from the game was a little
measured by a between-subject comparison). The null results as more motivating than learning from the animation. However, it could
concerns learning outcomes are particularly notable, as the study have also been a bit more mentally demanding. Neither of these
had reasonable power. Effects of interactivity on learning outcomes (opposing) effects was a big deal for children, and they probably
were not found to be moderated by levels of game experience and counterbalanced each other.
prior knowledge. How does this study fare in comparison to prior studies con-
ducted with adults? Prior studies (see Table 2) yielded a mixed spec-
trum of results as concerns learning outcomes: ranging from negative
6.1 | Contributions with large effect sizes to positive with large effect sizes. This study
lies somewhere in the middle of the road. Why have we not seen neg-
The fact that children preferred the game over the animation after ative results? Negative outcomes were reported only when the inter-
they were introduced both to the game and the animation is not par- activity condition was designed sub-optimally. This included situations
ticularly surprising. The point is that despite the fact that they pre- when the intervention was evidently more difficult (Song et al., 2014)
ferred the game, they did not learn better or worse from the game or its elaboration phase was problematic (Moreno & Mayer, 2005,
compared with the animation. Exp. 2). As we have already explained, these situations did not likely
On the theoretical level, the key point is that neither of the happen in the present case. So why have we not found positive
two main theories alone can explain our finding. Contrary to pre- results? There are four possible explanations to consider.
dictions from the cognitive-affective theory of learning from
media, and the expectations of game-based learning proponents, 1 Previously examined interactive treatments could have been better
children did not learn better from the game—despite the fact that than our game in targeting organizing and integrating processes.
they preferred it and thus were presumably more motivated by Because previous interventions were rarely described in a sufficient
it. However, contrary to predictions from cognitive load theory, way, it is difficult to determine whether this was the case or not.
they did not learn in a worse way either—despite the fact that one 2 The between-group difference in mental demand could be larger in
could have expected a higher cognitive load in the game our study than in previous studies. Because (as has already been
condition. said) additional mental demand imposed due to interactivity in our
The study has one less noticeable, but also important, out- study was unlikely a big deal for children, this explanation is
come: The children actually appeared to like the animation only implausible.
slightly less than the game, which is apparent from relatively high 3 The animation and the game in our study could be closer to each
scores of the enjoyment variable in the animation condition. other in motivational terms compared with previous studies. In
Therefore, even a learning experience with low levels of interactiv- other words, the difference between intervention versions could
ity can be fun. have been larger in the previous studies. This possibility cannot be
 13652729, 2020, 6, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12454 by Bbw Hochschule, Wiley Online Library on [06/04/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
TETOUROVÁ ET AL. 993

excluded. For example, Sawyer et al. (2017) used a 60–90 min. long settings, in which instructional materials “compete” for children's
game and Wang et al. (2011) used 2 hr long sessions, so the inter- attention, the results favour the interactive game. The latter did not
vention itself was probably also longer than an hour. Watching a hamper learning and the children preferred it (no learning would have
comparably long animation could become boring; our animation occurred, had the children not started to interact with the learning
was much shorter (Mtime = 13.60 min). Unfortunately, previous material in the first place).
studies did not report affective-motivational variables; except for On the theoretical level, it is important to highlight the link (previ-
Ritterfeld et al. (2009), who reported null results regarding gained ously rarely exposed) between studies on problem-solving interactiv-
topic interest. Notably, the intervention in the Ritterfeld et al.' ity within game-like environments (this one and those from Table 2)
study was also a short one (Mtime = 17.11 min). and worked example studies (see, for example, Renkl & Atkinson,
4 Positive effect of interactivity disappeared in the study by Moreno 2010; Renkl, 2017, for reviews). The latter examines learning from
and Mayer (2005; Exp. 3) when a proper elaboration mode was problem-solving versus from showing learners fully or partially solved
included, i.e., when learners had to explain correct solutions to problems (i.e., worked examples). The similarity comes from the
problems. In our study, elaboration was targeted by self-testing worked examples resembling the non-interactive (animation) condi-
through yes/no questions. These simple, closed-format questions tions in studies on game-based learning problem-solving interactivity.
represent arguably a less efficient elaboration mode compared with The problem-solving resembles the interactive conditions. The distinc-
what Moreno and Mayer did (see Fiorella & Mayer, 2015, Ch. 6 for tion between these two types of studies lies in the fact that those
rationale), but they could play a role, nevertheless. That is, answer- with worked examples (a) typically, but not always, concern acquisi-
ing these questions could, to some extent, target (in both our con- tion of cognitive skills (such as how to solve mathematical equations
ditions) organizing and integrating processes much like interactivity or a puzzle), and (b) rarely use game-like learning environments. How-
did in the game condition only. The effects of these questions and ever, the boundaries are not always clear cut (see, for example, van
of the interactivity (i.e., in the game condition) might not be fully Gog, 2011; van Gog et al., 2011; cf. also Wang et al., 2011).
additive. The point is that worked examples from various literature
strongly favours worked examples as opposed to problem-solving as
All in all, not enough data are available to reconcile the first, third concerns learning gains (see, for example, Renkl & Atkinson, 2010;
and fourth explanations described above. This is a task for future Renkl, 2017). Why does game-based problem-solving interactivity lit-
research. In our opinion, it would be especially useful to pay attention erature tend to favour games (which correspond to the problem-
to the motivational difference between the interactive and non- solving conditions) or report neutral results when studies with sub-
interactive conditions and the possible effects of elaboration strate- optimal interactivity are excluded? Is this because of game-like learn-
gies implemented within the treatments. ing environments' motivational power? Or because solving problems
Finally, we have not detected a moderation effect by prior knowl- within properly designed game-like environments does not over-
edge and game experience. The moderation effect by prior knowledge whelm mental resources, unlike in the problem-solving treatments in
probably has not been detected because all participants were most the worked examples research? Or is it due to different types of learn-
likely low-to-medium prior knowledge learners. When a participant ing (mental models acquisition vs. cognitive skills learning)? As things
had some prior knowledge, it was patchy and fragmented. We saw in currently stand, we do not know. These opposing patterns of findings
our data no “expert” with already existing, solid, mental models that beg for explanation and offer space for cross-field inspiration.
would merely need to be strengthened and expanded (for which For instance, can some game-based motivational approaches be
problem-solving can be particularly helpful; cf., e.g., Renkl, 2014). The used in the worked examples subfield? Or consider the following idea:
moderation effect by game experience might not have been detected worked examples studies examine various combinations of problems
because children did not really struggle with controlling the game. The and examples (e.g., example–problem–example–problem vs. problem–
game was child-friendly by design. example–problem–example; see, for example, van Harsel,
Hoogerheide, Verkoeijen, & van Gog, 2019 for a mini-review).
Starting with examples is probably more beneficial for initial knowl-
6.3 | Implications edge acquisition; whereas, problem-solving may be generally more
useful for advanced learners (e.g., Chen, Retnowati, & Kalyuga, 2019;
The fact that the children preferred the game rather than the anima- Reisslein et al., 2006; Renkl, 2014; but see also Coppens et al.,
tion in this study, but learned equally from both interventions, has 2019). By analogy, should animations that demonstrate how problems
useful practical implications. In formal schooling contexts, wherein are solved in a game-like environment be alternated with problem-
children can rarely choose learning materials themselves, the results solving within the environment? Should this order depend on learners'
favour non-interactive animations, as these are easier to implement in prior knowledge? In fact, in our study, several children suggested
schools (cf. Brom et al., 2015) and cheaper to develop. However, in exactly that in a post-study interview: that it may be beneficial to
informal settings, such as when children browse webpages with child- learn initially from the animation and then switch to the game (see
focused content in their leisure time, kids can choose among many Tetourova, 2019, foproblem-solving with information delivery
applications: both entertainment and educational ones. In these (i.e., delivery–problem–delivery–problem–…). In the context of mental
 13652729, 2020, 6, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12454 by Bbw Hochschule, Wiley Online Library on [06/04/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
994 TETOUROVÁ ET AL.

models acquisition, information delivery could play a similar role to All in all, the limitations do not undermine the study's key out-
that of showing a worked example in the context of cognitive skills comes, but there is, of course, room for improvement.
acquisition (i.e., information delivery supports construction of an initial
mental model much like a worked example supports initial skills acqui-
sition). Therefore, our procedure could have a similar impact on learn- 7 | CONC LU SIONS
ing processes that an example–problem–example–problem sequence
would have. This study showed that problem-solving interactivity that targets
All in all, more research is needed to elucidate how best to design mental models acquisition in a simulation game neither improved nor
instructional environments for learning from examples (or overseeing harmed learning in the case of 8- to 10-year-olds (compared to learn-
animations) and problem-solving (or interfacing with simulation game- ing from an information-equivalent animation). However, the children
like environments). Worked examples and game-based learning fields preferred the interactive intervention, when they could choose what
may benefit from each other. learning materials to interact with. This “half-full/half-empty-glass” of
results has important implications for instructional design: it favours
non-interactive interventions in settings where children cannot
6.4 | Limitations choose learning materials, but interactive ones when they can choose.
This study also contrasted the present findings with the findings from
We did not measure mental demands. This could have helped to inter- the worked examples literature (the latter is generally less favourable
pret the present findings. Mental demands are typically assessed using to problem-solving for initial knowledge acquisition). The worked
cognitive load measures (e.g., Brünken, Plass, & Leutner, 2003; examples and game-based learning subfields both study learning from
Leppink, Paas, Van Gog, van Der Vleuten, & Van Merrienboer, 2014). problem-solving, but from different perspectives. Highlighting the link
Unfortunately, we are unaware of such measures having been between these research strands paves the way for future cross-
validated for research using primary school children. This is a future fertilization between them.
challenge for general cognitive load research. We think, though, that
self-reports often used to tap mental demands would not likely work AC KNOWLEDG EME NT
for young audiences. The dual-task paradigm (e.g., Sweller et al., This study was conducted in inter-faculty Advanced Multimedia
2011, pp. 77–80) may, in our opinion, be more fruitful. Learning Laboratory (AMuLab). It was primarily funded by Czech
A possible criticism concerns the fact that we did not employ a Grant Science Foundation (GA ČR), Project “EduGames4K: Designing
yoked design, which would totally equate instructional content (across educational games for kids” (nr. 19-02532S). The project also received
matched pairs), save for interactivity. A consequence is that the time- institutional funding from Charles University (Project PRIMUS/
on-task differed between the conditions. However, in this study, cor- HUM/03 and Progress Q15). TT was partly supported by a student
relations between time-one-task and outcome variables tended to be grant (by Charles University's Faculty of Education) during preparation
small. At the same time, as already argued (Section 2.3), yoked designs of this manuscript. VŠ was partly supported by the European Regional
are problematic and could do more harm than good. Development Fund Project, “Creativity and Adaptability as Conditions
We had to exclude children with speech disorders due to the oral for the Success of Europe in an Interrelated World”
examination. Children with speech disorders could have been asked (No. CZ.02.1.01/0.0/0.0/16_019/0000734). We thank Anna Drobná,
to write down their answers, but not all children 8–10 years of age Karolina Faberová, Nikol Kopáňková, Nikola Sochová and Amalie
are able writers. An alternative would have been to design a multiple- Jaskmanická for helping with data collection. We also thank Tomáš
choice test for all children. However, based on our experience, it is Kozák for programming the experimental game, Radka Dvořáková for
better to use open-ended questions in a transfer knowledge test (see biological consultations, Jiří Lukavský for comments on the manu-
also Mayer, 2009). script, and Štěpánka Sunková and the whole New Media Dramaturgy
An obvious limitation is that we tested one particular game and Center, Czech Television (decko.cz) for helping us conduct the study.
one particular topic with a particular age group. We believe the find-
ings can be generalized to different treatments of similar length and CONFLIC T OF INT ER E ST
complexity; and also as concerns acquisition of mental models. Kristina Volná is employed by Czech TV, which is a public institution
However, the question remains whether they can be generalized to engaged (among others) in development of educational games. There-
different age groups; particularly, given the possible sensitivity of fore, she declares a potential conflict of interest. Other authors
problem-solving interventions to learners' mental capacity and their declare no conflicts of interest.
prior knowledge. It is also unclear whether our findings can be gen-
eralized to interventions of different lengths: the motivational bene- DATA AVAILABILITY STAT EMEN T
fits of game-based problem-solving may become more apparent in Because of the approval by the Ethics Review Board, it would be pos-
longer treatments. Finally, it would be beneficial if a similar study sible to share data only in ways it is impossible to identify individual
were conducted in a real environment: at school or during home children (for anyone, that is, including the children's parents and
learning. teachers). Anonymizing data would require removal of demographic
 13652729, 2020, 6, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12454 by Bbw Hochschule, Wiley Online Library on [06/04/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
TETOUROVÁ ET AL. 995

information, as this information could aid in identifying individual chil- Deci, E. L., Koestner, R., & Ryan, R. M. (1999). A meta-analytic review of
dren. Anonymizing would further require removal of all outliers experiments examining the effects of extrinsic rewards on intrinsic
motivation. Psychological Bulletin, 125(6), 627–668.
(e.g., children with very low or high prior knowledge), as this can also
Deci, E. L., & Ryan, R. M. (1985). Intrinsic motivation and self-determination
assist in identifying individual children. Only such a substantially in human behavior. New York, NY: Plenum.
reduced data set, of quite limited use, could, in principle, be shared. Faul, F., Erdfelder, E., Lang, A.-G., & Buchner, A. (2007). G* power 3: A
flexible statistical power analysis program for the social, behavioral,
and biomedical sciences. Behavior Research Methods, 39(2), 175–191.
ORCID
Fiorella, L., & Mayer, R. E. (2015). Learning as a generativeactivity: Eight
Cyril Brom https://orcid.org/0000-0001-5945-0514 learning strategies that promote understanding, New York: Cambridge
University Press.
ENDNOTES Gathercole, S. E., Pickering, S. J., Ambridge, B., & Wearing, H. (2004). The
1 structure of working memory from 4 to 15 years of age. Developmental
We tested whether the intervention and the learning topics are appropri-
Psychology, 40(2), 177–190.
ately complex (i.e., neither too easy nor too difficult) before the study
Hainey, T., Connolly, T. M., Boyle, E. A., Wilson, A., & Razak, A. (2016).
started (n = 45).
A systematic literature review of games-based learning empirical
2
For exploratory purposes, irrelevant for the present paper, eye move- evidence in primary education. Computers & Education, 102,
ments of 6 children from the game condition and 5 from the animation 202–223.
condition were registered by means of an eye tracker. Hannemann, O., Stárková, T., Javora, O., & Brom, C. (2018). Measuring
affective-motivational states of 8–10-year-olds playing learning
games: an explorative study. Poster presented at European Conference
RE FE R ENC E S of Game-Based Learning. Sophia-Antipolis, France.
Abdul Jabbar, A. I., & Felicia, P. (2015). Gameplay engagement and learning Hussein, M. H., Ow, S. H., Cheong, L. S., Thong, M. K., & Ebrahim, N. A.
in game-based learning: A systematic review. Review of Educational (2019). Effects of digital game-based learning on elementary science
Research, 85(4), 740–779. learning: A systematic review. IEEE Access, 7, 62465–62478.
Alfieri, L., Brooks, P. J., Aldrich, N. J., & Tenenbaum, H. R. (2011). Does Javora, O., Hannemann, T., Stárková, T., Volná, K., & Brom, C. (2019).
discovery-based instruction enhance learning? Journal of Educational Children like it more but don't learn more: Effects of aesthetic visual
Psychology, 103(1), 1–18. design in educational games. British Journal of Educational Technology,
Boyle, E. A., Hainey, T., Connolly, T. M., Gray, G., Earp, J., Ott, M., … 50(4), 1942–1960
Pereira, J. (2016). An update to the systematic literature review of Kalyuga, S. (2007). Expertise reversal effect and its implications for
empirical evidence of the impacts and outcomes of computer games learner-tailored instruction. Educational Psychology Review, 19(4),
and serious games. Computers & Education, 94, 178–192. 509–539.
Brom, C., Levčík, D., Buchtová, M., & Klement, D. (2015). Playing educa- Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance
tional micro-games at high schools: Individually or collectively? Com- during instruction does not work: An analysis of the failure of con-
puters in Human Behavior, 48, 682–694. structivist, discovery, problem-based, experiential, and inquiry-based
Brom, C., Šisler, V., Slussareff, M., Selmbacherová, T., & Hlávka, Z. (2016). teaching. Educational Psychologist, 41(2), 75–86.
You like it, you learn it: Affectivity and learning in competitive social Leppink, J., Paas, F., Van Gog, T., van Der Vleuten, C. P., & Van
role play gaming. International Journal of Computer-Supported Collabo- Merrienboer, J. J. (2014). Effects of pairs of problems and examples on
rative Learning, 11(3), 313–348. task performance and different types of cognitive load. Learning and
Brünken, R., Plass, J. L., & Leutner, D. (2003). Direct measurement of cog- Instruction, 30, 32–42.
nitive load in multimedia learning. Educational Psychologist, 38(1), Mayer, R. E. (2009). Multimedia learning (2nd ed.). New York: Cambridge
53–61. University Press.
Chen, O., Retnowati, E., & Kalyuga, S. (2019). Element interactivity as a Mayer, R. E. (2014). Computer games for learning: An evidence-based
factor influencing the effectiveness of worked example–problem solv- approach, Cambridge, MA: The MIT Press.
ing and problem solving–worked example sequences. British Journal of Mayes, J. (2015). Still to learn from vicarious learning. E-Learning and Digi-
Educational Psychology. https://onlinelibrary.wiley.com/doi/abs/10. tal Media, 12(3–4), 361–371.
1111/bjep.12317. [Epub ahead of print]. Moreno, R. (2005). Instructional technology: Promise and pitfalls. In Tech-
Clark, D. B., Tanner-Smith, E. E., & Killingsworth, S. S. (2016). Digital nology-based education: Bringing researchers and practitioners together
games, design, and learning a systematic review and meta-analysis. (pp. 1–19). Greenwich, CT: Information Age Publishing.
Review of Educational Research, 86(1), 79–122. Moreno, R., & Mayer, R. E. (2005). Role of guidance, reflection, and inter-
Clark, R. E., & Feldon, D. F. (2014). Ten common but questionable princi- activity in an agent-based multimedia game. Journal of Educational Psy-
ples of multimedia learning. In The Cambridge handbook of multimedia chology, 97(1), 117128.
learning (2nd ed., pp. 151–173). New York, NY: Cambridge University Ninaus, M., Greipl, S., Kiili, K., Lindstedt, A., Huber, S., Klein, E., …
Press. Moeller, K. (2019). Increased emotional engagement in game-based
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd learning–a machine learning approach on facial emotion detection
ed.). Hillsdale, NJ: Erlbaum. data. Computers & Education, 142, 103641.
Coppens, L. C., Hoogerheide, V., Snippe, E. M., Flunger, B., & van Gog, T. Patall, E. A., Cooper, H., & Robinson, J. C. (2008). The effects of choice on
(2019). Effects of problem–example and example–problem pairs on intrinsic motivation and related outcomes: A meta-analysis of research
gifted and nongifted primary school students' learning. Instructional findings. Psychological Bulletin, 134(2), 270–300.
Science, 47(3), 279–297. Rasch, D., Teuscher, F., & Guiard, V. (2007). How robust are tests for two
Craig, S. D., Sullins, J., Witherspoon, A., & Gholson, B. (2006). The deep- independent samples? Journal of Statistical Planning and Inference, 137
level-reasoning-question effect: The role of dialogue and deep-level- (8), 2706–2720.
reasoning questions during vicarious learning. Cognition and Instruc- Reisslein, J., Atkinson, R. K., Seeling, P., & Reisslein, M. (2006). Encoun-
tion, 24(4), 565–591. tering the expertise reversal effect with a computer-based
 13652729, 2020, 6, Downloaded from https://onlinelibrary.wiley.com/doi/10.1111/jcal.12454 by Bbw Hochschule, Wiley Online Library on [06/04/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
996 TETOUROVÁ ET AL.

environment on electrical circuit analysis. Learning and Instruction, 16 van Gog, T., Kester, L., & Paas, F. (2011). Effects of worked examples,
(2), 92–103. example-problem, and problem-example pairs on novices' learning.
Renkl, A. (2014). Toward an instructionally oriented theory of example- Contemporary Educational Psychology, 36(3), 212–218.
based learning. Cognitive Science, 38(1), 1–37. van Harsel, M., Hoogerheide, V., Verkoeijen, P., & van Gog, T. (2019).
Renkl, A. (2017). Instruction based on examples. In Handbook of research Effects of different sequences of examples and problems on motiva-
on learning and instruction (2nd ed., pp. 325–348). New York, NY: tion and learning. Contemporary Educational Psychology, 58, 260–275.
Routlege. VanLehn, K., Graesser, A. C., Jackson, G. T., Jordan, P., Olney, A., &
Renkl, A., & Atkinson, R. (2010). Learning from worked-out examples and Rosé, C. P. (2007). When are tutorial dialogues more effective than
problem solving. In Cognitive load theory (pp. 89–108). New York: reading? Cognitive Science, 31(1), 3–62.
Cambridge University Press. Vansteenkiste, M., Sierens, E., Soenens, B., Luyckx, K., & Lens, W. (2009). Moti-
Ritterfeld, U., Shen, C., Wang, H., Nocera, L., & Wong, W. L. (2009). vational profiles from a self-determination perspective: The quality of moti-
Multimodality and interactivity: Connecting properties of serious vation matters. Journal of Educational Psychology, 101(3), 671–688.
games with educational outcomes. Cyberpsychology & Behavior, 12(6), Wang, P.-Y., Vaughn, B. K., & Liu, M. (2011). The impact of animation
691–697. interactivity on novices' learning of introductory statistics. Computers &
Sawyer, R., Smith, A., Rowe, J., Azevedo, R., & Lester, J. (2017). Is more Education, 56(1), 300–311.
agency better? The impact of student agency on game-based learning. Weber, R., Behr, K.-M., & DeMartino, C. (2014). Measuring interactivity in
International conference on artificial intelligence in education, video games. Communication Methods and Measures, 8(2), 79–115.
Springer. pp. 335–346. Wouters, P., van Nimwegen, C., van Oostendorp, H., & van der Spek, E. D.
Scheiter, K. (2014). The learner control principle in multimedia learning. In (2013). A meta-analysis of the cognitive and motivational effects of
The Cambridge handbook of multimedia learning (pp. 487–512). UK: serious games. Journal of Educational Psychology, 105(2), 249–265.
Cambridge University Press. Wouters, P., & van Oostendorp, H. (2017). Overview of instructional tech-
Snow, E. L., Allen, L. K., Jacovina, M. E., & McNamara, D. S. (2015). niques to facilitate learning and motivation of serious games. In
Does agency matter?: Exploring the impact of controlled behaviors Instructional techniques to facilitate learning and motivation of serious
within a game-based environment. Computers & Education, 82, games (pp. 1–16). Cham, Switzerland: Springer.
378–392.
Song, H. S., Pusic, M., Nick, M. W., Sarpel, U., Plass, J. L., & Kalet, A. L. SUPPORTING INF ORMATION
(2014). The cognitive impact of interactive design features for learning
Additional supporting information may be found online in the
complex materials in medical education. Computers & Education, 71,
Supporting Information section at the end of this article.
198–205.
Sweller, J., Ayres, P., & Kalyuga, S. (2011). Cognitive load theory. New York,
NY: Springer.
The jamovi project (2019). jamovi (Version 1.0) [Computer Software]. How to cite this article: Tetourová T, Hannemann T, Javora O,
Retrieved from https://www.jamovi.org Volná K, Šisler V, Brom C. To solve or to observe? The case of
Tetourová, T. (2019). Interactivity principle in multimedia learning materials problem-solving interactivity within child learning games.
in 3rd and 4th grade primary school students. Master thesis. Faculty of
J Comput Assist Learn. 2020;36:981–996. https://doi.org/10.
Education, Charles University [in Czech]
van Gog, T. (2011). Effects of identical example–problem and problem– 1111/jcal.12454
example pairs on learning. Computers & Education, 57(2), 1775–1779.