Journal of Eye Movement Research

10(1):3,1-18

Eye tracking in Educational Science:

Theoretical frameworks and research
agendas
Halszka Jarodzka
Open University of the Netherlands, the Netherlands
& Lund University, Sweden

Kenneth Holmqvist Hans Gruber

Lund University, Sweden University of Regensburg, Germany
& UPSET, NWU Vaal, South Africa & Turku University, Finland

Eye tracking is increasingly being used in Educational Science and so has the interest of
the eye tracking community grown in this topic. In this paper we briefly introduce the
discipline of Educational Science and why it might be interesting to couple it with eye
tracking research. We then introduce three major research areas in Educational Science
that have already successfully used eye tracking: First, eye tracking has been used to im-
prove the instructional design of computer-based learning and testing environments, often
using hyper- or multimedia. Second, eye tracking has shed light on expertise and its devel-
opment in visual domains, such as chess or medicine. Third, eye tracking has recently
been also used to promote visual expertise by means of eye movement modeling examples.
We outline the main educational theories for these research areas and indicate where fur-
ther eye tracking research is needed to expand them.

Keywords: applied eye tracking, education, learning, expertise

best investigated field of applied eye tracking research

(Rayner, 1998, 2009). Later on, usability and human-
Introduction computer-interaction researchers discovered the value of
eye tracking for their purposes (Jacob & Karn, 2003). A
Eye tracking has been developed to measure ‘where rather young field of applied eye tracking research is the
we look at’. For a long time and up until now, optimizing one of Educational Science that we would like to intro-
the apparatuses to measure accurately and unobtrusively duce here to the readers. Let us begin with what Educa-
how the eyes move, considerations which eye movements tional Science actually entails.
can be distinguished from a neurological perspective (cf.
the discussion of whether post-saccadic oscillations are Educational Science investigates how people learn
separate eye movements or belong to saccades), and and how this learning can be fostered with instruction.
developing software to detect these different types of eye But what is learning? Kids at school begin with decipher-
movements were in focus. These topics are still ongoing ing single letters and end up analyzing complex texts and
and there is still plenty room for this fundamental eye relate these to accompanying graphs or pictures. Univer-
tracking research. But already from the beginning, these sity students begin with studying countless facts over
apparatuses were used – irrespective of the many funda- years to finally become highly specialized experts who
mental unknowns and imperfections – to apply them to effortlessly diagnose complex problems. Hence, learning
answer research questions from other fields. This applied is the act of acquiring or improving knowledge, skills or
eye tracking research field began with letting people view behavior. Its result is a persistent change of these. Learn-
art paintings (Yarbus, 1967). Quickly linguistics jumped ing follows a trajectory from an initial encounter with a
onto the eye tracking train and this became probably the topic or task, such as studying a textbook page for 30
minutes, to mastering it on high levels of expertise, in
Received: October 17, 2016; Published: February 4, 2017
Citation: Jarodzka, H., Holmqvist, K., & Gruber, H. (2017). Eye
tracking in Educational Science: Theoretical frameworks and
research agendas. Journal of Eye Movement Research, 10(1):3,1-18 1
Digital Object Identifier: 10.16910/jemr.10.1.3
ISSN: 1995-8692
This article is licensed under a Creative Commons Attribution 4.0
International license.
Journal of Eye Movement Research Jarodzka, H., Holmqvist, K., & Gruber, H. (2017).
10(1):3,1-18 Eye tracking in Educational Science:Theoretical frameworks and research agendas

professional development lasting for decades. Thus, learning materials, such as textbooks or e-learning set-
learning is rather a process than merely an outcome, such tings. But they may also be working environments, such
as a grade or a diploma. Researchers in Educational Sci- as a surgical room for medical residents or a flight simu-
ence investigate this process to understand how learning lator for pilots. Often, they can be so information-rich
is constituted and how it can be fostered through instruc- that they can easily overwhelm the learner. Basically,
tion. there are two possibilities to deal with this issue. First, the
environment can be adapted to the learner. This approach
Eye tracking (Holmqvist et al., 2011) has become an
is most effective for initial stages of learning and is called
important tool to investigate learning processes over the
Instructional Design. Instructional material that is de-
past years. The reason for this is that we take most infor-
signed to optimally make use of the human cognitive
mation in via our eyes; this is true when we learn, but
information processing system as well as the abilities of
also when we execute a professional task. Consider for
the learner enables the learner to autonomously and effi-
instance scientific illustrations. Such illustrations on the
ciently make progress. In later stages of learning, it is
composition or functioning of diverse systems have been
important to encounter the environments in their full
around since hundreds of years. Below you see an exam-
complexity. This is for instance the case in workplace
ple from the 19th century (Lilienthal, 1889) on the flight
learning. In such cases, the second option comes into
of birds (Figure 1). Not only professionals had to deal
play, namely, scaffolding the learner to the environment.
with such illustrations, but also students had to use them
This part of educational research is called expertise de-
to study the subject matter. Nowadays, with increasing
velopment. Again with the long-term aim to enable the
possibilities to create visualizations, their use, but also
learner to autonomously develop. The theories used in
their variability has mushroomed. For instance, profes-
Educational Science are based on findings from funda-
sionals have to operate complex computer-generated
mental research on cognition and perception, but are at
simulations (e.g., interactive 3D medical images), while
the same time applicable to concrete educational practice.
students have to learn from all sorts of visualizations,
such as videos, and often they have to integrate infor- In the following we will describe these two areas of
mation from many sources. And these are just few exam- research in education with concrete examples from our
ples of where eye tracking can aid in understanding and own research. Next we will show how both areas can be
even improving learning and its instruction within Educa- integrated into a training method of visual expertise,
tional Science. called eye movement modeling examples.

Instructional Design – adapting the

environment to the learner’s abilities
Theories of human learning – the working memory
perspective
Let us begin with the initial stage of learning: a per-
son who has little prior knowledge on a topic wants to
learn new facts from a textbook, for instance about the
functioning of a car engine. The material presented in this
book contains a text describing the functioning of this
engine, but also several graphs that show how the differ-
ent elements of the engine would move at different stages
Figure 1. Scientific illustration on the flight of birds. Otto of the stroke cycle. This person might experience quite
Lilienthal, Der Vogelflug als Grundlage der Fliegekunst, some difficulties to relate all this information into one
Berlin, 1889. coherent mental model in his or her mind. He or she
Nowadays, learning often takes place in environments might be also distracted by a picture of a fancy car placed
that are rich in information. These environments may be on this page. The research area of Instructional Design

2
Journal of Eye Movement Research Jarodzka, H., Holmqvist, K., & Gruber, H. (2017).
10(1):3,1-18 Eye tracking in Educational Science:Theoretical frameworks and research agendas

investigates how to construct learning material that opti- efficient (i.e., as much content learnt within as little time
mally supports the learner. One very important aspect of as possible). The most established guidelines include
this is how the material is visually presented.
 Seeking coherence of information. First and fore-
The strongest focus in Instructional Design lies on the most it is crucial to avoid unnecessary information
(visual) flow and processing of information to and within presented in instructional material, such as decorative
working memory. This view is based on (a simple version pictures. As the learner tries to make sense out of eve-
of) Baddeley’s working memory model (Baddeley, 2012) ry information given and integrate it with the other
and Paivio’s dual coding theory (Paivio, 1991). The two presented information and with own prior knowledge,
most influential theories on Instructional Design are the irrelevant information will only unnecessarily con-
Cognitive Theory of Multimedia Learning (CTML: sume cognitive capacities.
Mayer, 2009) and the Cognitive Load Theory (CLT:
 Avoiding redundant information. The exact same
Chandler & Sweller, 1991). Both theories assume that (a)
information should not be given in different formats,
the working memory capacity is limited and learning can
because the learner tries to integrate all information
only take place if enough capacity is available and not
with each other as well as with prior knowledge. This
consumed by ‘bad’ Instructional Design. Moreover, (b)
in turn costs cognitive capacities, which are not avail-
learning only takes place if the learner actively engages
able for learning any more. One common ‘bad’ exam-
with the learning material or the task. The Cognitive Load
ple is presenting a text on the slides and reading it out
Theory (Chandler & Sweller, 1991) mainly states that the
loud at the same time.
working memory capacity can be consumed by different
types of load that can either be attributed to the difficulty  Making use of multimedia. Even though the exact
of the task itself (known as intrinsic load), ineffective same information should not be presented in different
layout of the instructional material (known as extraneous modalities, preferable the same subject matter should
load), or active elaborations on the task content (known be presented in different ways. For instance, an ex-
as germane load). Only the latter results in learning. The planation of a car engine is easier to understand with
Cognitive Theory of Multimedia Learning (Mayer, 2009) an accompanying picture or animation.
focuses on the working memory’s dual coding in interac-
tion with the instructional material and long-term  Making use of different modalities. To account for
memory. This theory predicts how pictures and words are the dual-coding characteristics of working memory,
processed in working memory depending on their mo- instructional material should present related infor-
dality (written or spoken) and integrated with long-term mation in different modalities. For instance, a graph
memory content. For learning to occur, relevant infor- accompanied by an audio text instead of a written
mation from the material must be visually selected and text.
integrated, organized in mental models and integrated  Avoiding split attention by seeking contiguity. In-
with prior knowledge. If this happens, a person learned. It structional material should present related information
is easy to see that the theories include statements on per- that needs to be integrated in closely, both in space
ceptual processes (e.g., visual search of relevant infor- and time. For instance, the legend of a graph should
mation; integration of information from different better be incorporated in the graph itself than present-
sources), although these processes were not directly test- ed on the side.
ed when these theories were formed.
More principles were developed over time and fill en-
Both theories result in astonishingly similar guide- tire textbooks (Mayer, 2009), but these are the most fun-
lines on how to design (the layout of) instructional mate- damental ones. These guidelines sound valid and were
rial (Mayer & Moreno, 2003; Sweller, Van Merriënboer, often supported by empirical studies – but not always.
& Paas, 1998). The aim of these guidelines is to decrease
unnecessary cognitive processes (i.e., extraneous load) Testing learning theories in educational practice
and to foster cognitive processes leading to learning (i.e., The above described theories were developed based
germane load). These guidelines shall make learning on many empirical studies that were conducted under
specific circumstances. We will exemplify this with the

3
Journal of Eye Movement Research Jarodzka, H., Holmqvist, K., & Gruber, H. (2017).
10(1):3,1-18 Eye tracking in Educational Science:Theoretical frameworks and research agendas

studies of Mayer (for an overview of 15 years of studies: two versions of this material (Figure 2): In one version,
Mayer, 2009) and describe how new studies should en- all additional material was presented on one side of the
rich these findings. First, most studies were conducted screen and the task text on the other. This is a classic
with psychology students as participants. This is common split-attention design as the pupils must visually search
research practice, as psychology students have to partici- for the related information. In the second version, all
pate in research for course credits and form the backbone additional material was placed within the text, right
of a lot of psychological research. For many research where it was referred to. This corresponds to a classic
topics that should be equal across humans (e.g., percep- integrated design as it allows the pupils to process the
tion, memory) psychology students are valid participants. multimedia information right when it is needed.
For educational research, however, they represent a pre-
selected group with very specific characteristics that may
influence the outcomes (e.g., in Germany only students
with very high grades are allowed to enter psychology
study). Thus, we argue that it is crucial to test the actual
target group of a learning material when investigating
educational principles. Second, the illustrations used were
very specific. Mayer used in most studies short black and
white drawings (animated or static) showing the for-
mation of lightning (or a bicycle pump). Of course it was Figure 2. The computer-based testing environment in a
important to keep the material constant when investigat- split (left) and in an integrated design (right). Adapted
ing different principles. Nowadays, however, we must from Jarodzka, Janssen, et al. (2015), pp. 808 & 809.
acknowledge that this was a very specific format (simple
black & white drawings) and a specific topic (shouldn’t
lighting formation be known to university students?). Surprisingly, pupils achieved better test scores in the
Third, these studies used short, one sentence texts in split-version of the test (50% correct, vs. 44% correct in
English. This may have caused artefacts in the findings. the integrated format). Eye tracking data showed that
For instance, research suggests that a modality effect only pupils largely neglected the additional information in the
occurs for short sentences, while for long sentences only split-design (32 sec fixation time). Contrary to the predic-
the last part is affected (Rummer, Schweppe, tions of the CTMML (Mayer, 2009), pupils did not put a
Fürstenberg, Scheiter, & Zindler, 2011) or that it might lot of effort to integrate the related information that
even occur only for English text (Lindow et al., 2011). would have consumed up cognitive capacity (5 points on
We argue that it is necessary to test the guidelines and a 9-point score for both conditions). Actually, these pu-
principles found thus far on diverse material that proba- pils were ‘lazy’ (or clever!) and ignored everything that
bly uses more up-to-date multimedia. they figured was not mandatory to solve the task. This
In the following, we present two examples, where eye was indeed the better strategy as it turned out that this
tracking shed light on the processes underlying these additional information was not crucial to solve these tasks
effects that were carried out in ecologically valid scenari- correctly. So was the integrated design pointless? On the
os. In the first example, we tested the split-attention contrary! Eye tracking results showed that exactly the
effect (Jarodzka, Janssen, Kirschner, & Erkens, 2015). In same pupils processed all information in the integrated
our study, we used multimedia material on the topic of design (44 sec fixation time). Hence, they might have
arts that is used nation-wide for assessment of all Dutch built a richer mental model in these cases. Probably, the
pupils at secondary school level. Moreover, our partici- test items were just not appropriately designed to tackle
pants were 16 years old pupils. So, we used ecologically this richness of the mental model. Either way, learners
valid material that was tested with the actual target group. might not always be as eager to actively process all given
The material itself consisted not only of one task, but of information as multimedia theories assume them to be.
eight tasks. Each task consisted of a text paragraph de- In another example, we investigated the multimedia
scribing the task background and additional multimedia effect (Ögren, Nyström, & Jarodzka, 2016). In this study,
material, such as pictures, text or videos. We compared we used multimedia material on the topic of vector calcu-

4
Journal of Eye Movement Research Jarodzka, H., Holmqvist, K., & Gruber, H. (2017).
10(1):3,1-18 Eye tracking in Educational Science:Theoretical frameworks and research agendas

lus. Again, for our participants this was relevant educa- culty, looking at the area where the task performance
tional material, as they were university physic students. actively takes place (here: calculation area) was positive-
These students solved eight tasks. Each task was com- ly correlated with task performance. Consequently, we
posed of a text describing the problem including a formu- must specify the CTMML based on our findings: it is not
la, and a statement about this formula that the students enough that the learners process a graph; they must pro-
had to confirm or reject (i.e., task performance). Addi- cess it in the context of the main task question. Only then
tionally, half of the problems included a graph that pre- graphs are beneficial, otherwise they might even pursue
sented one exemplary instance of the formula (Figure 3). learners to be uncritical. Moreover, a recent study by
Krejtz, Duchowski, Krejtz, Kopacz, and Chrzastowski-
Wachtel (2016) has shown that the type of graph that is
presented plays a role: interactive graphs evoke most
systematic text-graph integrative saccades than static or
dynamic graphs. Future research should investigate,
whether this has also a positive effect on learning out-
comes.

Research agenda for Instructional Design theories

Figure 3. Exemplary task from the multimedia condition.
We can conclude from these two examples already
Adapted from Ögren et al. (2016).
that eye tracking can help to explain unexpected findings,
as it allows unique insights into processes underlying
CTMML would predict that such an additional visual- learning outcomes. One possible reason for the unex-
ization should enrich the mental model the students are pected findings might be that the perceptual processes
building and thus, lead to better performance. This was assumed by multimedia theories (CTML, CLT) were not
not what we found (56% correct with graphs vs. 52% directly tested with eye tracking when these theories were
correct without graphs). Instead, we found a bias in stu- developed. These theories have been very helpful heuris-
dents to confirm the statement, if a graph was present tics to design instructional material. However, now we
(65% confirmation vs. 47% rejection). This is in line with must unravel new evidence to further develop, specify,
findings that scientific pictures make text appear more correct, and form these theories. The following issues
credible (McCabe & Castel, 2008). Hence, our students should be considered in future eye tracking research to
probably saw the graph, judged it as being correct and achieve this:
concluded the same for the statement. Eye tracking data  In the latter example presented above (Ögren et al.,
revealed that in the multimedia tasks, students paid less 2016), we saw that the guidelines given, might need
attention to the task description (50% vs. 40%) and to the to be specified. Hence, it is crucial to test also the
statement (45% vs. 40%) – obviously, as they also looked other guidelines for Instructional Design with eye
at the graph (20%). The amount of looking at the graph tracking, but also under ecologically valid circum-
was not related to task performance (dwelling on graph stances (i.e., actual learning material with real stu-
when answering correctly 20% vs. incorrectly 19%). dents).
Looking at the statement, however, was positively related
to task performance (dwelling on statement when answer-  In the first example above (Jarodzka, Janssen, et al.,
ing correctly 43% vs. incorrectly 38%). Also, many tran- 2015), we saw that even if assuming that those guide-
sitions between the statement and the graph were posi- lines are appropriate, some basic pre-assumptions of
tively related to task performance (correct answer: 9 these theories might not be (e.g., that learners do
transitions vs. 6 transitions for incorrect answers). Lin their best to actively integrate material). Hence, it is
and Lin (2014) received similar findings when investigat- crucial to test also these. In particular, the many as-
ing geometrical problem solving with eye tracking: while sumptions about perceptual processes must be tested
looking at the graph was an indicator for perceived diffi- directly with eye tracking.

5
Journal of Eye Movement Research Jarodzka, H., Holmqvist, K., & Gruber, H. (2017).
10(1):3,1-18 Eye tracking in Educational Science:Theoretical frameworks and research agendas

 The research discussed thus far considered cognitive problems described in text form. They quickly see the
processes. However, metacognitive processes are al- crucial cues that indicate which type of formula should be
so crucial for learning (i.e., monitoring what I already used. Based on this info, they know which other infor-
can do what I still need to practice). However, too lit- mation they have to search for in the text and which they
tle research has been conducted on this important top- can ignore to fill in the formula. Next they solve the for-
ic until now (Van Gog & Jarodzka, 2013). mula and formulate a solution to the problem. This pro-
cedure describes an exemplary use of a schema (Van
 Finally, eye tacking research is conducted in laborato-
Lehn, 1996). Similar to a chunk, a schema is not only an
ries where one participant at a time is tested under
efficient way to store information in long-term memory,
minimal disturbance. This has, however, nothing to
but it also expands working memory: one entire schema
do with educational practice. From social psychology
functions as only one entity. Thus, plenty capacity is left
research, we know that performing a task in the pres-
over to collect new information to fill in the schema’s
ence of others might be inhibiting, but also facilitating
empty slots. If a schema includes a specific temporal
(Bond & Titus, 1983). Eye tracking research also
order, such as visiting a restaurant (enter a restaurant,
shows effects of social presence on attention (Oliva,
look for a table, order from menu, …), it is called a script
Niehorster, Jarodzka, & Holmqvist, in press;
(Schank & Abelson, 2013). Another form of knowledge
Richardson et al., 2012) Hence, future eye tracking
organization is forming short-cuts within long chains of
research should investigate social effects on processes
reasoning by encapsulating parts of it into entities that are
of learning, for instance within so-called digital class-
only unfolded into its pieces if necessary (Boshuizen &
rooms.
Schmidt, 1992; Schmidt & Boshuizen, 1992). The more
It has to be noted that eye tracking – in particular in knowledge a person has in a task and the more efficient it
methodological triangulation with other process data – is organized, the faster and more correct this person can
cannot only be used to derive instructional guidelines, but execute this task. Until he or she eventually becomes an
also to concretely usability test concrete computer-based expert (Ericsson, Charness, Feltovich, & Hoffman, 2006).
multimedia learning environments. For a comprehensive For certain professions, such as medicine, we already
description on how to proceed in such a case, see Groner know so much from research that these short-cuts and
and Siegenthaler (2009). organizations of knowledge can be described very specif-
ic (Jarodzka, Boshuizen, & Kirschner, 2012). In the cur-
rent section, we specifically focus on visual expertise and
Expertise development – scaffolding the what we know so far about its knowledge and skill organ-
learner to the environment ization.

Theories of human learning – the long-term

memory perspective The specific case of visual expertise.

So far, we have looked into initial learning processes. Expertise is defined as a consistently superior perfor-
The more a person knows about a task or a domain, the mance on a specified set of representative tasks for a
more we must take the long-term memory into account as domain (Ericsson & Lehmann, 1996; Ericsson & Smith,
well. In the long-term memory all knowledge is stored 1991). This superiority is due to the above described
and with increasing experience in a task it is re- efficient organization of large amounts of knowledge and
organized. This knowledge organization, in turn, chang- skills in a domain. This efficient knowledge organization
es the deal for the working memory. It changes it to this reflects in different aspects, depending on the task itself.
extent that Ericsson and Kintsch (1995) suggested the One example is the above mentioned well documented
concept of long-term working memory. For instance, cognitive chunking in chess (Chase & Simon, 1973; De
with increasing numerical skills, children do not have to Groot, 1946/2008). Typically, expert and novice chess
memorize six digits separately, but can form two chunks players are asked to build chess formation from memory;
of three digits each and thus increase their working a task in which experts excel largely (Freyhof, Gruber, &
memory capacity (Miller, 1956). With ongoing mathe- Ziegler, 1992; Gruber, 1991). Eye tracking research re-
matical education, children can even solve mathematical vealed that this chunking is also reflected in perceptual

6
Journal of Eye Movement Research Jarodzka, H., Holmqvist, K., & Gruber, H. (2017).
10(1):3,1-18 Eye tracking in Educational Science:Theoretical frameworks and research agendas

processes: experts look rather in between chess figures, so that it can be integrated. Consequently, the stimuli
while novices look at each single figure (Reingold, used in most visual expertise research so far are not rep-
Charness, Pomplun, & Stampe, 2001; Reingold & resentative for most expertise domains. Thus, we cannot
Sheridan, 2011). We see that the concept of chunking in simply generalize these findings to information-rich or
chess is reflected in two aspects: a cognitive recall per- even dynamic domains. Research in this field, is in focus
formance and perceptual processes. Similar findings of the following section.
occur also in other domains of expertise, such as playing
music (Lehmann & Gruber, 2006). In most cases, reading
from notes is an important part of playing music and thus, Research on visual expertise in information-rich
it is one aspect of musical expertise that is investigated environments
with eye tracking (Arthur, Blom, & Khuu, 2016;
Penttinen & Huovinen, 2011; Penttinen, Huovinen, & The concept of visual expertise is difficult to tackle as
Ylitalo, 2013, 2015). it entails so many different aspects (as already described
above). In most cases, it is thus necessary to approach
Reingold and Sheridan (2011) provide a comprehen- this concept from different angles by means of methodo-
sive overview of eye tracking research on visual exper- logical triangulation (Denzin, 2012; Thurmond, 2001).
tise. The authors draw two main conclusions from their One the one hand, eye tracking can tackle the perceptual
review. First, experts are able to encode domain related aspects of visual expertise, while other data sources com-
patterns in a superior way, which is due to their larger plete the picture on the more cognitive side, such as per-
visual span. Second, eye tracking data of experts often formance data, verbal data, and even drawings of what a
entails information that they were not aware of. This is a person thinks where he or she looked at. Due to the na-
clear indicator of experts’ tacit knowledge. The increased ture of this concept and the research tradition, verbal data
visual span is a reflection of the above described chunk- are most often used to investigate expertise (Ericsson &
ing in perceptual processes. The tacit knowledge could be Simon, 1993). They can take the form of interviews, self-
linked to encapsulated knowledge and its automated use. explanation, retrospective reports or thinking aloud (for
When reading this review, you will quickly realize that an overview of different forms of verbal data and how to
most research was conducted on the traditional expertise combine them with eye tracking see Chapters 3.4.8 and
domains of chess and medicine. These studies used static 4.7.3 in Holmqvist et al., 2011). If implemented carefully,
and perceptually simple stimuli, such as chess boards or verbal reports will not disturb the actual task perfor-
X-rays of the chest. mance. Instead, they will give us more information on the
However, a lot of visual expertise plays a role in per- reason why a person looked at a certain area. In the fol-
ceptually much more complex environments, such as air lowing, we present examples from own research using
traffic control (Beck, Trenchard, Van Lamsweerde, Gold- this methodological triangulation for investigating visual
stein, & Lohrenz, 2012), new medical imaging techniques expertise and its knowledge organization in information-
(Bertram et al., 2016), meteorology (Stofer & Che, 2014), rich environments.
etc. These environments are difficult for cognitive pro- One reason that this field is still so little investigated
cessing for two reasons (Atkinson & Shiffrin, 1968; (Reingold & Sheridan, 2011) besides its obvious rele-
Chandler & Sweller, 1991; Mayer, 2009). First, they are vance as described above, are software issues. In 2010,
information-rich (Dwyer, 1976; Schnotz & Lowe, we published the very first article investigating visual
2008). Hence, they entail large amount of information; expertise with eye tracking using video material and an
and a lot of it is irrelevant. On top of that, the relation of AOI analysis (Jarodzka, Scheiter, Gerjets, & Van Gog,
thematic relevance and visual saliency is often not opti- 2010). This study investigated expertise in the domain of
mal. Hence, it is challenging to select the relevant infor- marine zoology. In other words, seven professors and
mation. Moreover, these environments are dynamic PhD students, and 14 biology students classified the
(Hegarty, 1992; Lowe, 2003). Thus, information may be swimming modes of reef fish. In reality, marine zoolo-
transient. Also, several information elements may appear gists often execute their profession under water (either
(and disappear) simultaneously (cf. split-attention effect). snorkeling or diving). To get as close as possible to this
Consequently, it is challenging to keep information active situation, we asked participants to watch four videos of

7
Journal of Eye Movement Research Jarodzka, H., Holmqvist, K., & Gruber, H. (2017).
10(1):3,1-18 Eye tracking in Educational Science:Theoretical frameworks and research agendas

single fish swimming in a colorful reef for as long as they diagnoses), on the other hand, diagnosed these slides
wanted to. In this way, we created representative, but at equally well. However, they differed in how they pro-
the same time experimentally controllable tasks. After- cessed the slides. Experts relied on their first inspection
wards, they watched their own eye tracking recordings of the relevant area (fixation dispersion 1st trial part: 135)
and reported what they were thinking while approaching and then further checked the slide for other potentially
this task (Van Gog, Paas, Van Merriënboer, & Witte, relevant information (2nd trial part: 167). In their explana-
2005). As we wanted to compare where experts and tions they mainly focused on the typicality of the slide
where novices looked at, we used a cumbersome manual (e.g., high usage of comparative terms). Intermediates
procedure to define AIOs on videos, which delivered kept inspecting the relevant area throughout the entire
interesting findings: Experts clearly outperformed novic- trial duration (fixation dispersion 1st trial part: 192; 2nd:
es (experts: 4/4 points, novices: 3/4 points; ηp2 =.18), 165) and considered many potential diagnoses (e.g., a lot
which meant that they were indeed true experts in this of mentioning of pathologies). For their knowledge or-
task (not a trivial finding in expertise research!). Also, we ganization, we may conclude that experts have such con-
compared the sequences in which participants inspected solidated illness-scripts that they can rely on, which
the different body parts of the fish. Experts were more leaves them capacity to check for further potential prob-
diverse than novices (similarity of experts: 67%; novices: lems. Intermediates, instead, possess already according
72%; ηp2 =.08). Probably, novices just followed the most schemata, however, they still have to check many com-
visually salient features, which resulted in a rather similar peting schemata to reach a diagnosis. Even though this
scanpath. Experts, on the other hand, seem to have had study yielded interesting findings, the task we used was
different scripts to approach this task, which resulted in not really representative for this profession. Hence, in
different scanpaths. These different scripts might be due following studies (Jaarsma et al., 2015, 2016), we used a
to different forms of experience (i.e., when diving you digital version of a tissue sample that could be operated
see the fish from the side, while when snorkeling you see as under a regular microscope: zooming in and out as
it from above; consequently, you rely on different fea- well as panning around the slide. Hence, this was a highly
tures when classifying its motion). Indeed, dwell time representative task. Despite the progress in commercial
analyses of AOI data taken together with participants’ eye tracking software, using a stimulus that can be indi-
verbal reports, showed that part of the experts took a vidually changed that much (and that is not a website) is
short-cut: they first classified the fish and deduced from still challenging and requires a lot of manual work and
this, how it must swim (dwell time on according AOIs of programming. We found that experts were more efficient
experts: 375 ms; novices: 160 ms; ηp2 =.29; according as they used fewer microscopic movements (e.g., op-
verbal utterances of experts: 57; novices: 26; ηp2 =.56). In posed zooming movements: η2p = 0.03; expertise effect
sum, we found that visual expertise in marine zoology (a) for all navigation behavior: η2p= 0.11) and shorter reason-
leads to different types of scripts, probably depending on ing chains to reach a diagnosis (reasoning terms used by
the concrete experience in that task, and (b) which form experts: 109; intermediates: 63; novices: 159). This is in
these scripts can take. line with the findings from the first study that indicated
that experts possess consolidated illness-scripts that allow
In a following step, we moved towards an interactive
fast decision making. Also, navigation data showed that
task stimulus, namely digital pathology (Jaarsma et al.,
experts visited fewer diagnostically relevant areas (ex-
2016; Jaarsma, Jarodzka, Nap, Van Merriёnboer, &
perts: 3.05; intermediates: 3.98; novices: 4.05). This
Boshuizen, 2014, 2015). In the first study (Jaarsma et al.,
poses the question whether it is even possible to define
2014) we compared how participants of three expertise
areas as being relevant for each expertise group. It might
levels diagnosed 10 pathological slides based on a two
be difficult to grasp the effects, because experts under-
seconds inspection. They were eye tracked during this
stand the stimuli so quickly. Intermediates also showed
inspection and reported afterwards how they came about
processes that are in line with Study 1: they took longer
their diagnosis. Obviously, novices were incorrect (38%
to reach a decision (expert: 86 sec; intermediates: 110
correct diagnoses), incomplete and inconclusive in their
sec; novices: 152 sec) and looked more at relevant areas
diagnosis (hardly conclusive terms or diagnostic specifi-
while basing their diagnosis on many specific abnormali-
cations mentioned) and looked little at relevant areas (3
ties (novices: 35; intermediates: 96; experts: 94). Thus,
fixations). Experts (85%) and intermediates (87% correct

8
Journal of Eye Movement Research Jarodzka, H., Holmqvist, K., & Gruber, H. (2017).
10(1):3,1-18 Eye tracking in Educational Science:Theoretical frameworks and research agendas

intermediates already have established schemata. Howev- look more outside this area, including the starting and
er, they still need a lot of time to check them. Novices landing points of the planes. This strategy allowed them
again were simply all over the place and clearly lacked to plan ahead in this very dynamic environment. Hence,
any relevant knowledge (or its organization). the scripts individuals with higher expertise possess in
this task, must be updated dynamically if the task in-
Another expertise domain we have investigated is air
cludes more time pressure.
traffic control (Jarodzka, Gouw, Van Meeuwen, &
Brand-Gruwel, 2015; Van Meeuwen et al., 2014). Con-
trolling air traffic is a really challenging task: constantly
Research agenda for visual expertise research
flying in and departing airplanes need to be coordinated
with a high emphasis of safety, but also on environmen- From the research presented above, but also from oth-
tally friendly travel. 31 air traffic controllers of three er research on visual expertise of teachers (Lachner,
different expertise levels solved nine situations. Each Jarodzka, & Nückles, 2016; Wolff, Jarodzka, Van den
depicted a real radar screen, with airplanes (including Bogert, & Boshuizen, 2016), neurological pediatrists
type, height, and speed), sectors, and start and landing (Balslev et al., 2012), or radiology (Kok et al., 2015; Van
points. Participants reported the optimal order of arrival der Gijp et al., 2016) we have learned already a lot about
of the airplanes while their eye movements were record- visual expertise in information-rich environments. Ex-
ed. Individuals with higher levels of expertise clearly perts use chunks (e.g., air traffic control) and shortcuts
outperformed those of lower levels (experts: 4.63, inter- (e.g., marine zoology) and this can be also seen in their
mediates: 4.30; novices: 3.82; ηp2 = .49). Interestingly, perceptual processes and measured with eye tracking.
the performance of those with higher expertise was more Also, we have clearly seen the use of cognitive scripts or
similar than of those of lower expertise (experts: 0.59; schemata and their influence on the visual processing of
intermediates: 0.53; novices: 0.43; ηp2 = .44; in contrast an environment and vice versa in each profession. Often,
to our findings with marine zoologists: Jarodzka et al., even very concrete statements about the form of these
2010). In this profession it seems, thus, that there is one schemata or scripts could be made. Still, many open re-
optimal script to solve this task. Eye tracking analyses search questions remain.
revealed that individuals with higher expertise looked
 To which extent can we generalize these findings?
mainly at the aircrafts and at the background between
We have seen that sometimes even slight changes in
them (e.g., time to first fixation on aircraft for experts:
the task can lead to different outcomes (cf. air traffic
41.59 sec; intermediates: 54.6 sec; novices: 65.06; ηp2 =
control), while sometimes the changes go in the same
.37). This indicates that the script individuals with more
direction (cf. pathology). Also, some findings that are
expertise establish allows them to better focus on the
found in one profession (e.g., experts become more
relevant information and chunk single information enti-
similar in air traffic control) are not true for another
ties. Novices, on the other hand, had no appropriate strat-
profession (e.g., experts in marine zoology become
egy to relay on and fall back on the sub-optimal means-
more diverse). Hence, future research should consist-
end-strategy as indicated by them looking mainly at the
ently vary task characteristics and professions to
destination of the airplanes (e.g., time to first fixation on
understand, which aspects of visual expertise are ge-
destination for experts: 38.38 sec; intermediates: 36.62
neric and which domain-specific.
sec; novices: 25.37 sec; ηp2 = .36). We have to admit,
though, that participants only saw static screenshots of  A lot of research on visual expertise has been con-
radar screens. In a recent follow up study, we used a ducted on simplified tasks. This was largely due to
more representative task of this profession (Jarodzka. technological restrictions of the eye tracking appa-
Gouw, et al., 2015). In that twelve participants with vary- ratuses and software. Research should not be hold
ing expertise levels worked on a simulation of an actual back by technological obstacles, but rather feed their
airport. The situation was entirely realistic including development. In particular two issues must be tackled
communication with other co-workers. Already the first to foster ecologically valid research on visual exper-
eye tracking analyses reveal a drastic difference to the tise. First, the detection of smooth pursuit to enable
first study: novices mainly focus on the area of their own valid analysis of dynamic stimuli. Thereby, it is not
responsibility, while individuals with higher expertise enough to detect smooth pursuit with a stand-alone

9
Journal of Eye Movement Research Jarodzka, H., Holmqvist, K., & Gruber, H. (2017).
10(1):3,1-18 Eye tracking in Educational Science:Theoretical frameworks and research agendas

algorithm, but it must be implemented into existing experiment that imitation leads indeed to learning. Chil-
analysis software, so it can be used in applied re- dren watched videos of an adult playing with a ‘bobo
search as well. Second, more automated analyses for doll’, which is an inflatable, large doll that stands up
mobile eye tracking. Clearly, the truest way of analyz- again once it is tipped over. Depending on the experi-
ing visual expertise often requires real-world eye mental condition this adult was either behaving aggres-
tracking. However, cumbersome manual analyses of- sively (e.g., punching the doll) towards this doll or not.
ten hold researchers back. Once these children were confronted with this doll, they
treated it in a similar way as the model they saw in the
 The presented research has shown how much we can
video before (Bandura, Ross, & Ross, 1961).
benefit from methodological triangulation when in-
vestigating multifaceted concepts such as visual ex- Consequently, research on teaching and training has
pertise. In a next step, research should directly link picked up this approach. Indeed, decades of research have
the analysis of verbal and eye tracking data. Only shown that studying examples of a model successfully
in this way it will be possible to make more concrete executing a task is more efficient for learning than learn-
statements about the cognitive structures underlying ing by trial-and-error (Kirschner, Sweller, & Clark,
these processes. 2006). It is not trivial, though, to model a task. Many
critical processes are not observable from outside, such as
 Finally, it must become the ultimate aim of this re-
solving a mathematical equation. In such cases the model
search line to unravel the organization of knowledge
verbalizes his or her thoughts (cognitive apprenticeship:
and skills in long-term memory and how it develops
Collins, Brown, & Newman, 1989; process-oriented
with increasing expertise. Only then it is possible to
modeling examples: Van Gog, Paas, & Van Merriënboer,
draw meaningful conclusions from eye tracking data
2004). But what about perceptual processes in a visual
that go beyond superficial statements such as ‘experts
task? We know that simply telling beginners to “look the
had longer fixation durations’ that have virtually no
way experts do” does work, but does not necessarily
meaning for professional or educational practice (Kok
improve their performance (Kok et al., 2015). These
& Jarodzka, 2016).
beginners may now know where to look, but not why.
To address this issue, we developed eye movement
Eye movement modeling examples: Bridging modeling examples (EMME). These are video record-
Instructional Design and expertise research ings of a model executing a task and explaining how he
or she goes about that. On top of that, the model’s eye
Theories of human learning – training visual as- movements are tracked and replayed on top of the video
pects of expertise (Van Gog, Jarodzka, Scheiter, Gerjets, & Paas, 2009).
However, novices are often already overwhelmed with
So far, we have discussed how initial learning takes
information-rich material that forms the basis of visual
place, how it can be supported by Instructional Design,
tasks. Adding an eye movement display on top of that, is
and which role eye tracking can play in this. Then, we
likely to overwhelm them. An alternative is to display the
have shown how individuals develop further over time
model’s eye movements by reducing existing information
and until they become experts in visual domains. In this
on videos (Dorr, Vig, Gegenfurtner, Martinetz, & Barth,
section, we try to bring both research areas together to
2008; Nyström & Holmqvist, 2008). This results in a
show how this road to visual expertise can be supported
spotlight wandering across the video, while the rest of it
by instruction. This is not as trivial as it may sound, as
appears blurred. Figure 4 presents screenshots of both, a
Instructional Design entails the simplification of learning
traditional and a spotlight display used in EMME.
material, while expertise development requires to be
faced with the authentic, information-rich tasks.
One very powerful way of learning authentic tasks is
imitation. It is so inherent to our system that even two
weeks old babies imitate adults (Meltzoff & Moore,
1977). Bandura (1977) has shown in his classic bobo doll

10
Journal of Eye Movement Research Jarodzka, H., Holmqvist, K., & Gruber, H. (2017).
10(1):3,1-18 Eye tracking in Educational Science:Theoretical frameworks and research agendas

time, their eye movements were recorded to study wheth-

er they actually did follow the eye movement display of
the model on the videos. In the testing phase, participants
watched four new videos without any form of guidance
or verbal explanation. They had the task to classify these
videos accordingly. While watching the testing videos,
participants’ eye movements were recoded to investigate
the efficiency of their visual search of relevant infor-
Figure 4. Eye movement modeling examples with a tradi-
mation on the videos. Then, they indicated via a ques-
tional dot display (left) and a spotlight display (right).
tionnaire how they interpreted this information. Results
Material used in Jarodzka et al. (2010).
showed that both EMME videos guided the eye move-
Research on eye movement modeling examples ments of the participants to the spots where the model
looked at (measured as coherence between the model’s
Research described in the last section has shown that
and the learner’s scanpath: Spot = 15.10; Dot = 15.11;
experts dramatically differ from novices. Hence, there is
Control = 12.07; ηp2 = .39). Moreover, in the spotlight
no point in trying to ‘make novices act like experts’.
condition, participants showed a more efficient visual
Consequently, in our research, we have always used a
search on testing videos (measured e.g., time to first
systematic way to make the expert model act more
fixation on relevant areas: Spot = 1236 ms; Dot = 1530
didactical. On the one hand, the models in our studies
ms; Control = 1632 ms; ηp2 = .11), while participants in
were always not only experts in their domains, but also
the dot group exhibited better interpretation performance
highly experienced in teaching this domain. Hence, they
in comparison to the control group (measured as % cor-
knew from experience which difficulties students face in
rect: Dot = 74%; Spot = 69%; Control = 67%; ηp2 = .12).
these tasks and how to best explain these tasks to them.
On the other hand, we used a specific recording proce- We have conducted a similar study in the domain of
dure to ensure that the EMME videos were of high quali- diagnosing epileptic seizures in infants (Jarodzka,
ty. First, to ensure a close relation of the voice and the Balslev, et al., 2012). The experimental procedure was
eye movements of the models, we first show them the just as in the study described above, except from the task:
task itself (e.g., a video recording of something they need participants watched videos of infants either suffering a
to classify). Only after they are familiar with this specific form of epileptic seizure or a differential diagnosis. Even
task, we begin with the recording. Such recording proce- though both tasks sound very different, they had crucial
dure have resulted tight gaze-voice couplings elsewhere commonalities: participants had to identify relevant body
(Richardson & Dale, 2005). Second, to shift the models’ parts (fins that were used to produce propulsion vs. limbs
focus from the task to the novice recipient, they evaluate that might be affected by the disease) and to describe how
their own recordings based on several questions: Will a exactly these body parts move. Based on these two steps,
student know what each term means? Is the task ex- a classification or a diagnosis, respectively, can be made.
plained in comprehensible enough terms for students? Is Also, these steps rely on a visual inspection of a video
it explained in enough detail? Are all information that a input. A further difference to the fish locomotion study
student needs contained? Are all contained information was that display of the eye movements: The traditional
really important? Such questions have shown to improve display was shown as a circle instead of a dot to not oc-
written communication of experts to novices (Jucks, clude relevant information on the video (e.g., a twitching
Schulte-Löbbert, & Bromme, 2007). Third, if necessary, eye). The spotlight display was far more subtle than in
the models could revise their recordings. the fish locomotion study. Results showed an overall
advantage of the spotlight display on attention guidance
We have used such EMMEs, for instance, to train the
in the learning phase (measured as Euclidean distance to
classification of the locomotion patterns of reef fish
model’s gaze: Spot = 210; Circle = 238; Control = 237;
(Jarodzka, Van Gog, Dorr, Scheiter, & Gerjets, 2013). In
ηp2 = .13), visual search (measured as e.g., time until
the learning phase, participants studied four videos with
looking at relevant area: Spot = 189 ms; Circle = 274 ms;
either a dot display EMME, a spotlight EMME or a video
Control = 289 ms; ηp2 = .13) and interpretation perfor-
with verbal explanations only (Figure 4). In the mean-

11
Journal of Eye Movement Research Jarodzka, H., Holmqvist, K., & Gruber, H. (2017).
10(1):3,1-18 Eye tracking in Educational Science:Theoretical frameworks and research agendas

mance (measured as % correct: Spot = 60% Circle = sient and not salient. This is where EMME were most
53%; Control = 50%; ηp2 = .11) in the testing phase. helpful.
Similar training approaches have been used for visual  The role of the eye movement display design is an
tasks, which require hardly prior knowledge (Litchfield & entirely understudied aspect. Apart from two studies
Ball, 2011; Mason, Tornatora, & Pluchino, 2013; (Jarodzka, Balslev, et al., 2012; Jarodzka et al., 2013),
Skuballa, Fortunski, & Renkl, 2015), for expertise tasks none has compared different designs directly even
(Leff et al., 2015; McNamara et al., 2012; Sridharan, though these studies indicate that this might be a cru-
Bailey, McNamara, & Grimm, 2012), and even for cial success factor for EMME. Results showed that
problem solving in dyads (Cherubini, Nüssli, & Dillen- reducing information on a spotlight manner guides
bourg, 2010). However, these studies did not test whether visual attention on EMME videos best. Also, the spot-
the found performance differences could be transferred to light facilitates visual search on testing videos most.
similar tasks (as we did on our studies), i.e., whether However, the interpretation of relevant features is on-
learning took place. Thus, strictly speaking, these cannot ly enhanced, if a holistic processing is possible during
be seen as educational studies. learning.
 Moreover, the role of didactizing the expert model,
as we have done in our studies, has not been directly
Research agenda for EMME investigated. In fact, most studies provide hardly any
EMME as similar gaze-based approaches may be description on how the model’s eye movements were
helpful in training visual tasks. Still, we should not be- collected. This is surprising as we know very well
come too enthusiastic, as there are also enough examples from research to which large extent experts and nov-
where these approaches had no (single conditions in the ices differ in their processing and how unlikely it thus
two studies reported above) or even detrimental effects is that forcing experts’ processes upon novices can
(Skuballa, Schwonke, & Renkl, 2012; Van Gog et al., hardly work.
2009). Hence, the question is not whether EMME does
 Finally, the EMME methodology could be embedded
foster the performance of visual tasks (or even visual
into well-established methods of expertise trainings.
expertise), but rather, under which circumstances in does
For instance, the 4C-ID training (Van Merriënboer &
so. We thus recommend the following research questions
Kirschner, 2007) is an elaborated model to design a
to be addressed in the future:
curriculum for complex tasks. It includes modeling
 The role of the task and the stimulus characteristics: episodes that might easily be filled in with EMME for
The research on EMME covers a diversity of tasks specific visual tasks. Another example is deliberate
(from insight problem solving, to performance only, practice (Ericsson, Krampe, & Tesch-Römer, 1993).
to transfer and learning) and a diversity of stimuli This method involves a detailed study of own and
(from simple line drawings to complex videos). A others performance. If the task includes visual as-
systematic variation and concrete description of these pects, studying the eye movements of an expert (or
factors should shed more light into when EMME are one owns) might provide additional benefits.
effective. For instance, existing studies already indi-
cate that the visual complexity of the task is crucial:
Van Gog et al. (2009) used a task that could be exe-
cuted without perceptual input and found negative ef-
fects of EMME on performance (Van Marlen, Van
Wermeskerken, Jarodzka, & Van Gog, in press).
Jarodzka et al. (2013) used a fish locomotion classifi-
cation task where all relevant information was visual
salient. EMME was in part helpful in this case.
Jarodzka, Balslev, et al. (2012) used a pediatric neu-
rology task, where the relevant information was tran-

12
Journal of Eye Movement Research Jarodzka, H., Holmqvist, K., & Gruber, H. (2017).
10(1):3,1-18 Eye tracking in Educational Science:Theoretical frameworks and research agendas

Discussion The third topic we have presented are eye movement

modeling examples. This is the youngest topic within the
In the current paper we have introduced Educational field of applied eye tracking research in Educational
Science as a field of applied eye tracking research. We Science, but nonetheless, a very promising one. It
have structured it along three topics, namely Instructional addresses the question, how visual expertise could be
Design, expertise development, and eye movement trained with the help of instructional videos of real-world
modeling examples. The topic of Instructional Design tasks that are explained by experts in the field. These
investigates how learning of a new skill or knowledge by videos include an overlay of these experts’ visual focuses
optimally designing the according learning material. to support the learner in connecting the verbal
Educational theories on human cognitive processing, in explanation of the expert to the real-world complexity of
particular in the working memory, resulted in guidelines the task. Of course, this research topic gives us practical
on how to design such material and which processes implications for educational practice. But it also provides
learners should devote to efficiently achieve learning interesting research questions apart from education, such
gains. Up until now, eye tracking helped us to understand as: how to best guide eye movements of people on
how learners actually process such instructional material, videos? How to support speech comprehension with
which was not always in line with what theory predicted. displaying the eye movements of the speaker to the
Future eye tracking research on this topic can thus help to listener? Etc.
further corroborate, improve, and enrich these theories. It is important to keep in mind that the area of applied
Not only to understand and support processes of initial eye tracking in Educational Science is clearly applied
learning, but also to better understand how we as humans research. This means that the tasks and stimuli used are
process information in working memory under realistic very diverse and less well controlled in comparison to
circumstances. fundamental experiments in vision science, for instance.
The topic of expertise development investigates the However, they are ecologically valid. This is crucial for
other side of the learning spectrum, namely people, who this research to allow drawing actual conclusions for
already have a lot of experience and knowledge on a task. educational practice. Therefore, research questions
How do they process information? How do they differ should always be developed together with stakeholders
from people with slightly less or more experience? A from educational practice. And the models or frameworks
large body of expertise research started already many derived in research should always be tested ‘in the wild’
years ago to expend towards visual processes underlying (aka schools, universities). But this also means that we
expertise and thus, eye tracking research. This research can learn a lot from this research field on real-world
showed that, indeed, changes in long-term memory processing, which in turn can be fruitful to establish new
structures that come along the development of expertise research question for fundamental research.
influence not only working memory processing, but also Furthermore, this research area is still relatively new.
visual processing of the environment and vice versa. This means that there are no well-established eye tracking
Future eye tracking research on this promising topic must measures, like in reading research, that can be clearly
dive into more real-world scenarios with diverse tasks related to concrete processes. This is due to the fact, that
and information-rich, dynamic environments. Not only there is simply less research conducted as, for instance, in
will we understand in this way more about the reading. But the ecologically valid nature makes it almost
development and characteristics of visual expertise, but impossible to hope for such simple relations: each
we will also better understand how long-term memory learning environment, each expertise domain are so
structures influence the way we see and interpret our inherently different in terms of tasks and stimuli that the
environment, both in every day and in challenging eye tracking measures have to be found each time anew.
situations. The process of finding the appropriate measures must not
be driven by what is given by the manufacturers. Instead,
it is important to work along existing theories and
carefully operationalize measures that are clearly related
to concrete hypotheses.

13
Journal of Eye Movement Research Jarodzka, H., Holmqvist, K., & Gruber, H. (2017).
10(1):3,1-18 Eye tracking in Educational Science:Theoretical frameworks and research agendas

Acknowledgements Bertram, R., Kaakinen, J., Bensch, F., Helle, L., Lantto,
E., Niemi, P., & Lundbom, N. (2016). Eye
This paper is based on two keynote speeches of the movements of radiologists reflect expertise in
first author at the 4th Polish Eye Tracking Conference, CT study interpretation: A potential tool to
Warsaw, Poland (2016) and the 7th Scandinavian Work- measure resident development. Radiology,
shop on Applied Eye Tracking, Turku, Finland (2016). 281(3), 805-815. doi:
10.1148/radiol.2016151255
The authors declare that there is no conflict of interest Bond, C. F., & Titus, L. J. (1983). Social facilitation: A
regarding the publication of this paper. meta-analysis of 241 studies. Psychological
Bulletin, 94(2), 265-292. doi: 10.1037/0033-
2909.94.2.265
References Boshuizen, H. P. A., & Schmidt, H. G. (1992). On the
role of biomedical knowledge in clinical
Arthur, P., Blom, D., & Khuu, S. (2016). Music sight- reasoning by experts, intermediates, and
reading expertise, visually disrupted score and novices. Cognitive Science, 16(2), 153-184. doi:
eye movements. Journal of Eye Movement Re- 10.1207/s15516709cog1602_1
search, 9(7), 1-12. doi: 10.16910/jemr.9.7.1
Chandler, P., & Sweller, J. (1991). Cognitive load theory
Atkinson, R. C., & Shiffrin, R. M. (1968). Human and the format of instruction. Cognition and
memory: A proposed system and its control Instruction, 8(4), 293-332. doi:
processes. In K. W. Spence & J. T. Spence 10.1207/s1532690xci0804_2
(Eds.), The psychology of learning and
Chase, W. G., & Simon, H. A. (1973). The mind’s eye in
motivation: Advances in research and theory
chess. In W. G. Chase (Ed.), Visual information
(Vol. 2, pp. 89-192). New York: Academic
processing (pp. 215-281). New York: Academic
Press.
Press.
Baddeley, A. D. (2012). Working memory: Theories,
Cherubini, M., Nüssli, M. A., & Dillenbourg, P. (2010).
models, and controversies. Annual Review of
This is it! Indicating and looking in collabora-
Psychology, 63(1), 1-29. doi: 10.1146/annurev-
tive work at distance. Journal of Eye Movement
psych-120710-100422
Research, 3(5), 1-20. doi: 10.16910/jemr.3.5.3
Balslev, T., Jarodzka, H., Holmqvist, K., De Grave, W.
Collins, A., Brown, J. S., & Newman, S. E. (1989).
S., Muijtjens, A., Eika, B., . . . Scherpbier, A.
Cognitive apprenticeship: Teaching the craft of
(2012). Visual expertise in paediatric neurology.
reading, writing, and mathematics. In L. B.
European Journal of Paediatric Neurology,
Resnick (Ed.), Cognition and instruction: Issues
16(2), 161-166. doi: 10.1016/j.ejpn.2011.07.004
and gendas (pp. 453-494). Mahwah: Erlbaum.
Bandura, A. (1977). Social learning theory. Englewood
De Groot, A. D. (1946/2008). Thought and choice in
Cliffs: Prentice-Hall.
chess. The Hague: Mouton.
Bandura, A., Ross, S. A., & Ross, D. (1961).
Denzin, N. K. (2012). Triangulation 2.0. Journal of
Transmission of aggression through imitation of
Mixed Methods Research, 6(2), 80-88. doi:
aggressive models. Journal of Abnormal and
10.1177/1558689812437186
Social Psychology, 63(3), 575-582. doi:
10.1037/h0045925 Dorr, M., Vig, E., Gegenfurtner, K. R., Martinetz, T., &
Barth, E. (2008, October). Eye movement
Beck, M. R., Trenchard, M., van Lamsweerde, A.,
modelling and gaze guidance. Paper presented at
Goldstein, R. R., & Lohrenz, M. (2012,
the Fourth International Workshop on Human-
September). Searching in clutter: Visual atten-
Computer Conversation, Oxford, UK.
tion strategies of expert pilots. In Proceedings of
the Human Factors and Ergonomics Society an- Dwyer, F. M. (1976). Adapting media attributes for
nual meeting (Vol. 56, No. 1, pp. 1411-1415). effective learning. Educational Technology,
Santa Monica, USA: Sage. 16(8), 7-13.
Ericsson, K. A., Charness, N., Feltovich, P., & Hoffman,
R. R. (Eds.). (2006). The Cambridge handbook
of expertise and expert performance.
Cambridge: Cambridge University Press.

14
Journal of Eye Movement Research Jarodzka, H., Holmqvist, K., & Gruber, H. (2017).
10(1):3,1-18 Eye tracking in Educational Science:Theoretical frameworks and research agendas

Ericsson, K. A., & Kintsch, W. (1995). Long-term Jaarsma, T., Boshuizen, H. P. A., Jarodzka, H., Nap, M.,
working memory. Psychological Review, Verboon, P., & Van Merriënboer, J. J. G.
102(2), 211-245. doi: 10.1037/0033- (2016). Tracks to a medical diagnosis: expertise
295X.102.2.211 differences in visual problem solving. Applied
Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. Cognitive Psychology, 30(3), 314-322. doi:
(1993). The role of deliberate practice in the 10.1002/acp.3201
acquisition of expert performance. Jaarsma, T., Jarodzka, H., Nap, M., Van Merriёnboer, J.
Psychological Review, 100(3), 363-406. doi: J. G., & Boshuizen, H. P. A. (2014). Expertise
10.1037/0033-295X.100.3.363 differences under the microscope: processing
Ericsson, K. A., & Lehmann, A. C. (1996). Expert and histopathological slides. Medical Education,
exceptional performance: Evidence of maximal 48(3), 292-300. doi: 10.1111/medu.12385
adaption to task constraints. Annual Reviews in Jaarsma, T., Jarodzka, H., Nap, M., Van Merriënboer, J.
Psychology, 47(1), 273-305. doi: J. G., & Boshuizen, H. P. A. (2015). Expertise in
10.1146/annurev.psych.47.1.273 clinical pathology: Bridging the gap. Advances
Ericsson, K. A., & Simon, H. A. (1993). Protocol in Health Sciences Education, 20(4), 1089-1106.
analysis: Verbal reports as data. Cambridge: doi: 10.1007/s10459-015-9589-x
MIT Press. Jacob, R., & Karn, K. S. (2003). Eye tracking in human-
Ericsson, K. A., & Smith, J. (1991). Prospects and limits computer interaction and usability research:
in the empirical study of expertise. In K. A. Ready to deliver the promises. In J. Hyönä, R.
Ericsson & J. Smith (Eds.), Towards a general Radach & H. Deubel (Eds.), The mind's eye:
theory of expertise: Prospects and limits (pp. 1- Cognitive and applied aspects of eye movement
38). Cambridge, MA: Cambridge University research (pp. 573-605). Oxford: Elsevier.
Press. Jarodzka, H., Balslev, T., Holmqvist, K., Nyström, M.,
Freyhof, H., Gruber, H., & Ziegler, A. (1992). Expertise Scheiter, K., Gerjets, P., & Eika, B. (2012).
and hierarchical knowledge representation in Conveying clinical reasoning based on visual
chess. Psychological Research, 54(1), 32-37. observation via eye-movement modelling
doi: 10.1007/BF01359221 examples. Instructional Science, 40(5), 813-827.
Groner, R., & Siegenthaler, E. (2009). Improving the doi: 10.1007/s11251-012-9218-5
usability of eLearning tools: The IFeL Jarodzka, H., Boshuizen, H. P. A., & Kirschner, P. A.
Multifunctional Analysis and its application in (2012). Cognitive skills in medical diagnosis
distance teaching. Proceedings of the and intervention. In P. Lanzer (Ed.), Catheter-
ICDE/EADTU Conference in Maastricht, June based cardiovascular interventions; Knowledge-
2009. Retrieved from: based approach (pp. 69-86). Berlin, Germany:
http://www.ou.nl/Docs/Campagnes/ICDE2009/P Springer.
apers/Final_Paper_100Groner.pdf Jarodzka, H., Gouw, P., Van Meeuwen, L., & Brand-
Gruwel, S. (2015, August). Air traffic control:
Gruber, H. (1991). Wissensakquisition und Visual expertise in a dynamic problem solving
Gedächtnisleistung in Abhängigkeit vom task. Paper presented at the 18th European
Expertisegrad. Munich: University of Munich, Conference on Eye Movements, Vienna,
Chair of Education and Educational Psychology. Austria.
Hegarty, M. (1992). Mental animation: Inferring motion Jarodzka, H., Janssen, N., Kirschner, P. A., & Erkens, G.
from static displays of mechanical systems. (2015). Avoiding split attention in computer-
Journal of Experimental Psychology: Learning, based testing: Is neglecting additional
Memory, and Cognition, 18(5), 1084-1102. doi: information facilitative? British Journal of
10.1037//0278-7393.18.5.1084 Educational Technology, 46(4), 803-817. doi:
Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, 10.1111/bjet.12174
R., Jarodzka, H., & Van de Weijer, J. (2011). Jarodzka, H., Scheiter, K., Gerjets, P., & Van Gog, T.
Eye tracking: A comprehensive guide to methods (2010). In the eyes of the beholder: How experts
and measures. Oxford: Oxford University Press. and novices interpret dynamic stimuli. Learning
and Instruction, 20(2), 146-154. doi:
10.1016/j.learninstruc.2009.02.019

15
Journal of Eye Movement Research Jarodzka, H., Holmqvist, K., & Gruber, H. (2017).
10(1):3,1-18 Eye tracking in Educational Science:Theoretical frameworks and research agendas

Jarodzka, H., Van Gog, T., Dorr, M., Scheiter, K., & Lilienthal, O. (1889). Der Vogelflug als Grundlage der
Gerjets, P. (2013). Learning to see: Guiding Fliegekunst, Berlin, Germany.Retrieved from:
students’ attention via a model’s eye movements https://commons.wikimedia.org/wiki/File:Lilient
fosters learning. Learning and Instruction, 25, halFliegekunst.png
62-70. doi: 10.1016/j.learninstruc.2012.11.004 Lin, J. J., & Lin, S. S. (2014). Tracking eye movements
Jucks, R., Schulte-Löbbert, P., & Bromme, R. (2007). when solving geometry problems with handwrit-
Supporting experts’ written knowledge ing devices. Journal of Eye Movement Research,
communication through reflective prompts on 7(1), 1-15. doi: 10.16910/jemr.7.1.2
the use of specialist concepts. Journal of Lindow, S., Fuchs, H. M., Fürstenberg, A., Kleber, J.,
Psychology, 215(4), 237-247. doi: Schweppe, J., & Rummer, R. (2011). On the
10.1027/0044-3409.215.4.237 robustness of the modality effect: Attempting to
Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why replicate a basic finding. Zeitschrift für
minimal guidance during instruction does not Pädagogische Psychologie, 25(4), 231-243. doi:
work: An analysis of the failure of 10.1024/1010-0652/a000049
constructivist, discovery, problem-based, Litchfield, D., & Ball, L. J. (2011). Using another’s gaze
experiential, and inquiry-based teaching. as an explicit aid to insight problem solving. The
Educational Psychologist, 41(2), 75-86. doi: Quarterly Journal of Experimental Psychology,
10.1207/s15326985ep4102_1 64(3), 649-656. doi: 10.1037/a0020082
Kok, E., & Jarodzka, H. (2016). Before your very eyes: Lowe, R. K. (2003). Animation and learning: Selective
The value and limitations of eye tracking in processing of information in dynamic graphics.
medical education. Medical Education, 51, 114- Learning and Instruction, 13(2), 157-176. doi:
122. doi: 10.1111/medu.13066 10.1016/S0959-4752(02)00018-X
Kok, E., Jarodzka, H., De Bruin, A. B. H., BinAmir, H., Mason, L., Tornatora, M. C., & Pluchino, P. (2013). Do
Robben, S. G. F., & Van Merriënboer, J. J. G. fourth graders integrate text and picture in
(2015). Systematic viewing in radiology: Seeing processing and learning from an illustrated
more, missing less? Advances in Health Science science text? Evidence from eye-movement
Education, 21(1), 189-205. doi: patterns. Computers & Education, 60(1), 95-
10.1007/s10459-015-9624-y 109. doi: 10.1016/j.compedu.2012.07.011
Krejtz, K., Duchowski, A. T., Krejtz, I., Kopacz, A., & Mayer, R. E. (2009). Multimedia learning. Cambridge:
Chrząstowski-Wachtel, P. (2016). Gaze transi- Cambridge University Press.
tions when learning with multimedia. Journal of Mayer, R. E., & Moreno, R. (2003). Nine ways to reduce
Eye Movement Research, 9(1), 1-17. doi: cognitive load in multimedia learning.
10.16910/jemr.7.1.2 Educational Psychologist, 38(1), 43-52. doi:
Lachner, A., Jarodzka, H., & Nückles, M. (2016). What 10.1207/S15326985EP3801_6
makes an expert teacher? Investigating teachers’ McCabe, D., & Castel, A. (2008). Seeing is believing:
professional vision and discourse abilities. The effect of brain images on judgments of
Instructional Science, 44(3), 197-203. doi: scientific reasoning. Cognition, 107(1), 343-352.
10.1007/s11251-016-9376-y doi: 10.1016/j.cognition.2007.07.017
Leff, D. R., James, D. R., Orihuela-Espina, F., Kwok, K.- McNamara, A., Booth, T., Sridharan, S., Caffey, S.,
W., Sun, L. W., Mylonas, G., . . . Yang, G.-Z. Grimm, C., & Bailey, R. (2012, August).
(2015). The impact of expert visual guidance on Directing gaze in narrative art. Paper presented
trainee visual search strategy, visual attention at the Proceedings of the ACM Symposium on
and motor skills. Frontiers in Human Applied Perception, Los Angeles, USA.
Neuroscience, 9. doi:
Meltzoff, A. N., & Moore, M. K. (1977). Imitation of
10.3389/fnhum.2015.00526
facial and manual gestures by human neonates.
Lehmann, A. C., & Gruber, H. (2006). Music. In K. A. Science, 198(4312), 75-78. doi:
Ericsson, N. Charness, P. Feltovich, & R. R. 10.1126/science.198.4312.75
Hoffman (Eds.), The Cambridge handbook of
Miller, G. A. (1956). The magical number seven, plus or
expertise and expert performance (pp. 457-470).
minus two: Some limits on our capacity for
Cambridge: Cambridge University Press.
processing information. Psychological Review,
63(2), 81-97. doi: 10.1037/h0043158

16
Journal of Eye Movement Research Jarodzka, H., Holmqvist, K., & Gruber, H. (2017).
10(1):3,1-18 Eye tracking in Educational Science:Theoretical frameworks and research agendas

Nyström, M., & Holmqvist, K. (2008). Semantic override Richardson, D. C., & Dale, R. (2005). Looking to
of low-level features in image viewing - both understand: The coupling between speakers' and
initially and overall. Journal of Eye Movement listeners’ eye movements and its relationship to
Research, 2, 1-11. doi: 10.1007/s11251-016- discourse comprehension. Cognitive Science,
9397-6 29(6), 1045-1060. doi:
Ögren, M., Nyström, M., & Jarodzka, H. (2016). There's 10.1207/s15516709cog0000_29
more to the multimedia effect than meets the Richardson, D. C., Street, C. N. H., Tan, J. Y. M.,
eye: Is seeing pictures believing? Instructional Kirkham, N. Z., Hoover, M., & Ghane-
Science, 1-25. doi: 10.1007/s11251-016-9397-6 Cavanaugh, A. (2012). Joint perception: Gaze
Oliva, M., Niehorster, D., Jarodzka, H., & Holmqvist, K. and social context. Frontiers in Human
(in press). Influence of coactors on saccadic and Neuroscience, 6, 194. doi:
manual responses. i-Percetion. 10.3389/fnhum.2012.00194
Paivio, A. (1991). Dual coding theory: Retrospect and Rummer, R., Schweppe, J., Fürstenberg, A., Scheiter, K.,
current status. Canadian Journal of Psychology, & Zindler, A. (2011). The perceptual basis of
45(3), 255-287. doi: 10.1037/h0084295 the modality effect in multimedia learning.
Penttinen, M., & Huovinen, E. (2011). The early Journal of Experimental Psychology: Applied,
development of sight-reading skills in 17(2), 159-173. doi: 10.1037/a0023588
adulthood: A study of eye movements. Journal Schank, R. C., & Abelson, R. P. (2013). Scripts, plans,
of Research in Music Education, 59(2), 196-220. goals, and understanding: An inquiry into
doi: 10.1177/0022429411405339 human knowledge structures. Hillsdale: Law-
Penttinen, M., Huovinen, E., & Ylitalo, A.-K. (2013). rence Erlbaum Associates.
Silent music reading: Amateur musicians’ visual Schmidt, H. G., & Boshuizen, H. P. A. (1992).
processing and descriptive skill. Musicae Encapsulation of biomedical knowledge. In D.
Scientiae, 17(2), 198-216. doi: A. Evans & V. L. Patel (Eds.), Advanced models
10.1177/1029864912474288 of cognition for medical training and practice
Penttinen, M., Huovinen, E., & Ylitalo, A.-K. (2015). (pp. 265-282). New York: Springer Verlag.
Reading ahead: Adult music students’ eye Schnotz, W., & Lowe, R. K. (2008). A unified view of
movements in temporally controlled learning from animated and static graphics. In R.
performances of a children’s song. International K. Lowe & W. Schnotz (Eds.), Learning with
Journal of Music Education, 33(1), 36-50. doi: animation: Research and design implications
10.1177/0255761413515813 (pp. 304-356). New York: Cambridge
Rayner, K. (1998). Eye movements in reading and University Press.
information processing: 20 years of research. Skuballa, I. T., Fortunski, C., & Renkl, A. (2015). An eye
Psychological Bulletin, 124(3), 372-422. doi: movement pre-training fosters the
10.1037/0033-2909.124.3.372 comprehension of processes and functions in
Rayner, K. (2009). Eye movements and attention in technical systems. Frontiers in Psychology, 6.
reading, scene perception, and visual search. The doi: 10.3389/fpsyg.2015.00598
Quarterly Journal of Experimental Psychology, Skuballa, I. T., Schwonke, R., & Renkl, A. (2012).
62(8), 1457-1506. doi: Learning from narrated animations with
10.1080/17470210902816461 different support procedures: Working memory
Reingold, E. M., Charness, N., Pomplun, M., & Stampe, capacity matters. Applied Cognitive Psychology,
D. M. (2001). Visual span in expert chess 26(6), 840-847. doi: 10.1002/acp.2884
players: Evidence from eye movements. Sridharan, S., Bailey, R., McNamara, A., & Grimm, C.
Psychological Science, 12(1), 48-55. doi: (2012, March). Subtle gaze manipulation for
10.1111/1467-9280.00309 improved mammography training. Proceedings
Reingold, E. M., & Sheridan, H. (2011). Eye movements of the Symposium on Eye Tracking Research
and visual expertise in chess and medicine. In S. and Applications, Santa Barbara, USA.
P. Liversedge, I. D. Gilchrist, & S. Everling Stofer, K., & Che, X. (2014). Comparing experts and
(Eds.), Oxford handbook of eye movements (pp. novices on scaffolded data visualizations using
523-550). Oxford: Oxford University Press. eye-tracking. Journal of Eye Movement Re-
search, 7(5), 1-15. doi: 10.16910/jemr.7.5.2

17
Journal of Eye Movement Research Jarodzka, H., Holmqvist, K., & Gruber, H. (2017).
10(1):3,1-18 Eye tracking in Educational Science:Theoretical frameworks and research agendas

Sweller, J., Van Merriënboer, J. J. G., & Paas, F. (1998). Van Gog, T., Paas, F., Van Merriënboer, J., & Witte, P.
Cognitive architecture and instructional design. (2005). Uncovering the problem-solving
Educational Psychological Review, 10(3), 251- process: Cued retrospective reporting versus
296. doi: 10.1023/A:1022193728205 concurrent and retrospective reporting. Journal
Thurmond, V. A. (2001). The point of triangulation. of Experimental Psychology: Applied, 11(4),
Journal of Nursing Scholarship, 33(3), 253-258. 237-244. doi: 10.1037/1076-898X.11.4.237
doi: 10.1111/j.1547-5069.2001.00253.x Van Lehn, K. (1996). Cognitive skill acquisition. Annual
Van der Gijp, A., Ravesloot, C., Jarodzka, H., Van der Review of Psychology, 47, 513-539. doi:
Schaaf, M., Van der Schaaf, I., Van Schaik, J., 10.1146/annurev.psych.47.1.513
& Ten Cate, T. J. (2016). How visual search Van Marlen, T., Van Wermeskerken, M., Jarodzka, H., &
relates to visual diagnostic performance: a Van Gog, T. (in press). Showing a model’s eye
narrative systematic review of eye-tracking movements in examples does not improve
research in radiology. Advances in Health performance of problem-solving tasks.
Sciences Education, 1-23. doi: 10.1007/s10459- Computers in Human Behavior. doi:
016-9698-1 10.1016/j.chb.2016.08.041
Van Gog, T., & Jarodzka, H. (2013). Eye tracking as a Van Meeuwen, L. W., Jarodzka, H., Brand-Gruwel, S.,
tool to study and enhance (meta-)cognitive Kirschner, P. A., De Bock, J. J. P. R., & Van
processes in computer-based learning Merriënboer, J. J. G. (2014). Identification of
environments. In R. Azevedo & V. Aleven effective visual problem solving strategies in a
(Eds.), International handbook of metacognition complex visual domain. Learning and
and learning technologies (pp. 143-156). New Instruction, 32, 10-21. doi:
York: Springer Science+Business Media. 10.1016/j.learninstruc.2014.01.004
Van Gog, T., Jarodzka, H., Scheiter, K., Gerjets, P., & Van Merriënboer, J. J. G., & Kirschner, P. A. (2007). Ten
Paas, F. (2009). Attention guidance during steps to complex learning. Hillsdale: Erlbaum.
example study via the model’s eye movements. Wolff, C. E., Jarodzka, H., Van den Bogert, N., &
Computers in Human Behavior, 25(3), 785-791. Boshuizen, H. P. A. (2016). Teacher vision:
doi: 10.1016/j.chb.2009.02.007 Comparing expert and novice teachers’
Van Gog, T., Paas, F., & Van Merriënboer, J. J. G. perception of problematic classroom
(2004). Process-oriented worked examples: management scenes. Instructional Science,
Improving transfer performance through 44(3), 243-265. doi: 10.1007/s11251-016-9367-
enhanced understanding. Instructional Science, z
32(1/2), 83-98. doi: Yarbus, A. L. (1967). Eye movements and vision. New
10.1023/B:TRUC.0000021810.70784.b0 York: Plenum.

18