English Teaching, Vol. 76, No. 4, Winter 2021, pp.

33-52
DOI: https://doi.org/10.15858/engtea.76.4.202112.33
http://journal.kate.or.kr

E-Book, Audio-Book, or E-Audio-Book: The Effects of

Multiple Modalities on EFL Comprehension

Na-Young Kim*

Kim, Na-Young. (2021). E-book, Audio-book, or E-audio-book: The effects of

multiple modalities on EFL comprehension. English Teaching, 76(4), 33-52.
This study explores effects of e-book and audio-book on EFL comprehension. It was
conducted at a university in Korea with 75 college students. The whole experiment lasted
for five weeks in 2020. To determine effectiveness of multiple modalities, participants
were assigned to three different experimental groups: e-book, audio-book, and e-audio-
book (a combination of e-book and audio-book) groups. While the e-book group was
provided with passages in a text format, the audio group received the same passages in
an audio format. The e-audio-book group was given both text and audio formats. At the
beginning and end of the treatment, pretest and posttest were administered. As expected,
the e-audio-book group benefited more than the other two groups in listening
comprehension. However, they showed better performance in reading comprehension
than the audio-book group only. Additionally, the e-book group performed better in
reading comprehension compared to the audio-book group. Based on these findings,
pedagogical implications are made.

Key words: e-book, audio-book, multimedia learning, modality effect, EFL

comprehension

This paper was supported by the Sehan University Research Fund (2021-145).
*Author: Na-Young Kim, Professor, Department of General Education, Sehan University; 33, Sehandae-gil,
Sinpyeong-myeon, Dangjin-si, Chungcheongnam-do, Korea; E-mail: [email protected]
Received 26 August 2021; Reviewed 21 October 2021; Accepted 21 December 2021

© 2021 The Korea Association of Teachers of English (KATE)

This is an open access article distributed under the terms of the Creative Commons Attribution License 4.0, which
permits anyone to copy, redistribute, remix, transmit and adapt the work, provided the original work and source is
appropriately cited.
34 Na-Young Kim

1. INTRODUCTION

Over the past decades, technological advances have changed the modern world. The
development of technology and its proliferation has made teaching and learning more
meaningful, especially in the time of the COVID-19 pandemic. The demands and challenges
of the new normal education have changed the educational landscape by urging teachers to
look for electronic ways to do their jobs previously done by lectures and textbooks.
Establishing expertise in the use of technology in teaching and learning has become a must
particularly in this COVID-19 pandemic (Marpa, 2021).
As technologies have improved teaching and learning with greater availability of
multimedia materials (Mayer & Moreno, 2002), teachers can now use instructional materials
in various combinations of text, image, audio, and even video (Mayer, 2001). In particular,
the rapid advances in multimedia technology have allowed students to read books using
multiple modalities. As information can be presented in many different ways with multiple
modalities, students can read e-books, listen to audio books, or do both at the same time from
their smart phones, tablet PCs, and computers (Rogowsky, Calhoun, & Tallal, 2016). By
exploiting visual and auditory modalities, the students can construct a reading context,
enhancing their interest and attention (Tang & Posner, 2009). Consequently, this sensory
stimulation helps them learn independently and effectively (Holt, 2005; Lee & Joo, 2017).
The effectiveness of multiple modalities is one of the common beliefs surrounding
multimedia instruction. According to Mayer (2001), the information processed through more
than two sensory channels exceeds the processing capacity of one single channel by reducing
the cognitive load. By using multiple sensory modalities, limited working memory can also
be expanded. Therefore, the use of two channels – both visual and auditory channels – is
considered better than that of one channel – either visual or auditory channel – in multimedia
instruction. In this light, it has been said that reading and listening to books at the same time
has more positive effects on learning compared to doing only one of them.
However, according to Kalyuga (2000), adding audio (spoken words) to text (written
words) can have negative effects on learning. Although two sensory modalities (auditory
and visual channels) are used, they are both verbally presented information and are solely
processed using the verbal working memory system. This means that the verbal system is
overloaded while the nonverbal left unused. In particular, it can be worse if redundant verbal
information is presented concurrently in different modes. Kalyuga, Chandler, and Sweller
(1999) found that audio simultaneously provided with the identical text imposes additional
and unnecessary cognitive load and interferes with learning. That is, the use of the same
verbal information in both auditory and visual channels increases more cognitive load
because connecting corresponding information consumes extra cognitive resources.
According to Kalyuga (2000), human cognitive capacity is limited, and only a limited

E-Book, Audio-Book, or E-Audio-Book: The Effects of Multiple Modalities on EFL Comprehension

English Teaching, Vol. 76, No. 4, Winter 2021, pp. 33-52 35

amount of information can be processed at any one time. As simultaneously processing

several interdependent sources of information causes a heavy cognitive load, it has a negative
learning effect. Thus, the redundant use of multiple modalities may not be effective in
learning (Kalyuga, 2000).
Regarding the effectiveness of multiple modalities, many empirical studies have been
carried out. In particular, there have been research studies on the effects of using dual
modality mode (visual and auditory) on comprehension. However, previous research
findings regarding this issue have shown mixed results. While some scholars found an
advantage for dual modality presentation (Adesope & Nesbit, 2012; Chang, 2009;
Lewandowski & Kobus, 1993), others revealed its disadvantage (Diao & Sweller, 2007;
Kalyuga et al., 2004). Likewise, regarding the studies on e-book and audio-book, conflicting
findings have been observed. Whereas many scholars (Kartal & Simsek, 2017; Larson, 2015;
Liu, Cao, & Wu, 2019; Rahman & Hajar, 2020) witnessed the superior effects of reading
and listening to books concurrently, Rogowsky, Calhoun, and Tallal (2016) found that
students perform equally in comprehension tests regardless of whether they read e-book,
listened to audio-book, or both read and listened simultaneously. Previous researchers have
also acknowledged these contrasting findings in the literature and recognized the need for
resolution (Adesope & Nesbit, 2012; Moreno & Mayer, 2002).
The unexpected arrival of the COVID-19 pandemic has required a substantial change in
education systems all over the world. It has resulted in the closing of schools including
universities and colleges and restructured teaching and learning processes. Different
modalities using multimedia technology have been discussed in the wake of the ongoing
pandemic. However, the schools are not so certain yet about the best modality mode to be
employed in order to make learning accessible to students at all levels nationwide (Marpa,
2021). Rogowsky, Calhoun, and Tallal (2016) noticed that there is a lack of empirical
research that directly evaluates the effectiveness of different modalities. Given that much
research on multimedia learning is still needed especially in EFL settings (Soruc, 2015), it
is necessary to examine the modality effects on EFL learning in Korea.
Taking all this into consideration, this study aims to examine whether the combination of
text (e-book) and audio (audio-book) can also play a positive role in EFL reading and
listening comprehension. Given that fairy tales offer an authentic model of English language
(Kochiyama, 2013), this modality study was carried out using fairy tales in the Korean EFL
context where the authentic language input is rare. Research questions are postulated as
follows:

1. Can the simultaneous use of e-book and audio-book more promote EFL
listening comprehension than the use of only one of them?

© 2021 The Korea Association of Teachers of English (KATE)

36 Na-Young Kim

2. Are there any superior effects of the simultaneous use of e-book and audio-
book on EFL reading comprehension over the use of only one of them?

2. REVIEW OF THE LITERATURE

2.1. Effects of Modality Modes on Learning

Educational technologies have revolutionized the learning environment with new

techniques with a wide range of tool. In particular, as a commonplace instructional tool,
multimedia has been increasingly used for a variety of educational purposes. According to
Najjar (1995), one of the reasons for this increase is that multimedia tutorials allow
instructional designers to use a wide range of media to present their teaching materials. That
is, with the advancement of multimedia technologies, instructors can easily use multiple
modalities for their teaching (Mayer, 2001). For example, teachers can now provide their
students with visual information (e.g., on-screen text), giving related or supporting
information through audio channels.
According to Kalyuga (2000), using both visual and auditory modalities is highly
beneficial as the use of both the channels extends the capacity of working memory to handle
the information. In particular, when presented concurrently in both a reading and listening
format, the information elicits more elaborate memory traces and facilitates better retrieval
(Baddeley, 1992). Moreno and Mayer (2002) also insisted that “response times and memory
are facilitated when redundant signal information is presented simultaneously in two sensory
channels rather than in one” (p. 156). From this perspective, empirical research comparing
single and dual modality modes has been carried out to confirm the superior effects of dual
mode on learning. However, its findings have produced contradictory results.
Adesope and Nesbit (2012) conducted a meta-analysis to compare the effects of spoken-
only, written-only, and spoken–written presentations on learning and found the superior
effects of spoken–written (dual) over spoken-only (single) presentations. The authors found
that the dual mode is more beneficial than the single mode in education. Likewise, in
vocabulary learning, Nasser and McEwen (1976) confirmed that a combination of text-audio
presentation is better than either text-only or audio-only presentation. Similarly,
Lewandowski and Kobus (1993) found that students presented with the same words
simultaneously in auditory and visual modes recall more vocabulary than those presented
with the words in a single mode. Regarding comprehension, Chang (2009) also proved the
superior effects of the dual modality modes (both reading and listening) compared to the
single mode (listening only).
On the contrary, Diao and Sweller (2007) also found significant gains in comprehension

E-Book, Audio-Book, or E-Audio-Book: The Effects of Multiple Modalities on EFL Comprehension

English Teaching, Vol. 76, No. 4, Winter 2021, pp. 33-52 37

in the single modality mode (reading-only) rather than in the dual mode (both reading and
listening). This is an example of redundancy effect. According to them, learning is inhibited
as written texts and spoken words contain the same information while presented
simultaneously. Similarly, Kalyuga et al. (2004) reported that nonconcurrent spoken–written
presentations helped students recall more information than do concurrent spoken–written
ones. They also showed that simultaneous spoken–written presentation caused poorer recall
than did spoken-only presentation. In their study, the redundant information presentations in
different modalities (i.e., showing the same information in both spoken and written form)
did not bring about the positive effects on learning. This supports cognitive load theory,
suggesting that concurrent input from dual modes increases cognitive load and are
detrimental to learning compared to the presentation of information in a single modality
(Plass, Moreno, & Brunken, 2010).
Meanwhile, others have found no differences between single and dual modality modes.
After conducting a meta-analysis to compare the effects of spoken-only, written-only, and
spoken–written presentations on learning, Adesope and Nesbit (2012) reported that there is
no difference between spoken–written (dual mode) and written-only (single mode)
presentations. Koroghlanian and Sullivan (2000) also revealed that concurrent spoken–
written presentations do not produce learning benefits in comparison to written-only
presentations.
Considering the mixed results in modality research, it is difficult to draw conclusions
about its effects on learning. According to Atkinson, Clark, Harrison, Koenig, and Ramirez
(2007), redundant spoken-written presentations are advantageous under some conditions but
detrimental under other conditions. These contradictory findings may be attributed to
different research methods, learner characteristics, outcome variables, and features of
modality presentations (Adesope & Nesbit, 2012). Rogowsky, Calhoun, and Tallal (2016)
also claimed that the findings from these previous modality studies cannot be compared
directly considering the differences across studies regarding population characteristics,
instructional conditions, and the nature of the readings.
Technologies have improved teaching and learning processes with greater availability of
multiple modalities (Mayer & Moreno, 2002), and the unexpected COVID-19 pandemic has
required teachers to establish expertise in the use of multimedia technology. In this regard,
different modalities have been discussed to make teaching and learning more effective in the
wake of the ongoing pandemic. However, it is difficult to determine the best mode to be
employed (Marpa, 2021). There is a lack of empirical studies directly evaluating the
effectiveness of different modalities (Rogowsky, Calhoun, & Tallal, 2016). Furthermore,
research on multimedia learning is still rare especially in EFL settings (Soruc, 2015).
Therefore, it is necessary to investigate the modality effects on EFL learning.

© 2021 The Korea Association of Teachers of English (KATE)

38 Na-Young Kim

2.2. E-Book and Audio-Book

Multimedia technologies have created the learning environment that is beneficial to

students, matching their individual learning preferences (Karbalaei & Zare, 2019). By
providing information in a non-linear way with different modalities (Mayer, 2001; 2005),
multimedia has also become important in foreign language teaching and learning. In
particular, the rapid development in multimedia technology has allowed foreign language
students to read books in many different ways. Frequently used as teaching materials, books
have been published in different media forms such as conventional paper books, e-books,
and audio-books (Wei & Ma, 2020).
We have five senses – sight, hearing, smell, taste, and touch – and we use them to receive
external information. Now days, students read e-books, listen to audio-books, or do both at
the same time using multiple modalities (Rogowsky, Calhoun, & Tallal, 2016). They read
books using these diverse sensory stimuli. In particular, various types of sensory stimulation
including visual and auditory information help students to construct a reading context,
allowing them to learn independently (Holt, 2005). Tang and Posner (2009) indicated that
the sensory information can also enhance the students’ interest and attention. Consequently,
this makes the students’ learning more effectively (Lee & Joo, 2017).
An electronic book, called e-book, is a digital version of a printed book. Rao (2003)
defined e-books as “text in digital form, or digital reading materials, or a book in a computer
file format, or an electronic file of words and images displayed on a desktop, note‐book
computer, or portable device, or formatted for display on dedicated e‐book readers” (pp. 86-
87). In terms of audio book, it is a simple audio track or an audio representation of a written
book (Egidi & Furini, 2006). It has been attracting great interest and booming audibly in the
digital age (Liu, Cao, & Wu, 2019). By comparing the two modalities – auditory modality
(spoken words) and visual modality (written texts) – previous scholars have confirmed the
superior effects of audio-book over e-book (Gunduz, 2006; Kartal & Simsek, 2017; Liu, Cao,
& Wu, 2019; Montgomery, 2009; Turker, 2010).
For example, Montgomery (2009) revealed that English language learners’ reading and
academic performance increase when using audio books in their classroom. Gunduz (2006)
also highlighted that EFL students can develop their reading comprehension skills by
listening to texts. Furthermore, Kartal and Simsek (2017) showed that auditory materials
including audio books can enhance English as a foreign language listening skills. Liu, Cao,
and Wu (2019) discovered that listening to audio-books through the audio channel is more
effective than reading e-books through the visual channel when it comes to reading
comprehension.
Interestingly however, Moyer (2011) investigated whether there were any differences in
comprehension across printed book, e-book, and audio-book modalities, but found no

E-Book, Audio-Book, or E-Audio-Book: The Effects of Multiple Modalities on EFL Comprehension

English Teaching, Vol. 76, No. 4, Winter 2021, pp. 33-52 39

statistically significant differences among them. That is, comprehension levels were the
same regardless of the mode – whether it was either auditory channel (audio) or visual
channel (text). The author asserted that the lines between audio-books and e-books are
blurred as audio books come digitally while e-books provide text-to-speech functions.
According to Moyer (2011), this might lead to insignificant differences among them.
Studies comparing the effectiveness of single (either audio-book or e-book) and dual (both
audio-book and e-book) modality modes have also yielded inconclusive results. For instance,
Rogowsky, Calhoun, and Tallal (2016) witnessed no statistically significant differences in
comprehension between the dual modality of input (reading and listening to texts) and the
single modality of input (either reading or listening to texts). The participants in their study
performed equally in the comprehension test regardless of whether they read e-book, listened
to audio-book, or both read and listened simultaneously. Therefore, the authors suggested
that the dual mode might not be beneficial in learning.
However, with the concurrent use of audio-book and e-book, students can have a
multisensory approach to read books (Kartal & Simsek, 2017). Woodall (2010) also claimed
that listening while reading provides students with a multisensory reading experience, and
this especially eliminates frustrations for those who have difficulties with text-only material.
Likewise, Rahman, and Hajar (2020) insisted that reading and listening to text
simultaneously is beneficial especially to students who struggle to focus on reading activities.
According to them, by reading and listening at the same time, the students gained a deeper
understanding of the language (e.g., syntax, pronunciation, emphasis, etc.). Larson (2015)
added that using both e-book and audio-book can promote students' understanding,
developing their literacy experience.
Regarding reading comprehension skills, Liu, Cao, and Wu (2019) examined the effects
of audio-book provided with written texts and reported that reading and listening to books
at the same time was better than either reading or listening only. Likewise, Rahman and
Hajar (2020) discovered that EFL students who listened to and read the books
simultaneously performed better on the reading comprehension tests than those who read the
books only. According to them, audio-books provided with e-books are effective in
enhancing reading skills. Turker (2010) also proved the effects of reading while listening on
EFL reading comprehension, and particularly, found that its effects varied based on students’
proficiency levels. The findings confirmed its greater effects on students at intermediate
level than those at elementary level. Regarding listening comprehension, Kartal and Simsek
(2017) also revealed the concurrent use of e-book and audio-book had a positive impact on
English as a foreign language listening comprehension skills.
Previous EFL studies have revealed that books presented through multimedia technology
significantly improve students’ comprehension skills (Gunduz, 2006; Kartal & Simsek, 2017;
Larson, 2015; Liu, Cao, & Wu, 2019; Montgomery, 2009; Rahman & Hajar, 2020; Turker,

© 2021 The Korea Association of Teachers of English (KATE)

40 Na-Young Kim

2010; Woodall, 2010). However, a review of the previous research yields inconclusive
results. Addressing this gap, the primary objective of this study is to investigate the effects
of using audio-book and e-book on EFL comprehension. Particularly, this study focuses on
whether the concurrent use of audio-book and e-book also plays a positive role in listening
and reading comprehension.

3. METHODOLOGY

3.1. Participants

Participants in the study were comprised of 75 Korean EFL learners who enrolled in an
Intensive English course at a university in Korea. They were 43 male and 32 female
undergraduate students. Their age ranged from 20 to 27, with diverse majors including
aeronautics, business administration, practical arts, and so on. The participants were all
Korean and had spent more than 10 years learning the English language in the classroom in
Korea.
For the current study, participants were randomly assigned into three experimental groups.
There were 26 students in the e-book group. The audio-book group consisted of 24
participants while the combination of e- and audio-book group (the e-audio-book group) was
made up of 25. In order to determine whether the participants in the three groups were
statistically similar in English comprehension skills at the beginning, all groups were pre-
tested using a simulated TOEIC test.

TABLE 1
The Homogeneity Test
E-Book Audio-Book E-Audio-Book
(n = 26) (n = 24) (n = 25) F p
M SD M SD M SD
Listening 35.00 18.17 38.13 17.99 37.60 20.52 0.20 .82
Reading 42.69 24.95 37.29 24.49 45.20 27.29 0.61 .55

As can be seen from Table 1 above, no group differences were found in both listening and
reading sections at significance level .05. It was assumed that samples in the present study
were homogeneous before treatment. With the pre-test results, the level of English
proficiency among participants was also determined. The participants’ average pretest score
was 78.67 out of 200, which can be considered as pre-intermediate level of language
proficiency.

E-Book, Audio-Book, or E-Audio-Book: The Effects of Multiple Modalities on EFL Comprehension

English Teaching, Vol. 76, No. 4, Winter 2021, pp. 33-52 41

3.2. Data Collection

3.2.1. E-book, audio-book, and a combination of e- and audio-book

The purpose of this study was to explore the effects of using e-book and audio-book on
EFL comprehension. Divided into the three experimental conditions, e-book, audio-book,
and a combination of e- and audio-book (e-audio-book) groups, all participants received the
same instruction provided by the same teacher. They were provided with the same input (five
selected fairy tales) for extra in-class activities. However, the fairy tales were given in three
different ways depending on the participants’ experimental conditions.
For the experiment, five fairy tales were selected based on the previous research (Massi
& Adriana, 2001) and given to the students. The tales used for this study included Beauty
and the Beast, Cinderella, Pinocchio, Snow White, and The Ugly Duckling. According to
Lee (2003), fairy tales are appropriate EFL materials. They have commonly been used as
English reading materials in class (Lazar, 1993) and playing a role as a great vehicle for
enhancing EFL development (Massi & Adriana, 2001). The fairy tales offer an authentic
model of the students’ target language (Kochiyama, 2013) while motivating them to learn
the language (Kosal, 2008). They also provide language that is perfectly suited for EFL
students as the tales have narrative structure and key elements of the story including plot,
setting, characters, the organization of events, and the overall message (Massi & Adriana,
2001).
Excluding the time for pre- and post-tests, the intervention lasted for five weeks. During
the experimental period, all participants read, listened to, or read and listened to the selected
fairy tales. The participants in the e-book group were given the fairy tales in e-text format.
They read e-books using their smart phones in the classroom. In terms of the audio-book
group, the participants listened to the same five fairy tales in audio format. They listened to
audio-books with their own earphones and smart phones. The e-audio-book group listened
to audio recordings of the fairy tale stories through the earphones while at the same time
reading the identically written e-text on their smart phone screens. The whole experiment
was carried out in class to control the students’ disruptive behavior, help them in
participating in fairytale activities, etc.
The audio recordings of the five fairy tales lasted for 4 minutes on average. The average
number of words in the tales was 302. That is, the texts were averagely read at a speed of 75
words per minute. The three groups were given the same length of time to read, listen to, or
read and listen to the fairy tale stories, regardless of their experimental conditions.

© 2021 The Korea Association of Teachers of English (KATE)

42 Na-Young Kim

3.2.2. Comprehension pre- and post-tests

This study investigates the effects of using e-book and audio-book on EFL listening and
reading comprehension. Considering that ETS TOEIC tests consist of listening and reading
sections, the tests were adopted as pre and posttests. All participants took the TOEIC tests
both before and after the treatment. The tests were paper-and-pencil and multiple-choice
assessments. The listening and reading sections were timed sections with 100 questions in
each section. The test items were adopted from the two practice tests provided by the official
test preparation book, Tactics for TOEIC Listening and Reading Practice Test (Trew, 2008).
Based on the original test-taking time, 45 minutes were given for the listening section
whilst 75 minutes were for the reading section. The participants were asked to finish the tests
within the time allowed. Test scores were given by the number of correct answers. A
question with a single correct answer was worth a single point. Each answer was scored
either 1 or 0, and one question could not have more than one correct answer. The total scores
ranged from 0 to 200 with 200 questions in total.

3.3. Research Procedures

The present study aimed to examine the effects of e-book and audio-book on EFL students’
comprehension skills. The whole experiment was administered during the 2020 academic
year. Excluding the time for pre- and post-tests, the intervention lasted only for five weeks.
Due to the outbreak of COVID-19, the experiment was carried out for a relatively short
period of time. In Korea, social distancing was enforced, and students were not allowed to
attend their face-to-face class. Schools and universities opened, closed, and reopened their
school campus during the pandemic (Kim, Cha, & Kim, 2021).
A total of 75 Korean EFL learners were recruited for this study. They were college
students taking Intensive English course at a university in Korea. For the study, the
participants were assigned to three experimental groups: the e-book group, the audio-book
group, and the e-audio-book group. The TOEIC-based pre-tests were completed by all
groups. With the pre-test results, a homogeneity test was administered. There was no
statistically significant difference among the three groups in both listening and reading pre-
test scores (p > .05). It was found that the participants were homogeneous before treatment.
In order to compare the effectiveness of e-book and audio-book, the three groups were
given the same input – five fairy tales based on the previous research (Massi & Adriana,
2001) – in different ways. All participants in the three groups were asked to read, listen to,
or read and listen to the fairy tale stories depending on their experimental conditions: e-book,
audio-book, or e-audio-book. The treatment took five class sessions of two hours during five
weeks.

E-Book, Audio-Book, or E-Audio-Book: The Effects of Multiple Modalities on EFL Comprehension

English Teaching, Vol. 76, No. 4, Winter 2021, pp. 33-52 43

In the e-book group, the participants were provided with the reading passages in e-text
format. Regarding the audio-book group, the participants were given the audio recordings of
the same passages. In the e-audio-book group, the participants read and listened to the
passage simultaneously. All participants used their smart phones to read, listened to, or read
and listened to fairy tale stories. In particular, the participants in the audio-book and e-audio-
book group used earphones to listen to the tales in audio format. At the end of the treatment,
the TOEIC-based post-tests were administered to all groups. Given that most Korean college
students are requested to submit their TOEIC test scores to prove that they have adequate
English language skills (Kim, 2019), the TOEIC listening and reading tests were employed
as pre- and post-comprehension tests.

3.4. Data Analysis

Data obtained from this study were computed and analyzed with SPSS version 21.
Descriptive statistics were first administered for all analyses. After that, a one-way
multivariate analysis of variance (MANOVA) was conducted to test group differences in
mean scores for both pre and posttests. In order to verify which mean scores significantly
differ from which others, Scheffe post hoc test was also performed. The significance level
was set at .05.

4. RESULTS

The current research study focuses on the effects of simultaneous use of e-book and audio-
book on English comprehension skills. In particular, this study compares the effectiveness
of e-book, audio-book, and e-audio-book in EFL listening and reading. For the study, the
three experimental groups were given the same input, the five selected fairy tales. However,
the tales were provided in three different ways depending on the participants’ experimental
conditions.
In order to confirm the possible differences among the experimental groups before
treatment, a one-way MANOVA was first carried out for the pretest scores. The findings of
this homogeneity test showed no group differences in comprehension test scores (p > .05),
suggesting that all participants were homogeneous before they were treated (See Table 1 in
the previous section). Thus, another one-way MANOVA was conducted for the posttest
scores to determine whether the mean scores on the posttest vary and differ for experimental
conditions. Levene’s tests were also done to investigate whether the variance between the
groups were equal. The test results indicated equal variances for listening comprehension
(F = 2.29, p = .11) and reading comprehension (F = 1.25, p = .29), showing the homogeneity

© 2021 The Korea Association of Teachers of English (KATE)

44 Na-Young Kim

of variance.

TABLE 2
Group Differences in EFL Comprehension
E-Book Audio-Book E-Audio-Book
(n = 26) (n = 24) (n = 25) F p
M SD M SD M SD
Listening 41.54 16.84 39.79 19.42 65.40 25.57 11.68 .00*
Reading 52.88 22.28 35.83 18.63 57.40 21.61 7.20 .00*
* p < .05

The MANOVA results for the post-test scores revealed a statistically significant group
difference: F (4, 142) = 8.56, p = .00; Wilks’ Λ = 0.65, partial η2 = .19. It can be assumed
that participants’ post-test scores were significantly dependent on which experimental
condition they had been provided with (p = .00). To be more specific, it was found that the
experimental conditions – e-book, audio-book, and e-audio-book – had statistically
significant effects on listening (F (2, 72) = 11.68; p = .00; partial η2 = .25) and reading (F
(2, 72) = 7.20; p = .00; partial η2 = .17), as shown in Table 2 above. That is, the participants
in the current study benefited differently from the input delivery modes.
In order to verify which mean scores significantly differed from which other ones,
Scheffe post hoc test was then run. Table 3 below shows the main findings of the post hoc
test. The test results revealed that mean scores for listening comprehension were
significantly different between the e-book group and the e-audio-book group (p = .00) as
well as the audio-book group and the e-audio-book group (p = .00). Mean scores for reading
comprehension were also significantly different between the audio-book group and the e-
book group (p = .02) as well as the audio-book group and the e-audio-book group (p = .00).

TABLE 3
Scheffe Post Hoc Test
MD SE p
e-book < e-audio-book 23.86 5.86 .00*
Listening
audio-book < e-audio-book 25.61 5.97 .00*
audio-book < e-book 17.05 5.93 .02*
Reading
audio-book < e-audio-book 21.57 5.99 00*
* p < .05

As can be shown from Table 3 above, in listening comprehension, the e-audio-book group
scored significantly higher than the e-book and audio-book groups did. The mean score of
the e-audio-book group was 65.40 on the listening post-test while the e-book group received
41.54 and the audio-book group scored 39.79 on the same test.
The findings of the current study showed that providing audio recordings with the

E-Book, Audio-Book, or E-Audio-Book: The Effects of Multiple Modalities on EFL Comprehension

English Teaching, Vol. 76, No. 4, Winter 2021, pp. 33-52 45

identically written texts led to more positive effects on EFL listening comprehension in
comparison with providing either text or audio only. It can be suggested that this study
confirmed the superior effects of the concurrent use of e-book and audio-book (using both
text and audio simultaneously) on English listening comprehension than the single use of e-
book or audio-book (using either text or audio only). Concerning reading comprehension,
the audio-book group scored significantly lower than the e-book and e-audio-book groups
did. The mean score of the audio-book group was 35.83 on the reading post-test while the e-
book group obtained 52.88 and the e-audio-book group earned 57.40 on the same test.
The findings of the present study showed that the use of e-book led to better performance
on reading comprehension tests compared to that of audio-book. In other words, reading
books were more beneficial than listening to books when it comes to EFL reading. The
current study also confirmed the superior effects of the simultaneous use of e-book and
audio-book on reading comprehension in comparison with the single use of audio-book. That
is, providing fairy tale stories in both text and audio formats resulted in better performance
in EFL reading compared to providing them in audio format only. It can be concluded that
there is superiority of the e-audio-book on English reading comprehension over the audio-
book.

5. DISCUSSION

In order to explore their effects on EFL comprehension, the participants were divided into
three groups – the e-book, the audio-book, and the e-audio-book groups – and engaged in
class activities. Listening and reading pre and posttests were administered to all groups.
Regarding listening comprehension, the group comparison results of this study found the
superior effects of the concurrent use of e-book and audio-book over any single use of either
e-book or audio-book, as the previous scholars have noticed.
Chang (2009), for example, argued that the dual channel is better than the single channel
in comprehension. In particular, Kartal and Simsek (2017) proposed that the concurrent
spoken–written (dual mode) text presentations yield learning benefits in listening
comprehension in comparison to written-only (single mode) text presentations. Mohamed
(2018) also confirmed the superiority of the use of both textual and auditory channels over
the single use of auditory channel on developing listening comprehension. The findings of
this study support all these previous studies, suggesting that the dual use of e-book and audio-
book is more effective that any single use of either e-book or audio-book on listening
comprehension.
Moreover, the current study also reported the superiority of the simultaneous use of e-
book and audio-book over the single use of audio-book regarding EFL reading

© 2021 The Korea Association of Teachers of English (KATE)

46 Na-Young Kim

comprehension. According to Woodall (2010), listening while reading provides students

with a multisensory reading experience, eliminating frustrations for those who have
difficulties with text-only material. Rahman and Hajar (2020) also insisted that reading and
listening to text is beneficial especially to students who struggle to focus on reading activities.
In the same line, Liu, Cao, and Wu (2019) proved that reading and listening to books at the
same time is better than listening alone in reading comprehension. In particular, according
to Turker (2010), intermediate-level EFL students can more benefit from reading while
listening than those at elementary level can. In this regard, the findings of the present study
are in accordance with the previous research, showing the greater effects of using e-audio-
book on intermediate EFL students’ reading comprehensions.
Lastly, the findings of the group comparison revealed that reading e-book compared to
listening to audio-book resulted in more beneficial effects on reading comprehension.
Regarding these results, previous scholars have also confirmed the superiority of visual
modality (written texts) over audio modality (spoken texts). For example, Daniel and Woody
(2010) and Lund (1991) found that students reading texts (visual input) perform better in
recall tests than those listening to texts (auditory input). Reimer (2007) also revealed that
listening to an audio recording is inferior to reading texts. In particular, Yeh and Wang (2003)
reported that EFL students prefer visual materials (text) over auditory materials (audio).
According to them, the EFL students’ preference to visual over auditory learning styles can
explain the superior effects of the use of e-book over that of audio-book in the current study.
Previous research findings on the effectiveness of using dual modality mode (visual and
auditory) have yielded different results. They have not always shown positive effects of dual
modality presentation. For example, Kalyuga (2000) claimed that using text and audio
concurrently is negative in learning. Diao and Sweller (2007) asserted that the single mode
(reading-only) is more effective in comprehension than the dual modality mode (both
reading and listening). Rogowsky, Calhoun, and Tallal (2016) also found that students
performed equally in the comprehension test regardless of whether they read e-book, listened
to audio-book, or both read and listened simultaneously.
Furthermore, there have been some scholars showing the superiority of audio-book over
e-book regarding comprehension, as opposed to the findings of this study. For example, Liu,
Cao, and Wu (2019) discovered that listening to audio-books through the audio channel is
more effective than reading e-books through the visual channel when it comes to reading
comprehension. Montgomery (2009) revealed that EFL students’ reading skills increase
when using audio books in their classroom. Gunduz (2006) also claimed that EFL reading
comprehension can be enhanced by listening to texts.
Rogowsky, Calhoun, and Tallal (2016) pointed out a lack of experimental research
directly examining the effect of different modalities on comprehension. Even so, there have
been conflicting results. Scholars have recognized these inconclusive findings in the

E-Book, Audio-Book, or E-Audio-Book: The Effects of Multiple Modalities on EFL Comprehension

English Teaching, Vol. 76, No. 4, Winter 2021, pp. 33-52 47

literature and stressed the need for resolution (Adesope & Nesbit, 2012; Moreno & Mayer,
2002). Given that more research on multimedia learning is required especially in EFL
settings (Soruc, 2015), this study provides empirical evidence, reporting the effectiveness of
the use of multimedia modalities.

6. CONCLUSION

As Larson (2015) mentioned, e‐books and audio-books are not new, and they have been
utilized to improve foreign language comprehension. The current study was to examine the
effects of using both e-book and audio-book on comprehension skills especially in EFL
settings. Main findings are as follows: First, the participants benefited more from the use of
e-audio-book than the use of either e-book or audio-book in listening comprehension. The
simultaneous use of e-book and audio-book also led to better performance in reading
comprehension in comparison with the single use of audio-book. Lastly, the use of e-book
turned out to be more beneficial compared to that of audio-book in EFL reading
comprehension.
Previous research on the concurrent use of text and audio has shown mixed results. While
proponents of dual modality presentation have shown its advantages for comprehension
(Chang, 2009; Kartal & Simsek, 2017; Liu, Cao, & Wu, 2019; Mohamed, 2018; Rahman &
Hajar, 2020; Woodall, 2010), opponents have disagreed on its presence (Diao & Sweller,
2007; Kalyuga, 2000; Rogowsky, Calhoun, & Tallal, 2016). Researchers have also
understood these contradictory findings and highlighted the need for resolution (Adesope &
Nesbit, 2012; Moreno & Mayer, 2002b). They have suggested further studies, addressing
the dearth of research that explores the modality effects on comprehension (Rogowsky,
Calhoun, & Tallal, 2016), especially in EFL contexts (Soruc, 2015).
In the wake of the ongoing COVID-19 pandemic, the use of different modalities has
continuously been discussed in educational settings, but they are not so certain yet about the
best modality mode to be employed in order to make learning more effective (Marpa, 2021).
From this point of view, the current study addressed this gap by investigating the effects of
using audio-book and e-book on EFL comprehension. The findings of the present study
confirmed the positive effects of the use of e-audio-book on EFL listening and reading
comprehension skills, supporting the proponents of the dual modality mode. Although
scholars have noted that the use of same verbal information in both auditory and visual
channels increases extra cognitive load and interferes learning (Kalyuga, 2000; Kalyuga,
Chandler, & Sweller, 1999), the current study revealed that Korean EFL students still
benefitted from the concurrent use of text and audio.
The results can provide several practical and pedagogical implications. First of all,

© 2021 The Korea Association of Teachers of English (KATE)

48 Na-Young Kim

teachers in EFL fields can make use of e-audio-book in their class in an attempt to improve
their students’ comprehension. The concurrent use of e-book and audio-book is more
recommended rather than any single use of either e-book or audio-book. While reading and
listening to books, the students can improve their listening and reading comprehension skills.
Additionally, between the single use of e-book and audio-book, the use of e-book is more
recommended to improve EFL reading comprehension. The students can more benefit from
reading books than listening to books when it comes to improving their reading
comprehension skills.
However, it should be carefully considered that the findings of this study can be adapted
to intermediate EFL students. According to Turker (2010), although EFL students can
benefit from reading while listening to books, its effects vary based on their proficiency
levels. The modality of presentation can play a less significant role for students who are
already proficient in English (Rogowsky, Calhoun, & Tallal, 2016). Therefore, students’
English proficiency level should be taken account when using e-audio-book in their class.
Furthermore, although the whole experiment in the current study was conducted in class, it
can also be carried out online, considering the pandemic situation where establishing
expertise in the use of technology is a must.
Limitations and suggestions are also made. First, these findings cannot be generalized
with the small sample size (n = 75). All participants were comprised of Korean EFL students.
Their English proficiency level was pre-intermediate. Therefore, different results can be
acquired in different settings. Participants’ different learning styles should also be taken
account. Although Yeh and Wang (2003) reported that EFL students’ preference to visual
over auditory materials and this study proved the superior effects of e-book over audio-book,
they can vary from individual to individual. In addition, this was not a longitudinal study.
Due to the COVID-19, the whole experiment was carried out in a relatively short period of
time (5 weeks). Longitudinal studies can take place with the same dimension with this
current study. Lastly, five selected fairy tales were provided as input contents in the current
study. Although they were given through different modes, it should be considered that the
modality effects can also depend on the input contents and may yield different results. Fairy
tales are considered authentic (Kochiyama, 2013), but it cannot be said that they exactly
meet students’ authentic communicative needs.

Applicable level: Tertiary

E-Book, Audio-Book, or E-Audio-Book: The Effects of Multiple Modalities on EFL Comprehension

English Teaching, Vol. 76, No. 4, Winter 2021, pp. 33-52 49

REFERENCES

Adesope, O. O., & Nesbit, J. C. (2012). Verbal redundancy in multimedia learning

environments: A meta-analysis. Journal of Educational Psychology, 104(1), 250-263.
Atkinson, R. K., Clark, D., Harrison, C., Koenig, A., & Ramirez, F. (2007). Multimedia
learning environments: Do the redundancy and modality principles apply to English
language learners? Paper presented at the Annual Meeting of the American
Educational Research Association, Chicago, IL.
Baddeley, A. (1992). Working memory: The interface between memory and cognition.
Journal of Cognitive Neuroscience, 4(3), 281-288.
Chang, A. C. S. (2009). Gains to L2 listeners from reading while listening vs. listening only
in comprehending short stories. System, 37(4), 652-663.
Daniel, D., & Woody, W. D. (2010). They hear, but do not listen: Retention for podcasted
material in a classroom context. Teaching of Psychology, 37(3), 199-203.
Diao, Y., & Sweller, J. (2007). Redundancy in foreign language reading comprehension
instruction: Concurrent written and spoken presentations. Learning and Instruction,
17(1), 78-88.
Egidi, L., & Furini, M. (2006). From digital audiobook to secure digital multimedia-book.
Computers in Entertainment, 4(3), 1-19.
Gunduz, N. (2006). Contributions of E-audiobooks and podcast to EFL listening classes.
Seljuk University Joumal of Faculty of Letters, 21, 249-259.
Han, J.-I., & Kim, N.-Y. (2016). The effects of post-tasks CMC activities and task types on
Korean EFL learners’ oral performance. STEM Journal, 17(2), 109-135.
Holt, J. (2005). The underachieving school. Boulder, CO: Sentient.
Kalyuga, S. (2000). When using sound with a text or picture is not beneficial for learning.
Australasian Journal of Educational Technology, 16(2). 161-172.
Kalyuga, S., Chandler, P., & Sweller, J. (1999). Managing split‐attention and redundancy in
multimedia instruction. Applied Cognitive Psychology: The Official Journal of the
Society for Applied Research in Memory and Cognition, 13(4), 351-371.
Kalyuga, S., Chandler, P., & Sweller, J. (2004). When redundant on-screen text in
multimedia technical instruction can interfere with learning. Human Factors, 46(3),
567-581.
Karbalaei, A., & Zare, A. (2019). A comparison of the effect of textual, audio and textual-
pictorial and audio-pictorial annotations on enhancing reading comprehension
among Iranian EFL learners. Teaching English with Technology, 19(3), 40-67.
Kartal, G., & Simsek, H. (2017). The effects of audiobooks on EFL students’ listening
comprehension. The Reading Matrix: An International Online Journal, 17(1), 112-
123.

© 2021 The Korea Association of Teachers of English (KATE)

50 Na-Young Kim

Kim, H.-S., Cha, Y., & Kim, N.-Y. (2021). Students’ perspectives on pre-recorded video
lectures versus video-conferencing lectures in the untact era. Journal of Research in
Curriculum and Instruction, 25(2), 164-178.
Kim, N.-Y. (2019). Effects of captioning order on EFL listening and reading: Should captions
be first or second? English Language and Literature Teaching, 25(4), 1-22.
Kochiyama, A. (2013). Teaching gender awareness through fairy tales in the EFL classroom.
English Review: Journal of English Education, 2(1), 1-7.
Koroghlanian, C. M., & Sullivan, H. J. (2000). Audio and text density in computer-based
instruction. Journal of Educational Computing Research, 22(2), 217-230.
Kosal, İ. (2008). Enhancing writing and improvisational speaking skills through fairy tales
in EFL preparatory classrooms (Unpublished doctoral dissertation). Selçuk
University, Konya, Turkey.
Larson, L. C. (2015). E‐books and audiobooks: Extending the digital reading experience.
The Reading Teacher, 69(2), 169-177.
Lazar, G. (1993). Literature and language teaching. Cambridge, MA: Cambridge University
Press.
Lee, C. (2003). The uses of enchantment: Fairy tales as instructional materials to facilitate
primary English language education. Journal of National Taipei Teachers College,
16(1), 29-48.
Lee, Y., & Joo, J. (2017). Using design methods to improve design quality: Verbalizers vs
visualizers. The Design Journal, 20(1), 1471-1484.
Lewandowski, L. J., & Kobus, D. A. (1993). The effects of redundancy in bimodal word
processing. Human Performance, 6(3), 229-239.
Liu, H., Cao, S., & Wu, S. (2019). An experimental comparison on reading comprehension
effect of visual, audio and dual channels. Proceedings of the Association for
Information Science and Technology, 56(1), 716-718.
Lund, R. J. (1991). A comparison of second language listening and reading comprehension.
The Modern Language Journal, 75, 196-204.
Marpa, E. P. (2021). Technology in the teaching of mathematics: An analysis of teachers’
attitudes during the COVID-19 pandemic. International Journal on Studies in
Education, 3(2), 92-102.
Massi, M. P., & Adriana, M. B. (2001). Using fairytales to develop reading and writing skills.
The CATESOL Journal, 13(1), 161-171.
Mayer, R. E. (2001). Multimedia learning. New York: Cambridge University Press.
Mayer, R. E. (2005). Cognitive theory of multimedia learning. In R. E. Mayer (Ed.), The
Cambridge handbook of multimedia learning (pp. 31-48). New York: Cambridge
University Press.

E-Book, Audio-Book, or E-Audio-Book: The Effects of Multiple Modalities on EFL Comprehension

English Teaching, Vol. 76, No. 4, Winter 2021, pp. 33-52 51

Mayer, R. E., & Moreno, R. (2002). Animation as an aid to multimedia learning. Educational
Psychology Review, 14(1), 87-99.
Mohamed, M. M. K. (2018). Using audiobooks for developing listening comprehension
among Saudi EFL preparatory year students. Journal of Language Teaching and
Research, 9(1), 64-73.
Montgomery, J. R. (2009). Using audiobooks to improve reading and academic performance.
Retrieved on September 11, 2021 from https://files.eric.ed.gov/fulltext/ED50594
7.pdf
Moreno, R., & Mayer, R. E. (2002). Verbal redundancy in multimedia learning: When
reading helps listening. Journal of Educational Psychology, 94(1), 156-163.
Moyer, J. E. (2011). Teens today don’t read books anymore: A study of differences in
comprehension and interest across formats (Unpublished doctoral dissertation).
University of Minnesota, Minnesota.
Najjar, L. J. (1995). Dual coding as a possible explanation for the effects of multimedia on
learning. Retrieved on September 11, 2021 from https://smartech.gatech.edu/bitstr
eam/handle/1853/ 3573/95-29.pdf;sequence=1
Nasser, D. L., & McEwen, W. J. (1976). The impact of alternative media channels: Recall
and involvement with messages. AV Communication Review, 24(3), 263-272.
Plass, J. L., Moreno, R., & Brünken, R. (2010). Cognitive load theory. New York: Cambridge
University Press
Rahman, A., & Hajar, I. (2020). The effect of audiobook on reading comprehension of the
eleventh grade students of SMA Negeri 2 Buru. ELT Worldwide, 7(2), 104-115.
Rao, S. S. (2003). Electronic books: A review and evaluation. Library Hi Tech, 27(1), 85-93.
Reimer, S. (2007, August 14). Listening to books is not cheating. The Baltimore Sun.
Retrieved on September 11, 2021 from http://articles.baltimoresun.com/2007-08-
14/features/0708140192_1_.
Rogowsky, B. A., Calhoun, B. M., & Tallal, P. (2016). Does modality matter? The effects of
reading, listening, and dual modality on comprehension. SAGE Open, 6(3), 1-9.
Soruc, A. (2015). What makes redundant presentation of multimedia learning difficult?
Procedia-Social and Behavioral Sciences, 191, 2844-2848.
Tang, Y. Y., & Posner, M. I. (2009). Attention training and attention state training. Trends in
Cognitive Sciences, 13(5), 222-227.
Trew, G. (2008). Tactics for TOEIC Listening and Reading Test: Student’s book. Oxford,
England: Oxford University Press
Turker, S. (2010). The effectiveness of audio books on the reading comprehension of selected
texts by university EFL students at different proficiency levels (Unpublished doctoral
dissertation). Bilkent University, Ankara, Turkey.

© 2021 The Korea Association of Teachers of English (KATE)

52 Na-Young Kim

Wei, C. C., & Ma, M. Y. (2020). Designing attractive children’s picture books: Evaluating
the attractiveness factors of various picture book formats. The Design Journal, 23(2),
287-308.
Woodall, B. (2010). Simultaneous listening and reading in ESL: Helping second language
learners read (and enjoy reading) more efficiently. TESOL Journal, 1(2), 186-205.
Yeh, Y., & Wang, C. (2003). Effects of multimedia vocabulary annotations and learning
styles on vocabulary learning. CALICO Journal, 21(1), 131-144.

E-Book, Audio-Book, or E-Audio-Book: The Effects of Multiple Modalities on EFL Comprehension