Available online at www.sciencedirect.

com

Procedia Social and Behavioral Sciences 2 (2010) 4480–4486

WCES-2010

Evaluating working memory capacity and cognitive load in learning

from goal based scenario centered 3D multimedia
Eylem Kilica,b *, Zahide Yildirima
a
Faculty of Education,Middle East Technical University, Ankara, 06531, Turkey
b
Faculty of Education. Yuzuncu Yil University, Van, 65080, Turkey

Received November 4, 2009; revised December 7, 2009; accepted January 19, 2010

Abstract

The purpose of this study is to investigate working memory capacity and cognitive load in learning from Goal Based Scenario
centered 3D multimedia learning environment (GBSc3DM) designed based on Cognitive Load Theory (CLT). GBSc3DM was
developed in two versions. In the designed of the first version (+CLT) cognitive load principles were applied. In the second
version (-CLT), however, the principles were violated. 47 11th grade high school students were selected based on their working
memory capacity (WMC). A series of parametric and non parametric statistical techniques were used to analyze the data. The
findings are discussed in the following sections.
© 2010 Elsevier Ltd. Open access under CC BY-NC-ND license.

Keywords: Working memory capacity; cognitive load; goal based scenario; instructional design; multimedia; individual difference.

1. Introduction

Goal based scenario (GBS) offers realistic environments for complex learning tasks. The severe risk of this
approach is high task complexity. If learners cannot handle the high task complexity, the learners’ limited working
memory is overload. Cognitive load theory (CLT) provides valuable guidelines on how to deal with the overload
(Van Merriënboer, Kirschner & Kester, 2003). CLT “is concerned with the development of instructional methods
that efficiently use people’s limited cognitive processing capacity to stimulate their ability to apply acquired
knowledge and skills to new situations” (Paas, Tuovinen, Tabbers &:Van Gerven, 2003, p. 63). There are three types
of cognitive load which are intrinsic, extraneous and germane. Extraneous cognitive load is the result of
implementing “instructional techniques that require students to engage in activities that are not directed at schema
acquisition” (Sweller 1994: p299). It is the effort required by the learner to process poorly designed instruction
(Kirschner, 2002). In present study, the powerful CLT-training formats reducing extraneous cognitive load split
attention, redundancy, signaling and modality principle (Bannert, 2002), and multimedia and coherence principle
proposed in cognitive theory of multimedia learning (CTML) by Mayer (2001) were investigated. Multimedia
presentation refers to any presentation that contains printed/spoken text and static/dynamic illustrations (Clark &
Mayer, 2003; Mayer &Moreno, 2002). Modality refers to placing material into spoken forms of words rather than

* Eylem Kilic. Tel.: +0-312-210-3673; fax: +0-312-210-7986

E-mail address: [email protected]

1877-0428 © 2010 Published by Elsevier Ltd. Open access under CC BY-NC-ND license.
doi:10.1016/j.sbspro.2010.03.715
 Eylem Kilic and Zahide Yildirim / Procedia Social and Behavioral Sciences 2 (2010) 4480–4486 4481

printed word whenever the graphic and/or animation is the focus of the words and both are given simultaneously
(Clark & Mayer, 2003; Sweller,Van Merriënboer & Paas, 1998). Redundancy refers to presenting words in both text
and audio narration which hinder learning. Spilt attention refers to presenting words and pictures separately.
Learners must use their limited cognitive resource to use mentally organize and integrate the materials when they are
separated from each other on the screen. On the contrary, if they are integrated, learners can combine them in their
working memory and make meaningful connection between them. Coherence refers to presenting irrelevant sound,
picture and graphics which can hurt learning in learning materials. In line with the coherence principle, extraneous
picture and word should be eliminated (Clark & Mayer, 2003; Mayer and Moreno, 2002). Signaling refers to adding
non content information, visually or auditory, to the content in order to focus attention to those aspects which is
important while watching dynamic display (Sweller et al., 1998).
CLT and CTML based their assumption on limited working memory model proposed by Baddeley and Hitch
(1974). Working memory “refers to a brain system that provides temporary storage and manipulation of the
information necessary for such complex cognitive task as language comprehension, learning and
reasoning”(Baddeley, 1992, p. 256). Working memory consists of three subcomponents which are central executive,
phonological loop and visuospatial sketchpad. Phonological loop and visuospatial sketchpad are two slave system of
working memory (Baddeley, 1992). Central executive which is most crucial component of working memory is
responsible for coordinating information from two-slave system and manipulation of information for higher order
cognitive skills. Since the function of this system are related to the higher order cognitive skills like reading and
comprehension while keeping information in a short period, the variation of individual on working memory should
be consistent with the performance on cognitive tasks (Unsworth & Engle, 2007).
Although the central assumptions of cognitive theories are based on limited WMC, there is not enough research
to investigate the effects of WMC and its relation to cognitive load in multimedia. The effects of WMC might be
much more important than before since theories recently try to find out new instructional formats that impose
germane cognitive load which is related to the construction of meaningful schemas. However, most research in
cognitive theories focus on dual task paradigm to investigate the effects of two slave system and the principles in
multimedia (Mayer & Sims, 1994; Gyselinck, Cornaldi, Dubois, De Beni & Ehrlich, 2002; Pujari, 2007) and their
realtion to cognitive load (Pujari, 2007). The effect of WMC has been investigated for elderly people (Van Gerven,
Paas, Van Marrienboer & Schmidt, 2000; Van Gerven, Paas, Van Marrienboer, Hendrick & Schmidt, 2003; Van
Gerven, Paas, Van Marrienboer & Schmidt, 2006). Van Gerven et al (2000) propose some instructional formats that
aim to help elderly people because of decline in their WMC and reduced ability to distinguish between relevant and
irrelevant information. Van Gerven et al (2003) found that both young and elderly groups benefit most from bimodal
condition of instructional format and invested less mental effort, particularly; elderly participants benefit more from
bimodal condition. In a recent study, there was no significant difference between modality and variability of worked
example for elderly participants and they invested less mental effort in both conditions (Van Gerven et al., 2006).
Goal based scenario requires students to perform complex cognitive task by combining and maintaining verbal
and visual information. Therefore, rather than two slave systme, WMC was taken into account for this study. That is,
WMC is the measurement of central executive (Unsworth, Heitz, Schrock & Engle, 2005) and it is known that the
function of central executive is coordinating two slave systems and maintaining information, hence, to investigate
WMC as an individual difference for such a complex learning environment assumed to be much more appropriate
for the present study. The following research questions were quided this study:
1. To what extent cognitive load could explain the possible difference on learning outcome from goal based
scenario designed multimedia for the learners with different working memory capacity.
1.1. Is there a significant difference between high WMC, medium WMC and low WMC learners on
learning outcomes from the second version (-CLT)?
1.2. Is there a significant difference between high WMC, medium WMC and low WMC learners on
learning outcomes from the first version (+CLT)?
1.3. Is there a significant difference between the first version (+CLT) and the second version (-CLT) on
learning outcome?
1.4. Is there an interaction effect between learners’ working memory capacity and two version of
GBSc3DM on learning outcome?
4482 Eylem Kilic and Zahide Yildirim / Procedia Social and Behavioral Sciences 2 (2010) 4480–4486

2. Method

2.1. Design

2 x 3 factorial design was used in this study. The independent variables were the two versions of multimedia (the
first (+CLT) and the second (+CLT) versions of GBSc3DM), and X Working Memory Capacity (high WMC,
medium WMC and low WMC). The dependent variables were the students’ log files, achievement test scores as a
pre-test and post-test, and mental effort. Pre-test scores were analyzed across independent variables, and the result
was taken into consideration as a covariance when there was a significant difference.

2.2. Participants

47 11th grade students (25 females and 22 males) were selected based on their working memory capacity from
from one of the Anatolian High Schools participated in this study. Students from high WMC and low WMC were
randomly assigned to either of (+CLT or –CLT) multimedia learning environments.

2.3. Software development

The GBSc3DM was developed as a game based learning environment. The goal was to restart the mitosis and
meiosis process which did not take place because of viruses’ attacks to cells. To achieve the goal, the students had to
sequence the main phases of mitosis and meiosis in the correct order, and then should complete the sub phases. The
GBSc3DM included library of resources about the topic to provide support for the students. In the first version
(+CLT), the multimedia was developed based on “split attention, multimedia, modality, coherence, signaling and
redundancy” principles that reduce extraneous cognitive load. In the second version (-CLT), however, the
multimedia was developed without these principles. Other than the implementation or violation of the principles, the
remaining design and the content were the same in both versions. The information on implementation and violation
of the principles in the two versions of the multimedia is provided in Table 1.
Table 1. The design issues in the first (+CLT) and the second version (-CLT) of multimedia

VERSIONS
PRINCIPLE First version (+CLT) Second version (-CLT)
Split • The explanation for each button on the main screen was • The explanation for each button on the main screen placed at
Attention placed next to the related button the lower side of the screen
• Each hyperlink was opened in the same window in • Each hyperlink in the library was opened in a different
library window in the second version.
Multimedia • Picture and text were presented together to order main • Only text was presented to order main phases
phase
• Pictures and text were given together in library design • Text was given in library design
Modality • Explanations of 3D animation for mitosis and meiosis • Explanations of 3D animation for mitosis and meiosis and
and all sub phases were given in audio format all sub phases were given in text format
Redundancy • Both text and narration were given, however, choice of • Text was given with animation and no narration and switch
switching off the text or the audio explanations of off options were given. Since background music was
animations were given incorporated, the audio explanation was excluded.
Signaling • Key concepts in library were highlighted • Key concepts in library were not highlighted
Coherence • Irrelevant background music was eliminated • Irrelevant background music (without word) was
incorporated

ʹǤͶǤ Instruments

2.4.1. Automated opretaion digit span task (AOSPAN)

Automated operation span task (AOSPAN) was mouse driven, and the participants only need to click on the
mouse to complete the task. They also can complete the task independently of the researcher (Unsworth, Heitz,
Schrock & Engle, 2005). The participants were instructed to solve some mathematical problems, and then try to
recall the letters appeared among math operations in the correct order. AOSPAN has both good internal consistency
(alpha 0.78) and test–retest reliability (0.83) in the original study. The scores gained from the task range from 0 to 75
 Eylem Kilic and Zahide Yildirim / Procedia Social and Behavioral Sciences 2 (2010) 4480–4486 4483

2.4.2. Cell achievement test

A cell achievement test having 20 multiple choice questions with four options were used as the pre-test and the
post-test. The content validity of the test was ensured with subject area teachers. The reliability of the test was 0.73
for Cronbach’s alpha and 0.75 for Kuder Richardson-20 (Atilboz, 2004).

2.4.2. Log files

Log files were created and updated automatically as participants proceeded through the program. Seven
variables had been recorded. These were the scores gained from multimedia, total time spent in the environment,
time spent in sequencing the main phases, the number of errors made in sequencing the main and the sub phases, the
frequency of library use. Students earned 10 points for each right answer, and lost 10 point for each wrong answer.
The score was computed by subtracting total points of wrong answers from total points of right answers.

2.4.3. Subjective rating scale

Participants expressed their mental efforts with 9-point mental effort rating scale ranging from 1 (very, very low
mental effort) to 9 (very, very high mental effort). The validity and reliability of this scale was tested in Turkish
context by Kilic and Karadeniz (2004). The reliability of this scale was found to be 0.78 (Cronbach’s alpha).

2.5. Procedure
The study lasted two classroom hours. Before the study started, AOSPAN were administered to find out
participants’ working memory capacity. Among 118 participants , 47 of them were selected for the present study.
Firstly, cell achievement test was administrated to the selected participants. Then, the students were required to use
the multimedia (+CLT and –CLT), and log files were kept, and mental effort was measured. At the end of the study,
students were required to complete the cell achievement test as a posttest.

2.6. Data analysis

Analysis of Covariance (ANCOVA), Analysis of Variance (ANOVA), and Kruskall Wallis non parametric test
were used to the analyze data. If the assumption of two way ANCOVA was not meet, two way ANOVA was used,
and then if the assumption of two way ANOVA was not meet then Kruskal Wallis was used to analyze data. For all
statistical tests a significance level of .05 was maintained.

3. Result

3.1. Prior knowledge

A cell achievement test was administrated to find out the students’ prior knowledge in the beginning of the study.
The result of two way ANOVA showed that there was no significant difference in students’ prior knowledge
between two versions of multimedia, working memory capacity, and interaction between the two versions of
multimedia and working memory capacity, F(1, 41) = .815, p=.37; F(2, 41)= 2.968, p=.06; F(2, 41)= 2.678, p= .08.
Although there was no significant difference with regard to the prior knowledge, the p value for WMC and
interaction between WMC and the two versions of multimedia was very close to significance level. Hence, the
researchers decided to take students’ prior knowledge as a covariance.

3.2. Variables measured for the whole process

Two way ANVOCA was used to analyze the effects of multimedia versions and WMC on mental effort, post test
(cell achievement test), learning time and library use. No main effect was found for multimedia versions (F (1, 40)
= 0.36, MSE = 1.94, p = .55), working memory capacity (F (2, 40) = 1.16, MSE = 6.29, p = .32), and no interaction
effect was found for working memory capacity and multimedia versions ((F (2, 40) = 1.22, MSE = 0.55, p = .30) on
post-test. No main effect was found for multimedia versions (F (1, 40) = 0.65, MSE = 1.98, p =.42), working
memory capacity (F (2, 40) = 0.08, MSE = 0.26, p = .91), and no interaction effect was found for working memory
capacity and multimedia versions (F (2, 40) = .86, MSE = 2.62, p = 42) on invested mental effort. No main effect
was found for multimedia versions (F (1, 40) = 0.15, MSE = 70440.47, p = 69), working memory capacity (F (2, 40)
4484 Eylem Kilic and Zahide Yildirim / Procedia Social and Behavioral Sciences 2 (2010) 4480–4486

= 0.81, MSE = 364568, 97, p =.42), and no interaction effect was found for working memory capacity and
multimedia versions (F (2, 40) = 1.39, MSE = 627939, 25, p = .26) on learning time. No main effect was found for
versions of multimedia (F (1, 40) = 0.52, MSE = 45.90, p = .47), working memory capacity (F (2, 40) = 1.36, MSE =
119.92, p = .26), and no interaction effect was found for working memory capacity and multimedia versions (F (2,
40) = 2.32, MSE = 204.37, p = .11) on the frequency of library use.
Two ways ANOVA was used to analyze data for score gained from multimedia. Main effect was found for
multimedia versions (F (1, 41) = 4.26, p= 0.04, Ș2 p=.09. The strength of relationship between multimedia versions
and score was medium. Students’ score in the first version (+CLT) was higher (M=602.31 SD= 36.12) than that in
the second version (-CLT) of multimedia program (M=498.37, SD= 35.04). No main effect was found for working
memory capacity (F (2, 41) = 0.84, p= 0.43, Ș2 p=.04, and no interaction effect was found for working memory
capacity and multimedia versions (F (2, 41) = 0.04, p= 0.95, Ș2 p=.0.002 on score gained from multimedia.

3.3. Mitosis findings

A Kruskal-Wallis test was used to compare the six conditions on main phase time and the number of errors
made in sequencing main phases. No effects of confition was found for main phase time H (5) = 6.95, p=.22 and
for main phase error, H (1)= 2.79, p=.09. Two ways ANCOVA was used to analyze the data for each phase error.
A main effect was found for multimedia versions (F (1, 40) = 4.53, MSE = 1764.98, p = .03, Ș2p = .10). The strength
of relationship between multimedia versions and each phase error was medium. The adjusted mean for each phase
error rate was found significantly lower (M= 15.40) in the first version compared to second version (M= 27.90) of
multimedia. However, no main effect was found for working memory capacity (F (2, 40) = 1.97, MSE = 789.98, p
=.15), and interaction effect for working memory capacity and multimedia versions (F (2, 40) = .05, MSE = 21, 88, p
= .94) on each phase error.

3.4. Meiosis findings

A Kruskal-Wallis test was used to compare the six conditions on main phase time and the number of errors
made for each phase. No effects of condition was found for main phase time, H (5) = 8.62, p=.12. On the other
hand, a significant effect of condition was found, H (5) = 8.62, p=.12. Multiple comparisons among groups were
conducted with Mann Whitney U test. The test result comparing high WMC/first version and high WMC/second
version were significant, z= -2.893, p=0.002 and showed a significantly lower error rate for high WMC/first version
(mean rank= 3.92) than high WMC/second version (mean rank= 10.72). The test comparing high WMC/first version
and low WMC/second version were found significant, z=-2.152, p=0.03 and showed a significantly lower error rate
for high WMC/first version (mean rank= 4.50) than low WMC/second version (mean rank= 9.14). Two ways
ANCOVA was used to analyze the data for main phase error. No main effect was found for multimedia versions (F
(1, 40) = 0.5, MSE = 15.70, p = .46), working memory capacity (F (2, 40) = 0.97, MSE = 27.79, p = .39) and no
interaction effect was found for working memory capacity and multimedia versions ((F (2, 40) = 0.18, MSE = 5.38,
p = .332) on main phase error.

4. Discussion

The effect of WMC was only obtained in each phase error rate in meiosis. The possible reason for this finding
might be related to the task itself. Meiosis had 8 sub phases and each subphase has several sub items needs to be
sequenced correctly. This task is relatively difficult than the other task in multimedia and the task demanded more
attention to maintain task goal. In previous research, the difference between high WMC and low WMC can only
observed when task demanded more attention to complete the task (Kane & Engle, 2003). In another study,
performance difference between high and low WMC students found out in anti saccade task than the pro saccade
task (Kane, Bleckley, Conway & Engle, 2001). In conclusion, the task characteristic requiring demanded attention is
an important factor that reveals performance difference between high WMC and low WMC participants.
Both high and low WMC students in the first version (+CLT) made significantly lower error in sequencing each
phase of meiosis than the high and low WMC students in the second version (-CLT). It is assumed that high WMC
students are able to adjust their attention by zooming out and zooming in based on the task demand. Therefore, it is
 Eylem Kilic and Zahide Yildirim / Procedia Social and Behavioral Sciences 2 (2010) 4480–4486 4485

expected that high WMC students in second version (-CLT) performed relatively similar as the performance of high
WMC students in the first version (-CLT). Contrary to this prediction, the result of this study showed that high
WMC students performed better when cognitive load principles applied in learning environment. Therefore, it can
be concluded that working memory capacity act as enhancer for high WMC students (Hambrick & Engle, 2002) and
increase the effects of cognitive load principles on performance.
Low WMC students in the first version (-CLT) made lower error than the high WMC and low WMC students in
the second version (-CLT). It is assumed that the low WMC students performed better in the first version (+CLT)
compared to the second version (-CLT) because the low WMC learners has more capacity to store and process the
information. The result is consistent with this assumption. However, the low WMC students in the first version
(+CLT) made lower error than the high WMC student in the second version (-CLT). The reason for this result might
be related to the task purposes and individual characteristics. The learning environment was designed as a game
based learning environment and so the low WMC learner tried to maintain and coordinate their attention in order to
complete the given task in the first version (+CLT). As pointed out by Cankaya (2007), low WMC students in game
condition reading purposes make significantly more evaluative judgment than the test condition in verbal protocol
task. Furthermore, low WMC readers produced more predictive inferences than the high WMC group in verbal
protocol task and free recall. In addition, the participants’ characteristics such as age play an important role in
findings in that in a previous research it is found that low WMC adults made lower predictive judgment than high
WMC adults’ participants (Linderholm & Van den Broek, 2002. The target group is also adolescents in the current
study therefore it can be expected that the young-low WMC students performed differently than adult-low WMC
participants.

5. Conclusion and recommendation

In conlusion, the present study shows that the principles reducing extraneous cognitive load increased learning gains
of the students in learning from multimedia. It is also found that students having high and low working memory
capacity in the first version (+CLT) of multimedia performed better than the high and low WMC students in the
second version of (-CLT) multimedia. Therefore, it can be conluded that the principles aiming to reduce extranous
cognitive load free up students’ working memory capacity, and so allow them to use their capacity for effective
learning. However, further studies in different subject areas and at different grade levels are needed to make strong
conclusions.

References

Atilboz, N. G., (2004) Lise 1. Sinif Ogrencilerinin Mitoz ve Mayoz Bolunme Konulari ile Ilgili Anlama Duzeyleri ve Kavram Yanilgilari, Gazi
Universitesi Egitim Bilimleri Dergisi, 24, 147-157.
Baddeley, A. (1992) Working memory. Science, 255, 556–559.
Baddeley, A. D. & Hitch, G. Working memory. In G. H. Bower (Eds.), The psychology of learning and motivation (pp.47-89). New York:
Academic Press,
Bannert, M. B. (2002). Managing cognitive load: Recent trends in cognitive load theory. Learning and Instruction, 12, 139-146.
Cankaya, O. Z. (2007). Influence of Working Memory Capacity and Reading Purpose on Young Readers’ Text Comprehension, Unpublished
Master thesis, McGill University
Clark, R., & Mayer, R.E., (2003), e-Learning and the Science of Instruction. San Francisco, CA. Pfeiffer.
Gyselinck, V., Cornald,, C., Dubois, V., De-Beni, R., & Ehrlich, M. (2002). Visuospatial memory and phonological loop in learning multimedia.
Applied Cognitive Psychology, 16, 665-685.
Hambrick, D.Z & Engle, R.W (2002). Effects of domain knowledge, working memory capacity, and age on cognitive performance: An
investigation of the knowledge-Is-power hypothesis. Cognitive Psychology, 44, 339–387.
Kane, M. J., & Engle, R. W. (2003). Working memory capacity and the control of attention: The contributions of goal neglect, response
competition, and task set to Stroop interference. Journal of Experimental Psychology: General, 132, 47-70.
Kane, M. J., Bleckley, M. K., Conway, A. R. A., & Engle, R. W. (2001). A controlled-attention view of working memory capacity: Individual
differences in memory span and the control of visual orienting. Journal of Experimental Psychology: General, 130, 169-183.
Kilic E. & Karadeniz, S. (2004). Hiper Ortamlarda Ögrencilerin Bilissel Yuklenme ve Kaybolma Duzeylerinin Belirlenmesi. Kuram ve
Uygulamada E÷itim Yönetimi Dergisi, 40, 562-579.
Kirschner, P. A. (2002). Cognitive load theory. Learning and Instruction, 12, 1-10.
4486 Eylem Kilic and Zahide Yildirim / Procedia Social and Behavioral Sciences 2 (2010) 4480–4486

Mayer R. E., & Sims, V. K. 1994. For whom is a picture worth a thousand words? Extensions of a dual coding theory of multimedia learning.
Journal of Educational Psychology, 86, 389–401.
Mayer, R. E. (2001). Multimedia learning. New York: Cambridge University Press.
Mayer, R. E., Moreno, R. (2002). Animation as an aid to multimedia learning. Educational Psychology Review, 14, 87-99.
Pass, F., Tuovinen, J.E., Tabbers, H, Van Gevren, P.W.M. (2003). Cognitive Laod Measurement as a Means to Advance Cognitive Laod Theory.
Educational Psychologist, 38, 63-71.
Pujari, A. B. (2007). The effects of presentation format and individual difference in cognitive load. Unpublished master thesis, University of
Arkansas.
Sweller, J. (1994). Cognitive load theory, learning difficulty and instructional design. Learning and Instruction, 4, 295-312.
Sweller, J., Van Merriënboer, J. J. G., & Paas, F. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10,
251–295.
Unsworth, N., Heitz, R. P., Schrock, J. C., & Engle, R. W. (2005). An automated version of the operation span task. Behavior Research Methods,
37, 498–505.
Unsworth, N.,& Engle, R. W. (2007). The nature of individual differences in working memory capacity: active maintenance in primary memory
and controlled search from secondary memory, Psychological Review, 114, 104-132.
Van Gerven, P. W. M., Paas, F., Van Merriënboer,, J. J. G., & Schmidt, H. G. (2000). Cognitive load theory and the acquisition of complex
cognitive skills in the elderly: towards an integrative framework. Educational Gerontology, 26, 503–521.
Van Gerven, P. W. M., Paas, F., Van Merriënboer,, J. J. G., & Schmidt, H. G. (2006). Modality and variability as factors in training the Elderly,
Applied Cognitive Psychology, 20, 311-320.
Van Gerven, P.W.M., Paas, F., Merriënboer,, J. J. G., Hendriks, M., & Schmidt, H. G. (2003). The efficiency of multimedia training into old age.
British Journal of Educational Psychology, 73, 489–505.
Van Merriënboer, J. J. G., Kirschner, P. A., & Kester, L. (2003). Taking the load of a learner’s mind: Instructional design for complex learning.
Educational Psychologist, 38, 5–13.