See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/220886350

Comparing Learning Results of Web Based and Traditional Learning Students

Conference Paper in Lecture Notes in Computer Science · December 2010

DOI: 10.1007/978-3-642-17407-0_39 · Source: DBLP

CITATION READS

1 209

2 authors:

Julija Lapuh Bele Jože Rugelj

B2 d.o.o. / Ljubljana School of Business University of Ljubljana
27 PUBLICATIONS 198 CITATIONS 115 PUBLICATIONS 778 CITATIONS

SEE PROFILE SEE PROFILE

All content following this page was uploaded by Jože Rugelj on 03 January 2014.

The user has requested enhancement of the downloaded file.

 Comparing Learning Results of Web Based and
Traditional Learning Students

Julija Lapuh Bele1,2 and Joze Rugelj1

1
University of Ljubljana, Faculty of Education
2
B2, Vocational College
[email protected], [email protected]

Abstract. Appropriately designed web based learning materials are thought to

be more efficient than traditional ones because of interactivity and multimedia,
which enable clear explanations, motivating design, active learning, instant
feedback and other advantages. Our research question is whether students
achieve better learning results if they learn from web based learning materials
than from traditional ones. Although the research results show that students
spent less time learning from traditional learning materials than students
learning from web based learning materials, the differences of learning times
were not significant. However, we proved significant differences in terms of
gained knowledge. The students using web based learning materials performed
significantly better than students using traditional learning materials.

Keywords: web based learning, multimedia learning, efficiency of e-learning.

1 Introduction
Nowadays, efficiency and effectiveness are expected in business as well as in
education. Employed adult part-time students have to be very efficient as they have
less time for learning than regular students. In learning courses adjusted to part-time
students, institutions offer less so called “contact hours”. As learning goals and
competences are the same for all the students studying in the same study program,
part time students have to spend more time learning independently. Therefore we
decided to explore the learning results (i.e. learning time, gained knowledge) of
students in ICT course, who learnt mostly independently, self-paced, depending on
the chosen learning materials (i.e. web based or traditional).
As traditional learning materials (TLM) we consider materials whose printed
version retains all the functionality. They can be available via Internet as PDF files or
as printed materials. In this research web based learning materials (WBLM) are
hypermedia learning objects accessed via internet and LMS system. They integrate
dynamic multimedia (e.g. animations, simulations, video) and knowledge evaluation
with automated feedback.

2 Theoretical Framework
Appropriately designed WBLMs are supposed to be more effective than traditional
ones because of interactivity and multimedia, which provide clear explanations,

X. Luo et al. (Eds.): ICWL 2010, LNCS 6483, pp. 375–380, 2010.
© Springer-Verlag Berlin Heidelberg 2010
376 J.L. Bele and J. Rugelj

attractive and motivating design, active learning, instant feedback, and other
advantages. However, the use of multimedia does not assure better learning results
than traditional learning. The use of multimedia can enhance learning or impede it
with counterproductive overload of human sensory channels and working memory
[16], [10]. Therefore, traditional learning can be even more effective than web based
learning if the former is not adequately designed.
Many researchers prepared recommendations for efficient web based learning [1-6],
[12], [15]. The use of multimedia tools must be carefully planned and designed.
Recently, cognitive psychology has progressed and many of the major findings in this
field have been of use in instructional design of multimedia educational materials.
Mayer [10] presented cognitive theory of multimedia learning which draws on dual
coding theory, cognitive load theory and constructivist learning theory. He considered
Miller’s findings that human working memory is limited [12] and that people process
visual and verbal information in separate cognitive channels, which work
simultaneously [13]. Considering this theories Mayer [10-11] suggested a set of design
principles for multimedia learning materials: redundancy (i.e. use animation and
narration rather than animation, narration and on-screen text), coherence (i.e. exclude
extraneous words, pictures and sounds), spatial contiguity (i.e. present corresponding
words and pictures near rather than far from each other), temporal contiguity (i.e.
present corresponding words and pictures simultaneously rather than successively),
modality (i.e. present animation and audio narration rather than animation and on-screen
text), multimedia (i.e. present narrated or printed words and pictures rather than words
alone).
Mayer proved that students learn more if learning materials are designed according
to these principles. Nevertheless, these principles are just recommendations. Mayer
claims that design principles have stronger implications for low-knowledge students
than for high-knowledge students and for high-spatial students than for low-spatial
students [10]. Mayer also agrees that students with higher prior knowledge seem to be
able to construct a mental model of the described content also only from the text.
Schnotz & Bannert [13] indicate that presenting graphics is not always beneficial for
the acquisition of knowledge. Their study suggests that pictures facilitate learning if
individuals have low prior knowledge and if the subject matter is visualized in a task-
appropriate way. Also, we cannot suggest strict implementation of modality principle
if students are used to read subtitles [8-9]. In many countries with small native
language speaking population (e.g. Slovenia) movies in foreign languages are not
synchronized and people are used to read subtitles. Lapuh Bele and Rugelj [9]
verified the modality principle and found no significant differences in learning results
(i.e. gained knowledge, learning time) between students that learn from animations
with online text and students that learn from animations with narrated text. For that
reason we have designed WBLMs where students can choose between the animations
with narrated text and animations with online text. Although Mayer’s multimedia
principles are only recommendations, we have taken them, except modality, into
account for developing WBLMs used in this research.
Many researchers also suggest consideration of constructivist learning theory,
which emphasizes the importance of active learning. Sweller [16] emphasizes the
importance of student’s “cognitive” activity (e.g. resolving tasks), instead of
“behavioural” activity (e.g. clicking the mouse). Students’ cognitive activity is easier
 Comparing Learning Results of Web Based and Traditional Learning Students 377

to achieve using interactive tasks and smart questions for knowledge evaluation.
Many researchers underline the importance of meaningful feedback, which can also
be implemented automatically via internet or LMS system [2-6].

3 Methodology
Learning Materials. In this research we examined learning from TLMs and learning
from WBLMs delivered via internet and LMS system eCampus.
WBLMs used in our research are designed according to described recomendations.
They consist of printed and narrated text, pictures and animations. Many learning
pages have tasks with automatic feedback. After each chapter, students can self-assess
gained knowledge by solving tests with automatic feedback. Interactivity and
dynamic graphics are main differences between observed TLMs and WBLMs. In
TLMs static graphics and text is used instead of dynamic graphic elements. However,
both materials have some similarities such as the same subject of study and the same
didactical approach. The Kolb's learning styles model is considered and experiential
learning theory is implemented [7].

Hypotheses. We set the null hypothesis that here are no significant differences in
learning results (i.e. learning time, gained knowledge) between students using
different presentation modes (i.e. WBLMs, TLMs).

Participants and Procedure. Students of advanced ICT course participated in the

experiment. They were part time students at the school of economics and business
assistance.
In the beginning of the course we tested participants to measure their previous
knowledge about using Word and Excel. We used a questionnaire that was previously
verified with ITEM analysis. With the pre-test we measured knowledge and
understanding according to Bloom’s taxonomy. After the learning session we tested
participants again to find out how much knowledge they gained. We measured the
application of knowledge according to the Bloom’s taxonomy.
As students have different learning preferences, we did not force them to learn
from particular type of materials. Before the experiment we asked students to choose
the type of the learning material (i.e. WBLM, TLM) and to measure and record
learning time. In both learning modes students used computers to solve tasks in Excel
and Word. Before data analysis we checked in the LMS system if students who chose
TLMs learnt also from WBLMs. After excluding students who learnt from both types
of materials, we got the sample of 225 students that participated in the experiment.

Measurements. The measurement instruments were pre-knowledge test, knowledge

test and the recording of learning time made by the students. Independent variables
were code of participant, gender, age, and type of learning material. Dependent
variables were learning time; the pre-test result, the test result, and gained knowledge,
calculated as a difference between the test result and the pre-test result.
We used SPSS to make statistical analysis. In addition to descriptive statistics we
performed one-way ANOVA test to check the differences among observed groups.
378 J.L. Bele and J. Rugelj

4 Results and Interpretation

Participants. Among 225 participants were 94 males (42%) and 131 females (58%).
They were from 19 to 53 years old with the mean age of 32. 127 (56%) students chose
WBLMs and 98 chose TLMs.

Learning Time. First we tested if there are significant differences in learning times
between learners using TLMs and learners using WBLMs. 3 of 225 students did not
measure their learning times. Table 1 shows the descriptive statistics.

Table 1. Descriptive statistics – learning in hours

Learning material N Mean St. Dev. Min Max

Web based 126 47.63 19.97 10 100
Traditional 96 43.48 16.31 13 90
Total 222 45.83 18.55 10 100

Learning times were rather dispersed: from 10 to 100 hours. On average,

participants learning from TLMs learnt less time than the other group. Participants
that learnt from WBLMs spent 9.5% more time than participants learnt from TLMs.
We performed the Kolmogorov-Smirnov test to check if the data are normally
distributed. The learning time of web based learner and the learning time of
traditional learners were both significantly normal. Therefore, we could apply the
one-way ANOVA test. First we checked the homogeneity of variances with the
Leven’s statistics and we got F(1, 220) = 5.61, p<0.05. As Levene’s statistics had
significance p<0.05, we could not accept the homogeneity of variances assumption.
After carrying out the ANOVA procedure to find out if there are statistically
significant differences among the experimental groups, we used robust tests Brown-
Forsyth’s and Welch’s statistics. Performing one way ANOVA test, we got following
result: F(1, 220) = 2.746, p>0.05. Robust tests confirmed that the differences among
groups are not significantly different.
Although learners using TLMs on average spent less time than those who learnt
from the WBLMs, the differences among the groups are not statistically significant.
We expected that learning from the WBLM would be more time efficient than
learning from TLMs. It seems that running the animations, answering to online
questions, solving online tasks, communicating via LMS tools and navigating LMS
system took certain amount of time. Learners from TLMs read the content at their
own pace, probably too quickly. Consequently, as we will see in the following
section, they gained less knowledge than learners from WBLMs.

Gained Knowledge. We tested if there are significant differences in gained

knowledge between learners using TLMs and learners using WBLMs.
As participants had different previous knowledge, we decided to measure learning
results with the difference between the results of the test (i.e. course exam) and the
pre-test. We called this variable gained knowledge. Participants could achieve grades
5 to 10 on both tests. The next table (Table 2) shows that learners from WBLMs
achieved better results. Their mean of gained knowledge was higher.
 Comparing Learning Results of Web Based and Traditional Learning Students 379

Table 2. Descriptive statistics for gained knowledge

Learning material N Mean St. Dev. Min Max

Web based 127 3.17 1.43 0 5
Traditional 98 2.57 1.39 0 5
Total 225 2.91 1.44 0 5

Before running the ANOVA procedure, we tested the assumption of normality and
the assumption of homogeneity of variances. Based on the Kolmogorov-Smirnov test
we found out that the data distribution in each group is not significantly different from
normal distribution. Then we checked the homogeneity of variances with Levene’s
statistics and got the result F(1, 223) = 0.07, p>0.05. As Levene statistics had
significance p>0.05, we could accept the homogeneity of variances assumption and
carried on the ANOVA to find out if there are statistically significant differences
among the experimental groups. Obtaining one way ANOVA test we got following
result: F(1, 223) = 10.012, p<0.01.
Based on the descriptive statistics (Table 2) we conclude that students learning
from WBLMs gain more knowledge than students learning from TLMs. The results of
one-way ANOVA procedure confirm that differences among groups are significant.

5 Conclusions
The main question of our research was what kind of learning material, web based or
traditional, provided better learning results measured in learning time or in gained
knowledge.
WBLMs used in the experiment were properly designed to avoid cognitive
overload. The main differences between both types of materials were in interactivity
(e.g. online tasks and tests with automatic feedback were implemented in WBLMs)
and in the type of multimedia elements (e.g. animations and simulations were
implemented in WBLMs). The instructional design in WBLMs encouraged students
to be cognitively active. Learning goals were the same for both learning materials and
in both cases problem based learning was used as didactical method.
The experimental research showed that learners from WBLM spent a bit more time
for learning than learners from TLMs. Although on average learners using TLMs
spent less time than those learning from the WBLMs, the difference between the
groups is not statistically significant. Therefore we could not conclude that learning
from TLMs is more time efficient. We also found that learners from WBLMs
achieved better results as regards gained knowledge. The difference between observed
two groups, were significant. We can conclude that students learn more if they learn
from web based learning materials.

References
1. Achtemeier, S.D., Morris, L.V., Finnegan, C.L.: Considerations for developing evaluations
of online courses. Journal of Asychronous Learning Networks (JALN) 7(1), 1–13 (2003)
2. Ardito, C., et al.: An Approach to Usability Evaluation of e-Learning Applications.
Universal access in the information society 4(3), 270–283 (2006)
 380 J.L. Bele and J. Rugelj

3. Barron, J.: Top ten secrets of effective e-learning. Industrial and commercial
training 38(7), 360–364 (2006)
4. Chickering, A.W., Ehrmann, S.C.: Implementing the seven principles: Technology as
lever. AAHE Bulletin 49 (1996)
5. Downes, S.: E-learning 2.0. eLearn Magazine 10 (2005),
http://www.elearnmag.org/subpage.cfm?section=articles&
article=29-1
6. Ko, S., Rossen, S.: Teaching Online: A Practical Guide. Houghton Mifflin Company, New
York (2004)
7. Kolb, D.: Experiential learning: Experience as the source of learning and development.
Prentice-Hall, Englewood Cliffs (1984)
8. Lapuh Bele, J., Rugelj, J.: Efficient learning from multimedia web-based learning contents.
In: MÉNdez-Vilas, A. (ed.) Current developments in technology-assisted education, vol. 1,
pp. 396–400 (2006)
9. Lapuh Bele, J., Rugelj, J.: Comparing efficiency of web based learning contents on
different media. Int. j. emerg. technol. learn. 4(3), 31–35 (2009)
10. Mayer, R.E.: Multimedia learning. Cambridge University Press, New York (2001)
11. Mayer, R.E.: Cognitive Theory of Multimedia Learning. In: Mayer, R. (ed.) The
Cambridge handbook of Multimedia learning, pp. 31–48. Cambridge University Press,
New York (2005)
12. Miller, G.A.: The Magical Number Seven, Plus or Minus Two: Some Limits on Our
Capacity for Processing Information. The Psychological Review 63, 81–97 (1956)
13. Paivio, A.: Mental representations: A dual coding approach. Oxford University Press,
Oxford (1986)
14. Piaget, J.: The Origins of Intelligence in Children. International University Press, New
York (1952), http://www.piaget.org/free-books.html
15. Squires, D., Preece, J.: Predicting Quality in Educational Software: Evaluating for
Learning. Usability and the Synergy between them. Interacting with Computers 11, 467–
483 (1999)
16. Sweller, J.: Implications of Cognitive Load Theory for Multimedia Learning. In: Mayer, R.
(ed.) The Cambridge handbook of Multimedia learning, pp. 19–30. Cambridge University
Press, New York (2005)
17. Vygotsky, L.S.: Mind and society: The development of higher mental processes. Harvard
University Press, Cambridge (1978)

View publication stats