Journal of Theoretical and Applied Information Technology

© 2005 - 2010 JATIT & LLS. All rights reserved.

www.jatit.org 
 
THE SUCCESS OF LEARNING MANAGEMENT SYSTEM
AMONG DISTANCE LEARNERS IN MALAYSIAN
UNIVERSITIES
1
IBRAHIM ABOOD ALMARASHDEH, 2NORAIDAH SAHARI, 3NOR AZAN MAT ZIN,
4
MUTASEM ALSMADI
1,4
Student, School of Information Technology
2
Senior Lecturer, School of Information Technology
3
Assoc. Prof., School of Information Technology

Faculty of Information Science and Technology, National University of Malaysia

Selangor, MALAYSIA
E-mail: [email protected]

ABSTRACT

Continuing formal education is essential for distance learners to improve their learning skills and
knowledge to meet the challenge career in modern competitive world. This study examines the success
factors that influence learners' use of the Learning Management System (LMS) and tests the applicability of
the propose model, in the context of distance learning practices in higher education. A survey was
conducted to higher education learners who involved in distance learning instruction. This study used a set
of questionnaire which was adapted from the literatures to examine three groups of dimensions, system
design (system quality, service quality, information quality, usefulness and ease of use), system usage
(system use, behavioural intention to use and user satisfaction) and System Outcome (net benefit). Using
data from a survey to distance learning students (N=425), a path analysis revealed that system design has a
significant influence on user satisfaction and intention to use of the LMS which directly affect the use of
the system. Consequently user satisfaction and system used show strong impact to the net benefit.

Keywords: Information System Success, Technology Acceptance Model, Educational Technology Model,
System Design. System Usage, System Impact.

Online Distance Education (ODE) is still young

1. INTRODUCTION
in Malaysia and needs better requirements
understanding and more system enhancement. For
The rapid growth of the internet use has not instance, distance education strategies and
only generated a transformed interest in the role delivery modes were perceived by the distance
of new information and communication learning as not adapted to meet the needs of the
technologies (ICT) in higher education and larger intake of learners, and the learners'
learning (Dutton & Loader, 2002), but it has also diversity. This causes the issue of learning
affected the ways people teach and learn support in distance learning that should be looked
(DeLacey & Leonard, 2002; Radcliffe, 2002). At into (Dzakaria, 2003).
the same time, there has been growing concern
over the possible decline of traditional teaching According to Wahlstedt and Honkaranta (2007),
and learning practices and learning institutions, as the learning management systems (LMS) consist
distance education and virtual universities pedagogical devices, human interactions, learning
become practicable alternative platforms for contents and assessment supporting and
higher education. Students, instructors, and advancing traditional learning in school or in
administrators have continued to employ the higher education. LMS must satisfy the needs of
internet for their practices, and e-learning have the users: the students and the instructors. LMS
become a key item in educational agendas. was largely used as a useful content distribution
system. Instructors can use the LMS for

 
80
 Journal of Theoretical and Applied Information Technology

© 2005 - 2010 JATIT & LLS. All rights reserved.

www.jatit.org 
 
distributing courses and interact with students in determinants that lead to the actual usage of an
distance. There seems to be a gap between the information system (Davis, 1989; Wahlstedt and
reality and the many advanced teaching tools that Honkaranta, 2007). In this study, perceived
are provided in LMS, such as multimedia usefulness is defined as the degree to which an
materials, which were considered as possible individual believes that using a LMS would
means for enhancing teaching, but not utilized. To enhance his or her learning performance, while
bridge this gap LMS system should be build to be perceived ease of use is defined as the degree an
more adaptive and customizable. This is to individual believes that using LMS would be less
support teachers or instructors with different effort to learn how to use the system. Between the
computer level skills. According to Ramayah two, perceived ease of use has a direct effect on
(2005) many colleges are using LMS for e- both perceived usefulness and behavioural
learning courses and instruction, but many intention to use, which ultimately influence the
instructors restrict themselves to uploading course actual system use.
materials (such as syllabus, reading materials and
lecture slide) to the course web site and never use A number of studies have adapted TAM
the interactive features (chat, discussion forum, model and acceptance items to study student
email, messages). Some instructors may use the acceptance of new technologies (Pearson and
discussion board to generate class discussion Young, 2002; Selim, 2003; Ong et al., 2004; and
among students and themselves but the lack of Drennan et al., 2005). In a study of student
immediate feedback with the discussion board in acceptance of course websites, Selim (2003)
LMS has discouraged users to utilise them. found that perceived usefulness and perceived
Although many interactive features are available usability proved to be important factors for the
in the LMS, its capacity for use may still be acceptance and use of websites, as these two
limited because of its demand on the constructs accounted for 83% of the variance in
commitments from both instructor and students the acceptance and use. Ong et al. (2004) stated
during a specific time frame. An effective that computer self-efficacy appeared to be a
implementation of an LMS needs to determine the significant determinant of perceived usefulness,
critical features of the system and their perceived ease of use and perceived credibility in
implementation (Ravi and Pamela, 2004). using e-learning system in high-technology
companies. In their study, high computer self-
The aim of this research is to develop an effective efficacy showed more positive usefulness and
LMS for distance learning instruction that is ease of use beliefs.
usable and fully utilized by students, instructors
and administrators. Developing a good LMS is Pearson and Young (2002), in their research
not worthwhile if the acceptance of the summarizes the study of using the applicability of
technology is still low and the success factors are TAM in Malaysia and the conclusion of their
still unclear. In priory, the acceptance and success research indicates that all those researches have
factors of the LMS usage among stakeholders are supported the model in predicting and explaining
studied and led to the construction of educational the use or the intention to use of a particular
technology model (ETM). Therefore, the focus of technology. There is a general consensus that
this paper is to develop and verify the goodness of perceived usefulness has significantly related to
an ETM that can be used as a guideline to technology use. A person who finds a particular
develop or evaluate a LMS. technology useful will use more of the technology
as compared to another person who finds it not
2. TECHNOLOGY ACCEPTANCE useful. On the other hand, many researches find
MODEL (TAM) perceived ease of use is useful in predicting
perceived usefulness and not significantly relate
Understanding why people accept or reject new to usage or intention to use. Researchers have
information system (IS) was one of the biggest explored various external variables that can act as
challenges in the research of new technologies. previous circumstances (for example attitude
This study used Technology Acceptance Model towards computer) to these perceived usefulness
(TAM) that identifies computer technology usage and perceived ease of use in attempts to improve
behaviour by applying perceived usefulness (PU) the predictive power of the expanded TAM. Some
and perceived ease of use (PEU) as key of the results shown to be significant and thus it
can be concluded that particularly in the
 
81
 Journal of Theoretical and Applied Information Technology

© 2005 - 2010 JATIT & LLS. All rights reserved.

www.jatit.org 
 
Malaysian environment, perceived usefulness is model to measure knowledge management system
the driver to any technology acceptance and this (KMS). Using path analysis, they confirmed the
has to be tackled to enhance usage among KMS success model with five dimensions
individuals (Pearson and Young, 2002). Those
namely, system quality, knowledge/information
empirical studies using these models, concluded
that TAM (Figure 1) is the most influential model quality, perceived KMS benefit, user satisfaction
in predicting the acceptance of technology. and System Use. Wang (2007) had employed the
DL&ML success model in assessing the success
perceived of web-based learning system from the
usefulness
organizational approach.
external
behavioural actual
variables
intention system use
System
perceived
ease of use quality Intention to
use/Use
Information
Informatio
Figure 1: The Technology Acceptance Model Net Benefits
quality
n quality
User
The TAM recommends that information system
Service satisfaction
(IS) usage is determined by behavioural intention
quality
to used (BI), which is viewed as being mutually
determined by the user's attitude toward using,
perceived ease of use (PEU) and perceived
Figure 2: DL&ML IS Success Model
usefulness (PU) of the system (Davis et al., 1989).
Lee-Post (2009) defined the success of e-learning
3. DELONE & MCLEAN IS SUCCESS as a multi-faceted construct that can be assessed
MODEL along six dimensions which take place in three
stages. The first stage is to manage system design
The most popular IS success model was success by maximizing the three quality
proposed by DeLone and McLean (DL&ML) in dimensions which are system, information and
service quality. The second stage is to accomplish
1990 by highlighting six intercorrelated
system delivery success by optimizing the
dimensions. The dimensions are System Quality, intention to use dimension. The final stage is to
Information Quality, Use, User Satisfaction, obtain system outcome success by maximizing
Individual Impact and Organizational Impact. net benefits and user satisfaction dimensions
This model were further modified as IS become (Lee-Post 2009). Many other researchers have
more complicated by including Service Quality as adapted the success dimensions by DL&MS in
several types of IS (Wang and Liao, 2007; Yen
a new function. Individual Impact dimension was
and Lu, 2008; Lee and Chung, 2009)
removed and Intention to Use dimension which
was related to Use was added. In addition, In conclusion, the DL&MS success
Organizational Impact was replaced by Net dimensions have played important role in assess
Benefit (Delone & McLean, 1992; Delone & many IS success. Each success dimension is
McLean, 2003). The modified model is shown in quantified as a single numeric measure by
Figure 2. Since then many empirical studies have combining the ratings of its set of attributed
factors through an instrument. The overall success
been conducted by researchers to validate the
of IS can then be evaluated for every dimension.
model by applying it in different IS before it’s Any success dimension with lower score signifies
dimension could serve as appropriate measure a lack in that area and efforts can be committed
(Wu and Wang, 2006; Wang et al., 2007 and accordingly to fix the insufficiency.
Lee-Post and Holsapple, 2009).

Based on DL&ML modified model, Wu and

Wang (2006) reconstructed and validated the

 
82
 Journal of Theoretical and Applied Information Technology

© 2005 - 2010 JATIT & LLS. All rights reserved.

www.jatit.org 
 
4. CONSTRUCTION OF EDUCATIONAL validation of the model (Rai, Lang and Welker
TECHNOLOGY MODEL (ETM) 2002).

The research model was constructed as a result This study demonstrates the value of assessing
of a comparison between the TAM model and the DL&MS success from an information systems
DL&ML IS success model. The TAM model perspective in general, and specifically from the
emphasizes on measuring the user behaviour educational Technology perspectives.
using the perceived usefulness and ease of use as Educational technology model (ETM) is
the main variables in the usage dimension and constructed based on the development and
that would measure the actual use of a system. continual improvement in designing, developing
Based on the literature review, TAM does not and delivering the needed service to the distance
measure the overall system quality even its have learning stakeholders. If we want to evaluate the
strong effect in the user behaviour. According to success of any program or system used by the
Dishaw & Strong (1998), TAM has a weakness of people, we must evaluate the system in all phase
not incorporating the technology and task of designing process until it is used by users. In
characteristics in measuring acceptance. On the other words the overall success of a distance
other hand the DL&ML success model have learning technology initiative is depending on the
significant relation with user satisfaction, system success at each of the three phases of LMS
use, intention to use and net benefit but the IS development, namely, system or technology
success model did not mention the importance of design, usage and system impact. The following
capacity of use even this variable is very are brief descriptions of each dimension and
important to determine the direction upon which element of the Educational Technology Model
the success of the IS. It does not give a clear (ETM) which is shown in Figure 3.
interest in ease of use and the usefulness even the
previous studies find significant effect on user 1. The success of system design stage is
satisfaction and the acceptance of new technology depending on the success of five dimensions:
(Lynn, 1999; Alhelalat et al. 2008; Mäntymäki, system quality; information quality; service
2009; Li 2009; Ong et al., 2009). According to quality; perceived ease of use; and perceived
Urbach et.al. (2009), some researchers have found usefulness.
that the DL&ML model is incomplete and have 2. The success of the usage stage is affected by
suggested that extra dimensions should be system design stage and is depending on the
included in the model. Some of them have success of the system use, behavioral intention
constructed alternative models of success (Seddon to use and user satisfaction factors.
& Kiew; Seddon 1997, 1994). Despite of all 3. The success of the system outcome stage is
comments and weaknesses, DL&ML model still depending on the attainment of success of the
become a dominant model for measuring the usage stage and net benefit.
success. Other researchers have applied the model
in their researches and some have focused on the

USAGE
SYSTEM DESIGN
Intention to Use
System Quality

Information Quality OUTCOME

System Use
Service Quality Net Benefit
User
Perceive ease of use satisfaction

Perceived Usefulness

Figure 3: Proposed Educational Technology Success Model

 
83
 Journal of Theoretical and Applied Information Technology

© 2005 - 2010 JATIT & LLS. All rights reserved.

www.jatit.org 
 
The following are descriptions of the sub- using LMS, students would intend to use the
characteristic of the ETM measures: system if they feel that the system might make
- System quality (SyQ) can be defined as the their learning activities easier and smooth with less
stability, reliability and suitability of the hardware effort.
and software that provides the information that is - Behavioral Intention to Use (BI) in TAM
required. It is a broad concept that has been hypothesizes that the behavioral intention to use is
measured in different ways. The studies on System the single best predictor of actual system usage and
Quality are often associated with system well supported by past TAM research (Davis &
performance. System quality in a LMS, measures Venkatesh, 1996; Morris & Dillon, 1997). In the
the intrinsic features of the technology including LMS usage, this factor implies that students
system performance and user interface. Examples develop this behavior if the system design is
of system quality measures in LMS are response adequate and their satisfaction increase.
time, usability, availability, reliability, - User satisfaction (SAT) is frequently used to
completeness, system flexibility, and security. measure a system success. It is defined as the
- Information Quality (InQ) are concerned with overall evaluation of user’s experience in using a
information system including students records, system and the potential impact of the system.
reports, learning content and other information User satisfaction is related to user perception and
regarding learning activities. Most information attitudes towards the use of the system which are
quality measures are subjective, as they are derived influenced by user’s characteristics and
from the user perspectives. Criteria that can be experiences.
used for LMS in distance learning quality are - The actual system use is a measure of IS success. It
information completeness, accuracy, legibility, refers to voluntary instead of mandatory use and
timeliness, availability, relevancy, consistency and also relates to the person who uses it, their levels
reliability (DeLone and McLean, 2003 Lee-Post, of use, training, knowledge, belief, expectation and
2009). acceptance or resistance (DeLone and McLean,
- Service Quality (SvQ) is concerned with the 2003, Park et al., 2007). In using the LMS, this
overall support delivered by the service provider of factor measures the frequency and breadth of
distance learning or technology, regardless of distance learning technology inquiries and
whether the service is delivered by the internal functions. It measures the use of instructional
department of distance learning organization or features, interactive features, administrative
outsource to external providers. Service quality can features, visual features, support features and
be measured through access to technical support, technology features to assess the success of LMS.
quick responsiveness, assurance, empathy and - System impact or net benefit (NB) implies how the
follow up services (DeLone and McLean, 2003 system can benefit a single user, a group of users,
Lee-Post, 2009). an organization or an entire industry. It is
- Perceived Ease of Use (PEU) refers to the degree associated with users’ job performance, as well as
to which the user expects the target system is free change in work activity and improved productivity.
of effort (Davis et al., 1989). TAM hypothesizes Thus, individual Net Benefits can be assessed
that PEU has a significant positive impact on the using job effects, efficiency, effectiveness,
behavioral intention to use. In using LMS this decision quality, and error reduction (DeLone and
factor suggests that the easier the system is to McLean, 200, Lee-Post, 2009). In assessing LMS,
interact with, the greater the user’s ability will be net benefits capture the balance of positive and
to carry out the sequences of behavior needed to negative impacts on the behavior of users, which
use the system. If the LMS is considered to be easy includes system developers, administrator,
to use, it is more likely to be accepted by the instructor and student, or the entire distance
students. learning stakeholders.
- Perceived Usefulness (PU) comes from the
definition of the word useful which is “capable of
5. RESEARCH METHODOLOGY
being used advantageously.” Therefore, a system
that scores high in perceived usefulness will have a
positive user performance impact. (Davis et al., The purpose of this study is to validate the
1989) defines usefulness as “the degree to which a proposed success ETM by determining the success
person believes that using a particular system level of Malaysian students toward LMS usage. A
would enhance his or her job performance”. In survey is carried out on a sample of about 425

 
84
 Journal of Theoretical and Applied Information Technology

© 2005 - 2010 JATIT & LLS. All rights reserved.

www.jatit.org 
 
students from four Malaysian universities that offered is well-constructed and a coefficient in the 0.50 to
distance learning program and use LMS platform 0.60 indicates that a scale is less well-constructed.
(Open University Malaysia (OUM), Universiti Tun When executing a reliability test for internal item
Abdul Razak (UNITAR), University Utara Malaysia consistency, the researcher first ran the test for all the
(UUM), Universiti Sains Malaysia (USM) and items except those asking for participants’
Universiti Putra Malaysia (UPM)). The researchers demographic information. A coefficient alpha value
have used survey method, and questionnaires are of 0.963 is obtained, indicating a high reliability for
distributed to all students registered in distance 94 items.
learning department from those four Malaysian
universities. The questionnaire is developed based on
6. DEMOGRAPHIC INFORMATION
issues raised by past researchers and studies, as well
as concerns mentioned in the literature related to
ETM dimensions. In order to see if the result of this Table 2 presents summary of the background
research is consistent with past literatures, the informations. It can be seen that 49.4% of the
researcher tried to match specific items with LMS respondents are female whereas 50.6% of them are
issues. Items are measure using 5 point-Likert-scales male. Majority of the respondents aged between 20 to
ranging from strongly disagree to strongly agree. The 40 years old. Most of the respondents have used
respondents took approximately 15 minutes to computer for more than 1 year and 23.3% have used
complete the questionnaire. Respondents were computer for more than 10 years. All of them have
assured that their individual responses would be kept experience using LMS for more than one year.
confidential. The researchers use Statistical Package
for the Social Sciences (SPSS) 17.0 to analyze the Table 2 . Demographic Information
data and use AMOS 18 for hypotheses testing and to Gender Frequency Percent
determine the model fit through Structural Equation Male 215 50.6
Model (SEM). Female 210 49.4
Age Frequency Percent
The validity of this instrument was assessed Less than 20 45 10.6
through content validity, pre-testing and revision of 20 - 24 120 28.2
questionnaire. The content and construction of the 25 - 29 121 28.5
questionnaire were also evaluated by two judges. The 30 -40 113 28.6
judges are lecturers in the field of Instructional 41 - 50 14 3.3
Technology, in the Faculty of Information over 50 12 2.8
Technology and Science at University Kebangsaan Computer Usage Frequency Percent
Malaysia. The researchers then conduct reliability less than 1 year 10 2.4
test for each construct to assess the goodness and 1 - 3 years 94 22.1
reliability of the measures. Table 1 shows high 3 - 7 years 158 37.2
reliability of each construct as mentioned by Sekaran 7 - 10 years 64 15.1
(2003).
more than 10 years 99 23.3
LMS Usage Frequency Percent
Table 1 : Cronbach's Alpha
less than 1 year 114 26.8
Item Reliability
1 - 3 years 148 34.8
System quality 0.944 3 - 7 years 128 30.1
Information quality 0.919 7 - 10 years 19 4.5
Service quality 0.890 more than 10 years 16 3.8
Perceived usefulness 0.914 Total 425 100
Perceived ease of use 0.892
User satisfaction 0.845
7. MEASURE OF FIT
System use 0.954
Intention to use 0.822 The proposed research model incorporates
Net benefit 0.803 aspects of technology acceptance model and IS
success model. The model presents five observed or
According to Sekaran (2003), a reliability exogenous variables (PU, SYQ, INQ, SVQ, PEU)
coefficient, alpha of 0.80 to 0.90 indicates that a scale

 
85
 Journal of Theoretical and Applied Information Technology

© 2005 - 2010 JATIT & LLS. All rights reserved.

www.jatit.org 
 
and four endogenous variables (SAT, USE, NB, BI). model is validated using confirmatory factor analysis
Endogenous variables (or dependent variables), (CFA).
depend on other variables, and have single-headed
arrows pointing to them. Exogenous variables (or The CFA is carried out using SEM software AMOS
independent variables), do not depend on other 18. The objective of the CFA is to construct a
variables, and do not have single headed arrows structural model which aligns the tested measures to
pointing to them (Arbuckle, 2005). The model after the specific constructs, by constraining the variance
testing and modification is called the Educational of each measure to the specific latent construct it
Technology Model (ETM) throughout the rest of this should represent. In addition to assess the degree to
research. which each measure contributes to its latent
construct, the CFA also tests the separation between
Before analysing the structural model, it is constructs by evaluating the fit in the overall model.
necessary to understand how to evaluate the models. There are four groups of fit measures. The fit
Fit measures are grouped into various types and each measures within each group give the same rank of
type has its specific capability in model evaluation, ordering of models (Arbuckle 2005). The first group
such as measures of parsimony, minimum sample is RMSEA and TLI, the second groups is CFI, the
discrepancy function, measures based on the third group is CMIN and NFI, and the fourth group is
population discrepancy, comparison to a baseline GFI, and AGFI. Among the many measures of fit,
model, and a goodness of fit index (GFI) and other five popular measures are: Chi-square, normed chi-
related measures (Sahari et al.,2009; Arbuckle 1999, square (χ2 /df ), goodness of fit index (GFI), Tucker-
2005; Byrne 2001, 2006; Holmes-Smith 2000; Lewis Index (TLI) and Root Mean-Square Error of
MacCallum 1990; Steiger 1990). Nevertheless, Approximation (RMSEA) (Holmes-Smith 2000).
Arbuckle (2005) mentions that model evaluations is Table 3 shows the measures of fit. The overall fit for
one of the most difficult and unsettle issues related to the model is very good with Chi-sq/df is 2.80, at 11
structural equation modelling. In this research the df and is significant and P value 0.001.

Table3 Summary of the Fit Measures Used in this study

Fit Measures Standards Fit Model Fit

CMIN/Df A value close to 1 and not exceeding 3 indicates a good fit. 2.800

P Value A p value greater than 0.05 indicates an acceptable fit. 0.022

RMSEA A value should not greater than 0.1 0.490

TLI a value close to 1 indicates a very good fit. 0.986

CFI a value close to 1 indicates a very good fit. 0.996

GFI A value always less than or equal to 1 indicates a perfect fit. 0.989

AGFI A value always less than or equal to1 indicates a perfect fit. 0.953

RMR RMR of less than .05 indicates a perfect fit. 0.016

IFI IFI values close to 1 indicate a very good fit. 0.996

RFI RFI values close to 1 indicate a very good fit 0.973

NFI TLI values close to 1 indicate a very good fit. 0.992

8. HYPOTHESES TESTING
Structural equation modelling is well suited to test a
group of hypotheses simultaneously in the form of a

 
86
 Journal of Theoretical and Applied Information Technology

© 2005 - 2010 JATIT & LLS. All rights reserved.

www.jatit.org 
 
model with significant level 0.05. Nevertheless, it
helps to reveal these hypotheses and to consider each
one individually.

Table4 : Educational Technology Model (ETM) Hypotheses

No Hypotheses Reference Effect β Sig

1. PU SAT Matti,2009; Alhelalat et al. 2008 Supported 0.254 ***

2. SvQ SAT DeLone & McLean, 2003 Supported 0.242 ***
3. InQ SAT DeLone & McLean, 2003 Supported 0.211 ***
4. SyQ SAT DeLone & McLean, 2003 Supported 0.236 ***
5. PEU SAT Matti,2009; Alhelalat et al. 2008 Supported 0.084 .036
6. PEU BI Davis, 1989; Supported 0.102 .019
7. PU BI Davis, 1989; Supported 0.382 ***
8. SvQ BI DeLone & McLean, 2003 Supported 0.126 .009
9. InQ BI DeLone & McLean, 2003 Not Supported -0.169 ***
10. SyQ BI DeLone & McLean, 2003 Supported 0.149 ***
11. SAT BI DeLone & McLean, 2003 Supported 0.339 ***
12. BI USE Davis, 1989; DeLone & McLean, 2003 Supported 0.145 .045
13. SAT USE DeLone & McLean, 2003 Supported 0.150 .038
14. SAT NB DeLone & McLean, 2003 Supported 0.479 ***
15. USE NB DeLone & McLean, 2003 Supported 0.135 .002
Modification Indices of AMOS 18 suggests the
From Table 4, the hypotheses that describe the independent variable to be correlated, Table 5 below
relationship between independent variables (SyQ, shows the correlation result of the independent
InQ, SvQ, PU, PEU) and the dependent variable variables.
(SAT, BI) are significant except the relationship This study found that there is a good significant
between InQ and BI. It shows that PU, SyQ, SvQ, correlation between the DL&ML quality dimensions
InQ, PEU have significantly influence SAT, and PU, (system, information and service quality) and TAM
SyQ, SvQ, PEU have significantly influence BI. It independent variables (Perceived ease of use and
also shows that SAT has significant relation with BI, perceived usefulness). This finding confirm further
USE and NB. Nevertheless, BI has significant
relation with USE, and USE significantly influences research recommended by Ultan et al. (2006) which
NB. There are enough evidence associated with suggested to further explore the interdependency of
causal relationship between independent variables the factors of quality and success and the correlations
and dependent variables in the model which between factors. A correlation analysis in Table 5
facilitates the effort to understand the dynamics shows positive correlation between independent
between the constructs. The analyses showed that PU variables in ETM (i.e. system quality, information
and PEU are correlated as predicted in theoretical quality, service quality, perceived usefulness and
work. The reason for such correlation in the perceived ease of used). Each construct shares greater
relationships is likely due to the advanced stage of variance with its own set of indicators than with the
adoption and ongoing use of technology. Thus, the

 
87
 Journal of Theoretical and Applied Information Technology

© 2005 - 2010 JATIT & LLS. All rights reserved.

www.jatit.org 
 
other constructs representing a different set of satisfaction have positive influence on the net benefit.
indicators. This result is in line with study by Sharkey et. al
(2006) that suggested the need of more research to
Table 5: Correlations between Independent explore the correlations between the independent
Variables variables of user satisfaction and intention to use.

Estimate In the LMS context, we encounter that the

SvQ <--> SyQ 0.703 attitude of using the system is affected by beliefs
about system design which then affected the net
PU <--> SvQ 0.632
benefit. Users’ belief that good or bad system design
InQ <--> PEU 0.470 form their attitude or interest to use the system,
SyQ <--> PEU 0.491 which eventually give positive or negative impact on
PU <--> InQ 0.598 the behaviour. Users also belief that the behavioural
SvQ <--> PEU 0.558 intention to use is developed by the system design
and consequently, to affect user which also shaped by
SvQ <--> InQ 0.755
the system design.
PU <--> PEU 0.791
PU <--> SyQ 0.559
InQ <--> SyQ 0.621
9. DISCUSSION
10. CONCLUSION
 
This study have constructed and verified an
educational technology success model. Using LMS as The evaluation of learning management system
a theoretical framework, we construct five measures (LMS) is crucial to ensure their effective
in system design quality, three measures in system implementation and positive impact on distance
usage and net benefit in system outcome as learning delivery. Many institutions find it quite easy
educational technology success dimensions. The to start with a commercial LMS, but they encounter
empirical results provide appropriate support for many problems such as, pricing, linguistic,
assessment tools, and suitability to target users. A
model. Almost all hypothesized relationships were review of LMS evaluation studies was carried out
found to be significant. which indicated that the improvement of current
system is required. In order to satisfy this
The current LMS analysis considers the success requirement, a new framework is introduced for the
dimensions as system, service and information evaluation of Information Systems (IS) in educational
quality, usefulness and ease of use, behavioural settings. The proposed Educational Technology
intention to use, user satisfaction and system use to Model (ETM) framework with three characteristics
influence the net benefit. Thus, the developed ETM (System design, usage behaviour and system
considers the interrelationships and causal effects outcome) was developed after having critically
amongst the main parts of the measures: system appraised the existing findings of IS success and
design (system quality, information quality, service acceptance studies. ETM is constructed based on
quality, ease of use and usefulness); usage (system previous models of IS evaluation, in particular, the IS
use, user satisfaction and behavioural intention to Success Model and the Technology acceptance
use) and system outcome (net benefits). The model (TAM). The researchers incorporates the
empirical results of the study indicate that system concept of fit between technology design quality
quality, information quality, service quality, (perceived usefulness, perceived ease of use, system
perceived usefulness and perceived ease of use have quality, information quality and service quality);
significantly positive influence to user satisfaction. Usage (behavioural intention to use, user satisfaction
Except for information quality, all other factors in and system use) and system outcome (net benefit).
system design have directly influence the behavioural The paper outlines the methodology of survey
intention to use. In addition, behavioural intention to approach using data from 425 distance learners
use and user satisfaction have direct effect on system (N=425), and by Structural Equation Modelling
use. Nevertheless, the system use and user (SEM) analysis. The main findings of the research

 
88
 Journal of Theoretical and Applied Information Technology

© 2005 - 2010 JATIT & LLS. All rights reserved.

www.jatit.org 
 
are the identification of the factors that determine the Journal of Personality and Social Psychology, 6,
use and success of educational technology system. 466-487
Based on these factors we formulate a measurement [5]. Alhelalat, J. Ineson, E. Jung, T. 2008. The
model to verify the relationships among TAM and IS Evaluation of Hotel Website Quality, Usability
success dimensions. and Benefits: Developing a Testing Model.
The empirical research is carried out in order to Proceedings of Euro-HCRIE Conference, Dubai,
evaluate the reliability, validity, and consistency of UAE, October 2008.
the developed measurement instrument. The [6]. Byrne, BM 2001, Structural Equation Modelling
theoretical model is developed and tested so as to with AMOS: Basic Concepts, Applications, and
highlight the variables that play an important role in Programming, Lawrence Erlbaum Associates,
determining the success of educational technology Inc., Mahwah, New Jersey, USA.
system. The study of the theoretical model suggests [7]. Byrne, BM, 2006, Structural Equation Modelling
some managerial implications. In particular, the with EQS: Basic concepts, applications, and
success of LMS depends on the quality of system programming, 2 edn, Lawrence Erlbaum
design (system quality, information quality, service Associates, Inc., Mahwah, New Jersey, USA.
quality, perceived ease of use and perceived [8]. Cao, M. Zhang, Q. & Seydel, J. (2005). B2C e-
usefulness). The usage dimension (behavioural commerce web site quality: an empirical
intention to use, user satisfaction and system use) Industrial Management Data Systems.Volume:
depends on the system design and affect the system 105, Issue: 5, Pages: 645-661
net benefit. The resultant model demonstrates the [9]. Li, C. 2009. Extending the technology
interrelationships amongst the main conceptual parts, acceptance model to investigate the factors
including the specific LMS features. It is proposed affecting the user satisfaction in internet banking.
that all of these factors affect the benefits and impact Business and Information . July 06-08
users and organisations. [10]. Davis, F. D. 1989. “Perceived usefulness,
Validation of measures requires the assessment perceived ease of use, and user acceptance of
of measurement properties over a variety of samples information technology”. MIS Quaterly, 13,
in similar contexts. Furthermore sample from 1989: 319-40.
different background should be gathered to confirm, [11]. DeLone, W.H., and McLean, E.R. 2003.
evaluate, or refine the model. This also can be done "The DeLone and McLean Model of Information
probably by investigating this model to different Systems Success: A Ten-Year Update," Journal
systems. of Management Information Systems (19:4),
Spring, pp 9-30.
ACKNOWLEDGMENT [12]. Dishaw, M., & Strong, D. “Extending the
TAM with Task-Technology Fit Constructs”,
The authors would like to acknowledge Universiti Information &n Management, (36:1), July 1998,
Kebangsaan Malaysia for partly funding this project pp. 9-21.
under the young researcher grant, UKM-GGPM-ICT- [13]. Dutton, W. H., & Loader, B. D.
116-2010 and thanks to all individuals and (2002). Digital Academe: New Media and
organizations who have contributed to this study. Institutions in Higher Education and Learning.
London: Taylor and Francis/Routledge.
REFERENCE [14]. Dzakira H and Idrus R. (2003) “Teacher-
Learner Interactions in Distance Education: A
[1]. Ajzen I, 1985, From Intentions to Actions: a Case of Two Malaysian Universities”.Turkish
Theory of Planned Behavior, (in J. Kuhl and J. Online Journal of Distance Education-TOJDE
Beckmann (Eds.), Action Control: From July 2003 ISSN 1302-6488 Volume:4 Number:3
Cognition to Behavior), Springer Verlag, New http://tojde.anadolu.edu.tr/tojde11/articles/idrus.
York, 11-39 htm.
[2]. Ajzen I, 1988, Attitudes, personality, and [15]. Holmes-Smith 2000, 'Introduction to
behavior, Addison-Wesley Pub, Reading, Mass. Structural Equation Modelling using AMOS 4.0,
[3]. Ajzen I, 1991, The theory of planned behavior, 5th International and 8th National Research
Organizational Behavior and Human Decision Conference on Quality and Innovation
Processes, 50(2), 179-211 Management, The Euro- Australian Cooperation
[4]. Ajzen I, Fishbein M, 1970, The prediction of Centre(EACC), Victoria.
behavior from attitudinal and normative beliefs,

 
89
 Journal of Theoretical and Applied Information Technology

© 2005 - 2010 JATIT & LLS. All rights reserved.

www.jatit.org 
 
[16]. Hu, P. J.-H. (2003). Evaluating 2005
Telemedicine Systems Success: A Revised fromhttp://ifets.ieee.org/periodical/vol_2_2002/r
Model, Proceedings of the 36th Hawaii adcliffe.html
International Conference on System Sciences [27]. Rai, A., Lang, S. S. and Welker, R. B.
(HICSS 03), January 6-9, Big Island, Hawaii. (2002). Assessing the Validity of IS Success
[17]. James L. Arbuckle ,(2007), AMOS 16 user Models: An Empirical Test and Theoretical
guide. Amos Development Corporation SPSS, Analysis, Information Systems Research, 13 (1),
Inc, http://amosdevelopment.com 50-69.
[18]. Lee-Post, A.and Holsapple, C. W. (2006). [28]. Sahari, N,. A.A.A. Ghani, H. Selamat and
“Defining, assessing, and promoting e-learning A.S.M. Yunus. 2009. Development and
success: An information systems perspective”, validation of mathematics courseware usefulness
Decision Sciences Journal of Innovative evaluation instrument for teachers. J.Applied Sci.
Education 4(1), pp 67-85. 9: 535-541.
[19]. Lynn bouchard ,(1999)developing and [29]. Seddon, P. B. (1997). A Respecification and
evaluating methods for user satisfaction Extension of the DeLone and McLean Model of
measurement in practice. journal of information IS Success, Information Systems Research, 8 (3),
technology Management, Volume X Numbers 3- 240-253.
4, 1999 [30]. Seddon, P. B. and Kiew, M.-Y. (1994). A
[20]. MacCallum, RC 1990, 'The need for Partial Test and Development of the DeLone and
alternative measures of fit in covariance structure McLean Model of IS Success, Proceedings of the
modelling', Multivariate Behavioural Research, 15th International Conference on Information
vol. 25, pp. 157-62. Systems (ICIS 94), December 14-17, Vancouver,
[21]. Mäntymäki, M 2009. Customer Loyalty in Canada.
Social Virtual Worlds. 22nd Bled eConference [31]. Sekaran, U, 2003, Research methods for
Enablement: Facilitating an Open, Effective and business: a skill-building approach, 4 th edn,
Representative eSociety ,14 – 17 June 2009; John Wiley & Sons, Inc
Bled, Slovenia Turku Centre for Computer [32]. Sharkey, U., Scott, M. and Acton, T (2006)
Science (TUCS) Turku School of Economics, The influence of quality on the success of e–
Finland commerce systems., European Conference on
[22]. Urbach, N, Smolnik, S. & Riempp, G. 2009. Information Systems (Ed, Ljungberg J, A. M.),
Development and validation of a model for Goteborg, Sweden, pp. 1711-1722.
assessing the success of employee portals ,17th [33]. Steiger, JH 1990, 'Structural model
European Conference on Information Systems, evaluation and modification: an interval
2009 estimation approach', Multivariate Behavioural
[23]. Ong, C, S, Day. M, Hsu. L 2009.The Research, vol. 25, pp. 173-80.
measurement of user satisfaction with question [34]. Wahlstedt A., and Honkaranta A. (2007),
answering systems, Information & Management Bridging the Gap between Advanced Distributed
46 (2009) 397–403.www.elsevier.com/locate/im Teaching and the Use of Learning Management
[24]. park, N., Lee, K. M., & Cheong, P. H. Systems in the University Context, Seventh
(2007). University instructors' acceptance of IEEE International Conference on Advanced
electronic courseware: An application of the Learning Technologies (ICALT 2007).
technology acceptance model. Journal of [35]. Wang, Y.-S., Wang, H.-Y. & Shee, D. Y.
Computer-Mediated Communication, 13(1), (2007). Measuring e-learning systems success in
article an organizational context: Scale development
9. http://jcmc.indiana.edu/vol13/issue1/park.html and validation. Computers in Human Behavior,
[25]. Petter, S., DeLone, W., and McLean, E. Vol 23, Issue 4, 1792-1808.
(2008) “Measuring information systems success: [36]. Davis, F. D., Bagozzi, R. P., & Warshaw, P.
models, dimensions, measures, and R. (1989). User acceptance of computer
interrelationships”, European Journal of technology: A comparison of two theoretical
Information Systems 17, pp 236-263. models. Management Science, 35 (8), 982-1003.
[26]. Radcliffe, D. (2002). Technological and [37]. Drennan, J. Kennedy, J. & Pisarki, A. 2005.
pedagogical convergence between work-based Factors affecting student attitudes toward
and campus-based learning. Educational flexible online learning in management
Technology & Society, 5 (2). Retrieved May 12,

 
90
 Journal of Theoretical and Applied Information Technology

© 2005 - 2010 JATIT & LLS. All rights reserved.

www.jatit.org 
 
education. The Journal of Educational Research.
v98 i6. 331-340.
[38]. Wang, Y.S. and Liao, Y.W. 2007. Assessing
eGovernment systems success: a validation of
the DeLone and Mclean model of information
system success. Government Information
Quarterly. 25. 717-733.
[39]. Yen, C. and Lu, H. 2008. Effect of e-service
quality on loyalty intention: an empirical study
in online auction. Managing Service Quality.
18(2). 127-146.
[40]. Lee, K.C. and Chung, N. 2009.
Understanding factor affecting trust and
satisfaction with mobile banking in Korea: A
modified DeLone and McLean’s model
perspective. Interacting with Computers. 21.
385-392.
[41]. Lee-Post, A. 2009. E-Learning success
model: an information system perspective.
Electronic Journal of eLearning. 7(10). 61-70.

 
91