education

sciences
Article
Using PLS-SEM Model to Explore the Influencing Factors of
Learning Satisfaction in Blended Learning
Chun-Hsiung Huang

Department of Digital Content Design, Ling Tung University, Taichung 408213, Taiwan;
[email protected]

Abstract: This research explores the influencing factors of learning satisfaction in blended learning.
Three dimensions are proposed: perceived usefulness, perceived ease of use, and learning motivation.
It studied how these variables affect students’ learning satisfaction. The research hypotheses are: (1)
Perceived ease of use positively affects perceived usefulness; (2) Perceived usefulness and perceived
ease of use will have a positive effect on learning motivation; (3) Learning motivation positively affects
learning satisfaction; (4) Perceived usefulness has a positive intermediary effect on the relationship
between perceived ease of use and learning motivation. Participants included 173 freshmen who
took the first-year interactive game design course at Ling Tung University in Taichung, Taiwan.
The questionnaire survey method is applied in this research to analyze the relationship between
the variables and verify the hypothesis based on the collected 173 valid questionnaires. The partial
least square method structural equation model (PLS-SEM) is used to carry out structural equation
modeling to study the relationship between latent variables. It explains that the perceived ease of
use affects the perceived usefulness. Perceived usefulness and perceived ease of use have a positive
impact on learning motivation. Learning motivation has a positive impact on learning satisfaction.





 Perceived usefulness as an intermediary factor of perceived ease of use has an indirect impact on
learning motivation. The contribution of this research is to provide empirical evidence and explain
Citation: Huang, C.-H. Using
what factors may affect learning satisfaction. Some other related factors that may affect learning
PLS-SEM Model to Explore the
Influencing Factors of Learning satisfaction should be taken as the factors that teachers should pay attention to when implementing
Satisfaction in Blended Learning. blended learning.
Educ. Sci. 2021, 11, 249. https://
doi.org/10.3390/educsci11050249 Keywords: PLS-SEM; blended learning; perceived usefulness; perceived ease of use; learning moti-
vation; learning satisfaction
Academic Editor: Eleanor Dommett

Received: 12 April 2021

Accepted: 17 May 2021 1. Introduction
Published: 20 May 2021
Due to the prevalence of information technology, the teaching method of blended
learning that combines online teaching and face-to-face teaching has become a common
Publisher’s Note: MDPI stays neutral
method in current teaching strategies [1,2]. Blended learning can be divided into online and
with regard to jurisdictional claims in
offline teaching based on different places where teaching activities take place [3]. Through
published maps and institutional affil-
the combination of traditional face-to-face teaching and online teaching, the teaching
iations.
mode of blended learning can cultivate students’ independent learning and independent
problem-solving abilities. It can also promote students’ self-regulated and collaborative
learning [4]. Incorporating ICT (Information and Communication Technologies) teaching
technology allows teachers to find a teaching method that suits and tailors to their needs.
Copyright: © 2021 by the author.
It can also provide students with more active participation in learning [5]. Moreover, the
Licensee MDPI, Basel, Switzerland.
application of information technology to teaching has not only changed the way students
This article is an open access article
obtain learning materials, but also promoted their active and independent learning, the
distributed under the terms and
ability to think, and critical thinking [6].
conditions of the Creative Commons
Attribution (CC BY) license (https://
However, are the learners satisfied with this blended learning model? Additionally,
creativecommons.org/licenses/by/
what factors will directly or indirectly affect students’ learning satisfaction? What kind
4.0/).
of influence relationship exists among the various factors? These are the key factors

Educ. Sci. 2021, 11, 249. https://doi.org/10.3390/educsci11050249 https://www.mdpi.com/journal/education

Educ. Sci. 2021, 11, 249 2 of 17

to evaluate the design effects of teaching activities, and student learning satisfaction is
also an important part of learning effectiveness analysis [7]. The research literature of
blended learning seldom talks about the relationship between usefulness, ease of use,
learning motivation and learning satisfaction, and lacks understanding of the relationship
between usefulness, ease of use, and learning motivation. Therefore, the relationship
between usefulness and ease of use and learning motivation, as well as the degree to
which learning motivation affects learning satisfaction, is what this research wants to
emphasize. As students are the main body of learning, individual differences of students
have different subjective feelings and learning experiences for the learning process with
the teaching method of blended learning. Learning satisfaction has certain significance for
the design of teaching activities, improvement of learning effectiveness, and promotion of
individualized learning. This research attempts to establish a model of the factors which
influence learning satisfaction in interactive game design courses from the perspectives
of perceived usefulness, perceived ease of use, and learning motivation. The partial
least square method (PLS-SEM) is used to explain the structure model and analyze the
interrelationship among influencing factors. It will provide ideas and references for follow-
up theoretical research and teaching practice.

2. Literature Review and Hypotheses

2.1. Blended Learning
Blended learning combines two advantages of face-to-face and online learning [8].
Teaching and learning can make teaching activities more flexible through synchronous
and asynchronous methods. Blended learning is a methodology that involves teaching
and learning. It integrates e-learning with traditional learning theories and practices
that promote self-regulation and collaborativeness [9]. Blended learning takes learning
activities, learners and teachers as elements, and it can form a variety of blended learn-
ing forms [10]. In particular, the implementation of interactive game design curriculum
teaching allows teachers to apply a variety of synchronous or asynchronous teaching. Ad-
ditionally, students in groups can also discuss through various social software in different
places. The combination of online teaching materials and the traditional face-to-face teach-
ing mode can promote independent, group, and collaborative learning for students [11].
Blended learning can be mixed into various aspects, including blended offline and
online learning, self-directed and cooperative learning, and structured and unstructured
learning. This method of combining several different teaching methods to transfer knowl-
edge can be assisted by information technology, such as cooperative learning software,
online courses, and knowledge management tools [12]. In addition, the provision of online
teaching and advanced internet technology provides an effective way to enter online learn-
ing for the blended learning model. For students who participate in learning independently,
this method is also very successful and effective for adult learners [13].
In blended learning, interaction can be carried out face-to-face or online, which is
considered to be the advantage of this educational method [14]. Blended learning is based
on constructive learning theory, emphasizing student-centered learning. The gaining of
knowledge and abilities is completed by students in the process of active construction.
Students learn in a time, place, and learning pace or learning path that can be indepen-
dently controlled [15]. Blended learning courses can combine online and offline teaching
through appropriate teaching design, and they can interactively transform different learn-
ing methods to improve students’ learning motivation and interest [16]. The benefits can
provide students with a richer learning experience [17]. With the development of online
learning technology and the popularization of internet technology, blended learning is
quickly becoming the standard in higher education [18].

2.2. Perceived Usefulness and Perceived Ease of Use

In this study, we used perceived usefulness and perceived ease of use to check stu-
dents’ initial acceptance of blended learning. Perceived usefulness and perceived ease of
Educ. Sci. 2021, 11, 249 3 of 17

use are important factors in the Technology Acceptance Model. The Technology Accep-
tance Model (TAM) is a theory developed by Davis [19] to explain the determinants of
technology acceptance. Perceived usefulness refers to the degree to which one believes
using a particular system can improve one’s work performance. The perceived ease of use
refers to the degree of awareness that a person thinks a particular system is easy to use or
not. In the TAM, the perceived ease of use will affect the perceived usefulness, and these
two variables will also affect the behavioral intention of use at the same time.
Research on the use of new information technology in the learning environment
pointed out that Wu and Chen [20] conducted a survey on the use of MOOC environments
in China. This research found that perceived ease of use has an important impact on
perceived usefulness. Joo et al. [21] tried to integrate self-determination theory with
TAM. They tested the use intention of Korean college students to use K-MOOC learning
environments. The results also showed that perceived ease of use has an important impact
on perceived usefulness. Therefore, this research proposes the following hypothesis:

Hypothesis 1 (H1). Perceived ease of use positively affects perceived usefulness.

2.3. Learning Motivation

Motivation is a mental representation. It mainly refers to an internal driving force that
causes an individual to produce active behaviors, persevere and move toward a certain
goal [22]. Learning motivation is the motivation of learners to continue their learning
behavior or a demand for success in the learning process. Learning motivation plays an im-
portant role in the learning process. It helps students to continue learning with confidence
in the learning process [23]. Learning motivation can also be a mental journey that allows
learners to induce learning activities and move towards learning goals [24]. Learning
motivation can be divided into internal and external motivation. Internal motivation refers
to the motivation caused by the individual’s interest in the learning activity. External
motivation is the motivation that promotes individual participation in learning activities
caused by external factors [25]. Learning motivation is an indispensable internal driving
force in the learning process of students. It allows students to persevere in learning towards
their goal. Compared with externally motivated students, intrinsically motivated students
are more likely to persevere and maintain long-term learning motivation when facing
learning challenges [26]. Studies have shown that the relationship between teachers’ input
and students’ learning effectiveness is not a simple positive correlation [27]. The high input
of teachers may not be able to obtain high learning effectiveness from students. The effect
of teaching input on learning effectiveness will be affected by factors such as learning
motivation [28].
Nikou and Economides [29] combined technology acceptance theory and motivation
factors in a study. They proposed the Mobile Based Assessment–Motivational and Ac-
ceptance Model (MBA-MAM). The model combines the interpretation and prediction of
behavioral intentions using mobile learning devices. With the development of technology,
the use of various learning technologies has become a part of daily life. The learning
environment and the characteristics of the learners will greatly affect the experience and
performance of the way of using the learning system [30]. There are some qualitative
studies that provide useful insights into various types of motivation and participation pat-
terns from the learner’s perspective [31,32]. Liang et al. [33] found that learners’ cognition
of the usefulness of MOOC has a significant impact on learning participation. Thereby
it improves the learning outcomes. According to Venkatesh and Davis [34], perceptual
ease of use is a personal view of the ease of use of new technologies. It is considered to
be an incentive for students in order to adopt new technological advancements. Shiau
and Chau [35] also believed that if the technical tool or system is easy for students to use,
students will use this new service intentionally. Relevant studies using the TAM believed
that motivation factors can improve the predictability of the model [36,37]. We can extend
Educ. Sci. 2021, 11, 249 4 of 17

this argument to be that higher perceived ease of use and perceived usefulness will affect
students’ learning motivation. Therefore, the following hypotheses are made:

Hypothesis 2 (H2). Perceived usefulness positively affects learning motivation.

Hypothesis 3 (H3). Perceived ease of use positively affects learning motivation.

2.4. Learning Satisfaction

Learning satisfaction refers to the kinds of personal inner feelings of learners in the
learning process. The satisfaction and positive feelings are generated by the learning
process experienced by the individual in learning activities [38]. It mainly emphasizes
the individual’s subjective feelings about the content, methods, process, and results of
learning activities. In the learning process, if the learning activity meets or exceeds its
“expectations”, it is considered as “satisfied”, otherwise it is considered as “unsatisfied”.
Satisfactory experience will affect continuous participation in learning activities and in-
crease investment in learning activities [39]. Learners’ perception of learning results comes
from the difference between the actual learning content obtained in the learning process
and the learning results expected to be obtained, which is the learning satisfaction. Learn-
ing satisfaction will continue to affect students’ learning experience. When learners meet
or exceed expectations in their learning performance, they will have a sense of satisfaction.
Additionally, the learning process will continue and appear repeatedly, which will form
a virtuous circle [40,41]. Chou [42] verified the relationship between flipped classroom
teaching and self-efficacy through experimental research. Self-efficacy has an impact on
flipped classrooms with the help of learning satisfaction, which is conducive to enhance
students’ confidence and improve learning effects.
In previous studies, it was found that if there is a good large number of interactions
between learners, teachers, and learning materials, this will positively affect learning sat-
isfaction. Higher learning satisfaction will also affect learning motivation and learning
effectiveness [43,44]. Learning motivation is an important indicator that affects learning sat-
isfaction. Some experimental studies have found that students’ attitudes towards teachers,
their learning motivation and experience will affect their performance and satisfaction [45].
The stronger the learner’s learning motivation is, the more significant learning satisfaction
will be in the learning process. Based on the literature from above, this study concludes
that learning motivation and learning satisfaction in blended learning have a substantial
influence. Therefore, the following hypothesis is made:

Hypothesis 4 (H4). Learning motivation positively affects learning satisfaction.

3. Methodology
3.1. Teaching Method Design
This research focuses on blended learning with the concept of a flipped classroom [46].
The flipped classroom consists of two parts: one is the interactive group learning activities
inside the classroom, and the other is the direct computer-based individual teaching
activities outside the classroom. The core concept of the mixed teaching of the flipped
mode is to flip the teaching mode. In traditional teaching, instructors teach course content
in the classroom, and students discuss, practice, and complete their homework after class.
Flipped classroom teaching means that students watch the video of the course content
recorded by the instructors before class, and then go to the classroom to discuss, practice,
and complete their homework. In this way, students can implement the traditional face-to-
face classroom teaching and online learning at the same time.
The subjects of this research are the students from the Interactive Game Design
course in three freshman classes in both the day and the night division of the university.
The number of teaching sessions per week for this course is two classes, with a length of 50
Educ. Sci. 2021, 11, x FOR PEER REVIEW 5 of 16

Educ. Sci. 2021, 11, 249 5 of 17

The subjects of this research are the students from the Interactive Game Design
course in three freshman classes in both the day and the night division of the university.
The number of teaching sessions per week for this course is two classes, with a length of
50 min
min perper class-hour.
class-hour. A total
A total of nine
of nine weeksweeks is the
is the implementation.
implementation. TheThe teaching
teaching methodmethod
is a
is a mixture
mixture of traditional
of traditional face-to-face
face-to-face discussion
discussion and online
and online coursecourse
contentcontent learning.
learning. In termsIn
terms
of of online
online learning,learning,
students students learn courses
learn courses online by online by themselves
themselves within
within the coursethe course
schedule
schedule
given given
by the by the
teacher. In teacher. In face-to-face
traditional traditional learning,
face-to-face learning,
groups of 3–4groups
people of
per3–4 people
group are
per group
used are usedteaching
to implement to implement
methods teaching
such asmethods such as group
group guidance, groupdiscussion,
guidance, group dis-
and group
cussion,
work. and in
Lastly, group work.week,
the ninth Lastly,
theinteam
the ninth
members week, thetoteam
need members
complete need toadventure
a computer complete
a computer
game designadventure game scene,
with character, designandwith character,
story design.scene, and story design.

3.2. Research Structure

3.2. Research Structure and
and Hypothesis
Hypothesis
This
This research mainly exploresthe
research mainly explores relationship
the between
relationship betweenstudents’ perceived
students’ usefulness,
perceived useful-
perceived ease of use, learning motivation and learning satisfaction in the
ness, perceived ease of use, learning motivation and learning satisfaction in thecourse teaching
course
of “interactive
teaching game design”
of “interactive gameindesign”
blended inlearning. The structure
blended learning. of this research
The structure of thisisresearch
shown
in Figure in
is shown 1. Figure 1.

Figure 1. Hypotheses and research model.

According toto the

theresearch
researchpurpose
purposeand
andframework, this
framework, research
this proposes
research the the
proposes follow-
fol-
ing research
lowing hypotheses:
research H1–H5.
hypotheses: H1–H5.
1. The
The analysis
analysis of
of the
the differences in the research dimensions
dimensions of different
different background
background
variables.
variables.
2. The
The influence
influence relationship
relationship between research dimensions.

Hypothesis 55 (H5).
Hypothesis (H5). Perceived
Perceived usefulness
usefulness has
has aa positive
positive intermediary
intermediary effect
effect on
on the
the relationship
relationship
between perceived ease of use and learning motivation.
between perceived ease of use and learning motivation.

3.3. The Definition and Measurement of Research Dimensions

3.3. The
All Definition
dimensions andofMeasurement of Research
this study refer Dimensions
to relevant literature for the definition and opera-
tion of variables. The questionnaire was translated into for
All dimensions of this study refer to relevant literature Chinese for students
the definition to fill out.
and operation of
The definition
variables. of the research
The questionnaire wasoperation
translatedand
intothe questionnaire
Chinese items
for students areout.
to fill shown in Tables
The definition
1 and
of 2.
the research operation and the questionnaire items are shown in Tables 1 and 2.
Educ. Sci. 2021, 11, 249 6 of 17

Table 1. The operational definitions.

Dimensions Operational Definition

Students believe that the blended learning
Perceived Usefulness method is very helpful for obtaining new
knowledge.
Students believe that the blended learning
Perceived Ease of Use
method is easy to use.
Students are willing to participate in learning
Learning Motivation
using blended learning methods.
Students believe that the blended learning
Learning Satisfaction style takes subjective feelings about learning
activities.

3.4. Data Collection and Analysis Method

The teacher has told the students in advance before the course is implemented, and
after the course is implemented, the students are asked to volunteer to fill in the ques-
tionnaire. This research questionnaire uses Likert’s 5-point scale, with options including
“strongly disagree”, “disagree”, “normal”, “agree”, and “strongly agree”. For the freshmen
from the Digital Content Design Department of Ling Tung University in Taiwan, question-
naires were issued and completed with a total of 173 valid questionnaires collected after
9 weeks of teaching in a blended learning mode. Next, SPSS 22.0 (IBM, USA) software is
used to carry out descriptive statistics and independent sample t-test methods in order to
test the difference analysis of different background variables to research variables. Smart-
PLS 3.0 uses partial least squares structural equation modeling (PLS-SEM) to measure
the correlation between observation variables and latent variables through a reflective
measurement model.
Partial least squares structural equation modeling (PLS-SEM) is an analysis technique
for detecting or constructing predictive models. Especially for the causal model analysis
between latent variables, it is better than the general linear structural relationship model,
which is very suitable for exploratory research [49,50]. Compared with covariance-based
structural equation modeling (CB-SEM), which is evaluated by covariance matrix, the PLS-
SEM is suitable for small sample analysis [51]. Chin [52] suggested that the sample size
requirement of PLS should be collected at 10 times the dimension of most question items.
The dimension of most question items is learning motivation. There are 8 question items.
According to the principle of statistical conservativeness, a larger sample size should be
used. Therefore, the minimum sample size for research must be at least 80. The sample
size of this study is 173, which meets the minimum sample size requirement.
PLS-SEM is mainly designed to detect whether the causal relationship has a statis-
tically significant mutual linear relationship. It is rather suitable for the construction of
theoretical models. This study uses PLS-SEM as a method to explore the relationship be-
tween the research variables. It mainly uses PLS Algorithm and Bootstrapping to perform
the repetitive sampling 5000 times in order to derive path coefficients and significance [53].
It can discuss the correlation and influence between the dimensions.
Educ. Sci. 2021, 11, 249 7 of 17

Table 2. Questionnaire items and reference.

Dimensions Questions Reference

PU1: This way of learning in class enriches
learning activities.
PU2: This way of learning in class is very helpful
for me to acquire new knowledge.
PU3: The learning mechanism provided by this
way of learning in class makes the learning process
Perceived Usefulness smoother. Hwang et al. [47]
PU4: This way of learning in class helps me get
useful information when I need it.
PU5: This way of learning in class helps me learn
better.
PU6: This way of learning in class is more useful
than the traditional computer classroom.
PEOU1: The kind of operating system by this way
of learning in class is not difficult for me.
PEOU2: It only took me a short time to fully
understand how to apply this way of learning in
class.
PEOU3: The learning activities in this way of
Perceived Ease of Use learning in class are easy to understand and follow. Hwang et al. [47]
PEOU4: I quickly learned to apply this way of
learning in class.
PEOU5: It is not difficult for me to apply the
learning system in this way of learning in class.
PEOU6: I think the system interface of this way of
learning in class is easy to use.
LM1: I think this way of learning in class is
interesting.
LM2: I think this way of learning in class is
valuable.
LM3: I want to learn more in this way of learning
in class.
LM4: I think it is worth to apply this way of
learning in class.
Learning Motivation Hwang et al. [47]
LM5: To me, it is important to apply this way of
learning in class.
LM6: I know that learning to apply this way of
learning in class is very important in the future.
LM7: I will seek more information to learn how to
apply this way of learning in class.
LM8: I think it is important for every student to
learn to apply this way of learning in class.
LS1: I am satisfied with this way of learning in
class.
LS2: If I still have the opportunity to apply this
way of learning in class, I will be happy to do so.
LS3: I think it is a wise choice to study courses in
Learning this way of learning in class.
Sun et al. [48]
Satisfaction LS4: I feel very satisfied with this way of learning
in class.
LS5: I think this way of learning in class satisfies
my learning needs very well.
LS6: I will try to apply this way of learning in class
as much as possible to study courses.
Educ. Sci. 2021, 11, 249 8 of 17

4. Research Results
4.1. Analysis of Background Variables
The subjects of this research are a total of 173 students: two classes of freshmen from
the day division and one class of the night division of the Digital Media Design Department
of Ling Tung University in Taiwan. There are 54 males and 119 females: 123 students from
the day division and 50 students from the night division. The frequency distribution of
sample data is shown in Table 3.

Table 3. Frequency distribution of sample data.

Background
Items Number of People Percentage
Variables
Male 54 31.2%
Gender
Female 119 68.8%
Day-time classes 123 71.1%
School System
Evening classes 50 28.9%

4.2. Reliability and Validity Test

Reliability refers to the consistency of scale tools. The measurement indicators include
individual item reliability and internal consistency [54]. Among them, the individual item
reliability is tested by factor loading. The internal consistency is tested by latent variable
composition reliability (CR) and Cronbach’s alpha. The recommended value needs to be
greater than 0.7.
The validity refers to the correctness of the scale tool, and the measurement indicators
include convergent validity and discriminant validity. The convergent validity is mainly
to measure the correlation between items with the same dimension, and to detect the
average variance extraction (AVE). The recommended value needs to be greater than
0.5 [55]. The discriminant validity is to measure the correlation between items with
different facets, using the square root value of AVE to test. If the square root value of the
diagonal AVE is greater than the correlation coefficient value of the horizontal or vertical
column, it represents discriminative validity [56].
From Table 4, it is known that the factor loadings of the questionnaire items in this
research aspect are all greater than 0.7, which meets the verification standard. The Cron-
bach’s alpha and CR values of all dimensions are also greater than 0.7, indicating good
reliability and internal consistency. The AVE value of each dimension is greater than
0.5, indicating good convergent validity. Table 5 shows that the square root value of the
diagonal AVE is greater than other correlation coefficient values in the matrix. Detected by
heterotrait–monotrait analysis, Table 6 shows that all values are less than 0.9, indicating
good discriminant validity [57].

4.3. The Analysis of the Difference of Various Background Variables on the Research Dimension
4.3.1. Analysis of Gender Differences
The results of independent sample t-tests for students of different genders found that
the dimensions of perceived usefulness, learning motivation, and learning satisfaction
reached a significant level. There was no significant difference in the dimensions of
perceived ease of use. Thus, when blended learning is used in interactive game design
courses, the perceived ease of use dimension will not be different due to the difference
of gender. However, in terms of perceived usefulness, learning motivation, and learning
satisfaction, males are significantly higher than females. Gender difference analysis data
are shown in Table 7.
Educ. Sci. 2021, 11, 249 9 of 17

Table 4. Measurement model parameter estimation.

Dimensions Question Items Factor Loading Cronbach’s α CR AVE

PU1 0.904
PU2 0.931
Perceived PU3 0.922
0.959 0.967 0.832
Usefulness(PU) PU4 0.904
PU5 0.934
PU6 0.876
PEOU1 0.886
PEOU2 0.940
Perceived Ease of PEOU3 0.890
0.962 0.969 0.841
Use(PEOU) PEOU4 0.922
PEOU5 0.939
PEOU6 0.923
LM1 0.871
LM2 0.924
LM3 0.869
Learning LM4 0.938
0.964 0.969 0.798
Motivation(LM) LM5 0.899
LM6 0.897
LM7 0.902
LM8 0.845
LS1 0.904
LS2 0.942
Learning LS3 0.936
0.970 0.976 0.870
Satisfaction(LS) LS4 0.955
LS5 0.932
LS6 0.928

Table 5. Discriminant validity test (Fornell–Larcker).

Dimensions AVE PU PEOU LM LS

PU 0.832 0.912
PEOU 0.841 0.774 0.917
LM 0.798 0.865 0.754 0.893
LS 0.870 0.831 0.716 0.871 0.933
Note: The bold slash text is the square root value of AVE, and the rest are the correlation coefficients between the various dimensions.

Table 6. Heterotrait–monotrait ratio of correlations.

Dimensions PU PEOU LM LS
PU
PEOU 0.801
LM 0.897 0.780
LS 0.861 0.736 0.898
Educ. Sci. 2021, 11, 249 10 of 17

Table 7. Summary table of independent sample t-test analysis of gender on research variables.

Standard
Dimensions Gender Number Mean T Value
Deviation
Male 54 3.90 0.855
PU 2.690 **
Female 119 3.50 0.919
Male 54 4.14 0.799
PEOU 1.731
Female 119 3.91 0.807
Male 54 3.96 0.816
LM 2.882 **
Female 119 3.58 0.797
Male 54 3.92 0.914
LS 2.899 **
Female 119 3.50 0.877
** significant at p < 0.01.

4.3.2. Analysis of the Difference of School Systems

The results of independent sample t-tests for students of different academic systems
found that there are obvious differences between students in the day and the night division
in perceived usefulness and perceived ease of use. The students in the day-time classes are
significantly higher than the students in the evening classes. There is no obvious difference
in the dimensions of learning motivation and learning satisfaction. The analysis of school
system differences is shown in Table 8.

Table 8. Summary table of t-test analysis of independent samples of research variables by school sys-
tem.

School Standard
Dimensions Number Mean T Value
System Deviation
Day-time
123 3.72 0.919
PU classes 2.142 *
Evening
50 3.40 0.873
classes
Day-time
123 4.09 0.795
PEOU classes 2.763 **
Evening
50 3.72 0.794
classes
Day-time
123 3.75 0.810
LM classes 1.227
Evening
50 3.58 0.840
classes
Day-time
123 3.70 0.894
LS classes 1.533
Evening
50 3.47 0.929
classes
* significant at p < 0.05; ** significant at p < 0.01.

4.4. Structural Equation Modeling Analysis

When evaluating structural equation modeling, be sure to confirm that the problem of
collinearity has been eliminated. When the Variance Inflation Factor (VIF) is greater than 5,
it means that there may be a collinearity problem between the dimensions [58]. The VIF
value of the structural equation modeling in this study is less than 5, which is between 1 and
2.498, indicating no collinearity among the study dimensions. SRMR, NFI and RMS_theta
are commonly used indicators for PLS-SEM in order to evaluate the appropriateness of the
overall model. The range of the SRMR value is from 0 to 1. When SRMR is less than 0.08,
it can be regarded as a good fit of the model [59]. The range of the NFI value is between 0
and 1. The larger the value of NFI, the better performance it obtains. When NFI is greater
Educ. Sci. 2021, 11, 249 11 of 17

than 0.9, it indicates that the model fits well [60]. The RMS_theta value is only suitable for
evaluating reflective measurement models. An RMS_theta value less than 0.12 indicates
that the model fits well [61]. The SRMR value of the model evaluation verification in this
study is 0.054. Although the NFI value of 0.858 is less than 0.9, it is not much different. The
RMS_theta value is 0.153. Although it is greater than 0.12, it is also acceptable. Therefore,
the model in this study is reasonably well-fitted in general. The collinearity analysis and
model fit are shown in Table 9.

Table 9. Collinearity analysis and model fit.

Dimension Correlation VIF Model Fit

PEOU and PU 1.000
SRMR = 0.054
PU and LM 2.498
NFI = 0.859
PEOU and LM 2.498
RMS_theta = 0.154
LM and LS 1.000

Next, the model verification is analyzed and explained by the path analysis and R2 .
Educ. Sci. 2021, 11, x FOR PEER REVIEW 11 of 16
In the path analysis, the value of t is used to determine whether the hypothesis is true.
When t value > 1.96, it means a significant level of 0.05 (indicated by *). When t value >
2.58, it means that it reaches a significant level of 0.01 (indicated by **). When t value >
When t value > 1.96, it means a significant level of 0.05 (indicated by *). When t value >
3.29, it means that it reaches a significant level of 0.001 (indicated by ***). It can be seen
2.58, it means that it reaches a significant level of 0.01 (indicated by **). When t value >
from Table 10 that H1, H2, and H4 have reached a significant level with a p value less than
3.29, it means that it reaches a significant level of 0.001 (indicated by ***). It can be seen
0.001. H3 reaches a significant level with a p value less than 0.05. Therefore, the hypotheses
from Table 10 that H1, H2, and H4 have reached a significant level with a p value less than
of0.001.
H1, H3H2,reaches
H3, and H4 in thislevel
a significant study are
with a pvalid. Thethan
value less PLS-SEM path analysis
0.05. Therefore, model is shown
the hypotheses
inofFigure 2.
H1, H2, H3, and H4 in this study are valid. The PLS-SEM path analysis model is shown
in Figure 2.
Table 10. Path analysis verification.
Table 10. Path analysis verification.
Path Analysis Path Coefficient T Value p Value Hypothesis
Path Analysis Path Coefficient T Value p Value Hypothesis
PEOU→PU 0.774 20.020 *** 0.000 H1 valid
PEOU→PU 0.774 20.020 *** 0.000 H1 valid
PU→LM 0.701 9.050 *** 0.000 H2 valid
PEOUPU→LM
→LM 0.701
0.211 9.050 ****
2.580 0.000 0.010 H2 validH3 valid
PEOU→LM
LM→LS 0.211
0.871 36.434****
2.580 0.010 0.000 H3 validH4 valid
LM→LS 0.871
* significant at p < 0.05; *** significant at p < 0.001.36.434 *** 0.000 H4 valid
* significant at p < 0.05; *** significant at p < 0.001

Figure 2.
Figure 2. Model
Model of
ofPLS-SEM
PLS-SEMpath analysis
path diagram.
analysis diagram.

The R
The R2 value
2
valueisisused
usedtoto
evaluate thethe
evaluate explanatory ability
explanatory of theofmodel.
ability The R The
the model.
2 valueRis
2 value is
between 0 and 1. The higher the value, the higher the explanatory power. When
between 0 and 1. The higher the value, the higher the explanatory power. When the R2 value the R 2

isvalue
closeistoclose
0.50,tothe
0.50, the model has a moderate explanatory power. When the R value is
2
model has a moderate explanatory power. When the R2 value is close to
close to 0.75, the model has a high degree of explanatory power. It can be seen from Table
0.75, the model has a high degree of explanatory power. It can be seen from Table 11 that
11 that the explanatory power of perceived ease of use to perceived usefulness is 60.0%.
the explanatory power of perceived ease of use to perceived usefulness is 60.0%. Perceived
Perceived usefulness and perceived ease of use explanatory power to learning motivation
usefulness
is 76.5%. The and perceivedpower
explanatory ease of use explanatory
of learning power
motivation to learning
to learning motivation
satisfaction is 81.2%.is 76.5%.
Therefore, the model in this study explains the latent variables very well and it has a high
degree of explanatory power.
Effect size is the effect of exogenous variables on endogenous variables using the ex-
planatory effect value f2 to detect. When 0.02 < f2 ≤ 0.15, it is a small effect. When 0.15 < f2
2
Educ. Sci. 2021, 11, 249 12 of 17

The explanatory power of learning motivation to learning satisfaction is 81.2%. Therefore,

the model in this study explains the latent variables very well and it has a high degree of
explanatory power.

Table 11. R2 value and f2 value.

Path Analysis R2 R2 Adjusted f2

PEOU→PU 0.600 0.597 1.498
PU→LM 0.841
0.765 0.763
PEOU→LM 0.076
LM→LS 0.759 0.758 3.152

Effect size is the effect of exogenous variables on endogenous variables using the
explanatory effect value f2 to detect. When 0.02 < f2 ≤ 0.15, it is a small effect. When 0.15 <
f2 ≤ 0.35, it is a medium effect. Additionally, when f2 > 0.35, it is a large effect. It can be
seen from Table 11 that the explanatory effect value f2 of PEOU to LM is 1.498. It displays
a large-effect explanatory ability. The explanatory effect value f2 of PU to LM is 0.841,
which displays a large-effect explanatory ability. The explanatory effect value f2 of PEOU
to LM is 0.076, which displays a small-effect explanatory ability. The explanatory effect
value f2 of LM to LS is 4.313, which displays a large-effect explanatory ability. Except for
the fact that the explanatory effect value of perceived ease of use on learning motivation
displays a small-effect explanatory ability, the rest belongs to the large-effect explanatory
ability. This represents that exogenous variables are very capable of explaining endogenous
variables, with a high degree of explanatory effect value.
The conditions for the establishment of the intermediary effect firstly depend on the
path coefficient of the independent variable to the intermediary variable, and the path
coefficient of the independent variable to the dependent variable must both be significant.
Secondly, the path coefficient of the intermediate variable to the dependent variable must
also be significant. The independent variable produces an indirect effect on the dependent
variable through the intermediate variable. If it is greater than the direct effect of the
independent variable on the dependent variable, it can be judged that the intermediate
variable does have an intermediate effect [62].
The test of intermediary effect can also be detected by the t value of indirect effect.
A value of t greater than 1.96 indicates that there is an intermediary effect [63]. To assess the
size of the intermediary effect, the variance accounted for (VAF value) can be applied. VAF
< 20% means a slight intermediary effect. Furthermore, 20% < VAF < 80% means a partial
intermediary effect. VAF > 80% indicates a complete intermediary effect. In this study, PU
is used in the relationship between PEOU and LM, and it plays an intermediary role. The
indirect effect value is 0.543, and the t value is 9.453, which reaches a significant level. The
H5 hypothesis is established. That is, in the influence of perceived ease of use on learning
motivation, perceived usefulness has a partial intermediary effect. The intermediary effect
verification data are shown in Table 12.

Table 12. Intermediary effect verification.

Independent Intervening Dependent Direct Indirect

VAF Hypothesis
Variable Variable Variable Effect Effect
0.211 0.543
PEOU PU LM (t = 2.583 (t = 9.406 72.02% H5 valid
**) ***)
** significant at p < 0.01; *** significant at p < 0.001.

5. Discussion
The main purpose of this research is to explore the influencing factors of learning
satisfaction in blended learning with the PLS-SEM model. This research attempts to
Educ. Sci. 2021, 11, 249 13 of 17

explain the relationship between perceived usefulness, perceived ease of use, learning
motivation, and learning satisfaction. In the hypothetical model, we propose the impact
of two variables: perceived usefulness and perceived ease of use on learning motivation.
Additionally, we propose how learning motivation affects students’ learning satisfaction
with blended learning as well. In this section, we discuss findings related to the research
model.
First of all, in terms of the analysis of the difference of different background vari-
ables to the research dimensions: in terms of perceived usefulness, learning motivation,
and learning satisfaction, male students are significantly higher than female students.
There is no significant difference in the perceived ease of use dimension. In terms of
perceived usefulness and perceived ease of use, students from day-time classes are signifi-
cantly higher than students from evening classes. Compared to students from day-time
classes, students from evening classes think that this blended learning method is not easy
to use and that this blended learning method is not helpful for obtaining new knowledge.
It is possible that the students from evening classes need to work during the day and do
not have time to learn the course content recorded by the teacher in advance. There is no
obvious difference in the dimensions of learning motivation and learning satisfaction.
Second, this research found that perceived ease of use does affect perceived usefulness
(H1). This is consistent with the existing research [20,21]. Therefore, we speculate that
students consider the blended learning method to be intuitively easy to use, which is very
helpful for their learning. Considering that blended learning methods should be easy to
use, designing student-friendly teaching methods is essential so that students will not
encounter technical problems when using them for the first time. Additionally, they would
believe that blended learning is very useful for their learning activities and goals.
Third, both perceived usefulness and perceived ease of use have a significant positive
impact on learning motivation (H2, H3). The effect of perceived usefulness is far greater
than that of perceived ease of use. This research infers that having powerful functionality
is an important factor for students to consider learning motivation in blended learning.
The teaching strategy we apply is to promote the growth of students’ knowledge and
skills through useful teaching. Additionally, it will promote students’ learning motivation.
Therefore, the teaching content that students consider useful is the key to whether they can
learn successfully. Perceived ease of use will also affect learning motivation. This shows that
the easier to use blended learning, the more positive the evaluation of learning motivation.
Fourth, learning motivation owns a significant positive impact on students’ learning
satisfaction (H4). This result is the same as the previous study [45]. This shows that it
is important for learners to generate learning motivation for blended learning methods.
Students with motivation will be able to continue learning with confidence during the
learning process. Therefore, how to make students willing to use blended learning methods
to participate in learning can affect their learning satisfaction. Learning experience with
satisfaction will affect the continuous participation in learning activities and form a virtuous
circle eventually.
Finally, the perceived ease of use mediated by perceived usefulness has a positive
intermediary effect on learning motivation. Learning motivation is the willingness of
students to use blended learning methods and participate in learning. Perceived usefulness
far outweighs the impact of perceived ease of use on learning motivation. The significance
of this discovery lies in the fact that students believe the implementation of teaching
strategies in blended learning methods and the ability to acquire new knowledge. Useful
teaching content is the most important element. Therefore, when implementing blended
learning methods, using appropriate teaching strategies and providing useful content
teaching can make students willing to participate in learning. This affects their learning
satisfaction, and teachers should pay attention to these details.
Educ. Sci. 2021, 11, 249 14 of 17

6. Conclusions
6.1. Research Conclusion
In addition to the good reliability and validity of this research, it also confirmed
that there is no collinearity between the research facets when evaluating the structural
model. The SRMR value of this research model evaluation verification is 0.054, which is
less than 0.08. Although the NFI value of 0.858 is less than 0.9, it is not much different. The
RMS_theta value is 0.153. Although it is greater than 0.12, it is also acceptable. Overall, the
model in this study is reasonably well-fitted. It can be seen from the path relationship of
the model that Hypothesis 1 is valid, and perceived ease of use positively affects perceived
usefulness (β = 0.774, t = 20.020, p < 0.001). Hypothesis 2 is valid. Perceived usefulness
positively affects learning motivation (β = 0.701, t = 9.050, p < 0.001). Hypothesis 3 is
valid. Perceived ease of use positively affects learning motivation (β = 0.211, t = 2.580, p <
0.05). Hypothesis 4 is valid. Learning motivation positively affects learning satisfaction
(β = 0.871, t = 36.434, p < 0.001). Through the test of intermediary effect, Hypothesis 5 is
valid. Perceived usefulness has a positive intermediary effect on the relationship between
perceived ease of use and learning motivation (t = 9.406, p < 0.001).
The contributions of this research are as follows: it provides empirical data evidence of
students’ perceived usefulness, perceived ease of use, and learning motivation to learning
satisfaction in a blended learning system. According to data analysis, we found that
students’ willingness to use blended learning methods has a very significant impact on
their learning satisfaction. The key to making students willing to use blended learning
lies in the teaching strategies and useful teaching methods implemented by teachers. The
teaching methods and strategies implemented by teachers need to guide students to achieve
learning goals and maintain their learning progress. The powerful and functional blended
learning can make students think that it is very helpful to acquire new knowledge.

6.2. Research Limitations and Suggestions

We should also pay attention to some limitations of this research. First, this research
investigates the perception of blended learning in a specific university environment, ex-
cluding students who take different courses in different grades, such as language or music
courses. The sample size comes from 173 freshman students. It is suggested that future
related research can expand the sample size so that quantitative data make more extensive
inferences and macro explorations. Second, because this research is about the specific
courses offered by the university and measuring students’ learning motivation and learn-
ing satisfaction through subjective perception, the research results should be extrapolated
to other situations carefully. Third, it is recommended that qualitative research can be
added in the future to make further analysis of students’ learning motivation and learning
satisfaction. It will also explore how to improve teachers to face the introductory teaching
of blended learning and have more adequate preparation and response.

Funding: This research received no external funding.

Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: Data is contained within the article. For further questions, please
contact the author.
Conflicts of Interest: The author declares no conflict of interest.

References
1. Bonk, C.J.; Graham, C.R. The Handbook of Blended Learning: Global Perspectives, Local Designs; John Wiley & Sons: San Francisco,
CA, USA, 2006; pp. 3–21.
2. Drysdale, J.S.; Graham, C.R.; Spring, K.J.; Halverson, L.R. An analysis of research trends in dissertations and theses studying
blended learning. Internet High. Educ. 2013, 17, 90–100. [CrossRef]
Educ. Sci. 2021, 11, 249 15 of 17

3. Gilbert, R.; Joshi, L.; Kilpert, L.; Lightman, S. Blended learning: E-patients and patient perspectives in ophthalmology. Med. Educ.
2018, 52, 553–554. [CrossRef]
4. Sun, Z.; Xie, K.; Anderman, L.H. The role of self-regulated learning in students’ success in flipped undergraduate math courses.
Internet High. Educ. 2018, 36, 41–53. [CrossRef]
5. Dziuban, C.; Graham, C.R.; Moskal, P.D.; Norberg, A.; Sicilia, N. Blended learning: The new normal and emerging technologies.
Int. J. Educ. Technol. High. Educ. 2018, 15, 1–16. [CrossRef]
6. Yean, L.S. Promoting active learning and independent learning among primary school students using flipped classroom. Int. J.
Educ. 2019, 4, 324–341.
7. Piccoli, G.; Ahmad, R.; Ives, B. Web-based virtual learning environment: A research framework and a preliminary assessment of
effectiveness in basic IT skill training. MIS Q. 2001, 25, 401–426. [CrossRef]
8. De Leng, B.A.; Dolmans, D.H.J.M.; Donkers, H.H.L.M.; Muijtjens, A.M.M.; Vleuten, C.P.M. Instruments to explore blended
learning: Modifying a method to analysis online communication for the analysis of face-to-face communication. Comput. Educ.
2010, 55, 644–651. [CrossRef]
9. Fernandes, J.; Costa, R.; Peres, P. Putting Order into Our Universe: The Concept of Blended Learning—A Methodology within the
Concept-based Terminology Framework. Educ. Sci. 2016, 6, 15. [CrossRef]
10. Osguthorpe, R.T.; Graham, C.R. Blended learning environments: Definitions and directions. Q. Rev. Distance Educ. 2003, 4,
227–233.
11. Broadbent, J. Comparing online and blended learner’s self-regulated learning strategies and academic performance. Internet High.
Educ. 2017, 33, 24–32. [CrossRef]
12. Valiathany, P. Blended learning models. Learn. Circuits 2002, 3, 50–59.
13. Martyn, M. The hybrid online model: Good practice. Educ. Q. 2003, 26, 18–23.
14. Johnson, C.S. Collaborative technologies, higher order thinking and self-sufficient learning: A case study of adult learners. Res.
Learn. Technol. 2017, 25, 1–17. [CrossRef]
15. Spanjers, I.A.E.; Könings, K.D.; Leppink, J.; Verstegen, D.M.L.; De Jong, N.; Czabanowska, K.; Merriënboera, J.J.G. The promised
land of blended learning: Quizzes as a moderator. Educ. Res. Rev. 2015, 15, 59–74. [CrossRef]
16. Hainey, K.; Kelly, L.J.; Green, A. A blended learning approach to teaching CVAD care and maintenance. Br. J. Nurs. 2017, 26,
S4–S12. [CrossRef]
17. Castro, R. Blended learning in higher education: Trends and capabilities. Educ. Inf. Technol. 2019, 24, 2523–2546. [CrossRef]
18. Masi, A.C.; Winer, L.R. A university-wide vision of teaching and learning with information technologies. Innov. Educ. Teach. Int.
2005, 42, 147–155. [CrossRef]
19. Davis, F.D. Perceived usefulness, perceived ease of use, and user acceptance of information technology. Mis Q. 1989, 13, 319–340.
[CrossRef]
20. Wu, B.; Chen, X. Continuance intention to use MOOCs: Integrating the technology acceptance model (TAM) and task technology
fit (TTF) model. Comput. Hum. Behav. 2017, 67, 221–232. [CrossRef]
21. Joo, Y.J.; So, H.J.; Kim, N.H. Examination of relationships among students’ self-determination, technology acceptance, satisfaction,
and continuance intention to use K-MOOCs. Comput. Educ. 2018, 122, 260–272. [CrossRef]
22. Hulleman, C.S.; Durik, A.M.; Schweigert, S.A.; Harackiewicz, J.M. Task values, achievement goals, and interest: An integrative
analysis. J. Educ. Psychol. 2008, 100, 398–416. [CrossRef]
23. Glynn, S.M.; Aultman, L.P.; Owens, A.M. Motivation to learn in general education programs. J. Gen. Educ. 2005, 54, 150–170.
[CrossRef]
24. MacIntyre, P.D.; Blackie, R.A. Action control, motivated strategies, and integrative motivation as predictors of language learning
affect and the intention to continue learning French. System 2012, 40, 533–543. [CrossRef]
25. Deci, E.L.; Ryan, R.M. The “what” and “why” of goal pursuits: Human needs and the self-determination of behavior. Psychol. Inq.
2000, 11, 227–268. [CrossRef]
26. Huang, W.D. Evaluating learners’ motivational and cognitive processing in an online game-based learning environment. Comput.
Hum. Behav. 2011, 27, 694–704. [CrossRef]
27. Wu, E.; Yang, S.C. Examining the impact of online labeling on tutoring behavior and its effect on the English learning and
motivation of low-achieving university students. Comput. Assist. Lang. Learn. 2016, 29, 316–333. [CrossRef]
28. Lai, C. Modeling teachers’ influence on learners’ self-directed use of technology for language learning outside the classroom.
Comput. Educ. 2015, 82, 74–83. [CrossRef]
29. Nikou, S.A.; Economides, A.A. Mobile-based assessment: Integrating acceptance and motivational factors into a combined model
of self-determination theory and technology acceptance. Comput. Hum. Behav. 2017, 68, 83–95. [CrossRef]
30. Kim, J.H. A structural analysis of factors affecting learning flow of participants in lifelong education programs. Korean J.
Argricultural Educ. Hum. Resour. Dev. 2005, 37, 275–301.
31. Littlejohn, A.; Hood, N.; Milligan, C.; Mustain, P. Learning in MOOCs, motivation and self-regulated learning. Internet High.
Educ. 2016, 29, 40–48. [CrossRef]
32. Milligan, C.; Littlejohn, A. Why study on a MOOC? The motives of students and professionals. Int. Rev. Res. Open Distance Learn.
2017, 18, 93–102. [CrossRef]
Educ. Sci. 2021, 11, 249 16 of 17

33. Liang, D.; Jia, J.; Wu, X.; Miao, J.; Wang, A. Analysis of learners’ behaviors and learning outcomes in a massive open online
course. Knowl. Manag. E-Learn. 2014, 6, 281–298.
34. Venkatesh, V.; Davis, F.D. A theoretical extension of the technology acceptance model: Four longitudinal field studies. Manag. Sci.
2000, 46, 186–204. [CrossRef]
35. Shiau, W.L.; Chau, P.Y. Understanding behavioral intention to use a cloud computing classroom: A multiple model comparison
approach. Inf. Manag. 2016, 53, 355–365. [CrossRef]
36. Chen, H.R.; Tseng, H.F. Factors that influence acceptance of web-based e-learning systems for the in-service education of junior
high school teachers in Taiwan. Eval. Program Plan. 2012, 35, 398–406. [CrossRef]
37. Rupp, M.; Michaelis, J.; McConnell, D.; Smither, J. The role of individual difference on perceptions of wearable fitness device
trust, usability and, motivational impact. Appl. Ergon. 2018, 70, 77–87. [CrossRef]
38. Kuo, M.C.; Chang, P. A total design and implementation of an intelligent mobile chemotherapy medication administration. Stud.
Health Technol. Inform. 2014, 201, 441–446.
39. Martin, C.L. Enhancing children’s satisfaction and participation using a predictive regression model of bowling performance
norms. Phys. Educ. 1988, 45, 196–209.
40. Elliott, K.M. Key determinants of student satisfaction. J. Coll. Stud. Retent. Res. Theory Pract. 2002, 4, 271–279. [CrossRef]
41. David, W.L.; Joao, S.N. Determinants of undergraduate business student satisfaction. Res. High. Educ. J. 2010, 6, 1–26.
42. Chou, L.Y. The Effect of Flipped Classroom on Self-efficacy and Satisfaction of Computer Auditing. In Innovative Mobile and
Internet Services in Ubiquitous Computing, Proceedings of the 11th International Conference on Innovative Mobile and Internet Services in
Ubiquitous Computing (IMIS-2017), Torino, Italy, 10–12 July 2017; Springer: Berlin/Heidelberg, Germany, 2018; pp. 841–845.
43. Kuo, Y.C.; Walker, A.E.; Belland, B.R.; Schroder, K.E. A predictive study of student satisfaction in online education programs. Int.
Rev. Res. Open Distance Learn. 2013, 14, 16–39. [CrossRef]
44. Lovecchio, C.P.; DiMattio, M.J.K.; Hudacek, S. Predictors of undergraduate nursing student satisfaction with clinical learning
environment: A secondary analysis. Nurs. Educ. Perspect. 2015, 36, 252–254. [CrossRef]
45. Whillier, S.; Lystad, R.P. No differences in grades or level of satisfaction in a flipped classroom for neuroanatomy. J. Chiropr. Educ.
2015, 29, 127–133. [CrossRef] [PubMed]
46. Bishop, J.L.; Verleger, M.A. The Flipped Classroom: A Survey of the Research. In Proceedings of the ASEE National Conference
Proceedings, Atlanta, GA, USA, 23–26 June 2013; pp. 1–18.
47. Hwang, G.J.; Yang, L.H.; Wang, S.Y. A concept map-embedded educational computer game for improving students’ learning
performance in natural science courses. Comput. Educ. 2013, 69, 121–130. [CrossRef]
48. Sun, P.C.; Tsai, R.J.; Finger, G.; Chen, Y.Y.; Yeh, D. What drives a successful e-Learning? An empirical investigation of the critical
factors influencing learner satisfaction. Comput. Educ. 2008, 50, 1183–1202. [CrossRef]
49. Pavlou, P.A.; Fygenson, M. Understanding and predicting electronic commerce adoption: An extension of the theory of planned
behavior. MIS Q. 2006, 30, 115–143. [CrossRef]
50. Melchor, M.Q.; Julián, C.P. The impact of the human element in the information systems quality for decision making and user
satisfaction. J. Comput. Inf. Syst. 2008, 48, 44–52.
51. Ringle, C.M.; Sarstedt, M.; Straub, D.W. Editor’s comments: A critical look at the use of PLS-SEM in MIS quarterly. MIS Q. 2012,
36, iii–xiv. [CrossRef]
52. Chin, W.W. The Partial Least Squares Approach for Structural Equation Modeling. Modern Methods for Business Research; Marcoulides,
G.A., Ed.; Lawrence Erlbaum Associates: Mahwah, NJ, USA, 1998; pp. 295–336.
53. Henseler, J.; Chin, W.W. A comparison of approaches for the analysis of interaction effects between latent variables using partial
least squares path modeling. Struct. Equ. Modeling A Multidiscip. J. 2010, 17, 82–109. [CrossRef]
54. Hair, J.F.; Black, W.C.; Babin, B.J.; Anderson, R.E.; Tatham, R.L. Multivariate Data Analysis; Prentice Hall: Upper Saddle River, NJ,
USA, 1998; pp. 207–219.
55. Bagozzi, R.P.; Yi, Y. On the evaluation of structural equation models. J. Acad. Mark. Sci. 1988, 16, 74–94. [CrossRef]
56. Fornell, C.; Larcker, D.F. Evaluating structural equation models with unobservable variables and measurement error. J. Mark. Res.
1981, 18, 39–50. [CrossRef]
57. Henseler, J.; Ringle, C.M.; Sarstedt, M. A new criterion for assessing discriminant validity in variance-based structural equation
modeling. J. Acad. Mark. Sci. 2015, 43, 115–135. [CrossRef]
58. Hair, J.F.; Ringle, C.M.; Sarstedt, M. PLS-SEM: Indeed a silver bullet. J. Mark. Theory Pract. 2011, 19, 139–152. [CrossRef]
59. Hu, L.-t.; Bentler, P.M. Fit indices in covariance structure modeling: Sensitivity to underparameterized model misspecification.
Psychol. Methods 1998, 3, 424–453. [CrossRef]
60. Bentler, P.M.; Bonett, D.G. Significance tests and goodness–of–fit in the analysis of covariance structures. Psychol. Bull. 1980, 88,
588–600. [CrossRef]
61. Henseler, J.; Dijkstra, T.K.; Sarstedt, M.; Ringle, C.M.; Diamantopoulos, A.; Straub, D.W.; Ketchen, D.J.; Hair, J.F.; Hult, G.T.M.;
Calantone, R.J. Common beliefs and reality about partial least squares: Comments on Rönkkö & Evermann (2013). Organ. Res.
Methods 2014, 17, 182–209.
Educ. Sci. 2021, 11, 249 17 of 17

62. Baron, R.M.; Kenny, D.A. The moderator–mediator variable distinction in social psychological research: Conceptual, strategic,
and statistical considerations. J. Personal. Soc. Psychol. 1986, 51, 1173–1182. [CrossRef]
63. Zhao, X.; Lynch, J.; Chen, Q. Reconsidering Baron and Kenny: Myths and truths about mediation analysis. J. Consum. Res. 2010,
37, 197–206. [CrossRef]