Neural Computing and Applications

https://doi.org/10.1007/s00521-020-04767-0 (0123456789().,-volV)(0123456789().
,- volV)

ORIGINAL ARTICLE

Intelligent tutoring system using expert knowledge and Kohonen

maps with automated training
Sirlon Diniz de Carvalho1 • Francisco Ramos de Melo2 • Edna Lucia Flôres3 • Sandrerley Ramos Pires4 •

Luiz Fernando Batista Loja1

Received: 4 May 2019 / Accepted: 3 February 2020

Ó Springer-Verlag London Ltd., part of Springer Nature 2020

Abstract
This paper presents an intelligent tutoring system (ITS) model that is capable of driving the didactic transposition of
contents. Initially, the tutoring system reactions bases its behavior on rules defined by an expert teacher; after this, a neural
network that learns from the student’s behavior when they are studying adjusts these rules. This way, the neural network
improves the teacher’s rules and, consequently, defines a learning strategy that is more adaptive and reactive to the
student’s profile. Thus, it is possible to offer the student a personalized and individualized education form. The model is
able to guide the student throughout the didactic transposition of contents, aiding the consolidation of desired competencies
established on educational propositions. This work shows the development process of the ITS, including the expert
guidance system and the hybrid system, which improves the expert rules from SOM neural network use. The obtained
results indicate that the application of hybrid technology in ITSs is feasible because, for defining the teaching strategies, it
incorporates the teacher’s knowledge and by neural network use, it assimilates the students’ learning process behavior. The
results show that proposed model has great agreement between the actions of the ‘‘ITS’’ and the students’ actions. The
model showed satisfactory performance when compared to other systems proposed in the literature that use connectionist
approach in its conception.

Keywords Intelligent tutoring systems  Hybrid ITS  Didactic transposition of contents  Artificial neural networks 
Self-organizing maps

1 Introduction

Since the 1950s, people use the computers in education to

support the teaching and learning process. Early, computer
& Francisco Ramos de Melo
systems used in education were classified as computer-as-
[email protected]
sisted instruction (CAI) [2, 28]. With the advances of
Sirlon Diniz de Carvalho
artificial intelligence (AI) techniques, a new paradigm of
[email protected]
educational systems was introduced, namely intelligent
Edna Lucia Flôres
computer-assisted instruction (ICAI) or intelligent tutoring
[email protected]
system (ITS) [32]. In the first generation of ITS, the sys-
Sandrerley Ramos Pires
tems had the ability to represent certain domains and adapt
[email protected]
to some students’ characteristics. ICAI systems could
Luiz Fernando Batista Loja
achieve a wide domain representation, thus allowing its use
[email protected]
even when the designer does not fully describe the
1
Instituto Federal de Goiás, Goiânia, Anápolis, GO, Brazil knowledge.
2
Universidade Estadual de Goiás, Anápolis, GO, Brazil In order to be considered intelligent, a ITS must behave
3 similarly to a human tutor. It also must offer a teaching
Universidade Federal de Uberlândia, Uberlândia, MG, Brazil
4
process which is adaptive, responsive, flexible and
Universidade Federal de Goiás, Goiânia, GO, Brazil

123
 Neural Computing and Applications

personalized [10]. Intelligent tutoring systems apply sev- assimilate knowledge from the content. The content is
eral AI techniques in order to enable different forms of related to the knowledge that is conveyed, is what the
learning as adaptive, contextual, directed and flexible, student will learn [8, 24, 25].
according to student’s profile [31]. ITS using symbolic Face-to-face interaction is still the oldest and most
paradigms of AI, and a mediation of the teacher, has shown widely used method to transmit knowledge. A key aspect
promising results [11, 27, 33]. Others paradigms as con- of this model, within the physical environment where the
nectionist and nonlinear paradigms have also been used in teaching–learning process takes place, is the simultaneous
the design of intelligent tutoring systems and showing presence of both the receiving (student) and the transmit-
satisfactory results [3, 19]. ting (instructor) agents. In this kind of education, the
To each one of these used techniques in an ITS, there are knowledge transmission process lies in the interaction
several open problems to reach a ‘‘perfect’’ system. When between teacher and student; the teacher guides such pro-
an ITS uses connectionist paradigms, it is necessary to cess according to his or her perception of the student’s
determine correct parameters to obtain a good system reactions [12]. Operational and economic limitations, along
performance. It is hard to set the number of layers, the with the impossibility of individualized contact between
number of neurons by layers and others parameters to the teacher and the several students, have resulted in seri-
achieve an appropriate performance. However, a frequent ous imbalances in the outcome of the teaching–learning
and complex problem is to adapt ITS neural network to process.
students’ skills. This issue comes out because ITS only has With the expansion of technology and the assumption
contact with students after the course starts. Using an ITS that part of the teaching–learning process may not require
with symbolic paradigm, it will behave by a prior knowl- the physical presence of teachers, educators and psychol-
edge inserted in the system. An expert can insert this ogists developed the field of research and educational
knowledge, and in this case, ITS will ignore the particular technology called ‘‘Distance Learning’’ (DL) [7, 13]. This
students’ skills. field of research is an approach without the limitations
Despite all problems, the benefits of using information imposed by a face-to-face teaching environment. DL
technology for aiding teaching activities have increasingly allows a greater dissemination of knowledge; thus, it pro-
motivated the interest for researches in virtual environ- motes the education popularization [15].
ments for education [o]. In this context, the use of com- In a face-to-face teaching environment, the teacher
puter has supported teaching in the classroom and in interacts with students during the presentation of content,
distance learning environments. Currently, virtual univer- identifies their difficulties and offers options to conduct the
sities are an actuality, with proposals of training, under- learning process. In DL environment, the interaction
graduate and postgraduate levels of education [27]. between teacher and student occurs throughout the process
Thus, this work’s goal is to propose and implement a of generating the educational contents [12]. Due to its
hybrid ITS model. This model will allow an expert teacher, asynchronous aspect, DL does not enable immediate
aided by a self-organizing map (SOM) artificial neural interaction between teacher and student when learning
network, create an optimized form to the didactic trans- problems arise. Thus, its delay in the teacher’s intervention
position of contents [6]. The propose is to use the good jeopardizes the student’s learning. The interaction between
characteristics of two stated approaches and, as contribu- the different elements involved in a non-face-to-face edu-
tion of this work, increase a retraining process of neural cational process has established disciplinary ties between
network to adapt ITS with particular students’ skills of a the Exact Sciences and the Humanities [12].
specific course. In a distance learning environment, interactivity is a
complicating factor. During the delivery of the contents,
there may be a time lapse before the teacher administers his
2 Virtual learning environments pedagogical intervention toward the student’s issues. It is
hard to apply general pedagogical interventions in conse-
The process of transmitting knowledge can be represented quence of the plurality of student skills and other typical
by the combined actions of teacher and student, with the situations of a distance learning environment. An learning
purpose of assimilating knowledge. Three elements take analytics about the practice in virtual learning environ-
part in this representation: the teacher, the student and the ments is important to reflect on the challenges of this
delivered content. Students are the interacting part to which environment [30].
all the effort of the process is directed, in order to develop This context requires from distance learning systems the
his or her skills. The teacher is the key agent of the learning capability of perceive students’ skills and changes itself to
process. He is responsible for giving the direction and offer the content in the best way. The use of artificial
organization of the actions that enable students to intelligence techniques can aid these systems to perform

123
Neural Computing and Applications

this kind of task. There are several works in the literature

with many proposals to improve ITS to realize students’
skills and react from this obtained knowledge. It follows
some of these approaches.
Raabe and Giraffa [27] proposed an alternative to the
traditional ICAI model, which improved the classic struc-
ture by making use of the Theory of Mediated Learning
Experiences. In this model, the teacher becomes an active
part in the decision process, since their actions are included
in the tutoring model.
An intelligent tutoring system using artificial neural
networks was developed and presented promising results Fig. 1 Traditional structure of an ITS with expert system
[5, 22]. These jobs demonstrated the feasibility of applying
connectionist AI techniques in the tutoring systems. Their material and for handling the student’s progress. Finally,
models differ from traditional ICAIs because they do not the control module is responsible for the overall coordi-
rely solely from the teacher’s knowledge for modeling the nation of the tutoring system. The teachers’ conceptual
tutoring system. In their approach, a neural network knowledge is the main information origin to build the
influences in tutoring system’s actions. The system uses the domain basis of ICAI systems using the symbolic AI
students’ behavior to generating patterns used in the neural paradigm [31].
network training stage. Despite the good results and the The didactic transposition proposed by Chevallard [6]
feasibility of using connectionist models, the techniques inspired the levels’ framework. This transposition asserts
proposed require the collection of extensive data for that to students assimilate the content, it is necessary to
training the neural networks. Furthermore, these models modify the knowledge, enabling the learning process.
depended on the adjustment of neural networks parameters Thus, the teacher should structure the content according to
to specify the best formats, such as network layout, number what is relevant to the desired competencies. This approach
of neurons. allows the distribution of contents along a timeframe to
Martins and Afonseca [20] presented an ITS model in organize a sequence of linear or nonlinear concepts and
which the decision-making uses reinforcement learning. In relationships.
that model, the learning occurs during use of the tutoring In another job [21], a questionnaire was utilized to
system by students. Although that system has shown good survey teachers’ actions from several educational institu-
results, the study session began without any prior knowl- tions. The goal of this survey was to map the best teacher’s
edge being available. decision to each identified situation according to the stu-
Markowska-Kaczmar et al. [19] presented a survey of dent’s response to a given exercise. In his work, the
the main techniques used for developing e-learning sys- research created five level of difficulties to the content, and,
tems with adaptive features; the SOM network appears according with the rules in its knowledge basis, this content
among these techniques due to its capabilities of grouping is presented by the students aiming maximize the learning
and classifying. process.
Another work presents the importance and different
methods in personalized learning [14, 18]. 2.2 Kohonen maps

2.1 Symbolic approach to intelligent tutoring Teuvo Kohonen developed the self-organizing map neural
network, or SOM, in the 1980s [16]. The topological map
The most traditional ICAI models use symbolic AI. These observed in the cerebral cortex inspires its design. In the
systems usually comprise in five parts: the domain basis, SOM network, neurons that are near one each other on the
the student model, teaching strategies, interface control map layout are responsible for similar functions, as occurs
module and the user (the student). Figure 1 shows as these in the brain of more highly evolved animals.
five parts interact. The self-organizing maps are composed by two layers,
The domain basis is the component that acts as the the input layer and a second layer that is the own classi-
expert; it is the origin of all knowledge presented in the fying map. In the input layer, there are several sensing
course. The student model represents the knowledge and neurons to perceive each signal from the external world.
cognitive skills of the user at any given time. The teaching This RNA model uses to training a competitive learning
strategies encompass the pedagogical model in itself. The algorithm, where the neurons on the output layer compete
interface is responsible for the presentation of instructional to define which one has the greatest similarity with the

123
 Neural Computing and Applications

presented pattern. However, there are no physical con- 2.3.1 Map topological preservation
nections between them. A neighborhood relationship
determines the map topology. There are many possible This metric describes if after the training, the network
metrics to determine the winning neuron. Yet, the most topology maintains the features of the training data set. If
usual way to determine the winner neuron is the Euclidean the network maintains these characteristics, it indicates that
distance. The one that has the smallest Euclidean distance network learned with success. A simple form to measure
in relation to the presented pattern [17] is the winner the topological quality of a SOM network is using the
neuron. average of the distance between the first and the second
In SOM neural network, weights are associated with winner neurons. Thus, the highest is the obtained result and
each neuron and the number of inputs defines the dimen- the lower is the global ordination of the network. Equa-
sion of the weight vector. For example, if one considers tions (1–3) show how to calculate this measure to a two-
three inputs for the colors red, green and blue, there will be dimensional Kohonen map.
a weight vector with three elements. 0 0
h ¼ maxðLvLv j;jCv  Cv Þ ð1Þ
The basic algorithm for training an SOM neural network 
does not have a high complexity and two phases comprise h; se h [ 1
s¼ ð2Þ
the training process: global organization and fine-tuning 0; otherwise
(convergence). The topological organization of the SOM
1X N
network occurs in the initial cycles; however, as it is a QT ¼ s ð3Þ
stochastic process, the accuracy of the map depends on the N i¼1
number of cycles during the convergence or fine-tuning.
where QT topological quality, Lv line of the winner neuron;
There is no specific rule for choosing the number of iter- 0
Lv line of the second winner neuron, Cv column of the
ations or cycles, but studies have shown that five hundred 0

cycles per neuron are enough to obtain good statistical winner neuron, Cv column of the second winner neuron,
precision [17]. Therefore, the cycles’ number depends on N number of samples in the data set training.
the number of neurons in the network. The ability to group A map topological preservation quality indicates only
similar patterns together makes the SOM network a good the map ordination state of the network. To measure the
decision-making tool for tasks that involve data classifying precision of the network answers, it is necessary other
and clustering, among others. measure called map precision.
This paper considers that navigation form or the learning
strategy defined by SOM neural network maps the students 2.3.2 Map precision
learning profiles. This is the main hypothesis of this work.
A traditional problem when one uses a neural network is This metric shows the accuracy of the neuron answer to the
to define the best network parameters to execute the net- training data set used in training process. A usual way to
work training, as number of the neuron, the number of measure the map precision in a Kohonen network is using
layers and others characteristics. Normally, the most the average of the Euclidian distance sum retrieved from
common approach is to try several configurations until each used sample in training data set. Thus, the lower is the
reach a good result, or at least, a configuration that can result, better is the map precision. Equation (4) shows how
solve the problem at hand. can be applied, using as base the trained network weights
and the training data set.
2.3 Kohonen quality metrics 1X N
QT ¼ xi  wji ð4Þ
N i¼1
As the neural network training is a stochastic process, it is
difficult to measure the quality of learning process. Nev- where xi i-th pattern presented to network; and wji winner
ertheless, after the training process, as tutoring system will neuron weight.
use the neural network, it is important to know the quality These measures allow the verification of the training
of the network learning. This work uses two quality mea- process quality, indicating if the chosen network configu-
sures proposed by Carvalho [4]. These measures verify the ration is enough to correct classification.
quality of network training. They are the map topological The unsupervised learning feature allows the SOM
preservation and the map precision. neural network to perform the classification of data sets not
related to their respective classes. The wide variety of
actions and attributes in a learning environment are pre-
sented as a complicating factor in establishing its classifi-
cation. In this work, the SOM neural network presented

123
Neural Computing and Applications

itself as a viable alternative to the task, enabling the teaching strategy. Figure 2 illustrates the basic format of
structuring of the environment data, relating the diversity this design. The guiding navigator is the part of the inter-
of data in appropriate classes to provide a custom and face that is responsible for providing guidelines for the
favorable learning environment to the student [19]. study activity.
Kohonen’s maps were used in this work because it is one of Each proposed course in that structure should have its
the most applied models for unsupervised learning appli- didactic units organized and distributed in several levels,
cations. It is a classic ANN and used in several studies each one aiming to develop different skills.
where the objective is to classify. Works such as A sequence of texts, arranged in five distinct levels,
[1, 9, 23, 29] deal with the analysis of student performance composes the teaching unit, in addition by two tests, an
using SOM networks. initial and a final. The principal levels among those five are
easy, medium and advanced. The other two levels are
auxiliary: frequently asked questions (FAQs) and exam-
3 Didactic transposition of contents guided ples. Each one of the levels has an exercise related to the
by the hybrid model didactic content and to the level.
The students begin the course by performing an initial
This paper presents a proposal of hybrid ITS model. In our test in order to measure their prior knowledge about the
proposed approach, rules mapped from teacher’s knowl- topic. After the initial test, the tutoring system chooses an
edge and an artificial neural network SOM model guide the appropriate difficulty level to the students according to
didactic transposition of contents. The neural network their knowledge degree. After the student study, the
operates by adjusting the teacher’s rules according to how defined level and answer the proposed exercise, the system
the students navigate through the educational contents. shows a navigation page. This page informs to the student
Two phases compose the process of constructing the the teacher’s suggestion to carry on the study process, or,
proposed model. In the first phase, the tutoring system alternatively, which are the possible steps to be taken by
works using only the teaching strategies based on the rules own student. Figure 3 shows the navigation screen.
defined by expert, the teacher. In the second phase, the The teacher’s suggestion, the first option in Fig. 3,
hybrid decision-making alters the pedagogic strategy cre- shows the tutoring systems decision. This option directs the
ated in first phase. The latter phase uses SOM artificial student toward another level within the same topic or, if
neural networks to refine the expert strategy. deemed appropriate by the system, toward another didactic
The first phase approach allows students to learn in a
guided manner, according to the rules of the teacher who
created the course. However, the students have the freedom
to decide about how to navigate over contents, and which
study strategy they will adopt. Concurrently, this model is
also a source of data for training the neural network used in
the hybrid model.
The hybrid guidance is determined according to the
characteristics of student who used the system in its first
phase, where the expert rules define the decision-making
process. The students’ behavior, while conducting their
study activities, offers information to the tutoring system to
adapt itself, resembling human interaction. Thus, the
hybrid model acts from a domain basis composed by expert
rules and by neural network rules.

3.1 Expert’s guidance in the didactic

transposition of contents

During the development of the hybrid model, the first

development stage consisted of an ITS in which the
didactic transposition of contents is guided by expert rules.
The structure of this approach is similar to hybrid system.
The main difference is that the teacher, who generated the Fig. 2 Structure of the tutoring system with expert and hybrid
didactic contents, is responsible for defining the entirely decision

123
 Neural Computing and Applications

Fig. 3 Tutoring system’s navigation options

unit. Even after the students visit other topic levels, if they 3.2 The use of hybrid guidance in the didactic
wish to advance to another unit, the ITS will always lead to transposition of contents
the most appropriate level according to each student’s
profile. The ITS with hybrid guidance used for the didactic trans-
At the beginning of the navigation, the tutoring system, position of contents follows the same organization of
based on the score of the initial test and on the rule spec- didactic units shown in Fig. 2, i.e., an initial and final test,
ified by the teacher, defines the most appropriate level to be easy, medium and advanced levels, FAQs and examples.
selected. For navigating through the remaining units, the The greatest difference between this model and the one
system determines the next level to be followed according shown in the previous section is the teaching strategy. In
to the student’s global performance. This performance the hybrid ITS model, the decision-making provided by the
score is defined by the average score of all answered SOM neural network complements the guidance decision-
exercises and the rules defined by the teacher. making that already exists within the expert model. Thus,
In the expert guidance model, the teacher is responsible the purpose of this network is to adjust the teaching strat-
for defining a teaching strategy that will guide the student egy that guides the students throughout the didactic trans-
throughout the didactic transposition of contents. This position that is best suited to their profile and performance.
means that the teacher should define the strategy for nav- Following the same rationale defined for the expert
igating to different units and through the different levels guidance model described in the section before, the hybrid
within a unit. model ITS also presents to the student an initial and final
Aiming to facilitate the teacher’s work in generating of test. However, in this model, the system’s guidance defines
the content, the proposed system suggests the criteria for the guidance process according to the decisions of both
local navigation or expert decision, based on the research system structures, the neural network and the expert
presented [22]. That research considers the same five levels knowledge base. In this case, the neural network operates
used in the structure of our proposed tutoring system, by adjusting the expert’s teaching strategy based on the
without, however, the evaluate process of the contents at learning experience of students that used the expert system
the end of each topic. before. The characteristics of texts and exercises remain
In our proposed approach, the teachers, as the ones in unchanged in the hybrid ITS, i.e., they are the same as in
charge of defining the contents for their courses, have the expert model. Thus, both systems may use the same
complete freedom to modify the expert guidance according interface layouts, colors, texts, etc.
to their own experience or specificity of the topic at hand. A single neural network is responsible for guiding the
Figure 4 shows a simplified map of the expert suggestions student’s navigation in the hybrid ITS. After reading the
depending on the student’s success or failure to a given contents of a didactic unit and answering its respective
local exercise. exercise, the tutoring system submits collected data to the
Therefore, if the student fails in medium-level exercise, neural network for the decision-making process to take
for instance, the tutoring system will perceive in the new place. The neural network guides the transposition between
classifying process and will create a sequence of possible content levels, whereas the expert decision guides the
suggestions to continue the study. transposition between different didactic units.
The proposed ITS, through the expert decisions descri- As occurs in the expert guidance ITS, every step of the
bed in this section, may be used immediately after the student that used the hybrid system is stored in a database.
creation of the course, using only that knowledge basis to The system updates the ITS neural network just after the
help the learning process. At the same time, the tutoring end of data collect process. Thus, the ITS uses the new
system, with its hybrid model, acts gathering data for neural network status to decide about the next step that
training its neural networks. Next section describes how student should be guided.
this process works. These data can also be used later to improve the per-
formance of the neural network if they are used as the data
set for a new training stage. The collected data permit to

123
Neural Computing and Applications

Fig. 4 Suggestion’s tree of local decision after an exercise result

analyze the students’ behavior along their studying neighborhood reducing tax, the neighborhood shape and
activities. the learning tax [17].
The hybrid ITS supports any course or content, but it is The proposed model in this work uses an algorithm to
necessary to use the same distribution structure levels that execute an automatic training. This algorithm minimizes
allow the didactic transposition. After the creation of a new the necessity of a neural network professional to define the
course in this style, one could insert it on the tutoring best configuration to SOM network. To define the ideal
system. Furthermore, the automation implemented for network structure, the algorithm training several bi-di-
training the neural networks enables immediate utilization mensional SOM network, each one with different config-
of the new course. The training of neural network can uration parameters. After this, the algorithm selects the best
happen at any time, without assistance from a neural net- network. This optimized network must have the best
work specialist. topological preservation map and precision map [4].
Aiming guarantee the generalization in the training pro-
3.3 Defining the best network configuration cess, the algorithm also defines the number of neurons
based in the numbers of training patterns. The tutoring
The neural network training process is a complex task, system repeats this process to each new course inserted in
independently from the adopted model. This task requires the system. This model of training and the tutoring system
from the engineer a high knowledge and experience to architecture allows the use of several different SOM net-
define the adequate configuration, which depending on the work in the system, one to each course. After the training
treated problem. In particular, in the SOM networks, some parameters definition and the network training, the system
factors are responsible to its performance after the training builds the neuron labeling process.
as the neuron number, the initial weight values, the

123
 Neural Computing and Applications

3.4 Neuron labeling process 3.5 Development environment

In its simplest form, the SOM neural network has only one This work used the RubyÒ technology to build both
layer. This layer shows which neuron is the winning one, structures, the hybrid system and the expert tutoring sys-
i.e., the one that most adapted to a given pattern. In this tem, which operate on the server side. This technology
work, each neuron must be able to choose one among five brings dynamism to web pages and allows access to the
different levels of guidance (five predefined classes). For database stored on a PostgreSQL service. The values of
instance, from the medium level of a particular didactic temporal and behavior variables (latency, satisfaction, etc.)
unit, students can visit the easy level of that unit, the are retrieved with JavaScript. This ensures greater accuracy
medium level of the next topic, the advanced level, FAQs and fidelity of the records because the JavaScript works on
or examples. the client side (browser), thus avoiding the delays that are
In order to enable neurons to make various decisions, inherent to the communication.
this work developed a labeling process. This process The choice of technology used for developing the
associates with each neuron a decision probability vector. described systems follows the philosophy of free software,
The algorithm shown in Fig. 5 illustrates the labeling thus minimizing investments with proprietary software.
process used in this work. In the proposed algorithm, all A Xen-based cloud computing solution hosts the systems,
neurons in the network have an updated label that indicates which enables server virtualization with high performance.
the next step. This is possible by insertion of a labeling Figure 6 shows a screenshot of the proposed intelligent
process. tutoring system.
This process associates with each neuron a vector that
contains the five indication levels. The labeling process
works by updating, at each neuron, of the level (label) that 4 Experiments and results
is equivalent to the next visit made by the student, which
generated that pattern. For conducting the experiment, this work developed two
This updating process adds at neuron’s indication point systems with similar software interfaces: an expert navi-
(equivalent to the forthcoming visit of the pattern), the gation system and a hybrid navigation system. The dif-
similarity value (Euclidean distance) between the presented ference between these two systems is the decision
pattern and the weights of the neuron. This process labels mechanism for choosing the more convenient level from
every neuron in the network; however, the neuron must the contents organization. The hybrid navigation system
indicate the level where similar patterns are most repeated applies the modeling techniques proposed and already
(strengthened points). described in this paper.
The labeling process occurs only after the training stage
of the neural networks, which is the moment when every 4.1 Data collection
mapped example is available to the network. The mapping
strengthens the next step that the neuron should indicate In order to compile the experimental results, this work
according to the next step chosen by the (successful) stu- created four data sets to both approaches, the expert system
dent that generated that pattern. and the hybrid guiding system. Fifty-four students from
early years of vocational courses in Mechanics, Computer
Science and Chemistry from the Institute Federal de Goiás,

Fig. 5 Labeling algorithm used

in the SOM network (for all
neurons)

123
Neural Computing and Applications

Fig. 6 Screenshot of the tutoring system

in the federal state of Goiás, Brazil, participated in those created for the experiment has as the theme the research
experiments. presented in this paper and information about the system’s
Before the students began navigating the systems to navigation modes.
generate the experiments data, they watched an explanation
about the main characteristics of each system. The main 4.2 Training set
goal of these explanations was to minimize problems the
students could face during their navigation. They also had This work used the data of twenty-two students to compose
information about the software, such as the structure, the the set of examples used for training the neural networks.
development environment and the average time for com- The initial and final tests consisted of fifteen questions each
pletion of the proposed course. Furthermore, the course one and resulted in 660 answer records. Each of these

123
 Neural Computing and Applications

records contains information about the navigation process,

including the number of correct answers, latency (time in
milliseconds) and the satisfaction level.
The course used for generating the training data set
contains fifteen didactic units structured into five levels,
where each level has an associated exercise. Visits to these
units resulted in 624 records related to reading texts and the
same number of answers to the exercises, totaling 1248
patterns or examples.
For each navigation record, the following information
was stored: latency in milliseconds, degree of satisfaction,
sequence or order of visitation of the didactic unit and Fig. 7 Navigation behavior chart for the expert tutoring system
success status of the exercise. Therefore, nine entries
compose the patterns; they are: text reading latency, degree
of satisfaction with the text, response latency for the correctly. Those students who followed their own guidance
exercise, degree of satisfaction with the exercise, local answered 76.68% of the exercises correctly. This demon-
performance or success status of the exercise, global per- strates that this work could not disregard the students’
formance or overall average performance in the course, the ability to choose their own learning strategy. These results
average score in the initial test, current didactic unit and also suggest that the neural networks can learn the stu-
current level. dents’ strategies and thus improve the teacher’s decision.
This work assumes that the decision of the next level or For instance, a given student profile can learn better with
topic in the study process is a consequence of the current examples and others can perform better when observing a
situation, since it will influence the student’s next step. set of questions and answers. In this experiment, the degree
Thus, the data that composes the training set for the net- of satisfaction of students that used the expert ITS was
work are restricted to situations in which the student’s 99.2%.
current decision (time t) has positively influenced the result From the results obtained in this experiment, this work
of the next studied item (time t ? 1). concluded that the expert model is able to provide guidance
This means that the collected data were only included in for the studying activity and that the student is able to make
decisions that will lead to success in the next level or proper decisions and these decisions can be used for cre-
didactic unit; the system excluded the last records of each ating new teaching strategies.
student, since those records always lead to the final test.
Using the described criterion to select the valid data, the 4.4 Hybrid model’s behavior
number of records qualified to compose the set of examples
used for training the network reduced to 902, including After collecting the data from the ITS with expert guid-
information on reading texts and the answers to exercises. ance, the next step was training the neural networks to
evaluate the hybrid model’s performance.
4.3 Expert tutoring system’s behavior The hybrid tutoring system trained some neural network
models in order to evaluate which is the best model to
As previously discussed in this paper, the structure devel- adapt to the profiles, according to the nine selected attri-
oped for the tutoring systems enables students to navigate butes, already described in this work. At the end, the bi-
to any level of each didactic unit, or in the worst case, at dimensional SOM model had the best behaving, yielding
least one level for each unit. The bar chart depicted in satisfactory accuracy and topological sorting. The bar
Fig. 7 shows the navigation behavior for the expert tutoring chart in Fig. 8 illustrates the results obtained from the
system. navigation guided by the hybrid ITS and guided by stu-
We can observe in the chart from Fig. 7 that students dents’ decision.
who accepted the expert tutoring system’s guidance The chart in Fig. 8 shows that students who accepted the
obtained 52.99%, with a 40.7% success rate and 12.29% of guidance provided by the hybrid ITS were 72.79% of them,
failure at the exercises. The students that navigated through with 59.31% of correct answers to the exercises and
the system according to their own decisions obtained 13.48% of incorrect answers. The students who made their
47.01%, with 36.05% of success and 10.96% of failure in own navigation decision were 27.21%, with 21.09% of
the exercises. correct answers to the exercises and 6.12% of incorrect
These results also indicate that students who followed answers. This indicates that students who followed the
the expert guidance answered 76.80% of the exercises guidance provided by the hybrid ITS answered correctly

123
Neural Computing and Applications

based on students’ behavior. However, the greatest

advantage of our model is that it has prior expert knowl-
edge, which makes it possible to utilize the system, while it
collects continuously data for the construction of new
teaching strategies.
During one of the sessions of data collection, technical
problems affected the Internet link, and then we decided to
interrupt the session and continue in the next day. Although
the system supports resuming the course from the point
where it interrupted, we decided to disregard the data
related to this session. This decision based on the appre-
hension that the discontinuity could generate different or
Fig. 8 Navigation behavior chart for the hybrid tutoring system noisy patterns. Because of this, the experiment selects data
from twenty-two students to constitute the training set, and
81.48% of the exercises. Those students who followed their data collected from twenty-two other students for evalu-
own guidance achieved a success rate of 77.51% in the ating the performance of the hybrid intelligent tutoring
exercises, which is almost the same obtained result in system, totaling forty-four students.
before experiment.
The results obtained from the hybrid ITS also indicated 4.6 A Second experiment with proposed ITS
that the degree of satisfaction was 92.48%, having the
expert ITS as reference. It is important to note that both the Another important experiment carried out with the ITS
expert and hybrid systems utilized the same environment proposed here is described in the work of Pires et al. [26].
for data collection and kept the same structure, including In that work, the ITS is compared with a traditional dis-
layout, text and exercises. For the hybrid system, the tance learning system, using the Moodle software (Moodle
decrease in satisfaction may have been due to fatigue 2019). Pires created a course about labor law to run in the
caused by the increase in time of study, which nearly ITS and in the Moodle. A class with thirty students was
increased twofold. This is due to the nonlinear nature of the divided in two groups, with fifteen students with ITS and
neural networks, which for the hybrid ITS causes a greater fifteen with Moodle.
number of levels and topics being suggested by the system, After completing the course, the ITS applies a final
which ultimately keeps the student navigating the content assessment and presents the student’s performance auto-
for a longer time span. matically. For those students that tried traditional approach,
This work concluded, from the results obtained in this with Moodle, an assessment with the same ITS’s questions
second experiment, performed using the hybrid ITS, that was applied after they concluded the course. Figure 9
the hybrid system was able to learn from the behavior of presents a summary of the obtained results in both course
students for establishing a new teaching strategy. The modalities. Figure 9a shows that the numerical results are
increased value of acceptance or adherence to the system’s almost the same, and, by standard deviation, there are a
suggested guidance demonstrates this conclusion. major homogeneity in the traditional approach. Figure 9b
presents that the three best results are from students who
4.5 Comparison of the results took the course through the intelligent tutor system.
The numerical similarities obtained in the experiment
By comparing the results obtained in the experiments suggest that both approaches are suitable to the learning
conducted with the expert and hybrid systems, we could process. These same similarities were identified in the work
observe that there was a greater tendency for the students of Carvalho [4]. However, some non-numerical observa-
accept and follow the guidance provided by the hybrid ITS. tions indicate that the process using the tutor system brings
This guidance also provided higher success rate in exer- a series of gains that are not numerically measurable.
cises related to content levels specified by the hybrid sys- During the courses, students were observed to assess their
tem. The results also suggest that this model provided interest in conducting studies, productivity in each of
greater agreement between the system’s guidance and the approaches and difficulties with handling the technology.
students’ preferences, i.e., a reconciliation between the The main observations made were:
teaching strategy and the study strategy.
(1) Students had more difficulty in dealing with the
Comparing the proposed model with the work of other
interface of the intelligent tutor system. It is believed
researchers [5, 22], it is possible to verify that both models
that this difficulty was caused to a greater number of
have the ability of learning and defining teaching strategies

123
 Neural Computing and Applications

5 Conclusions

This paper proposed an intelligent tutoring system model

capable of guiding the didactic transposition of content.
The system creates the teaching strategies based on self-
organizing maps (SOM) neural networks and on knowl-
edge of the expert teacher.
Thus, it is possible to use a course, created using this
hybrid model, immediately after its creation. Initially, the
tutoring system issues its reactions based on the expert
teacher’s knowledge and, as the system is continuously
used, it refines this knowledge gradually based on the
decisions of the neural model. The neural model acquires
its learning according to the behavior of students,
approaching its actions from the students’ actions.
To avoid the necessity of a neural network professional,
the proposed model uses two criteria to do a neural network
automatic training. These criteria are the map topological
preservation and the map precision. They have shown to be
able to define the best configuration to a SOM neural
network.
Fig. 9 In a the mean and homogeneity of final assessment. In b the The results obtained with the expert tutoring system
bests students result suggest the viability of the proposed model, especially for
interactions needed to conclude the course and the demonstrating that students are capable of defining their
high number of options for the student to navigate study strategies.
the course. Experiments with the constructed hybrid tutoring sys-
(2) The level of motivation and attention during the tem, for the didactic transposition of contents, show its
study was higher in the course with the ITS, since the viability. This is due to its teaching strategy provoked a
constant interactions held the student’s attention. greater acceptance of the system’s directions and resulted
(3) In the traditional course, when accessing a text, the in higher success rates for the exercises.
student used to feel bored with the high number of This work concluded that this model is more adaptive to
contents pages. In the tutor, although the student had the student’s characteristics. Comparing this work with
the same amount of information, because it is others connectionist models, we can observe that proposed
fragmented, students did not perceive this easily. model works without the requirement of extensive data
(4) The need for student support to manipulate with the collection and without many neural network trainings.
environments was significantly greater in the course Moreover, differently of others model, the retraining pro-
with the intelligent tutor system. Nevertheless, all cess of neural networks occurs independently if the users
students involved in the test were already in the habit define the initial configuration to the neural network.
of using Moodle. Furthermore, the use of the hybrid tutoring system also
(5) Withdrawing the time spent by the student to becomes a source of data to improving the teaching
understand the intelligent tutor’s interface, it is strategies of the model itself, further enhancing the pro-
observed that the time spent in this approach is vided guidance for the didactic transposition of contents.
significantly less than in the traditional environment.
Acknowledgements Funding was provided by Fundação de Amparo à
From the teacher’s point of view, preparing the course is Pesquisa do Estado de Goiás (Grant No. 12345).
harder in ITS, because the same subject is written in three
levels of difficulty and two more additional perspectives. Compliance with ethical standards
On the other hand, the content generated results in more
Conflict of interest The authors declare that they have no conflict of
efficient tutoring.
interest.
Finally, that work concludes that the proposed ITS,
although more difficult to use, has important advantages
when compared to other options. These advantages justify
the use of ITSs instead of another approach.

123
Neural Computing and Applications

References 19. Markowska-Kaczmar U, Kwasnicka H, Paradowski M (2010)

Intelligent techniques in personalization of learning in e-learning
systems. Stud Comput Intel 273(2010):1–23
1. Alias UF, Ahmad NB, Hasan S (2015) Student behavior analysis
20. Martins W, Fonseca UR, Nalini LEG, Gomes VM (2007) Tuto-
using self-organizing map clustering technique. ARPN J Eng
riais Inteligentes Baseados em Aprendizado por Reforço: Con-
Appl Sci 10(23):17987–17995
cepção, Implementação e Avaliação Empı́rica. Anais do SBIE.
2. Anderson JR, Boyle CF, Reiser BR (1995) Intelligent tutoring
Workshop em Informática na Educação (SBIE) 2007 XVIII
systems. Science 228(1):456–462
Simpósio Brasileiro de Informática na Educação-SBIE-
3. Cabada RZ, Barrón ML, Garcia AR (2011) EDUCA: a web 2.0
Mackenzie
authoring tool for developing adaptive and inteligente tutoring
21. Melo FR (2012) Modelo Neural por Padrões Proximais de
systems using a Kohonen network. Expert Syst Appl
Aprendizagem para Automação Personalizada de Conteúdos
38:9522–9529
Didáticos. Ph.D. dissertation, FEELT, UFU, Uberlândia, MG,
4. Carvalho SD (2013) Modelo Hı́brido de Sistema Tutor Inteligente
Brazil
Utilizando Conhecimento do Especialista e Mapas de Kohonen
22. Melo FR, Flôres EL, Carvalho SD (2011) Multilevel content’s
com Treinamento Automatizado. Universidade Federal de
structure for personalization in conexionist intelligent tutor sys-
Uberlândia, Tese de Doutorado
tems. In: 8th International conference on information systems and
5. Carvalho SD, Melo FR, Martins W, Nalini LEG, Flôres EL
technology management, 2011, São Paulo. Proceedings of 8th
(2008) Sistemas Tutores inteligentes baseados em mapas auto-
CONTECSI. São Paulo, TECSI EAC FEA USP
organizáveis. In: VI CEEL: Conferência de estudos em engen-
23. Nohuddin P, Zainol Z, Nordin A (2018) Monitoring students
haria elétrica 2008, 2008, Uberlândia-MG. VI CEEL 2008.
performance using self organizing map trend clustering. ZUL-
Uberlândia-MG, pp 21–26
FAQAR Int J Def Sci Eng Technol 1(1):50–56
6. Chevallard Y (1998) La transposición didáctica del saber sábio al
24. Patten JV, Chao CI, Reigeluth CM (1986) A review of strategies
saber ensen * ado. Tradução de Cláudia Gilman, 3rd edn.
for sequencing and synthesizing instruction. Rev Educ Res
Buenos Aires, Aique, p 196
56(4):437–471
7. De Vaney A, Butler RP (2001) Voices of the founders: early
25. Phelan KC, Mendoza-Diaz NV, Mathews S (2002) An example
discourses in educational technology. In: Em Jonassen DH (ed)
of converting a traditional course into distance learning: an open
The handbook of research for educational communications and
discussion. In: 9th Annual international distance education con-
technology. The Association for Educational Communications
ference, Austin, Texas, USA
and Technology, Bloomington
26. Pires DGF, Pires SR, Lopes CR (2016) Análise comparativa entre
8. Fontenla J, Caeiro M, Llamas M (2010) A SOA architecture to
o uso de ambientes tradicionais de EAD e de sistemas tutores
improve the tailorability and extensibility of e-learning systems.
inteligentes: preparação, elaboração, aplicação e resultados, Atas
IEEE Latin Am Trans 8(2):135
CIAIQ2016: Investigação Qualitativa em Educação, Porto.
9. Forradellas MTS, Barberà-Mariné MG, Bariviera AF, Garbajosa-
CIAIQ 1:779–788
Cabello MJ (2012) Advantages of using self-organizing maps to
27. Raabe ALA, Giraffa LMM (2006) Uma Arquitetura de Tutor para
analyse student evaluations of teaching. Fuzzy Econ Rev
Promover Experiências de Aprendizagem Mediadas. In: XVII
XVII(1):53–71
SImpósio Brasileiro de Informática na Educação-SBIE2006,
10. Giraffa LMM, Viccari RM (1998) The use of agents techniques
Brası́lia-DF. v. 1. pp 589–598
on intelligent tutoring systems. In: Proceedings of 18th interna-
28. Rosenberg MJ (2001) E-learning strategies for delivery knowl-
tional conference of the Chilean computer science society,
edge in the digital age. McGraw-Hill, New York
Antofagasta, Chile, 1998, pp 78–83
29. Saxena K, Jaloree S, Thakur RS, Kamley S (2017) Self-orga-
11. Hafidi M, Bensebaa T (2013) Design and evaluation of an
nizing map (SOM) based modelling technique for student aca-
adaptive and intelligent tutoring system by expert system. Intel
demic performance prediction. Int J Future Revol Comput Sci
Decis Technol 7(253–264):253. https://doi.org/10.3233/IDT-
Commun Eng 3(9):115–120
130167
30. Stewart C (2017) Learning analytics: shifiting from theory to
12. Horton WK (2000) Designing web-based training: how to teach
practice. J Empower Teach Excell 1(1):10
anyone anything anywhere anytime. Wiley, Hoboken
31. Viccari RM, Giraffa LMM (1996) Intelligent tutoring systems:
13. Jaques PA, Seffrin H, Rubi G, Morais F, Ghilardi C, Bittencourt
functional approach x agents approach. In: Proceedings of 13th
AII, Isotani S (2013) Rule-based expert systems to support step-
Brazilian symposium on artificial intelligence, Curitiba, Brazil,
by-step guidance in algebraic problem solving: the case of the
p 235
tutor PAT2Math. Expert Syst Appl 40:5456–5465
32. Wenting Ma OO, Adesope JC, Nesbit QL (2014) Intelligent
14. Jevsikova T, Berniukevicius E, Kurilovas E (2017) Application
tutoring systems and learning outcomes: a meta-analysis. J Educ
of resource description framework to personalise learning sys-
Psychol 106(4):901–918
tematica review and methodology. Inf Educ 16(1):61–82
33. Zarandi MHF, Khademian M, Minaei-Bidgoli B, Türkşen IB
15. Jonassen DH (2001) The handbook of research for educational
(2012) A Fuzzy expert system architecture for intelligent tutoring
communications and technology. The Association for Educa-
systems: a cognitive mapping approach. J Intell Learn Syst Appl
tional Communications and Technology, Bloomington
4:29–40
16. Kohonen T (1982) Analysis of a simple self-organizing process.
Biol Cybern 44:135–140
17. Kohonen T (2001) Self-organizing maps. Springer, Berlim Publisher’s Note Springer Nature remains neutral with regard to
18. Kurilovas E (2016) Evaluation of quality and personalization of jurisdictional claims in published maps and institutional affiliations.
VR/AR/MR learning systems. Behav Inf Technol
35(11):998–1007

123