1931

IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 17, 2024

Predicting Student Performance in a Programming

Tutoring System Using AI and
Filtering Techniques
Miloš Ili´c , Goran Kekovi´c , Vladimir Miki´c , Katerina Mangaroska , Lazar Kopanja , and Boban Vesin

Abstract—In recent years, there has been an increasing trend of Digital Object Identifier 10.1109/TLT.2024.3431473
utilizing artificial intelligence (AI) methodologies over traditional the student’s behavioral characteristics are significant in
statistical methods for predicting student performance in learning interpreting their interaction with the e-learning system [1],
contexts.Notably,manyresearchershaveadoptedAItechniques
[2]. Educational data generated through e-learning systems can
without conducting a comprehensive investigation into the most
appropriate and accurate approach to employ. Additionally, be effectively processed using machine learning (ML)
determining the optimal input parameters for each AI technique techniques to obtain insights into student performance and
remains a pertinent question in this domain. This study employs learning outcomes [3], [4]. Effective prediction techniques and
machine learning (ML) and artificial neural networks (ANN) to features could assist educators and teachers in learning design
predict student grades within a programming tutoring system. The
and in creating appropriate instructional content to help
experiment involved university students whose interaction data
with the e-learning system were analyzed and used for predictions. learners study based on projected results [5], [6]. For example,
By identifying the structural relationships between the properties Lykourentzou et al. [7] clustered students into groups based on
of the input data, this research aims to determine the most efficient their predicted performance levels, allowing the instructor to
AI method for accurately predicting student performance in e- select educational materials more suitable to the skills and
learning systems. The structure of the input data in these systems
capabilities of each group of students. Also, predicting student
is described by variables related to individual student activities, so
correlations between variables were a natural starting point for performance can enable teachers to prevent students from
further theoretical considerations. In this manner, by applying a dropping out before final examinations, identify those who
filtering technique based on the minimum redundancy–maximum need additional help, and boost an institution’s ranking and
relevance (mrMR) criterion, it was shown that correlations among prestige [8].
predictors and between predictors and the target variable play a
Artificial intelligence (AI) methods for prediction offer
significant role in defining the appropriate model for predicting
student grades. The results showed that ANN (the Levenberg– many advantages over traditional statistical methods. They can
Marquardt algorithm with Bayesian regularization) outperformed managecomplexnonlinearrelationshipsindataanddiscovercom
MLmethods,achievingthehighestpredictionaccuracy.Theresults plicated patterns and trends [9]. AI methods are flexible and
obtained from this study can be of great importance for learning adaptable as they can learn from new data, enabling them to
technologies engineering and AI in general.
enhance their prediction ability and performance continuously
Index Terms—Artificial intelligence (AI), artificial neural [10]. The
networks (ANNs), e-learning, machine learning (ML), advantageofAImethodsisalsoinpredictionaccuracycompared
programming tutoring systems. to statistical methods [11].
Artificial neural networks (ANNs) as an AI technique are
I. INTRODUCTION
proven to be an efficient method for forecasting learners’
P PREDICTING student performance is an essential topic in
e-learning and education in general. In e-learning systems,
performance in e-learning environments [12], [13]. In addition
to ANN, ML methods are also widely used for this purpose
Manuscript received 24 July 2023; revised 24 February 2024 and 28 April
[14]. ML techniques are generally faster, which can be very
2024; accepted 16 July 2024. Date of publication 24 July 2024; date of current important for a large dataset [15]. However, ANNs are very
version 13 August 2024. This work was supported by the UTFORSK project of powerful in terms of their ability to learn complex patterns and
the Norwegian Directorate for Higher Education and Skills (HK-dir), under
Project UTF-2021/10186. (Corresponding author: Miloš Ili´c.)
identify hidden relations within the data using collections of
This work involved human subjects or animals in its research. Approval of all simple, trainable mathematical functions [16] and, thus, are
ethical and experimental procedures and protocols was granted by the used to solve more complicated problems than ML techniques,
Norwegian Agency for Shared Services in Education and Research-SIKT
such as natural language processing tasks [17] and computer
(former Norwegian Centre for Research Data - NSD).
Miloš Ili´c, Goran Kekovi´c, Vladimir Miki´c, and Lazar Kopanja are with vision tasks [18]. Also, ANNs are free from the limitations of,
the Faculty of Information Technologies, Alfa BK University, 11000 Belgrade, for example, regression or linear discriminant analysis and are
Serbia (e-mail: [email protected]; [email protected]; vladimir. often superior to ML techniques [19], [20], [21].
[email protected]; [email protected]).
Katerina Mangaroska and Boban Vesin are with the School of Business, Even though the data structure influences the selection of the
University of South-Eastern Norway, 3184 Vestfold, Norway (e-mail: katerina. optimal AI method, most of the researchers who designed
[email protected]; [email protected]). models for student performance prediction in e-learning

Authorized licensed use limited to: K. Ramakrishnan Health and Educational Trust. Downloaded on November 05,2024 at 05:08:31 UTC from IEEE Xplore. Restrictions apply.
1939-1382 © 2024 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.
 1932 IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 17, 2024

environments did not give enough consideration to this matter, II. RELATED WORK
as they only randomly chose techniques without previous data Predictive techniques of student performance in e-learning
analysis [22], [23], [24], [25], [26]. Without the data analysis, havebeenexploitedinanumberofrelatedresearchworks.These
researchers are unable to fully exploit the potential and techniques were used for different purposes, such as providing
capacity of AI methods and are unable to achieve maximal early insights about student results and achievements [31],
prediction accuracy [27]. Data analysis can enable the [32], [33]; identifying student types based on different
selection of input parameters, increasing the accuracy of the AI performance, engagement, and behavior levels [34], estimating
method [28]. In addition, descriptive analysis, as part of student satisfaction [35] and engagement [36], improving
statistical analysis, feedback for students from the tutoring agent to enable a
includesavisualrepresentationofvariables,calculationofmean personalized experience for each student [37], recognizing
values (or medians), and standard deviations, which gives the student difficulties [38], identifying students at risk [39], [40],
researcher a broader picture and enables a better choice of AI [41]; early alerting and advising future students [42], [43]; and
method for prediction [29], [30]. Also, correlations can include enhancing the quality of learning and student learning
or exclude an entire spectrum of ML methods. When ML experience [44].
methods become excluded, ANNs are the only logical option
with no restrictions regarding the properties of the input A. Benefits of Student Performance Prediction
variables. That is why initial data analysis should become the
standard when choosing a prediction technique. For these Generally, the benefits that student performance prediction
reasons, the focus must be on finding structural connections can bring to e-learning are numerous, benefiting teachers,
between the properties of the input data. That also determines students, and institutions alike.
the most favorable AI method for forecasting learner For teachers, learning performance prediction can grant
performance in e-learning environments and predicting valuable insights about students. Predicting student grades is
difficulties in e-learning courses. essential in the learning process, as it can help teachers grasp
The goals of this study are to: the learner’s full potential and give them enough time to take
1) investigate the correlations between particular student remedial actions [45]. It can assist teachers for the purposes of
activities so as to determine the optimal set of input learning design and in creating appropriate educational material
predictors for AI methods of predicting student grades; for students [5]. Also, it can assist them in identifying students
2) identify the most efficient method of AI for predicting who require additional help and support and prevent them from
studentgradesbasedontheresultsofcorrelationsbetween droppingoutbeforefinalexaminations[8].Thestudyoflearning
particular student activities. performance prediction provides a basis for teachers to modify
Studieswereconductedwithuniversitystudentstoachieveset and adapt their teaching approach for students experiencing
goals. The log data about their interactions with the e-learning difficulties by predicting their performance on upcoming tests,
system were analyzed and used to predict student performance. reducing the risk that students will fail to complete the course,
The first step for these experiments was the statistical analysis and ensuring the quality and standard of online learning [44].
of the predictors. Based on the results of this analysis, Students can study based on projected results if teachers use
appropriate AI methods were employed to create a model for prediction results adequately and create appropriate
predicting the students’ final performance (their grades). The instructional content according to the projected results [5].
filtration technique [minimum redundancy–maximum Prediction
relevance (mrMR) algorithm] was used to determine the cansupportstudentsthroughtheirchoiceofstudyplans,learning
optimal set of predictors, which increased the accuracy of the strategies, and learning tactics suited to their abilities [46]. Also,
following ML methods: support vector machines (SVM), k- it can assist students in choosing the appropriate course for them
nearest neighbors (kNN), and decision tree (DT). ANN, on the based on their performance in previous exams [47].
other hand, achieved a satisfactory level of accuracy without Student performance prediction can also provide important
the use of input parameter selection. information that institutions can use to make timely decisions,
The rest of this article is organized as follows. Section II changes, and modifications, leading to better student outcomes
presents the related work relevant to this study. Section III [48]. Based on these insights, institutions might establish a
presents the methodology used. Section IV presents the study strategic plan for the development of quality education [49].
results. Section V discusses the results and also presents the Predicting student performance can provide institutions with a
implications and limitations of the study. Finally, Section VI multitude of benefits that can increase student retention rates,
concludes this article and provides suggestions for future enableeffectiveenrollmentmanagement,empoweralumnimanag
research. ement, improve targeted marketing, and enhance the overall
effectiveness of institutions [50].
In that regard, student performance prediction can be
considered an effective method for improving and enhancing

Authorized licensed use limited to: K. Ramakrishnan Health and Educational Trust. Downloaded on November 05,2024 at 05:08:31 UTC from IEEE Xplore. Restrictions apply.
 ILIC et al.: PREDICTING STUDENT PERFORMANCE IN A PROGRAMMING TUTORING SYSTEM USING AI AND FILTERING TECHNIQUES´ 1933

online education, with benefits for teachers, students, and and computational performance, affect the choice of prediction
institutions, respectively. technique. In that way, by merely randomly choosing prediction
techniques, researchers cannot fully exploit the capabilities of
B. Prediction Techniques Used in E-Learning AI and achieve maximum prediction accuracy.
Many studies present models for predicting student In the literature, there are many papers using different
performance in e-learning environments using ANN or different approaches that are novel or not standard for learner
ML techniques, and most of these studies cover multiple performance prediction. For example, Jang et al. [41] predicted
prediction techniques, which are then used for comparing student performance in an online learning environment using
results (see Table I). The most common ML techniques found in ML and explainable AI. Jiao et al. [66] used genetic
the literature are (DT, naïve Bayes (NB), SVM, kNN, random programming to develop a quantitative prediction model that
forest (RF), logistic regression (LR) and linear regression [14]. outperformed ANN
When the prediction of student performance in e-learning is onthetrainingdataandhadcomparableresultswithANNonthe
addressed, most papers focus on the prediction of academic testing data while surpassing SVM in both training and testing
TABLE I data. Rakic et al. [68] used a multimethod approach for data
analysis (i.e., social network analysis, k-means clustering, and
linear regression), which provided more relevant results. Wang
et al. [62] developed a predictive model by utilizing
convolutional gated recurrent unit and explainable neural
networks, with this approach performing favorably compared
with similar methods. Sweta et al. [26] predicted learner
performance in an e-learning system using learning analytics.
Saleem et al. [56] proposed a model demonstrating an
intelligent decision support system that provides a complete and
effective method for forecasting student performance via ML
algorithms.
Models for predicting learner performance presented in
papers from Table I reported a wide range of accuracies, such
as < 80% [57], [61], [65], those with 80%–90% [22], [23], [25],
[51], [52], [56], [63], [64], [70], then higher accuracies, such
as 90%–
performance, i.e., their grades [60], [64], [74], [75]. Some 95% [1], [24], [54], and those with highest accuracies > 95%
authors categorize student grades into classes (for example, [59], [69], [71], [72], [77], where in this last group, one paper
good, average, and bad) and then use those classes as predicted can be singled out that reports the accuracy of 100% in which
variables [65]. However, authors often use this method of a hybrid model for learner grade prediction was developed,
prediction in order to acquire better results than those they with a generalized linear model and ANN [72].
would have obtained if they had predicted exact grades, as In the field of learner performance prediction, researchers
converting grades into two or three classes can be easier for are making a constant effort to ensure greater prediction
prediction. For example, Hashim et al. [8] did not exceed an accuracy. To obtain better results in accuracy, several studies
accuracy of 68.7% with grades, but they succeeded in achieving proposedhybridpredictivemodels,forexample,adaptiveneurofu
an accuracy of 88.8% when transforming grades in two classes. zzy inference system [69], neural networks based on particle
Also, many papers focus on predicting whether a student will swarm optimization [71], and a generalized linear model with
pass or fail the course [52]. In courses, it is also common to ANN [72]. Other efforts exist that use ensemble methods, such
developmodelsthatmakeweeklypredictionsinordertoprovide as bagging, boosting, stacking, and voting to improve
early predictions [41]. prediction accuracy [1], [52], [56], [58].
There are many standard approaches that authors use to
predict learner performance. The most common are developing
predictive models using only ML algorithms [8], [65]. Also, C. Filtering Techniques for Prediction Purposes
many authors have developed prediction models using only Filtering techniques are one of the approaches that can
ANN [20], [73], [74], [75]. There are many articles dealing only potentially lead to increased accuracy [78]. These techniques
with comparisons of prediction techniques in order to determine are based on correlation and statistical analysis. They are used
the best among them [23], [76]. However, there are to obtain an optimal set of predictors that can potentially lead
manystudiesinwhichauthorsjustrandomlyusesomeprediction to maximum prediction accuracy. Using filtering techniques
techniques in order to obtain and present some results without not only increases prediction accuracy but can also simplify
explaining why these techniques were chosen [22], [24], [25], prediction models, decrease the time required for their
[26]. Numerous aspects, such as data structure, target variable, computation, and prevent overtraining, all as a result of
Authorized licensed use limited to: K. Ramakrishnan Health and Educational Trust. Downloaded on November 05,2024 at 05:08:31 UTC from IEEE Xplore. Restrictions apply.
 1934 IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 17, 2024

reducing the number of predictors [61]. In addition, by for performance prediction. Authors who used both ANN and
selecting relevant variables, the model’s interpretability, ML
generalizability, and operational efficiency will also improve techniquesinthesameresearchtopredictstudentperformancein
[27]. Filtering techniques arepart of the input variable selection e-learning and used these results to compare accuracy
(IVS) techniques, and many authors use these techniques [1], indicated that the ANN models produced more accurate
[58], [65], [72] for predictor selection predictions in a significant number of cases. For example,
orotherIVStechniques,suchaswrapper[79]orembedded[80]. Arumugam et al. [51] used a DT (J48), a random tree, a
Compared to other IVS techniques, filtering techniques are multilayer perceptron neural network, and a radial basis neural
faster and more scalable, while they are also independent from network for this task, and the radial basis neural network
classifiers [81]. The most commonly known filtering achieved the highest accuracy. Su et al. [63] utilized ANN, RF,
techniques are the mrMR algorithm, chi-squared test, Pearson SVM, and LR, where ANN outperformed ML techniques.
correlation coefficient, information gain, etc. [82]. Rivas et al. [57] employed ANN, DT, RF and XGBoost, and
Applying filtering techniques to ML can greatly influence achieved the highest accuracy with ANN. Kokoç and Altun
prediction accuracy, as numerous studies have reported such [60] used NB, kNN, SVM, RF, ANN, and CN2 algorithm,
results [1], [27], [52], [67]. There are also a lot of papers in the where among the implemented models, ANN performed better
literature that did not include any filtering technique to get the then ML techniques. Santamaría-Lopez et al. [64] utilized
optimal set of predictors for ML techniques and instead ANN, RF, XGBoost, SVM, KNN, and LR, with ANN
included all variables; thus, they did not take advantage of the achieving the best results.
opportunity to enhance the prediction accuracy [8], [22], [23], The view that ANN achieves greater accuracy in data
[25], [51], [59]. However, most of the papers in the literature processing than ML cannot be generalized. Several studies
that included filtering techniques for feature selection showed presented that ML techniques outperformed ANN or had the
only those prediction results with selected variables, and they same results as ANN in predicting student performance. For
did not include the accuracy results of ML techniques with all example, Emirtekin et al. [59] used kNN, ANN, RF, SVM, NB,
variables [24], [54], [61]. Without testing ML algorithms with and LR, with kNN having the highest accuracy. Alsharhan et
and without variable selection using filtering techniques, it is al. [22] employed ANN, DT, and NB, and achieved the highest
not possible to confirm that the chosen variables do, in fact, average accuracy with NB. Sriram et al. [61] used ANN, SVM,
improve the accuracy of the model. Rarely, filtering techniques and linear regression, with ANN and SVM having the same
for ML might have the reverse effect, i.e., not increasing or highest average accuracy.
decreasing accuracy [83]. For these reasons, it is also essential Bearing in mind what was mentioned above, further studies
to perform a test with all parameters selected to determine how are needed to determine the specific cases in which ANN
much prediction accuracy has improved, if any at all. In that outperforms ML and vice versa.
way, readers will have a better insight into the percentage of
improvement when filtering techniques are applied. E. Type of Data Used for Prediction Model Building in E-
ApplyingfilteringtechniquestoANNisusuallynotnecessary. Learning
ANNs have no input parameter restrictions, as in the case of Various features are used for prediction and can be divided
ML methods [84]. That is why better results can be achieved into three main groups: demographic data, educational data, and
with different e-learning features.
ANNwhenusingallvariablesaspredictors[20],[70],[71],[74], Demographic data include gender, age, place of birth,
[75]. This cannot be generalized because there are some studies nationality, parent information [1], [59], etc. Examples of
that actually achieved better results with ANN when applying educational data are grades, overall score, homework, final
filtering techniques [63]. exam scores, faculty department, level of education, semester,
Considering the previously mentioned facts, ML can greatly attendance, absentee days [24], [25], [59], etc. E-learning
benefit from using filtering techniques to select input variables features can
for prediction purposes, whereas ANNs are more likely to additionallybecategorizedbytheirtypes:clickstreams(number of
achieve better results without them. clicks, mouse click left, mouse wheel click, mouse click right,
mouse movement) [52], time features (time spent on the system,
D. Predicting Student Performance—ANN versus ML time spent per task, average time on the task, start time, end
time, idle time) [61], [69], used educational resources (visited
ANNs are more robust compared with ML when it comes to
educational materials, video material, syllabus, topics, quiz
predicting student performance and can show superior
training) [1], [59], [71], success features (number of attempts,
accuracy over ML prediction algorithms [19]. ANNs are also
number of failures, number of successes, online quiz results,
much more scalable than ML techniques and can solve more
online test scores) [59], [73], and used services (forum, wiki,
complex problems [85].
glossary, announcements, messages, chat) [44], [71].
Most of the available studies have focused either on the use
of ANN [20], [73], [75] or on ML techniques [44], [53], [54]
Authorized licensed use limited to: K. Ramakrishnan Health and Educational Trust. Downloaded on November 05,2024 at 05:08:31 UTC from IEEE Xplore. Restrictions apply.
 ILIC et al.: PREDICTING STUDENT PERFORMANCE IN A PROGRAMMING TUTORING SYSTEM USING AI AND FILTERING TECHNIQUES´ 1935

Consideringthecategoriesofpredictionfeatures,differentautho demonstratetheaccuracyimpactoffilteringtechniquesbyshowing
rs use different features for predicting student performance. the improvement percentage after application. Both current and
Some performance-predictive models were built based only on future instructors of this programming tutoring system can
different e-learning features [44], [51], [52], [61], [69], [73]. benefit from the prediction models developed in this study, as
Somemodelsweredevelopedwithcombinede-learningfeatures they can utilize them to optimize learning and improve learner
and student demographic data to predict their performance [70], performance.Inaddition,nostudiesuseasimilarsetofvariables as
[71]. Others have combined e-learning features and student this one, which have been shown to be excellent predictors for
educational data for performance prediction [24], [25], [72]. In building prediction models. Also, learning designers can greatly
many cases for predicting performance, e-learning features benefit from this study by using it to improve the quality of the
were combined with both student demographic and educational curriculum, learning resources, and assessment in online
data [1], [8], [59]. environments in order to enhance students’ final performance.

F. Data Analysis for Prediction Purposes III. METHODOLOGY

TheresultsofusingAImethodsforclassificationorregression To achieve the study’s goals, we conducted an experiment
are the main focus of the majority of data research in the e- involving university students. We analyzed their interaction data
learning area [8], [23], [51], [59]. Data analysis for prediction with the e-learning system and used various AI techniques to
purposes is very important, yet many authors do not perform it create prediction models.
[22], [26], [59]. The analysis of input data can be essential for
choosing the proper prediction technique. There are many A. Data Collection
authors who mention that they conducted some data analysis
The experimental data for this study were collected from two
beforethepredictionprocess,buttheresultsoftheseanalysesare
generations of students at the University of South-Eastern
oftennotpresented[61],[72].Also,manystudiesdidnotpresent
Norway in the winter semesters of 2020–21 and 2021–22, when
descriptive statistics, preventing readers from having a better
both generations were in the second year of their bachelor’s
insight into the structure of the data [25], [53], [58]. In addition,
studies. These students attended an introductory programming
most studies do not provide scales for the variables used for
course and were asked to use the ProTuS tutoring system [91].
prediction, despite the fact that this information, together with
This system offers interactive material for practicing basic
descriptive statistics, might help readers better comprehend the
object-oriented concepts in the Java programming language and
data [20], [53], [56]. Descriptive analysis can provide a visual
has been used at different universities for over a decade. The
depiction of variables, which can be useful when selecting a
collected data include log data of student interactions with the
prediction method [29], [30]. Correlation analysis can be
ProTuS tutoring system. The final dataset includes interaction
essential for choosing a prediction technique, but it is also very
data of 121 participants (96 males and 25 females). In the study,
important for achieving the highest possible prediction
potential participants were precisely informed about the data
accuracy. Some authors report that the highest accuracy can be
collection and analysis process. This ensured their
achieved if variables are highly correlated with the output
understanding and voluntary consent to be part of the study,
variable and there is
meeting the requirements of informed consent under GDPR and
nomulticollinearitybetweenthepredictors[86],[87].Ithasbeen
making our methods and research goals accessible.
shown that removing parameters whose correlations are < 0.35
The content of this course is divided into 20 topics, each
can increase the precision of the applied ML methods [88]. Such
containing multiple interactive examples, challenges, and
and similar studies indicate that the choice of prediction
coding exercises [91]. Students could freely browse the course
technique is closely related to the data structure [89], [90]. Most
and
of the papers found in the literature do not conduct such
selecttopicsandcontentoftheirchoice.SincetheJavaprogrammin
analyses and only randomly choose prediction techniques [22],
g language is learned in this course, examples represent the
[23], [24], [25], [26]. Without a proper analysis of the input data,
explanation of basic Java programming concepts that students
researchersareunabletofullyutilizethepotentialandcapacities of
can read about. After each example, there is a challenge related
AI methods and reach maximum prediction accuracy.
to the example they had previously viewed, and students can
Although numerous researchers have addressed student
practice with code using the drag-and-drop technique and put
performance prediction in e-learning environments, most do not
pieces of code in the blank fields. Coding exercises are the
use the full potential of AI methods. Contrary to most studies
main task students need to solve in every topic, requiring them
that only randomly choose prediction techniques without doing
to write the entire code from scratch for a given problem. A
previous data analysis, this study aims to show the importance
topic is considered covered if the student has successfully
of predictor analysis in order to maximize prediction accuracy,
completed at least one coding exercise in the given topic.
show how to utilize the capabilities of AI methods fully, and
Students are not provided with unlimited time to complete the
also identify the most efficient method of AI for predicting
assessments. Instead, sessions are automatically closed if no
student grades. Unlike many other articles, we also aim to
activity is detected from the student over a 2-min period. This
Authorized licensed use limited to: K. Ramakrishnan Health and Educational Trust. Downloaded on November 05,2024 at 05:08:31 UTC from IEEE Xplore. Restrictions apply.
 1936 IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 17, 2024

protocol ensures that the assessment duration remains passed unit tests (pcrs_dist_success), and overall rating of the
controlled and comparable across all participants. student (elo_rating). The overview of the e-learning variables
The Elo rating method is used in the background to generate (predictors)usedinthepredictionprocesscanbeseeninTableII.
student ratings based on their ability to solve the problems
defined by coding exercises [33]. The Elo rating algorithm, C. Data Analysis and Methods
which was initially developed to rate players in chess
All incomplete data and outliers were removed in the first
tournaments, is used in online learning to represent learners’
step of the analysis. In accordance with the goals set in this
knowledge and difficulty of learning content [92]. The Elo
study and to observe the importance of certain properties of the
rating system operates on the premise that a student’s rating
input data, a popular mrMR algorithm was used [95]. This
increases or decreases based on the outcome of his or her
method is also known in the framework of input data filtering,
attempts to solve an exercise. A successful attempt to solve a
which includes various statistical analysis methods. This
high-difficulty exercise leads to a greater rating gain, whereas
criterion selects the most informative variables regarding
an unsuccessful one results in a comparatively smaller rating
minimum redundancy, maximum relevance, and a combination
loss, thus dynamically adjusting
of these criteria. Minimum redundancy implies minimum
thestudent’sratinginaccordancewiththeirperformancerelative
correlations between input parameters, while maximum
to the content’s difficulty level [91]. Although the Elo rating
relevance refers to maximum correlations between input and
algorithm comes with certain limitations, as any methods for
output variables. Within the mrMR algorithm, the F-statistic
skills estimation or adaptive assessment do, such as the item
was used to examine relevance in the case of continuous
response theory (IRT) [93], we selected the Elo method for the
variables grouped by classes of the predicted variable (student
following reasons: first, the Elo rating algorithm relies entirely
grade), and the
on the quantitative evaluation of a student’s performance, and
Pearsoncorrelationcoefficientwasusedtoexamineredundancy.
it can model skills that change over time, such as programming
Furthermore, the Greedy search algorithm was used to extract
skills [93]; second, when a new item is added in the system,
the input variables to maximize the objective function based
there is no need to assign the correct initial item difficulty, as
on mutual information criteria. The mutual information I(X,Y )
the system will learn and assign difficulty levels for
between the two discrete variables is given by the following
educational content based on the correctness of students’
formula [96]:
answers [93]; and the third and final reason, the Elo rating
algorithm can provide an accurate estimation of the task’
difficulty with a small sample size, something that is not (1)
possible with IRT models [94].
where p(x,y) is the mixed distribution of the variables X and Y
, while p(x) and p(y) are the marginal distributions of the
B. Variables aforementioned variables. The meaning of mutual information
The study looks into student interaction data collected by the can best be seen if we consider X ∩ Y = {0}, then p(x,y) = p(x)
system: overall time the student actively used the system · p(y), and from the formula above, it can be seen that I(X,Y )
(total_durationseconds), number of meaningful actions the = 0. Simply put, this quantity represents the amount of
student made in the system, such as visiting the content and information about one variable concerning another variable.
TABLE II The selection of input variables has a significant influence on
the generalization property of ML methods, which is why it is
a natural necessity to compare the results of the mrMR
algorithm with others in the IVS field. Considering the nature
of the target variable (discrete) and the predictors (continuous),
we considered another algorithm, where the importance of the
predictors was based on the ensemble of DTs (RF). The RF
classifier is optimized by determining the number of estimators
or DTs (323), as well as their depth (14). Since the
methodology of this algorithm is different compared to the
mrMR algorithm, the importance of the predictors is on a
different scale of values. Therefore, it is convenient to
introduce the relative importance of predictors according to the
following formula:
changing the lesson (interaction_num), number of topics the
student successfully covered (topics_covered), number of impi · 100% (2)
different coding exercises that were successfully submitted and impi (%) = max(imp)

Authorized licensed use limited to: K. Ramakrishnan Health and Educational Trust. Downloaded on November 05,2024 at 05:08:31 UTC from IEEE Xplore. Restrictions apply.
 ILIC et al.: PREDICTING STUDENT PERFORMANCE IN A PROGRAMMING TUTORING SYSTEM USING AI AND FILTERING TECHNIQUES´ 1937

and ED is the standard error function

where impi represents ith predictor (i = 1,2,...5). In this way, it is

possible to compare the importance of predictors between . (5)
various algorithms. The minimization of the function specified by formula (2) is
performed within the framework of Bayes’ theorem by
determining the values of the parameters α, β, and γ in an
iterative procedure until convergence is achieved. Fig. 1 shows
a graphical presentation of the Bayesian regularization
algorithm based on the Levenberg–Marquardt algorithm. The
expression J denotes the Jacobian obtained by methods from the
Levenberg–Marquardt algorithm. Therefore, an ANN model
was designed consisting of an input layer, a hidden layer (ten
neurons), and an output layer (one neuron). The mean square
error (mse) was used as the cost function and is given by the
formula (4), where yi and ti are the predicted and known values
of student grades, respectively, and are represented as A = 6, B
= 5, C = 4, D = 3, E = 2, and F = 1. The total number of samples
(students) was N = 121 and the following predictors were used
After determining the most informative variables by filtering as input parameters: total_durationseconds, interaction_num,
the input variables, three different ML methods were applied to elo_rating, topics_covered, and pcrs_dist_success, which
check the influence of this selection on the accuracy of student means that the input data matrix X was 121 × 5. The data were
grade prediction. In this sense, a DT was used where the divided so that 75% were for training and 25% for testing.
maximum number of splits was 20 and with Gini-index as the Standard, tansig, and logsig were used as transfer functions,
split criterion, SVM with Gaussian kernel in version oneversus- respectively. In each training epoch, the training and the test
one, and kNN with 10-nearest neighbor and Euclidian data are shuffled in order to achieve better generalization.
distancemetric.Afivefoldcross-validationwasusedfortraining Numerical simulations of the ANN were performed in the
and validation. In this way, the validation of the ML methods MATLAB programming language (MathWorks, version
was simulated, conditionally speaking, on five unseen datasets 2023b).
and better generalization was achieved.
In order to check the effect of the sample size on the accuracy
of ML methods, the popular and well-known synthetic minority
oversampling technique (SMOTE) algorithm was applied [97].
By applying this algorithm, synthetic data were generated: each
of the synthetic data was generated based on four original data,
whereby no original data could be taken into account more than
twice. In this way, the original set was expanded to 302 samples,
and the accuracy of all methods was checked again.
Given the fact that the prediction cannot be complete if the
influence of all predictors is not considered, ANN was used
since, in their case, no restrictions are set in the case of ML
methods.Sincetherewasamedium-sizedsampleinourcase,the
natural choice was an ANN based on the Levenberg–Marquardt
algorithm. In this sense, it has been shown that better results are
achieved if Bayesian regularization is based on this algorithm
[98]. In short, the essence of this method is to minimize the
objective function

F = αEW + βED (3)

where EW is the sum of squared neural weights for all layers k

(4)

Authorized licensed use limited to: K. Ramakrishnan Health and Educational Trust. Downloaded on November 05,2024 at 05:08:31 UTC from IEEE Xplore. Restrictions apply.
 1938 IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 17, 2024

TABLE III

Fig. 1. Algorithm of Bayesian regularized neural network.

IV. RESULTS
The first logical step in the analysis of the results is certainly
a descriptive analysis, and Table III gives the mean values of all
variables with the corresponding standard deviations.
also given in this table. An exception is a value for a variable
elo_rating. Greater variability means a greater probability that
part of the standard deviation of a specific parameter can be
explained by the variability of other parameters, which would
indicate a multicollinearity problem. The heterogeneity and
size of the student group may be causes of more significant
variability.
Applying the mrMR criterion implies sorting the input
variables from Fig. 2 according to the values for their
importance on the y-axis.
In this sense, the most informative variables are: elo_rating,
pcrs_dist_success, topics_covered, and total_durationseconds.
These variables were sufficient for prediction [95], since there
is a sharp drop between the values of the variables
total_durationseconds and interaction_num, and the
importance of variable interaction_num was < 0.5. This
position was verified by predicting student grades with three
different ML methods: DT, SVM, and kNN. The prediction
was made by first considering all the input variables and then
selecting only the variables according to the mrMR criterion.

Authorized licensed use limited to: K. Ramakrishnan Health and Educational Trust. Downloaded on November 05,2024 at 05:08:31 UTC from IEEE Xplore. Restrictions apply.
 ILIC et al.: PREDICTING STUDENT PERFORMANCE IN A PROGRAMMING TUTORING SYSTEM USING AI AND FILTERING TECHNIQUES´ 1939

Correlations of the predictors with the targeted variable were technique highly dependent on the data structure. In our case,
relatively high (≥0.68),which is there as on why the mrMR high correlation values between the predictors and the target
algorithm was imposed as a natural and logical choice. The variablehadadecisiveinfluenceonthepreprocessingtechnique.
percentage of accuracy achieved by ML techniques with The accuracy of these methods is much lower in the case of
allinputvariables isasfollows: DT 74.4%, SVM 81.8%, and theinputdatasetaugmentedwithsyntheticdatageneratedbythe
kNN 75.2%. The results with selected variables are DT 76%, SMOTEalgorithmandamountedtoDT(54.5%),SVM(55.4%),
SVM 84.3%, and kNN 77.7%. These results were verified by andkNN(58.4%)whenallinputvariablesaretakenintoaccount.
Accuracy is not much changed even when selected variables
comparing the mrMR algorithm with the RF classification
from the expanded set are taken into account: DT (52.8%),
algorithm in terms of relative importance (%), and the data are
SVM (58.1%), and kNN (60.1%). After forming the ANN
summarized in Table IV. and running the algorithm, an accuracy of 96.69% was
Visual inspection shows that the same trend between the achieved using all input variables, which means that the ANN
predictors also exists when an ensemble of classification trees successfully reproduced 117 out of 121 student grades in the
(RF) isapplied.Itcanalsobenotedthatthereisadiscrepancyinterms given sample (with selected variables, the
of the parameters total_durationseconds and interaction_num.
Applying the same parameter exclusion methodology, as in the
case of the mrMR algorithm, it can be seen that there is a sharp
drop in values between the parameters interaction_num and
total_durationseconds in the RF classification algorithm. Since
the value of the total_durationseconds parameter is minimal, it
was removed.
TABLE IV
RELATIVE IMPORTANCE OF PREDICTORS WITH THE MRMR ALGORITHM AND THE RF CLASSIFICATION ALGORITHM

Fig. 3. Prediction accuracy of ML methods. Fig. 4. Evolution of mse during epochs.

accuracy was 95.87%). Fig. 4 shows the evolution of mse over

The only way to check the discrepancy between these two time.
algorithms in terms of parameters total_durationseconds and Fig. 4 shows that the global minimum was reached after 240
interaction_num is to calculate the accuracy of ML methods epochs. It should be noted that the output variable of ANN also
with a set of parameters in two cases: total_durationseconds, appears in the form of a decimal number. That was resolved by
topics_covered, pcrs_dist_success, and elo_rating for the simply rounding to the nearest whole number. It is known and
mrMR algorithm, and interaction_num, topics_covered, described in the broad literature that, in classification tasks,
pcrs_dist_success, and elo_rating for the RF softmax layer and cross-entropy are used as cost functions [99].
classificationalgorithm.Inthefirstcase,theaccuracyofthemethods On the other hand, it compounds the procedure of predicting
wasDT76%,SVM84.3%,andKNN77.7%,whileinthesecond student grades to some extent, but in this case, there was no
case, it was DT 76%, SVM 83.5%, and KNN 75.2%. These needforthis,asthestandardandmuchsimplermsecostfunction
results are shown in Fig. 3. As can be seen, greater accuracy
achieves high accuracy. Based on the results obtained with the
was achieved by applying the mrMR algorithm, reflecting the
ANN, the following formula for predicting student grades was
fact
thattheapplicationofalgorithmsandmethodsisapreprocessing acquired:

Authorized licensed use limited to: K. Ramakrishnan Health and Educational Trust. Downloaded on November 05,2024 at 05:08:31 UTC from IEEE Xplore. Restrictions apply.
 1940 IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 17, 2024

Predicted value = logsig(W2· of lessons that students successfully go through during the
course. The total_durationseconds shows that the time spent on
tanh(W1 · X + b1)+ b2) (6) the online platform can also be used as a good predictor. The
interaction_num variable importance was < 0.5, which means
where: X(5×1) is the input data matrix (total_durationseconds, that the total number of student interactions with the e-learning
system is not considered a good predictor based on this criteria.
interaction_num, elo_rating, topics_covered, and pcrs_dist_ Fig. 5 represents a DT graph demonstrating a visualization
success), W1(10×5) and W2(1×10) neural weight matrices that provides insights into the underlying factors that affect
student performance. Visualizing insights is very important to
obtained from ANN simulations, and finally, b1(10×1) and enhance the interpretability of the results [101], [102].
b2(1×1)matricesofthecorrespondingbiasesalsoobtainedfrom Visualization can show patterns that help explain insights within
data and provide more intuitive explanations than text.
ANN simulations. Visualization can help simplify complex findings acquired by
AI methods, allowing teachers to interpret the results better.
V. DISCUSSION Thus, information present Edina visual context can help
This study used a filtering technique based on the mrMR teachers grasp information quickly. By analyzing the graph of
criterion to select variables. This technique is based on the DT method, the teacher can obtain valuable information
correlations and other statistical analysis tools and belongs to a from the input set. According to studies, proper visualization for
subgroup of IVS [100]. The correlation analysis of data based teachers, regardless of their level of skills and competencies,
on the mrMR criterion showed that the most informative allows them to more easily investigate prediction outcomes and
variables are: elo_rating, pcrs_dist_success, topics_covered, the factors that lead to them and, as a result, utilize them
and total_durationseconds. These four variables proved to be effectively to enhance teaching and learning quality [103],
sufficient for prediction regarding ML methods, which was not [104], [105].

Fig. 5. DT graph that provides insights into factors that affect student performance.
the case with ANN since, in their case, no restrictions are set Applying mrMR filtering to ML methods showed that higher
when it comes to input parameters (as shown in this study). The accuracy could be achieved with it rather than without it. All
elo_rating has the highest importance value of 0.8759, which three ML methods performed better using the input variables
means that the overall rating of the student on the online that were determined with this criterion (DT 76%, SVM 84.3%,
platform is very important for predicting student grades. Given andkNN77.7%),incontrasttotheresultsobtainedwithallinput
that elo_rating represents student ratings based on their ability parameters (DT 74.4%, SVM 81.8%, and kNN 75.2%). In the
to solve the problems defined by coding exercises, it is clear cases of SVM and kNN, an improvement of 2.5% was made,
why this is the most important variable. The pcrs_dist_success while for DT, the improvement was 1.6%. Results obtained via
variable is the next most important, with an importance value of ML methods showed that solid prediction accuracy could be
0.648. This indicates that the number of different coding achieved.ThebestresultsamongtheMLmethodswereobtained
exercises that the student successfully passed is also a very using SVM with a Gaussian kernel, where the accuracy was
importantitemforpredictingstudentgrades.Thetopics_covered 84.3%, with selected variables according to the mrMR criterion.
and total_durationseconds variables have similar importance Most other research has also demonstrated that using filtering
values of 0.5925 and 0.5917, respectively. When it comes to techniquesforMLproduceshigherpredictionaccuracy[1],[27],
topics_covered, it demonstrates the significance of the number [52], [67].
Authorized licensed use limited to: K. Ramakrishnan Health and Educational Trust. Downloaded on November 05,2024 at 05:08:31 UTC from IEEE Xplore. Restrictions apply.
 ILIC et al.: PREDICTING STUDENT PERFORMANCE IN A PROGRAMMING TUTORING SYSTEM USING AI AND FILTERING TECHNIQUES´ 1941

SectionIVshowedthattheincreaseinsamplesizedidnotlead significantly
toanincreaseintheaccuracyofMLmethods.Furtherincreasing affectstheaccuracyofMLmethods[110].Itisworthmentioning an
the noise (or reducing correlations) by an order of magnitude outstanding study, which can also be seen as an attempt to
relative to the output variable drastically reduced accuracy. standardize the sample size according to the accuracy of ML
Similar results were obtained in the paper of Bailly et al. [95], methods [111]. Through numerical experimentation on the
where they varied the sample size by generating synthetic data. arrhythmia dataset, the authors showed that with sample sizes
The performances of binary regression analysis and ANN were < 120,theaccuracyoscillationsofMLmethodsaresignificantly
measured on synthetically generated samples of different sizes. greater compared with sample sizes > 120. This sample size
It was found that sample size has no significant effect on the limit is very close to the sample size in this article (121). At the
accuracy of the mentioned methods. Nevertheless, a larger same time, grand and average effect sizes, according to Cohen
input dataset means better parameter space coverage. [112], were around 0.8. A further increase in the sample size (>
However, this does not necessarily mean that AI methods will 120) did not have a greater impact on the applied ML methods.
perform better. Also, Rajput et al. [111] showed by manipulating data quality
Forexample,byapplyingtheevolutionarycomputation-enabled (completeness, consistency, accuracy, modernity, etc.), that an
prediction method to only 35 students, a mathematical model increase in data quality causes a significant increase in the
was built to predict students’ academic performance in online accuracy of ML methods.
education [66]. Besides that, researchers have shown that the Based on the results presented in this study, it follows that
largest jump in accuracy of the ML methods is for sample sizes higher accuracy is achieved by applying ANN. The best results
of 100–200 [106]. In this manner, by building a species inpredictingstudentgradeswereobtainedusingtheLevenberg–
distribution model, the researchers showed that the maximum MarquardtalgorithmwithBayesianregularization(96.69%with
accuracy of various ML methods is achieved with sample sizes all input variables and 95.87% with selected variables).
of 50 and 100 [107]. These and many other examples indicate Applying the mrMR filtering in the case of ANN showed
that the other properties of the input data (correlation, quality, slightly worse prediction accuracy results than in the case of
balanced dataset, etc.) have the same or even greater influence using all input variables. The accuracy was reduced by 0.82%
in relation to the sample size. The sample size is determined using the mrMR criterion. This confirms that in the case of
mainly based on previous studies, and this problem is given ANN, no restrictions are set in terms of input parameters, as in
special attention within the framework of traditional statistical the case of ML methods, and that with ANN, higher accuracy
analysis. When it comes to ML, there is no big selection of can be
achievediftheinfluenceofallpredictorsisincluded.Mostofthe
researchersalsoreportedbetteraccuracyresultswithANNwhen
all of the predictors were taken into account for prediction [20],
[70], [71], [74], [75].
Fig. 6 shows the best ML methods and ANN results. These
are the results of DT, SVM, kNN, (with selected variables), and
ANN (with all input variables).
Even though ML methods give a solid prediction accuracy of
student grades, their accuracy is significantly lower than the
results of ANN (the Levenberg–Marquardt algorithm with
Bayesian regularization) with 96.69% accuracy. ANN managed
to outperform ML methods and proved to be robust and superior
Fig. 6. Prediction accuracy of ML methods and ANN. not only in terms of accuracy achieved but also in terms of
limiting predictor properties. Similar results were obtained by
other researchers where ANN models produced more accurate
predictions than ML methods [51], [57], [60], [63], [64]. Their
studies dedicated to this problem. Nevertheless, the study [108]
results confirm our findings from this study that ANNs are more
examined the influence of sample size on the accuracy of SVM
robust compared with ML when it comes to predicting student
methods and LR analysis. It was shown that smaller sample
performance in e-learning environments. The view that ANN
sizes (< 100) led to higher accuracy (> 95%), while datasets
achieves greater accuracy in predicting than ML cannot be
(100–1000) led to significantly reduced accuracy (60%–70%).
generalized, as in several cases, ML methods performed better
This result is in agreement with the results of this article.
than ANN [22], [59] or had equal average accuracy [61].
Similar results were obtained in the paper of Faber and Fonseca
All the facts presented in the discussion so far indicate that
[109], where they showed that increasing the sample size
the
beyond a certain limit did not increase the accuracy of ML
correlationsbetweenthepredictorsandtheoutputvariablecould
methods. There is another interesting result: the researchers
beimportantfactorsintheaccuracyoftheappliedmethods.This
reported that there is a limit above which sample size no longer
Authorized licensed use limited to: K. Ramakrishnan Health and Educational Trust. Downloaded on November 05,2024 at 05:08:31 UTC from IEEE Xplore. Restrictions apply.
 1942 IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 17, 2024

meansthattheIVSisalmostasimportantastheiranalysis.Inour learning designers since a higher rating of students in a system

case, varying the sample size did not significantly affect the leads to higher grades, which is also confirmed by the following
final results, but reducing the correlation between input and studies [115], [116]. Also, the number of different tasks or
output variables did. This becomes especially significant, assignments successfully solved in an e-learning system (in our
considering the importance of sample size estimation in case, coding exercises) shows that students will get better
statistical analysis. However, the relationship between IVS grades if they have a larger number of solved tasks and lower
techniques and sample size remains an open question. In grades if they have a smaller number of solved tasks, so this is
conjunction with this view, some authors have used techniques something that learning designers should pay attention to. Other
to reduce the dimensionality of the input set, such as principal authors also indicate this kind of relationship between the
component analysis and independent component analysis [113]. number of successfully completed assignments and learner
This approach makes sense in the case of large samples and performance [117], [118]. Learning designers also need to
reduces the time required for classification and regression tasks consider the number of topics or lessons students must cover or
using AI methods. The disadvantage of these techniques is that pass in order to perform well or get good grades. This is also
one cannot see the influence of each individual input variables, shown by the following studies [119], [120], in which a higher
which is very important in e-learning. There are other number of lessons passed by students led to their better
techniques of IVS, which can be classified into two groups: performance. The time students spend using a learning tool can
wrapper and embedded algorithms, and which can handle all be relevant to some extent for learning designers since opinions
sample sizes. are divided on this issue, as some studies report the irrelevance
In addition to the results of this study, other authors also of this feature in impacting student performance [121], while
experimented with the property of collinearity of input data, others emphasize it as an essential element and that students
which corresponds to the mr criterion. In the study conducted who spend more time on e-learning platforms mostly perform
by Akande et al. [88], it was shown that removing correlations better [122]. Everyone agrees on one thing regarding the total
whose values are < 0.35 leads to an increased accuracy of time spent in the system: spending less time in the system is
applied ML methods. These results are in line with our findings. associated with lower student performance. Learning designers
In addition, an increase in the correlation between input and should not be able to leverage the number of interactions that
output variables lessens this effect. In another study, Halkjrer students make in the e-learning system because there is no
and Winther [89] theoretically showed that large eigenvalues of specific relationship between student interactions and final
the autocorrelation matrix of the input data slow down linear performance in our case. Likewise, the study presented in [123]
perceptron training. They showed that the optimal value of the claimed that increased online interaction of students with an e-
learning constant, which speeds up perceptron training, is learning system did not significantly improve their
inversely proportional to the maximum eigenvalue of the performance.
autocorrelation matrix. What is particularly interesting from a pedagogical point of
Everything that has been said so far about the problem of view, which can be utilized by teachers as well as learning
multicollinearity and correlations of input variables has been designers, are the paths to the leaf nodes, which represent
discussed in the framework of techniques for filtering or
preprocessing input data. Most data research in the field of e- student grades, as shown in Fig. 5. So, for example, for the
learning focuses on the results of applying AI methods in terms values 1478 > elo_rating ≥ 1364 and pcrs_dist_success < 6.5,
of regressionorclassificationtasks.Thisarticleshowedthatspecial teachers can expect a student to get an E. Similarly for predictor
attentionmustbepaidtopreprocessing,thatis,totheselectionof values: elo_rating ≥ 1478, pcrs_dist_success ≥ 39.5 and
input variables. In this sense, it is important to delimit the cases
total_durationseconds ≥ 39784.8 the maximum grade of A can
when necessary so that the AI method can be applied when a
variable is discarded because it does not influence the target be expected, etc. Knowing which factors affect grades is
variable enough. something the teacher can use for student performance
improvement, which many studies confirm [124], [125], [126].
A. Implications Moreover,asdiscussedpreviously,otherinsightsgainedthrough
Considering that the Elo rating of students proved to be the variable importance can be utilized as checkpoints to assist
most informative predictor in our case, implementation of the teachers in reflecting and redesigning to improve the learning
Elo rating should be considered in developing e-learning process [103], [104].
systems, where, in addition to its use for adaptive assessment, Changing teaching methods and learning designs based on
the overall rating of students can be used for predicting learner prediction results can improve student performance, as
performance. The study in [114] showed that using only the Elo expressed in [127]. In our study, the prediction model was
rating of a student as a predictor can yield satisfactory results in developed on data obtained after the course; thus, for now, this
predicting their future performance. The overall rating of model can be applied only after the course but before exams,
students in e-learning systems also needs to be considered by
Authorized licensed use limited to: K. Ramakrishnan Health and Educational Trust. Downloaded on November 05,2024 at 05:08:31 UTC from IEEE Xplore. Restrictions apply.
 ILIC et al.: PREDICTING STUDENT PERFORMANCE IN A PROGRAMMING TUTORING SYSTEM USING AI AND FILTERING TECHNIQUES´ 1943

which the previously applied filtration method using mrMR criteria,

leavesenoughtimeforteacherstotakeappropriateactions.With which increased the accuracy of the prediction in relation to
this information, teachers can start a dialogue with the students the situation with all input variables. Finding a universal
who are projected to fail the exam or have lower predicted method for filtering data completely independent of both the
gradestounderstandwhythesestudentsarestrugglingandfailto ML method and the input data structure is very difficult.
keep up their learning performance. In this manner, teachers To overcome the discussed limitations, we applied ANN
can organize additional learning activities for them before based on the Levenberg–Marquardt algorithm with Bayesian
exams with an emphasis on topics and coding exercises that regularization, which involved utilizing all input parameters.
they have problems with—everything in an effort to raise This algorithm proved very robust, and an accuracy of 96.69%
students’ level of performance. Also, there is a possibility for wasachieved.TheLevenberg–Marquardtalgorithmprovedtobe
teachers to use this model for personalized interventions, as superior not only in terms of achieved accuracy but also in
other studies have successfully done it before to help students terms of limiting predictor properties. This indicates a future
achieve higher learning performance [128], [129]. However, it direction of research: to check the robustness of this algorithm
is important to utilize predicted outcomes for interventions in on other groups of students over time. In addition, we also plan
the early stages of the course, as studies show a significantly to check the algorithm effectiveness on other e-learning
higher impact on student performance than interventions platforms and compare it with other ANN algorithms.
applied toward the end of the course [130], [131]. Particularly interesting
From a student’s standpoint, accessing prediction outcomes arethequalitativerelationshipsbetweenthevariablesthatcanbe
enables a tailored learning experience that aligns with their observed on the DT graph. In this sense, using the DT method
competencies, potentially improving their performance for the selection of variables, which are then delegated to the
through informed study strategies [132], [133]. However, the ANN to predict the student’s grades, can be a challenging task.
utility of weekly performance predictions remains largely It is worth noting that the question of correlations of online
confined to instructors, limiting direct student benefit from learning features has been addressed. Consequently, another
such dynamic insights [50], [134]. research direction could involve exploring the impact of
additional AI methods beyond the ones discussed within the
scope of this article. This issue holds broader implications, not
only for e-
B. Limitations
learningbutalsoforAIingeneral.Buildingontheimplications
Although the results from this study speak in favor of the outlined in this study, future research should prioritize testing
importance of data preprocessing and the outcomes when the ANN model developed in this study for early predictions,
initial data analysis is performed before applying ML and ANN enabling teachers to gain insights into student performance as
techniques, they are also subject to certain limitations. First, early as possible, thus, empowering them to take appropriate
the participants involved in this study are bachelor students actions during the semester. Finally, future research should also
from a university in Norway, which limits the generalizability focusonfurtherrefiningthismodeltoenableweeklypredictions
of our findings to students from different populations with that students can use to monitor their progress and plan
different skill sets. Second, the generalizability of our findings activities based on these results.
is restricted by the learning tasks in this tutoring system that
require domain-specific knowledge of computer science and
carry variables that are specific to the domain of computer
science. Third, the effectiveness of the Elo rating hinges on
precisely assessing the complexity of coding tasks and peers’
performance, two factors that can create inconsistency. This
inconsistency might result in biased predictions. Investigating
different approaches or improvements to the Elo rating system
in future studies could help overcome these issues, leading to
a moreunbiasedanddependableevaluationofstudentabilitiesand
knowledge.
VI. CONCLUSION AND FUTURE WORK
In this study, models based on ML and ANN for predicting
student grades in the ProTuS intelligent tutoring system were
investigated. In developing these predictive models, features
related to student behavior in the online learning environment
were used. By using ML methods, satisfactory accuracy was
achieved: SVM (84.3%), KNN (77.7%), and DT (76%) with
Authorized licensed use limited to: K. Ramakrishnan Health and Educational Trust. Downloaded on November 05,2024 at 05:08:31 UTC from IEEE Xplore. Restrictions apply.
 1944 IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 17, 2024

REFERENCES [24] Y. Abubakar and N. B. H. Ahmad, “Prediction of students performance

in e-learning environment using random forest,” Int. J. Innov. Comput.,
[1] P. Kumari, P. K. Jain, and R. Pamula, “An efficient use of ensemble vol. 7, no. 2, pp. 1–5, 2017.
methods to predict students academic performance,” in Proc. 2018 4th [25] S. Sisovic, M. Matetic, and M. B. Bakaric, “Mining student data to assess
Int. Conf. Recent Adv. Inf. Technol. (RAIT), 2018, pp. 1–6. the impact of Moodle activities and prior knowledge on programming
[2] V. Miki´c, M. Ili´c, L. Kopanja, and B. Vesin, “Personalisation methods course success,” in Proc. 16th Int. Conf. Comput. Syst. Technol., 2015,
in e-learning-A literature review,” Comput. Appl. Eng. Educ., vol. 30, no. pp. 366–373.
6, pp. 1931–1958, 2022. [26] S. Sweta, S. Mahato, and L. K. Pathak, “Prediction of learner’s
[3] B. K. Yousafzai et al., “Student-performulator: Student academic performance in adaptive e-learning system using learning analytics,” in
performance using hybrid deep neural network,” Sustainability, vol. 13, Proc. IOP Conf. Series: Mater. Sci. Eng.. IOP Publishing, 2021, pp. 1–6.
no. 17, pp. 1–21, 2021. [27] F. Qiu et al., “E-learning performance prediction: Mining the feature
[4] C. Romero, M.-I. López, J.-M. Luna, and S. Ventura, “Predicting space of effective learning behavior,” Entropy, vol. 24, no. 5, pp. 1–22,
students’finalperformancefromparticipationinon-linediscussionforums,” 2022.
Comput. Educ., vol. 68, pp. 458–472, 2013. [28] S. Visalakshi and V. Radha, “A literature review of feature selection
[5] H. Nawang, M. Makhtar, and W. Hamzah, “A systematic literature techniques and applications: Review of feature selection in data mining,”
review on student performance predictions,” Int. J. Adv. Technol. Eng. in Proc. 2014 IEEE Int. Conf. Comput. Intell. Comput. Res., 2014, pp. 1–
Exploration, vol. 8, no. 84, pp. 1441–1453, 2021. 6.
[6] K. Mangaroska, B. Vesin, V. Kostakos, P. Brusilovsky, and M. N. [29] Y. Riahi, T. Saikouk, A. Gunasekaran, and I. Badraoui, “Artificial
Giannakos, “Architecting analytics across multiple e-learning systems to intelligence applications in supply chain: A descriptive bibliometric
enhance learning design,” IEEE Trans. Learn. Technol., vol. 14, no. 2, analysis and future research directions,” Expert Syst. with Appl., vol. 173,
pp. 173–188, Apr. 2021. 2021, Art. no. 114702.
[7] I. Lykourentzou, I. Giannoukos, G. Mpardis, V. Nikolopoulos, and V. [30] E. Alyahyan and D. Dü¸stegör, “Predicting academic success in higher
Loumos, “Early and dynamic student achievement prediction in education: Literature review and best practices,” Int. J. Educ. Technol.
elearning courses using neural networks,” J. Amer. Soc. Inf. Sci. Technol., Higher Educ., vol. 17, no. 3, pp. 1–21, 2020.
vol. 60, no. 2, pp. 372–380, 2009. [31] A. H. Nabizadeh, D. Gonçalves, S. Gama, and J. Jorge, “Early prediction
[8] A. S. Hashim, W. A. Awadh, and A. K. Hamoud, “Student performance of students’ final grades in a gamified course,” IEEE Trans. Learn.
prediction model based on supervised machine learning algorithms,” in Technol., vol. 15, no. 3, pp. 311–325, Jun. 2022.
Proc. IOP Conf. Series: Mater. Sci. Eng., 2020, vol. 928, no. 3, pp. 1–18. [32] J. Bravo-Agapito, S. J. Romero, and S. Pamplona, “Early prediction of
[9] X. Wu, X. Zhu, G.-Q. Wu, and W. Ding, “Data mining with Big Data,” undergraduate student’s academic performance in completely online
IEEE Trans. Knowl. data Eng., vol. 26, no. 1, pp. 97–107, Jan. 2014. learning: A five-year study,” Comput. Hum. Behav., vol. 115, pp. 1–25,
[10] A. L’heureux, K. Grolinger, H. F. Elyamany, and M. A. Capretz, 2021.
“Machine learning with Big Data: Challenges and approaches,” IEEE [33] K. Mangaroska, B. Vesin, and M. Giannakos, “Elo-rating method:
Access, vol. 5, pp. 7776–7797, 2017. Towards adaptive assessment in e-learning,” in Proc. 2019 IEEE 19th
[11] B. U. Islam, “Comparison of conventional and modern load forecasting Int. Conf. Adv. Learn. Technol. (ICALT), 2019, pp. 380–382.
techniques based on artificial intelligence and expert systems,” Int. J. [34] G. Barata, S. Gama, J. Jorge, and D. Gonçalves, “Early prediction of
Comput. Sci. Issues (IJCSI), vol. 8, no. 5, pp. 504–513, 2011. student profiles based on performance and gaming preferences,” IEEE
[12] J. L. Rastrollo-Guerrero, J. A. Gómez-Pulido, and A. Durán-Domínguez, Trans. Learn. Technol., vol. 9, no. 3, pp. 272–284, Jul./Sep. 2016.
“Analyzing and predicting students’ performance by means of machine [35] K. F. Hew, X. Hu, C. Qiao, and Y. Tang, “What predicts student
learning: A review,” Appl. Sci., vol. 10, no. 3, pp. 345–360, 2020. satisfaction with MOOCs: A gradient boosting trees supervised machine
[13] M. Ili´c, V. Miki´c, L. Kopanja, and B. Vesin, “Intelligent techniques in learning and sentiment analysis approach,” Comput. Educ., vol. 145,
e-learning: A literature review,” Artif. Intell. Rev., vol. 56, no. 12, pp. 2020, Art. no. 103724.
14907–14953, 2023. [36] P. Bhardwaj, P. Gupta, H. Panwar, M. K. Siddiqui, R. MoralesMenendez,
[14] R. Farhat, Y. Mourali, M. Jemni, and H. Ezzedine, “An overview of and A. Bhaik, “Application of deep learning on student engagement in e-
machine learning technologies and their use in e-learning,” in Proc. 2020 learning environments,” Comput. Elect. Eng., vol. 93, 2021, Art. no.
Int. Multi-Conf.:“Org. Knowl. Adv. Technol.” (OCTA), 2020, pp. 1–4. 107277.
[15] M. Vakili, M. Ghamsari, and M. Rezaei, “Performance analysis and [37] D. Riofrio-Luzcando, J. Ramirez, and M. Berrocal-Lobo, “Predicting
comparison of machine and deep learning algorithms for IoT data studentactionsinaproceduraltrainingenvironment,”IEEETrans.Learn.
classification,” 2020, arXiv:2001.09636. Technol., vol. 10, no. 4, pp. 463–474, Oct./Dec. 2017.
[16] M. Hassan and M. Hamada, “A neural networks approach for improving [38] M.Moresi,M.J.Gomez,andL.Benotti,“Predictingstudents’difficulties
the accuracy of multi-criteria recommender systems,” Appl. Sci., vol. 7, from a piece of code,” IEEE Trans. Learn. Technol., vol. 14, no. 3, pp.
no. 9, pp. 1–18, 2017. 386–399, Jun. 2021.
[17] Y. Goldberg, “A primer on neural network models for natural language [39] T. Anagnostopoulos, C. Kytagias, T. Xanthopoulos, I. Georgakopoulos,
processing,” J. Artif. Intell. Res., vol. 57, pp. 345–420, 2016. I. Salmon, and Y. Psaromiligkos, “Intelligent predictive analytics for
[18] R. K. Sinha, R. Pandey, and R. Pattnaik, “Deep learning for computer identifying students at risk of failure in Moodle courses,” in Proc. Intell.
vision tasks: A review,” 2018, arXiv:1804.03928. Tutoring Syst.: 16th Int. Conf., ITS 2020, Athens, Greece, 2020, pp. 152–
[19] Y. Baashar et al., “Toward predicting student’s academic performance 162.
using artificial neural networks (ANNs),” Appl. Sci., vol. 12, no. 3, pp. [40] R. Alcaraz, A. Martínez-Rodrigo, R. Zangróniz, and J. J. Rieta, “Early
1–16, 2022. predictionofstudentsatriskoffailingaface-to-
[20] M. Sayed and F. Baker, “E-learning optimization using supervised facecourseinpowerelectronicsystems,”IEEETrans.Learn.Technol.,vol.14
artificial neural-network,” J. Softw. Eng. Appl., vol. 8, no. 1, pp. 26–34, ,no.5,pp. 590–603, Oct. 2021.
2015. [41] Y. Jang, S. Choi, H. Jung, and H. Kim, “Practical early prediction of
[21] A. Remaida, B. Abdellaoui, A. Moumen, and Y. E. B. El Idrissi, students’performanceusingmachinelearningandexplainableAI,”Educ.
“Personalitytraitsanalysisusingartificialneuralnetworks:Aliteraturesurve Inf. Technol., vol. 27, no. 9, pp. 12855–12889, 2022.
y,”in Proc. 2020 1st Int. Conf. Innov. Res. Appl. Sci., Eng. Technol. [42] F. Braz, F. Campos, V. Stroele, and M. Dantas, “An early warning model
(IRASET), 2020, pp. 1–6. forschooldropout:Acasestudyine-learningclass,”inProc.Braz.Symp.
[22] A. Alsharhan, S. A. Salloum, and A. Aburayya, “Using e-learning factors Comput. Educ., 2019, vol. 30, no. 1, pp. 1441–1450.
to predict student performance in the practice of precision education,” J. [43] D.Baneres,M.E.Rodríguez-Gonzalez,andM.Serra,“Anearlyfeedback
Legal, Ethical Regulatory Issues, vol. 24, no. 6, pp. 1–14, 2021. prediction system for learners at-risk within a first-year higher education
[23] V. Vijayalakshmi and K. Venkatachalapathy, “Comparison of predicting course,” IEEE Trans. Learn. Technol., vol. 12, no. 2, pp. 249–263,
student’s performance using machine learning algorithms,” Int. J. Intell. Apr./Jun. 2019.
Syst. Appl., vol. 11, no. 12, 2019, Art. no. 34.
Authorized licensed use limited to: K. Ramakrishnan Health and Educational Trust. Downloaded on November 05,2024 at 05:08:31 UTC from IEEE Xplore. Restrictions apply.
 ILIC et al.: PREDICTING STUDENT PERFORMANCE IN A PROGRAMMING TUTORING SYSTEM USING AI AND FILTERING TECHNIQUES´ 1945

[44] F. Qiu et al., “Predicting students’ performance in e-learning using [66] E. S. Bhutto, I. F. Siddiqui, Q. A. Arain, and M. Anwar, “Predicting
learning process and behaviour data,” Sci. Rep., vol. 12, no. 1, 2022, Art. students’ academic performance through supervised machine learning,”
no. 453. in Proc. 2020 Int. Conf. Inf. Sci. Commun. Technol. (ICISCT), 2020, pp.
[45] Y.T.BadalandR.K.Sungkur,“Predictivemodellingandanalyticsofstudents’ 1–6.
grades using machine learning algorithms,” Educ. Inf. Technolog., vol. [67] P. Jiao, F. Ouyang, Q. Zhang, and A. H. Alavi, “Artificial
28, pp. 3027—3057, 2023. intelligenceenabled prediction model of student academic performance
[46] S. Li and T. Liu, “Performance prediction for higher education students in online engineering education,” Artif. Intell. Rev., vol. 55, no. 8, pp.
using deep learning,” Complexity, vol. 2021, pp. 1–10, 2021. 6321–6344, 2022.
[47] P. Balaji, S. Alelyani, A. Qahmash, and M. Mohana, “Contributions of [68] M. Abdullah, M. Al-Ayyoub, S. AlRawashdeh, and F. Shatnawi,
machine learning models towards student academic performance “Elearningdjust: E-learning dataset from Jordan University of Science
prediction: A systematic review,” Appl. Sci., vol. 11, no. 21, pp. 1–25, and Technology toward investigating the impact of COVID-19 pandemic
2021. on education,” Neural Comput. Appl., pp. 1–15, 2021.
[48] G. B. Brahim, “Predicting student performance from online engagement [69] S. Rakic, M. Pavlovic, S. Softic, B. Lalic, and U. Marjanovic, “An
activities using novel statistical features,” Arabian J. Sci. Eng., vol. 47, evaluation of student performance at e-learning platform,” in Proc. 2019
no. 8, pp. 10225–10243, 2022. 17th Int. Conf. Emerg. eLearning Technol. Appl. (ICETA), 2019, pp. 681–
[49] F. Ünal, “Data mining for student performance prediction in education,” 686.
Data Mining-Methods, Appl. Syst., vol. 28, pp. 423–432, 2020. [70] M. N. Raheema, A. M. Al-Khazzar, and J. S. Hussain, “Prediction of
[50] B. Albreiki, N. Zaki, and H. Alashwal, “A systematic literature review of students’ achievements in e-learning courses based on adaptive
student’ performance prediction using machine learning techniques,” neurofuzzy inference system,” Int. J. Fuzzy Log. Intell. Syst., vol. 22, no.
Educ. Sci., vol. 11, no. 9, pp. 1–27, 2021. 2, pp. 213–222, 2022.
[51] S. Arumugam, A. Kovalan, and A. Narayanan, “A learning performance [71] S. Aydo˘¸ gdu, “Predicting student final performance using artificial
assessment model using neural network classification methods of neural networks in online learning environments,” Educ. Inf. Technol.,
elearning activity log data,” in Proc. 2019 Int. Conf. smart Syst. Inventive vol. 25, no. 3, pp. 1913–1927, 2020.
Technol. (ICSSIT), 2019, pp. 441–448. [72] E. P. Saputra et al., “Prediction of evaluation result of e-learning success
[52] K. Fahd, S. J. Miah, and K. Ahmed, “Predicting student performance in based on student activity logs with selection of neural network attributes
a blended learning environment using learning management system base on PSO,” in J. Phys.: Conf. Ser. , 2020, pp. 1–6.
interaction data,” Appl. Comput. Informat., pp. 1–12, 2021. [73] Z.Kanetaki,C.Stergiou,G.Bekas,C.Troussas,andC.Sgouropoulou,“A
[53] H. Heuer and A. Breiter, “Student success prediction and the trade-off hybridmachinelearningmodelforgradepredictioninonlineengineering
between Big Data and data minimization,” in DeLFI 2018-Die 16. education,”InternationalJournalEng.Pedagogy,vol.12,pp. 4–24,2022.
ELearning Fachtagung Informatik, 2018, pp. 219–230. [74] K. Noama et al., “Improvement of e-learning based via learning
[54] T. Mahboob, S. Irfan, and A. Karamat, “A machine learning approach for management systems (LMS) using artificial neural networks,” Asian J.
student assessment in e-learning using Quinlan’s c4. 5, naive Bayes and Res. Comput. Sci., vol. 4, no. 1, pp. 1–9, 2019.
random forest algorithms,” in Proc. 2016 19th Int. Multi-Topic Conf. [75] F. Okubo, T. Yamashita, A. Shimada, and H. Ogata, “A neural network
(INMIC), 2016, pp. 1–8. approach for students’ performance prediction,” in Proc. seventh Int.
[55] W. Bagunaid, N. Chilamkurti, and P. Veeraraghavan, “AISAR: Artificial Learn. Analytics Knowl. Conf., 2017, pp. 598–599.
intelligence-based student assessment and recommendation system for [76] N. Z. Zacharis, “Predicting student academic performance in blended
e-learning in Big Data,” Sustainability, vol. 14, no. 17, pp. 1–22, 2022. learning using artificial neural networks,” Int. J. Artif. Intell. Appl., vol.
[56] F.Saleem,Z.Ullah,B.Fakieh,andF.Kateb,“Intelligentdecisionsupport 7, no. 5, pp. 17–29, 2016.
system for predicting student’s e-learning performance using ensemble [77] E. P. Saputra, I. Indriyanti, S. Supriatiningsih, T. Rahman, and A. H.
machine learning,” Mathematics, vol. 9, no. 17, pp. 1–22, 2021. Kahfi, “Comparison of machine learning algorithm for prediction
[57] A. Rivas, A. Gonzalez-Briones, G. Hernandez, J. Prieto, and P. Chamoso, learning achievement on e-learning students,” in Proc. AIP Conf. Proc.,
“Artificial neural network analysis of the academic performance of 2023, vol. 2714, no. 1, Art. no. 030005 .
students in virtual learning environments,” Neurocomputing, vol. 423, [78] M.K.KadhimandA.K.Hassan,“Towardsintelligente-learningsystems: A
pp. 713–720, 2021. hybrid model for predicating the learning continuity in Iraqi higher
[58] S.-S.M.Ajibadeetal.,“Utilizationofensembletechniquesforprediction education.,” Webol., vol. 17, no. 2, pp. 172–188, 2020.
oftheacademicperformanceofstudents,”J.Optoelectron.Laser,vol.41, no. [79] M.W.Mwadulo,“Areviewonfeatureselectionmethodsforclassification
6, pp. 48–54, 2022. tasks,” Int. J. Comput. Appl. Technol. Res., vol. 5, no. 6, pp. 395–402,
[59] E. Emirtekin, M. Karatay, and T. Ki, “Online course success prediction 2016.
of students with machine learning methods,” J. Modern Technol. Eng., [80] M. Ye et al., “SA-FEM: Combined feature selection and feature fusion
vol. 5, no. 3, pp. 271–282, 2020. for students’ performance prediction,” Sensors, vol. 22, no. 22, pp. 1–24,
[60] M. Kokoç and A. Altun, “Effects of learner interaction with learning 2022.
dashboards on academic performance in an e-learning environment,” [81] S. Syed Mustapha, “Predictive analysis of students’ learning
Behav. Inf. Technol., vol. 40, no. 2, pp. 161–175, 2021. performance using data mining techniques: A comparative study of
[61] K. Sriram, T. Chakravarthy, and K. Anastraj, “Students impenetrabilities feature selection methods,” Appl. System Innov., vol. 6, no. 5, pp. 1–24,
fromlearningsessiondatapredictionusingmachinelearningtechniques,” 2023.
Compliance Eng. J., vol. 10, no. 7, pp. 69–77, 2020. [82] N. Pudjihartono, T. Fadason, A. W. Kempa-Liehr, and J. M. O’Sullivan,
[62] X. Wang, P. Wu, G. Liu, Q. Huang, X. Hu, and H. Xu, “Learning “A review of feature selection methods for machine learning-based
performance prediction via convolutional GRU and explainable neural disease risk prediction,” Front. Bioinf., vol. 2, pp. 1–17, 2022.
networksine-learningenvironments,”Computing,vol.101,pp. 587–604, [83] D. Theng and K. K. Bhoyar, “Feature selection techniques for machine
2019. learning: A survey of more than two decades of research,” Knowl. Inf.
[63] Y.-S. Su, Y.-D. Lin, and T.-Q. Liu, “Applying machine learning Syst., vol. 66, pp. 1575–1637, 2023.
technologies to explore students’ learning features and performance [84] E. M. Karabulut, S. A. Özel, and T. Ibrikci, “A comparative study on the
prediction,” Front. Neurosci., vol. 16, pp. 1–9, 2022. effect of feature selection on classification accuracy,” Procedia Technol.,
[64] T. Santamaría-Lopez, D. Patiño-Perez, V. González-Ruiz, and L. vol. 1, pp. 323–327, 2012.
FloresCarvajal, “Implementation of machine learning techniques and [85] S. Gbashi et al., “Application of a generative adversarial network for
creation of an artificial neural network for the prediction of the academic multi-featured fermentation data synthesis and artificial neural network
performance of students in university environments that use e-learning (ANN) modeling of bitter gourd–grape beverage production,” Sci. Rep.,
and streaming,” DYNA-Ingeniería e Industria, vol. 98, no. 3, pp. 282– vol. 13, no. 1, pp. 1—15, 2023.
287, 2023. [86] O. I. Abiodun, A. Jantan, A. E. Omolara, K. V. Dada, N. A. Mohamed,
[65] and H. Arshad, “State-of-the-art in artificial neural network applications:
A survey,” Heliyon, vol. 4, no. 11, 2018, Art. no. e00938.
Authorized licensed use limited to: K. Ramakrishnan Health and Educational Trust. Downloaded on November 05,2024 at 05:08:31 UTC from IEEE Xplore. Restrictions apply.
 1946 IEEE TRANSACTIONS ON LEARNING TECHNOLOGIES, VOL. 17, 2024

Miloš Ili´c received the bachelor’s degree in information Katerina Mangaroska received the Ph.D. degree in
technologies and the master’s degree in information and computer science from the Norwegian University of
communication technologies from the Faculty of Science and Technology, Trondheim, Norway, in
Information Technologies, Alfa BK University, 2021.
Belgrade, Serbia, in 2016 She is currently an Associate Professor with the
University of South-Eastern Norway, Vestfold,
Norway. She is a Fulbright scholar. Her research
interests include learning analytics and computing
education research, and learning design, multimodal
learning analytics, artificial intelligence in education,
and human–computer interaction.
Goran Kekovi´c received a Ph.D. degree in electrical Lazar Kopanja received a Ph.D. degree in image processing, nanomaterials,
science from the School of Electrical Engineering, and magnetic properties from the Faculty of Technology and
University of Belgrade, Belgrade, Serbia, in 2008. Metallurgy,University of Belgrade, Belgrade, Serbia, in 2016. He is currently
a Computer Scientist with Alfa BK University, Belgrade, specializing in image
He is currently an Assistant Professor with the Faculty
analysis, artificial intelligence, mathematical models for shape measurement-
of Information Technology, Alfa BK University,
oriented purposes and e-learning systems. The number of citations is 883, and
Belgrade. At the beginning of his scientific career, he
an H-index of 14. His research interests include nanotechnologies, biometrics,
dealt with the theory of condensed state and later
bioinformatics, and data science.
switched to biosignal analysis (EEG) and medical data
processing, using methods of statistical analysis and
machine learning. These research works resulted in the
publication of 43 scientific papers that were
cited 154 times, with an H-index of 7. His current scientific interests include
data processingandartificialintelligence.Themaincontributionofhisresearchisthe
theoretical model of charge and energy transport during structural transitions of
polymersandtheinvestigationofmarkersofneurobiologicaldisordersinanimal and
human models by analyzing EEG signals.

Vladimir Miki´c received the bachelor’s degree in Boban Vesin received the M.Sc. degree in
information technology and the master’s degree in computer science and the
electrical and computer engineering from the Faculty Ph.D. degree in computer
of Information Technology, Alfa BK University, science from the
Belgrade, Serbia, in 2019 and 2020, respectively, University of Novi
where he is currently working toward the Ph.D. Sad, Novi Sad, Serbia, in
degree in electrical and computer engineering with 2007 and 2014,
the Faculty of Information Technology. respectively. He is
He is currently a Teaching Assistant with the currently an
Faculty of Information Technology, and the Faculty of Associate Professor
Mathematics and Computer Science, Alfa BK Uni- with the University of South-Eastern
versity. His research interests include e-learning personalization, intelligent e- Norway, Vestfold, Norway. He boasts an
learning techniques, and multimedia. extensive publication record of more than 40 scientific papers in esteemed
journals and international conference proceedings, spanning the domains of
personalized learning, software engineering, and learning analytics. His
research interests include the engineering of e-learning systems and learning
analytics, with a primary focus on aiding educators in understanding and
enhancing teaching processes.

Authorized licensed use limited to: K. Ramakrishnan Health and Educational Trust. Downloaded on November 05,2024 at 05:08:31 UTC from IEEE Xplore. Restrictions apply.