Intelligent Automation & Soft Computing Tech Science Press

DOI: 10.32604/iasc.2023.029698
Article

Recent Advances in Fatigue Detection Algorithm Based on EEG

Fei Wang1,2, Yinxing Wan1, Man Li1,2, Haiyun Huang1,2, Li Li1, Xueying Hou1, Jiahui Pan1,2,
Zhenfu Wen3 and Jingcong Li1,2,*

1
School of Software, South China Normal University, FoShan, 528225, China
2
Pazhou Lab, Guangzhou, 510330, China
3
Department of Psychiatry, New York University School of Medicine, New York, 10016, USA
*Corresponding Author: Jingcong Li. Email: [email protected]
Received: 09 March 2022; Accepted: 14 April 2022

Abstract: Fatigue is a state commonly caused by overworked, which seriously

affects daily work and life. How to detect mental fatigue has always been a hot
spot for researchers to explore. Electroencephalogram (EEG) is considered one
of the most accurate and objective indicators. This article investigated the devel-
opment of classification algorithms applied in EEG-based fatigue detection in
recent years. According to the different source of the data, we can divide these
classification algorithms into two categories, intra-subject (within the same sub-
ject) and cross-subject (across different subjects). In most studies, traditional
machine learning algorithms with artificial feature extraction methods were com-
monly used for fatigue detection as intra-subject algorithms. Besides, deep learn-
ing algorithms have been applied to fatigue detection and could achieve effective
result based on large-scale dataset. However, it is difficult to perform long-term
calibration training on the subjects in practical applications. With the lack of large
samples, transfer learning algorithms as a cross-subject algorithm could promote
the practical application of fatigue detection methods. We found that the research
based on deep learning and transfer learning has gradually increased in recent
years. But as a field with increasing requirements, researchers still need to con-
tinue to explore efficient decoding algorithms, design effective experimental para-
digms, and collect and accumulate valid standard data, to achieve fast and
accurate fatigue detection methods or systems to further widely apply.

Keywords: EEG; fatigue detection; deep learning; machine learning; transfer

learning

1 Introduction
Fatigue is defined as a state of declined physical or mental activity, usually due to continuous study or
work [1]. It can commonly be divided into physical fatigue and mental fatigue. Generally speaking, mental
and physical fatigue are intertwined and appear simultaneously [2]. Compared with physical fatigue, mental
fatigue is more complicated and more harmful. Mental fatigue is often the result of long-term accumulation
of cognitive activities in the brain, such as excessive thinking and exhaustion. After brain fatigue, cognitive

This work is licensed under a Creative Commons Attribution 4.0 International License, which
permits unrestricted use, distribution, and reproduction in any medium, provided the original
work is properly cited.
3574 IASC, 2023, vol.35, no.3

function is limited, and alertness decreases [3]. In a state of mental fatigue, people will feel dizzy,
unresponsive, distracted, and difficult to perform their thinking activities normally [4]. This affects
learning and work efficiency and leads to insomnia, anxiety, worry, and forgetting. Driver fatigue is
common mental fatigue in daily life, which is one of the most important reasons for many traffic
accidents. An earlier study by the American Automobile Association Road Safety Foundation estimated
that 7% of traffic accidents, 13% of hospital traffic accidents and 21% of fatal traffic accidents in relation
to driver fatigue [5]. According to the American Road Safety Foundation, about 30.8% of people have
been driving fatigued [6]. Therefore, how to detect fatigue correctly has become an urgent issue.
A variety of methods of fatigue detection have emerged in the past decades. We can divide them into
3 categories: questionnaires (e.g., Psycho-motor Vigilance Test (PVT) [7], Checklist Individual Strength
questionnaire (CIS) [8]), behavioral detection methods (e.g., head position [9], blink frequency [10]), and
methods based on physiological signals (e.g., EEG [11,12], electrooculogram (EOG) [13,14],
electromyography (EMG) [15,16], electrocardiogram (ECG) [17,18], pulse beat [19]). Among these, the
questionnaires are psychological tests, the advantage of which are cheap and easy to promote. However,
these can’t detect the state of the subjects in real-time and are also highly subjective [20]. Behavioral
detection methods use image recognition methods to identify the state of the subject’s head and eyes, the
head position and the blinking frequency of eyes change with fatigue. These methods require using a
camera to collect head features in real-time. The illumination greatly affects its accuracy and cannot adapt
to the ever-changing environment [21,22]. Physiological signals can accurately reflect the human state in
real-time. Bioelectricity is generated by cell activity. For instance, EOG is bioelectricity generated by eye
movement. EEG is produced by the human brain at the scalp and can directly reflect different activity
states of the brain. In a study on fatigue detection by Laurent et al. [23], the accuracy of EEG is
significantly better than ECG and EOG. EEG is recognized as one of the most objective and accurate
indicators of brain fatigue detection in all common physiological signals [24]. This study focuses on
investigating the development of EEG-based fatigue detection methods.
The entire EEG-based fatigue detection methods or systems commonly requires the three blocks: signal
acquisition, signal processing and analysis, feedback and application, as shown in Fig. 1. Specifically, signal
acquisition uses the EEG cap or other equipment to collect the fatigue and awake state EEG signals of the
subjects. The signal processing and analysis are the most critical steps, including the three processes of
preprocessing, feature extraction, and classification. The preprocessing is to delete the original EEG
signal’s artefacts such as EOG and EMG, and the feature extraction is to extract key features to help the
classifier to decode effectively such as Wavelet Transform, Autoregressive (AR), Fast Fourier Transform
(FFT), etc. Then the effective features such as entropy, time/frequency domain characterization are used
to train a model in classification to distinguish fatigue from the alert. We usually divide these
classification algorithms into three categories: machine learning, deep learning and transfer learning.
Machine learning methods are easy to implement and can achieve good results without a large training
dataset, such as Support Vector Machines (SVM), K-Nearest Neighborhood (KNN), etc. The deep
learning method needs to adjust the network structure and parameters, but it can get effective results with
enough training data. The commonly deep learning methods are Convolutional Neural Network (CNN),
Long-Short Term Memory (LSTM), etc. For transfer learning, it’s suitable for application with small or
even zero-shot training. At present, some transfer learning algorithms such as Transfer Component
Analysis (TCA), Domain Adversarial Neural Network (DANN) have been widely used in fatigue
detection based on EEG. Through the block of application and feedback the detection results can be
presented to remind the user that they are in a state of fatigue.
IASC, 2023, vol.35, no.3 3575

Figure 1: The framework of EEG-based fatigue detection system

The preprocessing includes filtering, epoch extraction, down-sampling, artifact removal, ICA
decomposition and other processes. The function of the preprocessing process is to filter and clean to
obtain better EEG data. Borowicz [25] designed time-domain linear filtering called Multichannel Wiener
Filter (MWF) to wipe out the artifacts in EEG data. After the preprocessing, there are a lot of methods to
extract the features such as Fast Fourier transform (FFT), wavelet transform. Wali et al. [26] applied the
Discrete Wavelet Transform (DWT) and FFT to get time-frequency features and obtained a 79.21%
accuracy classifier.
In order to avoid overlap with previous reviews, this study is the first to review the classification
algorithms and some feature extraction algorithms in EEG-based fatigue detection in recent years. We
divided these algorithms into two categories, intra-subject and cross-subject. Intra-subject refers to using
the same subject’s data to train and test, while cross-subject refers to the case that the training and testing
data come from different subjects. We summarized and analyzed these algorithms to lay the foundation
for future research on algorithms based on EEG fatigue detection technology. Moreover, we also
discussed the existing problems of EEG-based fatigue detection and analyzed the future development
direction of the field based on the existing foundation.
The remainder of this study will be organized as follows: Section 2 reviews the EEG-based fatigue
detection algorithms in intra-subject; we review the EEG-based fatigue detection algorithms in cross-
subject in Section 3; Section 4 provides a discussion about current problems and future directions, and
finally, Section 5 summarizes of the full text.

2 EEG-based Fatigue Detection in Intra-subject

This section will review in detail the research progress of EEG-based fatigue detection in intra-subject in
recent years. As shown in Fig. 2, the classification algorithms in intra-subject are divided into two categories:
traditional machine learning and deep learning. SVM [27,28], Random Forest (RF) [29], KNN [30], and
3576 IASC, 2023, vol.35, no.3

Artificial Neural Networks (ANN) [31] are commonly used as traditional machine learning classifiers in the
field of EEG-based fatigue detection. For deep learning algorithms, researchers exploit the characteristics of
each network to extract effective features and improve classification performance. CNN [32–34] is
commonly utilized to extract frequency domain characterization, and LSTM [35,36] is applied to analyze
time series. Besides, Deep Belief Network (DBN) [37] and some other deep learning networks have been
used in fatigue detection based on EEG.

SVM KNN
Machine Learning ……
RF ANN

Classification in
Intra-subject

CNN DBN
Deep Learning
LSTM ……

Figure 2: The EEG-based fatigue detection classification algorithms in intra-subject

2.1 Traditional Machine Learning Algorithms Applicated in Classification

Traditional machine learning classification algorithms are usually employed with feature extraction
methods. In the early days, researchers used traditional feature extraction algorithms with machine
learning algorithms to improve classification accuracy. Besides, researchers proposed some networks
which were composed of multiple feature extraction methods to automatically extract deep effective
features. According to the characteristics of the feature extraction algorithm, we could introduce the
development of traditional machine learning for EEG-based fatigue detection from two aspects:
traditional feature extraction methods and feature extraction network methods. Tab. 1 is the summary of
the machine learning articles about the author, feature extraction method, classification algorithm and
accuracy. In the table, the first four items use traditional feature extraction methods, and the remaining
two items apply feature extraction network to extract features.

Table 1: The summary of traditional machine learning algorithms applicated in EEG-based fatigue detection
Reference Feature extraction method Classification algorithm Accuracy (%)
Cui et al. [38] FFT FWET1 <NG>
Zhao et al. [27] MVAR2 KPCA-SVM8 81.64
Chai et al. [39] ERBM-ICA3, AR BNN9 84.3/83.0
Zhang et al. [31] DWT4 MLPNN10 96.5–99.5
Ma et al. [28] PCA5, PCANet6 SVM 95.14 ± 4.87
Tuncer et al. [30] NCA7, ReliefF and PCA KNN 97.29
Notes: 1FWET, The Feature Weighted Episodic Training; 2MVAR, Multivariate Autoregressive; 3ERBM-ICA, Entropy Rate Bound Minimization
Analysis; 4DWT, Discrete Wavelet Transform; 5PCA, Principal Component Analysis; 6PCANet, Principal Component Analysis Network; 7NCA,
Neighborhood Component Analysis; 8KPCA-SVM, Kernel Principal Component Analysis and SVM; 9BNN, Bayesian Neural Network; 10MLPNN,
Multilayer Perceptron Neural Network; <NG> Not given.
IASC, 2023, vol.35, no.3 3577

2.1.1 Traditional Machine Learning Methods Combined with Traditional Feature Extraction Methods
SVM is one of the most widely used method for EEG classification. It can accurately predict numerous
variables, while the performance is very competitive compared to other methods [40]. Zhao et al. [27] used
MVAR model to extract multi-channel EEG signal features, and then they used KPCA-SVM to get 81.64%
accuracy. KPCA was applied to reduce the dimension of feature vectors and accelerating the convergence
speed of SVM training. The major shortcoming of SVM is the key parameters are difficult to adjust for
best results [41]. ANN also needs to adjust several parameters, and it is prone to overfitting, but it has an
associative memory function and can adequately approach intricate nonlinear relationships [42]. Zhang
et al. [31] applied wavelet transform to extract three kinds of entropy from 20 healthy male’s (20–
35 years) EEG, EOG and EMG data. Then a four-layer (two hidden layers) MLPNN was used as the
classifiers. The results showed that their method could achieve about 96.5%–99.5% accuracy. What can
be known is the structure of MLPNN has a great influence on the model. Introducing the concept of
Bayesian into the neural network can produce a measure of uncertainty and treat the weights as random
variables, which is not easy to overfit [43]. Chai et al. [39] employed BNN to analyze EEG fatigue state
data and alert state data from 43 healthy participants. The accuracy for detecting fatigue status is 84.3%,
and for detecting alertness, the accuracy is 83.0%. It shows that BNN improves generalization ability
without being affected by data quality. However, BNN usually has more parameters, and the performance
in classification/regression problems on large-scale datasets does not have much advantage over ordinary
neural networks [44].
Besides, the above methods all require calibration work when applied to new subjects, which need the
collection of data from subject-specific to adjust the model parameters. To deal with this issue, Cui et al. [38]
applied 16 healthy subjects’ EEG data FWET to train and test FWET model. The results showed FWET
could get a good performance without calibration work. The author expanded domain generalization from
computer vision to brain-computer interface, which is applied to eliminate the calibration effort caused by
individual differences completely.

2.1.2 Traditional Machine Learning Methods Combined with Feature Extraction Network Methods
Traditional feature extraction is only a superficial level extraction, and some discriminative information
cannot be effectively extracted. Some shallow learning methods also require artificial design or application of
certain features as the machine learning input. Researchers combine multiple feature extraction methods to
form a feature extraction network to obtain more features in recent research. Compared with traditional
extraction methods, it has better performance and more useful features.
There are also many classification methods used in conjunction with feature extraction networks. Ma
et al. [28] combined PCA and PCANet to build a feature extraction network, and applied SVM for
classification. It is obvious that PCA could reduce the dimensionality of EEG data to prevent PCANet
from causing a dimensional explosion. At the same time, with the advantages of SVM processing small
sample data, the accuracy of 95.14 ± 4.87 had been achieved. Different from Ma et al., Tuncer et al. [30]
utilized Dynamic Center based Binary Pattern (DCBP) and Multi Threshold based Ternary Pattern
(MTTP) to generate features, and then employed NCA, ReliefF and PCA to build a feature selection
network. KNN was applied as a classifier because of its simplicity and excellent performance. This
feature generation and selection can shorten training time and improve classification accuracy without any
optimization algorithm.

2.2 Deep Learning Algorithms Applicated in Classification

Compared with machine learning, deep learning can easily utilize large amounts of data and has better
classification performance [45]. Many deep networks have been applied in EEG-based fatigue detection in
recent years. Tab. 2 summarizes all the deep learning articles about author, classification algorithm and
3578 IASC, 2023, vol.35, no.3

accuracy. The first three items in the table use CNN, the next two items employ LSTM, and the last two apply
other deep networks.

Table 2: The summary of deep learning algorithms used in EEG-based fatigue detection
Reference Classification algorithm Accuracy (%)
Zeng et al. [32] EEG-Conv, EEG-Conv-R 91.788, 92.682
Gao et al. [33] RN-CNN1 92.95
Gao et al. [34] ESTCNN2 97.37
Budak et al. [35] LSTM 94.31
Jiao et al. [36] LSTM 98.14
Yang et al. [46] CNBLS3 99.58
Chai et al. [37] sparse-DBN4 93.1
Notes: 1RN-CNN, a Recurrence Network-based CNN; 2ESTCNN, EEG-based Spatial–Temporal CNN; 3CNBLS,
Complex Network based Broad Learning System; 4sparse-DBNsparse-deep belief networks.

CNNs are commonly utilized in image recognition and classification [47]. It can automatically and
adaptively learn the feature space hierarchy at each level and reduce the number of learned parameters
[48], thereby improving training efficiency to avoid overfitting and reduce network complexity. CNNs
have been proverbially applied in EEG-based fatigue detection and has achieved good results in recent years.
Zeng et al. [32] designed two novel models named EEG-Conv and EEG-Conv-R to detect fatigue
through EEG. EEG-Conv is a traditional CNN, and EEG-Conv-R integrates CNN with deep residual
learning. EEG-Conv-R converges more rapidly and gets a higher accuracy in intra-subject compared to
EEG-Conv. The idea of residual learning in EEG-Conv-R has an excellent learning effect on nonlinear
EEG signals and can reduce the interferences of the signals. CNN is more appropriate for extracting
spatial features from EEG signals, while temporal features are easily ignored. Gao et al. [34] built a novel
ESTCNN whose core block is introduced to extract temporal features. And dense layers were employed
to merge spatial features and finish classification. ESTCNN has excellent performance whose
classification accuracy reached 97.37%. Meanwhile, ESTCNN reduces the processing of multi-channel
data, which is more conducive to realizing the BCI online fatigue detection system. Many researchers
also combined RNN with CNN to extract spatiotemporal features. Gao et al. [33] employed a Recurrent
Network (RN) to analyses time series. The information matrix of RN was used as the input of CNN. This
model applied multiple RNs for dimensionality reduction and feature extraction and then CNN for
classification. The accuracy of RN-CNN reached 92.95%, which has good classification performance.
RNN can easily to lead the problems of gradient vanishing and gradient explosion [49].
LSTM is a particular recurrent neural network (RNN) [50]. It is primarily to deal with gradient vanishing
and explosion issue during long sequence training [51]. Budak et al. [35] designed three building blocks to
detect drowsiness. These three building blocks used different methods to extract different features,
respectively. Then all the features of the three building blocks were adopted for the LSTM network to
classification. And the result of LSTM was combined with a majority voting layer. The final experiment
got an average accuracy of 94.31%. This method employed TQWT to extract different features and
combine them on the multi-voting mechanism of the LSTM network, which is reliable, robust and
efficient. LSTM may overfit due to less EEG data. Generative Adversarial Network (GAN) is widely
used in image generation and processing [52,53]. Now it is applied to expand the EEG data. Jiao et al.
[36] applied Conditional Wasserstein GAN to expand the sample, then utilized LSTM to process the time
information of EEG and EOG signals.
IASC, 2023, vol.35, no.3 3579

Although LSTM solves the gradient problem of traditional RNN, it also adds a lot of content and
parameters, making training more difficult [51]. Accordingly, researchers have also applied other deep
learning algorithms to EEG classification in recent years. The Broad Learning System (BLS) offers an
alternative to deep learning networks, which solves the problem of continuously increasing the network
layers or adjusting the parameters to improve the accuracy of deep models [54]. Yang et al. [46]
constructed a CNBLS which starts from the Multivariate Weighted Recurrence Networks (MWRNs).
They got an excellent result which the area under the curve (AUC) over 98% in all 11 subjects. What’s
more, CNBLS achieved superior average accuracy (99.58%), sensitivity (99.67%) and specificity
(99.48%). BLS has two major advantages: horizontal expansion and incremental learning because of its
single hidden layer structure. Different from deep Learning, BLS can use horizontal scaling to add new
enhancement nodes, feature nodes and input data to improve model performance.
Generally speaking, we use supervised learning to complete fatigue detection. But in real life, unlabeled
data is easily available, while labeled data is often hard to get, and labeling is time consuming and laborious.
Semi-Supervised learning is more suitable for real-world applications and has recently become a hot new
direction in deep learning. This method requires only a bit labeled samples and numerous unlabeled
samples. Chai et al. [37] improved the DBN and used 43 peoples’ EEG data to train and test the network.
They used a regularization term to add sparsity in sparse-DBN. It can prevent model overfitting and learn
low-level and high-level structures. The results showed that sparse-DBN has an excellent performance
whose accuracy achieved 93.1%. Forming the sparse-DBN using sparse-restricted Boltzmann machine
(RBM) enables learning of helpful low-level and high-level features for unlabeled data.

3 EEG-based Fatigue Detection in Cross-subject

Due to the large difference between EEG among individuals, a reliable model should be constructed
after training the collected data from the same person. Furthermore, this model is not suitable for new
users, and the original model is less robust. Therefore, it is critical to establish a model in real application
which can detect fatigue in cross-subject. In recent studies, a small number of researchers used deep
learning on EEG-based fatigue detection in cross-subject, e.g., enhancing CNN to improve cross-subject
accuracy. Most researchers tried to use transfer learning to reduce the variability between subjects, which
can transfer the information from the source domain to the target domain. A few algorithms about
domain adaptions were employed in this field, such as Maximum Independence Domain Adaptation
(MIDA), DANN, Easy Transfer Learning (EasyTL). Tab. 3 is the summary of all the cross-subject studies
about the author, classification algorithm and accuracy. The first two items in the table used CNN for
classification, the next three items applied domain adaptation to classification, and the last one improved
easy transfer learning (EasyTL) as the classifier.

Table 3: The summary of cross-subject algorithms used in EEG-based fatigue detection

Reference Classification algorithm Accuracy (%)
Paulo et al. [55]# CNN 75.87
Cui et al. [56]# CNN 78.35
Wu et al. [57] OwARR1 <NG>
Liu et al. [58]# MDIA, TCA2 73.01
Zeng et al. [59] GDANN3 91.63
Zeng et al. [60] InstanceEasyTL 88.33
Notes: #The study applied a public dataset to train and test the network [61]; 1OwARR, Online weighted Adaptation
Regularization for Regression; 2MIDA, Maximum Independence Domain Adaptation; 3GDANN, Generative-DANN;
<NG> Not given.
3580 IASC, 2023, vol.35, no.3

Paulo et al. [55] applied a Butterworth filter to get the power density of the theta (4–8 Hz), alpha
(8–13 Hz), and beta (13–30 Hz). Then they calculated attention metrics associated with fatigue. The
recurrence plot and the gramian angular fields were applied to convert the above features to images
which was the input of CNN. As a result, they achieved an average precision of 75.87% on 27 subjects
for leave-one-out cross-validation. Cui et al. [56] tried to improved an interpretation technique called
class activation map (CAM) to discover the factors that favor and disfavor classification. Separable
convolution was utilized to reduce parameters and speed up network convergence. And a seven layers
CNN was used to recognize fatigue. The results showed that the average accuracy of the network reached
78.35% in the leave-one-out cross-validation of 11 subjects.
The deep learning algorithms currently seem to have poor theoretical interpretability in cross-subject
classification, and the experimental results need to be improved. Therefore, many researchers have
focused on the study of transfer learning algorithms to improve the performance of EEG-based fatigue
detection in cross-subject. The most commonly transfer learning method is domain adaptation, in which
the source task is the same as the target task, but the data distribution in the two domains are different.
And the source domain has numerous labeled samples, but the target domain has no (or only very few)
labeled samples [62]. Wu et al. [57] developed an algorithm called OwARR that combined fuzzy sets
with domain adaptation to cut back the number of alignment data for specific objects. But when using
OwARR for each source domain, it consumed a lot of time. Therefore, a source domain selection (SDS)
method was employed for reducing the calculated cost of OwARR. The results showed that OwARR has
high applicability, and SDS could reduce the computing time of OwARR by about half. TCA can
mitigate the distribution mismatch between source and target data. Liu et al. [58] used RF to
select channels and TCA to classify the selected one channel. At the same time, a transfer learning
algorithm called MIDA was applied for classifiers in all channels. The results showed that the
performance of MIDA is slightly better than that of TCA, which indicates that although TCA could
transfer knowledge, it will affect the classification performance. In a recent study, GAN was utilized to
effectively reduce the difference between training and test distribution and improve generalization
performance. Zeng et al. [59] combined GAN and DANN to propose a Generative-DANN model
(GDANN). In the cross-subject EEG-based fatigue detection, GDANN could achieve 91.63% accuracy. It
could be concluded that for subjects with better DANN classification, GDANN can slightly improve
accuracy, while for subjects with poor DANN performance, GDANN can significantly improve. As
shown in Fig. 3, GDANN applied a generate network from GAN to generate the fake data and
discriminant network to select the most suitable source domain subjects. Then using the selected data and
fake data to train the DANN.
In addition to domain adaptation, other transfer learning methods were also used in brain-computer
interface. For EEG-based fatigue detection, there are not many methods currently applied. EasyTL was
designed by Wang et al. [63]. It does not need to select model and adjust hyperparameter. It can learn
non-parametric transfer features and classifiers at the same time. Zeng et al. [60] improved EasyTL and
proposed a classifier called InstanceEasyTL for cross-subject EEG-based fatigue detection. It shows that
the InstanceEasyTL could use fewer data to get a higher accuracy and robustness model. In the same
cross-subject experiment, InstanceEasyTL could reach an average accuracy of 88.33%, much higher than
DANN (70.75%) and EasyTL (70.91%).
IASC, 2023, vol.35, no.3 3581

Figure 3: The structure of GDANN [59]. (The process of minimizing Nf, Ny, and maximizing Nd to train a
DANN, which involves sifting the data for the previous Ns source-domain subjects most resemblance to the
target domain and using random noise to generate data to train a domain-aligned model together)

4 Limitations and Development Directions of EEG-based Fatigue Detection

Over the past few decades, EEG-based fatigue detection has made great progress. However, it still has
great limitations. Since the EEG signal is usually feeble and the signal-to-noise ratio is very low, for most
algorithms in intra-subject, accurate feature extraction and efficient classification algorithms are usually
required to obtain high accuracy. Meanwhile, for the traditional machine learning algorithms, it is difficult
to take advantage of deep features, so high accuracy is hard to achieve when utilizing traditional feature
extraction methods [64]; but when using feature extraction network, a large amount of data is required to
train the network to ensure accuracy. And when we employ deep learning as a classifier, it is hard to
explain the training process of the model [65]. What’s worse, it takes a long time and an amount of EEG
data to build the model while the data can’t be collected in large quantities [66]. Besides, these trained
models are not available online because of the complicated process. Although there are so many
problems to be solved, it can be seen from the literature that in recent years, most researches tend to use
deep learning. With numerous data, establishing a deep learning algorithm with strong interpretability,
high accuracy, and real-time is an important future direction.
Because the EEG signal is non-stationary, the algorithms in intra-subject cannot achieve good results
when using cross-subject training data [67]. However, it is often arduous to require each subject to do
extensive calibration exercises in practical applications. The algorithms in cross-subject can build a
universal classifier to a certain extent [60]. However, in terms of the current transfer learning results, the
accuracy of cross-subjects is too low, which needs to be improved urgently. In the case of small or zero
training samples, improving the accuracy and scope of transfer learning methods is another direction. At
present, researchers need to improve the feature extraction method and classification method. Some
traditional algorithm ideas, such as BP neural network [68], may have certain reference value.
3582 IASC, 2023, vol.35, no.3

EEG-related studies have shown that multimodal-based systems can extract or learn the features of
multiple signals. Their overall performance is higher than single-modality systems [69–72]. In the study
of EEG emotion recognition, some authors fused facial expressions with EEG to analyze [71,72].
Therefore, it is also a future research direction to study multi-modal algorithms to combine other signals,
such as behavioral signals, EOG and EEG, to improve fatigue detection. According to the research of
Laurent et al. [23], combining EEG features with ECG and EOG features can obviously improve
classification performance for short time windows.
Both deep learning and traditional machine learning methods require numerous labeled samples.
Furthermore, the transfer learning method is more effective if there are more samples. Therefore, it is
very important to collect many training samples in the future. How to design new effective paradigms,
improve the wearable and portable sensor and save huge useful EEG data in the case of information
security are important issues. As an ever-evolving field of study, fatigue detection research increasingly
requires multidisciplinary collaborative efforts.

5 Conclusions
The fast-paced life in today’s society can easily lead to fatigue which can directly or indirectly cause
harm to the human body. There is no doubt that high-precision, real-time, and universal fatigue detection
is of great significance. This study reviewed the progress of EEG-based fatigue detection in recent years
from two types of algorithms: intra-subject and cross-subject. The summary of these methods provided a
solid theoretical foundation for future algorithmic research in this field. Moreover, we also analyzed the
existing problems of EEG-based fatigue detection and provided some directions for solutions. As an
evolving research area, fatigue detection needs to combine multi-disciplinary knowledge to improve the
performance. While improving the detection algorithm, designing new effective paradigms and portable
equipment, and collecting valid standard data are also urgent.
Funding Statement: This project was funded by the National Natural Science Foundation of China (Grant
Nos. 61906019, 62006082 and 62076103), the Guangdong Basic and Applied Basic Research Foundation
(Grant Nos. 2021A1515011853, 2021A1515011600 and 2020A1515110294), Guangzhou Science and
Technology Plan Project (Grant No. 202102020877) and the Guangzhou Science and Technology Plan
Project Key Field R&D Project (202007030005).

Conflicts of Interest: The authors declare that they have no conflicts of interest to report regarding the
present study.

References
[1] L. S. Aaronson, C. S. Teel, V. Cassmeyer, G. B. Neuberger, L. Pallikkathayil et al., “Defining and measuring
fatigue,” Image: The Journal of Nursing Scholarship, vol. 31, no. 1, pp. 45–50, 1999.
[2] S. Nozaki, M. Tanaka, K. Mizuno, S. Ataka, H. Mizuma et al., “Mental and physical fatigue-related biochemical
alterations,” Nutrition, vol. 25, no. 1, pp. 51–57, 2009.
[3] R. D. Moore, M. W. Romine, P. J. O’connor and P. D. Tomporowski, “The influence of exercise-induced fatigue
on cognitive function,” Journal of Sports Sciences, vol. 30, no. 9, pp. 841–850, 2012.
[4] S. Kar, M. Bhagat and A. Routray, “EEG signal analysis for the assessment and quantification of driver’s fatigue,”
Transportation Research Part F: Traffic Psychology and Behaviour, vol. 13, no. 5, pp. 297–306, 2010.
[5] B. C. Tefft, Acute Sleep Deprivation and Risk of Motor Vehicle Crash Involvement, Washington, DC, USA: AAA
Foundation for Traffic Safety, 2016. [Online]. Available: https://aaafoundation.org/acute-sleep-deprivation-risk-
motor-vehicle-crash-involvement/.
[6] T. Johnson, 2017 Traffic Safety Culture Index, Washington, DC, USA: AAA Foundation for Traffic Safety, 2018.
[Online]. Available: https://aaafoundation.org/2017-traffic-safety-culture-index/.
IASC, 2023, vol.35, no.3 3583

[7] M. Basner and J. Rubinstein, “Fitness for duty: A 3-minute version of the psychomotor vigilance test predicts
fatigue-related declines in luggage-screening performance,” Journal of Occupational and Environmental
Medicine, vol. 53, no. 10, pp. 1146–1154, 2011.
[8] A. J. Beurskens, U. Bültmann, U. Kant, J. H. Vercoulen, G. Bleijenberg et al., “Fatigue among working people:
Validity of a questionnaire measure,” Occupational and Environmental Medicine, vol. 57, no. 5, pp. 353–357,
2000.
[9] M. R. Smith, A. J. Coutts, M. Merlini, D. Deprez, M. Lenoir et al., “Mental fatigue impairs soccer-specific
physical and technical performance,” Med. Sci. Sports Exerc., vol. 48, no. 2, pp. 267–276, 2016.
[10] M. H. Sigari, M. Fathy and M. Soryani, “A driver face monitoring system for fatigue and distraction detection,”
International Journal of Vehicular Technology, vol. 2013, no. 263983, pp. 1–11, 2013.
[11] H. Wang, A. Dragomir, N. I. Abbasi, J. Li, N. V. Thakor et al., “A novel real-time driving fatigue detection system
based on wireless dry EEG,” Cognitive Neurodynamics, vol. 12, no. 4, pp. 365–376, 2018.
[12] J. Hu and J. Min, “Automated detection of driver fatigue based on EEG signals using gradient boosting decision
tree model,” Cognitive Neurodynamics, vol. 12, no. 4, pp. 431–440, 2018.
[13] M. Kołodziej, P. Tarnowski, D. J. Sawicki, A. Majkowski, R. J. Rak et al., “Fatigue detection caused by office
work with the use of EOG signal,” IEEE Sensors Journal, vol. 20, no. 24, pp. 15213–15223, 2020.
[14] M. Song, L. Li, J. Guo, T. Liu, S. Li et al., “A new method for muscular visual fatigue detection using
electrooculogram,” Biomedical Signal Processing and Control, vol. 58, pp. 101865, 2020.
[15] Q. Abbas, M. E. Ibrahim, S. Khan and A. R. Baig, “Hypo-driver: A multiview driver fatigue and distraction level
detection system,” Computers, Materials & Continua, vol. 71, no. 1, pp. 1999–2007, 2022.
[16] P. A. Karthick, D. M. Ghosh and S. Ramakrishnan, “Surface electromyography based muscle fatigue detection
using high-resolution time-frequency methods and machine learning algorithms,” Computer Methods and
Programs in Biomedicine, vol. 154, pp. 45–56, 2018.
[17] L. Wang, H. Wang and X. Jiang, “A new method to detect driver fatigue based on EMG and ECG collected by
portable non-contact sensors,” Promet-Traffic&Transportation, vol. 29, no. 5, pp. 479–488, 2017.
[18] F. Wang, H. Wang and R. Fu, “Real-time ECG-based detection of fatigue driving using sample entropy,” Entropy,
vol. 20, no. 3, pp. 196, 2018.
[19] S. J. Jung, H. S. Shin and W. Y. Chung, “Driver fatigue and drowsiness monitoring system with embedded
electrocardiogram sensor on steering wheel,” IET Intelligent Transport Systems, vol. 8, no. 1, pp. 43–50, 2014.
[20] B. L. Egleston, S. M. Miller and N. J. Meropol, “The impact of misclassification due to survey response fatigue on
estimation and identifiability of treatment effects,” Statistics in Medicine, vol. 30, no. 30, pp. 3560–3572, 2011.
[21] W. J. Chang, L. B. Chen and Y. Z. Chiou, “Design and implementation of a drowsiness-fatigue-detection system
based on wearable smart glasses to increase road safety,” IEEE Transactions on Consumer Electronics, vol. 64,
no. 4, pp. 461–469, 2018.
[22] F. Bi, X. Ma, W. Chen, W. Fang, H. Chen et al., “Review on video object tracking based on deep learning,”
Journal of New Media, vol. 1, no. 2, pp. 63–74, 2019.
[23] F. Laurent, M. Valderrama, M. Besserve, M. Guillard, J. P. Lachaux et al., “Multimodal information improves the
rapid detection of mental fatigue,” Biomedical Signal Processing and Control, vol. 8, no. 4, pp. 400–408, 2013.
[24] F. Wang, S. Wang, X. Wang, Y. Peng and Y. Yang, “Design of driving fatigue detection system based on hybrid
measures using wavelet-packets transform,” in Proc. of the 2014 IEEE Int. Conf. on Robotics and Automation,
Hong Kong, HK, China, pp. 4037–4042, 2014.
[25] A. Borowicz, “Using a multichannel Wiener filter to remove eye-blink artifacts from EEG data,” Biomedical
Signal Processing and Control, vol. 45, pp. 246–255, 2018.
[26] M. K. Wali, M. Murugappan and B. Ahmmad, “Wavelet packet transform based driver distraction level
classification using EEG,” Mathematical Problems in Engineering, vol. 2013, pp. 297587, 2013.
[27] C. Zhao, C. Zheng, M. Zhao, Y. Tu and J. Liu, “Multivariate autoregressive models and kernel learning algorithms
for classifying driving mental fatigue based on electroencephalographic,” Expert Systems with Applications, vol.
38, no. 3, pp. 1859–1865, 2011.
3584 IASC, 2023, vol.35, no.3

[28] Y. Ma, B. Chen, R. Li, C. Wang, J. Wang et al., “Driving fatigue detection from EEG using a modified PCANet
method,” Computational Intelligence and Neuroscience, vol. 2019, pp. 4721863, 2019.
[29] J. Hu, “Comparison of different features and classifiers for driver fatigue detection based on a single EEG
channel,” Computational and Mathematical Methods in Medicine, vol. 2017, pp. 5109530, 2017.
[30] T. Tuncer, S. Dogan, F. Ertam and A. Subasi, “A dynamic center and multi threshold point based stable feature
extraction network for driver fatigue detection utilizing EEG signals,” Cognitive Neurodynamics, vol. 15, pp.
223–237, 2021.
[31] C. Zhang, H. Wang and R. Fu, “Automated detection of driver fatigue based on entropy and complexity
measures,” IEEE Transactions on Intelligent Transportation Systems, vol. 15, no. 1, pp. 168–177, 2013.
[32] H. Zeng, C. Yang, G. Dai, F. Qin, J. Zhang et al., “EEG classification of driver mental states by deep learning,”
Cognitive Neurodynamics, vol. 12, no. 6, pp. 597–606, 2018.
[33] Z. K. Gao, Y. L. Li, Y. X. Yang and C. Ma, “A recurrence network-based convolutional neural network for fatigue
driving detection from EEG,” Chaos: An Interdisciplinary Journal of Nonlinear Science, vol. 29, no. 11, pp.
113126, 2019.
[34] Z. Gao, X. Wang, Y. Yang, C. Mu, Q. Cai et al., “EEG-based spatio–temporal convolutional neural network for
driver fatigue evaluation,” IEEE Transactions on Neural Networks and Learning Systems, vol. 30, no. 9, pp.
2755–2763, 2019.
[35] U. Budak, V. Bajaj, Y. Akbulut, Q. Atila and A. Sengur, “An effective hybrid model for EEG-based drowsiness
detection,” IEEE Sensors Journal, vol. 19, no. 17, pp. 7624–7631, 2020.
[36] Y. Jiao, Y. Deng, Y. Luo and B. L. Lu, “Driver sleepiness detection from EEG and EOG signals using GAN and
LSTM networks,” Neurocomputing, vol. 408, pp. 100–111, 2020.
[37] R. Chai, S. H. Ling, P. P. San, G. R. Naik, T. N. Nguyen et al., “Improving EEG-based driver fatigue classification
using sparse-deep belief networks,” Frontiers in Neuroscience, vol. 11, pp. 103, 2017.
[38] Y. Cui, Y. Xu and D. Wu, “EEG-based driver drowsiness estimation using feature weighted episodic training,”
IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 27, no. 11, pp. 2263–2273, 2019.
[39] R. Chai, G. R. Naik, T. N. Nguyen, S. H. Ling, Y. Tran et al., “Driver fatigue classification with independent
component by entropy rate bound minimization analysis in an EEG-based system,” IEEE Journal of
Biomedical and Health Informatics, vol. 21, no. 3, pp. 715–724, 2016.
[40] N. Kumar, K. Alam and A. H. Siddiqi, “Wavelet transform for classification of EEG signal using SVM and ANN,”
Biomedical and Pharmacology Journal, vol. 10, no. 4, pp. 2061–2069, 2017.
[41] S. Thakur, R. Dharavath and D. R. Edla, “Spark and rule-KNN based scalable machine learning framework for
EEG deceit identification,” Biomedical Signal Processing and Control, vol. 58, pp. 101886, 2020.
[42] J. V. Tu, “Advantages and disadvantages of using artificial neural networks versus logistic regression for
predicting medical outcomes,” Journal of Clinical Epidemiology, vol. 49, no. 11, pp. 1225–1231, 1996.
[43] F. Burden and D. Winkler, “Bayesian regularization of neural networks,” Artificial Neural Networks, vol. 458, pp.
23–42, 2008.
[44] M. S. Khan and P. Coulibaly, “Bayesian neural network for rainfall-runoff modeling,” Water Resources Research,
vol. 42, no. 7, pp. W07409, 2006.
[45] Y. LeCun, Y. Bengio and G. Hinton, “Deep learning,” Nature, vol. 521, no. 7553, pp. 436–444, 2015.
[46] Y. Yang, Z. Gao, Y. Li, Q. Cai, N. Marwan et al., “A complex network-based broad learning system for detecting
driver fatigue from EEG signals,” IEEE Transactions on Systems, Man, and Cybernetics: Systems, vol. 51,
pp. 5800–5808, 2019.
[47] J. Qin, W. Pan, X. Xiang, Y. Tan and G. Hou, “A biological image classification method based on improved
CNN,” Ecological Informatics, vol. 58, pp. 101093, 2020.
[48] Y. Luo, J. Qin, X. Xiang, Y. Tan, Q. Liu et al., “Coverless real-time image information hiding based on image
block matching and dense convolutional network,” Journal of Real-Time Image Processing, vol. 17, no. 1, pp.
125–135, 2020.
IASC, 2023, vol.35, no.3 3585

[49] Y. Huang, L. Shen and H. Liu, “Grey relational analysis, principal component analysis and forecasting of carbon
emissions based on long short-term memory in China,” Journal of Cleaner Production, vol. 209, pp. 415–423,
2019.
[50] A. Sherstinsky, “Fundamentals of recurrent neural network (RNN) and long short-term memory (LSTM)
network,” Physica D: Nonlinear Phenomena, vol. 404, pp. 132306, 2020.
[51] Ö. Yildirim, “A novel wavelet sequence based on deep bidirectional LSTM network model for ECG signal
classification,” Computers in Biology and Medicine, vol. 96, pp. 189–202, 2018.
[52] W. Fang, E. Gu, W. Yi, W. Wang and V. S. Sheng, “A new method of image restoration technology based on
WGAN,” Computer Systems Science and Engineering, vol. 41, no. 2, pp. 689–698, 2022.
[53] P. Gong, J. Liu and S. Lv, “Image denoising with GAN based model,” Journal of Information Hiding and Privacy
Protection, vol. 2, no. 4, pp. 155–163, 2020.
[54] C. L. P. Chen and Z. Liu, “Broad learning system: An effective and efficient incremental learning system without
the need for deep architecture,” IEEE Transactions on Neural Networks and Learning Systems, vol. 29, no. 1, pp.
10–24, 2017.
[55] J. R. Paulo, G. Pires and U. J. Nunes, “Cross-subject zero calibration driver’s drowsiness detection: Exploring
spatiotemporal image encoding of EEG signals for convolutional neural network classification,” IEEE
Transactions on Neural Systems and Rehabilitation Engineering, vol. 29, pp. 905–915, 2021.
[56] J. Cui, Z. Lan, O. Sourina and W. Müller-Wittig, “EEG-based cross-subject driver drowsiness recognition with an
interpretable convolutional neural network,” IEEE Transactions on Neural Networks and Learning Systems,
vol. 2022, pp. 1–13, 2022.
[57] D. Wu, V. J. Lawhern, S. Gordon, B. J. Lance and C. T. Lin, “Driver drowsiness estimation from EEG signals
using online weighted adaptation regularization for regression (OwARR),” IEEE Transactions on Fuzzy
Systems, vol. 25, no. 6, pp. 1522–1535, 2016.
[58] Y. Liu, Z. Lan, J. Cui, O. Sourina and W. Müller-Wittig, “Inter-subject transfer learning for EEG-based mental
fatigue recognition,” Advanced Engineering Informatics, vol. 46, pp. 101157, 2020.
[59] H. Zeng, X. Li, G. Borghini, Y. Zhao, P. Aricò et al., “An EEG-based transfer learning method for cross-subject
fatigue mental state prediction,” Sensors, vol. 21, no. 7, pp. 2369, 2021.
[60] H. Zeng, J. Zhang, W. Zakaria, F. Babiloni, B. Gianluca et al., “InstanceEasyTL: An improved transfer-learning
method for EEG-based cross-subject fatigue detection,” Sensors, vol. 20, no. 24, pp. 7251, 2020.
[61] Z. Cao, C. H. Chuang, J. K. King and C. T. Lin, “Multi-channel EEG recordings during a sustained-attention
driving task,” Scientific Data, vol. 6, no. 1, pp. 1–8, 2019.
[62] S. J. Pan and Q. Yang, “A survey on transfer learning,” IEEE Transactions on Knowledge and Data Engineering,
vol. 22, no. 10, pp. 1345–1359, 2009.
[63] J. Wang, Y. Chen, H. Yu, M. Huang and Q. Yang, “Easy transfer learning by exploiting intra-domain structures,”
in Proc. of the 2019 IEEE Int. Conf. on Multimedia and Expo, Shanghai, SH, China, pp. 1210–1215, 2019.
[64] Y. Yang, D. Li and Z. Duan, “Chinese vehicle license plate recognition using kernel-based extreme learning
machine with deep convolutional features,” IET Intelligent Transport Systems, vol. 12, no. 3, pp. 213–219, 2018.
[65] C. Wu, M. J. F. Gales, A. Ragni, P. Karanasou and K. C. Sim, “Improving interpretability and regularization in
deep learning,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, vol. 26, no. 2, pp. 256–
265, 2017.
[66] L. Sun, H. Gao, S. Pan and J. X. Wang, “Surrogate modeling for fluid flows based on physics-constrained deep
learning without simulation data,” Computer Methods in Applied Mechanics and Engineering, vol. 361, pp.
112732, 2020.
[67] M. A. Duhayyim, H. M. Alshahrani, F. N. Al-Wesabi, M. A. Al-Hagery, A. M. Hilal et al., “Intelligent machine
learning based EEG signal classification model,” Computers, Materials & Continua, vol. 71, no. 1, pp. 1821–
1835, 2022.
[68] X. R. Zhang, X. Sun, W. Sun, T. Xu and P. P. Wang, “Deformation expression of soft tissue based on BP neural
network,” Intelligent Automation & Soft Computing, vol. 32, no. 2, pp. 1041–1053, 2022.
3586 IASC, 2023, vol.35, no.3

[69] Y. Li, J. Pan, J. Long, T. Yu, F. Wang et al., “Multimodal BCIs: Target detection, multidimensional control, and
awareness evaluation in patients with disorder of consciousness,” Proceedings of the IEEE, vol. 104, no. 2, pp.
332–352, 2015.
[70] H. V. Chand and J. Karthikeyan, “CNN based driver drowsiness detection system using emotion analysis,”
Intelligent Automation & Soft Computing, vol. 31, no.2, pp. 717–728, 2022.
[71] Y. Huang, J. Yang, P. Liao and J. Pan, “Fusion of facial expressions and EEG for multimodal emotion
recognition,” Computational Intelligence and Neuroscience, vol. 2017, pp. 1–8, 2017.
[72] Y. Huang, J. Yang, S. Liu and J. Pan, “Combining facial expressions and electroencephalography to enhance
emotion recognition,” Future Internet, vol. 11, no. 5, pp. 105, 2019.