Classification of fine hand movements of the same limb

through EEG signals

Jorge A. Sanchez, Javier M. Antelis, and Omar Mendoza-Montoya.

Tecnologico de Monterrey, Escuela de Ingenieria y Ciencias
Monterrey, NL 64849
[email protected]

Abstract
Discriminating fine movements within the same limb using electroencephalography
(EEG) signals is a current challenge to non-invasive BCIs systems due to the close
spatial representations on the motor cortex area of the brain, the signal-to-noise
ratio, and the stochastic nature of this kind of signals. This research presents
the performance evaluation of different strategies of classification using Linear
Discriminant Analysis (LDA) method and power spectral density (PSD) features
for three tasks: make a fist, open the hand, and keep the anatomical position of the
hand. For this, EEG signals were collected from 10 healthy subjects and evaluated
with different cross-validation methods: Monte Carlo, to implement an Offline
Analysis And Leave-one-out for a pseudo-online implementation. The results show
that the average accuracy for classifying the start of each task is approximately 76%
for offline and Pseudo-online Analysis, classifying just the start of movement is
54% and 62% respectively for same both methods and 45% for and 32% classifying
between classes. Based on these results, it can be said that the implementation
of a BCI based on PSD features and LDA method could work to detect the start
of one of the proposed tasks, but to discriminate the movement it is necessary to
implement a different strategy for increase accuracy in the classification problem.

1 Introduction
A brain-computer interface (BCI) is a technology that allows a human brain to communicate with the
outside world through an artificial pathway Chaudhary et al. [2016]. It is a system that translates a
mental task into commands that can operate a device Wolpaw [2007]. This translation can be done
through the recording of brain activity by invasive and non-invasive techniques Lebedev and Nicolelis
[2006] and the controlled devices can allow the user’s spelling with a virtual keyboard Scherer
et al. [2004], navigating the World Wide Web Birbaumer et al. [2004] or controlling exoskeletons
and prostheses Al-Quraishi et al. [2018]. While these BCI examples achieve their goals with some
degree of success, they are based on discrete task classification, and the need to classify tasks that
involve continuous estimation of parameters such as the fine motion of a human hand remains a latent
challenge for the successful operation of non-invasive BCI Hosseini and Shalchyan [2022].
One of the most used technologies to implement non-invasive BCIs is electroencephalography
(EEG) due to its good temporal resolution, cost-effectiveness and portability Hooda et al. [2020],
nevertheless, the use of these types of signals to discriminate the motor task of different movements
within the same limb is an actual challenge due to the stochastic nature of these signals, the signal-to-
noise ratio, and close spatial representations in the motor cortex area for these kinds of tasks Yong
and Menon [2015], Shiman et al. [2017].
The protocol proposed in this research involves motor execution: the test subject executes and
maintains by 5 seconds the movement that is requested through a visual stimulus. This type of

36th Conference on Neural Information Processing Systems (NeurIPS 2022).

task induces changes in power values across the frequency bands that EEG signals are commonly
analyzed, but it is expected that the relevant changes for this process will be in the power values of
the sensorimotor band Graimann and Pfurtscheller [2006], Hooda et al. [2020]. The idea is that while
the test subject performs these tasks, brain activity is recorded, and then power spectral density (PSD)
features are extracted to be used as discriminant variables between these movements. It is because
power spectral changes are observed during motor tasks such as movement execution, imagery, or
attempt Hernández-Rojas et al. [2020].

2 Materials and Methods

The present research evaluates different classifiers based on power spectral density (PSD) features
and linear discrimination analysis (LDA) to determine two things: first, if it is possible to determine
the fine movement onset of the hand, and second if it is possible to discretize between the movements
of the same hand. This analysis is done with electroencephalography signals acquired from 10 test
subjects (4 females and 6 males with an average age of 23).

2.1 Experimental Protocol

To collect the raw signal, a experimental protocol was run that guided test subjects through visual
stimuli by attention, execution, and rest sections. These images that were presented to the subjects, as
well as the duration times of each image, are shown in figure 1. This protocol allowed measuring the
electroencephalography signals during 90 trials in each session, which means that for each subject 30
trials per class were recorded.

(a) Images presented and their duration in a trial for the different sections of an experimental session.

(b) Images presented to test subjects to tell them what type of task to perform.
Figure 1: Visual stimuli by attention, execution, and rest sections

A trial is a time section of 11 seconds, as can be seen in figure 1, which consists of three sections:
attention, execution, and rest. In the first stage of attention a fixation cross is placed on the screen
indicating to the test subject that they should be aware of the type of movement that has to do after
this cross. This visual stimulus is presented for 3 seconds and the subject is asked to try not to blink
or make sudden movements while the screen presents this image.
After presenting the fixation cross, a visual stimulus is presented for 5 seconds that indicates the
movement that must be executed. This image can instruct the test subject to make a fist, extend the
hand, or simply do nothing. During this stage of motor execution, the subject is asked to execute the
requested movement with the greatest possible force and to hold it until rest advice appears.
The final step in the recording process of a trial is the resting stage that cover lasts 5 seconds. An
image is presented with the word "descanso" that indicates to the test subjects that they can blink,
adjust their position to be more comfortable, and prepare for the next trial that will start with the next
fixation cross.

2
This experiment aims to evaluate the difference in brain activity during the fixation cross attention
task, which is the first stage of the instruction given to the test subject, and the different motor tasks
(including holding anatomical position) and evaluate de differences between the performed tasks.

2.2 Data Collection

To collect the data analyzed in this research, a high-performance neuroscience research system was
used, the g.HIamp by g.Tec. With this device, 32 EEG channels were recorded simultaneously with a
sampling frequency of 1200 Hz. The configuration of channels that was used in this investigation is
shown in figure 2

Figure 2: 32-channel configuration for recording EEG signals

2.3 Pre-processing

to pre-processing the signals, the first step is to resample the frequency to 256 Hz, then a band-pass
Butterworth filter was applied from 2 to 30 Hz. This frequency range is sufficient for the feature
extraction needs that are used to train and test the classifiers. In addition to this bandpass filter, a
notch filter was applied to reject the noise that the electrical network could apply to the analyzed
signal.
Once these filters were applied, the signals obtained were analyzed with the intention of eliminating
noisy data from channels and entire trials that, due to artifacts, movements, or any other situation,
generate noise and can generate bad classifications or a malfunction in the implemented classifiers.

2.4 Power Spectral Density (PSD)

The features used to classify the different tasks in this research is the power spectral density (PSD)
of the EEG signals. It is a widely used feature in tasks such as motor execution, motor imagination,
or movement intention because a change in spectral power can be observed in these types of tasks
Hernández-Rojas et al. [2020]. This feature is a way to represent the distribution of the frequency
components of the analyzed signal, that is, it can represent the proportion of the total power of
the signal that is contributed by each frequency component Dempster [2001]. For each of the
classification methodologies, the PSD was calculated in a range from 2 to 30 Hz with a resolution of
1 Hz in analysis windows of one second.

2.5 Feature selection

The features used correspond to Power Spectral Density and are calculated with a resolution of 1 Hz,
this means that for each data 893 features were obtained, with 28 frequency-related values per each
of the 32 electrodes. To adjust the number of features to 3 times the amount of data, Spearman’s
correlation coefficients were measured and the features that obtained the lowest values of correlation
with the classified task were discarded.

3
2.6 Classification methodologies

Different linear discriminant analysis (LDA) classifiers were implemented in this research due to its
low computational cost Ahmadizadeh et al. [2021]. The objective of this research is to evaluate the
possibility of implementing a human-machine interface capable of classifying the start of a movement
and discretize between different movements in the same hand, For this, two types of cross-validation
were implemented: Monte Carlo to perform an offline analysis and leave-one-out to implement a
pseudo-online analysis. With each of these validations, the performance of the implemented classifiers
was analyzed to detect the start of each of the tasks described in section 2.1 and to discretize between
them.

2.6.1 Offline Analysis

The Monte Carlo Cross-validation technique was implemented for the offline analysis, where the
training set corresponds to 80% of the recorded trials and the remaining 20% to test it. This process
was repeated 500 times and for each time the data sets were randomly defined. To obtain the features,
two analysis windows were used as shown in Figure 3. The analysis window for the fixation cross
was taken in the last second of that section and the analysis window for each tasks execution was
taken from 3.2 seconds to 4.2 seconds, that is, 0.2 seconds after the image indicating the execution of
the movement has been presented.

Figure 3: Monte Carlo cross Validation windows for offline Analysis

Classifying the start of the movement: To classify the task onset, the data corresponding to each
one was isolated and the PSD features were obtained for the fixation cross and task execution analysis
windows, then the Cross-validation technique was applied to the data. After eliminating the noise
trials, an average of 28 trials for each of these tasks by subject, that is, to achieve this classification
objective, there were by subject approximately 56 data per task; 44 for training and 12 for testing.
After classifying each of these tasks, the second objective was to classify the start of a movement, this
is, only the trials corresponding to making a fist and extending the hand were added to the data set.

Classifying between tasks To discretize between the three classes of making a fist, extending
the hand, and maintaining the anatomical position, The features of PSD were acquired and the
same technique of cross-validation was implemented. But to achieve this objective, the features
corresponding to the fixation cross were avoided, a data set was created only with the information
corresponding to the beginning of the movement, that is, from 3.2 seconds to 4.2 seconds.Figure 4
shows the position of the analysis windows for both training and testing data sets.

Figure 4: windowing strategy for classification between classes using Monte Carlo cross-validation

2.6.2 Pseudo-online Analysis

The implementation of the leave-one-out Cross-validation technique allows a pseudo-online analysis
to be carried out with the same objectives as the analysis presented in section 2.6.1 for the offline
analysis: evaluate the performance of a classifier to detect the movement onset and to discretize

4
between the classes that are evaluated in this research. For this Analysis all the trials except one are
used to train the LDA classifier and the rest is used for testing it. This is repeated until each of the
trials is used as test data. Figure 5 shows the windows for the pseudo-online analysis. For the training
data set, the PSD features for the fixation cross and tasks execution windows are obtained in the same
way as they were taken for the offline analysis. Regarding the test data set, a mobile window was
implemented with the same duration as the training windows. This mobile window gets the PSD
features through time in trial test from its first position between 1 and 2 seconds to its last position
between 7 and 8 seconds with an overlapping of 0.1 seconds.

Figure 5: Leave-one-out cross validation windows for pseudo-Online analysis

Classifying the start of the movement In the same way that was done with the offline methodology,
for this methodology the classification between the fixation cross and each tasks was carried out,
followed by the classification of the start of the movement. For these two types of tests, the features
with which the classifiers were trained were obtained from fixed windows that are shown in Figure 5:
a window for the fixation cross from 2 to 3 seconds and another after 3.2 seconds for the execution of
each task.

Classifying between tasks For the classification between tasks, PSD features of each one of the
executions of the tasks were obtained, but also features of the fixation cross were used to train and
test the classifier. this implies a bias problem in the implementation of the classifier because for each
of the recorded trials there is a set of data for the fixation cross and another for the task, which means
that when the features were obtained, there were 3 times more data for the fixation cross than for
each one of the tasks. To solve this, the data from the fixation cross were randomly chosen and the
size of the data was adjusted to the maximum size of the data of the remaining classes.

3 Results
the results of this performance evaluation of classification, show an average accuracy of approximately
76% for the detection of the start of each task and 78% for the detection of the start of a movement in
the offline analysis, regarding the online analysis, the results show approximately 76% on average
for both: the start of each task and each movement. For the classification between the tasks, the
results were much lower, having a result of approximately 45% for the offline analysis and 35% for
the pseudo-online analysis.The results of the processes developed to evaluate the performance of the
classifiers proposed here are shown below.

3.1 Trial Rejection

The elimination of trials and noisy channels is done to prevent classifications from being made on
signals that are not related to mental tasks but that may be the result of muscle artifacts or poor
electrode positions that can be generated by movements of the test subjects. Table 1 shows the results
of the trials and the number of electrodes that remained at the end of this process. The electrodes that
were eliminated for each of the subjects that were part of this investigation are also named.

3.2 Offline classification

The first objective of the offline methodology was to see the performance of classification for
the detection of the start of the movement. Table 2 shows the average results after executing the
methodology with the Monte Carlo Cross-validation technique 500 times. It can be seen that the one
who had the best performance was subject 4 with an average classification of 91%, while subject two
had the lowest classification rate with 59%. Regarding the classified tasks, it can be seen that they

5
 Test Subject # of trials # of EEG Channels Eliminated EEG Channels
S1 83 32
S2 87 29 C4 CP5 O2
S3 85 31 CP5
S4 84 30 CP5 O2
S5 90 30 C4 CZ
S6 87 29 FZ CP6 O2
S7 82 29 P8 OZ O2
S8 89 30 T8 CP6
S9 78 29 P3 T8 FZ
S10 87 32
Table 1: Summary of the data after the elimination of trials and noisy channels

all had a similar classification average, with opening the hand being the lowest average with 73%
followed by making a fist with 74%, and maintaining the anatomical position with 75%.

Task S1 S2 S3 S4 S5 S6 S7 S8 S9 S10
Make a fist 0.66 0.60 0.74 0.92 0.68 0.82 0.84 0.59 0.80 0.71
Extend the hand 0.64 0.60 0.82 0.94 0.67 0.81 0.73 0.58 0.77 0.77
Do not move 0.87 0.56 0.71 0.87 0.63 0.79 0.81 0.68 0.79 0.81
Table 2: Average accuracy for the classification of each of the tasks

Table 3 shows the results average of movement initiation classification. It can be seen that subject 4
again has the best performance with 77% accuracy and subjects 2, 5, and 8 have the worst performance
with 61 and 62%.

S1 S2 S3 S4 S5 S6 S7 S8 S9 S10
Mean accuracy 0.77 0.61 0.79 0.93 0.62 0.83 0.84 0.61 0.82 0.82
Table 3: Average accuracy for classification of movement start for each of the subjects

Evaluating the classification between the proposed classes, table 4 presents the average values of
precision and recall for each class by subject. It can be seen that subject 9 is the one with the best
performance with an accuracy of 59% and that subject 8 is the one with the lowest performance with
an accuracy of 33%. Likewise, it can be seen that the class with the best recall on average is the one
related to not moving the hand with approximately 53%.

S1 S2 S3 S4 S5 S6 S7 S8 S9 S10
Accuracy of classifier 0.44 0.38 0.48 0.43 0.39 0.53 0.49 0.33 0.59 0.43
Recall for make a fist 0.37 0.40 0.46 0.36 0.36 0.42 0.39 0.32 0.59 0.44
Recall for extend the hand 0.29 0.43 0.45 0.50 0.43 0.47 0.52 0.31 0.58 0.37
Recall for do not move 0.68 0.32 0.56 0.47 0.40 0.72 0.58 0.40 0.62 0.52
Table 4: Average accuracy for classification of movement start for each of the subjects

3.3 Pseudo-online classification

The objective of the implementation of an online classifier is to evaluate the performance of an

eventual brain-machine interface implementation. Initially, the behavior of the classifier is evaluated
to detect the start of each task. For this, the classifier was evaluated using the test data shown in the
section 2.6.2 and an array of performance values was obtained. Table 5 shows the max accuracy
values for each of the evaluated tasks. It can be observed in this type of analysis that the best
performance is again obtained by subject 4 with an average value of 89% in the detection of each of
these tasks. Even so, the 3 classes presented have similar maximum performance, were making a fist
has 76%, extending the hand 78%, and not executing any movement 80%.

6
 Task S1 S2 S3 S4 S5 S6 S7 S8 S9 S10
Make a fist 0.75 0.68 0.89 0.87 0.62 0.76 0.89 0.63 0.81 0.67
Extend the hand 0.74 0.66 0.88 0.95 0.63 0.79 0.85 0.73 0.70 0.82
do not move 0.86 0.69 0.76 0.85 0.65 0.83 0.86 0.71 0.91 0.89
Table 5: Average accuracy for classification of movement start for each of the subjects

Figure 6 shows the accuracy curve obtained for subject 4 and the task of reaching out. In this graph,
each value of the x-axis represents the accuracy obtained by the classifier with the features of the last
second, that is, for t=2 seconds the value in the graph (0.72) corresponds to the accuracy obtained
when classifying the window that goes from 1 to 2 seconds. It is easy to notice that in the period
between 3 and 4.2 seconds the classifier has more difficulty determining if the movement is finishing
or paying attention. It should be remembered that after 3 seconds the test subject is presented with
the image that indicates to execute of the movement and depending on the configuration that is being
used in this pseudo-online analysis only up to 4 seconds, the obtained features have no data from the
fixation cross period.

Figure 6: Accuracy curve for Leave-one-out cross-validation for subject 4 and "make a fist"

Table 6 presents the mean accuracy values obtained through the leave-one-out cross-validation
methodology for detecting the start of a movement. It can be seen that subject 4 has the best
performance with 92% and subject 8 has the lowest accuracy. the figure 7 shows the accuracy
curve for subject 8, although it has a classification class rate of 0.53, the maximum classification
value reaches 0.65 for time values of 2.7 and 4.5 seconds and, as in the previous case, the accuracy
decreases when the analyzed data contains information of both classes: the fixation cross and the
motor execution.

S1 S2 S3 S4 S5 S6 S7 S8 S9 S10
Mean Accuracy 0.59 0.54 0.54 0.80 0.49 0.63 0.75 0.53 0.58 0.55
Table 6: Maximum accuracy for classification of movement start for each of the subjects

As expected from the results obtained in the offline strategy, the multiclass classification with the
technique described in this section shows lower precision values with respect to the detection of
movement initiation and the detection of initiation of each of the classes. Table 7 shows the average
precision for each of the subjects evaluated, however these results can be confusing: according to
these values, subject 4 has an average precision of 42%; however, in figure 8 it can be seen that the
precision can become 77% before parsing the window in which there is task execution information
and the fixation cross.

7
 Figure 7: Accuracy curve for Leave-one-out cross-validation for movement onset for subject 8

S1 S2 S3 S4 S5 S6 S7 S8 S9 S10
Mean Accuracy 0.34 0.25 0.31 0.42 0.25 0.33 0.39 0.25 0.33 0.33
Table 7: Mean accuracy for classification of movement start for each of the subjects

Figure 8: Accuracy curve for Leave-one-out cross-validation for movement onset for subject 8

4 Discussion

One of the questions that this research address is whether it is possible to implement a brain-machine
interface based on electroencephalography. The results of this research reveal the first topic of
discussion in this research. It is about the role that electroencephalography would play in the eventual
implementation of a human-machine interface to predict motor execution. As can be seen, the
start of movement has a good performance with LDA classifiers and PSD features. however, when
determining what type of movement, the subject wants to make, it is necessary to add other types of
technologies to implement a hybrid interface that improves system performance.
The other topic that needs to be discussed is the type of task that is being classified when detecting the
start of each one evaluated in this work. The results raise doubts as to which is the participation of the
motor task and which of the attention task when the intention of the movement is classified through
the methodology proposed. In future works, it will be necessary to implement different protocols to
verify how it affects the attention task to the results obtained.

8
References
Ujwal Chaudhary, Niels Birbaumer, and Ander Ramos-Murguialday. Brain–computer interfaces for communica-
tion and rehabilitation. Nature Reviews Neurology, 12(9):513–525, 2016.
Jonathan R Wolpaw. Brain-computer interfaces (bcis) for communication and control. In Proceedings of the 9th
international ACM SIGACCESS conference on Computers and accessibility, pages 1–2, 2007.
Mikhail A Lebedev and Miguel AL Nicolelis. Brain–machine interfaces: past, present and future. TRENDS in
Neurosciences, 29(9):536–546, 2006.
Reinhold Scherer, GR Muller, Christa Neuper, Bernhard Graimann, and Gert Pfurtscheller. An asynchronously
controlled eeg-based virtual keyboard: improvement of the spelling rate. IEEE Transactions on Biomedical
Engineering, 51(6):979–984, 2004.
N Birbaumer, T Hinterberger, AA Karim, A Kubler, N Neumann, and R Veit. Brain-computer communication
using self-control of slow cortical potentials (scp). In Proceedings of the 2nd International BCI Workshop
and Training Course. Graz, Austria, pages 1–4, 2004.
Maged S Al-Quraishi, Irraivan Elamvazuthi, Siti Asmah Daud, S Parasuraman, and Alberto Borboni. Eeg-based
control for upper and lower limb exoskeletons and prostheses: A systematic review. Sensors, 18(10):3342,
2018.
Seyyed Moosa Hosseini and Vahid Shalchyan. Continuous decoding of hand movement from eeg signals using
phase-based connectivity features. Frontiers in Human Neuroscience, page 408, 2022.
Neha Hooda, Ratan Das, and Neelesh Kumar. Fusion of eeg and emg signals for classification of unilateral foot
movements. Biomedical Signal Processing and Control, 60:101990, 2020.
Xinyi Yong and Carlo Menon. Eeg classification of different imaginary movements within the same limb. PloS
one, 10(4):e0121896, 2015.
Farid Shiman, Eduardo López-Larraz, Andrea Sarasola-Sanz, Nerea Irastorza-Landa, Martin Spüler, Niels
Birbaumer, and Ander Ramos-Murguialday. Classification of different reaching movements from the same
limb using eeg. Journal of neural engineering, 14(4):046018, 2017.
Bernhard Graimann and Gert Pfurtscheller. Quantification and visualization of event-related changes in oscilla-
tory brain activity in the time–frequency domain. Progress in brain research, 159:79–97, 2006.
Luis Guillermo Hernández-Rojas, Omar Mendoza Montoya, and Javier M Antelis. Anticipatory detection of
self-paced rehabilitative movements in the same upper limb from eeg signals. IEEE Access, 8:119728–119743,
2020.
John Dempster. Chapter six - signal analysis and measurement. In John Dempster, editor, The Laboratory Com-
puter, Biological Techniques Series, pages 136–171. Academic Press, London, 2001. doi: https://doi.org/10.
1016/B978-012209551-1/50039-8. URL https://www.sciencedirect.com/science/article/pii/
B9780122095511500398.
Chakaveh Ahmadizadeh, Mahta Khoshnam, and Carlo Menon. Human machine interfaces in upper-limb
prosthesis control: a survey of techniques for preprocessing and processing of biosignals. IEEE Signal
Processing Magazine, 38(4):12–22, 2021.

Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper’s
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes] See the 2 section
(c) Did you discuss any potential negative societal impacts of your work? [No]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to
them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions of all theoretical results? [N/A]
(b) Did you include complete proofs of all theoretical results? [N/A]

9
3. If you ran experiments...
(a) Did you include the code, data, and instructions needed to reproduce the main ex-
perimental results (either in the supplemental material or as a URL)? [No] Just the
instructions to reproduce the experiments and results.
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they
were chosen)? [Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experi-
ments multiple times)? [N/A]
(d) Did you include the total amount of compute and the type of resources used (e.g., type
of GPUs, internal cluster, or cloud provider)? [N/A]
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [N/A]
(b) Did you mention the license of the assets? [N/A]
(c) Did you include any new assets either in the supplemental material or as a URL? [N/A]

(d) Did you discuss whether and how consent was obtained from people whose data you’re
using/curating? [No] The test subjects gave their informed consent, but it is not stated
in the document.
(e) Did you discuss whether the data you are using/curating contains personally identifiable
information or offensive content? [No] However, the information of the subjects is
only referring to biomedical data
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, if
applicable? [Yes] see the section 2
(b) Did you describe any potential participant risks, with links to Institutional Review
Board (IRB) approvals, if applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount
spent on participant compensation? [No] the test subjects participated voluntarily
without receiving any payment other than contributing to Mexican science.

10