IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 70, NO.

8, AUGUST 2023 2787

Explainable Deep Learning-Based Approach for

Multilabel Classification of Electrocardiogram
Ganeshkumar M. , Vinayakumar Ravi , Sowmya V, Gopalakrishnan E.A, and Soman K.P

Abstract—Recently computer-aided diagnosis methods have of diseases that affects the heart and its blood vessels. It is
been widely adopted to aid doctors in disease diagnosis making usually caused due to the build-up of fatty deposits inside the
their decisions more reliable and error-free. Electrocardiogram arteries [1]. The number of deaths globally due to CVD has
(ECG) is the most commonly used, noninvasive diagnostic tool for
investigating various cardiovascular diseases. In real life, patients been increased from 12.3 million (25.8%) in 1990 to 17.9
suffer from more than one heart disease at a time. So any practical million deaths (32.1%), in 2015 [2], [3]. So, an automated heart
automated heart disease diagnosis system should identify multiple disease diagnosis method is needed to aid doctors in an accurate
heart diseases present in a single ECG signal. In this article, we diagnosis of various CVDs. Electrocardiogram (ECG) is one of
propose a novel deep learning-based method for the multilabel the most commonly used diagnostic tools for the identification
classification of ECG signals. The proposed method can accurately
identify up to two labels of an ECG signal pertaining to eight of various cardiovascular diseases. Many methods have been
rhythm or morphological abnormalities of the heart and also the proposed in the literature for automated heart disease diagnosis
normal heart condition. Also, the black-box nature of deep learning using ECG signals [4]–[6]. However, patients suffer from more
models prevents them from being applied to high-risk decisions like than one heart disease at the same time. So, for automated heart
the automated heart disease diagnosis. So in this article, we also disease diagnostic methods to be practical, it has to identify
establish an explainable artificial intelligence (XAI) framework for
ECG classification using class activation maps obtained from the multiple heart diseases present in a single ECG signal.
Grad-CAM technique. In the proposed method, we train a convolu- In this article, we propose a fully automated method for
tional neural network (CNN) with constructed ECG matrices. With the multilabel classification of ECG into eight cardiovascular
the experiments conducted, we establish that training the CNN by diseases: 1) Atrial fibrillation (AF), 2) First-degree atrioven-
taking only one label for each ECG signal data point is enough for tricular block (I-AVB), 3) left bundle brunch block (LBBB),
the network to learn the features of an ECG point with multilabel
information in it (multiple heart diseases at the same time). During 4) right bundle brunch block (RBBB), (5) premature atrial
classification, we apply thresholding on the output probabilities of contraction (PAC), 6) premature ventricular contraction (PVC),
the softmax layer of our CNN, to obtain the multilabel classification 7) ST-segment depression, 8) ST-segment elevation and the
of ECG signals.We trained the model with 6311 ECG records and normal heart condition. In the proposed method we train a CNN
tested the model with 280 ECG records. During testing, the model using ECG matrices. The ECG matrix is constructed by taking
achieved a subset accuracy of 96.2% and a hamming loss of 0.037
and a precision of 0.986 and a recall of 0.949 and an F1-score of beats from different leads of the patient’s ECG signal in each
0.967. Considering the fact that the model has performed very row of the matrix. During classification, we show that for a
well in all the metrics of multilabel classification, the model can multilabel test point, the output probabilities from the softmax
be directly used as a practical tool for automated heart disease layer corresponding to correct labels will be of the same order
diagnosis. of magnitude. Further, by applying some simple thresholding
Index Terms—Deep learning, electrocardiogram (ECG), on the output probabilities of the softmax layer of our CNN,
explainability, explainable AI, multilabel classification. we classify multilabel ECG recording with up to two labels
accurately.
I. INTRODUCTION Also, the problematic black-box nature of deep learning mod-
ARDIOVASCULAR disease (CVD), one of the major els poses a requirement for an explainable artificial intelligence
C causes of premature death throughout the world is a class (XAI). So that users and domain experts can analyze the various
features learned by the neural network for its classification task.
Manuscript received 21 May 2021; revised 30 July 2021; accepted 10 August In the current article, we also establish an XAI framework
2021. Date of publication 14 September 2021; date of current version 16 June
2023. Review of this manuscript was arranged by Department Editor N. Gerdsri. for the ECG classification task using class activation maps.
(Corresponding author: Ganeshkumar M.) Thereby, making sure that the neural network has learned the
Ganeshkumar M., Sowmya V, Gopalakrishnan E.A, and Soman K.P are with right features of different diseases considered and not some local
the Center for Computational Engineering and Networking (CEN), Amrita
School of Engineering, Amrita Vishwa Vidyapeetham, Coimbatore 601103, noises present in the dataset. The neural network erroneously
India (e-mail: [email protected]; [email protected]; learning the local noises present in the dataset will lead to
[email protected]; [email protected]). catastrophic misidentification of heart diseases, when tested with
Vinayakumar Ravi is with the Center for Artificial Intelligence, Prince
Mohammad Bin Fahd University, 34754 Khobar, Saudi Arabia (e-mail: ECG signals other than the ones present in the dataset used. So,
[email protected]). the proposed XAI framework validates that the neural network
Color versions of one or more figures in this article are available at has learned the right features and makes the predictions from it
https://doi.org/10.1109/TEM.2021.3104751.
Digital Object Identifier 10.1109/TEM.2021.3104751 highly confident. The contributions of this article are as follows.

0018-9391 © 2021 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See https://www.ieee.org/publications/rights/index.html for more information.

Authorized licensed use limited to: Bar Ilan University. Downloaded on March 12,2024 at 10:40:49 UTC from IEEE Xplore. Restrictions apply.
2788 IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 70, NO. 8, AUGUST 2023

The model was trained on 24 106 samples and tested on 8036

samples. The model achieved a micro average F1-score of 0.863.
Zhu et al. [9] constructed a CNN that outputs 21 sigmoid prob-
ability distributions leading to the multilabel classification of
ECG signals into 21 rhythm and conduction abnormalities. The
authors used 12-lead ECG data and constructed 5000 × 12 di-
mension matrices by cropping all the ECG signals of 12 leads to a
length of 5000 ms. These ECG matrices were further used to train
their CNN. The model was trained on 180 112 ECGs and tested
Fig. 1. Different segments of an ECG signal. on 828 ECGs. The model achieved a mean F1 score of 0.887.
Cheng et al. [10] proposed a novel neural network architecture
for the multilabel classification of arrhythmias from compressed
1) Proposing a novel deep learning-based framework for the
ECG signals. Their neural network outputs a binary vector of
multilabel classification of ECG signals.
length 4, each element in the vector represents the existence
2) Establishing that training the CNN by taking only one
of one of the four types of arrhythmias. They used the sum
label for each ECG data point is enough for the neural
of binary cross-entropy loss over these four different labels
network to capture features of multilabel points.
as their final loss to optimize the neural network. The authors
3) Providing an XAI framework for ECG classification, using
used the MIT-BIH arrhythmia database which contains 48 ECG
the class activation maps obtained from the gradient-
recordings each spanning 30 min. Each of these ECG recordings
weighted class activation mapping (Grad-CAM) tech-
was divided into segments of 2 s and 80% of them were used for
nique, by establishing that the right segments of input ECG
training and 20% of them were used for testing. When tested with
are getting activated while classifying it to its correspond-
original ECGs (without any compression) the model achieved a
ing heart disease.
hamming loss of 0.0031 and a macro F1-score of 0.9848.
Fig. 1 shows the different segments of an ECG signal, which
Jia et al. [11] proposed a novel ensemble neural network
can be used as a reference throughout this article. The rest of
architecture for multilabel classification of ECG signals into the
this article is organized as follows. Section II describes the
following heart diseases: Atrial fibrillation, first-degree atrioven-
existing methods available in the literature for the multilabel
tricular block, complete right bundle branch block, left anterior
classification of ECG signals, Section III the proposed method,
fascicular block, premature ventricular contraction, premature
Section IV experiments and findings, Section V evaluation
atrial contraction, early repolarization, and T wave change and
metrics, Section VI results obtained, Section VII discussion
normal heart condition. The ensemble architecture used had two
on the performance of the method, and finally, Section VIII
modules, the first module uses a sequence generation model [12],
concludes this article.
which outputs a subset sequence of label space leading to the
multilabel classification of ECG. The second module identifies
II. LITERATURE REVIEW each label individually, by having an individual classifier for
Sun et al. [7] proposed an ensemble classifier for the multil- each label. The individual classifiers take extracted features from
abel classification of ECG signals [7]. The model was capable of a base network as input and identify a particular label. The final
multilabel classification of ECG signals into the following heart multilabel classification results are obtained by applying the
diseases: Sinus arrhythmia, sinus bradycardia, atrial premature voting ensemble strategy to predictions from the two modules.
beats, atrial fibrillation, atrioventricular heart-block, complete The model was trained and tested on 6500 ECG samples. The
right bundle branch block, and left ventricular high voltage. model achieved a Macro-F1 score of 0.872.
ECG signals were first preprocessed to remove any noise and Table I summarizes all the above discussed existing meth-
baseline wandering, then 169 features were extracted from the ods for the multilabel classification of ECG signals. From
ECG signals using the WFDB toolbox. Finally, an ensemble Table I we can see that none of the methods available in the
model that combines several multilabel classification algorithms literature are capable of identifying ST-segment elevation and
is used for the multilabel classification of the ECG signals. 46729 ST-segment depression in the multilabel classification of ECG
ECG records were used in this study, with 60% of samples used signals. ST-segment elevation and depression are characteristic
for training and 40% samples used for validating the model. features of many important heart diseases, mainly myocardial
The model achieved an accuracy of 75.2% and a hamming loss infarction [13]. Our proposed method overcomes this limitation
of 0.062 and a precision of 0.808 and a recall of 0.716 and an by including the identification of ST-segment elevation and
F1-score of 0.752. ST-segment depression in the multilabel classification of ECG
Cai et al. [8] proposed a novel neural network architecture signals.
called the multi-ECGNet for multilabel classification of ECG
signals. Multi-ECGNet integrates the advantages of various neu-
ral network architectures like one-dimensional convolution, skip A. XAI Methods Based on Grad-CAM
connections, depth-wise separable convolution, and squeeze- This section describes some of the works in the litera-
and-excitation (SE) module. The model was capable of mul- ture, pertaining to different applications, where the Grad-
tilabel classification on 55 different categories of arrhythmias. CAM technique was adopted to create an explainable model.

Authorized licensed use limited to: Bar Ilan University. Downloaded on March 12,2024 at 10:40:49 UTC from IEEE Xplore. Restrictions apply.
M. et al.: EXPLAINABLE DEEP LEARNING-BASED APPROACH FOR MULTILABEL CLASSIFICATION OF ECG 2789

TABLE I
REVIEW OF EXISTING METHODS FOR THE MULTILABEL CLASSIFICATION OF ECG SIGNALS

Shui Hua Wang et al. [14] proposed a novel neural network to learn the relative features among the batch of CT slices.
architecture called graph rank-based pooling neural network The RAPNN is a VGG-16 architecture that utilizes a novel
(GRAPNN) for the diagnosis of pulmonary tuberculosis using rank-based pooling layer instead of the traditional max-pooling
CT images. layer. The Grad-CAM technique is utilized on the GRAPNN to
The authors first extracted image-level features from the CT make sure that it is paying attention to the areas of lesions (CT
slices using rank-based pooling neural network (RAPNN) and manifestations of pulmonary tuberculosis), thereby creating an
further a graph convolutional neural network (GCN) is adopted XAI framework.

Authorized licensed use limited to: Bar Ilan University. Downloaded on March 12,2024 at 10:40:49 UTC from IEEE Xplore. Restrictions apply.
2790 IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 70, NO. 8, AUGUST 2023

Fig. 2. Flow diagram of our proposed method.

The authors in [15] derived a CNN architecture from VGG-16 classification of ECG. Also, we utilize our trained CNN to
model and added convolutional block attention modules with it establish the XAI framework for the ECG classification task.
and used it to detect COVID-19, pneumonia, and tuberculosis
from CT images. The authors used the same Grad-CAM tech- A. Preprocessing
nique to make sure that the features learned by the CNN are
The preprocessing stage aims to remove the common noises
appropriate manifestations of those diseases in the CT images.
which get added to ECG signals while recording them. We
Authors in [16] proposed a novel patch shuffle stochastic
adopted the same preprocessing steps followed in the arti-
pooling neural network (PSSPNN) to detect COVID-19, pneu-
cle [18], in which authors identified arrhythmias from ECG
monia, and tuberculosis from CT images. Stochastic pooling
signals using CNNs. The steps followed are: 1) Baseline-wander
neural network (SPNN) is a traditional neural network in which
removal and 2) powerline interference removal. The presence of
the max and average pooling layers are replaced by a random-
these noises (baseline-wander and powerline interference) is un-
ized stochastic pooling layer. In SPNN, patch shuffling was
desirable as sometimes the CNN might confuse them as a feature
introduced, in which each minibatch of images and feature
to be learned. However, these noises are completely irrelevant
maps are partitioned into nonoverlapping patches and they are
for the identification of any heart diseases and their removal
shuffled randomly to create local variations which could prevent
helps CNN to learn the appropriate distinguishing features of
overfitting. The Grad-CAM technique was utilized to create an
various heart diseases.
XAI model.
1) Baseline Wander Removal: Baseline wander is the drift
Shui Hua Wang et al. [17] proposed a novel transfer learning
of ECG recordings from its isoelectric level (no positive or
algorithm for pretrained deep learning models and used it to de-
negative charges of electricity). Baseline wander is usually a
tect COVID-19, pneumonia, and tuberculosis from CT images.
low-frequency noise [19]. During the preprocessing stage, a
The authors also used heat maps generated from the Grad-CAM
Butterworth high-pass filter with a cutoff frequency of 0.5 Hz is
technique to create an explainable model.
used to remove baseline wander in ECG signals. Also to get a
zero-phase shift, the filter is applied in both forward and back-
III. PROPOSED METHOD ward directions. The Butterworth high-pass filter is designed
using the SciPy package.1 Fig. 3 shows a sample ECG signal
The flow diagram of the proposed method is shown in Fig. 2, it
from our dataset before and after baseline wander removal.
broadly consists of four steps: 1) Preprocessing the ECG signals,
2) ECG matrix formation, 3) training the CNN, and 4) applying
thresholding on softmax probabilities leading to the multilabel 1 [Online]. Available: https://www.scipy.org/

Authorized licensed use limited to: Bar Ilan University. Downloaded on March 12,2024 at 10:40:49 UTC from IEEE Xplore. Restrictions apply.
M. et al.: EXPLAINABLE DEEP LEARNING-BASED APPROACH FOR MULTILABEL CLASSIFICATION OF ECG 2791

Fig. 3. Sample ECG signal before and after Baseline wander removal.

Fig. 4. Sample ECG signal before and after power line interference removal.

2) Power Line Interference Removal: Power line interfer-

ence is a 50-Hz noise that gets added to the ECG signal due
to Electromagnetic interference from the power line [20]. A
Butterworth low-pass filter with a cutoff frequency of 50 Hz is
used to remove the power line interference in our ECG signals.
Fig. 4 shows a sample ECG signal from our dataset before and
after power line interference removal.

B. ECG Matrix Formation

After preprocessing the patient’s ECG recordings, we con-
structed ECG matrices by beat segmenting the various leads. Our
dataset consists of 12 lead ECG recordings. Sujadevi et al. [21]
Fig. 5. Visualization of the constructed ECG matrix.
identified that the ECG signals from the leads: aVF, V5, and
V6 are enough for accurate identification of the heart diseases
using deep learning algorithms in the same dataset we used. This
finding greatly reduced the size of data that has to be processed. 1) Beat Segmentation and Period Normalization: As shown
Also, gave us the possibility to conduct a lot of experiments in Fig. 1, an ECG recording of a human heart beat starts with a
in a short time. Out of the 12 leads in the dataset, the ECG P wave, forms a QRS complex in the middle, and ends with a T
signals from only three leads (aVF, V5, and V6) were taken and wave. For beat segmenting ECG signals, the R-peaks in all three
beat segmented. After beat segmentation, 10 beats from each of the leads are detected. The time-span of the PR interval (the
of the three chosen leads were extracted and period normalized time from the start of the P wave to the start of the QRS complex,
to 400 ms (milliseconds). Further, ECG matrices of dimension as shown in Fig. 1) in any normal ECG recording is between
30 × 400 were constructed taking all 30 beats (10 from each of 120 to 200 ms [22]. This duration of PR interval can be utilized
the three leads) in each row of the matrix. Fig. 5 visualizes our to beat segment the ECG signals as it can point out the starting
constructed ECG matrix (Leads × Time). The steps followed for point of any beat relative to its R peak. We experimentally found
beat segmentation and period normalization are described below. that setting up the PR interval to 170 ms worked well for beat

Authorized licensed use limited to: Bar Ilan University. Downloaded on March 12,2024 at 10:40:49 UTC from IEEE Xplore. Restrictions apply.
2792 IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 70, NO. 8, AUGUST 2023

Fig. 6. Architecture of the CNN used for the multilabel classification of ECG signals.

segmenting our dataset. The following procedure is adopted for from the VGG-16 gave a better performance while evaluating
beat segmenting the ECG signals: After detecting the R-peaks, with the test set. Also, the number of neurons in the fully
we considered a beat to be the signal spanning between connected layers was reduced compared to that of VGG-16 and
170 ms left from the current R-peak to 170 ms left from the next dropouts with a rate of 0.5 were introduced in between fully
R-peak. That is the signal which starts from the PR interval of the connected layers to avoid overfitting.
current beat and spans till the beginning of the PR interval of the All the convolutional layers of the CNN had 3×3 Kernels. The
next beat. number of filters in each convolution layer is also mentioned in
For R-peak detection, we used multiple methods available Fig. 6 (e.g., 64). Pool/2 in Fig. 6 indicates a max-pooling opera-
in the literature: Pan and Tompkins method [23], stationary tion with 2×2 stride. The fully connected layers are denoted by
wavelet transform method [24], Christov method [25], Hamilton “fc” along with the number of neurons in those layers. We tried
method [26], Engelse and Zeelenberg method [27], and two tuning various hyperparameters of the CNN like the number of
moving average method [28]. The accuracy of R-peaks detection layers, filter size, and the number of filters in each convolutional
varied in each of these methods. During our experiments, we layer. However, there were no improvements in the performance
found that a particular R-peak detection method worked well when deviated from the hyperparameter configurations of the
for a particular ECG data point and other methods gave some standard VGG-16 architecture. We trained our CNN using an
false positives or false negatives. So for R-peak detection, all Adam optimizer with a learning rate of 0.0001 and a batch size
abovementioned methods were applied one by one, until an of 32.
appropriate number of R-peak is detected with respect to the The CNN was trained by taking only one label for each of
length of that particular ECG signal. For period normalizing the ECG recordings. While experimenting with the test ECG
the ECG beats, we adopted a similar procedure as that of the data points, we observed that CNN automatically captured the
one used in the papers [29], [30], where authors processed ECG features of multilabel ECG points, this is further validated in the
signals by constructing ECG matrices and tensors, respectively. experiments section.
The R-peaks of all the beats were aligned at 200 ms and zeros
are padded appropriately at the start and end to period normalize D. Thresholding on Softmax Layer Probabilities and
the beats to 400 ms. This alignment of R-peaks at 200 ms Multilabel Classification of ECG
is an important normalization step. CNNs are discriminative
After training the proposed CNN with constructed ECG ma-
models, they learn the features by discriminating the data points
trices, during testing, we applied thresholding on the softmax
belonging to different classes. The misalignment in R-peaks can
layer probabilities to pick the right labels for our test ECG
therefore act as noise, as CNN may confuse it as characteristic
points. Leading to the multilabel classification of ECG signals.
features of some heart disease, which makes it difficult for the
This thresholding technique was formulated with the help of
CNN to learn the actual features of the diseases.
experiments we conducted and it is described in the Experiments
section with supporting results.
C. Training the CNN
After constructing the ECG matrices for all the ECG record- E. XAI Using Class Activation Maps
ings, the CNN is trained with those matrices. Since the con- The XAI framework for the ECG classification task is estab-
structed ECG matrices are 2-D, we were able to process them lished using class activation maps obtained from a technique
with traditional 2D CNNs. Fig. 6 describes the CNN architecture called Grad-CAM [31]. Grad-CAM generates a map of weights
used. Our CNN is adapted from the standard VGG-16 architec- indicating the important regions in the input used by the CNN for
ture. One convolution + max-pooling block was removed from predicting its class label. Grad-CAM uses the values of gradients
the standard VGG 16 architecture to make it compatible with the flowing into the final convolutional layer to produce such class
dimensions of our input and to reduce the number of trainable activation maps. The detailed working of Grad-CAM can be
parameters. Thereby making the training of CNN faster. We found in the article [31]. We picked sample ECG matrices that
experimentally found that removing the 2nd convolutional block are being correctly classified by our CNN and obtained their

Authorized licensed use limited to: Bar Ilan University. Downloaded on March 12,2024 at 10:40:49 UTC from IEEE Xplore. Restrictions apply.
M. et al.: EXPLAINABLE DEEP LEARNING-BASED APPROACH FOR MULTILABEL CLASSIFICATION OF ECG 2793

Fig. 7. Computing the mean of activations across the beat space of the ECG matrix.

class activation maps using Grad-CAM. After obtaining the class TABLE II
CLASS DISTRIBUTION OF OUR TEST SET
activations maps for the considered ECG matrices (Leads ×
Time), the mean of activations across the leads is computed, to
obtain the activation along the beat space of those ECG matrices.
Fig. 7 describes this process. The obtained average activation
along the beat space indicates the important segments of the
ECG signal used by the CNN to predict that particular heart
disease successfully.
Further, the obtained average activation along the segments
of the ECG beat is analyzed and a correlation is established
with the ECG manifestations of that particular heart disease.
Thereby making sure that the CNN has learned the right features
for classifying ECG signals according to the heart diseases they
belong to. Thus establishing an XAI framework for the ECG
classification task. was not able to capture the features of ECG recordings with
three labels. The CNN was trained with 6311 ECG recordings.
We used the validation set given in our dataset for testing our
IV. EXPERIMENTS
model. Table II gives the number of ECG recordings in different
The CNN and Grad-CAM models were implemented using classes of our test set, multilabel points are considered in all the
TensorFlow2 and Keras3 package and we ran our experiments classes they belong to.
in a 12 GB NVIDIA Tesla K80 GPU.
B. Softmax Probability Thresholding for Multilabel
A. Dataset Used Classification of ECG Signals
The dataset we used was taken from “The China Physio- After training the proposed CNN by taking one label for
logical Signal Challenge 2018: Automatic identification of the each ECG matrix, while testing we observed that CNN was
rhythm/morphology abnormalities in 12-lead ECGs.” A detailed automatically able to capture the features of the multilabel ECG
description of the dataset is published in an article by Feifei Liu points. CNNs capture the features of a particular class with some
et al. [32]. The dataset contains 12 leads ECG recordings lasting of its neurons. The respective neurons get activated whenever
from 6 to 60 s. ECG recordings were sampled at 500 Hz. We had the CNN detects those features in input, which in turn increases
to drop some recordings from the training set and the validation the softmax probability of that particular class. Whenever the
set of our dataset, as we could not extract enough number of beats CNN detects the features of multiple classes in the input, the
from them, due to their shorter length. Our dataset also had six respective neurons which capture the features of those classes
three-label points, which we removed, as our proposed method get activated, leading to increased softmax probabilities of all
those classes. During our experiments we found out that for
2 [Online]. Available: https://www.tensorflow.org/ a multilabel ECG data point, the output probabilities obtained
3 [Online]. Available: https://keras.io/ from the softmax layer corresponding to its right class labels

Authorized licensed use limited to: Bar Ilan University. Downloaded on March 12,2024 at 10:40:49 UTC from IEEE Xplore. Restrictions apply.
2794 IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 70, NO. 8, AUGUST 2023

Algorithm 1: Multilabel Classification of ECG.

Input:
X ECG matrices of the training set
X̂ ECG matrices of the test set
Output:
C Multiple class labels
for each ECG matrix x in X do
train CNN(x)
end for
for each Test ECG matrix x̂ in X̂ do
SoftmaxProbabilities ← CNN(x̂)
Top1Probability = max(SoftmaxProbabilities)
Top1Index = argmax(SoftmaxProbabilities)
SoftmaxProbabilities = deletemax
(SoftmaxProbabilities)
Top2Probability = max(SoftmaxProbabilities)
Top2Index = argmax(SoftmaxProbabilities)
if Top1Probability and Top2Probability in same order
Fig. 8. Output probabilities from the softmax layer for two sample multilabel of magnitude then
test points. C ← Top1Index,Top2Index
else
C ← Top1Index
are of the same order of magnitude. Fig. 8 describes the same, end if
showing the output probabilities of two sample multilabel test end for
points, with a red ellipse indicating the probabilities of their
correct labels.
Also, we found that single label ECG test points do not B. Subset Accuracy
produce same order of magnitude probabilities. So the following Subset Accuracy is a measure of exactly correct classifications
thresholding is applied for picking the class labels of any test given by the model, i.e.,
point: Top-2 probabilities from the softmax layer are taken and
1

if they are of the same order of magnitude, both of the labels are Subset Accuracy (h) = |X| [h(x) = y]. (2)
assigned to the test point. Otherwise, only the class label which x∈X
has got the largest probability is assigned. Thus leading to a
multilabel classification of ECG signals up to two labels. Pseudo C. Precision
code of our proposed method for multilabel ECG classification Precision is a measure of, what fraction of positive identifica-
is described as Algorithm-1. tions were actually correct, i.e.,

V. EVALUATION MEASURES FOR MULTILABEL CLASSIFICATION 

l
tpj
j=1
This section describes the multilabel classification evaluation precision (h) = 
l
(3)
(tpj +fpj )
metrics we calculated while testing our model. Assuming X to j=1

be our test dataset and l to be the maximum number of labels

any test point can belong to, the metrics are defined as follows. where tp denotes the number of true-positive identifications and
fp denotes the number of false-positive identifications.
A. Hamming Loss
D. Recall
Hamming loss is calculated individually for each test point,
it is the fraction of wrongly classified labels to the total number Recall is a measure of what fraction of actual positives were
of labels. The final Hamming loss is obtained by computing the identified correctly, i.e.,
mean of all the Hamming losses obtained for each test point

l
tpj
1
 1

l
recall (h) =
j=1
(4)
HammingLoss(h) = [(Lj ∈ h(x)) ⊗ (Lj ∈ y)] 
l
|X|
x∈X
l
j=1
(tpj +fnj )
j=1
(1)
where ⊗ is logical X-OR operation, h(x) is the output obtained where tp denotes the number of true-positive identifications
from the model, and y is the original class labels the test point by the model and fn denotes the number of false-negative
belongs to. Lj denotes any jth label. identifications by the model.

Authorized licensed use limited to: Bar Ilan University. Downloaded on March 12,2024 at 10:40:49 UTC from IEEE Xplore. Restrictions apply.
M. et al.: EXPLAINABLE DEEP LEARNING-BASED APPROACH FOR MULTILABEL CLASSIFICATION OF ECG 2795

Fig. 9. Loss and accuracy curve of the CNN during training.

TABLE III
PERFORMANCE OF THE PROPOSED METHOD IN VARIOUS EVALUATION METRICS
FOR THE MULTIABEL ECG CLASSIFICATION TASK

E. F1-Score
F1-score is the harmonic mean between precision and recall
Fig. 10. Average activations across the beat space of an ECG sample with AF.
2(precision (h)∗ recall(h))
F1 = precision (h)+recall(h) . (5)

VI. RESULTS performed extremely well in terms of precision (0.986). A good

precision means fewer false positives.
This section describes the results we obtained for the multil- The proposed method also achieved a good recall of 0.949. A
abel ECG classification task and for the XAI framework. Also, good recall means fewer false negatives. F1-score gives the over-
provides a visualization of our test set. all performance of the model and the proposed method achieved
a very good F1- score of 0.967. Hamming loss penalizes the
A. Results of Multilabel ECG Classification miss-classification of individual labels for each test point. The
proposed method achieved an average hamming loss of 0.037.
Multilabel classification of ECG signals was done by applying
the following thresholding on the softmax layer of CNN: Top-2
B. XAI Framework
probabilities from the softmax layer are taken and if they are
of the same order of magnitude, both of the corresponding This section shows the average activations across the beat
labels are assigned to the test point. Otherwise, only the class space, obtained for a sample ECG signal, belonging to each of
label which has got the largest probability is assigned. Table III the diseases considered in our study. We analyze the activations
shows the scores obtained by our proposed method in evaluation obtained for each of the diseases and establish a correlation
metrics: Subset accuracy, hamming loss, precision, recall, and with their respective characteristic variations in ECG. Thereby,
F1- score. The values of these metrics are calculated using the making sure that the model has learned the right features for its
equations described in Section V. classification task and not some undesirable tricks.
Fig. 9 shows the loss and accuracy curve obtained on both 1) Atrial Fibrillation (AF): Frequent elevation of ventricular
training and validation set during the training of CNN. Subset rate (heart rate) is the characteristic nature of AF [33]. Due to
accuracy is the strictest metric, it indicates the fraction of test which the duration of a heartbeat is shortened. Fig. 10 shows
points that have all their labels identified correctly. Our proposed the average activations across the beat space of an ECG sample
method performed extremely well in terms of subset accuracy with AF. The shortened beats in an AF ECG sample leads to the
by achieving a value of 0.962. Also, the proposed method padding of zeros at the beat’s end to normalize it to 400 ms. This

Authorized licensed use limited to: Bar Ilan University. Downloaded on March 12,2024 at 10:40:49 UTC from IEEE Xplore. Restrictions apply.
2796 IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 70, NO. 8, AUGUST 2023

Fig. 11. Average activations across the beat space of an ECG sample with
I-AVB.

Fig. 13. Average activations across the beat space of an ECG sample with
PAC.

Fig. 12. Average activations across the beat space of an ECG sample with
LBBB.

is due to the period normalization procedure we adopted. This

zero padding at the end is taken as the discriminative feature by
our CNN to identify ECG points with AF, which is evident from
the activations shown in Fig. 10.
2) First-Degree Atrioventricular Block (I-AVB): Prolonged
PR interval (>300 ms) is the characteristic feature of the disease Fig. 14. Average activations across the beat space of a mormal ECG sample.
I-AVB [34]. Also, the P wave will be buried in the proceeding
T wave in an ECG with I-AVB [34]. Fig. 11 shows the average
activations across the beat space of an ECG sample with I-AVB. in the regions of abnormal P wave, normal QRS complex, and
From Fig. 11, we can see that our CNN is correctly getting normal T wave, in accordance with PAC’s characteristic features.
activated in the prolonged P wave region and in the regions 5) Normal Heart Condition: A normal ECG signal will have
where the P wave is buried in proceeding T wave in accordance a normal P wave, a normal QRS complex, and a normal T
with the characteristic features of the I-AVB. wave [37]. Fig. 14 shows the average activations across the beat
3) Left Bundle Brunch Block (LBBB): The characteristic fea- space of a normal ECG sample. From Fig. 14, we can see that
ture of LBBB is a broad QRS complex (duration >120 ms) [35]. our CNN’s activations are in the appropriate regions of normal
Also, a depressed ST segment and a T wave opposite to that of P wave and normal QRS complex and normal T wave.
the QRS complex will be present in an ECG with LBBB [35]. 6) ST-Segment Depression: A normal P wave, a normal T
Fig. 12 shows the average activations across the beat space of wave, and a depressed ST-segment is the characteristic feature of
an ECG sample with LBBB. From Fig. 12, we can see that an ECG signal with ST-segment depression [38]. Fig. 15 shows
CNN’s activations are relatively high in the region of the broad the average activations across the beat space of an ECG sample
QRS complex and the ST-segment depression and in the region with ST-segment depression. From Fig. 15, we can observe that
where there is an inverted T wave. our CNN is getting precisely activated in the regions of normal
4) Premature Atrial Contraction (PAC): An ECG signal with P wave, depressed ST-segment, and a normal T wave.
PAC typically has a normal QRS complex and a normal T wave, 7) ST-Segment Elevation: An elevated ST-segment is the
but an abnormal P wave in them [36]. Fig. 13 shows the average characteristic feature of ECG signals with ST-segment eleva-
activations across the beat space of an ECG sample with PAC. tion [38]. They also have a normal QRS complex and a normal
From Fig. 13, we can observe that our CNN is rightly activated T wave [38]. Fig. 16 shows the average activations across the

Authorized licensed use limited to: Bar Ilan University. Downloaded on March 12,2024 at 10:40:49 UTC from IEEE Xplore. Restrictions apply.
M. et al.: EXPLAINABLE DEEP LEARNING-BASED APPROACH FOR MULTILABEL CLASSIFICATION OF ECG 2797

Fig. 15. Average activations across the beat space of an ECG sample with Fig. 17. Average activations across the beat space of an ECG sample with
ST-segment depression. RBBB.

Fig. 16. Average activations across the beat space of an ECG sample with Fig. 18. Average activations across the beat space of an ECG sample with
ST-segment elevation. PVC.

beat space of an ECG sample with ST-segment elevation. From see that our CNN is properly getting activated at the broad QRS
Fig. 16, we can observe that our CNN is precisely activated in complex region of the ECG.
the regions of normal QRS complex, elevated ST-segment, and
a normal T wave. With this, we make sure, that our CNN has C. Visualization
learned the right features for classifying an ECG signal with To visualize our test set, we extracted the features from
ST-segment elevation, according to its characteristic features. the last convolution layer of our trained CNN for all the test
8) Right Bundle Branch Block (RBBB): A wide slur S wave points, vectorized and reduced them to a dimension of two
in leads I, V5, and V6 is the characteristic feature of RBBB [35]. using the t-distributed stochastic neighbor embedding (t-SNE)
V5 and V6 are the two out of the three leads our CNN is trained technique [40]. Fig. 19 visualizes our test set in 2-D. The legend
with. Fig. 17 shows the average activations across the beat space in the figure describes the mapping of colors to different diseases
of an ECG sample with RBBB. The signal shown in Fig. 17 is the test points belong to (considering only the first label of each
from the V5 lead. From Fig. 17, we can see that our CNN’s test point). From the figure, we can understand that the data is
average activation across the beat space is relatively high in the highly nonlinearly separable.
wide slur S wave region, in correspondence to the characteristic
features of RBBB.
9) Premature Ventricular Contraction (PVC): An abnor- VII. DISCUSSION AND FUTURE WORK
mally broad QRS complex is the characteristic feature of ECGs Our proposed method achieved a state-of-the-art performance
with PVC [39]. Fig. 18 shows the average activations across the in the multilabel ECG classification task, also providing an
beat space of an ECG sample with PVC. From Fig. 18, we can XAI framework for its classifications. This section provides a

Authorized licensed use limited to: Bar Ilan University. Downloaded on March 12,2024 at 10:40:49 UTC from IEEE Xplore. Restrictions apply.
2798 IEEE TRANSACTIONS ON ENGINEERING MANAGEMENT, VOL. 70, NO. 8, AUGUST 2023

Fig. 19. Visualization of the test set using t-SNE.

TABLE IV the disadvantages of all the other methods by including the iden-
COMPARISON OF THE PERFORMANCE OF THE PROPOSED METHOD WITH
EXISTING METHODS
tification of ST-segment depression and ST-segment elevation.
Thereby, aiding doctors in identifying critical heart diseases like
myocardial Infarction.

B. Limitations of the Proposed Method and Future Works

Our proposed method currently identifies only eight critical
heart diseases belonging to both rhythm and morphological
abnormality categories. In the future, our model can be further
improved to identify wider varieties of heart diseases, by in-
cluding ECG recordings from different datasets in the training.
Thereby, making it a complete, one-stop automated heart disease
diagnosis system. Multilabel ECG data for different subcate-
gories of the considered heart diseases can be collected and
experiments can be conducted to see if our proposed method
comparison of the proposed method with existing methods dis- can identify them without compromising on the performance.
cussed in the literature review and also describes the limitations
of the proposed method and the future works possible. VIII. CONCLUSION
This research article proposes a novel deep learning method
A. Comparison of the Proposed Method for Multilabel for the multilabel classification of ECG signals into eight rhythm
Classification of ECG Signals With Existing Methods
or morphological abnormalities of the heart and also the normal
The bold entry in Table IV indicates the performance of heart condition. Further, this work establishes an XAI framework
the proposed method. Table IV compares the performance of for its ECG classification task. A detailed investigation of the
our proposed method with existing methods discussed in the existing methods available for the multilabel classification of
literature review. All the existing methods compared utilize ECG signals was done. The performance of the existing methods
different datasets other than the one we used. Considering the was compared with the our proposed method. Our proposed
F1-scores given in Table IV, our proposed method outperformed method outperformed most of the existing methods available
most of the existing methods available for the multilabel ECG for multilabel classification of ECG signals and it included the
classification [7]–[9], [11]. The method in [10] outperformed identification of ST-segment elevation and ST-segment depres-
our proposed method by a slight margin (F1-score of 0.98), sion. Our XAI framework made sure that the CNN had learned
however, their method is capable of identifying only four types the right features for its classification task. Thereby, overcoming
of arrhythmias but cannot identify various rhythm and mor- the problematic black-box nature of the CNN and adding utmost
phological abnormalities. Also, the methods in [8] and [10] do confidence to the results given by our model. Thus, our proposed
not include the identification of wide varieties of rhythm and method can be directly deployed to aid doctors in heart disease
morphological abnormalities. Our proposed method overcomes diagnosis.
Authorized licensed use limited to: Bar Ilan University. Downloaded on March 12,2024 at 10:40:49 UTC from IEEE Xplore. Restrictions apply.
M. et al.: EXPLAINABLE DEEP LEARNING-BASED APPROACH FOR MULTILABEL CLASSIFICATION OF ECG 2799

ACKNOWLEDGMENT [19] C. Haritha, M. Ganesan, and E. P. Sumesh, “A survey on modern trends in

ECG noise removal techniques,” in Proc. IEEE Int. Conf. Circuit, Power
The authors would like to sincerely thank all the reviewers and Comput. Technol., 2016, pp. 1–7, doi: 10.1109/ICCPCT.2016.7530192.
editors for their valuable guidance and suggestions to enhance [20] S. Thalkar and D. Upasani, “Various techniques for removal of power
line interference from ECG signal,” Int. J. Sci. Eng. Res., vol. 4, no. 12,
the quality and depth of the article. pp. 12–23, 2013.
[21] V. G. Sujadevi and K. P. Soman, “Towards identifying most important leads
for ECG classification. A data driven approach employing deep learning,”
Procedia Comput. Sci., vol. 171, pp. 602–608, 2020.
[22] E. Z. Soliman, M. Cammarata, and Y. Li, “Explaining the inconsistent asso-
REFERENCES ciations of PR interval with mortality: The role of P-duration contribution
to the length of PR interval,” Heart Rhythm, vol. 11, no. 1, pp. 93–98,
[1] World Health Organization, Prevention of Cardiovascular Disease. Pocket
2014.
Guidelines for Assessment and Management of Cardiovascular Risk.
[23] J. Pan and W. J. Tompkins, “A real-time QRS detection algorithm,” IEEE
Africa Who/Ish Cardiovascular Risk Prediction Charts for the African
Trans. Biomed. Eng., vol. BME-32, no. 3, pp. 230–236, Mar. 1985.
Region. Geneva, Switzerland: World Health Organization, 2007.
[24] V. Kalidas and L. Tamil, “Real-time QRS detector using stationary wavelet
[2] H. Wang et al., “Global, regional, and national life expectancy, all-cause
transform for automated ECG analysis,” in Proc. IEEE 17th Int. Conf.
mortality, and cause-specific mortality for 249 causes of death, 1980–2015:
Bioinf. Bioeng., 2017, pp. 457–461.
A systematic analysis for the Global Burden of Disease Study 2015,”
[25] I. I. Christov, “Real time electrocardiogram QRS detection using combined
Lancet, vol. 388, no. 10053, pp. 1459–1544, 2016.
adaptive threshold,” Biomed. Eng. Online, vol. 3, no. 1, 2004, Art. no. 28.
[3] M. Naghavi, H. Wang, R. Lozano, A. Davis, X. Liang, and M. Zhou,
[26] P. Hamilton, “Open source ECG analysis,” in Proc. IEEE Comput. Car-
“GBD 2013 mortality and causes of death collaborators. Global, regional,
diol., 2002, pp. 101–104.
and national age–sex specific all-cause and cause-specific mortality for
[27] W. A. H. Engelse and C. Zeelenberg, “A single scan algorithm for
240 causes of death, 1990–2013: A systematic analysis for the Global
QRS-detection and feature extraction,” Comput. Cardiol., vol. 6, no. 1979,
Burden of Disease Study 2013,” Lancet, vol. 385, no. 9963, pp. 117–171,
pp. 37–42, 1979.
2015.
[28] M. Elgendi, M. Jonkman, and F. De Boer, “Frequency bands effects on
[4] R. J. Martis, U. R. Acharya, C. M. Lim, K. M. Mandana, A. K. Ray, and C.
QRS detection,” in Proc. 3rd Int. Conf. Bio-Inpsired Syst. Signal Process.,
Chakraborty, “Application of higher order cumulant features for cardiac
2010, pp. 428–431.
health diagnosis using ECG signals,” Int. J. Neural Syst., vol. 23, no. 4,
[29] H. Lee and K. M. Buckley, “ECG data compression using cut and align
2013, Art. no. 1350014.
beats approach and 2-D transforms,” IEEE Trans. Biomed. Eng. IEEE
[5] E. H. Houssein, I. E. Ibrahim, N. Neggaz, M. Hassaballah, and Y. M.
Trans. Biomed. Eng., vol. 46, no. 5, pp. 556–564, May 1999.
Wazery, “An efficient ECG arrhythmia classification method based on
[30] S. Padhy and S. Dandapat, “Third-order tensor based analysis of multilead
Manta ray foraging optimization,” Expert Syst. Appl., vol. 181, 2021,
ECG for classification of myocardial infarction,” Biomed. Signal Process.
Art. no. 115131.
Control, vol. 31, pp. 71–78, 2017.
[6] R. J. Martis, U. R. Acharya, K. M. Mandana, A. K. Ray, and C.
[31] R. R. Selvaraju, M. Cogswell, A. Das, R. Vedantam, D. Parikh, and D.
Chakraborty, “Application of principal component analysis to ECG signals
Batra, “Grad-CAM: Visual explanations from deep networks via gradient-
for automated diagnosis of cardiac health,” Expert Syst. Appl., vol. 39,
based localization,” in Proc. IEEE Int. Conf. Comput. Vis., 2017, pp. 618–
no. 14, pp. 11792–11800, 2012.
626.
[7] Z. Sun, C. Wang, Y. Zhao, and C. Yan, “Multi-label ECG sig-
[32] F. Liu et al., “An open access database for evaluating the algorithms
nal classification based on ensemble classifier,” IEEE Access, vol. 8,
of electrocardiogram rhythm and morphology abnormality detection,” J.
pp. 117986–117996, 2020.
Med. Imag. Health Inform., vol. 8, no. 7, pp. 1368–1373, 2018.
[8] J. Cai, W. Sun, J. Guan, and I. You, “Multi-ECGNet for ECG arrythmia
[33] V. Fuster et al., “ACC/AHA/ESC 2006 guidelines for the management of
multi-label classification,” IEEE Access, vol. 8, pp. 110848–110858, 2020.
patients with atrial fibrillation: A report of the american college of cardiol-
[9] H. Zhu et al., “Automatic multilabel electrocardiogram diagnosis of heart
ogy/American heart association task force on practice guidelines and the
rhythm or conduction abnormalities with deep learning: A cohort study,”
European society of cardiology committee for practice guidelines (writing
Lancet Digit. Health, vol. 2, no. 7, pp. 348–357, 2020.
committee to revise the 2001 guidelines for the management of patients
[10] Y. Cheng, Y. Ye, M. Hou, W. He, and T. Pan, “Multi-label arrhythmia
with atrial fibrillation): Developed in collaboration with the European heart
classification from fixed-length compressed ECG segments in real-time
rhythm association and the heart rhythm society,” Circulation, vol. 114,
wearable ECG monitoring,” in Proc. 42nd Annu. Int. Conf. IEEE Eng.
no. 7, pp. 257–354, 2006.
Med. Biol. Soc. 2020, pp. 580–583.
[34] A. H. Kashou, A. Goyal, T. Nguyen, and L. Chhabra, Atrioventricular
[11] D. Jia et al., “An ensemble neural network for multi-label classification
Block. Treasure Island, FL, USA: StatPearls Publishing, 2019.
of electrocardiogram,” in Proc. Int. Workshop Mach. Learn. Med. Eng.
[35] B. Surawicz, R. Childers, B. J. Deal, and L. S. Gettes, “AHA/ACCF/HRS
Cardiovasc. Healthcare, 2019, pp. 20–27.
recommendations for the standardization and interpretation of the elec-
[12] P. Yang, X. Sun, W. Li, S. Ma, W. Wu, and H. Wang, “SGM: Sequence
trocardiogram: Part III: Intraventricular conduction disturbances a sci-
generation model for multi-label classification,” 2018, arXiv:1806.04822.
entific statement from the American heart association electrocardiog-
[13] K. Thygesen et al., “Fourth universal definition of myocardial infarction
raphy and arrhythmias committee, council on clinical cardiology; the
(2018),” Eur. Heart J., vol. 40, no. 3, pp. 237–269, 2019.
American college of cardiology foundation; and the heart rhythm so-
[14] S.-H. Wang, V. Govindaraj, J. M. Gorriz, X. Zhang, and Y.-D. Zhang,
ciety. Endorsed by the international society for computerized electro-
“Explainable diagnosis of secondary pulmonary tuberculosis by graph
cardiology,” J. Amer. College Cardiol., vol. 53, no. 11, pp. 976–981,
rank-based average pooling neural network,” J. Ambient Intell. Humanized
2009.
Comput., 2021, doi: 10.1007/s12652-021-02998-0.
[36] J. Heaton and S. Yandrapalli, Premature Atrial Contractions (PAC). Trea-
[15] S.-H. Wang, S. Fernandes, Z. Zhu, and Y.-D. Zhang, “AVNC: Attention-
sure Island, FL, USA: StatPearls, 2020.
based VGG-style network for COVID-19 diagnosis by CBAM,” IEEE
[37] B. Surawicz, T.-C. Chou, and T. K. Knilans, Chou’s Electrocardiography
Sensors J., to be published, doi: 10.1109/JSEN.2021.3062442.
in Clinical Practice: Adult and Pediatric. Philadelphia, PA, USA: W. B.
[16] S.-H. Wang, Y. Zhang, X. Cheng, X. Zhang, and Y.-D. Zhang, “PSSPNN:
Saunders, 2001.
PatchShuffle stochastic pooling neural network for an explainable diagno-
[38] A. H. Kashou, H. Basit, and A. Malik, ST Segment. Treasure Island, FL,
sis of COVID-19 with multiple-way data augmentation,” Comput. Math.
USA: StatPearls, 2020.
Methods Med., vol. 2021, 2021, Art. no. 6633755.
[39] E. Burns and R. Buttner, “Premature ventricular complex (PVC),” Ac-
[17] S.-H. Wang, D. R. Nayak, D. S. Guttery, X. Zhang, and Y.-D. Zhang,
cessed: Nov. 7, 2020. [Online]. Available: https://litfl.com/premature-
“COVID-19 classification by CCSHNet with deep fusion using trans-
ventricular-complex-pvc-ecg-library/
fer learning and discriminant correlation analysis,” Inf. Fusion, vol. 68,
[40] L. Van der Maaten and G. Hinton, “Visualizing data using t-SNE,” J. Mach.
pp. 131–148, 2021.
Learn. Res., vol. 9, no. 86, pp. 2579–2605, 2008.
[18] E. H. Houssein, D. S. Abdelminaam, I. E. Ibrahim, M. Hassaballah, and Y.
M. Wazery, “A hybrid heartbeats classification approach based on marine
predators algorithm and convolution neural networks,” IEEE Access, vol. 9,
pp. 86194–86206, 2021.

Authorized licensed use limited to: Bar Ilan University. Downloaded on March 12,2024 at 10:40:49 UTC from IEEE Xplore. Restrictions apply.