IAES International Journal of Artificial Intelligence (IJ-AI)

Vol. 13, No. 1, March 2024, pp. 1090~1103

ISSN: 2252-8938, DOI: 10.11591/ijai.v13.i1.pp1090-1103  1090

Deep learning based biometric authentication using

electrocardiogram and iris

Ashwini Kailas, Geevagondanahalli Narayanappa Keshava Murthy

Department of Electronics and Instrumentation Engineering, Siddaganga Intitute of Technology, Visveswaraya Technological
University, Tumakuru, India

Article Info ABSTRACT

Article history: Authentication systems play an important role in wide range of applications.
The traditional token certificate and password-based authentication systems
Received May 23, 2023 are now replaced by biometric authentication systems. Generally, these
Revised Sep 27, 2023 authentication systems are based on the data obtained from face, iris,
Accepted Nov 9, 2023 electrocardiogram (ECG), fingerprint and palm print. But these types of
models are unimodal authentication, which suffer from accuracy and
reliability issues. In this regard, multimodal biometric authentication
Keywords: systems have gained huge attention to develop the robust authentication
systems. Moreover, the current development in deep learning schemes have
Biometric authentication proliferated to develop more robust architecture to overcome the issues of
Classification tradition machine learning based authentication systems. In this work, we
Deep learning have adopted ECG and iris data and trained the obtained features with the
Electrocardiogram help of hybrid convolutional neural network- long short-term memory
Feature extraction (CNN-LSTM) model. In ECG, R peak detection is considered as an
Iris important aspect for feature extraction and morphological features are
extracted. Similarly, gabor-wavelet, gray level co-occurrence matrix
(GLCM), gray level difference matrix (GLDM) and principal component
analysis (PCA) based feature extraction methods are applied on iris data.
The final feature vector is obtained from MIT-BIH and IIT Delhi Iris dataset
which is trained and tested by using CNN-LSTM. The experimental analysis
shows that the proposed approach achieves average accuracy, precision, and
F1-core as 0.985, 0.962 and 0.975, respectively.
This is an open access article under the CC BY-SA license.

Corresponding Author:
Ashwini Kailas
Department of Electronics and Instrumentation Engineering, Siddaganga Intitute of Technology,
Visveswaraya Technological University
Tumakuru, India
Email: [email protected]

1. INTRODUCTION
There is a tremendous growth in the field of security and privacy preserving techniques by means of
authentication systems. These security systems are widely adopted in various real-time online and offline
applications such as biomedical systems, cloud computing, computer vision. Generally, these authentication
systems are classified as password-based authentication, multifactor authentication, certificate-based
authentication and token-based authentication. However, these systems have their different applications,
advantages and disadvantages. But when considering the human authentication system, then identifying the
liveliness of human plays an important role, Therefore, in this work we focus on biometric authentication
system. The term "biometrics" refers to our biological, behavioural, or physical traits, and it is seen as a
legitimate substitute for passwords, signatures, and other forms of identification. Evidently, biometric

Journal homepage: http://ijai.iaescore.com

Int J Artif Intell ISSN: 2252-8938  1091

systems are designed to automatically recognise or validate certain persons based on their physical,
physiological, or behavioural attributes, such as their iris, face, stride, keystroke dynamics, or other
characteristics. In order to increase security, convenience, and social inclusion as well as to offer prospective
applications in several scientific and industrial domains, biometric technologies are widely being
implemented. The current advancements in machine learning techniques also increased the reliability of these
authentication systems.
The traditional authentication systems were based on the unimodal systems which considered single
biometric system for authentication however these systems suffer from various issues such as noisy data,
inter-class similarity, intra-class variability in different applications. Some of unimodal authentication
researches include face authentication [1], [2], Iris authentication [3], [4], ECG authentication [5], [6] and
fingerprint authentication [7], [8]. However, these systems suffer from accuracy and afore mentioned related
issues. Therefore, need of robust authentication system gained attention due their significant applications. In
order to handle these issues, several researches have been presented which are mainly focused on developing
the multimodal authentication system, for instance, Zhang et al. [9] presented a robust multimodal
authentication system which is based on the voice and face data. Tarannum et al. [10] used a combination of
IRIS, facial and fingerprint data to achieve the increased authentication performance. Currently,
optimization-based methods also play important role in classification tasks by reducing the error in attributes
and finding the best solution for dimension reduction and feature selection. Based on this concept, Sujatha et
al. [11] used iris, finger vein, finger print data along with genetic algorithm-based optimization strategy.
Similarly, the traditional classification methods suffer from several issues such as these methods rely on data
pre-processing, overfitting, under fitting, speed, scalability and accuracy. Therefore, deep learning-based
methods have gained huge attention in machine learning based applications. Hammad and Wang [12]
presented a combination of ECG and fingerprint biometrics to develop the convolution neural network. Zhao
et al. [13] used palm print and dorsal hand vein and introduced deep learning model for authentication.
Alkeem et al. [14] adopted deep learning for ECG based authentication systems.
Electrocardiogram (ECG) verification, in comparison, is difficult to compromise without the user's
knowledge. ECG biometric feature collection necessitates the use of specialized apparatus, such as an
electrocardiograph, which makes this technique challenging to replicate. The ECG technique usually takes 10
seconds or longer to record ECG signals and reach an adequate degree of accuracy for authentication. ECG
signal helps asses the heart's electrical conductivity as well as its cardio vascular alterations. There are two
main types of data that an ECG provides. First, a doctor measures the time intervals on the ECG to ascertain
how long the electrical wave takes to travel through the heartbeat. The amount of time it takes a wave to
move from one area of the heart to another area of the body reveals whether electrical activity is regular,
slow, quick, or irregular. Secondly, by monitoring the amount of electrical activity flowing through the heart
muscle, a cardiologist may determine whether areas of the heart are too big or overworked. The normal ECG
is depicted in Figure 1. According to this figure, we can observe several entities in the given ECG signal such
as different waves such as P, Q, R, S and T wave. Based on this, several attributes can be extracted such as
PR interval, ST interval, QRS interval and Q-T interval.

Figure 1. Normal ECG signal [15]

Deep learning based biometric authentication using electrocardiogram and iris (Ashwini Kailas)
1092  ISSN: 2252-8938

Similarly, iris is a small, annular structure in the eye, regulates the pupil's size and diameter, hence
regulating how much light reaches the retina and Figure 2 shows a sample iris image which shows that
complete eye image is comprised of pupil, iris, eyelid, collarette and sclera. Thus, identification of iris
becomes an important aspect to perform various tasks on iris Therefore, in this work; we focus on developing
a multimodal authentication system by combining ECG and iris image modalities.

Figure 2. Eyelid image [16]

ECG and Iris-based authentication systems have their own set of challenges that must be overcome
for their effective implementation. One of the primary challenges in ECG-based authentication systems is the
need for high-quality ECG signals. Poor quality signals can lead to inaccurate detection of R-peaks, which
can significantly impact the accuracy of the authentication system. Additionally, ECG signals can be affected
by factors such as noise, motion artifacts, and electrode placement, which can further reduce the accuracy of
the system. Iris-based authentication systems also face several challenges. One significant challenge is the
need for high-quality iris images, as poor-quality images can result in inaccurate feature extraction.
Furthermore, factors such as occlusion, dilation, and age-related changes in the iris can affect the accuracy of
the authentication system. Another challenge is the need for proper illumination and focus during image
capture, as variations in lighting and focus can impact the quality of the iris image.
The main aim of this research is to develop a novel and robust multimodal authentication system by
considering ECG and Iris data. In order to achieve this, the proposed model uses feature extraction method
where R peak detection and morphological feature extraction methods are employed on ECG signals. For
iris, Gabor wavelet, gray level co-occurrence matrix (GLCM) and gray level difference matrix (GLDM)
feature extraction methods are employed. Later, the obtained features are combined together to formulate the
feature vector. Later, we trained a deep learning classifier by using convolutional neural network long short-
term memory CNN-LSTM approach.
As discussed before, the multimodal authentication plays important role in various real-time security
applications. Several biometrics combinations have been employed to improve the robustness of
authentication systems where ECG and Iris based authentication systems have been adopted widely. In this
section, we present a brief literature review about these multimodal authentication systems. Regouid et al.
[17] focused on ECG, ear and Iris to develop a multimodal biometric authentication system to overcome the
challenges of traditional unimodal systems. ECG facilitates the liveness information, ear biometrics helps to
obtain the rich and stable information, and iris features helps to ensure the promising reliability and accuracy.
This scheme performs normalization and segmentation as pre-processing step. Later, 1D-LBP, Shifted-1D-
LBP and 1D-MR-LBP features are extracted from ECG signal. Ear and iris images are transformed into 1D
signal. Finally, the K nearest neighbors (KNN) and radial basis function (RBF) classifiers are applied to
classify the users as genuine or imposter.
Jiang et al. [18] focused on incorporating the authentication for body area sensor networks by
combining iris and ECG features. This process follows two-level authentication where first level focus on iris
authentication and later ECG authentication is performed to improve the overall security. El-Rahiem et al.
[19] presented a multimodal authentication system by using ECG and finger vein. The complete process is
divided into three stages as: pre-processing where data normalization and filtering techniques are applied,
feature extraction phase uses deep CNN model for feature extraction from ECG and finger vein. Finally,
different classifiers such as KNN, support vector machine (SVM), artificial neural network (ANN), random
forest (RF) and naïve bayes (NB) are used to classify the obtained features. Moreover, this model uses multi-
canonical correlation analysis (MCCA) to increase the speed of authentication.
Jadhav et al. [20] presented a multimodal biometric authentication system by using the combination
of palm-print, iris, and face. The feature extraction process uses deep learning model for robust feature
extraction from raw input images. Further, modified group search optimization (MGSO) approach is

Int J Artif Intell, Vol. 13, No. 1, March 2024: 1090-1103

Int J Artif Intell ISSN: 2252-8938  1093

employed to obtain the optimized and dimensionality reduced features. The classification phase uses teacher
learning based deep neural networks (TL-DNN) model to reduce the classification error.
Hammad et al. [5] developed a combination of ECG and fingerprint-based authentication system.
CNN model is used for generating the combined feature set and biometric templates are generated from these
features. These features are trained and classified by using Q-Gaussian multi support vector machine (QG-
MSVM) to increase the classification performance. Singh and Tiwari. [21] combined ECG, sclera, and
fingerprint to develop a multimodal biometric system. This combination is carried out in two modules: i)
decision-level fusion where combined Whale Optimization (WOA-ANN) is used to generate the sequential model
and ii) score-level fusion model which uses salp swarm optimization-deep belief network (SSA-DBN) model.
Zhang et al. [9] developed face and voice based multimodal authentication system. This model uses
local binary pattern (LBP) method for face feature extraction and voice activity detection (VAD) for voice
sample analysis to increase the voice detection accuracy. This model also uses a feature fusion strategy to
fuse the face and voice attributes efficiently.
Amritha et al. [22] emphasized on score level and feature level fusion to improve the accuracy of
multimodal authentication system. This model includes ECG, face, and fingerprint for feature extraction and
fusion. The score and feature level fusion are performed separately and later obtained scores are normalized
by employing overlap extrema-based min–max (OVEBAMM) method. Huang et al. [23] reported that the
performance of ECG based authentication system which is affected by the different types of noises and
sample variation. To overcome these issues, authors presented local binary pattern-based feature extraction
along with the robust multi-feature collective non-negative matrix factorization (RMCNMF) approach to
address the noise and sample variation issues. This process helps to learn the latent semantic space with the
help of Convolution non-negative matrix factorization (CNMF) method. Kumar et al. [24] focused on face
and gait biometrics as multimodal authentication system. This approach uses principal component analysis
(PCA) and deep neural network approach. The deep learning model replaced Euclidean distance with cross
entropy function. The PCA is used for feature extraction and reconstruction of faces and DNN helps to
improve the accuracy where final matching score is obtained by applying SoftMax function.

2. METHOD
This section presents the proposed solution for feature extraction and classification for multimodal
authentication system. Figure 3 depicts the complete process of proposed approach The proposed ECG
feature extraction phase includes deep learning-based R Peak detection and further, several morphological
features [25] are obtained based on the R peak. Later, Iris feature extraction stage includes Gabor-wavelet,
GLCM, GLDM and PCA based feature extraction. These features are further processed through the CNN-
LSTM classifier module to authenticate the users.

Figure 3. Complete architecture of proposed authentication system

2.1. ECG feature extraction

This section presents the proposed solution for ECG feature extraction. In ECG signal processing, R
Peak detection plays important role therefore, we present the R peak detection method. Currently, several
methods have been described to detect the R peaks. In this work, we adopted the deep learning method and
present an encoder and decoder-based CNN architecture for peak detection by segmenting the input signal.
Figure 4 depict the architecture of encoder and decoder module. The encoder path is known as contracting
path which contains repeated convolution operations followed by rectified linear unit (ReLU) and max
pooling. This structure is used to perform the down-sampling operations. Similarly, the expanding path
performs up sampling operations followed by convolutions and ReLU operations.
Deep learning based biometric authentication using electrocardiogram and iris (Ashwini Kailas)
1094  ISSN: 2252-8938

Figure 4. Encoder/decoder modules in CNNs

In this work, we formulated the peak detection problem as 1D segmentation problem for R peak
segmentation by using deep learning approach. The input dataset contains 𝑖 input ECG signals which are
normalized between +1 and -1. The output of this module is obtained as 1D segmentation map which
contains the R peak locations in the given signals. The segmentation map obtained from processing the signal
through encoder and decoder module is expressed as (1):

𝑧 (𝑖) = 𝐸(𝑥 (𝑖) , 𝜃1 )𝑦̂ (𝑖) = 𝐷(𝑧 (𝑖) , 𝜃2 ) (1)

Where 𝐸 represents the encoder block, 𝐷 represents the decoder block, 𝜃1 & 𝜃2 are the weight
vector which are optimized by incorporating the binary cross entropy loss between actual and predicted R
peaks. The loss function is given as (2):
1
𝐿𝑜𝑠𝑠 = ∑𝑁 ̂ + (1 − 𝑦𝑖 ) × log(1 − 𝑦𝑖 )
𝑖=1 𝑦𝑖 × log 𝑦 (2)
𝑁

− Based on the R peaks, we obtain RR interval which can be used to approximate the P and T waves. The
T wave lie next to 1st R peak whereas P-wave is present in the 2nd R peak in the current RR interval.
− In order to select the T-wave, 15% of RR interval is added to its 1st R Peak location and continue adding
55% of RR interval to the same location.
− In order to select the P-wave, 65% portion of RR interval is added to the 1 st R peak location and
continues adding till 95% to the same location.
− P and T wave peaks are identified based on the highest value in their corresponding window.
− The Q peak is estimated by identifying the minimum value in window starting from 20ms before R
peak.
− In order to select the S peak, the lowest value in R Peak is identified from its peak to 20ms after the
peak.
Based on these peaks, we estimate several attributes such as PR interval, QRS detection, QT
interval, QT corrected and vent rate (BPM). These parameters can be computed as follows:
− PR Interval: it can be computed as (3):

(𝑅𝑙𝑜𝑐(𝑖) −𝑃𝑙𝑜𝑐(𝑖) )
𝑡𝑝𝑟 (𝑖) = (3)
𝑓𝑠

Where 𝑓𝑠 denotes the sampling frequency, 𝑅𝑙𝑜𝑐 represents the locations of R peak, 𝑃𝑙𝑜𝑐 represents the
locations of 𝑃 peaks
− QRS duration: it is computed as (4):

(𝑆𝑙𝑜𝑐(𝑖) +𝑥)−(𝑄𝑙𝑜𝑐(𝑖) −𝑥)

𝑡𝑞𝑟𝑠 (𝑖) = (4)
𝑓𝑠
Where 𝑥 is the immediate 5 ms from signal which are added to the 𝑆𝑙𝑜𝑐 and subtracted from 𝑄𝑙𝑜𝑐 because
QRS duration is identified from start of Q till end of S peak.

Int J Artif Intell, Vol. 13, No. 1, March 2024: 1090-1103

Int J Artif Intell ISSN: 2252-8938  1095

− QT interval: this interval can be computed as (5):

𝑇𝑙𝑜𝑐(𝑖) +(𝑡𝑟𝑟(𝑖) ×0.13)−(𝑄𝑙𝑜𝑐(𝑖) −𝑥)

𝑡𝑞𝑡 (𝑖) = (5)
𝑓𝑠

Where 𝑇𝑙𝑜𝑐 shows the locations of 𝑇 peaks, a constant 0.13 is multiplied with 𝑡𝑟𝑟 , and 𝑥 is subtracted from
the 𝑄𝑙𝑜𝑐

2.2. Iris feature extraction

This subsection describes he proposed approach for iris segmentation and feature extraction. The iris
segmentation plays important role where we consider and inner and outer boundaries of iris. The Figure 5
depicts the inner and outer boundary region. In this, we extend our previous feature extraction technique for
iris image authentication. Figure 5(a) depicts the original image whereas Figure 5(b) shows the identification
of inner and outer boundaries which can be useful in segmentation process.

(a) (b)

Figure 5. Inner and outer boundary representation, (a) original image and (b) boundary detection

For simplicity, it is assumed that the iris image acquisition devices extract the square region of
image where center of iris image is closer to the center of square region. In order to obtain the boundary
information, we initiate the iterative process from the center point of image which is denoted by 𝑃(𝑋𝐼 , 𝑌𝐼 ).
Then, the profile operations are performed in vertical and horizontal directions which are denoted as 𝑉𝑃 and
𝐻𝑃 , respectively. The center point is denoted by 𝑃(𝑋0 , 𝑌0 ). In order to find the boundaries, we focus on
estimating the radius which is computed as (6).

𝑟𝑠∗ = arg min 𝐷𝑠 (𝑟) (6)

𝑟∈𝑅

Where 𝐷 represents the left, right, top and bottom points of the image. Further, this image is
processed to obtain the partial derivatives and normalized contour integrals. This includes Gaussian
smoothing function, and smoothing scale which examines entire image repeatedly over the given parameter
set (𝑥0 , 𝑦𝑜 , 𝑟). This can be expressed as (7).

𝜕 𝐼(𝑥,𝑦)
𝑚𝑎𝑥(𝑟,𝑥0 ,𝑦0) |𝐺𝜎 (𝑟) ∗ ∮ 𝑑𝑠| (7)
𝜕𝑟 2𝜋𝑟

In order to obtain the normalized output, all points which are inside the iris region are mapped to
polar coordinates (𝑟, 𝜃) where 𝑟 represents the interval [0,1] and 𝜃 denotes the angle as [0, 2𝜋]. Based on
this remapping, we obtain the two circular edges which are represented in Figure 6.

Figure 6. Radius estimation

Deep learning based biometric authentication using electrocardiogram and iris (Ashwini Kailas)
1096  ISSN: 2252-8938

This helps to obtain the inner and outer radius on the acquired image which is helpful in cropping
and segmenting the iris region. Figure 7 shows the outcome of inner and outer boundary localization and
segmentation. The localized image Figure 7(a) demonstrates the identified boundaries of iris and removing
pupil and other components generates the segmented image as depicted in Figure 7(b).

(a) (b)

Figure 7. Iris localization and segmentation, (a) localization and (b) segmentation

2.2.1. Gabor-wavelet feature extraction

This portion of image is considered as region of interest (ROI) of the image which is used for further
feature extraction process. In this work we apply Gabor –wavelet feature, GLCM [26], GLDM [27] and PCA
[28]. The 1D-Gabor filter can be expressed as (8).

−(log(𝜔 ⁄𝜔0 ))2

𝐺(𝜔) = exp ( ) (8)
2(log(𝜎 ⁄𝜔0 ))2

Where 𝜔0 denotes the central frequency. In next stage, we convert the polar coordinates to Cartesian
coordinates, thus, the resulting frequency domain can be expressed as (9).

−(ln(𝑢1⁄𝑓0 ))2 −𝑣12

𝐺(𝜔) = exp ( ) ∗ exp ( ) (9)
2(log(𝜎𝑢 ⁄𝑓0 ))2 2𝜎𝑣2

Where 𝑓0 denotes the central frequency, 𝜎𝑢 is the bandwidth controller for 𝑢1, 𝜎𝑣 bandwidth
controller for 𝑣1 , 𝜃 denotes the orientation of filters. Based on this, the output of Gabor filter is correlated
with the input image and Gabor kernel 𝑝_𝑘(𝑥) and the Gabor filter can be expressed as (10).

𝑎𝑘 (𝑥0 ) = ∫ ∫ 𝐼(𝑥)𝑝𝑘 (𝑥 − 𝑥0 )𝑑𝑥 (10)

Where Gabor filter can be expressed as (11).

𝑘2 𝑘2 𝜎2
𝑝𝑘 (𝑥) = exp (− 𝑥 2 ) (exp(𝑖𝑘𝑥) − exp (− )) (11)
𝜎 2𝜎 2 2

Where 𝑘 denotes the vector used to describe the wave. In this work we use 𝜎 = 𝜋 and five spatial
𝜋 𝜋 𝜋 𝜋 𝜋
frequencies with varied wavenumbers as 𝑘𝑖 = ( , , , , ) along with 8 orientations from 0 to 𝜋.
2 4 8 16 32

2.2.2. GLCM and GLDM feature extraction

Further, we apply gray level co-occurrence matrix (GLCM) based texture feature extraction which is
applied on the segmented iris ROI. To extract the GLCM features, we consider different offsets ranging as
0°, 45°, 90°, and 135°. With the help of these offsets, we compute contrast, correlation, energy, homogeneity
and entropy features. Table 1 shows the expression to compute features.
Similarly, we consider GLDM feature extraction model which is based on the absolute difference
between two pixels given in a gray level and separated by a displacement 𝛿 which is denoted as motion
vector as (12).

𝛿 = |𝑆(𝑥, 𝑦) − 𝑆(𝑥 + ∆𝑥, 𝑦 + ∆𝑦)| (12)

The probability density function can be expressed as (12).

𝐷(𝑖|𝛿) = 𝑃𝑟𝑜𝑏[𝑆𝛿 (𝑥, 𝑦) = 𝑖] (13)

Int J Artif Intell, Vol. 13, No. 1, March 2024: 1090-1103

Int J Artif Intell ISSN: 2252-8938  1097

Where ∆𝑥 and ∆𝑦 are integer parameters, 𝑆𝛿 (𝑥, 𝑦) denotes the input image. In order to formulate the
feature vector, we concatenate contrast, entropy, angular second moment from PDF. Further, the obtained
feature is processed through the PCA approach which has stages:
− In first phase, it computes the mean of each vector as (14):
1
𝑥𝑚 = ∑𝑁
𝑘=1 𝑥𝑘 (14)
𝑁

further, the obtained mean is subtracted from the data vectors to generate the zero mean vectors as (15):

𝑥𝑧 = 𝑥𝑖 − 𝑥𝑚 (15)

Where 𝑥𝑧 denotes the zero mean vector, 𝑥𝑖 represents the each element of the column vector and 𝑥𝑚 is the
mean of each column vector.
− In next stage, covariance matrix is computed as (16):

𝑐 = [𝑥𝑧 𝑇 ∗ 𝑥𝑧 ] (16)

later, we compute Eigen vectors and Eigen values which are further multiplied with the zero mean vector to
produce the feature vector. The Eigen vector is represented as (17):

[𝑐 − 𝛾𝑖]𝑒 = 0 (17)

and, the obtained feature vector is expressed as (18):

𝑓𝑖 = [𝑥𝑧 ]𝑒 (18)

This complete process generates a fused feature vector which is used for training purpose.

Table 1. Feature computation formulas

Feature Description Formula

Contrast It measures the local information variation in the image 𝑓1 = ∑|𝑖 − 𝑗|2 𝑝(𝑖, 𝑗)
𝑖,𝑗
𝑓2
Correlation It measures the joint probability occurrence of specified pixel (𝑖 − 𝜇𝑖)(𝑗 − 𝜇𝑖)𝑝(𝑖, 𝑗)
=∑
𝜎𝑖 𝜎𝑗
𝑖,𝑗

Energy
It provides the details of squared elements contain in the GLCM matrix. it is also 𝑓3 = ∑ 𝑝(𝑖, 𝑗)2
known as angular second moment or uniformity 𝑖,𝑗
𝑝(𝑖, 𝑗)
Homogeneity It measures the nearness of GLCM component distribution 𝑓4 = ∑
1 + |𝑖 − 𝑗|
𝑖,𝑗
𝑀−1

Entropy It is used to determine the statistical attributes 𝑓5 = − ∑ 𝑝𝑘 log 2(𝑝𝑘 )

𝑘=0

2.3. CNN-LSTM classifier construction

In this section we briefly describe the CNN and LSTM model and present a combined architecture
for classifier construction. The current advancements in machine learning field have reported the importance
of deep learning approaches where CNN has been adopted widely to achieve the efficient classification
accuracy [5]. The CNN architectures are based on the feedforward neural network process and it is capable to
extract the robust features from convolution structures. In this work, we adopt a single-dimensional
convolutional layer with 𝑚 number of filters and 𝑘 number of kernels. For each input signal, the
corresponding attributes are fed into the corresponding different layer and the obtained output is then
processed through the bidirectional long short-term memory (BiLSTM) block to obtain the final
classification.
The LSTM architecture consist of three different gates as input gate 𝑖𝑔 , forget gate 𝑓𝑔 and output
gate 𝑜𝑔 . Moreover, it contains a dedicated memory cell 𝑐𝑡 which helps to learn the long-term dependencies in
the given sequence, and hidden state ℎ𝑡 . These components of LSTM are represented as (19):

Deep learning based biometric authentication using electrocardiogram and iris (Ashwini Kailas)
1098  ISSN: 2252-8938

𝑖𝑡 = 𝜎(𝑤𝑖 [ℎ𝑡−1 , 𝑥𝑡 ] + 𝑏𝑖 )

𝑓𝑡 = 𝜎(𝑤𝑓 [ℎ𝑡−1 , 𝑥𝑡 ] + 𝑏𝑓 )

𝑜𝑡 = 𝜎(𝑤𝑜 [ℎ𝑡−1 , 𝑥𝑡 ] + 𝑏𝑜 )

𝑐𝑡 = 𝑓𝑡 ⊙ 𝑐𝑡−1 + 𝑡𝑡 ⊙ tanh(𝑤𝑐 [ℎ𝑡−1 , 𝑥𝑡 ] + 𝑏𝑐 )

ℎ𝑡 = 𝑜𝑡 ⊙ tanh(𝑐𝑡 ) (19)

Where 𝑤𝑖 , 𝑤𝑜 and 𝑤𝑓 represents the weights of neurons. Similarly, 𝑏𝑖 , 𝑏𝑜 and 𝑏𝑓 represents the
biases, 𝜎 denotes the sigmoid function and ⊙ represents the elementwise multiplication and tanh() is the
hyperbolic tangent function. The traditional LSTM model is extended to formulate the bidirectional LSTM
where two LSTMs operate simultaneously in forward and backward direction. The backward LSTM module
is used to capture the past contextual information whereas forward LSTM captures the future information.
The final hidden state can be expressed as (20):

⃗⃗⃗𝑡 , ⃖⃗⃗⃗
ℎ𝑡 = 𝜇(ℎ ℎ𝑡 ) (20)

3. RESULTS AND DISCUSSION

In previous section, we have described the complete proposed approach to process the ECG and Iris
data to formulate a robust multimodal authentication system. In this section, we present the experimental
analysis of proposed approach and compare its obtained performance with existing schemes. In this
experiment, we have considered two different datasets as IIT Delhi Database and MIT-BIH dataset from
Physionet.

3.1. Dataset details

IIT Delhi Database: this dataset is considered for Iris image analysis. It has been acquired in
Biometrics Research Laboratory by using JIRIS, JPC1000, digital CMOS camera. All of the images in the
currently accessible database are in bitmap (*.bmp) format and obtained from 224 users. The database has
176 men and 48 women, all of whom fall into the age range of 14 to 55. A total of 224 separate folders, each
linked to an integer identity or number, make up the database's 1120 photos. All of the photographs in this
collection have a resolution of 320 by 240 pixels and were taken inside. Figure 8 depicts some sample images
from this dataset.
MIT BIH dataset: The MIT-BIH dataset encompasses 48 hours recording of two-channel
ambulatory ECG signals obtained from 47 subjects. Out of these 47 subjects. Out of these 48 hours, 23
readings are chosen form 24 hours’ ambulatory signals which includes mixed population of inpatient and
outpatient as 60% and 40% subjects respectively. Remaining 25 readings are obtained from the same set of
the signal which includes clinically vital arrhythmias. For simplification, these signals were processed
through digitization phase which generates the digitized signal at 360 samples per second per channel. This
digitization is done with 11-bit resolution over 10mV range. These signals are annotated by expert
cardiologists and entire dataset contains 110,000 annotations. Figure 8 from Figures 9(a)-9(f) depicts some
sample of original signal, different noise types and combined signal with different noises.

Figure 8. Sample images from IITD Iris dataset

Int J Artif Intell, Vol. 13, No. 1, March 2024: 1090-1103

Int J Artif Intell ISSN: 2252-8938  1099

(a) (b)

(c) (d)

(e) (f)

Figure 9. Sample ECG signals obtained from MIT-BIH dataset, (a) signal 100, (b) signal 101, (c) signal 102,
(d) signal 103, (e) signal 104, and (f) signal 105

This dataset is further divided in a ratio where 70% data is used for training and 30% data samples
are used for testing purpose. We perform random shuffling the given dataset to ensure that the data points are
in a random order. This helps in preventing any ordering biases that might exist in the data. The proposed

Deep learning based biometric authentication using electrocardiogram and iris (Ashwini Kailas)
1100  ISSN: 2252-8938

classification model uses certain hyperparameters to improve the learning process. The complete process is
performed for k fold cross-validation with k=5. These hyperparameters are demonstrated in Table 2.

Table 2. Hyperparameters
Simulation Parameter Considered Value
Loss Function Cross Entropy
Optimizer Adam
Batch Size 32
Number of Epochs 200
Learning Rate 0.001
Dropout 0.5

3.2. Performance measurement parameters

The performance of proposed approach is measured based on confusion matrix calculation. The
confusion matrix generated with the help of true positive, false positive, false negative and true negative.
Table 3 shows a sample representation of confusion matrix.

Table 3. Confusion matrix

Predicted class
Actual class
Genuine User Imposter User
Genuine User True Positive False Negative
Imposter User False Positive True Negative

Based on this confusion matrix, we measure several statistical performance parameters such as
accuracy, precision, F1-score by using proposed approach. Accuracy is the measurement of correct instance
classification out of total number instances. The accuracy is measured:
𝑇𝑃+𝑇𝑁
𝐴𝑐𝑐 =
𝑇𝑃+𝑇𝑁+𝐹𝑃+𝐹𝑁

Then, we compute precision of the proposed approach. It is computed by taking the ratio of true
positive and (true and false) positives.
𝑇𝑃
𝑃=
𝑇𝑃+𝐹𝑃

Finally, we compute the F-measure based on the sensitivity and precision values, which is expressed:
2∗𝑃∗𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦
𝐹=
𝑃+𝑆𝑒𝑛𝑠𝑖𝑡𝑖𝑣𝑖𝑡𝑦

3.3. Comparative analysis with combined feature extraction process

With the help of aforementioned performance measurement parameters, we measure the
performance of proposed approach. In order to classify the ECG and iris dataset, we consider 50 samples
where these data samples are stored in each folder which is allocated their corresponding class. We measure
the performance in terms of precision, sensitivity, specificity, F1-score and accuracy for different classifiers.
Moreover, we evaluate the performance for ECG and Iris separately and later measured the performance for
combined feature extraction. Table 4 shows the obtained classification performance.

Table 4. Classification performance for different classifiers

Classifier Precision F1-score Sensitivity Specificity Accuracy
Iris ECG combine Iris ECG combine Iris ECG combine Iris ECG combine Iris ECG combine
SVM 0.925 0.910 0.928 0.942 0.951 0.961 0.941 0.951 0.941 0.941 0.935 0.925 0.931 0.955 0.940
KNN 0.930 0.923 0.936 0.951 0.952 0.961 0.945 0.955 0.950 0.942 0.930 0.928 0.941 0.9450 0.951
Neural Network 0.935 0.941 0.940 0.952 0.952 0.962 0.961 0.955 0.950 0.945 0.938 0.940 0.945 0.936 0.955
Naïve Bayes 0.938 0.945 0.951 0.955 0.951 0.965 0.962 0.961 0.962 0.941 0.941 0.965 0.940 0.941 0.951
Random Forest 0.941 0.952 0.951 0.958 0.963 0.965 0.961 0.951 0.952 0.942 0.945 0.961 0.945 0.955 0.960
Decision Tree 0.945 0.948 0.952 0.96 0.962 0.965 0.966 0.9612 0.955 0.945 0.955 0.962 0.955 0.951 0.971
Bagged ensemble 0.946 0.951 0.958 0.960 0.968 0.969 0.968 0.9712 0.951 0.955 0.956 0.965 0.950 0.958 0.965
Proposed 0.952 0.955 0.962 0.965 0.975 0.975 0.979 0.9750 0.978 0.9650 0.965 0.9710 0.961 0.9680 0.985

Int J Artif Intell, Vol. 13, No. 1, March 2024: 1090-1103

Int J Artif Intell ISSN: 2252-8938  1101

3.4. Comparative analysis with different attacks on the input data

In order to show the robustness of proposed approach, we incorporated several attacks to the
original data and measured the performance of proposed model. For ECG signal, we consider three different
types of noises as baseline wander, muscle artifacts and power line interference. On the other hand, we
consider different attacks on iris data such as cropping, rotation, adding noise, image blurring. Figure 9
depicts different attacks on image data where Figure 10(a) shows the original image, Figure 10(b) depicts the
image with cropping attack, Figure 10(c) depicts the image after rotation attack, Figure 10(d) is the noisy
image and Figure 10(e) shows the image after motion blur attack.
For this experiment, we measure the outcome of proposed approach and compared with existing
classification schemes. Table 5 shows the comparative performance in terms of precision and F1-score for
different attacks such as Cropping, Rotation, Noise, and Blurring.
Similarly, we have measured the performance in terms of Sensitivity and Specificity by considering
aforementioned attacks. The obtained performance is presented in Table 6. Finally, we present the outcome
of proposed approach in terms of classification accuracy and compare it with traditional classifiers. Table 7
shows the obtained performance. Table 6 shows that the proposed approach is able to achieve desired
accuracy for four different types of attacks and outperforms traditional classification method.

(a) (b) (c) (d) (e)

Figure 10. Sample attack images, (a) original image, (b) image cropping, (c) image rotation attack,
(d) noisy image, and (e) motion blur attack

Table 5. Precision and F1-score performance for different attacks

Classifier Precision F1 score
Cropping Rotation Noise Blurring Cropping Rotation Noise Blurring
SVM 0.825 0.830 0.851 0.861 0.821 0.830 0.815 0.821
KNN 0.803 0.841 0.833 0.835 0.803 0.8125 0.8210 0.841
Neural Network 0.821 0.844 0.850 0.840 0.821 0.8210 0.8350 0.851
Naïve Bayes 0.833 0.850 0.855 0.855 0.833 0.8315 0.8410 0.856
Random Forest 0.850 0.8350 0.825 0.861 0.856 0.8410 0.8331 0.862
Decision Tree 0.855 0.841 0.803 0.850 0.881 0.8511 0.8516 0.852
Bagged ensemble 0.870 0.856 0.825 0.861 0.901 0.8920 0.8620 0.860
Proposed Model 0.9210 0.892 0.912 0.910 0.921 0.9210 0.918 0.901

Table 6. Sensitivity and Specificity performance for different attacks

Classifier Sensitivity Specificity
Cropping Rotation Noise Blurring Cropping Rotation Noise Blurring
SVM 0.7915 0.8011 0.8215 0.810 0.8345 0.8511 0.8410 0.810
KNN 0.812 0.8050 0.835 0.805 0.855 0.820 0.842 0.845
Neural Network 0.805 0.810 0.840 0.815 0.82 0.795 0.844 0.813
Naïve Bayes 0.815 0.829 0.841 0.8351 0.83 0.83 0.851 0.818
Random Forest 0.8351 0.815 0.844 0.8401 0.866 0.843 0.855 0.836
Decision Tree 0.8401 0.808 0.805 0.830 0.850 0.844 0.842 0.849
Bagged ensemble 0.854 0.851 0.815 0.841 0.830 0.858 0.845 0.865
Proposed Model 0.9012 0.8915 0.923 0.915 0.905 0.911 0.846 0.895

Table 7. Classification accuracy performance for different attacks

Classifier Accuracy
Cropping Rotation Noise Blurring
SVM 0.845 0.813 0.821 0.84
KNN 0.861 0.835 0.835 0.835
Neural Network 0.868 0.840 0.840 0.840
Naïve Bayes 0.895 0.855 0.835 0.855
Random Forest 0.890 0.861 0.840 0.861
Decision Tree 0.885 0.840 0.855 0.850
Bagged ensemble 0.895 0.855 0.861 0.8910
Proposed Model 0.934 0.8950 0.921 0.9325

Deep learning based biometric authentication using electrocardiogram and iris (Ashwini Kailas)
1102  ISSN: 2252-8938

4. CONCLUSION
In this work, we have mainly focused on development of multimodal biometric authentication
system with improved accuracy and robustness. The proposed approach considers ECG and iris data due to
their significant advantages in biometric authentication for physiological and physiological behavior. The
proposed model performs deep learning based segmentation method for R peak segmentation from ECG
signals. Further, different morphological features are extracted with the help of this peak data. Similarly,
wavelet, GLCM and GLDM feature extraction processes are applied to extract the robust features from iris
images. Finally, CNN-LSTM based hybrid classifier is used to learn these patterns and classify the users as
genuine or imposter. The comparative study shows the proposed approach achieves average performance as
0.962, 0.975, 0.978, 0.9710 and 0.985 in terms of Precision, F1-score, Sensitivity, Specificity, and Accuracy.
However, this process is tested on limited number of attacks such as Cropping, Rotation, Noise, and blurring
with less intensity of attack whereas real-time attacks could be more intense. Moreover, this work can be
extended by incoproating multimodal authentication system along with liveliness detection of subjects.

REFERENCES
[1] W. Xu et al., “RFace: anti-spoofing facial authentication using COTS RFID,” in IEEE INFOCOM 2021 - IEEE Conference on
Computer Communications, May 2021, pp. 1–10. doi: 10.1109/INFOCOM42981.2021.9488737.
[2] J.-H. Im, S.-Y. Jeon, and M.-K. Lee, “Practical Privacy-preserving face authentication for smartphones secure against malicious
clients,” IEEE Transactions on Information Forensics and Security, vol. 15, pp. 2386–2401, 2020, doi:
10.1109/TIFS.2020.2969513.
[3] M. K. Morampudi, M. V. N. K. Prasad, and U. S. N. Raju, “Privacy-preserving iris authentication using fully homomorphic
encryption,” Multimedia Tools and Applications, vol. 79, no. 27–28, pp. 19215–19237, Jul. 2020, doi: 10.1007/s11042-020-
08680-5.
[4] A. R. Khan et al., “Authentication through gender classification from iris images using support vector machine,” Microscopy
Research and Technique, vol. 84, no. 11, pp. 2666–2676, Nov. 2021, doi: 10.1002/jemt.23816.
[5] M. Hammad, P. Pławiak, K. Wang, and U. R. Acharya, “ResNet‐attention model for human authentication using ECG signals,”
Expert Systems, vol. 38, no. 6, Sep. 2021, doi: 10.1111/exsy.12547.
[6] M. Ingale, R. Cordeiro, S. Thentu, Y. Park, and N. Karimian, “ECG biometric authentication: A comparative analysis,” IEEE
Access, vol. 8, pp. 117853–117866, 2020, doi: 10.1109/ACCESS.2020.3004464.
[7] X. Yin, S. Wang, M. Shahzad, and J. Hu, “An IoT-oriented privacy-preserving fingerprint authentication system,” IEEE Internet
of Things Journal, vol. 9, no. 14, pp. 11760–11771, Jul. 2022, doi: 10.1109/JIOT.2021.3131956.
[8] W. Yang, S. Wang, K. Yu, J. J. Kang, and M. N. Johnstone, “Secure fingerprint authentication with homomorphic encryption,” in
2020 Digital Image Computing: Techniques and Applications (DICTA), Nov. 2020, pp. 1–6. doi:
10.1109/DICTA51227.2020.9363426.
[9] X. Zhang, D. Cheng, P. Jia, Y. Dai, and X. Xu, “An efficient android-based multimodal biometric authentication system with face
and voice,” IEEE Access, vol. 8, pp. 102757–102772, 2020, doi: 10.1109/ACCESS.2020.2999115.
[10] A. Tarannum, Z. U. Rahman, L. K. Rao, T. Srinivasulu, and A. Lay-Ekuakille, “An efficient multi-modal biometric sensing and
authentication framework for distributed applications,” IEEE Sensors Journal, vol. 20, no. 24, pp. 15014–15025, Dec. 2020, doi:
10.1109/JSEN.2020.3012536.
[11] E. Sujatha, J. Sathiya Jeba Sundar, P. Deivendran, and G. Indumathi, “Multimodal biometric algorithm using iris, finger vein,
finger print with hybrid GA, PSO for authentication,” 2021, pp. 267–283. doi: 10.1007/978-981-15-8335-3_22.
[12] M. Hammad and K. Wang, “Parallel score fusion of ECG and fingerprint for human authentication based on convolution neural
network,” Computers & Security, vol. 81, pp. 107–122, Mar. 2019, doi: 10.1016/j.cose.2018.11.003.
[13] S. Zhao, W. Nie, and B. Zhang, “Deep learning-based hyperspectral multimodal biometric authentication system using palmprint
and dorsal hand vein,” in AI and Deep Learning in Biometric Security, First edition. | Boca Raton, FL: CRC Press, 2021. |: CRC
Press, 2021, pp. 1–22. doi: 10.1201/9781003003489-1.
[14] E. Al Alkeem et al., “Robust deep identification using ECG and multimodal biometrics for industrial internet of things,” Ad Hoc
Networks, vol. 121, p. 102581, Oct. 2021, doi: 10.1016/j.adhoc.2021.102581.
[15] R. Avanzato and F. Beritelli, “Automatic ECG diagnosis using convolutional neural network,” Electronics, vol. 9, no. 6, p. 951,
Jun. 2020, doi: 10.3390/electronics9060951.
[16] G. Kaur, A. Girdhar, and M. Kaur, “Enhanced iris recognition system – an integrated approach to person identification,”
International Journal of Computer Applications, vol. 8, no. 1, pp. 1–5, Oct. 2010, doi: 10.5120/1182-1630.
[17] M. Regouid, M. Touahria, M. Benouis, and N. Costen, “Multimodal biometric system for ECG, ear and iris recognition based on
local descriptors,” Multimedia Tools and Applications, vol. 78, no. 16, pp. 22509–22535, Aug. 2019, doi: 10.1007/s11042-019-
7467-x.
[18] D. Jiang, G. Zhang, O. W. Samuel, F. Liu, and H. Xiao, “Dual-Factor WBAN Enhanced Authentication System Based on Iris and
ECG Descriptors,” IEEE Sensors Journal, vol. 22, no. 19, pp. 19000–19009, Oct. 2022, doi: 10.1109/JSEN.2022.3198645.
[19] B. A. El-Rahiem, F. E. A. El-Samie, and M. Amin, “Multimodal biometric authentication based on deep fusion of
electrocardiogram (ECG) and finger vein,” Multimedia Systems, vol. 28, no. 4, pp. 1325–1337, Aug. 2022, doi: 10.1007/s00530-
021-00810-9.
[20] S. B. Jadhav, N. K. Deshmukh, and V. T. Humbe, “HDL-PI: hybrid deeplearning technique for person identification using
multimodal finger print, iris and face biometric features,” Multimedia Tools and Applications, vol. 82, no. 19, pp. 30039–30064,
Aug. 2023, doi: 10.1007/s11042-022-14241-9.
[21] S. P. Singh and S. Tiwari, “A dual multimodal biometric Authentication system based on WOA-ANN and SSA-DBN
Techniques,” Sci, vol. 5, no. 1, p. 10, Mar. 2023, doi: 10.3390/sci5010010.
[22] S. Amritha Varshini and J. Aravinth, “Hybrid level fusion schemes for multimodal biometric authentication system based on
matcher performance,” 2021, pp. 431–447. doi: 10.1007/978-981-33-6862-0_35.
[23] P. Huang, L. Guo, M. Li, and Y. Fang, “Practical privacy-preserving ECG-based authentication for IoT-based healthcare,” IEEE
Internet of Things Journal, vol. 6, no. 5, pp. 9200–9210, Oct. 2019, doi: 10.1109/JIOT.2019.2929087.

Int J Artif Intell, Vol. 13, No. 1, March 2024: 1090-1103

Int J Artif Intell ISSN: 2252-8938  1103

[24] A. Kumar, S. Jain, and M. Kumar, “Face and gait biometrics authentication system based on simplified deep neural networks,”
International Journal of Information Technology, vol. 15, no. 2, pp. 1005–1014, Feb. 2023, doi: 10.1007/s41870-022-01087-5.
[25] W. Xu, L. Wang, B. Wang, and W. Cheng, “Intelligent recognition algorithm of multiple myocardial infarction based on
morphological feature extraction,” Processes, vol. 10, no. 11, p. 2348, Nov. 2022, doi: 10.3390/pr10112348.
[26] F. Özbilgin, Ç. Kurnaz, and E. Aydın, “Prediction of coronary artery disease using machine learning techniques with iris
analysis,” Diagnostics, vol. 13, no. 6, p. 1081, Mar. 2023, doi: 10.3390/diagnostics13061081.
[27] R. Gutsche et al., “Radiomics outperforms semantic features for prediction of response to stereotactic radiosurgery in brain
metastases,” Radiotherapy and Oncology, vol. 166, pp. 37–43, Jan. 2022, doi: 10.1016/j.radonc.2021.11.010.
[28] H. Hafeez, M. N. Zafar, C. A. Abbas, H. Elahi, and M. O. Ali, “Real-time human authentication system based on iris
recognition,” Eng, vol. 3, no. 4, pp. 693–708, Dec. 2022, doi: 10.3390/eng3040047.

BIOGRAPHIES OF AUTHORS

Ashwini Kailas holds a Doctor of Informatics degree from Kyoto University,

Japan in 2004. He also received his B.Sc. and M.Sc. (Mathematics) from Assiut University,
Egypt in 1993 and 1997, respectively. He is currently an associate professor at Computer
Science Department in Jamoum, Umm Al-Qura University, Makkah, Saudi Arabia. He is also
an associate professor of artifcial intelligence in Assiut University since January 2012. His
research includes meta-heuristics, global optimization, machine learning, data mining,
bioinformatics, graph theory and parallel programming. He has published over 70 papers in
international journals and conferences. From July 2005 to July 2007, he was a JSPS research
fellow in Kyoto University, Japan. He can be contacted at email: [email protected] or
[email protected].

Geevagondanahalli Narayanappa Keshava Murthy is currently working as an

Assistant Professor, in the Department of Electronics and Instrumentation Engineering, with
23 years of teaching experience and 2 years of industry experience. He received his B.E
degree from the esteemed Siddaganga Institute of Technolgy, Tumakuru in the year 1996, M.E
from UVCE, Bangalore in 2001 and was awarded Ph.D. degree in 2016 from VIT University,
one among top 12 universities of India. His research interests include Smart sensors,
Biomedical Engineering, Biosciences and Cognitive neuroscience. He has more than 13
papers published in peer-reviewed international journals and more than 10 conferences in his
credit and has received research award twice for his publications at VIT University. He has
served as session chair for few conferences and is BOE member for few colleges. As an
extension of his work and knowledge currently he is guiding 2 Ph. D students. He also has
delivered lectures in various engineering colleges and Universities. He can be contacted at
email: [email protected].

Deep learning based biometric authentication using electrocardiogram and iris (Ashwini Kailas)