Deep Learning for Biomedical Data Analysis Techniques,

Approaches, and Applications

Visit the link below to download the full version of this book:

https://medipdf.com/product/deep-learning-for-biomedical-data-analysis-technique
s-approaches-and-applications/

Click Download Now

Editor
Mourad Elloumi
Computing and Information Technology
The University of Bisha
Bisha, Saudi Arabia

ISBN 978-3-030-71675-2 ISBN 978-3-030-71676-9 (eBook)

https://doi.org/10.1007/978-3-030-71676-9

© Springer Nature Switzerland AG 2021

This work is subject to copyright. All rights are reserved by the Publisher, whether the whole or part of
the material is concerned, specifically the rights of translation, reprinting, reuse of illustrations, recitation,
broadcasting, reproduction on microfilms or in any other physical way, and transmission or information
storage and retrieval, electronic adaptation, computer software, or by similar or dissimilar methodology
now known or hereafter developed.
The use of general descriptive names, registered names, trademarks, service marks, etc. in this publication
does not imply, even in the absence of a specific statement, that such names are exempt from the relevant
protective laws and regulations and therefore free for general use.
The publisher, the authors, and the editors are safe to assume that the advice and information in this book
are believed to be true and accurate at the date of publication. Neither the publisher nor the authors or
the editors give a warranty, expressed or implied, with respect to the material contained herein or for any
errors or omissions that may have been made. The publisher remains neutral with regard to jurisdictional
claims in published maps and institutional affiliations.

This Springer imprint is published by the registered company Springer Nature Switzerland AG
The registered company address is: Gewerbestrasse 11, 6330 Cham, Switzerland
Contents

Part I Deep Learning for Biomedical Data Analysis

1-Dimensional Convolution Neural Network Classification
Technique for Gene Expression Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3
Samson Anosh Babu Parisapogu, Chandra Sekhara Rao Annavarapu,
and Mourad Elloumi
Classification of Sequences with Deep Artificial Neural Networks:
Representation and Architectural Issues . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27
Domenico Amato, Mattia Antonino Di Gangi, Antonino Fiannaca,
Laura La Paglia, Massimo La Rosa, Giosué Lo Bosco, Riccardo Rizzo,
and Alfonso Urso
A Deep Learning Model for MicroRNA-Target Binding . . . . . . . . . . . . . . . . . . . . 61
Ahmet Paker and Hasan Oğul
Recurrent Neural Networks Architectures for Accidental Fall
Detection on Wearable Embedded Devices. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81
Mirto Musci and Marco Piastra

Part II Deep Learning for Biomedical

Image Analysis
Medical Image Retrieval System Using Deep Learning Techniques . . . . . . . 101
Jitesh Pradhan, Arup Kumar Pal, and Haider Banka
Medical Image Fusion Using Deep Learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129
Ashif Sheikh, Jitesh Pradhan, Arpit Dhuriya,
and Arup Kumar Pal
Deep Learning for Histopathological Image Analysis. . . . . . . . . . . . . . . . . . . . . . . . 153
Cédric Wemmert, Jonathan Weber, Friedrich Feuerhake,
and Germain Forestier

v
vi Contents

Innovative Deep Learning Approach for Biomedical Data

Instantiation and Visualization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 171
Ryad Zemouri and Daniel Racoceanu
Convolutional Neural Networks in Advanced Biomedical
Imaging Applications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 197
Daniel A. Greenfield, Germán González, and Conor L. Evans

Part III Deep Learning for Medical Diagnostics

Deep Learning for Lung Disease Detection from Chest X-Rays Images . . . 239
Ebenezer Jangam, Chandra Sekhara Rao Annavarapu,
and Mourad Elloumi
Deep Learning in Multi-Omics Data Integration in Cancer Diagnostic . . . 255
Abedalrhman Alkhateeb, Ashraf Abou Tabl, and Luis Rueda
Using Deep Learning with Canadian Primary Care Data for
Disease Diagnosis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273
Hasan Zafari, Leanne Kosowan, Jason T. Lam, William Peeler,
Mohammad Gasmallah, Farhana Zulkernine, and Alexander Singer
Brain Tumor Segmentation and Surveillance with Deep Artificial
Neural Networks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 311
Asim Waqas, Dimah Dera, Ghulam Rasool, Nidhal Carla Bouaynaya,
and Hassan M. Fathallah-Shaykh

Index . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 351
 Part I
Deep Learning for Biomedical
Data Analysis
1-Dimensional Convolution Neural
Network Classification Technique for
Gene Expression Data

Samson Anosh Babu Parisapogu, Chandra Sekhara Rao Annavarapu,

and Mourad Elloumi

Abstract In the field of bioinformatics, the development of computational methods

has drawn significant interest in predicting clinical outcomes of biological data,
which has a large number of features. DNA microarray technology is an approach
to monitor the expression levels of sizable genes simultaneously. Microarray
gene expression data is more useful for predicting and understanding various
diseases such as cancer. Most of the microarray data are believed to be high
dimensional, redundant, and noisy. In recent years, deep learning has become
a research topic in the field of Machine Learning (ML) that achieves remark-
able results in learning high-level latent features within identical samples. This
chapter discusses various filter techniques which reduce the high dimensionality
of microarray data and different deep learning classification techniques such as
2-Dimensional Convolution Neural Network (2D- CNN) and 1-Dimensional CNN
(1D-CNN). The proposed method used the fisher criterion and 1D-CNN techniques
for microarray cancer samples prediction.

Keywords Gene expression data · Deep learning · Convolution neural network ·

Machine learning · Classification

1 Introduction

Computational molecular biology is an interdisciplinary subject that includes dif-

ferent fields as biological science, statistics, mathematics, information technology,
physics, chemistry and computer science. The analysis of biological data involves
the study of a wide range of data generated in biology. This biological data is gener-

S. A. B. Parisapogu · C. S. R. Annavarapu ()

Department of Computer Science and Engineering, Indian Institute of Technology, Dhanbad,
Jharkhand, India
e-mail: [email protected]; [email protected]
M. Elloumi
Faculty of Computing and Information Technology, The University of Bisha, Bisha, Saudi Arabia

© Springer Nature Switzerland AG 2021 3

M. Elloumi (ed.), Deep Learning for Biomedical Data Analysis,
https://doi.org/10.1007/978-3-030-71676-9_1
4 S. A. B. Parisapogu et al.

Table 1 Gene expression Sample 1 ... Sample j ... Sample M

data format
Gene 1 a11 ... a1j ... a1M
... ... ... ... ... ...
Gene i ai1 ... aij ... aiM
... ... ... ... ... ...
Gene N aN 1 ... aNj ... aN M

ated from different sources, including laboratory experiments, medical records, etc.
Different types of biological data include nucleotide sequences, gene expression
data, macromolecular 3D structure, metabolic pathways, protein sequences, protein
patterns or motifs and medical images [1]. Unlike a genome, which provides only
static sequence information, microarray experiments produce gene expression pat-
terns that provide cell functions dynamic information. Understanding the biological
intercellular and intra-cellular processes underlying many diseases is essential for
improving the sample classification for diagnostic and prognostic purposes and
patient treatments.
Biomedical specialists are attempting to find relationships among genes and
disease or formative stages, as well as relationships between genes. For example,
an application of microarrays is the revelation of novel biomarkers for cancer,
which can give increasingly exact determination and monitoring tools for early
recognition of a specific subtype of disease or assessment of the viability of a
particular treatment protocol. Different technologies are used to interpret these
biological data. For example, microarray technology is useful for measuring the
expression levels of a large number of genes under different environmental con-
ditions, and Next Generation Sequencing (NGS) Technology for massively parallel
DNA sequencing. This kind of experiments on a large amount of biological data
leads to an absolute requirement of collection, storage and computational analysis
[2].
In the last decade, biological data analytics has improved with the development
of associated techniques such as Machine Learning (ML), Evolutionary Algorithms
(EA) and Deep Learning (DL). These techniques are capable of handling more
complex relationships in the biological data. For example, the prediction of cancer
disease from the microarray data can be carried out using different ML algorithms
(classification and clustering). While dealing with the microarray datasets, which
has high dimensionality, are usually complex and noisy makes the classification
task inconvenient [3, 4]. Table 1 presents the gene expression data format.
Due to this high dimensionality and redundancy, usual classification methods
became challenging to apply on gene expression data efficiently. To reduce the
problem of high dimensionality, improving learning accuracy and removing
irrelevant data from gene expression data, many filter [48] and wrapper [49]
approaches were applied. The filter method selects feature subsets independently of
any learning algorithm and relies on various measures of the general characters
of the training data. The wrapper method uses the predictive accuracy of a
1-Dimensional Convolution Neural Network Classification Technique for Gene. . . 5

predetermined learning algorithm to determine the goodness of the selected subsets

and is computationally expensive. Some of the filter methods are as follows:
Z-Score, Minimum Redundancy and Maximum Relevance (mRMR) [5], T-Test
[6], Information Gain [7], Fisher criterion [8] and K-Means-Signal-to-Noise Ratio
(KM-SNR) ranking [9] etc. The effective gene selection aims to select the small
subset of essential genes that are highly predictive and maximizing the ability of
classifiers to classify the samples accurately. The task of finding reducts is reported
to be NP-hard [10, 11].
Artificial intelligence (AI) is the reproduction of human intelligence by machine
and computer systems. This evolution of reproduction done by learning—rules to
use information, reasoning—rules to reach proper execution and self-correction—
take appropriate actions based on learning, reasoning procedures. ML is made
use of computational methods to get better machinery performances by detecting
influential patterns and inconsistency information. ML makes decisions based on
what they memorize or learn from data.
DL solves problems that were difficult with ML. DL uses Artificial Neural
Networks (ANNs) [33]. An ANN acts very much like a human brain to increase
computational work and provide accurate results. At present, AI is getting smarter
by changing the way of its branches, namely ML and DL with enormous com-
putational powers. Convolutional Neural Network (CNN) mimics brain function in
processing information [19].
In this chapter, 1D-CNN, which is a multilayered DL algorithm, is proposed
to classify microarray cancer data to recognize the kind of disease. CNN has the
capacity in managing with insufficient data and boosting classification performance.
Furthermore, CNN is also influential in detecting latent characteristics of cancer
from comparable types. The organization of the chapter is as follows. Section 2
describes the related works of gene expression data classification. The preliminaries,
which includes various filter and DL classification techniques, are explained in
Sect. 3. Section 4 provides the details of the selected datasets. Section 5 describes
the proposed approach, and Sect. 6 presents the experimental results. Finally, Sect. 7
concludes the chapter.

2 Related Works

The accessibility of open repositories of data that are appropriate for AI

research is highly essential to the field of biomedical informatics. The
University of California, Irvine (UCI) ML repository (https://archive.ics.uci.edu/
ml/index.php), which contains a collection of databases, and data generators, which
are used by the ML community for the empirical analysis of ML algorithms. It has
enabled many numbers of analysts to demonstrate the performance of new statistical
and ML algorithms. ML has excellent accuracy results when it has pre-processed
data, but in real-time applications, it is not so easy to obtain pre-processed data.
6 S. A. B. Parisapogu et al.

Microarray information has been used in past researches to perform malignant

growth, such as cancer classification by utilizing ML methodologies. Decision Tree
(DT) was the most primitive ML method acquainted in comparing human proteins
to informative genes in proteins containing ailment [29]. In analyzing microarray
data, two conventional techniques which have been engaged are classification
and clustering. Moreover, there are various procedures which have been executed
beforehand in classifying microarray data, which includes k-Nearest Neighbours
(KNN) [30], Support Vector Machines (SVM) [31], Multilayer Perceptron (MLP)
[32], and variants of ANNs [33].
The breast cancer and leukemia datasets in performing selection of informative
genes from microarray data reasoned that KNN classifier performed superiorly than
random forest in terms of accurate classification [34]. From a hybrid classifier with a
combination of Particle Swarm Optimization (PSO) and adaptive KNN in choosing
useful genes recognizes a bunch of genes that meet the criteria of classification [35].
Deep Neural Network (DNN) has multiple neural layers where each layer
describes hidden information from real-world raw data [12–14]. DL has been
applied in many fields of computational biology. In [15], the author designed a deep
learning-based framework for the prediction of drug-target interaction. In [16, 17],
deep learning was applied for the prediction of eukaryotic protein subcellular
localization. In [18], deep learning algorithm based on the 2D-CNN was applied for
classification of microarray gene expression data.

3 Preliminaries

In this section, we present the details of three various filter approaches that can apply
on microarray gene expression data, deep learning and CNN.

3.1 Forward Selection and Minimum Redundancy: Maximum

Relevance Criterion Filter Approach

The Minimum Redundancy - Maximum Relevance (MRMR) criterion filter

approach can be used to rank the genes based on their relevance and redundancy
values. This criterion has been explained in [20–22] with forward selection search
strategy. Given a set XS of selected variables, the method updates XS with the
variable Xi ∈ XR that maximizes vi − zi , where vi represents the relevance terms
and zi represents the redundancy terms. In detail, vi is the relevance of Xi to the
output of Y alone, and zi is the average redundancy of Xi to each selected variables
Xj ∈ XS :

vi = I (Xi ; Y ) (1)
1-Dimensional Convolution Neural Network Classification Technique for Gene. . . 7

1 
zi = I (Xi ; Xj ) (2)
|XS |
Xj ∈XS

XiMRMR = arg max {vi − zi } (3)

Xi ∈XR

In every step, this method chooses the variable having the best trade-off between
relevance and redundancy. This selection approach is fast and efficient. At step d of
forwarding search, the algorithm computes n − d evaluations where each evaluation
requires the estimation of d + 1 bivariate densities (one for each already selected
variable and one for with the output). Therefore, MRMR avoids the estimation of
multivariate densities by using multiple bivariate densities.
In [20], the authors justification of MRMR is as follows:

I (X; Y ) = H (X) + H (Y ) − H (X, Y ) (4)

with


n
R(X1 ; X2 ; . . . ; Xn ) = H (Xi ) − H (X) (5)
i=1

and


n
R(X1 ; X2 ; . . . ; Xn ; Y ) = H (Xi ) + H (Y ) − H (X, Y ) (6)
i=1

Hence

I (X; Y ) = R(X1 ; X2 ; . . . ; Xn ; Y ) − R(X1 ; X2 ; . . . ; Xn ) (7)

where:
• The minimum of the 2nd term R(X1 ; X2 ; . . . ; Xn ) is reached for independent
variables since, in that case, H (X) = Σi H (Xi ) and R(X1 ; X2 ; . . . ; Xn ) =
Σi H (Xi ) − H (X) = 0. Hence, if XS is already selected, a variable Xi should
have a minimal redundancy I (Xi ; XS ) with the subset. However, according to
the authors the approximation of I (Xi ; XS ) is with |S|
1
Σj ∈S I (Xi ; Xj ).
• The maximum of the 1 term R(X1 ; X2 ; . . . ; Xn ; Y ) is attained for maximally
st

dependent variables.
Qualitatively, in a sequential setting where a selected subset XS is
given, independence between the variables in X is achieved by minimizing
|XS | ΣXj ∈XS I (Xi ; Xj )  I (Xi ; XS ) and maximizing dependency between the
1

variables of X and Y , that is, by maximizing I (Xi ; Y ).

8 S. A. B. Parisapogu et al.

3.2 Fisher Criterion Filter Approach

In recent studies, the focus has been on the Fisher ranking measure [23], and this
metric has proven its robustness against data scarcity [24] in eliminating the weak
features, in this work, we used Fisher criterion to rank the genes. The Fisher criterion
calculation for genes is as followed in Eq. 8.

(μj 1 − μj 2 )2
F C(j ) = (8)
σj21 − σj22

Where, μj c is the sample mean of gene j in class c and σj2c is variance of gene j
in c. The top N genes possessing the highest Fisher value are to be selected for the
next step.

3.3 k-Means and Signal-to-Noise Ratio Filter Approach

Using this k-Means and Signal-to-Noise Ratio (KM-SNR) filter method [25], ini-
tially, all the genes are grouped into different clusters by k-means (KM) algorithm.
Then genes in each cluster are positioned separately using the Signal-to-Noise Ratio
(SNR) ranking method. These two methods are applied to overcome the redundancy
issue of the gene selection process and to decrease the search space complexity. Best
ranked genes from each cluster then sent to the next stage.

3.3.1 k-Means Algorithm

The k-Means clustering is the most utilized unsupervised learning approach,

based on limiting a formal objective function and also minimizing the maximum
distance from its closest center to every point. One of the excellent heuristics for
understanding k-Means problem depends on finding a locally minimal solution [26].
The k-Means clustering aims to divide n data points of d dimensional space into k
groups in which every data point belongs to the group with the closest mean (i.e.,
minimizing the within-cluster sum of squares), filling in as a model of the group.
These data points clustered based on feature similarity. The data points inside a
partition belong to the cluster. These partitions are also called Voronoi partitions
[50], which is a process of segmenting a space into regions, around a set of points
called seeds. The following Algorithm 1, represents the pseudocode of k-Means
clustering algorithm.
1-Dimensional Convolution Neural Network Classification Technique for Gene. . . 9

Algorithm 1: k-Means clustering algorithm

Input: k = Number of clusters to be made and
S = Set of features or Dataset
Output: k clusters
// Initialization:
1: Set the cluster value k
2: Randomly select k features as initial centers from the dataset
// Loop process
3: while not convergence do
4: Assign each data point to its nearest centroid according to
squared Euclidean distance
5: update mean of each cluster
6: end while

3.3.2 Signal-to-Noise-Ratio Ranking Approach

The Signal-to-Noise-Ratio (SNR) is a measurement of the expression patterns

relationship between signal strength and noise of the features. It is also known
as the proportion of signal to the noise power. Here, signal strength refers to the
maximal distinction in the mean expression between two classes and noise refers
to the minimal standard deviation of expression within each category [27, 28]. By
using this method, we rank the features depending on SNR score value. The SNR
score value is mathematically calculated using the following Eq. 9.

Sv = (μ1 − μ2 )/(σ1 + σ2 ) (9)

Here, Sv represents the SNR value, μ1 and μ2 are the means of class1 and class2
respectively. And, σ1 and σ2 are the standard deviations of class1 and class2
respectively.

3.4 Deep Learning

DL begins to administer our day by day life, exhibiting such arrangements that must
be envisioned in science fiction movies just a decade earlier. Indeed, the presentation
of the AlexNet, maybe one can think about, has started with the pivotal article
published in the journal, Science, in 2006 by Hinton and Salakhutdinov [36], which
described the importance of “the depth” of an ANN in ML. It fundamentally calls
attention to the way that ANNs with few hidden layers can have an amazing learning
capacity, that further improve with increasing depth—or equivalently the number of
hidden layers. Thus comes the term “Deep” learning, a specific ML branch, which
can handle intricate patterns and objects in enormous size datasets.
DL is a way to deal with ML that has drawn slowly on our knowledge of
the human mind, statistics and applied math as it developed over the past several
10 S. A. B. Parisapogu et al.

decades. In recent years, it has seen enormous growth in its popularity and
usefulness, due in large part to all-powerful computers, more substantial datasets
and techniques to train deeper ANNs. DL has solved increasingly complicated
applications with increasing accuracy over time.
Because of the capacity of learning on multilayered representations, DL is
prevalent in drawing results from complex issues. In this sense, DL is the most
progressive way to be utilized in collecting and processing abstract information
from several layers. Such attributes present DL as an appropriate way to be
considered in dissecting and contemplating gene expression information. The ability
to learning multilayered representations makes DL a flexible procedure in creating
progressively accurate outcomes in a speedier time. Multi-layered representation is
a component that structures the general architecture of DL [37].
ML and DL contrast in terms of performance relying upon the amount of data.
For low dimensionality dataset, DL works inefficiently, as it requires data consisting
of high dimensionality to comprehend learning to be carried out [38].

3.5 Convolutional Neural Network

During the most recent decade, Convolutional Neural Networks (CNNs) [51] has
turned into the de facto standard for different Computer Vision and ML tasks.
CNNs are feed-forward Artificial Neural Networks (ANNs) [52] with alternating
convolutional and subsampling layers. Profound 2D-CNNs with many hidden layers
and with many parameters can learn intricate patterns giving that they train on
a gigantic size visual database with ground-truth labels. Figure 1 visualizes the
pipeline of usual CNN architecture.1
1D-CNNs have been proposed in a few applications, for example, customized
biomedical data classification and early finding, structural health observing,

Dog
Person
Cat

Bird
Convolution Max pooling Fish
Fox

Convolutional Layers + Pooling layers Fully connected layers

Fig. 1 The pipeline of usual CNN architecture [39]

1 Image taken from https://www.sciencedirect.com/science/article/pii/S0925231215017634#

f0010.
1-Dimensional Convolution Neural Network Classification Technique for Gene. . . 11

anomaly detection and identification in motor fault detection. Moreover, the real-
time and minimum-cost hardware usage is plausible because of the reduced and
straightforward configuration of 1D-CNNs that perform only 1D convolutions. The
following subsections present a comprehensive review of the general architecture
and principals of 2D-CNNs and 1D-CNNs [40]. The CNNs has three main
components, such as the input layer, hidden layer, and latent layers. These latent
(hidden) layers may categorize as a fully-connected layer, a pooling layer, or a
convolutional layer. The definitions and details are as follows [39, 41]:

Convolution layer is the first layer in CNN architecture. The procedure of

convolution deals with the iterative execution of explicit function towards the
output of a different function. This layer comprises of various maps of neu-
rons, portrayed as maps of features or filters. It is moderately indistinguishable
in size to the dimensionality of the input. Neural reactivity deciphers through
quantifying discrete convolution of receptors. The quantification manages to
figure absolute neural weights of input and assigning activation function

Max pooling layer concerns with delivering a few grids from the splitting
convolution layers output. In matrices, most of the grid values used to be
sequenced. Operators are used in performing the calculation on every matrix
to quantify average or maximize value.

A fully connected layer is a practically complete CNN, involving 90% of

architectural parameters of CNN. The layer enables input to be transmitted in
the ANN with preset vector lengths. A layer changes dimensional data before
classification. The convolutional layer also undergoes a transformation, which
enables the retaining of data integrity.

3.5.1 2D Convolutional Neural Networks

Although it has been right around thirty years after the first CNN proposed, present-
day CNN structures still share the underlying properties with the absolute initial
one, for example, convolutional and pooling layers. To begin with, the ubiquity
and the broad scope of utilization areas of deep CNNs can ascribe to the following
advantages:
12 S. A. B. Parisapogu et al.

(24,24) (1,1)
(21,21)
(7,7) (4,4)
Convolution y1
Kx = Ky = 4 Pooling y2
sx = sy = 3 Pooling
Convolution sx = sy = 4
Kx = Ky = 4

2nd Pooling Fully-connected

Input Image 1st Convolution Layer 1st Pooling Layer 2nd Convolution Layer and Output Layers
Layer

Fig. 2 The sample illustration of CNN with two convolution and one fully-connected layers [40]

1. CNNs intertwine the feature extraction and classification procedures into a single
learning body. They can learn to optimize the features during the training stage
legitimately from the raw input.
2. As the CNN neurons are connected sparsely along with tied weights, CNNs can
process more inputs with an extraordinary computational proficiency compared
with the regular fully connected Multi-Layer Perceptrons (MLP) networks.
3. CNNs are resistant to little changes in the input information, including transla-
tion, scaling, skewing, and distortion.
4. CNNs can adapt various sizes of inputs.
In a conventional MLPs, each hidden neuron contains scalar weights, input and
output. In any case, because of the 2D nature of pictures, every neuron in CNN
contains 2-D planes for weights, known as the kernel, and input and outputs which
are known as a feature map. The classification of a 24 × 24 pixel grayscale image
of two categories by conventional CNN is shown in Fig. 2 [40]. This sample CNN
consists of two convolution layers and two pooling layers. The output of the second
pooling layer handled by a fully-connected layer and followed by the output layer
that produces the classification result.
The interconnections assigned with the weighting filters (w) and a kernel size
of (Kx , Ky ), which feeds the convolutional layers. As the convolution happens
inside the boundary limits of the image, the feature map dimension is decreased
to (Kx − 1, Ky − 1) pixels from the width and height, respectively. The values
(Sx , Sy ) initialized in pooling layers as subsampling factors. In the sample Fig. 2,
the kernel sizes of the two convolution layers assigned as Kx = Ky = 4, while
the subsampling elements set as Sx = Sy = 3 for the first pooling layer and
Sx = Sy = 4 for the subsequent one. Note that these values purposely are chosen so
that the last pooling layer (i.e. the input of fully-connected layer) outputs are scalars
(1 × 1). The output layer comprises of two fully-connected neurons relating to the
number of classes to which the image is categorized. The following steps show a
complete forward-propagation process of the given example CNN:
1. For the CNN, a grayscale 24 × 24-pixel image fed as the input layer.
2. Every neuron of the 1st convolution layer performs a linear convolution between
the image and related filter to create the input feature map of the neuron.