IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS, VOL. 23, NO.

5, SEPTEMBER 2019 2099

Hippocampus Analysis by Combination

of 3-D DenseNet and Shapes for Alzheimer’s
Disease Diagnosis
Ruoxuan Cui , Manhua Liu , and the Alzheimer’s Disease Neuroimaging Initiative

Abstract—Hippocampus is one of the first involved re- accounts for 60–80% of the cases [1]. It was proven that AD
gions in Alzheimer’s disease (AD) and mild cognitive impair- causes cell death and damage of tissue nerves throughout the
ment (MCI), a prodromal stage of AD. Hippocampal atrophy brain. Mild cognitive impairment (MCI) is a prodromal stage
is a validated, easily accessible, and widely used biomarker
for AD diagnosis. Most of existing methods compute the of AD including the progressive MCI (pMCI) and stable MCI
shape and volume features for hippocampus analysis us- (sMCI). pMCI means that the MCI subjects will convert to
ing structural magnetic resonance images (MRI). However, AD after some time while sMCI subjects are stable. Currently,
the regions adjacent to hippocampus may be relevant to AD, there are no effective cure to stop or reverse AD progress, but
and the visual features of the hippocampal region are impor-
some treatments can be developed to delay its progression, es-
tant for disease diagnosis. In this paper, we have proposed
a new hippocampus analysis method to combine the global pecially if AD is diagnosed at the early stage. Magnetic res-
and local features of hippocampus by three-dimensional onance images (MRI) is widely used as imaging modality for
densely connected convolutional networks and shape anal- AD diagnosis in clinic. It can non-invasively capture the internal
ysis for AD diagnosis. The proposed method can make use body structures and the regional brain atrophy, helping us un-
of the local visual and global shape features to enhance the derstand the anatomical and functional changes [2]. Structural
classification. Tissue segmentation and nonlinear registra-
tion are not required in the proposed method. Our method MRI scans provide detailed information about the anatomical
is evaluated with the T1-weighted structural MRIs from 811 structures and the morphology of brain tissues such as white
subjects including 192 AD, 396 MCI (231 stable MCI and 165 matter (WM), gray matter (GM) and cerebrospinal fluid (CSF).
progressive MCI), and 223 normal control in Alzheimer’s dis- In recent years, extensive efforts have been done to find the
ease neuroimaging initiative database. Experimental results biomarkers and develop computer-aided diagnosis system us-
show the proposed method achieves a classification accu-
racy of 92.29% and area under the ROC curve of 96.95% for ing pattern recognition methods to decode the disease states
AD diagnosis. Results comparison demonstrates the pro- with structural MRIs [2], [3].
posed method performs better than other methods. One popular method is based on the regions of interest (ROIs),
Index Terms—Alzheimer’s disease, hippocampus, 3D
which partition the MR brain image into multiple anatomical
densenet, deep learning, structural magnetic resonance regions by warping of a labeled atlas, and the regional measure-
image. ments such as volumes and shapes are computed as the features
for AD diagnosis [3], [4]. Zhang et al. [4] proposed to extract
I. INTRODUCTION tissue volumes from 93 ROIs to train the support vector ma-
LZHEIMER’S disease (AD) is an irreversible and chronic chine (SVM) classifier for AD diagnosis. A approach was pro-
A neurodegenerative disease with progressive impairment of
the memory and cognitive functions. In the World Alzheimer
posed to combine the marginal fisher analysis with norm based
multi-kernel learning to achieve the sparsity of ROIs, which
Report 2015, AD is the most common form of dementia as it can simultaneously select the relevant brain regions and learn
a dimensionality transformation for classification [5]. Recently,
deep learning networks have been investigated to learn the image
Manuscript received May 4, 2018; revised August 28, 2018 and Octo-
ber 10, 2018; accepted November 9, 2018. Date of publication November features and classifiers for AD/MCI diagnosis [3], [6]–[11]. A
20, 2018; date of current version September 4, 2019. This work was deep learning network on stacked auto-encoder (SAE) was pro-
supported in part by the National Natural Science Foundation of China posed to learn the complicated features inherent in the low-level
(No. 6181101049, 61375112, and 61773263), in part by the National Key
Research and Development Program of China (No.2016YFC0100903), ROI features, which were combined with the original features to
in part by the National Key Basic Research Program of China enhance the classification [3]. The whole brain is parcellated into
(973 Project, No.2015CB931802), and in part by the SMC Excellent a number of brain regions with a deep belief network trained on
Young Faculty Program of SJTU. (Corresponding author: Manhua Liu.)
The authors are with the Department of Instrument Science and each region, and multiple deep belief networks was ensembled
Engineering, School of EIEE, Shanghai Engineering Research Cen- for AD diagnosis [10]. The ROI based method can make use
ter for Intelligent Diagnosis and Treatment Instrument, Shanghai Jiao of all brain regions to achieve good classification performance.
Tong University, Shanghai 200240, China (e-mail:,[email protected];
[email protected]). There are still some limitations in these methods. First, the def-
Digital Object Identifier 10.1109/JBHI.2018.2882392 inition of ROIs requires accumulation of long-term experience

2168-2194 © 2018 IEEE. Personal use is permitted, but republication/redistribution requires IEEE permission.
See http://www.ieee.org/publications standards/publications/rights/index.html for more information.

Authorized licensed use limited to: Netaji Subhas University of Technology New Delhi. Downloaded on February 12,2024 at 10:28:43 UTC from IEEE Xplore. Restrictions apply.
 2100 IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS, VOL. 23, NO. 5, SEPTEMBER 2019

of researchers. Second, ROI segmentation is affected by the the 3D patches of each hippocampus to learn the visual features
individual differences and subjective factors of experts. Third, for image classification. Third, the shape analysis is performed
the morphological abnormalities caused by the brain disorders on the hippocampus mask using SPHARM-PDM tool to ex-
may involve multiple ROIs or part of the pre-defined ROIs, so the tract the shape descriptors, followed by a Multi-Layer Percep-
performance may not be stable. To avoid the multiple process- tron (MLP) for classification task. Finally, the outputs of 3D
ing steps in feature extraction, the residual and plain 3D CNN DenseNets and MLP are combined for disease classification.
architectures have been investigated for automatic feature gen- Compared to the existing methods, our proposed method has
eration and more straightforward analysis of whole brain MRI following improvements: 1) The deep DenseNet can jointly
scans [11]. The landmark-based deep learning framework was learn the features of image patches and disease classifier, while
proposed to automatically extract patch-based representation of the shape features capture the global shapes of hippocampus
MRI for AD diagnosis [6], [9]. mask. These two types of features can provide complementary
Among all parcellated ROIs in brain, hippocampus is proven information and thus are combined to enhance disease classifi-
to be a vitally important region for AD. There are many studies cation. 2) Training a DenseNet requires a large image set, which
in the literature to extract features from hippocampus region is not applicable for AD and MCI diagnosis. Instead of train-
for AD diagnosis using structural MRI [12]–[17]. Most exist- ing a deep DenseNet with the whole brain image, we build a
ing methods are based on the shape and volume analysis of DenseNet on the local patches from hippocampus, which can
bilateral hippocampi. Hippocampal atrophy is one of the most reduce the number of parameters and alleviate the problem of
validated, easily accessible and widely used biomarkers of AD. small image set. 3) No tissue segmentation and nonlinear reg-
The reduction of hippocampus volume in brain structure can istration are required in image processing, which can simplify
be visible and measured using structural MRI. Thus, there are the diagnosis procedure and save computation costs. The rest
some studies proposed to compute the hippocampal volumes of this paper is organized as follows. In Section II, we present
using MRI for AD diagnosis [13], [15], [17]. In [18], a fully au- the image set used in this study and the proposed method in
tomatic method using probabilistic and anatomical priors was details. In Section III, we provide the experimental results and
proposed for hippocampus segmentation and the hippocampus discussion. A conclusion is given in Section IV.
volumes were computed for AD classification. In addition, a fast
multi-atlas segmentation method was proposed to automatically
measure hippocampal volume for discriminating AD or MCI II. PROPOSED METHOD
from NC [13], [17]. However, volumetric analysis only assesses In this section, we presented the proposed classification al-
global hippocampal changes and suffers from the variety of gorithm for hippocampus analysis. Our method works on the
hippocampus volumes among different individuals. Therefore, T1-weighted MR brain images which are widely available, non-
shape analysis was used to capture the hippocampus morphol- invasive and often used as the first biomarker in AD diagnosis.
ogy for AD diagnosis [12], [14], [19], [20]. It can unveil the local Fig. 1 shows the flowchart of our proposed algorithm, which
atrophy of hippocampus and is more sensitive in the early stage consists of four main steps: hippocampus segmentation and
of AD. In [12], the hippocampus shape analysis was performed patch extraction, construction of 3D DenseNet models, MLP
by mapping the segmentation mask into spherical harmonic based shape analysis and final ensemble classification. More
shape description (SPHARM-PDM) and then the original sur- details of these steps were provided in the following subsec-
face locations were described with sets of coefficients weighting tions.
spherical harmonic basis functions.
However, there are still some limitations on the existing hip-
pocampus analysis methods. First, both volumetric and shape A. The Image Set and Processing
analysis of hippocampus depend on the accurate segmentation The structural MR brain images studied in this work were
of brain ROIs. It is not an easy task to achieve the accurate obtained from ADNI database, which can be freely down-
ROI segmentation. Second, hippocampus is not sufficient for loaded from the website (www.loni.ucla.edu/ADNI). In 2003,
discrimination of MCI from NC. Other regions adjacent to hip- the ADNI was launched as a $60 million, 5-year public–private
pocampus such as parahippocampus and amygdala are also in- partnership to test whether serial MRI, Positron Emission To-
volved in AD. Third, the visual features of MR images derived mography (PET), other biological markers, and clinical and
from the hippocampal region can be of great help for AD diag- neuropsychological assessment can be combined to measure
nosis [16], [21]. the progression of MCI and early AD. Detailed information
To overcome the above limitations, this paper proposes a about data acquisition is available at the ADNI website.
new method for hippocampus analysis based on combination of In this study, we use the T1-weighted MR images acquired
3D Densely Connected Convolutional Networks (3D DenseNet) with 1.5T scanners from 811 subjects as shown in Table I.
and traditional shape features for AD diagnosis, which inte- The MR images are first preprocessed by using a nonpara-
grates the multi-level and multi-type features for improving the metric nonuniform intensity normalization (N3) algorithm [22]
disease classification. First, hippocampus segmentation is per- to correct the non-uniform intensity. After correction, skull-
formed to generate the hippocampus masks and two 3D image stripping is performed by the method [23] followed by cere-
patches centered on the centroids of bilateral hippocampi are bellum removal with Freesurfer software. We manually check
extracted for each MRI. Second, motivated by the success of the skull-stripped images to ensure clean and dura removal.
Convolutional Neural Networks, a 3D DenseNet is trained with The rigid registration is used to align the MR images to a
Authorized licensed use limited to: Netaji Subhas University of Technology New Delhi. Downloaded on February 12,2024 at 10:28:43 UTC from IEEE Xplore. Restrictions apply.
 CUI AND LIU: HIPPOCAMPUS ANALYSIS BY COMBINATION OF 3-D DENSENET AND SHAPES FOR ALZHEIMER’S DISEASE DIAGNOSIS 2101

Fig. 1. The flowchart of our proposed algorithm consisting of four main steps: hippocampus segmentation and patch extraction, construction of
3D DenseNet models, MLP based shape analysis, and final ensemble classification for AD and MCI diagnosis.

TABLE I gyrus. The patches from left and right hippocampus have
DEMOGRAPHIC CHARACTERISTICS OF THE STUDIED SUBJECTS FROM ADNI
DATABASE (THE VALUES ARE DENOTED AS MEAN ± STANDARD DEVIATION)
overlapping of 6–16 voxels.

C. The Deep 3D DenseNet Model

After extraction of patches from hippocampus region, we
learn features from image patches for classification. Convolu-
tional neural networks (CNNs), which alternatively stacks con-
volutional and pooling layers followed by fully connected and
softmax layers, were investigated to learn features for image
classification. To enhance the representation power, more layers
are added to build the deep CNNs, which increases the informa-
template space with FMRIB Software Library (FSL) 5.0 from tion loss when the input information passes through many layers
https://fsl.fmrib.ox.ac.uk/. to reach the end of network. Thus, the Densely Connected Con-
volutional Networks (DenseNet) was proposed to connect each
B. Hippocampus Segmentation and Patch Extraction layer to every other layer in a feed-forward fashion, which in-
creased direct connections between the low and high layers [24].
Hippocampus is often considered as one of the first affected
In the DenseNet, the layers from different levels are densely con-
brain regions in AD. There are two hippocampi (left and
nected to improve information flow between layers. Compared
right) in brain. For hippocampus analysis, we first segment the
to deep CNN, DenseNets can alleviate the vanishing-gradient
hippocampus from other regions using FSL 5.0 and a binary
problem because there is a direct connection from the low to
mask is generated for each hippocampus. We also manually
high layers. In addition, feature propagation is strengthened to
check the results to ensure the correct segmentation. For the
reuse the low-level features with less information loss and the
failures, we remove the subjects. The MRIs of 834 subjects
number of parameters is reduced.
were initially downloaded from ADNI to test our algorithm.
In this work, we propose to construct a 3D DenseNet for
There are 23 failed subjects of 7 AD, 4 pMCI, 6 sMCI and 6
learning the features from 3D image patches for AD diagnosis.
NC. After removing the failed subjects, the final sample size
Fig. 2 shows the structure of 3D DenseNet model consisting
is 811 including 192 AD, 165 pMCI, 231 sMCI and 223 NC
of a convolutional layer, 4 dense blocks, 3 transition layers, an
subjects. Then, we calculate the centroid of each hippocampus
average pooling layer and a softmax layer. First, convolutional
based on the mask. Finally, a 3D patch of fixed size is extracted
layer is added after the input layer with a stride of 2 × 2 × 2,
from the MR images to be centered on hippocampus centroid as
followed by 4 dense blocks. The dense block uses a dense con-
the inputs of DenseNets. To achieve the robustness of location
nectivity through which the lth layer receives the output feature
variance, each hippocampus centroid is shifted by ±2 voxels in
maps of all preceding layers as [24]:
x, y and z coordinates to extract more patches for training. In the
test phase, no shift is for patch extraction. Since some regions xl = Hl ([x0 , x1 , . . . , xl − 1]) (1)
surrounding the hippocampus are also useful for AD diagnosis,
the patch size is so large to cover the whole hippocampus and where [x0 , x1 , . . . , xl−1 ] is the concatenation of the feature
its surrounding regions such as amygdala and parahippocampal maps from all previous layers into a single tensor, and Hl denotes
Authorized licensed use limited to: Netaji Subhas University of Technology New Delhi. Downloaded on February 12,2024 at 10:28:43 UTC from IEEE Xplore. Restrictions apply.
 2102 IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS, VOL. 23, NO. 5, SEPTEMBER 2019

TABLE II
THE ARCHITECTURE OF THE 3D DENSENET

Fig. 2. The structure of 3D DenseNet consisting of a convolutional

layer, four dense blocks, three transition layers, an average pooling and
a softmax layers.

a composite nonlinear transformation function of four consecu-

tive operations: batch normalization, leaky rectified linear units,
3 × 3 × 3 convolution and dropout. In each dense block, every
dense layer receives the feature maps of all previous dense layers
by shortcut connection. The dense layer includes one 1 × 1 × 1
and one 3 × 3 × 3 convolutional layers, two batch normalization
layers and two activation layers. Dense block 1 and 2 consist
of three dense layers, while the other two dense blocks contain
two dense layers. Between two dense blocks, a transition layer
is set to achieve dimension reduction of the feature maps. It con-
sists of five consecutive operations: batch normalization, leaky
rectified linear units, a 1 × 1 × 1 convolution, dropout and a
3 × 3 × 3 convolution with a stride of 2 × 2 × 2. Following
the last dense block, an average pooling and a softmax classifier
are appended to reduce the feature dimension and make the
classification. The subject labels are used through back-
propagation for updating the weights of DenseNet and learning
the relevant features. All layers receive the direct supervision
from the loss function through shortcut connections. The 3D
DenseNets for the left and right hippocampi have the same
structure but they are trained with different patches. Table II
lists the output size and the parameters of each network layer in
the 3D DenseNet model. Fig. 3. The flowchart of hippocampal shape analysis, where SPHARM
coefficients are computed with SPHARM-MAT, followed by MLP model
for classification task.
D. Shape Analysis
It was proven that the AD progression has a strong correla- on the sphere. Finally, surface alignment is conducted for group
tion with the hippocampal atrophy. Although the 3D DenseNet analysis across different surface samples.
is effective to learn the visual features of MR images, the hip- After the above operations, we obtain the SPHARM coeffi-
pocampal shapes are also important features to differentiate the cients for each hippocampus mask, which are used as the shape
hippocampal atrophies. Thus, we perform the shape analysis on features to train a Multi-Layer Perceptron (MLP) model with
the hippocampal mask for classification. Fig. 3 illustrates the three full connected layers and one softmax layer. Similar to the
flow chart of the hippocampal shape analysis which consists of DenseNet models mentioned above, two MLP models sharing
shape feature extraction and classification. For feature extrac- the same structure are constructed for the bilateral hippocampi.
tion, the tool of SPHARM-MAT [22] (SPHARM Modeling and In total, the MLP model contains ten layers, including the in-
Analysis Toolkit) on MATLAB 2012b is used for shape analy- put layer, two batch normalization layers, two ReLU activation,
sis in this work. It consists of four processing steps: topology two dropout, two full-connected layers and one softmax layer.
fix, spherical parameterization, SPHARM expansion and sur- It performs well with a simple structure and less number of
face alignment. First, the voxel-wise surfaces of all binary hip- parameters for shape analysis.
pocampus masks are fixed to have a spherical topology. Second,
we create spherical parameterization for surface meshes and
E. Final Classification
generate a continuous mapping from the hippocampus surface
to the surface of a unit. Third, SPHARM expansion is performed In the above sections, the DenseNets capture the visual fea-
on the surface into a complete set of spherical harmonic basis tures while the shape analysis by MLP is effective to capture the
functions, which are essentially Fourier basis functions defined hippocampal shape features for classification. In this work, we
Authorized licensed use limited to: Netaji Subhas University of Technology New Delhi. Downloaded on February 12,2024 at 10:28:43 UTC from IEEE Xplore. Restrictions apply.
 CUI AND LIU: HIPPOCAMPUS ANALYSIS BY COMBINATION OF 3-D DENSENET AND SHAPES FOR ALZHEIMER’S DISEASE DIAGNOSIS 2103

TABLE III
COMPARISON OF DIFFERENT PATCH SIZES FOR CLASSIFICATIONS OF AD VS. NC, PMCI VS. SMCI, AND MCI VS. NC

investigate two strategies to combine these networks to improve Tensorflow backend. For training the DenseNets, the initial
classification. First, we take the weighted sum of prediction weights for whole network is uniform, and Adam optimizer is
scores from 4 separately training networks. Considering that adopted with a low learning rate of 1 × 10−4. The networks are
DenseNet can capture more discriminative features than shape stable after iteration of 200 epochs. The batch sizes are set to 64.
analysis, the weights are set to 0.2 and 0.3 for each MLP and ReLU activation is used for each neuron of DenseNet. All the ex-
DenseNet output scores, respectively (all weights are summed periments were conducted in the environment of Ubuntu14.04-
to 1). Second, instead of taking the sum of output scores, we x64/GPU of NVIDIA GeForce GTX 1080 Ti.
use two full connected layers to combine the 4 networks which To avoid the overfitting problem, we used three techniques
are initially trained individually and finely tuned by end-to-end in building the DenseNet model. First, we shift the centroid of
training. We have implemented and compared these two meth- each hippocampus by ±2 voxels in x, y and z coordinates to
ods and found that the combined network by full connection extract more patches from training subjects to greatly augment
performs slightly better than the weighted sum. The combined the training data. Second, we constructed a simplified version
network is an end-to-end training to optimize the parameters of of DenseNet with the numbers of dense layers in the dense
whole network. blocks set to [2, 2, 2, 2], which is much smaller than those of
In summary, our proposed method combines the deep learn- [6], [12], [24], [16] in the original DenseNet proposed by Huang
ing model and MLP based shape analysis to make full use of et al. [24]. This reduces the depth of network structure and helps
the shape and local visual features of MR brain images for to avoid overfitting. Third, a dropout layer was added in each
AD diagnosis. Specially, we learn the visual features using 3D convolutional layer of DenseNet model to avoid overfitting.
DenseNets and extract the shape features with SPHARM-MAT. To evaluate the performance, we used a 5-fold cross-
These two methods are further combined to make the final clas- validation strategy to train and test the proposed method. Each
sification. Compared to the existing methods for classification time, 1 fold of the data set was used for testing, while the other 4
of MR brain images, our proposed method has the following folds was further split into training and validation parts. The val-
advantages. First, both the local and global hippocampal infor- idation part is used to adjust the training process for the optimal
mation are used to enhance the classification. Second, instead parameters. Four measures, i.e., classification accuracy (ACC),
of using the whole image, the proposed method extracted the sensitivity (SEN), specificity (SPE), receiver operating charac-
patches from hippocampal regions to train 3D DenseNets. This teristic (ROC) curve and the area under ROC curve (AUC), are
can avoid building very deep DenseNets and the training data computed for evaluation. ACC is computed as the proportion
can be augmented by shifting patches. Third, no tissue segmen- of correctly classified subjects among the whole population.
tation and nonlinear registration are required in image process- SEN is the proportion of correctly classified positive samples
ing, which can simplify the diagnosis procedure and save the (AD/MCI subjects) among the total number of positive samples.
computation costs. SPE is the proportion of correctly classified negative samples
(NC subjects) among the total number of negative samples. The
III. EXPERIMENTAL RESULTS ROC curve is generated by plotting the true positive rate (TPR)
against the false positive rate (FPR) at various thresholds on the
In this section, we first introduce the datasets and implementa- class prediction scores.
tion of our method. Then, we present the extensive experiments
to test our method on the classifications of AD vs. NC, pMCI vs.
sMCI and MCI vs. NC. Finally, we compare our method with B. Test the Effects of Patch Size
other methods and give the discussion.
The first experiment is to test the effects of different patch
sizes on the classification performance of DenseNet. Large patch
A. Datasets and Implementation
covers more neighboring information of hippocampus. In the
The proposed method is tested on the T1-weighted MR brain experiment, we extracted the patches by gradually increasing its
images from ADNI database. The image preprocessing was size from 57 × 45 × 53, 62 × 48 × 58 to 68 × 54 × 64. Table III
conducted as illustrated in Section II. The proposed method is compares the classification performances of three different patch
tested on classifications of AD vs. NC, pMCI vs. sMCI and MCI sizes for AD vs. NC, pMCI vs. sMCI and MCI vs. NC. From
vs. NC. The MR images were taken from the baseline visits of these results, we can see the classification performances are
811 ADNI participants for evaluation. The proposed method improved by increasing the patch size from 57 × 45 × 53 to
is implemented with python 2.7.9 and Keras library on the 62 × 48 × 58. But they cannot be improved by further increasing
Authorized licensed use limited to: Netaji Subhas University of Technology New Delhi. Downloaded on February 12,2024 at 10:28:43 UTC from IEEE Xplore. Restrictions apply.
 2104 IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS, VOL. 23, NO. 5, SEPTEMBER 2019

TABLE IV
COMPARISON OF DIFFERENT DEEP LEARNING MODELS FOR CLASSIFICATIONS OF AD VS. NC, PMCI VS. SMCI, AND MCI VS. NC

TABLE V
COMPARISON OF DIFFERENT METHODS FOR CLASSIFICATIONS OF AD VS. NC, PMCI VS. SMCI, AND MCI VS. NC

the patch size. Thus, the 62 × 48 × 58 patch is used to train the D. Comparison With Other Methods
DenseNets in our following experiments.
The third experiment is to compare our proposed method to
other methods that are also based on T1-weighted structural
C. Test the Effectiveness of 3D DenseNet and
MRI data of ADNI in the literature. Specifically, our method
Shape Analysis
is compared to three conventional features such as hippocam-
The second experiment is in depth analysis of all stages in the pal volume, GM volumes of ROIs and voxel-wise features in
proposed method. We first compare classification performances [4], [18], [27]. In [18], a fully automatic method was proposed
using the DenseNet with different number of dense blocks and for hippocampus segmentation to compute the hippocampal vol-
two well-known CNN models: LeNet [25] and VGGNet [26], umes. The GM volumes of 93 ROIs by parcellation of the whole
as shown in Table IV. The LeNet and VGGNet are implemented brain were computed for classification [4]. The voxel-wise GM
with the released codes by replacing the 2D convolution with density maps were used for classification [27]. For fair compar-
3D. We can see VGGNet performs better than LeNet and the ison, we implemented these methods with our best efforts on
performance of DenseNet is gradually improved by increasing the same training and test data sets.
the number of dense blocks till 4. The DenseNet model with 4 In our experiments, the same image processing and classifier,
dense blocks achieves the best performance because of its reuse i.e., softmax classifier, were used for all methods while feature
of multi-level rich features. extractions were different. To extract the conventional features,
We further test the classification performances of 3D tissue segmentation and rigid registration were performed using
DenseNet, shape analysis and two different combination meth- the FAST in the FSL package [28] and HAMMER [29], respec-
ods, separately. One combination method is by weighted sum of tively. The image is mapped into 93 ROIs to compute their GM
four output prediction scores from MLP based shape analysis volumes as the features. To extract the voxel-wise features, the
and DenseNets. Another method appended the full connected warped tissue volumes by HAMMER [29] reflected the spatial
layers to combine the features extracted from shape analysis and density of tissues in an original brain and were downsampled
DenseNets for classification. The classification performances by 4 and t-test is used to select the 10000 most discriminative
of these methods for AD and MCI diagnosis are compared features for classification. Table VI shows the comparison of
in Table V. We can see that the combination method by full classification performances by our proposed method and other
connection performs slightly better than the scores summation. three feature extractions with their ROC curves shown in Fig. 5.
Fig. 4(a), (b) and (c) illustrate the ROC curves for classifications It is worthy to note that this evaluation is based on different fea-
of AD vs. NC, pMCI vs. sMCI and MCI vs. NC, respectively. ture extractions, not the design of classifier, so the results may
The results show that the DenseNet performs better than the be different from those reported in literature.
shape analysis, especially for classification of AD vs. NC. The In addition, we compare our results with the reported results
classification performances are further improved by combina- by the deep learning methods on ADNI in Table VII. We can
tion of DenseNet and shape analysis with the full connection. see that our method performs better than other methods for

Authorized licensed use limited to: Netaji Subhas University of Technology New Delhi. Downloaded on February 12,2024 at 10:28:43 UTC from IEEE Xplore. Restrictions apply.
 CUI AND LIU: HIPPOCAMPUS ANALYSIS BY COMBINATION OF 3-D DENSENET AND SHAPES FOR ALZHEIMER’S DISEASE DIAGNOSIS 2105

Fig. 4. Comparison of ROCs by shape analysis, DenseNet and proposed combination method for (a) AD VS. NC, (b) pMCI vs. sMCI, and
(c) MCI vs. NC.

TABLE VI
COMPARISON OF OUR METHOD WITH OTHER METHODS IN THE LITERATURE FOR CLASSIFICATION OF AD VS. NC, PMCI VS. SMCI, AND MCI VS. NC

Fig. 5. Comparison of ROCs of our proposed method with other four methods for (a) AD vs. NC, (b) pMCI vs. sMCI, and (c) MCI vs. NC,
respectively.

TABLE VII
COMPARISON OF OUR RESULTS WITH THE RESULTS OF OTHER DEEP LEARNING METHODS FOR CLASSIFICATION OF AD VS. NC, PMCI VS. SMCI,
AND MCI VS. NC

classification of AD vs. NC. Specifically, the whole brain im- performance for classification of other groups. A landmark-
ages were used to train the residual and plain 3D CNNs for AD based deep learning framework was proposed for AD diagnosis
and MCI diagnosis [11]. Our method performs much better than [6]. This framework learns an end-to-end classifier to com-
this method for all classifications. The whole brain is parcellated bine both the local and global features from landmarks. The
into 116 regions with each trained on a deep belief network result comparison further validates the efficacy of our proposed
for ensemble classification using MRI and PET images [10]. method for AD and MCI diagnosis. It is worth noting that the
Our method using only MRI achieves better results than this differences of these results may be caused not only by the deep
ensemble methodw for AD classification, but has lower learning methods and but also by different ADNI subjects.

Authorized licensed use limited to: Netaji Subhas University of Technology New Delhi. Downloaded on February 12,2024 at 10:28:43 UTC from IEEE Xplore. Restrictions apply.
 2106 IEEE JOURNAL OF BIOMEDICAL AND HEALTH INFORMATICS, VOL. 23, NO. 5, SEPTEMBER 2019

Fig. 6. The convergence of loss functions for training the classification models of (a) AD vs. NC, (b) pMCI vs. sMCI, and (c) MCI vs. NC.

E. Discussion REFERENCES
From the above results, we can see that the classification per- [1] M. Prince, A. Wimo, M. Guerchet, G. C. Ali, Y. T. Wu, and M. Prina,
“World Alzheimer report 2015. The global impact of dementia. An anal-
formance is low by using the hippocampal volumes because ysis of prevalence, incidence, cost and trends,” Alzheimer’s Dis. Int.,
they have limited information for AD diagnosis. Our proposed London, U.K., Tech. Rep., 2015.
method achieves a very competitive classification accuracy of [2] J. CR, Jr. et al., “Introduction to the recommendations from the National
Institute on Aging—Alzheimer’s association workgroups on diagnostic
92.29% for classification of AD vs. NC, which is higher than the guidelines for Alzheimer’s disease,” Alzheimers Dementia, J. Alzheimers
ROI-based method (86.51%) and voxel-based method (86.02%). Assoc., vol. 7, pp. 257–262, 2011.
Our method also achieves the accuracy of 75.00% for classifi- [3] H. I. Suk, S. W. Lee, and D. Shen, “Latent feature representation
with stacked auto-encoder for AD/MCI diagnosis,” Brain Struct. Funct.,
cation of pMCI vs. sMCI, higher than the ROI-based method vol. 220, pp. 841–859, 2015.
(73.23%) and voxel-based method (72.98%). [4] D. Zhang, Y. Wang, L. Zhou, H. Yuan, and D. Shen, “Multimodal classifi-
Compared to the ROI methods that use multiple brain regions cation of Alzheimer’s disease and mild cognitive impairment,” Neuroim-
age, vol. 55, pp. 856–867, 2011.
[10], [30], the proposed method focuses on the hippocampus [5] P. Cao et al., “Nonlinearity-aware based dimensionality reduction and
with the following advantages. First, the proposed method can over-sampling for AD/MCI classification from MRI measures,” Comput.
make full use of the visual and shape features of hippocampus, Biol. Med., vol. 91, pp. 21–37, 2017.
[6] M. Liu, J. Zhang, E. Adeli, and D. Shen, “Landmark-based deep multi-
while the ROI methods just use the hippocampal volume which instance learning for brain disease diagnosis,” Med. Image Anal., vol. 43,
is not enough to represent the hippocampus. Second, different pp. 157–168, 2017.
from the ROI methods, no tissue segmentation and nonlinear [7] J. Zhang, M. Liu, and D. Shen, “Detecting anatomical landmarks from
limited medical imaging data using two-stage task-oriented deep neural
registration are required in the proposed method, which can networks,” IEEE Trans. Image Process., vol. 26, no. 10, pp. 4753–4764,
reduce the computation costs and fast the process. The disad- Oct. 2017.
vantage is that the regions far from the hippocampus will not be [8] M. Liu, J. Zhang, E. Adeli, and D. Shen, Deep Multi-Task Multi-Channel
Learning for Joint Classification and Regression of Brain Status. New
considered in the proposed method. York, NY, USA: Springer, 2017.
As for the computational complexity, the proposed method [9] M. Liu, J. Zhang, D. Nie, P. T. Yap, and D. Shen, “Anatomical landmark
includes the offline training and online test stages. Fig. 6(a), (b) based deep feature representation for MR images in brain disease diag-
nosis,” IEEE J. Biomed. Health Informat., vol. 22, no. 5, pp. 1476–1485,
and (c) show the convergence of the loss functions for training Sep. 2018.
the classification models of AD vs. NC, pMCI vs. sMCI and [10] A. Ortiz, J. Munilla, J. M. Górriz, and J. Ramı́rez, “Ensembles of deep
MCI vs. NC, respectively. The results show that the AD vs. NC learning architectures for the early diagnosis of Alzheimer’s disease,” Int.
J. Neural Syst., vol. 26, 2016, Art. no. 1650025.
model converges after 150 epochs which is faster than others. [11] S. Korolev, A. Safiullin, M. Belyaev, and Y. Dodonova, “Residual and
Other models converge after 200 epochs. In the test stage, it plain convolutional neural networks for 3D brain MRI classification,” in
takes about 0.058 s on average for a subject, which demonstrate Proc. IEEE Int. Symp. Biomed. Imag., 2017, pp. 835–838.
[12] O. Lindberg et al., “Hippocampal shape analysis in Alzheimer’s dis-
the usefulness of the proposed method in real application. ease and frontotemporal lobar degeneration subtypes,” J. Alzheimers Dis.,
vol. 30, pp. 355–365, 2012.
IV. CONCLUSION [13] J. Lötjönen et al., “Fast and robust extraction of hippocampus from MR
images for diagnostics of Alzheimer’s disease,” Neuroimage, vol. 56,
This paper has proposed a hippocampus analysis method pp. 185–196, 2011.
[14] M. F. Beg, P. R. Raamana, S. Barbieri, and L. Wang, “Comparison
based on combination of 3D DenseNets and shape analysis of four shape features for detecting hippocampal shape changes in
for AD and MCI diagnosis using MR images. The DenseNets early Alzheimer’s,” Statist. Methods Med. Res., vol. 22, pp. 439–462,
are constructed to learn the visual features of hippocampus re- 2013.
[15] K. K. Leung et al., “Automated cross-sectional and longitudinal hippocam-
gion. The final image classification is made by combination of pal volume measurement in mild cognitive impairment and Alzheimer’s
the 3D DenseNets and shape analysis. Experimental results and disease,” Neuroimage, vol. 51, pp. 1345–1359, 2010.
comparison on T1-weighted structural MR images from ADNI [16] L. Xin, X. Hong, Z. Zhen, and L. Tong, “3D texture analysis of hippocam-
pus based on MR images in patients with Alzheimer disease and mild
database demonstrate that the proposed method have promising cognitive impairment,” J. Beijing Univ. Technol., vol. 38, pp. 942–948,
performances for AD and MCI diagnosis. 2012.

Authorized licensed use limited to: Netaji Subhas University of Technology New Delhi. Downloaded on February 12,2024 at 10:28:43 UTC from IEEE Xplore. Restrictions apply.
 CUI AND LIU: HIPPOCAMPUS ANALYSIS BY COMBINATION OF 3-D DENSENET AND SHAPES FOR ALZHEIMER’S DISEASE DIAGNOSIS 2107

[17] C. Platero and M. C. Tobar, “A fast approach for hippocampal segmenta- [24] G. Huang, Z. Liu, and K. Q. Weinberger, “Densely connected convolu-
tion from T1-MRI for predicting progression in Alzheimer’s disease from tional networks,” in Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern
elderly controls,” J. Neurosci. Methods, vol. 270, pp. 61–75, 2016. Recog., pp. 2261–2269, 2017.
[18] M. Chupin et al., “Fully automatic hippocampus segmentation and classi- [25] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, “Gradient-based learning
fication in Alzheimer’s disease and mild cognitive impairment applied on applied to document recognition,” Proc. IEEE, vol. 86, no. 11, pp. 2278–
data from ADNI,” Hippocampus, vol. 19, pp. 579–587, 2009. 2324, Nov. 1998.
[19] X. Tang, D. Holland, A. M. Dale, L. Younes, and M. I. Miller, “Baseline [26] K. Simonyan and A. Zisserman, “Very deep convolutional networks for
shape diffeomorphometry patterns of subcortical and ventricular structures large-scale image recognition,” CoRR, vol. abs/1409.1556, 2014.
in predicting conversion of mild cognitive impairment to Alzheimer’s [27] R. Li et al., “Deep learning based imaging data completion for improved
disease,” J. Alzheimers Dis., vol. 44, pp. 599–611, 2015. brain disease diagnosis,” Med. Image Comput. Comput. Assist. Interv.,
[20] H. C. Achterberg et al., “Hippocampal shape is predictive for the develop- vol. 17, pp. 305–312, 2014.
ment of dementia in a normal, elderly population,” Hum. Brain Mapping, [28] Y. Zhang, M. Brady, and S. Smith, “Segmentation of brain MR im-
vol. 35, pp. 2359–2371, 2014. ages through a hidden Markov random field model and the expectation-
[21] O. B. Ahmed, J. Benois-Pineau, M. Allard, C. B. Amar, and G. Catheline, maximization algorithm,” IEEE Trans. Med. Imag., vol. 20, no. 1, pp. 45–
“Classification of Alzheimer’s disease subjects from MRI using hippocam- 57, Jan. 2001.
pal visual features,” Multimedia Tools Appl., vol. 74, pp. 1249–1266, 2015. [29] D. Shen and C. Davatzikos, “HAMMER: Hierarchical attribute matching
[22] J. G. Sled, A. P. Zijdenbos, and A. C. Evans, “A nonparametric method mechanism for elastic registration,” IEEE Trans. Med. Imag., vol. 21,
for automatic correction of intensity nonuniformity in MRI data,” IEEE no. 11, pp. 1421–1439, Nov. 2002.
Trans. Med. Imag., vol. 17, no. 1, pp. 87–97, Feb. 1998. [30] M. Liu, D. Zhang, and D. Shen, “Relationship induced multi-template
[23] Y. Wang, J. Nie, P. T. Yap, F. Shi, L. Guo, and D. Shen, “Robust learning for diagnosis of Alzheimer’s disease and mild cognitive im-
deformable-surface-based skull-stripping for large-scale studies,” Med. pairment,” IEEE Trans. Med. Imag., vol. 35, no. 6, pp. 1463–1474,
Image Comput. Comput. Assist. Interv., vol. 14, pp. 635–642, 2011. Jun. 2016.

Authorized licensed use limited to: Netaji Subhas University of Technology New Delhi. Downloaded on February 12,2024 at 10:28:43 UTC from IEEE Xplore. Restrictions apply.