Received August 5, 2020, accepted August 21, 2020, date of publication August 25, 2020, date of current version

September 4, 2020.
Digital Object Identifier 10.1109/ACCESS.2020.3019327

Classification of Breast Cancer Histopathological

Images Using Discriminative Patches Screened
by Generative Adversarial Networks
RUI MAN 1, PING YANG 1, AND BOWEN XU 2
1 Smart City College, Beijing Union University, Beijing 100101, China
2 Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China

Corresponding author: Ping Yang ([email protected])

ABSTRACT Computer-aided diagnosis (CAD) systems of breast cancer histopathological images automated
classification can help reduce the manual observation workload of pathologists. In the classification
of breast cancer histopathology images, due to the small number and high-resolution of the training
samples, the patch-based image classification methods have become very necessary. However, adopting a
patches-based classification method is very challenging, since the patch-level datasets extracted from whole
slide images (WSIs) contain many mislabeled patches. Existing patch-based classification methods have
paid little attention to addressing the mislabeled patches for improving the performance of classification.
To solve this problem, we propose a novel approach, named DenseNet121-AnoGAN, for classifying breast
histopathological images into benign and malignant classes. The proposed approach consists of two major
parts: using an unsupervised anomaly detection with generative adversarial networks (AnoGAN) to screen
mislabeled patches and using densely connected convolutional network (DenseNet) to extract multi-layered
features of the discriminative patches. The performance of the proposed approach is evaluated on the publicly
available BreaKHis dataset using 5-fold cross validation. The proposed DenseNet121-AnoGAN can be better
suited to coarse-grained high-resolution images and achieved satisfactory classification performance in 40X
and 100X images. The best accuracy of 99.13% and the best F1score of 99.38% have been obtained at the
image level for the 40X magnification factor. We have also investigated the performance of AnoGAN on the
other classification networks, including AlexNet, VGG16, VGG19, and ResNet50. Our experiments show
that whether it is at the patient-level accuracy or at the image-level accuracy, the classification networks with
AnoGAN have provided better performance than the classification networks without AnoGAN.

INDEX TERMS Breast cancer histopathological images, densely connected convolutional networks,
discriminative patches, generative adversarial networks, image classification.

I. INTRODUCTION About 42,170 women in U.S. are expected to die in 2020 from
Breast cancer is the top cancer in women, impacting breast cancer [2].
2.1 million women each year, and also causes the greatest Due to the high death rate of breast cancer, women
number of cancer-related deaths among women. Breast are suggested to do regular screenings via mammograms
cancer is a serious disease that can start in almost any organ or and computerized tomography (CT) [3]. If abnormal cells
tissue of the body when abnormal cells grow uncontrollably, are found, biopsy procedure is performed to diagnose the
go beyond their usual boundaries to invade adjoining parts abnormality in breast. Usually, the collected sample is stained
of the body or spread to other organs [1]. According to the with hematoxylin and eosin (H&E). Hematoxylin reacts to
data provided by the American Cancer Society, in 2020 in Deoxyribonucleic Acid (DNA) and it stains purple or blue
U.S., there will be an estimated 276,480 new cases of color to the nuclei, while Eosin reacts to proteins and it stains
invasive breast cancer and 48,530 new cases of non-invasive pink color to other structures [4].
breast cancer expected to be diagnosed in women. Diagnosis from a histopathological image is considered as
the gold standard in diagnosing all kinds of cancer, including
The associate editor coordinating the review of this manuscript and breast cancer [5]–[7]. However, histopathological analysis
approving it for publication was Kumaradevan Punithakumar . is a very time-consuming professional task that depends on

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
155362 VOLUME 8, 2020
R. Man et al.: Classification of Breast Cancer Histopathological Images Using Discriminative Patches Screened

the experience of the pathologist, and the diagnosis can be the number of samples is small and the size of images
influenced by factors such as the pathologist’s fatigue and is large, which makes it difficult or even impossible to
decreased attention [7], [8]. Therefore, there is an urgent train a deep learning model based on CNN. In addition,
need for computer-aided diagnosis (CAD) systems to provide directly resizing the whole histopathology images to the
an objective assessment to pathologists and improve the input size for the deep learning model will lose a host of
diagnostic efficiency [9], [10]. detailed feature information. Consequently, some researchers
With the advancements in medical image processing proposed the patch-based image classification methods to
and deep learning, classification of breast histopathological solve this problem. Spanhol et al. [30] adopted the random
images has become an important area for research [11], [12]. extracting patches strategy and the sliding window strategy
Due to the high-resolution breast cancer histopathological to extract the image patches of the BreakHis dataset. They
images, the exiting traditional machine learning methods and trained AlexNet [31] based on the extracted image patches as
the deep neural network models used to directly analyze input and combined the patch-level classification results with
the whole slide images (WSIs) have caused a very complex three fusion rules for final classification. Araújo et al. [32]
architecture that hard to train [13]. During the past few proposed a convolutional neural network (CNN) architecture,
decades, some researchers proposed the strategies that relied which is designed to extract features from patch-level dataset
on the segmentation of nucleus, and then used the extracted of 512 × 512 pixels. By training the network, images were
handcrafted features to train a classifier [12], [14]–[16]. classified into four classes, normal, benign, in situ carcinoma,
Kowal et al. [14] segmented the nucleus by color-based and invasive carcinoma, and into two classes, carcinoma,
clustering, and George et al. [15] used the circular Hough and non-carcinoma. The image patches extraction strategy
transform to detect the location of the nucleus, then refined enabled CNN to train the WSIs with a certain resolution.
feature-based candidates via watersheds algorithm [17]. Hou et al. [33] proposed a patch-level convolutional neural
These studies allowed to extract features that are usually network (CNN) for high-resolution WSIs classification
related to morphology, topology, and texture. The calculated which has two-level model. The first level (patch-level)
features can then be used to train one or more classifiers. model is an Expectation Maximization (EM) that can
Kowal et al. [14] achieved an accuracy rate of 84%-93% on automatically identify patches for patch-level CNN training,
500 images from 50 patients and George et al. [15] achieved and the second level (image-level) model is multiclass logistic
an accuracy between 72% and 97% on 92 images. In addition regression or support vector machine (SVM). Alom et al. [34]
to the nuclei-related information, Belsare et al. [16] also proposed a method to classify breast cancer histopathology
considered to segment the epithelial layer around the images using the Inception Recurrent Residual Convolutional
cell cavity by using the spatio-color-texture graph, and Neural Network (IRRCNN) model. Random patches were
statistical texture features were used to train the final cropped to create a patch dataset for training and testing the
classifier. Belsare et al. [16] reported the accuracy rates IRRCNN model, then used the Winner Take ALL (WTA)
between 70% and 100% of 70 breast histology H&E method [35] to generate the final classification results.
datasets from 40X magnification level. Spanhol et al. [18] Although the above researches show that patch-based
constructed a public dataset called BreaKHis and explored image classification methods have been widely used in
the effectiveness of six state-of-the-art handcrafted fea- various breast cancer histopathology datasets. Adopting a
tures descriptors, i.e., Local Binary Pattern (LBP) [19], patch-based classification method is very challenging. This
Completes Local Binary Pattern (CLBP) [20], Local Phase is because labeled data is critical to the performance of the
Quantization (LPQ) [21], Gray-Level Co-Occurrence Matrix deep learning approaches. Automated image classification
(GLCM) [22], Parameter-Free Threshold Adjacency Statis- tasks require large amounts of annotated data. Because
tics (PFTAS) [23], and Oriented FAST and Rotated BRIEF of the complexity of breast cancer histopathology images,
(ORB) [24]. Then they made experiment on four different the annotation process is laborious and costly. As only
classifiers and reported the accuracy between 80% and 85%. the image-level label is given in the datasets, the label
The results obtained from different handcrafted features of the whole input histopathological images is assigned to
given above were considered to be relatively acceptable the corresponding generated patches. However, there are
results, but highly unstable. As a matter of fact, the main benign areas in the malignant WSIs, which makes the
limitation of these traditional methods is that the quality patch-level label maybe not consistent with the image-level
of the model depends on the extracted features, however, label, and only a small part of extracted image patches is
obtaining highly representative features is a very complicated correctly labeled. This can result in training with mislabeled
task. Even if we choose the most appropriate descriptor, patches. When the training model receives the incorrect
or combine various descriptors together to improve their label information, the classification performance will be
recognition ability, the results obtained are still relatively low reduced.
and unstable between different magnification levels [25]. To address these mislabeled patches and further improve
Recently, the convolutional neural network (CNN) has the accuracy of classification. We propose a novel
been employed in visual classification system [26]–[29]. approach, named DenseNet121-AnoGAN, for classifying
In the classification of breast cancer histopathology images, histopathological images into benign and malignant classes.

VOLUME 8, 2020 155363

 R. Man et al.: Classification of Breast Cancer Histopathological Images Using Discriminative Patches Screened

FIGURE 1. Overview of the proposed framework used for classification of breast cancer histopathological images.

The proposed approach consists of two major parts: using an The rest of this paper is organized as follows: in Section II,
unsupervised anomaly detection with generative adversarial we give the information about the dataset and describe the
networks (AnoGAN) [36] to screen mislabeled patches proposed method. Section III provides the experiments and
as well as using densely connected convolutional network results. Discussions are shown in Section IV. In Section V,
(DenseNet) [37] to extract multi-layered features of the we summarize the conclusion of this paper.
discriminative patches. The main contributions of our work
II. METHODOLOGY
can be summarized as follow:
As shown in FIGURE 1, the proposed approach mainly
1) We propose a screening patches method based on includes three main steps, described as below.
an unsupervised anomaly detection with generative 1) Pre-processing: To solve the stain variability of
adversarial networks (AnoGAN). We use benign patches BreaKHis dataset, the stain normalization pre-
to train AnoGAN. The data distribution of benign processing of histopathological images is firstly carried
patches can be obtained by AnoGAN and it will generate out. Secondly, to increase the number of training
a fake patch with a probability distribution similar to that samples, we use patch extraction and data augmentation
of the benign patch. By defining the threshold of residual algorithm on benign images and use patch extraction
loss and discrimination loss between the malignant patch algorithm on malignant images.
to be tested and fake patch, this well-trained AnoGAN 2) Screening patches: We use benign patches to train
can yield the high anomaly score of the malignant AnoGAN, which will generate a fake patch G(z) of
patches. However, the anomaly score of mislabeled random sample z with a probability distribution similar
patches in malignant patches is low, which can screen to that of the benign patch. The trained parameters of the
the most discriminative histopathological image patches generator and discriminator are kept fixed. We calculate
and improve the classification performance of the the anomaly score between the malignant patch to be
subsequent network. tested and fake patch. This well-trained AnoGAN can
2) We design a breast cancer histopathological images yield the high anomaly score of the malignant patches
classification method based on DenseNet121. We note and yield the low anomaly score of mislabeled patches
that the presented research rarely involves state-of- in malignant patches. The patches labeled correctly in
the-art network architecture, e.g. DenseNet. DenseNet malignant are processed with data augmentation.
achieves multi-scale feature extraction by integrating 3) Classification: We use the discriminative patches to train
convolutional neural networks into dense blocks. DenseNet121. During testing, 100 random patches with
3) Experiments were conducted on the BreaKHis dataset the size of 224 × 224 pixels are cropped from each
using 5-fold cross validation. The results demonstrate image in the testing set. These patches are passed to the
that the proposed approach for breast cancer histopathol- well-trained DenseNet121, and we use majority voting
ogy image classification has an excellent performance for obtaining the final image label from the individual
in both image-level and patient-level classification. The patch classifications.
best accuracy of 99.13% and the best F1score of 99.38% In this section, we introduce the detail of the main tech-
have been obtained at the image level for the 40X nologies for the classification of breast cancer histopatholog-
magnification factor. ical images used in the overall framework.

155364 VOLUME 8, 2020

R. Man et al.: Classification of Breast Cancer Histopathological Images Using Discriminative Patches Screened

FIGURE 2. H&E stained images from BreaKHis, (A): Adenosis, (B): Fibroadenoma, (C): Phyllodes Tumor, (D): Tabular adenoma, (E): ductal
carcinoma, (F): lobular carcinoma, (G): Mucinous carcinoma, (H): Papillary carcinoma.

A. DATASET TABLE 1. Image and patient distribution among the main categories and
each sub-category.
The dataset used in this work is BreaKHis, the latest public
breast cancer histopathological images dataset, which was
collected through a clinical study in 2014. During this period,
all patients referred to the P&D Laboratory (Brazil) with a
clinical indication of breast cancer were invited to participate
this study [18]. The institutional review board approved
the study and all patients signed written informed consent.
All data were anonymized. Samples were generated from
breast tissue biopsy slides and stained with hematoxylin
and eosin (H&E). The samples were collected by a surgical
open biopsy (SOB), prepared for histological research and
labeled by pathologists of the P&D laboratory. Each case
was diagnosed by an experienced pathologist and confirmed
by immunohistochemical analysis and other complementary
exams [38].
To date, BreaKHis dataset is composed of 7909 histopatho-
logical biopsy images collected from 82 patients. Images
were acquired in three-channel RGB color space, with a
dimension of 700 × 460 using four magnification factors
(40X, 100X, 200X, and 400X). Each images is labeled as
either benign or malignant categories, and also distributed
into eight sub-categories: Adenosis (A), Fibroadenoma of the digital scanners, the material and manufacturing
(F), Phyllodes Tumor (PT), and Tubular Adenoma (TA) technology of the staining supplier, and the different staining
for benign images and Ductal Carcinoma (DC), Lobular protocol in the different labs, it may cause large color
Carcinoma (LC), Mucinous Carcinoma (MC) and Papillary differences in the histopathological images. Therefore, stain
Carcinoma (PC) for malignant ones. The distribution of normalization is a fundamental and necessary step in the
BreakHis images and patients into four magnification levels pre-processing of H&E stained breast cancer histopathology
for both main tumor categories and each sub-category is images.
provided in TABLE 1. FIGURE 2 shows samples from eight Many methods have been proposed for stain normalization
sub-categories breast tumors in 40X magnification factor. [39]–[41]. In this paper, we use a stain normalization
method proposed by Vahadane et al. [41] on BreaKHis
dataset. This method adopts a novel structure-preserving
B. STAIN NORMALIZATION PRE-PROCESSING color normalization (SPCN) scheme. It transforms the stain
A deep learning-based method for the classification of breast separation problem into a non-negative matrix factorization
cancer histopathology images, which relies on training set to (NMF) [42] to which we add a sparseness constraint, which
capture a wide range of changes to distinguish the differences is called sparse non-negative matrix factorization (SNMF).
between intra-class and inter-class. Due to the color response One advantage of this method is that the color basis is

VOLUME 8, 2020 155365

 R. Man et al.: Classification of Breast Cancer Histopathological Images Using Discriminative Patches Screened

patches generated from per image is defined by Equation (1):

 nx
P 

 i 
 i=1 
× α ,
 n
Nj =  (1)

 xj 

FIGURE 3. H&E stained images normalization, (A): The target image, (B): where Nj is the number of patches generated from per image
original image, (C): image after stain normalization.
in the jth category, xi refers the number of the ith category,
xj refers the number of the jth category, and n is the number
of categories. In our experiment, we set the fixed parameter
determined in an unsupervised manner, and there is no need α to 64. Then all classes have the roughly equal number of
to manually label the pure stains in different areas. The patches.
working principle of SPCN is to replace the color basis of a The main advantage of using image patches in training
source image with the color of a pathologist-preferred target for each category is that it retains the local discrimination
image while reliably keeping the source image structural information of the histopathological image, which helps the
information intact, and still maintaining its original staining model to learn local features [48]. The random image patches
concentration. FIGURE 3 shows images before and after stain generation strategy can also reduce the size of the training
normalization. images and increase the number of training samples at the
same time.
C. PATCH EXTRACTION AND DATA AUGMENTATION Data augmentation is an integral part of deep learning since
The performance of deep learning model depends on the it helps to overcome overfitting on models by increasing
large number of samples used for training. The number of the number of training samples [43]. For breast cancer
training samples for each category in the BreaKHis dataset histopathological images, pathologists can examine a tissue
is limited. So we must increase the number of training slide from different angles without tampering the diagnostic
samples through patch extraction and data augmentation results. So these images are rotation-invariant. We use
algorithm to overcome the overfitting problem in the the data augmentation algorithm to increase the prediction
network. accuracy of the CAD systems, while increasing the number of
Because of the high-resolution of breast cancer histopa- training samples without changing the tissue morphology and
thology images, direct training will lead to excessive cell structure of the image. The data augmentation algorithm
memory consumption and long training time. Inspired by is given in Algorithm I.
Spanhol et al. [30] and Krizhevsky et al. [43], we apply patch
extraction and data augmentation algorithm to increase the D. SCREENING PATCHES
number of training samples and use these training samples The strategy for sampling patches from breast cancer
to train the proposed model. Then we use majority voting histopathological images is described in Section II-C. As only
for obtaining the final image label from the individual image-level labels are given in the breast cancer histopatho-
classifications. It is worth mentioning that we avoid using logical image classification task, the label of the whole
smaller image patches of size 32 × 32 or 64 × 64 [30]. image is usually assigned to the corresponding generated
This is because in the BreaKHis dataset, the label has been image patches. Therefore, the patch-level labels may not
assigned to the whole input breast cancer histopathological be consistent with the image-level labels. These mislabeled
image with a size of 700 × 460, and there is no guarantee that patches may affect the training of subsequent network and
a smaller image patch with a size of 32 × 32 or 64 × 64 will reduce the classification performance. To avoid mislabeled
carry sufficient diagnostic information. Therefore, we divide image patches when we use patch-based classification
the images of size 700 × 460 into the patches of size 224 method, inspired by Schlegl et al. [36], we propose a
× 224 that would provide a larger field of view and carry screening mislabeled patches method based on an unsuper-
more local discrimination features in contrast to the smaller vised anomaly detection with generative adversarial networks
patches [44]–[47]. (AnoGAN). FIGURE 4 shows the framework of screening
As can be seen from TABLE 1, BreaKHis dataset has a patches using AnoGAN.
problem with data imbalance. The imbalance ratio between
malignant and benign classes is 0.45 at the image level and 1) GENERATIVE ADVERSARIAL NETWORK
0.41 at the patient level. In classification tasks, the data The generative adversarial network (GAN) [49] consists of
imbalance problem may cause the discrimination ability of two adversarial models, a generator G and a discriminator D.
the computer-aided diagnosis (CAD) systems to be biased The generator network captures the data distribution and
towards the majority class. To minimize the influence of data maps G(z) of random samples z, 1D vectors of uniformly
imbalance on the model performance, we adopt a random distributed input noise sampled from the latent space Z ,
patches extraction strategy. In the jth category, the number of to data space. The discriminator network estimates the

155366 VOLUME 8, 2020

R. Man et al.: Classification of Breast Cancer Histopathological Images Using Discriminative Patches Screened

FIGURE 4. The framework of screening patches from Anomaly Detection with Generative Adversarial Networks (AnoGAN). Generative adversarial training
is performed on benign data and testing is performed on both, unseen healthy cases and anomalous data.

Algorithm 1 Data Augmentation of BreakHis Histopatholog- Let x be data representing an image. For the generator
ical Images network, let z be a latent space vector sampled from uniform
Input: Breast cancer histopathological image Ik from distribution. G(z) refers the generator function which maps z
BreakHis after stain normalization pre-rocessing. to data space. The generator can generate fake samples from
Output: Augmented images {Ia1 , Ia2 , . . . , Ian }. the estimated distribution pg by estimating the training data
Functions: comes from pdata . D(x) represents the probability that x came
RanPatchGen() represents the methods of random from the training data rather than the generator. D(G(z)) is
patches extraction with Eq.(1); the probability that the output of the generator G is a real
Rotation() represents the methods of rotation with πQ 4 image. As Goodfellow et al. [49] described, a discriminator D
variations with Q in {0, 1, . . . , 7}; tries to maximize the probability of classifying reals and fakes
Filp() represents the methods of horizontal and vertical (logD(x)), and a generator G simultaneously tried to fool a
reflection. discriminator D via minimizing log(1 − D(G(z))). Therefore,
Step 1: we can find D and G through the following two-player
Take histopathological image Ik after stain normaliza- minimax game, with the value function V (G, D) [49]:
tion from training set;  
Step 2: minmax V (D, G) = Ex∼pdata (x) log D(x)
G D
Apply random patches extraction alogotithm on image  
+Ez∼pz (x) log(1 − D(G(z))) (2)
Ik :
RanPatchGen() = {Ik1 , Ik2 , . . . , Ikn } In order to screen the most discriminative breast can-
Step 3: cer histopathological image patches in malignant images,
Apply affine transformations on image patches the data distribution of benign image patches can be obtained
{Ik1 , Ik2 , . . . , Ikn }: by GAN, and when we use this GAN to learn the data
for Iki = Ik1 : Ikn do distribution of malignant image patches, there are obvious
Rotation() differences, which provides the possibility to screen the most
Filp() discriminative histopathological image patches and improve
end for the classification performance of the subsequent network.
Break;
2) MAPPING NEW IMAGES TO THE LATENT SPACE
When adversarial training is completed, the generator has
probability that a sample comes from the real data rather learned the mapping G(z) = z → x from the latent space
than the generator network. During the training process, representations z to the benign image patches x. However,
the generator network is optimized by the results of the GAN does not automatically generate the inverse mapping
discriminator network to improve the generating ability,and µ(x) = x → z from the test image patches x to the latent
generates the image as much closer to x as possible space representations z for free, and it needs to find z
to ‘‘fool’’ the discriminator network. At the same time, iteratively [36]. The transition of the latent space is smooth.
the discriminator network also optimizes itself to be better In other words, the images generated from two points at close
at flagging the generated samples. Goodfellow et al. [49] distances in the latent space are very similar [50]. Given a
compared the generative adversarial network (GAN) to the malignant image patch x, we aim to find a point z in the
minimax two-player game between the generator G and the latent space, which corresponds to the image G(z) that is
discriminator D. visually most similar to the malignant image patch x and that

VOLUME 8, 2020 155367

 R. Man et al.: Classification of Breast Cancer Histopathological Images Using Discriminative Patches Screened

FIGURE 5. Structure diagram of breast cancer histopathology image classification based on DenseNet.

is located on the data distribution of benign image patches. E. DENSELY CONNECTED CONVOLUTIONAL NETWORK
Inspired by the feature matching [51], in order to find the TOPOLOGY
best z, the following steps are used: Densely connected convolutional network (DenseNet) [37]
Step1: Define loss function, which represents the loss of combines the advantages of ResNet [52] and Highway [53]
latent space vector mapping to the image patches. to alleviate the vanishing-gradient problem in deep neural
Step2: Randomly sample z1 from the latent space distribu- networks. The idea of DenseNet is to ensure maximum
tion Z and feed z1 into a well-trained generator to obtain the information flow between layers in the network. so we
generated image G(z1 ). Use the loss function to calculate the directly connect all layers (with matching feature-map sizes).
loss. The patch-level breast cancer histopathology images clas-
Step3: Calculate the gradient of the loss function about z1 , sification algorithm includes: input the most discriminative
and use the gradient descent method to continuously update patches screened by AnoGAN, use DenseNet to extract
the coefficients of z1 . During the iteration process via γ = features, and softmax classifier. First, the preprocessed
1, 2, . . . , 0 backpropagation steps to optimize the position image patches are used as the input of the model. During
of z in the latent space Z . Until finding the most similar the training, DenseNet can extract the features of the
image G(z0 ). patches. Finally, the extracted feature vector is sent to the
softmax classifier to complete the classification of breast
3) LOSS FUNCTION cancer histopathology images. The structure of breast cancer
We use a loss function that maps malignant image patches to histopathology image classification model used DenseNet is
the latent space. This loss function includes two components, shown in FIGURE 5.
a residual loss and a discrimination loss. The dense block is the main part of DenseNet. The main
Residual Loss The residual loss is used to measure the characteristic is that each layer connects to every other layer
dissimilarity between the generated image G(zγ ) and the in a feed-forward fashion and passes its own feature maps to
malignant image patches x. all subsequent layers. It can promote better information and
X
 gradient flow, alleviate the vanishing-gradient problem, and
LResidual (zγ ) = x − G zγ  (3) the network can converge better [37]. Assuming that an image
patch x0 passes through the DenseNet, the network comprises
For an ideal normal query situation, the image patches x
L layers, each layer implements a non-linear transformation
and G(zγ ) are the same. In this case, the residual loss is zero.
H (·), and xl is the output of the l th layer. The output of the
Discrimination Loss Inspired by the proposed feature
l th layer is given in Eq.(6):
matching technique, we regard the discriminator as a feature
extractor, and the output of a certain layer of the discriminator
is used as the function f (·) to specify the statistics of an input xl = Hl ([x0 , x1 , . . . , xl−1 ]) , (6)
image. The discriminator loss reflects the difference of the
extracted features by the discriminator on the two feature where [x0 , x1 , . . . , xl−1 ] refers to the concatenation of the
maps. feature-maps produced in layer 0, 1, . . . , l − 1. H (·) includes
X
 three consecutive operations: batch normalization (BN) [54],
LDiscriminator (zγ ) = f (x) − f (G zγ ) (4) rectified linear unit (ReLU) [55] and convolution (Conv).
If each function Hl (·) produces k feature maps, the l th layer
To map to the latent space, we define the total loss as the consequently has k0 + k × (l − 1) input feature maps,
weighted sum of residual loss and discrimination loss: where k0 is the number of channels in the input layer.
L(zγ ) = (1−λ) · LResidual (zγ )+(λ) · LDiscriminator (zγ ) (5) The hyperparameter k is also called growth rate of the
DenseNet. FIGURE 6 illustrates the structure of dense block
Thus, an anomaly score, which expresses the fit of a query schematically.
image x to the model of benign image patches, can be directly DenseNet is divided into multiple dense blocks. These lay-
obtained from the total loss function in Eq.(5). This model ers between dense blocks are called transition layers, which
yields a large anomaly score for malignant image patches take care of down-sampling applying a batch normalization,
whereas a small anomaly score for benign image patches. a 1 × 1 convolution, and a 2 × 2 average pooling.

155368 VOLUME 8, 2020

R. Man et al.: Classification of Breast Cancer Histopathological Images Using Discriminative Patches Screened

FIGURE 6. A 4-layer dense block with a growth rate of k = 4. Each layer takes all preceding feature-maps as input.

We define the ith input image patch xi with the label yi . The global patient level accuracy as:
The DenseNet optimization is supervised by the softmax P
PatientScore
loss(L) [56] which can be written as PatientLevelAccuracy = (9)
Np
 
In the second case, the evaluation metric is image level
1 X 1 X  efyi 
accuracy. Let Nall be the number of breast cancer images of
L= Li = − log  X  , (7)
 
N N  f
e 
j testing set. If the CAD systems classify correctly Nr breast
i i
j cancer images, the image level accuracy is:
Nr
where fj denotes the jth element (j ∈ [1, K ], K is the number ImageLevelAccuracy = (10)
Nall
of classes) of the vector of class scores f , and N is the number
of training image patches. Conventionally, during cancer diagnosis, malignant case is
Compared with the traditional convolutional neural net- considered to be positive while benign case is considered to
work, dense connectivity strengthens the feature propagation be negative. The sensitivity (also called recall) of the CAD
of breast cancer histopathological images, improves the systems is more important in clinically diagnosis. Therefore,
information flow between the various layers, and greatly we not only use the first two evaluation metrics but also other
enhances the feature reuse. Therefore, the DenseNet can evaluation metrics such as precision, recall, F1score are used
automatically learn the discriminative features in breast to evaluate the performance of breast cancer classification.
cancer histopathological images and increase the accuracy of The metrics are calculated respectively as follows:
classification. TruePositives
Precision = (11)
TruePositives + FalsePositives
III. EXPERIMENTS AND RESULTS TruePositives
A. PERFORMANCE EVALUATION Recall = (12)
TruePositives + FalseNegatives
The purpose of the proposed BreaKHis dataset is to form 2 × Recall × Precision
the benchmark of breast cancer CAD systems. For this F1score = (13)
Recall + Precision
reason, BreaKHis authors proposed two classification level
evaluation metrics [18]. In order to visualize the classification performance, we also
The first one is patient level accuracy that reflects the use the confusion matrix that is a specific contingency table.
achieved performance in a patient-wise level. Let Nnp be the
number of pathological images of each patient, Nrp be the B. EXPERIMENTAL PROTOCOL
number of correctly classified images of each patient, and Np Following the standard labeling conventions used in med-
be the total number of patients. The patient score for each ical research, the label ‘‘positive’’ refers to malignant
patient is as follows: images, and ‘‘negative’’ refers to benign images [38]. For
further reducing the color inconsistency and improving
Nrp
PatientScore = (8) efficiency in learning high-level features, stain normal-
Nnp ization pre-processing method described in Section II-B

VOLUME 8, 2020 155369

 R. Man et al.: Classification of Breast Cancer Histopathological Images Using Discriminative Patches Screened

TABLE 2. AnoGAN architecture for screening the discriminative patches. TABLE 3. Densely Connected Convolutional Networks121 (DenseNet121)
architecture for BreakHis patches. The growth rate for DenseNet121 is
k = 32. Note that each ‘‘conv ’’ layer shown in the table corresponds the
sequence BN-ReLU-Conv.

was employed on BreaKHis datasets. In order to prevent

generating random results, we used 5-fold cross validation to
evaluate the proposed method for each magnification factor.
We divided the BreaKHis dataset into five folds and each fold
contained 20% of the overall samples. During training four of Algorithm 1 to increase the number of malignant patches.
the folds were used as the training set whereas the remaining Finally, we used the discriminative patches for training
set was used for testing. DenseNet121. During testing, 100 random patches with the
We applied the random patches extraction strategy men- size of 224 × 224 pixels were cropped from each image in
tioned in Section II-C on the training set, so a roughly equal the testing set. These patches were passed to the well-trained
number of patches were generated for each category could DenseNet121 and the class label of the image was obtained
we get. The size of the patches is 224 × 224 pixels because by majority voting from the individual patch classifications.
it has been shown to be particularly relevant to CNN-based TABLE 2 shows the details of AnoGAN architectures and
classification [44]–[47]. TABLE 3 shows the details of DenseNet121 architectures.
For screening the most discriminative patches, we applied Firstly, we performed 200 epochs utilizing Adam optimizer
the data augmentation algorithm described in Section II-C on with the learning rate 0.001 for training AnoGAN. The
benign patches for training AnoGAN. The malignant patches trained parameters of the generator and discriminator were
were sent to the well-trained AnoGAN for testing, and the dis- kept fixed. We ran 500 backpropagation steps for mapping
criminative malignant patches were screened by the anomaly malignant patches to the latent space. We set λ = 0.1 in
score. Then we used affine transformations mentioned in Equations (5) (0.1 is an empirical value found in the original

155370 VOLUME 8, 2020

R. Man et al.: Classification of Breast Cancer Histopathological Images Using Discriminative Patches Screened

TABLE 4. The mean and standard deviations of classification accuracy of DenseNet121 on BreakHis (without and with data balance). The best mean
results are in bold.

TABLE 5. The mean and standard deviations of classification accuracy of DenseNet121 on BreakHis (without and with AnoGAN). The best mean results
are in bold.

TABLE 6. The mean and standard deviations of precision, recall, and F1score computed from DenseNet121 on BreaKHis (without and with AnoGAN). The
best mean results are in bold.

paper [36]). Secondly, we used Adam optimizer with a batch breast cancer histopathological images classification task.
size of 64 to train the classification model. The learning First, the random patches extraction strategy and data
rate was set as 0.001. Our experiments were implemented in augmentation algorithm described in Section II-C were
Python using Pytorch as deep learning framework backend performed on the training set to obtain roughly equal numbers
and conducted on three NVIDIA GeForce GTX 1080 Ti of patches in both classes (benign and malignant) of size
GPUs with 24GB RAM. 224 × 224. These patches were used to train DenseNet121.
Second, we randomly extracted 64 patches with the size
C. EXPERIMENTAL RESULTS of 224 × 224 from each image in the training set and
This section presents the experimental results of the used the affine transformations mentioned in Algorithm 1 to
proposed approach evaluated on the BreaKHis dataset. increase the number of training samples. DenseNet121 was
In Section III-C-1, we verify the classification of BreakHis trained with the training patches unevenly distributed to
dataset affected by data imbalance. Section III-C-2 presents the two classes (benign and malignant). TABLE 4 shows
the performance of the proposed model (DenseNet121- the accuracy performance of the two experiments. In the
AnoGAN). Additionally, the performance of AnoGAN on the case of the BreakHis dataset, especially the majority class
existing classification networks is evaluated and presented in consists of the images of malignant tissue. It can be seen
Section III-C-3. that high data imbalance significantly affects the performance
of DenseNet121, slight data imbalance is actually beneficial
1) THE IMPACT OF DATA IMBALANCE ON THE for the performance of DenseNet121. This conclusion is
PERFORMANCE OF THE DENSENET consistent with Koziarski [57]. Therefore, we also applied
In this section, we experimentally evaluate the impact of the random patches extraction strategy and data augmentation
data imbalance on the DenseNet121 performance in the algorithm described in Section II-C in subsequent experi-

VOLUME 8, 2020 155371

 R. Man et al.: Classification of Breast Cancer Histopathological Images Using Discriminative Patches Screened

FIGURE 7. Confusion matrices of DenseNet121-AnoGAN model which has the best score in the BreaKHis testing set among
5-fold cross validation, (A): Confusion matrix of 40X magnification, (B): Confusion matrix of 100X magnification, (C): Confusion
matrix of 200X magnification, (D): Confusion matrix of 400X magnification.

ments. It can solve the problem of high data imbalance and accuracy, the classification network with AnoGAN screening
increase the number of training samples. the discriminative patches has further improved the accuracy
of the classification network without using AnoGAN for
2) THE PROPOSED MODEL RESULTS patches screening. The best accuracy of 99.13% has been
DenseNet121-AnoGAN is a novel network for breast cancer obtained at the image level for the 40X magnification factor.
histopathological images classification which can screen In TABLE 6, the assessment of the proposed model based
the discriminative patches and improve the performance of on evaluation metrics like precision, recall, and F1score are
classification. In order to verify the effect of using the further presented. At 40X magnification factor, we achieved
proposed approach, we conducted two sets of experiments. the best precision of 99.53%, the best recall of 99.16%, and
In the first set of experiments, we did not use AnoGAN the best F1score of 99.38%.
for patches screening, and all patches were trained based on A false negative means that a subject with breast cancer
the DenseNet121. In the second set of experiments, the dis- is misclassified as not having the disease on the basis of
criminative patches were screened by AnoGAN. Then we the classification model. The subject is given a misleading
used these discriminative patches to train the DenseNet121. result that she is free of breast cancer and thus does not
TABLE 5 provides the accuracy performance of two sets undergo more suitable diagnostic tests. FIGURE 7 shows
of experiments at the corresponding magnification factors. the confusion matrices of the DenseNet121-AnoGAN model
It can be noticed that there is a significant improvement which has the best score in testing set among 5-fold cross
in the performance of DenseNet121 when AnoGAN is validation. In the confusion matrices, we can see that
employed. For breast cancer histopathological images with the proposed model produces few false negatives at all
magnification factors of 40X, 100X, 200X, and 400X, magnification factors, which proves that the proposed model
whether it is at the patient-level accuracy or at the image-level can further improve the performance of the computer-aided

155372 VOLUME 8, 2020

R. Man et al.: Classification of Breast Cancer Histopathological Images Using Discriminative Patches Screened

FIGURE 8. The receiver operating characteristic (ROC) curves of DenseNet121-AnoGAN models based on BreakHis dataset, (A): ROC
curve of 40X magnification, (B): ROC curve of 100X magnification, (C): ROC curve of 200X magnification, (D): ROC curve of 400X
magnification.

diagnosis (CAD) systems of breast cancer. The performance be seen that all the classification networks with AnoGAN
of DenseNet121-AnoGAN is further analyzed by using to screen patches achieve better performance than the
receiver operating characteristic (ROC) curves corresponding classification networks without AnoGAN. While comparing
to each magnification factor (see FIGURE 8). the performance of the classification network, it can be
observed that the ResNet50-AnoGAN achieves the overall
3) ANOGAN ON THE EXISTING CLASSIFICATION NETWORKS best accuracy of 86.72% at the patient level and ResNet50-
RESULTS AnoGAN achieves the overall best accuracy of 87.02% at
In this section, we presented the performance of the method the image level. In the task of breast cancer histopathological
of screening patches by AnoGAN on the other classifi- images classification, these classification networks only learn
cation networks. We tested AlexNet [31], VGG16 [58], low-level features, such as colors, textures, and edges. How-
VGG19 [58], and ResNet50 [52] on the original patches as ever, DenseNet121 can concatenate features from different
well as the discriminative patches screening by AnoGAN. layers, strengthen features propagation, encourage feature
The experimental results are shown in TABLE 7 (the best reuse, and also has the narrow layers which means that the
mean results are in bold). From the experimental results, it can model has fewer parameters to train, so DenseNet121 makes

VOLUME 8, 2020 155373

 R. Man et al.: Classification of Breast Cancer Histopathological Images Using Discriminative Patches Screened

TABLE 7. The mean and standard deviations of classification accuracy of the existing classification networks on BreaKHis (without and with AnoGAN).
The best mean results are in bold.

the classification task easier and more efficient to train than AnoGAN and it will generate a fake patch with a probability
any other network. distribution similar to that of the benign patch. By defining
the threshold of residual loss and discrimination loss between
IV. DISCUSSION the malignant patch to be tested and fake patch, this
Breast cancer is one of the common types among hundreds well-trained AnoGAN can yield the high anomaly score
of cancer diseases. The incidence of this disease is increasing of the malignant patches. However, the anomaly score of
day by day, especially among women. If the disease is not mislabeled patches in malignant patches is low. Therefore,
diagnosed in time, the mortality rate will be fairly high. In this we can use the obvious differences produced by AnoGAN
work, we propose a novel approach for the classification of to screen the discriminative patches in malignant patches
breast cancer histopathology images, named DenseNet121- and improve the classification performance of the subsequent
AnoGAN. Many researchers have conducted studies on network. Compared with the patch-based image classifica-
the BreakHis dataset. The performance comparison of the tion methods, the proposed approach named DenseNet121-
proposed model with the existing studies using the BreaKHis AnoGAN can effectively solve the problem of mislabeled
dataset is shown in TABLE 8. patches in malignant patches and improve the classification
Compared with all the studies given in TABLE 8, our performance. Nuclei and tissue organization are related to
proposed model obtained the best performance for 40X and the diagnosing process [32]. As the magnification increases,
100X histopathology images. In particular, the accuracy at the the number of nuclei you are able to see in the patches
image level of our proposed model for the 40X magnification will decrease, resulting in incomplete nuclei edge-related
factor is 99.13%, the best precision is 99.53%, the best recall features extracted. The classification network is based on
is 99.16%, and the best F1score is 99.38%. The classification the extracted features from different scales, including nuclei
performance of our proposed model has clearly outperformed and tissues organization. If the classification network cannot
the methodologies of Spanhol et al. [18], Spanhol et al. [30], extract these relevant features at high-level magnifications,
Spanhol et al. [59], and Kumar and Rao [60]. As can be the accuracy of classification network will decline. Although
seen from TABLE 8, our proposed model has the best compared with the existing studies, the proposed model is
performance in low-level magnifications i.e. 40X and 100X insufficient in the classification performance of high-level
compared with the methodologies of Gupta and Bhavsar [61], magnifications, our proposed DenseNet121-AnoGAN can
Sudharshan et al. [38], and Gour et al. [62]. be better suited coarse-grained high-resolution images from
This is the first attempt that we use AnoGAN for breast tissue biopsy slides stained with hematoxylin and eosin
screening discriminative patches to deal with the mislabeled (H&E) and achieved satisfactory classification performance
patches. We use benign patches to train AnoGAN. The at 40X and 100X magnifications. It has laid a foundation for
data distribution of benign patches can be obtained by helping pathologists to diagnose diseases in the future.

155374 VOLUME 8, 2020

R. Man et al.: Classification of Breast Cancer Histopathological Images Using Discriminative Patches Screened

TABLE 8. Performance comparison with existing studies using the BreaKHis dataset in terms of patient level accuracy and image level accuracy.

V. CONCLUSION For breast cancer histopathological images with the magni-

In this paper, we propose a novel approach, named fication factors of 40X, 100X, 200X, and 400X, our exper-
DenseNet121-AnoGAN, for the classification of breast iments show that whether it is at the patient-level accuracy
cancer histopathology images, which screens patches based or the image-level accuracy, the method of screening the
on an unsupervised anomaly detection with generative discriminative patches by AnoGAN has further improved the
adversarial networks (AnoGAN). The proposed model can accuracy of the method without using AnoGAN for patches
effectively solve the problem of mislabeled patches when screening.
we adopt a patch-based classification method and improve Although the proposed model is very effective for
the performance of classification. We have experimentally breast cancer diagnosis in low-level magnifications i.e.
evaluated the proposed model for binary classification 40X and 100X, future work can explore different acti-
using 5-fold cross validation in the BreaKHis dataset at vation functions in the final layer of CNN architectures,
four different magnification factors (40X, 100X, 200X, the optimization of the hyperparameters, and the size
400X). The best accuracy of 99.13% and the best F1score of patches to improve the accuracy at high-level mag-
of 99.38% have been obtained at the image level for the 40X nification. Furthermore, the problem of data imbalance
magnification factor. In addition, we have also preliminarily is ubiquitous in the medical domain, we should explore
explored the impact of data imbalance on the classification some approaches for dealing with data imbalance in the
network and investigated the performance of the method of future.Before being used for clinical diagnosing, we need to
screening patches by AnoGAN on the other classification net- validated on the other breast cancer histopathological image
works, including AlexNet, VGG16, VGG19, and ResNet50. datasets.

VOLUME 8, 2020 155375

 R. Man et al.: Classification of Breast Cancer Histopathological Images Using Discriminative Patches Screened

REFERENCES [22] R. M. Haralick, K. Shanmugam, and I. Dinstein, ‘‘Textural features for

[1] Latest Global Cancer Data: Cancer Burden Rises to 18.1 Million New image classification,’’ IEEE Trans. Syst., Man, Cybern., vol. SMC-3, no. 6,
Cases and 9.6 Million Cancer Deaths in 2018, Int. Agency Res. Cancer, pp. 610–621, Nov. 1973.
Lyon, France, 2018. [23] L. P. Coelho, A. Ahmed, A. Arnold, J. Kangas, A.-S. Sheikh, E. P. Xing,
[2] C. Wild, E. Weiderpass, and B. Stewart, World Cancer Report: Cancer W. W. Cohen, and R. F. Murphy, ‘‘Structured literature image finder:
Research for Cancer Prevention. Lyon, France: The International Agency Extracting information from text and images in biomedical literature,’’
for Research on Cancer (IARC), 2020. [Online]. Available: https:// in Linking Literature, Information, and Knowledge for Biology. Cham,
publications.iarc.fr/Non-Series-Publications/World-Cancer-Reports/ Switzerland: Springer, 2010, pp. 23–32.
World-Cancer-Report-Cancer-Research-For-Cancer-Prevention-2020 [24] E. Rublee, V. Rabaud, K. Konolige, and G. Bradski, ‘‘ORB: An efficient
[3] S. C. DeWit, H. Stromberg, and C. Dallred, Medical-Surgical Nursing: alternative to SIFT or SURF,’’ in Proc. Int. Conf. Comput. Vis., Nov. 2011,
Concepts & Practice. Amsterdam, The Netherlands: Elsevier, 2016. pp. 2564–2571.
[4] D. Bardou, K. Zhang, and S. M. Ahmad, ‘‘Classification of breast cancer [25] Y. Benhammou, B. Achchab, F. Herrera, and S. Tabik, ‘‘BreakHis based
based on histology images using convolutional neural networks,’’ IEEE breast cancer automatic diagnosis using deep learning: Taxonomy, survey
Access, vol. 6, pp. 24680–24693, 2018. and insights,’’ Neurocomputing, vol. 375, pp. 9–24, Jan. 2020.
[5] Institute of Medicine (US) and National Research Council, Mammography [26] M. Hammad, P. Pławiak, K. Wang, and U. R. Acharya, ‘‘ResNet—
and Beyond: Developing Technologies for the Early Detection of Breast Attention model for human authentication using ECG signals,’’ Expert
Cancer. Washington, DC, USA: National Academies Press, 2001. Syst., p. e12547, Mar. 2020, doi: 10.1111/exsy.12547.
[6] J. R. Won, D. Gao, C. Chow, J. Cheng, S. Y. Lau, M. J. Ellis, C. M. Perou, [27] M. Hammad, Y. Liu, and K. Wang, ‘‘Multimodal biometric authentication
P. S. Bernard, and T. O. Nielsen, ‘‘A survey of immunohistochemical systems using convolution neural network based on different level fusion
biomarkers for basal-like breast cancer against a gene expression of ECG and fingerprint,’’ IEEE Access, vol. 7, pp. 26527–26542, 2018.
profile gold standard,’’ Modern Pathol., vol. 26, no. 11, pp. 1438–1450, [28] A. Alghamdi, M. Hammad, H. Ugail, A. Abdel-Raheem, K. Muhammad,
Nov. 2013. H. S. Khalifa, and A. A. A. El-Latif, ‘‘Detection of myocardial infarction
[7] M. N. Gurcan, L. E. Boucheron, A. Can, A. Madabhushi, N. M. Rajpoot, based on novel deep transfer learning methods for urban healthcare
and B. Yener, ‘‘Histopathological image analysis: A review,’’ IEEE Rev. in smart cities,’’ Multimedia Tools Appl., pp. 1–22, Mar. 2020, doi:
Biomed. Eng., vol. 2, pp. 147–171, 2009. 10.1007/s11042-020-08769-x.
[8] G. Aresta, T. Araújo, S. Kwok, S. S. Chennamsetty, M. Safwan, [29] M. Hammad, S. Zhang, and K. Wang, ‘‘A novel two-dimensional ECG
V. Alex, B. Marami, M. Prastawa, M. Prastawa, M. Chan, M. Donovan, feature extraction and classification algorithm based on convolution neural
G. Fernandez, J. Zeineh, M. Kohl, C. Walz, F. Ludwig, S. Braunewell, network for human authentication,’’ Future Gener. Comput. Syst., vol. 101,
M. Baust, Q. D. Vu, and P. Aguiar, ‘‘BACH: Grand challenge on breast pp. 180–196, Dec. 2019.
cancer histology images,’’ Med. Image Anal., vol. 56, pp. 122–139, [30] F. A. Spanhol, L. S. Oliveira, C. Petitjean, and L. Heutte, ‘‘Breast
Aug. 2019. cancer histopathological image classification using convolutional neural
[9] F. Oloumi, R. M. Rangayyan, P. Casti, and A. L. Ells, ‘‘Computer- networks,’’ in Proc. Int. Joint Conf. Neural Netw. (IJCNN), Jul. 2016,
aided diagnosis of plus disease via measurement of vessel thickness in pp. 2560–2567.
retinal fundus images of preterm infants,’’ Comput. Biol. Med., vol. 66, [31] Y. Lecun, L. Bottou, Y. Bengio, and P. Haffner, ‘‘Gradient-based
pp. 316–329, Nov. 2015. learning applied to document recognition,’’ Proc. IEEE, vol. 86, no. 11,
[10] Z. Gandomkar, P. Brennan, and C. Mello-Thoms, ‘‘Computer-based image pp. 2278–2324, Nov. 1998.
analysis in breast pathology,’’ J. Pathol. Informat., vol. 7, no. 1, p. 43, 2016. [32] T. Araújo, G. Aresta, E. Castro, J. Rouco, P. Aguiar, C. Eloy, A. Polónia,
[11] J. Xie, R. Liu, J. Luttrell, and C. Zhang, ‘‘Deep learning based analysis of and A. Campilho, ‘‘Classification of breast cancer histology images using
histopathological images of breast cancer,’’ Frontiers Genet., vol. 10, p. 80, convolutional neural networks,’’ PLoS ONE, vol. 12, no. 6, Jun. 2017,
Feb. 2019. Art. no. e0177544.
[12] C. Kaushal, S. Bhat, D. Koundal, and A. Singla, ‘‘Recent trends in [33] L. Hou, D. Samaras, T. M. Kurc, Y. Gao, J. E. Davis, and J. H. Saltz,
computer assisted diagnosis (CAD) system for breast cancer diagnosis ‘‘Patch-based convolutional neural network for whole slide tissue image
using histopathological images,’’ IRBM, vol. 40, no. 4, pp. 211–227, classification,’’ in Proc. IEEE Conf. Comput. Vis. Pattern Recognit.,
Aug. 2019. Jun. 2016, pp. 2424–2433.
[13] A. Das, M. S. Nair, and S. D. Peter, ‘‘Computer-aided histopathological [34] M. Z. Alom, C. Yakopcic, M. S. Nasrin, T. M. Taha, and V. K. Asari,
image analysis techniques for automated nuclear atypia scoring of ‘‘Breast cancer classification from histopathological images with inception
breast cancer: A review,’’ J. Digit. Imag., pp. 1–31, Jan. 2020, doi: recurrent residual convolutional neural network,’’ J. Digit. Imag., vol. 32,
10.1007/s10278-019-00295-z. no. 4, pp. 605–617, Aug. 2019.
[14] M. Kowal, P. Filipczuk, A. Obuchowicz, J. Korbicz, and R. Monczak, [35] J. Lazzaro, S. Ryckebusch, M. A. Mahowald, and C. A. Mead, ‘‘Winner-
‘‘Computer-aided diagnosis of breast cancer based on fine needle biopsy take-all networks of O (n) complexity,’’ in Proc. Adv. Neural Inf. Process.
microscopic images,’’ Comput. Biol. Med., vol. 43, no. 10, pp. 1563–1572, Syst., 1989, pp. 703–711.
Oct. 2013. [36] T. Schlegl, P. Seeböck, S. M. Waldstein, U. Schmidt-Erfurth, and G. Langs,
[15] Y. M. George, H. H. Zayed, M. I. Roushdy, and B. M. Elbagoury, ‘‘Remote ‘‘Unsupervised anomaly detection with generative adversarial networks
computer-aided breast cancer detection and diagnosis system based on to guide marker discovery,’’ in Proc. Int. Conf. Inf. Process. Med. Imag.
cytological images,’’ IEEE Syst. J., vol. 8, no. 3, pp. 949–964, Sep. 2014. Cham, Switzerland: Springer, 2017, pp. 146–157.
[16] A. D. Belsare, M. M. Mushrif, M. A. Pangarkar, and N. Meshram, [37] G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, ‘‘Densely
‘‘Classification of breast cancer histopathology images using texture connected convolutional networks,’’ in Proc. IEEE Conf. Comput. Vis.
feature analysis,’’ in Proc. TENCON IEEE Region 10 Conf., Nov. 2015, Pattern Recognit., Jul. 2017, pp. 4700–4708.
pp. 1–5. [38] P. J. Sudharshan, C. Petitjean, F. Spanhol, L. E. Oliveira, L. Heutte,
[17] L. Vincent and P. Soille, ‘‘Watersheds in digital spaces: An efficient and P. Honeine, ‘‘Multiple instance learning for histopathological breast
algorithm based on immersion simulations,’’ IEEE Trans. Pattern Anal. cancer image classification,’’ Expert Syst. Appl., vol. 117, pp. 103–111,
Mach. Intell., vol. 13, no. 6, pp. 583–598, Jun. 1991. Mar. 2019.
[18] F. A. Spanhol, L. S. Oliveira, C. Petitjean, and L. Heutte, ‘‘A dataset [39] E. Reinhard, M. Adhikhmin, B. Gooch, and P. Shirley, ‘‘Color transfer
for breast cancer histopathological image classification,’’ IEEE Trans. between images,’’ IEEE Comput. Graph. Appl., vol. 21, no. 4, pp. 34–41,
Biomed. Eng., vol. 63, no. 7, pp. 1455–1462, Jul. 2016. Jul. 2001.
[19] T. Ojala, M. Pietikainen, and T. Maenpaa, ‘‘Multiresolution gray-scale and [40] M. Macenko, M. Niethammer, J. S. Marron, D. Borland, J. T. Woosley,
rotation invariant texture classification with local binary patterns,’’ IEEE X. Guan, C. Schmitt, and N. E. Thomas, ‘‘A method for normalizing
Trans. Pattern Anal. Mach. Intell., vol. 24, no. 7, pp. 971–987, Jul. 2002. histology slides for quantitative analysis,’’ in Proc. IEEE Int. Symp.
[20] Z. Guo, L. Zhang, and D. Zhang, ‘‘A completed modeling of local binary Biomed. Imag. From Nano to Macro, Jun. 2009, pp. 1107–1110.
pattern operator for texture classification,’’ IEEE Trans. Image Process., [41] A. Vahadane, T. Peng, A. Sethi, S. Albarqouni, L. Wang, M. Baust,
vol. 19, no. 6, pp. 1657–1663, Jun. 2010. K. Steiger, A. M. Schlitter, I. Esposito, and N. Navab, ‘‘Structure-
[21] V. Ojansivu and J. Heikkilä, ‘‘Blur insensitive texture classification using preserving color normalization and sparse stain separation for histological
local phase quantization,’’ in Proc. Int. Conf. image signal Process. Cham, images,’’ IEEE Trans. Med. Imag., vol. 35, no. 8, pp. 1962–1971,
Switzerland: Springer, 2008, pp. 236–243. Aug. 2016.

155376 VOLUME 8, 2020

R. Man et al.: Classification of Breast Cancer Histopathological Images Using Discriminative Patches Screened

[42] D. D. Lee and H. S. Seung, ‘‘Learning the parts of objects by non-negative [60] K. Kumar and A. C. S. Rao, ‘‘Breast cancer classification of image using
matrix factorization,’’ Nature, vol. 401, no. 6755, pp. 788–791, Oct. 1999. convolutional neural network,’’ in Proc. 4th Int. Conf. Recent Adv. Inf.
[43] A. Krizhevsky, I. Sutskever, and G. E. Hinton, ‘‘ImageNet classification Technol. (RAIT), 2018, pp. 1–6.
with deep convolutional neural networks,’’ in Proc. Adv. Neural Inf. [61] V. Gupta and A. Bhavsar, ‘‘Sequential modeling of deep features for
Process. Syst., 2012, pp. 1097–1105. breast cancer histopathological image classification,’’ in Proc. IEEE Conf.
[44] K. Das, S. P. K. Karri, A. Guha Roy, J. Chatterjee, and D. Sheet, Comput. Vis. Pattern Recognit. Workshops, Jun. 2018, pp. 2254–2261.
‘‘Classifying histopathology whole-slides using fusion of decisions from [62] M. Gour, S. Jain, and T. Sunil Kumar, ‘‘Residual learning based CNN for
deep convolutional network on a collection of random multi-views at breast cancer histopathological image classification,’’ Int. J. Imag. Syst.
multi-magnification,’’ in Proc. IEEE 14th Int. Symp. Biomed. Imag. (ISBI), Technol., vol. 30, no. 3, pp. 621–635, Sep. 2020.
Apr. 2017, pp. 1024–1027.
[45] W. Zhi, H. W. F. Yueng, Z. Chen, S. M. Zandavi, Z. Lu, and Y. Y. Chung,
‘‘Using transfer learning with convolutional neural networks to diagnose
breast cancer from histopathological images,’’ in Proc. Int. Conf. Neural
Inf. Process. Cham, Switzerland: Springer, 2017, pp. 669–676.
[46] K. Das, S. Conjeti, A. G. Roy, J. Chatterjee, and D. Sheet, ‘‘Multiple
instance learning of deep convolutional neural networks for breast RUI MAN was born in Dezhou, China, in 1996.
histopathology whole slide classification,’’ in Proc. IEEE 15th Int. Symp. She received the bachelor’s degree from Qufu
Biomed. Imag. (ISBI), Apr. 2018, pp. 578–581. Normal University, Qufu, China, in 2018. She
[47] Z. Gandomkar, P. C. Brennan, and C. Mello-Thoms, ‘‘MuDeRN: Multi- is currently pursuing the master’s degree with
category classification of breast histopathological image using deep Beijing Union University. Her research interests
residual networks,’’ Artif. Intell. Med., vol. 88, pp. 14–24, Jun. 2018. include medical image processing and deep learn-
[48] Y. Ono, E. Trulls, P. Fua, and K. M. Yi, ‘‘LF-Net: Learning local ing.
features from images,’’ in Proc. Adv. Neural Inf. Process. Syst., 2018,
pp. 6234–6244.
[49] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley,
S. Ozair, A. Courville, and Y. Bengio, ‘‘Generative adversarial nets,’’ in
Proc. Adv. Neural Inf. Process. Syst., 2014, pp. 2672–2680.
[50] R. A. Yeh, C. Chen, T. Y. Lim, A. G. Schwing, M. Hasegawa-Johnson,
and M. N. Do, ‘‘Semantic image inpainting with deep
generative models,’’ 2016, arXiv:1607.07539. [Online]. Available:
http://arxiv.org/abs/1607.07539 PING YANG received the Ph.D. degree in control
[51] T. Salimans, I. Goodfellow, W. Zaremba, V. Cheung, A. Radford, and theory and control engineering from the China
X. Chen, ‘‘Improved techniques for training gans,’’ in Proc. Adv. Neural University of Mining and Technology, Beijing,
Inf. Process. Syst., 2016, pp. 2234–2242.
in 2006. She is currently an Associate Professor
[52] K. He, X. Zhang, S. Ren, and J. Sun, ‘‘Deep residual learning for
with the Smart City College, Beijing Union
image recognition,’’ in Proc. IEEE Conf. Comput. Vis. Pattern Recognit.,
Jun. 2016, pp. 770–778. University. Her works have been published in
[53] R. Kumar Srivastava, K. Greff, and J. Schmidhuber, ‘‘Highway several international conferences and journals. Her
networks,’’ 2015, arXiv:1505.00387. [Online]. Available: research interests include information acquisition
http://arxiv.org/abs/1505.00387 and processing, image processing, and the Internet
[54] S. Ioffe and C. Szegedy, ‘‘Batch normalization: Accelerating deep network of Things.
training by reducing internal covariate shift,’’ 2015, arXiv:1502.03167.
[Online]. Available: http://arxiv.org/abs/1502.03167
[55] X. Glorot, A. Bordes, and Y. Bengio, ‘‘Deep sparse rectifier neural
networks,’’ in Proc. 14th Int. Conf. Artif. Intell. Statist., 2011, pp. 315–323.
[56] W. Liu, Y. Wen, Z. Yu, and M. Yang, ‘‘Large-margin softmax loss for
convolutional neural networks,’’ in Proc. ICML, 2016, vol. 2, no. 3, p. 7.
BOWEN XU was born in Dezhou, China, in 1996.
[57] M. Koziarski, ‘‘Radial-based undersampling algorithm for classification
of breast cancer histopathological images affected by data imbalance,’’ He received the bachelor’s degree from the
in Proc. 12th Int. Congr. Image Signal Process., Biomed. Eng. Informat. Shandong University of Science and Technology,
(CISP-BMEI), Oct. 2019, pp. 1–5. Qingdao, China, in 2018. He is currently pursuing
[58] K. Simonyan and A. Zisserman, ‘‘Very deep convolutional networks the master’s degree with the Beijing University
for large-scale image recognition,’’ 2014, arXiv:1409.1556. [Online]. of Technology. His research interests include
Available: http://arxiv.org/abs/1409.1556 anomaly detection and water quality time series
[59] F. A. Spanhol, L. S. Oliveira, P. R. Cavalin, C. Petitjean, and L. Heutte, prediction.
‘‘Deep features for breast cancer histopathological image classification,’’
in Proc. IEEE Int. Conf. Syst., Man, Cybern., Oct. 2017, pp. 1868–1873.

VOLUME 8, 2020 155377