J. Med. Biol. Eng.

(2016) 36:344–356
DOI 10.1007/s40846-016-0138-8

ORIGINAL ARTICLE

Bleeding and Tumor Detection for Capsule Endoscopy Images

Using Improved Geometric Feature
Erzhong Hu1 • Hidenori Sakanashi2 • Hirokazu Nosato2 • Eiichi Takahashi2 •

Yasuo Suzuki3 • Ken Takeuchi3 • Hiroshi Aoki3 • Masahiro Murakawa2

Received: 6 May 2015 / Accepted: 26 August 2015 / Published online: 6 June 2016
 Taiwanese Society of Biomedical Engineering 2016

Abstract This paper presents a method for detecting pre-processing method that uses a non-linear conversion
bleeding and tumors within capsule endoscopy (CE) ima- model for the HSV color space is also proposed. Anomaly
ges. Because CE can be used for visual, non-invasive detection is implemented using a support vector machine
examinations of the small bowel, it has recently become classifier. The results of experiments, conducted with
widely used. However, as a capsule progresses along the normal, bleeding, and tumor images obtained from 28
gastrointestinal tract, it collects vast quantities of images patients, demonstrate both the feasibility and superiority of
that make diagnosis a very time-consuming task. To the proposed method for both bleeding and tumor detection
address this problem, many computational approaches for tasks.
anomaly detection have been proposed. Common to most
approaches is the belief that color, texture, and shape are Keywords Capsule endoscopy  Anomaly detection 
the most promising features for detecting anomalies within Local-contrast-enhanced higher-order local auto-
CE images. However, given that the requirements for each correlation (LCE-HLAC)  Non-linear conversion of HSV
type of feature vary according to the anomaly, generally, it color space  Support vector machine (SVM)
is essential to apply a complicated combination of tech-
niques for multiple-feature extraction. In this study, in
order to realize a scheme that covers the features of color, 1 Introduction
texture, and shape and can be applied to lesion areas of
various sizes, a geometric image feature called local-con- Gastrointestinal (GI) diseases, and GI cancers in particular,
trast-enhanced higher-order local auto-correlation (LCE- are among the most serious threats to human health. In
HLAC) is proposed. Moreover, although the HSV color 2012, GI cancer, including both colorectal and stomach
space is generally regarded as being appropriate for the cancer, was reported as being the most frequently diag-
analysis of CE images, imbalances in the distributions of nosed form of cancer worldwide [1]. Most GI diseases can
utilized hue components limit discriminatory performance be cured if diagnosed and treated early. In terms of diag-
for normal and anomalous images. Accordingly, an image nosis approaches, in addition to blood tests, radiography,
ultrasound, computed tomography (CT) and magnetic
resonance imaging (MRI) [2], endoscopy is important for
& Erzhong Hu the examination of some diseases such as suspected
[email protected] intestinal tumors [3]. However, the small intestine (the long
1
Department of Intelligent Interaction Technologies,
and winding middle section of the GI tract) is difficult to
University of Tsukuba, Tsukuba 305-8573, Japan examine with traditional endoscopy technologies, such as
2 gastroscopy and colonoscopy [4].
Artificial Intelligence Research Center, National Institute of
Advanced Industrial Science and Technology (AIST), Capsule endoscopy (CE) involves a pill-shaped device
Tsukuba 305-8568, Japan that provides patient-friendly, visual intestinal examina-
3
Department of Gastroenterology, Sakura Medical Center, tions. The latest models can capture video data at about 2–6
Toho University, Sakura 285-8741, Japan frames per second as the device moves along the GI tract

123
Bleeding and Tumor Detection for Capsule Endoscopy Images Using Improved Geometric Feature 345

and transmits the data to an external receiver. CE makes it polyps). Moreover, segmentation methods [20] and edge
possible to examine the entire small bowel, and in partic- detection methods [21] have been applied. Apart from
ular facilitates effective diagnosis of diseases such as these specific feature-based approaches, some studies have
bleeding and Crohn’s disease [5, 6]. A full-length CE video employed more complicated combinations of feature
usually lasts for approximately 8 h and contains in excess extraction techniques. For example, one study [22]
of 50,000 consecutive image frames [7]. On average, an employed MPEG-7 visual descriptors, including a variety
examiner requires between 45 and 120 min to screen the of color, texture, and shape descriptors, for detecting
images [8], but this is dependent both on the examiner’s bleeding, ulcers, and polyps. In the bleeding and ulcer
skill and experience and also on image conditions, such as detection method proposed by Szczypinski et al. [23],
mucosa tone, illumination, and amount of residue. More- anomaly detection was executed using software [24] that
over, when anomalies are suspected, examination time calculated various statistics, such as image histograms,
tends to be longer, adding further burden on the limited gradient magnitude, and grey-level co-occurrence matrices.
time of examiners [9]. Although CE manufacturers have Thus, the three features of color, texture, and shape are
developed and bundled assistance software for clinical generally regarded as providing the best descriptions for
diagnosis, some reports claim that accuracy levels are the detection of anomalies. However, the requirements of
unsatisfactory and oversights may occur due to the large these features vary according to the kind of lesion being
quantities of images and limited diagnosis times [10]. examined for anomalies. For example, when detecting
Considering this issue, automatic assistance for anomaly tumor areas that usually have a reddish appearance with
detection seems to be a promising means of improving both basically round contours, it is essential to employ suit-
the efficiency and quality of diagnoses. Proposed automatic able description schemes for both color and shape features.
anomaly detection methods can be divided into two broad At the same time, as already noted, most of the techniques
categories, namely predefined-condition-based evaluations for feature extraction have focused on only one of the three
and geometric-feature-based evaluations. As previously feature types, such as LBP and its improved RIU-LBP
reported [9, 11–13], predefined-condition-based evalua- form, which are suitable for texture features. To obtain
tions implement anomaly detection via pixel matching, for descriptions for more than one type of feature [22, 23], it is
either single or multiple color components within the RGB, often necessary to use a complicated combination of
HSV, or CIE L*a*b* color spaces, according to predefined techniques for multiple feature extraction.
conditions for specified lesions. However, because lesion Previous studies [25, 26] by Hu et al. focused on the
appearances vary considerably and are mutable, it is almost various anomalies of CE images using feature extraction
impossible to formulate a generalized definition for all based on higher-order local auto-correlation (HLAC) [27].
cases of a given anomaly. Moreover, mismatched defini- Analyses have revealed that HLAC can encompass the
tions can be a source of oversights, so the accuracy levels three features of color, texture, and shape, and that it has a
of normal and anomaly discriminations cannot be com- number of advantages, including low computational cost,
pletely guaranteed. shift invariance (independence of object location), and
In contrast, geometric-feature-based evaluations appear robustness to image noise. HLAC has also been demon-
to be a more promising direction. Such methods generally strated to be effective in many applications involving
adopt specialized geometric feature extraction that utilizes image classification, such as object counting [28], cancer
color, texture, and shape features, and involve statistical detection from pathology images [29, 30], and video
classification techniques that can make judgments on surveillance [31]. However, in the case of anomaly detec-
newly added normal or anomaly samples without matching tion from CE images, HLAC has been demonstrated less
with predefined conditions. For example, Li et al. [14, 15] effective for local small-scale lesions, which can be easily
adopted the chromaticity moment for the extraction of overlooked and thus lead to false-negative predictions.
color features, and Barbosa et al. [16] proposed the Accordingly, for the detection of bleeding and tumors, it
extraction of texture features using a co-occurrence matrix is desirable to have a scheme that incorporates color, tex-
combined with discrete wavelet transformation. Another ture, and shape at low computational cost. Moreover, it is
popular texture descriptor is the local binary pattern (LBP) also essential that the method can be applied to bleeding
feature. Ulcer and tumor detection based on rotation-in- and tumor areas of various sizes. In order to fulfill these
variant uniform LBP (RIU-LBP) has been reported requirements, this study proposes an improved geometric
[17, 18]. Compared to color wavelet covariance [19], LBP feature called local-contrast-enhanced higher-order local
has been shown to be more effective for tumor detection. auto-correlation (LCE-HLAC), which is an extension of
Shape features have also been considered for anomalies HLAC feature but also emphasizes descriptions for local
that are known to exhibit distinctive figurations (such as areas. Furthermore, to enhance discriminatory performance

123
346 E. Hu et al.

for bleeding and tumor images, this study also proposes an than bleeding images, in terms of tumor area, size, shape,
approach for image pre-processing based on a non-linear and color across cases.
conversion model for the HSV color space.
2.2 Process of Proposed Method

2 Materials and Methods The proposed method of anomaly detection consists of two
stages, namely training and testing (Fig. 2), and three main
2.1 Detection Targets processes, namely image pre-processing, geometric feature
extraction, and classification. During both stages, to cope
Bleeding and tumors are adopted as detection targets with imbalances within the selected color space, image pre-
because they are common intestinal symptoms that have processing involving a non-linear conversion of the color
specific appearances and characteristics. Bleeding images space is executed. Then, the proposed approach for geo-
for different patients can sometimes have different metric feature extraction is applied to obtain image fea-
appearances due to the conditions of bleeding (blood’s tone tures. The features extracted from the training dataset,
and quantity) and individual variations. For example, in which include both normal and anomalous images, are used
Fig. 1a, b more shed blood can be observed than in Fig. 1c, to train the support vector machine (SVM) classifier, which
d. Such distinctions are strongly affected by the seriousness then classifies the test dataset, which also consists of nor-
of bleeding, and also by the distance between the bleeding mal and anomalous images.
and the location where the image is captured. Images taken
closer to the bleeding spot often contain more shed blood. 2.3 Non-linear Conversion of HSV Color Space
Tumors are defined as abnormal masses of tissue
attached to the lumen surface, which can be either benign The RGB color space is the most widely used color space
or malignant. Bowel polyps, such as those shown in for image analysis. However, red, green, and blue com-
Fig. 1e–g, are benign tumors. Figure 1e, f present the most ponents are highly correlated and are not perceptually
common single polyps, which are not as harmful as uniform for human eyes [32, 33]. The HSV color space
malignant tumors. Single polyps are usually reddish in represents hue (color), saturation (colorfulness), and value
appearance, and are either partially or completely round. (brightness) from the respective RBG channels:
Polyps collected from a segment affected with the Cron- 8
> GB
khite-Canada syndrome (Fig. 1g) are also benign tumors. >
> 60  þ 0 ; if MAX ¼ R
>
> MAX  MIN
However, unlike single polyps, multiple polyps are wide- >
<  BR 
spread. To prevent benign tumors from becoming malig- Hue0 ¼ 60  MAX  MIN þ 120 ; if MAX ¼ G
>
> RG
nant, clinically, early detection for benign tumors is >
> 60  þ 240 ; if MAX ¼ B
>
>
regarded very important. The tumor shown in Fig. 1h is : MAX  MIN
0; if R ¼ G ¼ B
another case of a tumor selected from our database. It has
ð1Þ
been diagnosed as duodenum papilla cancer, which is a 
kind of malignant tumor that must be treated soon using 0 0
Hue ; if Hue  0
Hue ¼ ð2Þ
clinical procedures. From investigating various kinds of 360 þ Hue0 ; if Hue0 \0
tumor, tumor images can vary in appearance even more

Fig. 1 Examples of a–d bleeding and e–h tumor images Fig. 2 Processes of proposed method of anomaly detection

123
Bleeding and Tumor Detection for Capsule Endoscopy Images Using Improved Geometric Feature 347

MAX  MIN constrained deviations are then delivered to geometric

Saturation ¼ ð3Þ
MAX features when feature extraction is performed on the hue
Value ¼ MAX ð4Þ channel image, and thus, generate quite limited discrim-
inability. In order to balance the unevenly utilized range
where MAX is the largest value among R, G and B, and along the hue axis and to further improve discrimination
MIN is the smallest. As an indication of color variety, the for bleeding and tumor detection, the following non-linear
hue channel is independent of brightness, as the plane of conversion is introduced:
the hue channel is always orthogonal to the brightness 8
>
<0 ðif Huebefore ¼ 0 or 80 \Huebefore  360Þ
(value) axis. This property is particularly suitable for CE Hueafter ¼ 360 ðif 0 \Huebefore \10Þ
images. For all forms of highlights, shades, and shadows in >
:
1:0877ð80Huebefore Þ ðif 10  Huebefore  80Þ
the intestinal lumen, the hue channel is essentially
ð5Þ
unaffected.
Based on evaluations of a large number of CE images, where Huebefore is directly calculated from the RGB color
Fig. 3a shows the distribution of their hue components. The space. To make use of the full width along the hue axis, the
overwhelming majority of non-zero components span the proposed non-linear conversion involves reversal and
range of [10, 80], leaving the proportions of components expansion. First, the components for Huebefore within the
within the range of (0, 10) and (80, 360) at less than range of [10, 80] are inversed to 80  Huebefore . Then, the
7.1 9 10-4 and 8.3 9 10-5, respectively. However, full [0, 360] range is used by expanding the inversed
according to the hue channel model, red-pink components, components from the [10, 80] section with the exponential
which are generally regarded as characteristics of lesion function 1:0877ð80Huebefore Þ , so that the circular hue channel
areas, have smaller values and are closer to 0 than are model can be fully utilized. As shown in Fig. 3b, the
yellow-green components. Clearly, the narrow range of the proposed non-linear conversion (Fig. 3c) balances the
hue component distribution leads to very limited deviations unevenly utilized range along the hue axis, enhancing
within the numerical representations along the hue axis, numerical discrimination for feature extraction.
especially for reddish lesion-related components. These

Fig. 3 Non-linear conversion

of HSV color space. a Ordinary
hue components, b converted
hue components, and
c conversion function

123
348 E. Hu et al.

2.4 Local-Contrast-Enhanced Higher-Order Local number of benefits, including low computational cost, shift
Auto-Correlation invariance (independence of object location), and robust-
ness to noise.
2.4.1 Foundations and Motivations Nevertheless, in contrast to its efficacy for detecting
anomalies within large areas, as in Fig. 1g, the HLAC
Our explanation of the proposed method of feature feature is somewhat deficient in coping with some images
extraction starts by outlining the fundamental form for that include smaller local anomalies, as in Fig. 1e, f, h.
HLAC. N-th order HLAC is described by the following Since HLAC calculates a summation of uniformly weigh-
auto-correlations: ted auto-correlations across the entire image and lacks
X detailed representations for small-scale anomalous regions,
RN ða1 ; . . .aN Þ ¼ IðrÞIðr þ a1 Þ    I ðr þ aN Þ ð6Þ
r
features extracted from these local anomalous regions tend
to be submerged and are thus not fully reflected within the
where I is an intensity image and r is the location vector, so feature. Given this concern, it is necessary to develop an
that I ðrÞ is the reference pixel. ai ði ¼ 1; 2; . . .NÞ corre- approach for feature extraction that is more efficient for the
sponds to the displacement vector surrounding r. Changing local details associated with small-scale anomalies, and yet
the order N leads to changes in the numbers of terms in still retains the major advantages of HLAC. Local
I ðrÞ    I ðr þ aN Þ. To balance efficacy and the computa- anomalies usually have morbid appearances within small
tional burden, N is generally set as N [ {0, 1, 2} [28]. lesion areas, with distinct changes from the surrounding
Figure 4 shows mask patterns that represent combinations normal intestinal wall. As a solution, we introduce evalu-
of the reference pixel and surrounding correlative pixels ations of local contrast, which are particularly appropriate
when computing the auto-correlations. Specifically, zeroth- for the detection of small-scale anomalies, into the HLAC
order HLAC obtains the intensity of the reference pixel, feature. This extension is called LCE-HLAC. By employ-
and first-order and second-order HLAC calculate auto- ing LCE-HLAC, it is possible to formulate for peculiar
correlations between the reference pixel and surrounding pixels or areas that differ from their surroundings.
correlative pixels, indicating the apparent edge curves and
evaluating curvature, respectively. Overlapping is also 2.4.2 Definition of LCE-HLAC and Interception
considered, such that squared and cubic auto-correlations
of the central pixel itself are also calculated. Moreover, To realize evaluations of local contrast, the auto-correla-
mask patterns arranged on the same rows can also take into tions are divided according to intensity differences between
consideration uniformity within various orientations (hor- I ðrÞ and neighboring pixels I ðr þ aÞ, which are measured
izontal, vertical, and diagonal) and generate a variety of by a set of functions. Similar to the basic HLAC, N is also
mask patterns based on different N and ai values. Given the restricted as N[ {0, 1, 2}. In each case, the auto-correla-
addition and multiplication operations, HLAC has a tions for LCE-HLAC are defined, respectively, as follows:
X  
RN¼0 ðfk0 Þ ¼ I ðrÞfk0 I ðrÞ; I ðr þ aÞ ; k0 2 f1; 2; 3g
r2I
ð7Þ
X
RN¼1 ða1 ; fk1 Þ ¼ I ðrÞI ðr þ a1 Þfk1 ðIðrÞ; I ðr þ a1 ÞÞ;
r2I ð8Þ
k1 2 f1; 2; 3g
RN¼2 ða1 ; a2 ; fk1 ; fk2 Þ
X
¼ I ðrÞI ðr þ a1 ÞI ðr þ a2 Þfk1 ðIðrÞ; I ðr þ a1 ÞÞ ð9Þ
r2I
 fk2 ðIðrÞ; I ðr þ a2 ÞÞ; k1 ; k2 2 f1; 2; 3g

where fk corresponds to mutually exclusive functions.


1; I ðrÞ  I ðr þ ai Þ [ T
f1 ðI ðrÞ; I ðr þ ai ÞÞ ¼ ð10Þ
0; I ðrÞ  I ðr þ ai Þ  T

1; I ðrÞ  I ðr þ ai Þ\  T
f2 ðI ðrÞ; I ðr þ ai ÞÞ ¼ ð11Þ
0; I ðrÞ  I ðr þ ai Þ   T

Fig. 4 HLAC mask patterns for gray-scale images

123
Bleeding and Tumor Detection for Capsule Endoscopy Images Using Improved Geometric Feature 349


1; jIðrÞ  Iðr þ ai Þj  T LCE-HLAC benefit from further enhancements for evalu-
f3 ðI ðrÞ; I ðr þ ai ÞÞ ¼ ð12Þ ating edge curves. For example, Fig. 5b demonstrates that
0; jIðrÞ  Iðr þ ai Þj [ T
some diagonal second-order LCE-HLAC pattern masks are
For ai 2 RN , f1 , f2 , and f3 execute the respective com- capable of obtaining edge information when scanning
parisons between all pairings of neighboring pixels and the beyond the boundaries of a tumor area. Due to the obvious
reference pixel. Since f1 , f2 , and f3 represent mutually difference between normal and tumor sides, comparisons
exclusive conditions, at any given point, only one of the executed by the judgment functions f1 , f2 , and f3 sort the
three functions can be activated and conduct the corre- auto-correlations of object pixels into different pattern
sponding calculation of auto-correlations. T is the judgment masks. In this way, edge characteristics are easily reflected
threshold. Smaller (larger) T values make feature extraction within the feature vector. Moreover, because LCE-HLAC
more (less) sensitive to local contrasts. Since T is a dom- comprehensively covers all pixel intensities, co-occurrence
inant factor on sensitivity to local contrast information, relations within various patterns, and contrastive relations
preliminary discussions about determining its thresholds for the reference and surrounding pixels, it effectively
are provided in the following section. In the case of N = 0, encompasses all three feature types of color, texture, and
given that only the reference pixel is involved in auto- shape.
correlation, we introduce I ðr þ aÞ, which indicates the In terms of computational complexity, considering all
average intensity of d surrounding pixels, to replace the dimensions shown in Table 1, the feature vector of
I ðr þ ai Þ. LCE-HLAC involves a higher degree of dimensionality
when extended from the original HLAC. Given that various
1X d  
I ð r þ aÞ ¼ I r þ aj ð13Þ contrast conditions are being considered, the dimensions of
d j¼1
the LCE-HLAC feature vectors increase from 35 to 225.
LCE-HLAC can be thought of an extension of HLAC. However, little additional computational cost is incurred
Figure 5a shows examples of mask patterns when N is 0, 1, because the mutually exclusive nature of the judgment
and 2, respectively. In each situation, the mask patterns for functions means that auto-correlation calculations are
auto-correlation are expanded according to the evaluations executed sparsely under exclusive conditions, and no
for local contrasts by the judgment functions fk . In the overlapping calculations are involved. Thus, it is clear that
zeroth-order case (N = 0), apart from intensity, the the computational complexity of LCE-HLAC is effectively
description of sharply fluctuating pixels, possibly associ- constrained. Naturally, LCE-HLAC also inherits the other
ated with part of a small anomalous area or discrete HLAC advantages of shift invariance, additivity, and
anomalous pixels, is clearly distinguished from those robustness to noise.
referring to insignificant changes. First- and second-order
2.4.3 Determining Threshold T

In order to acquire an applicable T, we utilize the dataset to

be introduced in Sect. 3.1. As the detection of tumor
images is generally regarded to be a more difficult task than
bleeding detection, we created a small dataset of CE ima-
ges, which includes 600 normal and 300 tumor randomly
selected images, and employed principal component anal-
ysis (PCA; a simple and frequently used multivariate
analysis method) to construct a subspace of the feature
vectors and evaluate the performances of alternative
T values. Specifically, feature vectors extracted from nor-
mal images are firstly used to construct a normal feature

Table 1 Dimensionality of HLAC and LCE-HLAC

HLAC LCE-HLAC

Zeroth-order 1 3
First-order 5 15
Fig. 5 a Extension of rule for mask patterns from HLAC to LCE- Second-order 29 207
HLAC. b Example of how LCE-HLAC mask patterns extract edge Total 35 225
features

123
350 E. Hu et al.

subspace. Then, the distances between the LCE-HLAC hyperplane on which image feature vectors can be sepa-
feature vectors for the remaining 300 normal images and rated by the maximized margin between classes [35].
300 tumor images and the established subspace are calcu- Consider a set of data samples of the form {(y1 ; z1 ),
lated and equated as distinction degrees. We then analyze (y2 ; z2 ),…,(yn ; zn )}. yi 2 f1; þ1g indicates the labels of
these distinction degrees and produce their receiver oper- the classes that the feature vectors z1 ,z2 ,…,zn (zi 2 RN Þ
ating characteristic (ROC) curves (Fig. 6a). Finally, we belong to. In this work, for anomaly detection, positive
calculate the area under the curve (AUC) for all curves to samples are valued ?1, while negative samples that cor-
determine the best-performing threshold T, which corre- respond to the normal class are valued -1. Then, the
sponds to the largest AUC value. Based on our investiga- hyperplane can be written as:
tions of various images, T is assigned with a set of values z 2 F : ðw  zÞ  b ¼ 0 ð14Þ
ranging from 5 to 50. As shown in Fig. 6b, T = 10 leads to
the largest AUC, T values of below 40 lead to no significant where x is the weight vector and b is a non-negative
differences, and T values of over 40 tend to be detrimental parameter. If data samples can be separated linearly, then
to performance. In our experiments, LCE-HLAC with two parallel hyperplanes, H1 : (x z) b = 1 and H2 : (x
T = 10 is applied to validate its efficacy and for compar- z) b = þ1, should exist. Since the distance between the
ison with other approaches. two hyperplanes (margin) is kw2 k, to obtain the maximized
margin, calculations for satisfying the parameters x and b
2.5 SVM-Based Classification are required. Accordingly, the problem is equivalent to
minimizing kwk2 =2 = wt w/2, with constraints on points
SVM [34] is one of the most popular state-of-the-art yi (x zi ) -b C 1, i = 1, 2,…, n. By introducing the
classifiers. The principle behind SVM is to construct a Lagrange multiplier ai C 0, i = 1, 2,…, n, the problem is
further transformed to one of finding a solution that mini-
mizes the following formula:
X
N
1X N
LðaÞ ¼ ai  ai aj yi yj zti zj ð15Þ
i¼1
2 i;j¼1

As a result, the proper parameter w can be expressed as:

X
N
w ¼ ai yi zi ð16Þ
i¼1

where ai is the solution for Eq. (15). Finally, the resultant y
on a new sample vector znþ1 is calculated as follows:
!
X N
 t
y ¼ sgn ai yi zi znþ1  b ð17Þ
i¼1

When the input data are inseparable by a linear SVM as

above, a kernel function k(zi , zj ) = (U(zi )U(zj )) is applied
[36]. zi and zj are n-dimensional inputs. U is a map from the
n-dimension to an m-dimensional space (m [ n). Com-
monly, kernel functions include linear functions, polyno-
mial functions, and radial basis functions.

2.6 Experimental Data

CE data were collected from videos for 28 clinical patients.

Observations and diagnoses were carried out by two
experienced doctors. For each patient, the expert doctors
compiled detailed reports including diagnostic comments
and recording time ranges for normal/lesion segments.
Fig. 6 Performance obtained with various threshold T values in
preliminary PCA validations. a ROC curves and b comparison of Based on these reports, we first sampled video segments
AUC values that had been documented as containing certain kinds of

123
Bleeding and Tumor Detection for Capsule Endoscopy Images Using Improved Geometric Feature 351

data. Then, we extracted from these segments complete L*a*b*, another color space that separates color from
sets of images (480 9 480 pixels), before excluding any intensity, was utilized as another control group. Given that
seriously distorted images, such as those full of bubbles, the V (value) channel of the HSV color space and the L
strong bile, residues, or taken under extreme lighting channel of the CIE L*a*b* color space both refer to
conditions. The remaining images were subsequently cat- brightness, which is strongly related to lighting conditions,
egorized and labeled. In this way, we obtained a complete we excluded these channels and only utilized the H–S
dataset of 69 video segments that yielded 5642 normal channels and a*–b* channels, respectively. We concate-
images from 33 segments, 5476 bleeding images from 22 nated the feature vectors obtained from each channel to
segments, and 1164 tumor images from 14 segments create a long feature vector in the experiments.
(Table 2). Before being used, the data for each category In order to measure the efficacy of the proposed method
were double checked by the doctors to ensure that all the and to compare with other control methods, we first
image data were consistent with their opinions. Note that obtained the directly predicted results that encompass the
all of these segments were separated so that the diversity of fundamental evaluation criteria sensitivity and specificity,
the dataset could be guaranteed. and an average accuracy covering all test samples. The
Within an entire CE video, suspected lesions usually definitions of sensitivity and specificity are given as
occur in partial sections, and so, generally speaking, it is follows:
much easier to obtain normal data than lesion data, which
Number of correct positive predictions
is reflected in the imbalance of normal, bleeding, and tumor Sensitivity ¼
Number of positives
images within our database. However, to eliminate any
influences of this imbalance that may cause failures for ð18Þ
small partitions of the training data, while simultaneously Number of correct negative predictions
fully utilizing all the lesion data, we created non-overlap- Specificity ¼
Number of negatives
ping training and testing datasets that consisted of the same ð19Þ
number of normal images as lesion images (Table 2). For
each category, we first separated the normal/lesion images Specificity is an indicator of normal samples (nega-
from each patient into two equal parts, and then we tives), while sensitivity is an indicator of anomaly samples
assigned one part for each patient to create the training (positives). Both high sensitivity and high specificity cor-
dataset and assigned the other part to create the testing respond to excellent performance for binary classification.
dataset. In accordance with the principle of SVM, we Furthermore, we obtained the ROC curves for all normal
labeled all normal images and lesion images as -1 and ?1, and anomaly test samples. SVM was employed to estimate
respectively. positive probability for all test samples. ROC curves were
traced out by changing the threshold for positive proba-
2.7 Procedures and Evaluation Criterion bilities. Accordingly, in the results relating to the ROC
curves, curves with large AUC corresponding to the
In order to demonstrate the performance of the proposed dynamic combination of high sensitivity and specificity are
color space conversion and the feature extraction method, expected.
analyses were conducted for experiments on bleeding and
tumor detection. In evaluating the performance of the
proposed method, we also employed the basic HLAC 3 Results and Discussion
feature and the RIU-LBP feature for comparison. For
evaluating the proposed color space conversion, feature Summaries of results for bleeding and tumor detection are
extraction was implemented on both the proposed non- presented in Tables 3 and 4, respectively. In Table 3, there
linear conversion of the HSV color space (cHSV, the same are no conspicuous differences among the different groups
below) and the ordinary HSV color space. Moreover, CIE of approaches. The best performance in terms of total

Table 2 Categories of
Category Normal Bleeding Tumor
experimental data
Number of patients 14 14 8
Number of segments 33 22 14
Number of images 5642 5476 1164
Number of images used for bleeding detection 5476 5476 –
Number of images used for tumor detection 1164 – 1164

123
352 E. Hu et al.

accuracy (98.80 %) was obtained by the combination of the with the CIE L*a*b* color space. With the CIE L*a*b*
original HLAC feature and the proposed cHSV color space color space, only 79.21 % of tumor images were detected
model. However, the highest sensitivity (99.49 %), which correctly. The situation for the group utilizing the original
depicts the best bleeding detection result, was obtained by HLAC feature is similar. The HLAC feature yields mat-
the proposed method using the LCE-HLAC feature and ched performances adopting both the ordinary HSV color
color space conversion. For all groups, the average accu- space and the cHSV color space, but the sensitivity suffers
racies are larger than 97.48 %, with no sensitivity and a severe drop to 74.40 % when the CIE L*a*b* color space
specificity values of less than 96.90 and 96.09 %, respec- is introduced. For the RIU-LBP feature group, however,
tively. Such high accuracies clearly indicate that all method unlike the groups of LCE-HLAC and HLAC, both the
groups had considerable efficacies for bleeding samples. CIEL*a*b* and original HSV color spaces performed
The ROC curves, as shown in Fig. 7a, verify this result. similarly for RIU-LBP feature extraction. It can also be
With respect to the detection of tumors, as presented in inferred that the proposed cHSV color space is adaptive for
Table 4, the results yielded by the different methods RIU-LBP feature, since it obtained a total accuracy of
diverge more widely than those in Table 3. The best per- 88.75 % and a sensitivity of 90.89 %, which are slightly
formance was obtained by the proposed method, with a higher than those for the other color spaces. Generally
sensitivity of 96.74 %, a specificity of 88.14 %, and a total speaking, the color space did no substantially affect the
accuracy of 92.44 %. In terms of sensitivity, tumor RIU-LBP group’s performances.
detection based on the combination of the LCE-HLAC Figure 7b presents ROC curves for all groups. Consis-
feature and the proposed cHSV color space was 3.1 per- tent with the predicted results of accuracy, the proposed
centage points better than the best results obtained by the method, with both the LCE-HLAC feature and color space
HLAC group. The highest specificity (94.33 %) was conversion, had the largest AUC value (0.977, see
obtained when the LCE-HLAC feature and the original Table 4). From this result, we can deduce that the conse-
HSV color space were adopted. This indicates that quences of both false-positive predictions and false-nega-
although the efficacies in terms of total accuracy and sen- tive predictions are well restrained by the proposed
sitivity achieved using the LCE-HLAC feature and cHSV method.
color space surpass those obtained upon the other color Moreover, the results for tumor detection vary greatly
spaces, false-positive predictions were best controlled by according to the technique adopted. From further analysis,
using the original HSV color space. we identified that images with multiple tumor areas, such
For the first group of LCE-HLAC, in addition to being as in Fig. 8a, tend to have concentrations of strong reddish
the best accuracy combination, the method using the components. In contrast to Fig. 8b–d, which show smaller
ordinary HSV space reached 91.41 % for total accuracy local tumors, the histogram of cHue components in Fig. 8a
and 88.49 % for sensitivity. However, decreases emerged shows an obvious shifting to the right side. However, the
when the LCE-HLAC feature extraction was combined majority of cHue components in Fig. 8b–d are similar to

Table 3 Classification results

Accuracy (%) Sensitivity (%) Specificity (%) AUC
for bleeding detection in terms
of accuracies and AUC values LCE-HLAC
(areas under ROC curves)
cHSV 98.54 99.49 97.59 0.997
HSV 97.94 98.39 97.48 0.991
CIE L*a*b* 97.48 98.87 96.09 0.996
HLAC
cHSV 98.80 99.20 98.39 0.997
HSV 98.47 99.09 97.85 0.994
CIE L*a*b* 97.66 96.90 98.43 0.994
RIU-LBP
cHSV 98.08 97.81 98.36 0.997
HSV 98.32 98.98 97.66 0.997
CIE L*a*b* 98.21 99.12 97.30 0.994

123
Bleeding and Tumor Detection for Capsule Endoscopy Images Using Improved Geometric Feature 353

Table 4 Classification results

Accuracy (%) Sensitivity (%) Specificity (%) AUC
for tumor detection in terms of
accuracies and AUC values LCE-HLAC
(areas under ROC curves)
cHSV 92.44 96.74 88.14 0.977
HSV 91.41 88.49 94.33 0.965
CIE L*a*b* 84.02 79.21 88.83 0.913
HLAC
cHSV 91.07 93.13 89.00 0.965
HSV 91.67 93.64 89.69 0.961
CIE L*a*b* 82.05 74.40 89.69 0.874
RIU-LBP
cHSV 88.75 90.89 86.60 0.938
HSV 87.46 88.66 86.25 0.944
CIE L*a*b* 86.68 87.97 85.40 0.956

divergent detection results indicate that LCE-HLAC is the

most suitable for the detection of local tumors. Compared
to HLAC, the edge characteristic between tumor area and
the adjacent normal lumen area are more effectively
extracted by LCE-HLAC.
These results clearly demonstrate that the combination
of LCE-HLAC feature extraction and non-linear conver-
sion for the HSV color space is efficient for both bleeding
and tumor detection. Its performance is superior to that for
the control conditions consisting of various technique
combinations for tumor detection.

4 Conclusion

This paper proposed a detection method for bleeding and

tumors that utilizes the LCE-HLAC feature for geometric
feature extraction. LCE-HLAC encompasses the three main
image features of color, texture, and shape. In addition, the
LCE-HLAC feature is capable of providing detailed
descriptions of local contrasts for local bleeding and tumor
areas. Despite the increased complexity of the extended
technique, computational cost is well controlled. In conjunc-
tion with the LCE-HLAC feature extraction, a non-linear
conversion for the HSV color space was proposed. By
adopting SVM-based classification, the proposed method was
evaluated in experiments of bleeding and tumor detection.
To create a reliable demonstration, a dataset of CE
Fig. 7 Classification results with ROC curves for a bleeding and images was collected from 28 patient cases. In contrast to
b tumor detection
methods based on HLAC and RIU-LBP features, the pro-
those in the normal intestinal image, such as in Fig. 8e. posed method had the best performance (98.54 % accu-
Moreover, pixel values of local tumors are assumed to be racy) for bleeding detection and (92.44 % accuracy) tumor
depicted only within the small region beyond 160. The detection. The experimental results also demonstrate that

123
354 E. Hu et al.

Fig. 8 Histogram analysis for

tumor and normal images.
a Tumor image with multiple
tumor areas, b–d tumor images
with local tumor areas, and
e normal intestinal image

the non-linear conversion of the HSV color space is par- 96.74 % for tumor detection when the proposed non-linear
ticularly suitable for LCE-HLAC feature extraction. Sen- conversion of HSV color space substituted for the original
sitivities were significantly enhanced from 88.49 to HSV color space.

123
Bleeding and Tumor Detection for Capsule Endoscopy Images Using Improved Geometric Feature 355

References based feature selection. The IEEE Transactions on Information

Technology in Biomedicine, 16, 323–329.
1. International Agency for Research on Cancer, World Cancer 19. Karkanis, S. A., Iakovidis, D. K., Maroulis, D. E., Karras, D. A., &
Factsheet 2012, 2013. Tzivras, M. (2003). Computer-aided tumor detection in endoscopic
video using color wavelet features. The IEEE Transactions on
2. University of Rochester Medical Center, ‘‘Digestive diagnostic
procedures,’’ http://www.urmc.rochester.edu/encyclopedia/con Information Technology in Biomedicine, 7, 141–152.
tent.aspx?ContentTypeID=85&ContentID=P00364. Accessed 24 20. Hwang, S., & Celebi M. E. (2010). Polyp detection in wireless
Nov 2015. capsule endoscopy videos based on image segmentation and geo-
metric feature. In Proceedings of IEEE International Conference on
3. Xynopoulos, D., Mihas, A. A., Paraskevas, E., Dimitroulopoulos,
D., Heuman, D. M., & Mihas, A. A. (2002). Small bowel tumors. Acoustics, Speech and Signal Processing (pp. 678–681).
Annals of Gastroenterology, 15, 18–35. 21. Lee, J., Oh, J. H., Shah, S. K., Yuan, X., & Tang, S. J. (2007).
4. Rey, J.-F., Gay, G., Kruse, A., & Lambert, R. (2004). ESGE Automatic classification of digestive organs in wireless capsule
guideline for video capsule endoscopy. Endoscopy, 36, 656–658. endoscopy videos. In Proceedings of ACM Symposium on
5. Hara, A. K., Leighton, J. A., Heigh, R. I., Sharma, V. K., Silva, A. C., Applied computing (pp. 1041–1045).
De Petris, G., et al. (2006). Crohn disease of the small bowel: Pre- 22. Coimbra, M. T., & Cunha, J. P. S. (2006). MPEG-7 visual
liminary comparison among CT enterography, capsule endoscopy, descriptors-contributions for automated feature extraction in
small-bowel follow-through, and ileoscopy. Radiology, 238, 128–134. capsule endoscopy. IEEE Transactions on Circuits and Systems
6. Doherty, G. A., Moss, A. C., & Cheifetz, A. S. (2011). Capsule for Video Technology, 16, 628–637.
endoscopy for small-bowel evaluation in Crohn’s disease. Gas- 23. Szczypiński, P., Klepaczko, A., Pazurek, M., & Daniel, P. (2014).
Texture and color based image segmentation and pathology
trointestinal Endoscopy, 74, 167–175.
7. Delvaux, M., & Gay, G. (2008). Capsule endoscopy: Techniques detection in capsule endoscopy videos. Computer Methods and
and indications. Best Practice & Research Clinical Gastroen- Programs in Biomedicine, 113, 396–411.
terology, 22, 813–837. 24. Szczypiński, P., Strzelecki, M., Materka, A., & Klepaczko, A.
8. Yagi, Y., Vu, H., Echigo, T., Sagawa, R., Yagi, K., Shiba, M., (2009). Mazda—A software package for image texture analy-
et al. (2007). A diagnosis support system for capsule endoscopy. sis. Computer Methods and Programs in Biomedicine, 94,
Inflammopharmacology, 15, 78–83. 66–76.
9. Gan, T., Wu, J. C., Rao, N. N., Chen, T., & Liu, B. (2008). A 25. Hu, E., Nosato, H., Sakanashi, H., & Masahiro, M. (2012).
feasibility trial of computer-aided diagnosis for enteric lesions in Anomaly detection for capsule endoscopy images using higher-
capsule endoscopy. World Journal of Gastroenterology, 14, order local auto correlation features. In Proceedings of IEEE
6929–6935. International Conference on Systems, Man, and Cybernetics (pp.
2289–2293).
10. Signorelli, C., Villa, F., Rondonotti, E., Abbiati, C., Beccari, G.,
& de Franchis, R. (2005). Sensitivity and specificity of the sus- 26. Hu, E., Nosato, H., Sakanashi, H., & Masahiro, M. (2013). A
pected blood identification system in video capsule endoscopy. modified anomaly detection method for capsule endoscopy ima-
Endoscopy, 37, 1170–1173. ges using non-linear color conversion and higher-order local
auto-correlation (HLAC). In Proceedings of IEEE International
11. Penna, B., Tillo, T., Grangetto, M., Magli, E., & Olmo, G. (2009).
A technique for blood detection in wireless capsule endoscopy Conference on Engineering in Medicine and Biology Society (pp.
images. In Proceedings of European Signal Processing Confer- 5477–5480).
ence (pp. 1865–1868). 27. Otsu, N., & Kurita, T., (1988). A new scheme for practical
12. Lau, P. Y. (2007). Detection of bleeding patterns in WCE video flexible and intelligent vision systems. In Proceedings of IAPR
using multiple features. In Proceedings of IEEE International Workshop on Computer Vision (pp. 431–435).
Conference on Engineering in Medicine and Biology Society (pp. 28. Kobayashi, T., Hosaka, T., Mimura, S., Hayashi, T., & Otsu, N.
5601–5604). (2008). HLAC approach to automatic object counting. In Pro-
13. Al-Rahayfeh, A. A., & Abuzneid, A. A. (2010). Detection of ceedings of Bio-inspired, Learning and Intelligent Systems for
bleeding in wireless capsule endoscopy images using range ratio Security (pp. 40–45).
color. International Journal of Multimedia and Its Applications, 29. Nosato, H., Kurihara, T., Sakanashi, H., Murakawa, M.,
Kobayashi, T., Furuya, T., et al. (2011). An extended method of
2(2), 1–10.
14. Li, B., & Meng, M. Q. (2009). Computer-based detection of higher-order local autocorrelation feature extraction for classifi-
bleeding and ulcer in wireless capsule endoscopy images by cation of histopathological images. IPSJ Transactions on Com-
chromaticity moments. Computers in Biology and Medicine, 39, puter Vision and Applications, 3, 211–221.
141–147. 30. Qu, J., Nosato, H., Sakanashi, H., Terai, K., & Hiruta, N. (2012).
15. Li, B., & Meng, M. Q. (2009). Computer aided detection of Cancer detection from pathological images using higher-order
bleeding regions for capsule endoscopy images. IEEE Transac- local autocorrelation feature. In Proceedings of IEEE Interna-
tions on Biomedical Engineering, 56, 1032–1039. tional Conference on Signal Processing (pp. 1198–1201).
16. Barbosa, D. J., Ramos, J., & Lima, C. S. (2008). Detection of 31. Mimura, S., Itoh, K., Kobayashi, T., Takigawa, T., Tajima, A.,
small bowel tumors in capsule endoscopy frames using texture Sawamura, A., & Otsu, N. (2008). The cow gait recognition using
analysis based on the discrete wavelet transform. In Proceedings CHLAC. In Proceedings of Bio-inspired, Learning and Intelli-
gent Systems for Security (pp. 56–57).
of IEEE International Conference on Engineering in Medicine
and Biology Society (pp. 3012–3015). 32. Li, C. H., & Yuen, P. C. (2000). Regularized color clustering in
17. Li, B., & Meng, M. Q. (2009). Texture analysis for ulcer detec- medical image database. IEEE Transactions on Medical Imaging,
tion in capsule endoscopy images. Image and Vision Computing, 19, 1150–1155.
33. Mete, M. (2008). Color analysis. Delineation of malignant areas
27, 1336–1342.
18. Li, B., & Meng, M. Q. (2012). Tumor recognition in wireless in histological images of head-neck cancer (pp. 28–31). Little
capsule endoscopy images using textural features and SVM- Rock: University of Arkansas.

123
356 E. Hu et al.

34. Vapnik, V. N. (1998). Statislical Learning Theory. NewYork: 36. Press, W. H., Teukolsky, S. A., Vetterling, W. T., & Flannery, B.
Wiley. P. (2007). The kernel trick. Numerical recipes: The art of sci-
35. Onoda, T. (2001). Summary of support vector machine. Opera- entific computing (3rd ed., pp. 889–892). New York: Cambridge
tions Research, 225–230. University Press.

123