IEEE-20180

Novel Algorithm for Segmentation and

Automatic Identification of Pests on Plants using
Image Processing

Sushma R. Huddar*, Swarna Gowri*, Keerthana K. *, Vasanthi S.** and Sudhir Rao Rupanagudi***
*Student, Department of Telecommunication, Atria Institute of Technology, Bangalore, India
"Head of Department, Department of Telecommunication, Atria Institute of Technology, Bangalore, India
***Technical Research Associate, WoridServe Education, Bangalore, India
* {sushmahuddar210, swarna.jul, keerthanayadav91}@gmail.com, **[email protected],***[email protected]

Abstract- Enormous agricultural yield is lost every year, In order to address these disadvantages, several present
due to rapid infestation by pests and insects. A lot of day pest detection and control methodologies exists
research is being carried out worldwide to identify scientific
which include image processing based pest identification
methodologies for early detection/identification of these bio­
and also video analysis for sticky traps [10], [11]. These
aggressors. In the recent past, several approaches based on
two methodologies involve several complex image
automation and image processing have come to light to
address this issu e. Most of the algorithms concentrate on
processing algorithms to achieve the same and are limited
pest identification and detection, limit ed to a greenhouse to a greenhouse environment. The setup consists of a
environment. Also , they involve several complex calculations camera being focused on a sticky trap. Images are
to achieve the same. In this paper, we propose a nove l and captured whenever an insect comes into contact with the
unique algorithm to segregate and detect pests using image trap. An average of pests accumulated on the trap on a
processing. The proposed methodology involves reduced particular day gives us the density of pest population in
computational complexity and aims at pest detection not
the greenhouse. Remedial measures can be taken based
only in a greenhouse environment but also in a farm
on the density.
environment as well. The whitefly, a bio-aggressor which
In this paper we introduce a novel segmentation
poses a threat to a multitude of crops, was chosen as the pest
of interest in this paper. The algorithm was tested for
algorithm based on image processing to successfully
several whiteflies affecting different leaves and an accuracy identity pests on the leaves itself, instead of the sticky
of 96% of whitefly detection was achieved. The algorithm traps. This new methodology combined with the robotic
was developed and implemented using MATLAB pest detection vehicle discussed in [12] can be
programming language on MATLAB 7.1 build 2011a. incorporated in farms and provide an immediate cost
effective measure to eradicate pests in real-time. It also
provides a viable measure to reduce the amount of
I. INTRODUCTION
pesticide sprayed thus reducing the harmful effect on the
Agriculture is one of the most important sources for environment. The general working of the system has been
human sustenance on Earth. Not only does it provide the elaborated in section II and the algorithm itself in section
much necessary food for human existence and III. Results for the same and accuracy obtained in
consumption but also plays a major vital role in the detecting pests have been discussed in Section IV.
economy of the country [1], [2]. Several millions of We have chosen whiteflies as the pest of interest for
dollars are spent worldwide for the safety of crops, this paper since it is the most hazardous pest causing
agricultural produce and good, healthy yield. It is a matter widespread destruction to a large number of crops such as
of concern to safeguard crops from Bio-aggressors such tomato, cabbage, broccoli etc. Also, the whitefly due to
as pests and insects, which otherwise lead to widespread its small size is very difficult to identity and poses a
damage and loss of crops [3]. In a country such as India, challenge to image processing based pest detection
approximately 18% of crop yield is lost due to pest systems [13].
attacks every year which is valued around 90,000 million
rupees [4]. Conventionally, manual pest monitoring II. SYSTEM OVERVIEW

techniques, sticky traps [5], black light traps [6] are being The general working of the pest detection system using
utilized for pest monitoring and detection in farms. image processing is based on the following four steps [3],
Manual pest monitoring techniques are time consuming [14]:
and subjective to the availability of a human expert to
detect the same [7]. Sticky traps and black light traps are A. Color Conversion

less effective and also prone to cause harm to Digitally acquired images of the crops (leaves) are
environmental friendly insects [8], [9]. As a preventive obtained with the help of a low resolution color camera.
measure farmers spray pesticides in bulk, wh ich are Color conversion is performed on this picture from (Red,
detrimental and hazardous to the ecosystem [5]. Green and Blue) RGB to luminance and chroma
components (YCbCr). Based on experiments either Y, Cb

ICCCNT'12
26th_28th July 2012, Coimbatore, India
 IEEE-201S0

or Cr values can be utilized for further processing. The The resultant image obtained is a monochrome image on
advantage is that instead of storing 3 bytes of color data a which segmentation processes are performed.
single byte of data could be used for further processing Segmentation is performed on the color converted
thus saving memory and processing time [15]. image in order to segregate whiteflies from the leaf
background as explained in Section II. The image is now
B. Segmentation
scanned pixel-wise from left to right starting from
The next step in the detection process is Segmentation. topmost pixel.
Segmentation is performed on the color converted image,
Consider, a (i, j) to be the Cb value of the current pixel
to separate the object of interest from the leaf. Generally,
Gaussian Mixture Models and Watershed segmentation under consideration. A unique segmentation algorithm
algorithms are employed to extract the whiteflies from the based on relative pixel intensity is now applied on this
background as discussed in [16]. These algorithms involve pixel to identify whether the pixel belongs to the
a large number of arithmetic calculations in the form of background or is part of the whitefly. Fig. I shows the Cb
divisions, multiplications and calculating mean which in pixel values of the whitefly and the leaf, magnified. It
turn leads to an exponential increase in execution time. must be noted that in Fig. 1, a (i,j) and the surrounding
This issue is addressed with the help of a simplified pixels having intensities of Cb components greater than
approach which involves reduced computation lOO belong to the whitefly. Pixels a (i, j- q», a (i- q>, j)
complexity. This is based on a novel algorithm based on and other pixels with intensities less than lOO belong to
relative difference in pixel intensities (RDI) and is the background. It can be concluded from this that there
explained in detail in section III.
is a large variation in intensity of Cb values between the
C. Reduction in Noise
pixels which belong to the whitefly and those that belong
to the leaf. Based on this inference, it can be clearly seen
During the extraction process using segmentation,
that a difference in pixel intensities could therefore be
superfluous information in the form of dust, water droplets
and other parts of the leaf are visible in the resultant used to segregate the whitefly.
segmented image. It is necessary to eliminate this noise, in
" " " " 4"':
order to obtain only the whiteflies. This is performed 56 S! 51 51 51: 54

using the erosion algorithm [17]. The resultant image " 56 " 5' 56 54 0 :-:- a(i-<j>, j)

consists of pixels only belonging to the whitefly. Image 56 5' " " 51 " . , 12 . , " "
enhancement in the form of dilation is also performed to 12 '2 !7 " eo n
���e):��J_ �:.�_ ��_ ..e2
" 59 " .3 . ,
improvise the clarity of the segmented whiteflies. 59 ® 61 75 .2

" 55 ,. 101 112 : 11;;': 112 � 101 : 108 i 108

D. Counting the whiteflies .5 .. " 73 -101 ��-4'r�; �1���T;�;r;;;1�-;�
A counting algorithm is implemented to count the
number of whiteflies on each affected leaf. This is based a(i,j-<j» a(i,j)
on Moore Neighbor tracing and Jacob's stopping criterion
Fig. I. A magnified view of the Cb components of the whitefly and
[IS]. Based on the average number of pests detected in 1% the leaf background. Variations in pixel intensities can be clearly
of plants per square area of the field [19], a data analysis seen.
could be performed to decide the amount of pesticide to
be sprayed. According to the algorithm, the relative difference of
Cb intensities between the current pixel under
III. DETECTION OF WHITEFLY USING RDI consideration (a) and surrounding pixels are calculated
and compared with a threshold value e. The threshold
The general steps to perform whitefly detection and
value was determined experimentally and was found to
fmding the total count of whiteflies involve the steps
be 55. This relative difference in pixel intensities is
explored in II. Low resolution images obtained from the
calculated with pixels located q> locations to the left and
camera are first subjected to a color conversion
top of a. These differences are represented by PI and P2
algorithm. Segmentation is then performed based on the
respectively and closely guarded by (2) & (3).
color converted values obtained.
Through experimentation it was found that
PI= a (i, j)- a (i, j-q» (2)
segmentation based on the intensity component (Y) did
P2 = a (i, j)- a (i-q>, j) (3)
not provide consistent results due to variation in lighting
conditions. The red chroma (Cr) component also did not
Through extensive experimentation, the value of q> was
provide sufficient information for segmentation of the
found to be 15. If PI or P2 is found to be greater than the
whitefly. Satisfactory segmentation results with minimum
threshold value e, then the resultant pixel 'I' (i, j) is
noise were obtained for different illuminations of the leaf
identified to be belonging to the foreground. Otherwise it
using only the blue chroma (Cb) components. Hence, the
is identified to constitute the background. This has been
picture is first converted to Cb using (1).

{
represented in (4).
Cb=O.J6*R-0.33*G+O.500*B (1)
255.' if (PI or P2) 2: e
'I' (i,j) =
Where R, G, B are the red, green and blue pixel
O·, if (PI or P2) < e (4)
components of the color image of the pest affected leaf.

ICCCNT'12
26th_2Sth July 2012, Coirnbatore, India
 IEEE-20lS0

In other words, in the resultant segmented image '¥, the the whiteflies have been accurately detected and
pixels with value '255' represent the whiteflies & those encircled.
with a value of '0' belong to the leaf.
As mentioned earlier in Section II, the resultant
segmented image consists of not only whiteflies but also
noise in the form of water droplets, dust e.t.c. In order to
eliminate these pixels which are erroneously detected as
whiteflies, the erosion process is carried out as explained
in [14]. A 3x3 structuring element is utilized for the same
purpose. This was carried out twice. Erosion manages to
eliminate the noise but degrades the actual size of the
whitefly. In order restore the size of the whiteflies, (a) (b)

dilation operation is carried out using the same

structuring element.
Once all the whiteflies have been successfully � �.
1\."
extracted from the image of the leaf, a counting algorithm .. .� ...
� \
is deployed using the Moore Neighbor tracing and
Jacob's stopping criterion as explained in [IS].
Therefore it can be clearly seen that the RDI algorithm
�� .. ,
utilizes only two subtractions and one comparison , "
�
operation per pixel to separate the whitefly. This forms a
main significant highlight of utilizing the RDI algorithm (c) (d)

for segregating as opposed to complex computations

involved in other methods. Also the algorithm is less
susceptible to variations in outdoor light illuminating the
leaf (intensity).
The results for the segmentation algorithm explained in
this section along with the results obtained after noise
removal and counting have been elaborated in section IV.

IV. RESULTS

The algorithm was designed and developed using the

(e) (1)
MA TLAB programming language on the MA TLAB Fig. 2. Output obtained at various stages of the whitefly detection
version 7.1 platform. The methodology was tested for algorithm where (a) is the image of the pest affected leaf. (b) the
over lOO pictures from various sources for whiteflies on Cb components of the original image. (c) resultant image after
implementing RDT algorithm. (d) and (e) outputs after first and
different leaves under different lighting conditions. The
second stage of erosion. (I) four whiteflies successfully detected
outputs of the various experiments conducted are after dilation and counting. Image Source:
explained below. http://www.maherjabado.com
In order to test the correct working of the algorithm, an
image of a leaf infected with whiteflies which was In order to further validate the working of the RDI
captured under normal lighting conditions was obtained algorithm, the steps mentioned above were performed on
(Fig. 2a). As explained in Section II and III, the pest a brighter image (Fig. 3a). Satisfactory results were
affected leaf image is now color converted to Cb obtained as shown in Fig. 3b. It can be ascertained from
components (Fig. 2b). The novel RDI algorithm is now the results that the algorithm exhibits considerable
applied and a segmented image is obtained. It can be seen precis ion even if there is modification in external light
from Fig. 2c, that the segmented image contains illuminating the object.
unwanted details like veins of the leaf and other dust
particles. In order to improve the accuracy of detection,
we need to eliminate this noise. We therefore perform
erosion. Fig. 2d shows the result of this filtering process.
It is very evident that a part of the significant noise still
remains unfIltered. In order to improvise on the accuracy
of detecting whiteflies the erosion process is performed
once again to remove the residual noise. It can be
observed from Fig. 2e that the size of the whiteflies
diminishes as a result of the erosion process. In order to (a) (b)
restore the size, we perform dilation. A counting
algorithm based on Moore Neighbor tracing and Jacob's Fig. 3. RDI algorithm on images captured in high intensity light (a)
stopping criterion is then performed to count the total image of a leaf taken in bright light; (b) resultant segmented image
after filtering of noise. Image Source: http://www. sbproducts. co.uk/
number of whiteflies. It can be observed in Fig. 2f that

ICCCNT'12
26th_2Sth July 2012, Coimbatore, India
 IEEE-201S0

Another important aspect to be considered is the

presence of pests other than the whiteflies. In order to test
our algorithm for this condition, an image of a leaf
affected by the whiteflies along with the thrip pest
(encircled) was selected as shown in Fig. 4a. The result
reveals that the presence of the thrip has no significant
bearing on the outcome of the algorithm and only the
objects of interest i.e. the whiteflies were detected.
In [16], an algorithm based on region based (a) (b)
segmentation and watershed algorithm utilizing video
cameras was used as a special case to identify two Fig. 6. Testing for false alarms (a) Image of a leaf unaffected by
whiteflies in close proximity. In order to test this whiteflies; (b) resultant image after processing indicating no
whitefl y infestation
condition using the RDI algorithm, the same picture (Fig.
5a) was utilized and experimented on. We notice that the
V. CONCLUSION
occluding whiteflies are impeccably detected as two
separate distinct whiteflies as shown in Fig. 5b. In this paper, we describe a unique methodology to

Finally, in order to inspect for false alarms, the automatically detect pests in plants. A distinct algorithm
algorithm was tested on an image containing no for detecting whiteflies affecting various leaves based on
the relative difference intensity (RDI) algorithm was
whiteflies as shown in Fig. 6a. A leaf with white dust
proposed which utilizes only two subtractions and one
particles and light intensity veins was chosen for this
comparison per pixel. The algorithm not only works for
purpose. The resultant segmented image, shown in Fig. greenhouse based crops but also agricultural based crops
6b contains no pixels that are classified as whiteflies, thus as well. Numerous factors like variation of light intensity
proving the resistance of the RDI algorithm to false illuminating the leaf, presence of insects or pests apart
alarms. from those that are of our interest and also whiteflies
situated closely were considered in order to verify the
accuracy of the algorithm. The algorithm was tested for
over 100 pictures from various sources and an accuracy
rate of 96% was observed. By modifying the algorithm
slightly it can be extended for detection of various other
pests as well. The algorithm is based on the assumption
that the leaves are stationary when the image is captured.
In cases where there is movement of leaf due to wind, a
video camera can also be used. With the help of this
methodology pesticides can be sprayed only when
(a) (b) presence of pests are detected, thus safeguarding the
environment and also mankind.
Fig. 4. Results of RDI Algorithm for insects other than whitefly
(a) image of a leaf affected by whiteflies and thrip (encircled); REFERENCES
(b) result after applying the algorithm, identifying only whiteflies.
Image Source: Joe Spencer, Tllinois Natural History Survey [ I] X. Li, Y. Deng, and L. Ding, "Study on precision agriculture
monitoring framework based on WSN," international Conference
on Anti-counterfeiting, Security and identification, vol. 2, pp. 182
- 185, August 2008.
[2] M. Gravlee. (2009, Feb 08). The importance if Agriculture
[Online]. Available: http://www.ezinearticles.coml?The-
Importance-of-Agriculture&id= 1971475.
[3] K. S. Srinivas, B. S. Usha, S. Sandya, and Sudhir R. Rupanagudi,
"Modelling of Edge Detection and Segmentation Algorithm for
Pest Control in Plants", International Conference in Emerging
Trends in Engineering, pp. 293-295, May 2011.
[4] Press Trust of India. (2009, Feb 26). India loses Rs 90k-cr crop
yield to pest attacks every year [Online]. Available:
(a) (b) http://www. financialexpress. comlnews/india-Ioses-rs-90kcr-crop­
yield-to-pest-attacks-every-year 14280911
Fig. 5. Examining the RDl algorithm for closely spaced whiteflies [5] C. Jongman et ai, "Automatic identification of whiteflies, aphids
(a) Image of a leaf on which two whiteflies are situated close to and thrips in greenhouse based on image analysis", International
each other [16]; (b) result of implementing RDT algorithm and journal if Mathematics and computers in simulation, vol. 1, pp.
noise reduction process. The two occluding whiteflies are now 48-53,2007
clearly distinct. [6] M. N. Urban us, and H. J. Tantau, "Suitability of a UV Lamp for
Trapping the Greenhouse whitefly Trialeurodes vaporariorum
Westwood (hom: Aleyrodidae)", Agriculture Engineering
International.' the CIGR Ejournal, Vol. 7, September 2005.
[7] V. Martin, and S. Moisan, "Early pest detection in Greenhouses",
International Conference on Pattern Recognition, December
2008.

ICCCNT'12
26th_2Sth July 2012, Coimbatore, India
 IEEE-201S0

[8] C Chu, and T. J. Henneberry, "Arthropod Management: [14] Rafael C Gonzalez, Richard E Woods, Digital image Processing,
Development of a New whitefly trap", The Journal of Cotton 2nd ed. , Prentice Hall, chapters 9 - 12.
Science, Vol. 2, pp. 104-109, 1998. [15] Saravanan, C, "Color Image to Grayscale Image Conversion",
[9] !PM Thailand. (2008, Mar 14). iPM Components [Online]. Computer Engineering and Applications (ICCEA), Second
Available: http://www. thailand. ipm- international Conference, pp - 196 - 199, 2010
info.or glcomponents/traps. htm. [16] P. Boissard, V. Martin, S. Moisan, "A cognitive vision approach
[10] V. Martin et aI. , "Towards a video camera network for early pest to early pest detection in greenhouse crops", Computers and
detection in greenhouses", international conference on Electronics in Agriculture, Vol. 62, pp. 81-93, 2008.
diversifying crop protection, October 2008. [17] Gonzalez, R. C, R. E. Woods, and S. L. Eddins, Digital image
[11] I. Bechar, S. Moisan, M. Thonnat, and F. Bremond, "On-line Processing Using MATLAB, Gatesmark Publishing, 2009.
video recognition and counting of harmful insect", international [18] M. Berges, E. C Kara, E. Goldman, and A. Rowe, "Towards
Conference on Pattern Recognition, September 2010. automated detection and state tracking of artificial light sources
[12] Y. Li, C. Xia, and J. Lee, "Vision-based pest detection and from sequential pictures inside buildings", EG1CE Workshop, pp.
automatic spray of greenhouse plant", iEEE international 1-10,2011.
Symposium on industrial Electronics (ISiE), pp. 920-925, July [19] Sardi Pest & Diseases (2012). White fly [Online]. Available:
2009. www. sardi. sa.gov. aulpestdiseases/horticuIture/hor ticuItural_pestsli
[13] P. M. Ezop (2010). Sap suckers and magnolia trees [Online]. ntegratedj)est_managementlresources/greenho usej)ests/white_fl
Available: www.gardenguides. comlI00641-sap-suckers-magnolia­ y.
trees.html.

ICCCNT'12
26th_2Sth July 2012, Coimbatore, India