Research Article: Aphid Identification and Counting Based On Smartphone and Machine Vision
Research Article: Aphid Identification and Counting Based On Smartphone and Machine Vision
Journal of Sensors
Volume 2017, Article ID 3964376, 7 pages
https://doi.org/10.1155/2017/3964376
Research Article
Aphid Identification and Counting Based on Smartphone and
Machine Vision
Suo Xuesong, Liu Zi, Sun Lei, Wang Jiao, and Zhao Yang
Mechanical and Electrical Engineering, Agricultural University of Hebei, Baoding, Hebei 071001, China
Received 27 April 2017; Revised 20 June 2017; Accepted 17 July 2017; Published 29 August 2017
Copyright © 2017 Suo Xuesong et al. This is an open access article distributed under the Creative Commons Attribution License,
which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.
Exact enumeration of aphids before the aphids outbreak can provide basis for precision spray. This paper designs counting software
that can be run on smartphones for real-time enumeration of aphids. As a first step of the method used in this paper, images of
the yellow sticky board that is aiming to catch insects are segmented from complex background by using GrabCut method; then
the images will be normalized by perspective transformation method. The second step is the pretreatment on the images; images
of aphids will be segmented by using OSTU threshold method after the effect of random illumination is eliminated by single image
difference method. The last step of the method is aphids’ recognition and counting according to area feature of aphids after extracting
contours of aphids by contour detection method. At last, the result of the experiment proves that the effect of random illumination
can be effectively eliminated by using single image difference method. The counting accuracy in greenhouse is above 95%, while
it can reach 92.5% outside. Thus, it can be seen that the counting software designed in this paper can realize exact enumeration of
aphids under complicated illumination which can be used widely. The design method proposed in this paper can provide basis for
precision spray according to its effective detection insects.
poor timeliness. Therefore, to obtain images in the complex 𝜃 makes the boundary energy more and more little by
environment and high impact of the scene, pests can be iterative method; after iteration, the parameters of the Gauss
identified and counted without relying on the feedback of the mixture model of target and background are higher.
PC side.
𝜃 = {𝜋 (𝛼, 𝑝) , 𝜇 (𝛼, 𝑝) , ∑ (𝛼, 𝑝) , 𝛼 = 0, 1, 𝑘
2. Materials and Methods (4)
= 1 ⋅ ⋅ ⋅ 𝑘} .
The acquisition of experimental images was obtained at the
West Campus of Agricultural University of Hebei; image 2.2. Correction of Yellow Sticky Board. Because the shooting
acquisition equipment is Samsung Galaxy 4 phone that has angle is not fixed, the positions of yellow sticky board which
a Qualcomm eight-core processor with Android 2.3 system, has been extracted are not fixed. In order to improve the
API 25. It is a common type with satisfactory parameters. accuracy of aphid identification and counting, the yellow
The chapter contains two parts of picture distillation; in sticky board must be corrected to the same size and the same
the first part, the GrabCut algorithm [12] is used to extract position when a target image has been segmented; getting the
yellow sticky board image from complex background and elevation view, the aphids can be counted accurately.
correct it to the same size and position by perspective Transformation from distortion to elevation view can
correction method in order to extract aphid image [13]; in be completed by using perspective correction method [14]
the second part, the method of weighted average gray scale which uses the four vertexes of distortion image and elevation
is used to make the yellow sticky board image preprocessing, view as the correction basis. The top left vertex (𝑥, 𝑦) is the key
single image difference method is used to eliminate the of correction. Firstly, the corrected image matrix [𝑥2 𝑦2 𝑧2 ]
influence of illumination, and OSTU threshold segmentation is obtained by using the distorted image matrix [𝑥1 𝑦1 𝑧1 ];
and morphological erosion method is used to extract yellow you can get the top left vertex (𝑥, 𝑦) by using the relationship
sticky board image, laying the groundwork for the subsequent between [𝑥1 𝑦1 𝑧1 ]. Secondly, according to top left vertex,
counting program. the length and width of the rectangle can be used to obtain
the information of four vertices.
2.1. Extraction the of Yellow Sticky Board Image. GrabCut 𝑎11 𝑎12 𝑎13
is an improved algorithm based on GraphCut. GrabCut is [ ]
𝐷 = [𝑥2 𝑦2 𝑧2 ] = [𝑥2 𝑦2 𝑧2 ] [𝑎21 𝑎22 𝑎23 ] ,
an algorithm that uses the RGB three-channel hybrid Gauss
model for color segmentation; segmentation can be com- [𝑎31 𝑎32 𝑎33 ]
pleted by using region pixel set which contains background. 𝑥 𝑎 𝑥 + 𝑎21 𝑦1 + 𝑎31 (5)
The algorithm belongs to iterative segmentation, and the 𝑥 = 2 = 11 1 ,
𝑧2 𝑎13 𝑥1 + 𝑎23 𝑦1 + 𝑎33
segmentation accuracy is higher. Gibbs energy function is a
specific vocabulary that means a formula that can calculate 𝑦2 𝑎12 𝑥1 + 𝑎22 𝑦1 + 𝑎32
𝑥= = .
the weights and cost. 𝑧2 𝑎13 𝑥1 + 𝑎23 𝑦1 + 𝑎33
𝐸 (𝛼, 𝑝, 𝜃, 𝑧) = 𝑈 (𝛽, 𝑝, 𝜃, 𝑧) + 𝑉 (𝛼, 𝑧) . (1) Figure 1 is the elevation view of yellow sticky board by
using GrabCut and perspective correction method. We can
see that the picture has a clear boundary and regular shape
𝑈 is a method when the target pixels are classified as which will make the aphids extracted easily.
foreground or background models, which are obtained by
matrix 𝐷 iteration of classifying and rewriting pixel: 2.3. The Image of the Yellow Sticky Board Preprocessing. The
color image change into gray image can reduce amount of
𝑈 (𝛼, 𝑝, 𝜃, 𝑧) = ∑ 𝐷 (𝛼𝑛 , 𝑝𝑛 , 𝜃, 𝑧𝑛 ) , computation in the image processing process and reduce the
𝑛 complexity of the calculation [15, 16], the gray scale value
1 obtained by the weighted gray scale method is more in line
𝐷 (𝛼𝑛 , 𝑝𝑛 , 𝜃, 𝑧𝑛 ) = log 𝜋 (𝛼𝑛 , 𝑝𝑛 ) + log det ∑ (𝛼𝑛 , 𝑝𝑛 ) with the perception of the corresponding gray scale of each
2 (2) pixel, and the brightness is moderate; it is conducive to the
1 −1 next aphid image extraction.
+ [𝑧 − 𝜇 (𝛼𝑛 , 𝑝𝑛 ) ]
2 𝑛
−1 −1 2.4. Aphids Extraction
⋅ ∑ (𝛼𝑛 , 𝑝𝑛 ) [𝑧𝑛 − 𝜇 (𝛼𝑛 , 𝑝𝑛 )] .
2.4.1. Image Difference. Due to the influence of light, even
𝑉 is the boundary energy smoothing term, when the if more than two thresholds segmentations are proposed;
boundary pixels are discontinuous; it will do some boundary each of the pictures taken in the natural environment has
pixel classification processing: an uncertain gray value, so the threshold is uncertain too.
Aphid color is relatively single, so the impact of light is
2 mainly reflected in the background; an image can be seen to
𝑉 (𝛼, 𝑧) = 𝛾 ∑ [𝛼𝑥 , 𝛼𝑦 ] exp − 𝛽 𝑧𝑥 − 𝑧𝑦 . (3) consist of foreground and background; when the background
(𝑥,𝑦)∈𝑐 is removed, the influence of light is removed. Regarding the
Journal of Sensors 3
Figure 1: Perspective correction. Figure 2: Image difference method for extracting aphids.
Table 1: Part of the connection domain number, pixel values, and Main
the relationship between the lengths of pests. interface
the parameters of nuclear value Size (𝑤, ℎ) as Size (51, 51); 110
then use the Imgproc.GaussianBlur method to complete
100
the process of fuzzy background. Use Core.absdiff() for
differential operation. Use the Imgproc.THRESH OTSU for 90
threshold operation after difference background image. The
differential images need to undergo morphological corrosion 80
to remove the effects of the aphids. Set the morphological 70
action as MORPH ERODE and set the shape of structure
element kernel1 as MORPH ELLIPSE with a new Size (5, 5) 60
by using getStructuringElement(). Elimination of adhesions 50
will be completed.
50 60 70 80 90 100 110 120 130
(iii) Image Counting Module. The image counting module APP counts number
filters the contours based on the total pixel range in each
Greenhouse
connected domain and stores the count on the counter. The
Orchard
output value is the number of aphids. Linear fit of greenhouse
The counting process can be seen as a traversal of the
“funnel”; then we should define two containers: the first con- Figure 6: The comparison of accuracy between the manual count
tainer is stored in all the contour data, whether or not meeting and the APP count.
the characteristics of the requirements, and the second con-
tainer is stored after the screening of the contour information.
Traverse the contour type to Imgproc.RETR CCOMP and the counting results of the aphids in the greenhouse, and
output it as Imgproc.CHAIN APPROX SIMPLE. The “for- Figure 5(b) shows the counting results of the aphids in the
cycle” is the key of selecting. Set 15 px to 18 px as the shape of orchard.
aphids image in the connected domain and set scallar (255, Based on the least squares method, a linear regression
255, 255) to show aphids as white. analysis has been carried out by taking 10 sets of data from
Table 2 in the greenhouse to get the fitting curve. 𝑥-axis is
3. Results and Discussions the APP counts number and 𝑦-axis is the manual counts
number. As shown in Figure 6, the relationship of the fitting
Using the aphid counting software to count the images in between the mobile APP count and the manual count is
greenhouse and the images in orchards, Figure 5(a) shows 𝑦 = 1.04846𝑥 − 2.29425; the fit degree 𝑅2 factor is up to
6 Journal of Sensors
Image source Image number Number of manual counts/month APP counts number/month Relative error/% Accuracy/%
(1) 52 52 0 100
(2) 80 79 1.25 98.75
(3) 62 61 1.61 98.38
(4) 84 82 2.38 97.61
(5) 86 87 1.16 98.83
Greenhouse
(6) 55 53 3.7 96.22
(7) 54 54 0 100
(8) 69 67 2.9 97.01
(9) 70 70 0 100
(10) 121 116 4.13 95.86
(11) 71 66 7.04 92.95
Orchard
(12) 138 128 7.24 92.75
0.99504, indicating that the aphid recognition and mobile and convenient. The method proposed in this paper can
APP counting can be used to achieve functions of recognition extract and correct the image of yellow sticky board, eliminate
and counting. Adding two sets of orchard data from Table 2 influence of illumination on the image, and count aphids
into the coordinate system, we can find that the two points accurately.
are in the vicinity of the curve, indicating that the method is Due to the limitations of the season and the study site, the
basically effective. Because the orchard environment is more aphid counting software needs to design a memory storage
complex, the accuracy rate of the method is slightly worse module to achieve the same number of yellow sticky insect
than the greenhouse; as shown in Table 2, the greenhouse boards to make trend prediction function.
accuracy rate is up to 95% and orchard accuracy rate is up
to 92%. In general, the aphids identification and counting Conflicts of Interest
method is highly reliable.
The authors declare that they have no conflicts of interest.
4. Conclusion
Acknowledgments
In the past, aphid identification focuses on the separation of
a single kind of aphids, few on different kinds of pests. The This work is supported by the new facilities of agricultural
count number is less, and the aphids’ antennae, foot, and machinery research and development (17227206D).
wings need to be extracted precisely, because the operation
complexity is high; most of the application software that References
implements the aphid recognition and counting function is
concentrated on the PC side. The traditional counting APP [1] X. Zhan-li, L. Xiao-xia, Z. Qing-wen et al., “Characteristics of
collects the photos through the Internet to transfer data by aphid age identification of wheat long pipe,” Acta Entomologica
the PC processing, and then the results return back to the Sinica, vol. 01, pp. 81–87, 2014.
mobile App, the process Overreliance on the PC side and [2] R. Cassani, T. H. Falk, F. J. Fraga, P. A. Kanda, and R. Anghinah,
the Internet; its real time performance is poor. Collection of “Towards automated EEG-Based Alzheimer’s disease diagnosis
aphids requires special equipment; this professional equip- using relevance vector machines,” in Proceedings of the 5th
ment is not convenient and also very expensive. The device ISSNIP - IEEE Biosignals and Biorobotics Conference (BRC ’14),
that collects the aphids is not universal. pp. 1–6, IEEE, May 2014.
This paper uses yellow sticky board in the natural [3] S. Chakraborty and A. C. Newton, “Climate change, plant
diseases and food security: an overview,” Plant Pathology, vol.
environment to attracts aphids and some other pests, and
60, no. 1, pp. 2–14, 2011.
collects images in the real growth environment of plants.
[4] M. Pal and G. M. Foody, “Evaluation of SVM, RVM and SMLR
Aphid counting software overcomes the complex natural
for accurate image classification with limited ground data,” IEEE
environment and the impact of light caused by image pro- Journal of Selected Topics in Applied Earth Observations and
cessing difficulties; it does not rely on the PC side and the Remote Sensing, vol. 5, no. 5, pp. 1344–1355, 2012.
network feedback; it can quickly identify and count aphids, [5] H. N. Patel, R. K. Jain, and M. V. Joshi, “Fruit detection using
which are the representative of the pest group; it has very improved multiple features based algorithm,” International
practical value. The accuracy of the aphid counting software Journal of Computer Applications, vol. 13, no. 2, pp. 1–5, 2011.
is compared with the result of manual counting; the accuracy [6] J. Luo, D. Wang, Y. Dong, W. Huang, and J. Wang, “Developing
rate of the aphid counting software is more than 95% in an aphid damage hyperspectral index for detecting aphid
the greenhouse environment; the accuracy rate of the aphid (Hemiptera: Aphididae) damage levels in winter wheat,” in
counting software is more than 92%. The operation is smooth Proceedings of the 2011 IEEE International Geoscience and
Journal of Sensors 7
Rotating
Machinery
International Journal of
The Scientific
(QJLQHHULQJ Distributed
Journal of
Journal of
Journal of
Control Science
and Engineering
Advances in
Civil Engineering
Hindawi Publishing Corporation Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014 http://www.hindawi.com Volume 2014
Journal of
Journal of Electrical and Computer
Robotics
Hindawi Publishing Corporation
Engineering
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014 http://www.hindawi.com Volume 2014
VLSI Design
Advances in
OptoElectronics
,QWHUQDWLRQDO-RXUQDORI
International Journal of
Modelling &
Simulation
$HURVSDFH
Hindawi Publishing Corporation Volume 2014
Navigation and
Observation
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
in Engineering
Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014
(QJLQHHULQJ
+LQGDZL3XEOLVKLQJ&RUSRUDWLRQ
KWWSZZZKLQGDZLFRP 9ROXPH
Hindawi Publishing Corporation
http://www.hindawi.com
http://www.hindawi.com Volume 201-
International Journal of
International Journal of Antennas and Active and Passive Advances in
Chemical Engineering Propagation Electronic Components Shock and Vibration Acoustics and Vibration
Hindawi Publishing Corporation Hindawi Publishing Corporation Hindawi Publishing Corporation Hindawi Publishing Corporation Hindawi Publishing Corporation
http://www.hindawi.com Volume 2014 http://www.hindawi.com Volume 2014 http://www.hindawi.com Volume 2014 http://www.hindawi.com Volume 2014 http://www.hindawi.com Volume 2014