A Tomato Leaf Diseases Classification Method Based on Deep Learning

Ding Jiang, Fudong Li, Yuequan Yang, Song Yu

College of Information Engineering, Yangzhou University, Yangzhou 225127
E-mail: [email protected], [email protected], [email protected], [email protected]

Abstract: In the process of planting crops, the detection of diseases in the leaf parts is one of the key links to the pre-
vention and control of crop diseases. This paper takes tomato leaves as experimental objects, and uses the deep learning
method to extract the disease features on leaf surface, including three most common species (Spot blight, Late blight and
Yellow leaf curl disease). After continuous iterative learning, the network can predict the category of each disease pic-
ture. For each of the three diseases, 1000 pictures were selected, divided into 900 pictures for training set (2700 in total)
and 100 pictures for test set (300 in total). The experiment takes Resnet-50 as the basic network model. For comparison,
the activation function of the network was changed into Leaky-ReLU and the kernel size of the first convolutional layer
was modified to 11×11. After the improvement, the training accuracy in training set is 98.3% (increased by 0.6%) and the
test accuracy in test set is 98.0% (increased by 2.3%).
Key Words: Leaf Disease Classification, Feature Extraction, Deep Learning

tified four common wheat diseases. S.D.KHIRADE et al. [4]

1 INTRODUCTION discussed the possibility of using leaf images to detect plant
With the development of modern ecological agriculture, diseases, as well as some segmentation and feature extrac-
more attention has been paid to crop yield and quality. The tion algorithms used in plant disease detection.
incidence of crops has increased year by year, and the types However, all the disease detection methods mentioned
of diseases have become more and more complicated. If above are based on traditional image processing and clas-
only relying on professionals to observe the disease of crops sification algorithms. Most of these features are artificially
with naked eyes in the planting area, it is easy to be affected designed and extracted. For different problems, the ex-
by environmental factors and human factors, and it relies tracted features will be different. Some machine learning
heavily on professional experience. More importantly, methods can get a good identification effect, but they still
automatic monitoring and processing of diseases cannot be have poor flexibility and generalization ability when the
achieved. Therefore, it is particularly important to study blade disease is not obvious or the data set is large, which
more stable and efficient methods for the prevention and makes it difficult to achieve accurate identification effect on
diagnosis of crop diseases. practical applications.
In the application research of crop disease detection, tradi- 2 RELATED WORK
tional computer vision methods mostly need to segment leaf
lesions, such as pixel-level segmentation, edge segmenta- In recent years, with the rapid development of computer
tion, region segmentation and multi-scale segmentation. vision and deep learning methods, the accuracy and flexi-
Noise reduction, corrosion, enhancement and other tech- bility of image recognition have been greatly improved.
niques are applied to process image color space features and Deep learning has the advantage of extracting classification
texture features, and then appropriate lesion features and features directly without the need to design classifiers [5].
classifiers are chosen for detection. Srivastave A et al. [1] Deep learning method is suitable for classification in vari-
used a statistical threshold method to achieve image seg- ous occasions and has a strong generalization, especially for
mentation of grape diseased leaves in a three-dimensional the extraction of complex features or special features.
color space, and then judged downy mildew based on color
Liu Y Y et al. [6] of Gansu Agricultural University used the
differences. Wang S Z et al. [2] proposed a kernel K-means
regional Faster-RCNN model to train the collected images,
clustering algorithm for plant leaf disease identification,
and classified and identified the images by classifier. The
using vector median filtering to remove noise, extracting
accuracy of diseased leaf recognition was up to 75.52% and
leaf disease feature vectors, and mapping input space sam-
the lowest was 60.56%. In 2017, Ramcharan et al. [7] of the
ples to high-dimensional feature space for K- Mean clus-
Pennsylvania State University of the United States applied a
tering and plant disease identification. Wang M L et al. [3]
learning migration training deep convolutional neural net-
converted the RGB color space to the HSV space to extract
work to identify three diseases of cassava and two diseases
color features and geometric features, and effectively iden-
of pests. Srdjan S et al. [8] first distinguished the plant
leaves and the surrounding environment, and then identified
This work is supported by National Nature Science Foundation under 13 different types of plant diseases collected on the Internet.
Grant hhhhhhh

978-1-7281-5855-6/20/$31.00 2020
c IEEE 1446

Authorized licensed use limited to: Cornell University Library. Downloaded on August 24,2020 at 02:46:50 UTC from IEEE Xplore. Restrictions apply.
The Caffe frame was used, with a maximum accuracy of leaf diseases were all up to more than 2000, with a total of
98.21%. 6,794 images. These images will form a sample library for
the learning process [11]. Fig 2 shows some examples of the
3 PROPOSED METHODS processed images.
In this paper, tomato leaves were taken as the experimental
object, and Resnet-50 residual network was adopted as the
basic model. In the experiment, the feature of leaf disease
position was automatically extracted by the convolutional
layers, and the disease classification was finally determined
after iterative learning. In addition, random data augmen-
tation was carried out to prevent over-fitting in the exper-
iment. Leaky-ReLU [9] activation function and the larger
11×11 convolution kernel size were used to modify the
network, which increase the receptive field [10] and im-
prove the ability of the network to capture detailed features.
Through comparative experimental analysis, the perfor-
mance of the network is increased by 2.3% in test set.

4 DATA
As one of the most common agricultural crops, tomato is
Fig 2. Examples of the processed images (including Gaussian
widely cultivated in north and south of China. With the
noise and Affine transformation)
development of ecological agriculture, higher requirements
have been put forward for the detection and prevention of 5 NETWORK MODEL AND PARAMETERS
crop diseases. In the experiment, a total of 3000 images of
the three most common tomato leaf diseases (Spot blight, 5.1 Deep Residual Network
Late blight and Yellow leaf curl) were selected. The training As the depth of the network increases, there is a degradation
set and test set were allocated according to the ratio of 9:1, problem, which is that as the network gets deeper and
including a total of 2700 training pictures and 300 test deeper, the accuracy of the training flattens out, but the
pictures. Among them, leaves occupy the main part of the training errors get bigger. To overcome this problem,
picture, avoiding the messy texture background, so as to Kaiming He et al. [12] proposed the deep residual network
achieve better disease feature extraction effect. The data and achieved excellent results in the ILSVRC2015 compe-
were obtained from AI Challenger and some examples are tition.
shown in Fig 1.
Resnet-50 network model takes Bottleneck structure as
residual module. Direct connection channels, or short-cut
structures, have been added to the network, allowing a
certain proportion of the previous network layer's output to
be retained. This simple addition does not add any extra
parameters or computations to the network, but can greatly
increase the training speed of the model with better training
effect. When the network goes deeper, this structure can
solve the problem well that the gradient disappears during
back propagation.
The distribution of input data in each layer of the network is
changing all the time. The update of training parameters in
the front layer will lead to the change of the distribution of
input data in the later layer. Moreover, the first few layers of
the network make small changes, and the next few layers
Fig 1. Examples of three common tomato leaf diseases. Top: Spot
gradually accumulate and amplify the changes. The role of
blight. Middle: Late blight. Bottom: Yellow leaf curl.
Batch Normalization is to standardize these input values and
Before training, the image resolution is uniformly scaled to reduce the scale difference to the same range, which alle-
224×224 to ensure the rationality of network input. Due to a viates the impact on the back-layer network.
small number of training samples, random data augmenta-
The default image input size for the network is 224×224, so
tion was carried out on the selected tomato leaf pictures in
this is the reason for the uniform size of the data before (see
order to prevent the over-fitting phenomenon in training
Section 4). The convolution kernel size of the first layer is
process.
7×7, which is used to extract the primary characteristics of
Gaussian noise and Affine transformation were chosen to the pictures. The figure features then get into the bottleneck
process the leaf data in this paper, which is helpful to ex- residual block structure for deeper and higher level features
pand the amount of data. Each kind of processed pictures of extraction. For each convolutional layer, it is followed by a

2020 Chinese Control And Decision Conference (CCDC 2020) 1447

Authorized licensed use limited to: Cornell University Library. Downloaded on August 24,2020 at 02:46:50 UTC from IEEE Xplore. Restrictions apply.
 Fig 3. Schematic diagram of Resnet-50.
Batch Normalization layer and a ReLU activation function the receptive field of feature extraction and improve the
layer to improve the convergence speed. Batch Normaliza- network's ability to capture feature details of diseases.
tion of data is carried out in each hidden layer. This reduces
The learning rate directly determines the convergence effect
the problem of different distribution between practical
of the network. The weights are randomly initialized while
application pictures and training pictures, and makes each
initial learning rate was set to 0.001, which makes the gra-
layer of the networks relatively independent. Generally, the
dient descent at a relatively fast speed at the beginning of
data at the end of the network structure (the fully connected
training. When iteration reaches a certain number, the
classification layer) maps the output results to the interval of
learning rate is decayed to 0.0001, which is 1/10 of the
(0, 1) by Softmax function, so as to calculate the loss func-
origin. With the gradual decrease of learning rate, the
tion. Fig 3 shows the model.
training process becomes relatively slow, and the gradient
Loss function is generally composed of several parts. For
descent gets closer to the minimum error point more accu-
classification problems, cross entropy function is most
rately, so that sufficient convergence can be achieved.
commonly used as the error cost. In fact, L2 regularization
term is usually added in the training process to prevent Batch Gradient Descent (BGD) was used in the experiment,
over-fitting. The loss function is defined as and the number of pictures in each small batch was set to
e xi 16(due to the limitations of GPU memory size). In consid-
f ( x)i = xj
, (1) eration of the small batch-size, Batch Normalization is not
¦j e used in the actual experiment.
n
Loss( x, y, ω ) = −¦ yi log f ( x) i + λ ω 2 , (2)
2 6 EXPERIMENT
i =1 6.1 Training Methods and Procedures
where f (x)i is the i-th item in classification vector after the
A total of 6,794 images were included in the training set
Softmax function, which means the i-th actual output after data augmentation, and three common tomato leaf
probability. yi is the expected output probability and Ȝ is diseases were trained and classified. In the experiment, the
the Regularization factor. acquired training pictures were sorted and classified in
5.2 Parameter selection advance and placed in the folder corresponding to the name
of the disease category, so that the respective names were
ReLU activation function (modified linear element) is taken as the category "labels". Different activation functions
adopted in the Resnet-50 model. Compared with the tradi- and convolution kernel sizes were compared with the
tional nonlinear activation function, ReLU avoids the original Resnet-50 network. Before training, the weight
problem of slow learning rate in deep back propagation. decay was 0.00001 and the Adam optimizer was used for
However, ReLU itself may also cause certain inactivation iterations. The training results were recorded every 20 it-
problems, that is, some neurons may not be activated, which erations and the training model was saved every 100 itera-
means certain parameters cannot be updated. Consider the tions. The code was run on Ubuntu16.04 with tensorflow
potential impact of this problem, we tried to modify it to while a NVIDIA RTX2060 graphics card is used for GPU
Leaky-ReLU activation function, and this function was acceleration. The training process and procedure is shown
applied with a certain slope in the negative interval, so that in figure 4.
the neurons in this interval could maintain activity. In the
initial stage of feature extraction, 7×7 convolution kernel 6.2 Results
size was used in the first convolutional layer. In this paper, Firstly, the experiment is done on the original Resnet-50
we changed it to 11×11 convolution kernel size to increase model, while the ReLU activation function was modified to

1448 2020 Chinese Control And Decision Conference (CCDC 2020)

Authorized licensed use limited to: Cornell University Library. Downloaded on August 24,2020 at 02:46:50 UTC from IEEE Xplore. Restrictions apply.
 Fig 4. Training flow chart. Mini-batch=16. About 10000 iterations.
Leaky-ReLU and the kernel size of the first convolutional
layer was changed to 11×11 for comparative experiments.
The purpose of this approach is to reduce the impact of
ReLU inactivation and improve the network performance to
some extent by enhancing the ability to capture detailed
features. Fig 5 shows the comparison of the change diagram
of loss value for the three experiments. Loss value is rec-
orded every 200 iterations.

Fig 7. Training accuracy. Leaky-ReLU with 7×7 kernel size.

Fig 5. Training loss. Blue curve: ReLU with 7×7 kernel size;
Green curve: Leaky-ReLU with 7×7 kernel size; Red curve:
Leaky-ReLU with 11×11 kernel size.
Fig 6, Fig 7 and Fig 8 (smoothed from tensorboardX) rec-
ords the training accuracy curve of the three experiments,
with a total of 24 epochs.
Fig 8. Training accuracy. Leaky-ReLU with 11×11 kernel size.
In the third experiment, the activation function was replaced
with Leaky-ReLU and the size of the convolution kernel in
the first layer was changed to 11×11. Compared with the
performance of the original network, the modified network
converges faster in the initial stage and converges better in
the later stage of training (see Fig 5 and Fig 8).
Table 1 records the accuracy and experiment time of the
three experiments on the training set and test set. Table 2
records the specific number of misclassified samples. The
data in the table is from the log file after the test program,
which records the classification results of each test picture.
For each leaf disease picture, the highest probability will be
Fig 6. Training accuracy. ReLU with 7×7 kernel size. the final predicted category and placed in the first place.

2020 Chinese Control And Decision Conference (CCDC 2020) 1449

Authorized licensed use limited to: Cornell University Library. Downloaded on August 24,2020 at 02:46:50 UTC from IEEE Xplore. Restrictions apply.
 Table1. Accuracy and Time Second, the segmentation and localization of the disease
area can be combined to make the effect more reliable and
Experiments Train (%) Test (%) Time(min)
abundant. Finally, due to the lack of a relatively complete
ReLU,7×7 97.7 95.7 51 and high quality crop disease database, the classification of
L-ReLU,7×7 98.1 97.3 53 some rare diseases or species is still not ideal. How to es-
tablish a more comprehensive and high quality disease data
L-ReLU,11×11 98.3 98.0 54
system is still the most important part of crop disease pre-
Table2. Misclassified samples vention.

Error Spot blight Late blight Yellow leaf curl REFERENCES

13 4 2 7 [1] Srivastava A, Ma S, Inoue K. Development of a sensor for automatic
detection of downey mildew disease [J]. 2004:562 - 567.
8 1 3 4
[2] WANG S Z, HE D J, LI W, et al. Plant leaf disease identification
6 2 1 3 based on kernel k-means clustering algorithm [J]. Agricultural machinery,
2009, 40(3): 152-155.
According to the experimental results in table 1, this method
[3] WANG M L, NIU X J, ZHANG H M, et al. Research on feature
achieved some improvement on the network by using extraction and recognition of common diseases of wheat leaf [J]. Com-
Leaky-ReLU activation function. In addition, after modi- puter Engineering and Applications, 2014, 50(7): 154 – 157.
fying the convolution kernel to a larger size of 11×11, the
[4] KHIRADE S D, PATIL A B. Plant disease detection using image
performance of the convolution kernel was further im- processing[C]// Proceedings of the 2015 International Conference on
proved compared with the original network, achieving Computing Communication Control and Automation. Piscataway: IEEE
98.3% accuracy in the training set and 98.0% accuracy in Computer Society, 2015: 768 – 771.
the test set, respectively. Generally speaking, expanding the [5] JIA S P, GAO H J, HANG X. Advances in image recognition tech-
size of the convolution kernel can provide a larger receptive nology of crop diseases and insect pests based on deep learning [J].
field, but it will also lead to the problem of bigger calcula- Chinese journal of agricultural machinery, 2019, 50(S1):313-317.
tion of the convolution process, which affects the network [6] LIU Y Y,FENG Q,YANG S. Detecting grape diseases based on
training speed to some extent. However, it is not difficult to convolutional neural network [J].Journal of Northeast Agricultural Uni-
find that the training time in experiment 3 is not greatly versity, 2018, 49( 3) : 73 – 83.
affected due to the small-scale sample training samples and [7] RAMCHARAN A, BARANOWSKI K, MCCLOSKEY P, et al. Deep
stable GPU acceleration. As can be seen from table 2, the learning for image-based cassava disease detection [J]. Frontiers in Plant
wrong disease classification is mainly focused on the Yel- Science, 2017, 8: 1852.
low leaf curl while the identification errors of Spot blight [8] Srdjan S, Marko A, Andras A, et al. Deep Neural Networks Based
and Late blight are relatively few. Recognition of Plant Diseases by Leaf Image Classification: [J]. Compu-
tational Intelligence and Neuroscience, 2016, (2016-6-22), 2016,
7 CONCLUSION 2016(6):1-11.
[9] Zhang X, Zou Y, Shi W. Dilated convolution neural network with
In this paper, a classification method of tomato leaf diseases Leaky-ReLU for environmental sound classification[C]// 2017 22nd
based on deep learning was introduced to identify and International Conference on Digital Signal Processing (DSP). IEEE, 2017.
classify three common tomato leaf diseases: Spot blight,
[10] Bolei, Zhou, Aditya Khosla, Agata Lapedriza, Aude Oliva, and
Late blight and Yellow leaf curl. Based on the improved Antonio Torralba. "Object detectors emerge in Deep Scene CNNs." 2015
Resnet-50 model, 6794 leaf pictures (after data augmenta- International Conference on Learning Representations, May 7-9, 2015.
tion) of three different diseases were selected and prepared
[11] Zhang H, Cheng Q, Wu Y J, Wang Y X, Zhang C M, Yin f w. a wheat
for batch learning and training. After about 24 epochs, the disease identification method based on convolutional neural network [J].
recognition accuracy of the three diseases reached 98.3% in Shandong agricultural science, 2018, 50(03):137-141.
training set and 98.0% in test set, respectively. [12] K. He, X. Zhang, S. Ren, J. Sun, "Deep residual learning for image
The pictures were sorted out in advance and named uni- recognition", Computer Vision and Pattern Recognition (CVPR), 2016.
formly in one folder, instead of making labels for each one [13] Yan B, Zhou P, Yan L. Identification of small sample crop diseases
by hand. At the same time, a small number of pictures are based on migration learning [J]. Modern agricultural science and tech-
nology, 2019(06):87-89.
easy to obtain and handle in real practice, making training
process more flexible and efficient [13]. It has certain
practical value for the development of modern ecological
agriculture. In addition, this method can be applied to many
situations. For other crops with analogous disease features,
it can also achieve good results with a certain amount of
pictures collected and processed.
Of course, this paper may make some improvements. First
of all, for the classification of multiple diseases of a single
species, the experiment has achieved a certain effect. In the
future, efforts should be made in the direction of simulta-
neous detection of multiple species and multiple diseases.

1450 2020 Chinese Control And Decision Conference (CCDC 2020)

Authorized licensed use limited to: Cornell University Library. Downloaded on August 24,2020 at 02:46:50 UTC from IEEE Xplore. Restrictions apply.