2020 International Conference on Contemporary Computing and Applications (IC3A)

Dr. A.P.J. Abdul Kalam Technical University, Lucknow. Feb 05-07, 2020

Agribot: A Natural Language Generative Neural

Networks Engine for Agricultural Applications
Bhavika Arora* Dheeraj Singh Chaudhary Mahima Satsangi
Dept. of Electrical Engineering Dept. of Electrical Engineering Dept. of Electrical Engineering
2020 International Conference on Contemporary Computing and Applications (IC3A) 978-1-7281-5432-9/20/$31.00 ©2020 IEEE 10.1109/IC3A48958.2020.233263

Dayalbagh Educational Institute Dayalbagh Educational Institute Dayalbagh Educational Institute

Agra, India Agra, India Agra, India
[email protected] [email protected] [email protected]

Mahima Yadav Lotika Singh Prem Sewak Sudhish

Dept. of Electrical Engineering Member, IEEE Senior Member, IEEE
Dayalbagh Educational Institute Dept. of Physics & Computer Science Dept. of Physics & Computer Science
Agra, India Dayalbagh Educational Institute Dayalbagh Educational Institute
[email protected] Agra, India Agra, India
[email protected] [email protected]

Abstract—In this paper, we present an artificially Chatbots can be classified into two categories:(i) rules-
intelligent chatbot which would help farmers by providing based, operating by means of specific commands (or
solutions to various farming related problems and facilitate keywords), which generally obey well-defined navigation
their decision-making process. The bot not only provides flows and produce targeted conversations; (ii) AI-based,
answers to frequently asked questions but also lays emphasis making use of more advanced technologies such as machine
on crop disease detection and weather forecasting. We learning, NLP, among other artifices to increase its capacity
developed an end to end trainable sequence-to-sequence for dialogue and interaction.
learning model with the objective of achieving conversational
task-oriented system based on minimal assumption on its Other than this, division of chatbots is also carried out
sequence structure. Our approach exploits a multilayered into the categories of conversational and task-oriented bots
Long Short-Term Memory (LSTM) unit which maps the input based on the intended use case. Conversational bots aim to
sequence to a corresponding output sequence by converting it simulate the general conversation to entertain the user,
into a vector of fixed dimensionality in between. To achieve the without any target goal. The more the bot is able to indulge
disease detection, a Convolutional Neural Network the user with itself, the more is the performance level of the
architecture is implemented in which a multilayered bot. A task-oriented bot aims to interpret and perform the
architecture is developed and trained from scratch which action the user is requesting as fast as possible. The task
would classify the plant images into different classes. For
includes placing an order, scheduling an event or
conversational system module we have used the Kisan Call
troubleshooting. Replacement of humans with task-oriented
Center (KCC) dataset which contains logs of calls at KCC by
farmers whereas for disease detection module plant village
bots in industries is its main advantage.
dataset is used. After training 98% accuracy was achieved for Further distinctions include Retrieval-based and
conversational system module on training data and 94% for Generative models. Retrieval-based models have a repository
disease detection module on test data. of pre-defined responses they can use. Unlike, generative
models can generate the responses they’ve never seen before
Keywords—End-to-End Trainable Task-Oriented Systems,
[3]. The drawback with retrieval-based system is that we
Recurrent Neural Networks, Sequence-to-Sequence Learning,
have to anticipate all possible use cases and construct a set of
Memory Networks
all possible answers that are specific to this set. Therefore,
I. INTRODUCTION we devised a generative model which requires a large
amount of training data of human interactions to be able to
Farmers face a lot of issues - be it how to combat specific respond to an unseen input.
crop diseases, fertilizers choices, pest infestations or
unpredictable weather. There are KCC units to answer all For agricultural purposes, it is important that the data
these type of queries. But “They (KCC) don’t pick the call, about field conditions, such as air and soil temperature,
ever”. Also advice received on phone calls are hard to weather forecast, soil moisture, disease detection capabilities
remember. Another option for farmer is to find the solution and other relevant variables, be rapid and easily available for
through google apps but the problem is “You ask one thing use by farm management systems, by specialists, or the
(to Google) and get thousand responses. What to do!” [1]. In farmer itself in the decision-making process. For this
order to overcome the above mentioned issue we have purpose, a chatbot was developed using sequence-to-
devised an interactive chatbot for farmers. sequence learning, a type of generative approach [4]. The
model learns a mapping between questions and its
Chatbots have existed since 1960’s [2]. Initially chatbots appropriate responses which helps in evaluating strengths
used only simple keyword matching technique which is now and weaknesses of the system.
replaced with different techniques such as pattern matching,
keyword extraction, natural language processing, machine The system developed involves the creation of three
learning, and deep learning. modules – conversation system module, disease detection
module and weather prediction module. One more is the top
*This is the corresponding author.

978-1-7281-5432-9/20/$31.00 ©2020 IEEE 28

module connecting all the three modules. The concepts of all II. RELATED WORKS
the modules are described in the section 4 and 6 of this A few conversational systems, crop disease detection
research paper. The rest of the paper is organized as follows. applications and weather forecast based advisory systems
Section 2 is devoted to related work. Section 3 provides an can be seen in references [1, 4-8] presented in Table 1.
overview of dataset. Sections 4 and 5 describe the
methodology used, observations and inferences. Section 6
presents the results. Finally, Section 7 presents the
conclusions and future work.

TABLE I. OVERVIEW OF PREVIOUS WORKS

Target
Author Method Result Limitation/ comment
application
Proposal to model responding
style of human and then
generating personalized
outputs. Also introduced the
Neural
new scheme called domain
Personalized Introducing contextual
Wei-Nan Zhanget Sequence to Sequence adaptation scheme.
Response information will improve the
al. [5]. Learning, Neural
Generation as results of personalized
Networks Comparison of responses
Domain responding models.
generated with novel human
Adaptation
in an offline mode and with
real time conversations in an
online model for evaluation of
the model.
Used multilayered LSTM to
Sequence to map input sequence to output
A Phrase based statistical
Sequence Learning sequence.
Ilya Sutskever [4] Deep Neural Networks system achieves less BLEU
with Neural Compared LSTM system and
score than a LSTM system.
Networks phrase based statistical
system.
Using Deep
Deep convolution neural Large no of parameters,
Mohammed Learning for 39 class, accuracy up to
networks, transfer High computation time, No
Brahimi et al. [6] Image-Based Plant 99.76%.
learning integration with chatbot.
Disease Detection
Profile Collector Module
Improving Web Large no. of areas through
(PCM), Context Manager
Andy M. Connor Information contextual knowledge base Data analysis required for
Module (CMM), Result
et al. [7] Retrieval Using semantic differential and the aspect validation.
Analyser (RA),
Shared Contexts Likert scale value of ‘five’.
Contextual Profiles
Development of A
REST API For
Forecast (temperature,
Obtaining Site- REST API, Postman
humidity), observation was No cloud building energy
Hu Du et al. [8] Specific Historical environment (JavaScript
conducted using a local simulation application.
and Near-Future language)
weather station.
Weather Data in
EPW Format
FarmChat: A
Natural Language Speech based conversational No feature of disease
Mohit Jain et al. Conversational
Processing, Google system to resolve farmers detection with image of
[1] Agent to Answer Translator, IBM Watson queries diseased plant given as input
Farmer Queries
Integrated three modules –
conversational system
module, disease detection
Training own
module, weather forecasting
architecture from scratch Answers to agriculture-based
to one application.
using CNN and seq2seq queries, detect disease if
- Present Study
model applying rms image is given as input and
Our Agribot can predict the
propagation and Adam also forecast weather.
disease of the plant when
optimizer respectively.
given image as an input
whereas Farmchat doesn’t
have feature of this kind.

29
 III. DATASETS Village organization [14]. It is a public dataset of 54,306
In this section, we will discuss about the datasets used images of diseased and healthy plant leaves collected under
both for conversational system module and disease detection controlled conditions. Each class label is a crop-disease pair,
module. and we make an attempt to predict the crop-disease pair
given just the image of the plant leaf. The data was analyzed
A. Dataset Creation For Conversational System Module with the help of python and Jupyter Notebook. The details
Datasets play a major role in Artificial Intelligence. The of the dataset are as follow.
type of training is one of the factor on which the efficiency x No of classes: 37
of the system depends. The quality of training depends on x No of species: 13
the quality of datasets. Datasets in every field are not readily x No of diseases present in the dataset: 21
available. Different forums are used in order to fulfill general
x No of training images present in the dataset:
conversational systems. But, unfortunately task-oriented
datasets are limited in number. Accessibility to them is 39372
another issue. The only publicly available task-oriented data x No of testing images present in the dataset:
is the one collected by certain companies or collected using 9843
Wizard of Oz [9] techniques and crowdsourcing [10]. x Size of each image: (64*64)
Crowdsourcing proves to be an efficient technique to obtain x No of channel in an image: 3 (RGB)
data by letting different users chat and then recording their
conversations and transforming them into useful datasets. Sample of the classes present in the dataset are
‘Apple___Apple_scab’, ‘Peach___healthy’ and
Another way to create dataset is by using web-scraping
‘Tomato___Target_Spot’. Sample species present in dataset
[11]. Web-scraping is a technique by which one can access
are ‘Apple’, ‘Blueberry’, and 'Strawberry'. Sample diseases
required data from a website and can store it in a database.
present in dataset are ‘Apple_scab’, ‘Bacterial_spot’ and
Anything that is available on a webpage can be scraped using
‘Tomato_Yellow_Leaf_Curl_Virus’.
web-scraping.
Exploratory Data Analysis (EDA) is an approach for
Our dataset is a collection of query-response pairs
data-analysis and summarizing their characteristics visually.
gathered from various sources like agrifarming [12], KCC
Matplotlib, seaborn and image_grid libraries are used to
dataset and chatterbot-corpus-master [13]. Agrifarming is a
produce the visualizations. A sample visualization using
blog which contains information related to agriculture
image grid can be seen in Fig 1. The first row visualizes the
farming in India. The information it contains is in the form of
disease ‘Tomato Yellow Leaf Curl Virus’ in the plant
Frequently Asked Questions (FAQs), questions and answers
‘Tomato’.
were therefore extracted using web-scraping and stored in
separate files. Another source was KCC dataset, which
contains the logs of calls at KCC by farmers. Each call log
contains 11 fields namely date and time of the call, crop,
query type, query and answer by KCC expert. Dataset also
includes conversations from chatterbot-corpus-master. It’s a
multilingual dialog corpus. Each corpus is collection of input
statements and their responses. Agribot used only English
corpus to train itself. The KCC dataset and scraped dataset Fig. 1. Visualization of a dataset using image grid
will provide technical knowledge to the bot and general
conversational dataset can help in taking out normal IV. METHODOLOGY
conversations.
This section describes the implementation and how the
Steps for web-scraping to collect query-response pairs training was done both for conversational system module and
are: disease detection module. Also, the section contains the
implementation of weather prediction module.
x Import necessary libraries – requests, re, pandas,
beautiful soup. A. Approaches to Build Conversational System Module
x Make an array of URLs say URL. Sequence-to-sequence model is used to build
conversational system model. It is a multilayer perceptron
x Take two empty list one for question and Recurrent Neural Network (RNN). Another term for this is
another one for answer say ques, ans. an encoder-decoder model [15]. It is a generative type
intelligent model. It means that sequence to sequence model
x Search for the required pattern using beautiful can automatically learn from data and can generate response
soup and regular expressions and store it in the word by word based on the query. These models are not easy
list. to learn as they need to learn the structure of the sentence by
x Use pandas dataframes to represent data in themselves.
tabular form and import it in database. Sequence to sequence models comprises of 2 recurrent
To access the scraped dataset one can mail at neural networks. Namely - Encoder: symbols to fixed length
[email protected]. vector representation encoding, Decoder: vector length
representation to symbols decoding. The following diagram
B. Dataset Description For Disease Detection Module is the structure of seq-to-seq model comprising of encoder,
Dataset used for disease detection module is of Plant decoder depicting the process learning with the unrolled
RNNs in Fig. 2.

30
 layer. Embedding layer maps a word from our vocabulary (in
the form of dictionary) to a dense vector of a given size.
Dense vector will represent synonyms as one dimension so,
synonyms will be considered as the same thing and not
different. We used word2vec [17] technique to generate
Fig. 2. Basic architecture of seq-2-seq model embedding layer.
In this architecture, every box represents an RNN cell We need to predict the correct response for the query
which can be either LSTM (Long Short Term Memory) unit asked by the user which requires proper training of the
or a GRU (Gated Recurrent Unit). Both the encoder and model. So, conversational system model was trained using
decoder are then jointly trained to maximize the conditional cross entropy loss. Tensorflow has a sequence_loss function
probability of a target sequence (i.e. X, Y, Z) given an input defined in the module tf.contib.seq2seq. We have used the
sequence (i.e. A, B, C). Each encoder and decoder have its Adam optimizer to minimize the loss. We have used tf.train
own RNN network. Output of one layer is fed as input of with a learning rate of 0.01. By default, the network
another layer in encoder. The last state of encoder is called architecture is a standard encoder/decoder with two LSTM
context or thought vector. Context is passed as initial state to layers (hidden size of 128) and an embedding size for the
the decoder. For every previously generated symbol at each vocabulary of 6000. The maximum sentence length is set to
time step decoder generates a new symbol. 50 words but can be decreased on the cost of loss of
information.
A few constraints associated with the model are as
follow. Training generates the weights file which is used to
generate the response. The model is trained such that it gives
1) The model cannot handle variable length input the responses with minimum error, i.e. gives the responses
sequence. So, input sequence needs to be converted to fixed with maximum probability i.e., p(response|context). The
length vector representations. input and output symbols can be considered as a sequence of
2) Another problem is large vocabulary size. For each words. These sequences of words are converted to fixed
word in the output, decoder has to run softmax [16] which length vector. Model learns automatically an output for a
creates vector of probability that help us generate the final corresponding input. Final response is generated using
output. This increases the training time. softmax activation function.
The above stated problems could be solved using the B. Strategy for Disease Detection Module
following techniques. In this subsection we will discuss the implementation of
1) Padding: Input vectors are of variable length. These disease detection module of our Agribot. The step by step
input vectors cannot be given to sequence to sequence implementation including the pre-processing of the data,
model as this model requires fixed length input vectors. architecture and the training of the module is discussed here.
Variable length vectors can be converted to fixed length Component heads identify the different components of your
using padding. paper and are not topically subordinate to each other.
The following tokens are used to fill in the sequence. Data must be clean, formatted and restructured before it
a) EOS: Inserted at the end of statement. can be used for machine learning algorithms. So, the first
b) PAD: To fill the gap between variable length step was to pre-process the data. The pre-processing steps
sentences. include creating the structured data and then splitting the data
c) GO: At the beginning of the output symbols. into test and train data (80% training and 20% validation
data).
d) UNK: For word not in vocabulary.
Consider the question-answer pair as ‘How was your Next step is to create a classification model.
day? --- It was good.’ Classification model can be built either from scratch or using
Let us assume we want input and output to be of same transfer learning. Transfer learning will be a better approach
length then the above pair will be converted as follows. if original image size is used, but in our case it’s not so. Due
to memory issues size of the image was reduced to 64x64
x [PAD, PAD, PAD, PAD, PAD, PAD, “?”, pixels thus, building model from scratch is a better option.
“your”, “was”, “How”, “day”] The architecture created contains sequence of layers shown
x [GO, “It”, “was”, “good”, “.”, EOS, PAD, in Fig 3. The model created was compiled using rmsprop as
PAD, PAD, PAD] an optimizer and categorical cross entropy as a loss. Also to
2) Bucketing: If maximum length of vectors is set as 100 save the best weights during training model checkpoint file
and input vector is of size 4, then it causes huge wastage of was created.
space. So, concept of bucketing is used. Examples of list of The model created was trained for 50 epochs using batch
buckets be [ (6,10), (10,15), (20,25), (40,50) ]. size of 20.
If we put the input sequence in the bucket (6, 10) then
A. Implementation of Weather Prediction Module
the above question answer pair will be encoded to
x [ PAD, “?”, “day”, “your”, “was”, “How” ] Weather prediction is another feature that is added to the
Agribot. The ‘OpenWeatherMap’ API is used for the same.
x [ GO, “It”, “was”, “fine”, “.”, EOS, PAD, The response from these APIs is weather forecast for next 5-
PAD, PAD, PAD ] 10 days. API includes parameters like temperature, relative
humidity, pressure and wind speed. OpenWeatherMap is an
3. Word Embedding: It is done for the first layer of the administration that gives climate information, including
network. The first layer is therefore called the embedding current climate information, estimates, and verifiable

31
information to the engineers of web administrations and
versatile applications.

(a)

(b)

(c)
Fig. 4. (a) Prediction when trained for 1000 epoch (b) Prediction when
trained for 1187 epochs (c) Best Prediction

B. Observations of Disease Detection Module

Initially to build the architecture, we started with few
layers but increased them later to improve the results. Filters
help in extracting more features from the image so increased
the number of filters each of size (2,2) in every next layer.
Accuracy with other sizes of filters were also used but (2,2)
gave the best result. Large number of parameters led to
overfitting and took more time to train. So, to reduce the
dimensions of convolutional layer we used maxpooling
layer. To refine the model, drop out layers were used in the
architecture but that reduces the performance to around 70%
Fig. 3. Sequence of layers in CNN architecture
[18].
C. Observations for Weather Prediction Module
Workflow:
Initially we started with connecting weather API with our
1. Import required libraries
program. We can give access location in any form like in
2. GET form of city coordinate or zip code but for accuracy and ease
{http://api.openweathermap.org/data/2.5/weather?} of use we use city name as input. In case if we enter wrong
3. Input City Name and the date for forecasting input, it won’t process further and return invalid city. With
4. Send the request to API the help of API, we can do 5 days forecasting. In the end we
5. Return response object have integrated complete program with telegram to create a
6. Check input city is valid or not chatbot.
7. Get the data for corresponding day & time from
openweathermap API. VI. RESULTS
8. Print temperature, weather status, humidity status Based on the research described, AI-based chatbot was
and other details implemented. After achieving reasonable predictions, we set
up a webhook connection with our Telegram Bot API to
V. OBSERVATIONS AND INFERENCES make our chatbot interactive for users. The functionalities
This section discusses the various observations taken and developed for the Agribot are demonstrated.
inferences derived from them.
A. Study of Observations for Conversational System
Module
The model is trained only for 1187 iterations for limited
data-points on Google Colaboratory. The predictions
obtained for iterations 1000 as well as 1187 were quite
reasonable with an accuracy of approximately 98%. The
results are shown in Fig 4.
The expected outcome for Fig 4(a) is “Spray Plano fix
5ml/15 Lit of Water.” The actual outcome is not similar with
the expected answer but the chatbot understood the context
that it needs to spray plano for flower drop control. The
expected outcome for Fig 4(b) is “Denitrifying bacteria are
anaerobic in nature and do not need oxygen for processing
nitrate to nitrogen gas.” The expected outcome is similar
with the actual outcome in this case. The expected outcome
for Fig 4(c) is “Spray Proclaim 5 gm/15lit of Water.” In this
case the expected and actual outcomes are exactly same. Fig. 5. Conversations on Agribot

32
 The chatbot developed can assist farmers in the areas of language. So, language translator can be used. The weather
crop disease detection, problem solving, and weather prediction module can be modified so that it accesses the
prediction shown in Fig. 5, 6, 7. location without user interruption. In order to improve the
performance in future, we need to use original size images
Plant disease detection module trained on the plant taken in real world environment.
village dataset, is one of the special features of our chatbot.
The telegram messenger was able to output the predicted REFERENCES
disease class as its response. The Agribot can be more
[1] M. Jain, P. Kumar, I. Bhansali, Q. Vera Liao, K, Truong and S. Patel,
generalized in terms of conversations if the model gets “FarmChat: A Conversational Agent to Answer Farmer Queries. Proc.
trained on large number of data-points. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 4”, Article
170, pp. 22, 2018.
[2] J. Weizenbaum. “ELIZA–a Computer Program for the Study of
Natural Language Communication BetweenMan and Machine”. In:
Communications of the ACM 9.1, 1966, pp. 36–45.
[3] D. Britz “Deep Learning for Chatbots, Part 2 – Implementing a
Retrieval-Based Model in Tensorflow”, July 2016 URL:
http://www.wildml.com/category/neural-networks/
[4] I. Sutskever, O. Vinyals, and Q. V. Le. “Sequence to Sequence
Learning with Neural Networks”. In: Advances in Neural Information
Processing Systems, pp. 3104–3112, 2014.
[5] W. N. Zhang, T. Liu, Y. Wang, Q. Zhu “Neural Personalized
Response Generation as Domain Adaptation” arxiv, 2019
[6] M. Brahimi, M. Arsenovic, S. Laraba, S. Sladojevic, K. Boukhalfa, A.
Moussaoui “Deep Learning for Plant Diseases: Detection and
Saliency Map Visualisation,” springer, 2018, pp 93-117.
Fig. 6. Path of image of a plant given as a input will predict the type of [7] A.M. Connor, D.K. Limbu, S.G. MacDonell, & R. Pears, “Improving
disease web information retrieval using shared contexts,” International
Journal of Information Sciences and Computer Engineering, vol. 1,
VII. CONCLUSIONS AND FUTURE WORK pp.26-35, 2010.
[8] H. Du, P. Jones, E. L. Segarra, C. F. Bandera, “Development Of A
The objectives were possible to achieve with the help of REST API ForObtaining Site-Specific Historical And Near-Future
sequence to sequence learning, OpenWeatherApi and other Weather Data In EPW Format,” 4th Building Simulation and
Optimization Conference, Cambridge, sept. 2018.
machine learning algorithms that combines with the features [9] P. Budzianowski, T.H. Wen, B. H. Tseng, I. Casanueva, S. Ultes, O.
of the electronic message service Telegram. Ramadan, M. Gašić, “MultiWOZ - A Large-Scale Multi-Domain
Wizard-of-Oz Dataset for Task-Oriented Dialogue Modelling”,
Conference on Empirical Methods in Natural Language Processing,
pp. 5016–5026, 2018.
[10] R. Tao, H. R. Maier. “Crowdsourcing Methods for Data Collection in
Geophysics: State of the Art, Issues, and Future Directions”, Reviews
of Geophysics banner, Vol. 56, 4, pp. 698-740, 2018
[11] V. Krotov, L. Silva, “Legality and Ethics of Web Scraping”, Twenty-
fourth Americas Conference on Information Systems, New Orleans,
ResearchGate, 2018.
[12] Agrifarming, URL: https://www.agrifarming.in/
[13] Chatterbot-corpus URL:
https://chatterbot-corpus.readthedocs.io/en/latest/, 2017
[14] S. P. Mohanty, D. P. Hughes, M. Salathé “Using Deep Learning for
Image-Based Plant Disease Detection”, frontiers, 2016.
[15] M. Chen, Y. Li, R. Li “Research on Neural Machine Translation
Model”, IOP Conf. Series: Journal of Physics, 2019
[16] Z. Yuan , J. Li , Z. Li , C. Ding , A. Ren , B. Yuan , Q. Qiu , J. Draper
and Y. Wang, “ Softmax Regression Design for Stochastic
Computing Based Deep Convolutional Neural Networks”, Great
Lakes Symposium on VLSI, PP. 467–470, 2017.
[17] D. Matthews, S. Kriegman, C. Cappelle and J. Bongard, "Word2vec
to behavior: morphology facilitates the grounding of language in
machines," IEEE/RSJ International Conference on Intelligent Robots
and Systems (IROS), Macau, 2019.
[18] N. Srivastava “Dropout: A Simple Way to Prevent Neural Networks
from Overfitting”, Journal of Machine Learning Research, vol. 15, pp.
Fig. 7. Weather prediction using Agribot given location as input 1929-1958,2014.

Mostly farmers speak in their regional languages. We

need tool which can inter-relate our system and their

33