Towards Smart Farming: Accurate Prediction of

Paddy Harvest and Rice Demand

M.R.S.Muthusinghe, Palliyaguru S.T., W.A.N.D. Weerakkody, A.M. Hashini Saranga, W.H. Rankothge
Sri Lanka Institute of Information Technology
Sri Lanka

Abstract— Rice is the predominant staple food in Asian

countries. It has a major impact on the social and economic The current policy of the Government of Sri Lanka
development of these countries. Therefore, it is very important (GoSL) intends maintaining self-sufficiency in rice
to keep the sustainability between paddy cultivation and production and consumption. The GoSL guarantees a
consumer demand. Paddy crop yield and demand for rice of a minimum price to farmers via the Paddy Marketing Board
country depend on numerous factors such as rainfall, and in addition, also maintains a maximum retail price for
humidity, citizen’s life styles etc. Hence, the prediction of consumers [3].
future harvest and demand is a complex process. There is a
requirement for a platform that predicts on future harvest and
A fine understanding and better prediction mechanisms
demands based on all affecting factors.
for paddy crop yield and rice demand are much important as
We have proposed a platform that targets the smart the fluctuations of these factors have a major impact on food
farming concepts for paddy, with following modules: (1) a security and keeping sustainability between demand and
prediction module to predict paddy harvest and (2) a supply. To ensure the sustainability there is a requirement
prediction module to predict rice demand. We have developed
for a platform that first identifies the factors effecting paddy
the prediction modules using two machine learning algorithms:
(1) Recurrent Neural Network (RNN) and (2) Long Short-
harvest and rice demand and then predicts on future harvest
Term Memory (LSTM). The performances of algorithms were and demand. We have proposed a platform that targets the
evaluated using real data sets for the Sri Lankan context. Our smart farming concept for paddy with following modules:
results show that the prediction modules are giving accurate (1) a prediction module to predict paddy harvest and (2) a
results in a short time. prediction module to predict rice demand. Smart Farming
Keywords— Smart Farming, Predictions, Recurrent
(SF) involves the incorporation of information and
Neural Networks (RNN), Long Short-Term Memory (LSTM) communication technologies into agricultural production
systems as an enabler of more efficient, productive and
profitable farming enterprises [8].
I. INTRODUCTION
We have developed the prediction modules using two
Rice is one of the oldest cereal grains and believed to machine learning algorithms: (1) Recurrent Neural Network
have been grown for at least 5000 years. It is a staple food (2) Long Short-Term Memory (LSTM) model. The
for more than half of the world's population, particularly performances of algorithms were evaluated using real data
those living in southern and eastern Asia. Especially in Sri sets for the Sri Lankan context. Our results show that the
Lanka rice has a major impact on the social and economic prediction modules are giving accurate results in a short
development of the country. Rice is the staple food of 18.6 time. Performance of prediction modules were analyzed
million Sri Lankans and is the livelihood of more than 1.8 using following matrices: Mean Squared Error (MSE), Root
million farmers [1]. More than 30% of the Sri Lankan labor Mean Squared Error (RMSE), training score and test score
force is directly or indirectly involved in the rice sector [1]. of the models. The harvest prediction module gave 78% for
It is occupying 34% of the total cultivated area in Sri Lanka the train score with 0.04 MSE value and 75% test score with
during Maha (from May to end of August) and Yala 0.11 MSE value. The demand prediction module gave 79%
(September to March). Sri Lanka currently produces 2.7 for the train score with 0.17 MSE value and 74% test score
million tons of rough rice annually and satisfies around 95% with 0.34 MSE value.
of the domestic requirement.
The rest of the paper is organized as follows. Section II
presents the related work. Section III presents the
When considering the demand and consumption of rice
background of prediction algorithms and technological
in Sri Lanka, so far there has not been any competitive
approaches for the study. Section IV introduces the
substitutes for rice. By year 2017 the average consumption
prediction modules to predict paddy harvest and rice
of rice was 110.2 kg/per person in Sri Lanka [3]. Rice
demand. In section V, the result and discussions are
provides 45% total calorie and 40% total protein
presented. Final remarks mentioned in Section VI.
requirement of an average Sri Lankan [2]. It is expected that
the demand for rice will increase at 1.1% per year and to
II. RELATED WORK
meet this demand, the rice production should grow at the
rate of 2.9% per year. Increasing the cropping intensity and As rice is the predominant staple food in Asian countries
national average yield are the options available to achieve there have been several research works carried out on
this production targets [2]. predicting paddy harvest and rice demand. In this section we
will discuss the most recent works, especially in the Asian
context.

1
978-1-5386-5051-6/18/$31.00 ©2018 IEEE
 The authors of [9] present a review on use of machine Unlike a traditional neural network, in which it assumes
learning techniques for Indian rice crop prediction. They that all inputs are independent of each other, an RNN will
discuss the experimental results obtained by applying make use of the sequential information among input data.
sequential minimal optimization (SMO) classifier using the They are called recurrent because every element in the
WEKA tool on the dataset of 27 districts of Maharashtra sequence is subjected to the same task, while output is being
state, India. The parameters considered for the study were depend on computational values of previous nodes.
precipitation, minimum temperature, average temperature, RNN uses its internal state (memory) to process
maximum temperature, reference crop evapotranspiration, sequences of inputs which are related to each other. It keeps
area, production and yield for the Kharif season (June to memory on these relationships while training itself. So, the
November) for the years 1998 to 2002. relation among all the previous inputs helps in predicting a
better output. The RNNs are applicable to tasks such as
The trends in paddy production in Sri Lanka have been connected handwriting recognition, speech recognition, etc.
analyzed in [5]. The objectives of the study were to identify As shown in the Figure 1 [13], a block of neural network: A,
the past, present and future trends of paddy production in takes xt as the input and outputs the value ht. The
Sri Lanka and to develop a time series model to detect the information passed from one step of the network to another
long-term trend and prediction for three leading years of via a loop. A recurrent neural network can be considered as
paddy production. They have identified ARIMA (2, 1, 0), as multiple copies of the same network, where each one passes
the model which has the minimum AIC (Akaike Information a message to the next successor. This is known as output
Criterion) and BIC (Bayesian Information Criterion), which feedback.
was choosen as most suitable model that best fitted to the
data set.

The study [6] discussed reasons for problems in paddy

industry such as, failed to satisfy poor farmers even with the
self-suffiency due to increased average yeild and extent of
cultivation. The study has attempted to identify the past
trends in paddy production and to understand the reasons for Fig. 1. RNN structure [13]
those underlying problems considering, paddy production
related variables such as; extent sown under different water Figure 2 [13] gives an idea on what happens when we
sources, crop failure percentage, production, average yield unfold the loop of the RNN. The chain-like structure defines
etc.These variables have been analyzed using time series that RNNs are mostly applied with sequences and lists.
analysis techniques and statistical techniques.

When considering the paddy price fluctuation over past

years, it is very important to have a proper forecasting
technique for rice demand. This helps in avoiding
unnecessary price increases during rice shortages and
managing the surpluses in time. The study [7] aimed at
designing an information management and a decision
support system for paddy cultivation enabling both decision Fig. 2. Output feedback for RNN [13]
making authorities and farmers to make effective decisions.
They have considered a number of parameters affect the rice B. Long Short-Term Memory (LSTM) model
demand and predicted the expected price of the paddy in
advance before the end of each season. So it helps to prepare The LSTMs are a specific sub class of RNNs which are
for the situations coming up by adopting counter strategies. capable of learning long term dependencies. This concept
was initially introduced by Hochreiter & Schidhuber in 1997
III. TECHNOLOGICAL BACKGROUND [11].
All RNNs have a set of chain like of repeating modules
In this section, a brief introduction to the technologies used of neural network. As shown in figure 3, in standard RNNs,
in the prediction modules will be presented. a single repeating module will have a single tanh layer.

As mentioned in the previous sections, our prediction

modules were based on two machine learning algorithms:
(1) Recurrent Neural Network and (2) Long Short-Term
Memory (LSTM) model.

A. Recurrent Neural Network (RNN) model

An RNN is a class of artificial neural network. The
Connections between each unit is formed as directed graph
in a sequence manner [10]. This make it possible to identify Fig. 3. Standard simple RNN cell structure [12]
dynamic temporal behaviour for a time sequence.

2
 LSTMs also form in a chain like structure,but the (the respondent agreed with the system, but rely on
repeating module organizes in a different way. In a LSTM alternative methods for the predictions), (3) Not suitable
network, a module may have four neural network layers (the respondent disagreed with the proposed system is not
with three gates. The structure of a strandard LSTM cell is necessary). The summary of the survey conducted with
shown in figure 4. farmers and rice mill owners are summarized in the Table I.
According to summary, 56.7% farmers have suggested
the system as the most suitable solution. 32.5% have
suggested suitable and 17.5% suggested as not suitable.
Among rice mill owners nearly 50.0% of average has agreed
as system is most suitable. 38.9% have agreed as suitable
and only 11.1% has suggested as not suitable.

TABLE I. FEASIBILITY STUDY SUMMARY

Farmers Rice mill owners

District Most Not Most Not
Suitable Suitable
suitable suitable suitable suitable
Fig. 4. Standard LSTM cell structure [12] Anurad 80% 20% 0% 50% 50% 0%
apura
The first gate is the “forget gate layer” that decides Kurune 50% 37.5% 12.5% 66.67% 33.3% 0%
what information is going to be throw away from cell state. gala
40% 40% 40% 33.3% 33.3% 33.3%
It takes previous cell output h t-1 and current cell input Xt Matara
and applies sigmoid activation layer () to get a value Total 56.7% 32.5% 17.5% 50% 38.9% 11.1%
between 0 and 1 for each hidden unit. A “1” represents
“keep this” while a “0” represents “get rid of this”.
The survey results clearly showed that the prediction
Next gate is a “update gate”, which passes the same
platform is a timely necessary solution for both farmers and
input (ht1 and xt) into two layers: (1) a sigmoid activation rice mill owners to predict harvest and demand.
layer () and (2) a tanh activation layer (tanh). Then it
performs an element-wise multiplication between two
B. Data sets to evaluate the proposed prediction platform
outputs of the two layers to get the result r. Next, the
elment wise addition is carried out to the output of “forget
gate” and result r. Finally, the state is updated with new To train and evaluate the proposed prediction platform,
information. we needed historical statistics related to paddy harvest and
demands in entire Sri Lanka. However, as the first step, we
The third gate is the “output gate” which controls what decided to carry out our case study for selected few districts
information to be passed to the next state. The output form in Sri Lanka. We faced the challenge of selecting few
previous layer act as an filter to dicide what output ht should districts that would represents the general context of Sri
pass to the next LSTM cell. Lanka.
According to our survey results, we identified “rainfall”
IV. PREDICTION PLATFORM as one of the major affecting factors for the rice production
in Sri Lanka. Based on the average annual rainfall, Sri Lanka
can be broadly divided into three climatic zones: Dry Zone
As mentioned in the previous sections, to ensure the (DZ), Intermediate Zone (IZ) and Wet Zone (WZ).
sustainability between demand and supply, there is a Therefore, covering all three climate zones, we decided to
requirement for a platform that first identifies the factors carry out our study for following districts: Anuradhapura
effect paddy harvest and rice demand, and then predicts on (dry zone), Kurunegala (intermediate zone) and Matara (wet
future harvest and demand. We implemented a single zone).
platform for harvest and demand prediction of rice. The
platform can be used by farmers, rice mill owners and rice We collected statistics related to paddy harvest and
consumers and government authorities etc. In this section, demands of these three districts from different sources. All
we discuss the proposed solution platform. the data sets include data of year 1990 to 2017.
First, the district wise paddy cultivation data for Yala and
A. Requirments of the proposed prediction platform
Maha seasons were collected from Department of
Before proposing a solution platform for the prediction Agriculture, Sri Lanka. Next, the paddy purchasing data was
process, it was very important to understand the current collected from Paddy Marketing Board, Sri Lanka (PMB).
context of paddy harvest and rice demand in the world. The secondary data set for the paddy purchasing, was also
Therefore, as our case study, we selected the Sri Lankan collected from Sri Lanka - Department of Census and
context and gathered information by interviews and Statistics.
questionnaires with 80 farmers and 30 rice mill owners in
Anuradhapura, Kurunegala and Matara districts in Sri Lanka Third, the national macro and micro economic factors,
We tried to identify the feasibility and necessity of based on consumer cost of living, farmer’s cost of living,
prediction platform for rice. The responses were categorized producer’s prices for paddy and national GDP rates were
into 3 categories: (1) Most suitable (the respondent collected from Central Bank of Sri Lanka. Next, in order to
suggested that the system is highly essential), (2) Suitable identify the numerous parameters that affect the annual
paddy harvest amounts, the information from Rice research

3
and Development Institute, Bathalagoda, were analyzed. and “Test set”. Then the train and test set were split into
Finally, to identify regional based factors impact on paddy input (X) and output (Y) vectors. Finally, the inputs are
harvesting practices, purchasing prices and consumer’s reshaped into the 3D format expected by LSTM, namely
behaviour, the different Govi Jana Kendra centers were [samples, time steps, features].
referred. We defined the LSTM network with 50 neurons in the
first hidden layer and one neuron in the output layer for
C. Harvest prediction module predicting harvest. We used the Mean Absolute Error
(MAE) loss function and the efficient Adam version of
The main goal of harvest prediction module was to identify
stochastic gradient descent as evaluation models. The model
the best model that fit for harvest prediction patterns with a will be fit for 200 training epochs with a batch size of
higher accuracy level. 72.This value for number of iteration is selected as the
We started the process with a data set which reports the optimum value which gives higher accuracy specific to this
paddy harvest on each season (yala and maha) for 12 years at scenario by trial and error approach.
the Kurunegala, Anuradhapura and Matara district in Sri We kept track of both the training and test losses during
Lanka. The data included the year, the paddy harvest, and the training by setting the validation_data argument in the fit
factors that affect paddy harvest, including paddy yield area function.
and rainfall. Selection of these parameters were based on the Finally, we combined the forecast with the test dataset
co-efficient values of each variable to identify the degree of and invert the scaling. With forecasts and actual values in
change in the paddy production variable for every 1 unit their original scale, we calculated an error score for the
change in input parameters. model. In this case, we calculated the Root Mean Squared
Error (RMSE) that gives error in the same units as the
The complete list of raw data fields are as follow: variable itself.
1. id: row number D. Demand prediction module
2. district: district of paddy yield
3. season: season of paddy yield The main goal of demand prediction module was to identify
4. year: year of data in the row the best model that fit for demand prediction patterns with a
5. product: harvest amount reported in a season higher accuracy level. The demand prediction module
6. area: paddy yield area of a given district for the followed the same steps as harvest prediction module. The
given season same methods for data preparation and evaluation matrices
7. rain: average rainfall reported during the crop were used as the harvest prediction module. The complete
season list of raw data fields used for demand prediction are:
When preparing the data set first, we used the year as 1. id: row number
an index in Pandas (the python library used for the data 2. year: year of data in a row
analysis). Then all the null values in data were replaced as 3. income: per capita income on district basis
“0”. Then all the fields that were not important for the 4. substitute: average consumption amounts of
prediction dropped based on the co-efficient values. Finally, substitute foods
as shown in Table I our data set was ready for the 5. population: population per year on district basis
experiments. 6. consumption: consumption rates on district basis

We used this data and framed a forecasting problem The formatting of the data followed similar steps as the
where given the paddy cultivated area and rainfall for prior harvest prediction module. A sample of final dataset is given
year and then it forecasts the harvest production for the next in Table III.
year. TABLE III DEMAND PREDICTION - DATA SET
TABLE II HARVEST PREDICTION - DATA SET Year Population Per capita Substitute Consum
by income cereal ption of
Year Paddy Area cultivated Rainfall district (Rs) consumption rice (Mt)
production (Mt) (ha) (mm) (Mt)
2006 77257 16944 57 1989 12689 2600 2.3 80.3
2007 56492 11759 62 1990 14846 3549 2.6 95.2
2008 170500 38786 58 1991 18728 3549 2.6 117.5
2009 63275 14297 51 1992 19500 3800 2.2 118.6
2010 89625 18340 42 1993 19600 3540 2.5 119.9

Next step was fitting the LSTM to the problem. This Next, the data were feature normalized as input set, X:
involved preparing the dataset as a supervised learning {income, substitute, population} and output set, Y:
problem and normalizing the input variables. We framed the {consumption}. To fit an LSTM on the multivariate input
supervised learning problem as predicting the harvest at the data, the data split into “Train set” (70%) and “Test set”
current year (t) given the harvest production and affecting (30%). Finally, the data fetched into keras LSTM network
factors at the prior time step. Once all features are with 100 initial neurons and 1 output node. The batch
normalized, the dataset is transformed into a supervised processing ran for 90 epochs (iterations) with batch-size of
learning problem. 10. Again the number of iterations are chosen based on trial
To fit an LSTM on the multivariate input data, first we and error approach to get the optimum value.
split the prepared dataset into two sets named “Train set”

4
 V. RESULTS AND DISSCUSSION

In this section we are going to discuss the results of the

experiments that were carried out to measure the
performances of our prediction modules: harvest prediction
and demand prediction.

A. Harvest prediction

For the harvest prediction module we ran the harvest

prediction LSTM network for 200 iterations with the harvest
prediction dataset. Performances of the harvest prediction
module were analyzed using following matrices: Mean
Squared Error (MSE), Root Mean Squared Error (RMSE),
training score and test score of the models.
The harvest prediction model gave 78% for the train
Fig. 6. MAE distribution for train set and test set- Demand prediction
score with 0.04 MSE value and 75% test score with 0.11
MSE value. The figure 5 shows the MAE distribution for
train set and test set in harvest prediction module. VI. FINAL REMARKS
To ensure the sustainability between rice demand and
supply, there is a requirement for a platform that first
identifies the factors effecting paddy harvest and rice
demand, and then predicts on future harvest and demand.
We have proposed a platform that targets the smart farming
concept for paddy, with following modules: (1) a prediction
module to predict paddy harvest and (2) a prediction module
to predict rice demand.
We have developed the prediction modules using two
machine learning algorithms: (1) Recurrent Neural Network
(RNN) and (2) Long Short-Term Memory (LSTM). The
performances of algorithms were evaluated using real data
sets for the Sri Lankan context. Our results show that the
prediction modules are giving accurate results in a short
time. Performance of prediction modules were analyzed
using following matrices: Mean Squared Error (MSE), Root
Mean Squared Error (RMSE), training score and test score
of the models. The harvest prediction model gave 78% for
the train score with 0.04 MSE value and 75% test score with
Fig. 5. MAE distribution for train set and test set - Harvest prediction 0.11 MSE value. The demand prediction model gave 79%
for the train score with 0.17 MSE value and 74% test score
with 0.37 MSE value.
B. Demand Prediction Module
As the future work first, we are planning to conduct
more experiments to see how we can improve the accuracy
Similar to the harvest prediction module, for the demand of our prediction models. Furthermore, we are planning to
prediction module, we ran the demand prediction LSTM use bigger data sets. Next, we are planning to complete our
network for 90 iterations with the demand prediction smart farming approach for paddy harvest by proposing an
dataset. optimized algorithm for harvest distribution. We are
The demand prediction model gave 79% for the train specially looking at approximation algorithms such as
score with 0.17 MSE value and 74% test score with 0.37 Genetic Algorithms [10], Iterated Local Search [15] etc.
MSE value. The figure 6 shows the MAE distribution for which has been proven as efficient in resource optimization
train set and test set in demand prediction module. and planning problems [16, 17, 18].

5
 REFERENCES Conference on Computer Science and Software Engineering, Khon
Kaen, Thailand, July 2016, pp. 1-5.

[1] M. Dhanapala, "The Wet Zone Rice Culture in Sri Lanka: A Rational
Look", Journal of the National Science Foundation of Sri Lanka, vol. [10] "Recurrent neural network", En.wikipedia.org, 2018. [Online].
33, no. 4, p. 277, 2005. Available: https://en.wikipedia.org/wiki/Recurrent_neural_network.
[Accessed: 14- Aug- 2018].
[2] "Agriculture in Sri Lanka", En.wikipedia.org, 2018. [Online].
Available: https://en.wikipedia.org/wiki/Agriculture_in_Sri_Lanka. [11] S. Hochreiter and J. Schmidhuber, "Long Short-Term Memory",
[Accessed: 14- Aug- 2018]. Neural Computation, vol. 9, no. 8, pp. 1735-1780, 1997.
[12] "Understanding LSTM Networks -- colah's blog", Colah.github.io,
[3] "Crop forecast", Doa.gov.lk, 2018. [Online]. Available:
2018. [Online]. Available: http://colah.github.io/posts/2015-08-
https://doa.gov.lk/index.php/en/18-english-news/307-crop-forecast-4.
Understanding-LSTMs/. [Accessed: 15- Aug- 2018].
[Accessed: 14- Aug- 2018].
[4] "Understanding The Recurrent Neural Network – Mindorks – [13] "An Introduction to Recurrent Neural Networks", Towards data
Medium", Medium, 2018. [Online]. Available: science, 2018. [Online]. Available:
https://medium.com/mindorks/understanding-the-recurrent-neural- https://towardsdatascience.com/an-introduction-to-recurrent-neural-
network-44d593f112a2. [Accessed: 14- Aug- 2018]. networks-72c97bf0912. [Accessed: 14- Aug- 2018].
[5] V. Sivapathasundaram and C. Bogahawatte, "Forecasting of Paddy [14] M. Mitchell, An Introduction to Genetic Algorithms, Cambridge,
MA, USA: MIT Press, 1998.
Production in Sri Lanka: A Time Series Analysis using ARIMA
Model", Tropical Agricultural Research, vol. 24, no. 1, p. 21, 2015. [15] A. Ishigaki and S. Takaki, “Iterated Local Search Algorithm for
Flexible Job Shop Scheduling”, 2017
[6] A. Razmy Mohamed and A. Ahmed Naseer, "Trends in paddy
production in Sri Lanka", Ir.lib.seu.ac.lk, 2005. [Online]. Available: [16] W. Rankothge, F. Le, A. Russo, J. Lobo, " Experimental results on
http://ir.lib.seu.ac.lk/123456789/47. [Accessed: 30- Oct- 2018]. the use of genetic algorithms for scaling virtualized network
functions", Proc. IEEE SDN/NFV, pp. 47-53, 2015.
[7] Ponweera, P. and Premaratne, S. (2018). “Information and decision
support system to enrich paddy cultivation in Sri Lanka”. [online] [17] W. Rankothge, J. Ma, F. Le, A. Russo, J. Lobo, "Towards making
Dl.lib.mrt.ac.lk. Available at: http://dl.lib.mrt.ac.lk/handle/123/8438. network function virtualization a cloud computing service", Proc.
IEEE IM, pp. 89-97, 2015.
[8] Dieisson Pivotoa Paulo, Dabdab Waquil et al., "Scientific
development of smart farming technologies and their application in [18] W. Rankothge, F. Le, A. Russo, J. Lobo, " Optimizing Resource
Brazil", Information Processing in Agriculture, vol. 5, no. 1, p. 31, Allocation for Virtualized Network Functions in a Cloud Center
2018. Using Genetic Algorithms ", Proc. IEEE IM, pp. 89-97, 2015.
[9] N. Gandhi and L. Armstrong et al., "Rice Crop Yield Prediction in
India using Support Vector Machines", in International Joint