Received November 14, 2020, accepted November 21, 2020, date of publication January 20, 2021, date of current

version March 9, 2021.

Digital Object Identifier 10.1109/ACCESS.2021.3051196

Big Data and Machine Learning With

Hyperspectral Information in Agriculture
KENNETH LI-MINN ANG 1 , (Senior Member, IEEE),
AND JASMINE KAH PHOOI SENG 2 , (Member, IEEE)
1 School of Science and Engineering, University of the Sunshine Coast, Petrie, QLD 4502, Australia
2 School of Engineering and Information Technology, University of New South Wales, Canberra, ACT 2612, Australia
Corresponding author: Kenneth Li-Minn Ang ([email protected])

ABSTRACT Hyperspectral and multispectral information processing systems and technologies have demon-
strated its usefulness for the improvement of agricultural productivity and practices by providing useful
information to farmers and crop managers on the factors affecting crop status and growth. These technologies
are widely used in a range of agriculture applications such as crop management, crop yield forecasting, crop
disease detection, and the monitoring of agriculture land usage, water, and soil conditions. Hyperspectral
information sensing can acquire several hundred spectral bands that cover the electromagnetic spectrum
of an observational scene in a single acquisition. The resulting hyperspectral data cube contains a large
volume of spatial and spectral information. The hyperspectral sequence of images or video further increases
the data generation velocity and volume which lead to the Big data challenges particularly in agricultural
remote sensing applications. This paper is structured to first give a comprehensive review of representative
studies to provide insights into significant research efforts in agriculture using Big data, machine learning
and deep learning with the focus on frameworks or architectures, information processing and analytics
with hyperspectral and multispectral data. The potential for utilizing Big data, machine learning and deep
learning for hyperspectral and multispectral data in agriculture is very promising. The paper then further
explores the potential of using ensemble machine learning and scalable parallel discriminant analysis which
takes into consideration the spatial and spectral components for Big data in agriculture. To the best of our
knowledge, no similar review study on agriculture with Big data, machine learning and deep learning for
hyperspectral and multispectral information processing has been reported. Furthermore, the potential of
ensemble machine learning and scalable parallel discriminant analysis has not been explored in agriculture
information processing. Experiments and data analytics have been performed on hyperspectral data from
agriculture for validation. The results have shown the good performance of our approach.

INDEX TERMS Agriculture, big data, machine learning, parallel computing, hyperspectral, multispectral.

I. INTRODUCTION increasingly being developed and deployed for many indus-

The authors in [1] project that an increase of approximately tries, professions, and trade sectors.
25% to 70% above current production levels may be needed to For the agriculture sector, Big data provides farmers
meet the global crop demand in 2050. This makes it important with useful and actionable information on weather and sea-
for farmers and crop growers to utilize emerging technologies sonal patterns, rain and water cycles, fertilizer requirements,
to improve productivity to feed the growing global popula- and other critical information for harvesting and decision-
tion. The technology and data driven economy and its focus making. This enables farmers, agricultural suppliers and other
on developing intelligent instrumentation, sensing, robotics, stakeholders to make smart decisions such as the cycles for
artificial intelligence (AI), machine learning, Big data and crops planting to increase profitability and the planning of
data analytics is expected to play a transformative role in optimal harvesting times leading to improved farm yields.
agriculture to raise the rate of food production. Big data is To address the issues of the deployment of Big data in agri-
culture and Big data which are produced from large-scale net-
The associate editor coordinating the review of this manuscript and worked sensing systems, some authors [2], [3] have presented
approving it for publication was Liang-Bi Chen . some reviews for Big data in agriculture. The authors in [2]

This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/
VOLUME 9, 2021 36699
 K. L.-M. Ang, J. K. P. Seng: Big Data and Machine Learning With Hyperspectral Information in Agriculture

presented a review to develop insights into the usefulness of the development of advanced algorithms termed as deep neu-
Big data applications in smart farming and the related socio- ral networks (DNN) algorithms and approaches. The authors
economic challenges. The authors in [3] presented a review in [7] defined DNN as computational models that are com-
on some significant research efforts utilizing Big data for crop posed of multiple processing layers to learn representations of
protection focusing on weed management and control. data with multiple levels of abstraction. DNN methods have
A major source of Big data for agriculture comes from significantly improved the state-of-the-art in many fields such
hyperspectral and multispectral information processing and as speech recognition, visual object recognition, object detec-
remote sensing systems. Remote sensing applications and tion, drug discovery and genomics.
systems generate a huge amount of earth observation data This paper gives the following contributions. This paper
from many sources (e.g. satellite-based systems, unmanned is structured to first give a comprehensive review of repre-
aerial vehicles (UAVs), ground-based structures) and con- sentative studies to provide insights into significant research
tribute significantly to the volume of Big data to be processed. efforts in agriculture using Big data, machine learning and
Agricultural remote sensing is one of the key enabling tech- deep learning with the focus on frameworks or architec-
nologies to fulfill the potential for precision agriculture. Com- tures, information processing and analytics with hyperspec-
pared to traditional agriculture approaches, remote sensing tral and multispectral data. The potential for utilizing Big
approaches for agriculture has the advantages of consider- data, machine learning and deep learning for hyperspectral
ing the within-field variability for site-specific management and multispectral data in agriculture is very promising. The
instead of uniform management for the sites [4]. The use- paper then further explores the potential of using ensemble
fulness of agricultural remote sensing lies in its utilization machine learning and scalable parallel discriminant analy-
of global positioning location and geographic information to sis which takes into consideration the spatial and spectral
produce the spatially-varied data for precision agricultural components for Big data in agriculture. To the best of our
information processing and deployment operations. Agricul- knowledge, no similar review study on agriculture with Big
tural remote sensing is a specialized field to produce the data, machine learning and deep learning for hyperspectral
image and spectral data in large volume, variety and complex- and multispectral information processing has been reported.
ity to enable decision-making for farmers and crop growers in Furthermore, the potential of ensemble machine learning and
many areas (e.g. decision support systems for irrigation and scalable parallel discriminant analysis has not been explored
fertilization, pest management, crop disease detection, and in agriculture information processing. Experiments and data
monitoring of land usage, water and soil properties). analytics have been performed on hyperspectral data from
Agricultural remote sensing applications can utilize vari- agriculture for validation. The results have shown the good
ous data sources including hyperspectral and multispectral performance of our approach.
data. Hyperspectral and multispectral remote sensing can The remainder of the paper is structured as follows:
acquire several hundred spectral bands that cover the elec- Section II first gives a review of Big data and machine learn-
tromagnetic spectrum of an observational scene in a single ing for hyperspectral and multispectral data in agriculture.
acquisition. The resulting hyperspectral data cube contains a Section III presents the ensemble machine learning and scal-
large volume of spatial and spectral information. The hyper- able parallel discriminant analysis (EML-SPDA) for agricul-
spectral sequence of images or video further increases the ture applications and analytics. This section also presents
data generation velocity and volume which lead to the Big and gives details and discussions of experiments and data
data challenges and increase the complexity for information analytics. Section IV concludes the paper with some remarks
processing and analysis caused by the hyperspectral or multi- on future works and challenges.
spectral data. The vast amounts of generated data from hyper-
spectral and multispectral data sources require automated II. REVIEW OF BIG DATA AND MACHINE LEARNING
modeling and analysis techniques such as machine learning. TECHNIQUES FOR HYPERSPECTRAL AND
The field of machine learning has been defined by [5] as hav- MULTISPECTRAL DATA IN AGRICULTURE
ing the goal to program computers to use example data or past The authors in [8] presented a review on the utilization
experience to solve a given problem. The techniques which and deployment of Big data analysis in agriculture. The
have been developed for machine learning is particularly authors in [3] focused on Big data and machine learning
useful to handle the volume and large-scale requirements for for crop protection. The authors in [9] provided a review of
Big data applications. the research focused on the applications of data science and
Examples of applications of machine learning in agricul- machine learning which are relevant to agricultural systems.
ture can be found in [6]. These applications include crop The authors in [2] presented a review of Big data in smart
and yield prediction, disease and weed detection, species farming. These papers presented reviews on Big data or data
recognition, soil and water management, animal welfare and science related to agriculture, but none of them focused on
livestock management. crop quality for crop management, Big data and machine learning utilizing hyperspectral data
animal welfare and livestock production for livestock man- for agriculture. There are some authors [4], [10] which have
agement, water management, soil management, etc. Recent provided a general discussion on Big data in remote sensing.
techniques in the field of machine learning have resulted in It is noted that these review papers which either focus on

36700 VOLUME 9, 2021

K. L.-M. Ang, J. K. P. Seng: Big Data and Machine Learning With Hyperspectral Information in Agriculture

(i) Big data or data science in agriculture or (ii) reviews on TABLE 1. Summary of satellites and its imagery capabilities.
machine learning [6] or deep learning [11] for agriculture.
Other related works on Big data and sensing systems in smart
cities and urban environments can be found in [12] and [13].
The remainder of this section gives an overview of technolo-
gies and surveys the potential of Big data, machine learning,
AI and deep learning with the focus on spectral, hyper-
spectral and multispectral data information and processing
for agriculture. The works have been summarized into four
categories: (1) Big data sources with spectral information;
(2) Big data with hyperspectral analytics in agriculture; (3)
Machine learning techniques for hyperspectral data analytics
in agriculture; and (4) Deep learning techniques for hyper-
spectral data analytics in agriculture.

A. BIG DATA SOURCES WITH SPECTRAL INFORMATION

(BIG SPECTRAL DATA)
Modern hyperspectral sensor technologies have the capabil-
ities of generating very high dimensional imagery with a
large number of spectral bands and signatures through the
use of sensor optics with a large number of bands and spectral
signatures. These technologies make it possible to distinguish
materials through spectral information and to provide detailed many new challenges to be addressed in Big data information
information about the sensed scene. The sensor technologies processing for agriculture information processing.
from satellite-based hyperspectral imaging systems are also
capable of covering vast areas of the earth with high spatial, B. BIG DATA WITH HYPERSPECTRAL ANALYTICS IN
spectral and temporal resolutions. A hyperspectral image of AGRICULTURE
a single scene can be represented as a large volume three- This sub-section discusses several representative studies for
dimensional (3D) data cube with two spatial dimensions and the application of Big data with hyperspectral analytics in
one spectral dimension. agriculture. A summary of the representative works is shown
Sequential scenes are comprised of multiple large volume in Table 2. Agriculture relies on healthy soils to produce
data cubes and pose significant challenges for Big data. For quality crops and pastures. One of the real-world Big data
convenience, we use the term Big spectral data to describe challenges initiates from the domain of soil spectroscopy
Big data sources with spectral information. There are two which aims to identify and establish soil spectral libraries
main sources for Big spectral data: (1) Big spectral data from (SSLs) and signatures. The authors in [16] proposed an
satellite imagery; and (2) Big spectral data from unmanned evolutionary fuzzy rule-based system which was applied to
aerial vehicles (UAVs). An example of Big spectral data from real world agricultural Big data. Their work utilized large
satellite imagery is Sentinel-2. Sentinel-2 provides multispec- datasets (GEO-GRADLE and LUCAS SSL libraries) from
tral imaging (MSI) functionalities with spatial, spectral and the area of soil spectroscopy. In this work, the authors pro-
temporal resolutions, and also has two spectral bands in the posed a two-stage MapReduce scheme and several adapta-
red-edge region for distinguishing the different agricultural tions for Big data processing. Their approach adapted an
crops [14]. Table 1 shows a summary of satellites and its evolutionary fuzzy rule-based algorithm for Big Data termed
hyperspectral/multispectral data capabilities from different as DECO3RUM. Their experimental work used real world
countries in the world. These medium-resolution and high- Big data with hyperspectral information from the area of soil
resolution satellites generate huge volumes of hyperspectral spectroscopy. The data samples were diverse and distributed
or multispectral data which are rapidly increased as Big data across a variety of soil and land cover types. The model was
or termed as Big spectral data. A second data source for Big evaluated in a Hadoop cluster and simulated on eight virtual
spectral data derives from unmanned aerial vehicles (UAVs). servers over a hardware configuration with two Intel Xeon
As discussed by [15], there are two main classifications of processors and 128GB of RAM.
UAV platforms (fixed-wing UAVs and rotary-wing UAVs). The authors in [17] proposed a parallel computing
Rotary-wing UAVs can be further classified into helicopter approach for hyperspectral identification and classification of
UAVs and multi-rotor UAVs. Examples of multi-rotor UAVs oilseed rape waterlogging stress levels. Their work combined
are quadcopters, hexacopters and octocopters. These Big hyperspectral imaging and parallel computing to address the
spectral data from satellite imagery and UAVs require differ- challenges of agricultural Big data. In their study, hyperspec-
ent approaches for information processing and analytics due tral images of these siliques for two oilseed rape varieties
to their volume, complexity and characteristics. These lead to (NY 22 and NZ 19) were captured using Resonon Pika XC

VOLUME 9, 2021 36701

 K. L.-M. Ang, J. K. P. Seng: Big Data and Machine Learning With Hyperspectral Information in Agriculture

TABLE 2. Summary of representative works for big data with hyperspectral analytics in agriculture.

camera, followed by the exposure to three different water beet, cucumber, maize silage, onion, winter wheat, potatoes).
logging stress levels (0, 3 and 6 days). Their implementation Their results showed that combining the vegetation index fea-
used six servers, routing and switching devices to form the tures with the spectral and spatial features improved the clas-
parallel computing framework using Spark machine learning sification accuracy to 98%. The authors in [19] proposed an
library and HDFS (Hadoop Distributed File System). The image classification approach for a study in Florida utilizing
Spark library was used to program and develop two clas- unsupervised learning for hyperspectral agricultural images
sification algorithms (artificial neural network (ANN) and termed as ISODATA (Iterative Self-Organizing Data Analysis
support vector machine (SVM)). The SVM used the one- Technique Algorithm). Their experimental work used the
against-rest classifier for multiple binary classification. The ENVI (Environment of Visualizing Images) [37] software for
ANN and SVM were used as classifiers for the hyperspectral geospatial imagery. After performing PCA, the ISODATA
data and images using the parallel computing platform. The algorithm was applied to classify the hyperspectral images
data from five spectral bands (512, 621, 689, 953 and 961nm) for various class types (Water, Shadow, Wet, Fertile soil, Land
were used as the inputs into the classifiers. For the multiclass and Forest). The performance was evaluated and the overall
classification, the classification accuracy and F1 score of accuracy of the classification process was 75.6%. Another
the ANN were higher compared to SVM. For the binary study proposed by the authors in [80] proposed a graph-
classification, the SVM gave higher accuracy and F1 score. based learning approach termed as local geometric structure
Their results indicated that the ANN was more suitable for Fisher analysis (LGSFA) for dimensionality reduction. The
multi-class classification on the parallel platform whereas the authors showed that their approach was effective in revealing
SVM performed better in binary classification problems. the manifold structure for high-dimensional hyperspectral
The authors in [4] proposed a remote sensing data man- data, and their experimental results demonstrated classifi-
agement approach using the four-layer-twelve-level (FLTL) cation results comparable to other state-of-the-art methods.
framework as shown in Figure 1. The FLTL is an adaptation Further information on graph-based learning approaches for
of the five-layer-fifteen-level (FLFL) framework proposed by hyperspectral information can be found in the survey paper
the authors in [20]. The FLTL structure gives a framework for by the authors in [81].
the management of remote sensing and Big data for precision
agriculture at regional and farm scales. The production of
crop maps is essential for crop classification and the identi- C. MACHINE LEARNING TECHNIQUES FOR
fication of different crops. There are two challenges for crop HYPERSPECTRAL DATA ANALYTICS IN AGRICULTURE
classification and identification due to the spectral similarity In the field of agricultural remote sensing, hyperspectral
and the huge size of the input data. The authors in [18] pro- image classification has become an important topic. Hyper-
posed crop classification technique which combine various spectral data have complex characteristics and a nonlinear
features (spectral, spatial and vegetation index features) to relationship amongst the spectral bands and its various com-
address the spectral similarity challenge for Big data in agri- ponent materials. This makes the accurate classification of the
culture. Their technique involves dimensionality reduction sensed scene a challenging task. This subsection presents a
using PCA (principal component analysis), MNF (minimum review of more recent works on machine learning techniques
noise transform) in the first stage, followed by the support for multispectral and hyperspectral data analytics in agricul-
vector machine (SVM) supervised classification. Their work ture. A summary of the representative works is shown in
used six crops to perform the experimental evaluation (sugar Table 3.

36702 VOLUME 9, 2021

K. L.-M. Ang, J. K. P. Seng: Big Data and Machine Learning With Hyperspectral Information in Agriculture

The authors in [22] proposed a spatial-spectral classifica-

tion framework for Sentinel-2 time series data for land cover
mapping. Their approach utilized mathematical morphology
and image processing techniques to extract the spatial trends
from satellite image time series (SITS) data. These data were
then combined with the available spectral and temporal infor-
mation to improve the discrimination ability among different
land cover classes. The obtained spatial–spectral represen-
tation was classified with a random forest (RF) classifier.
Experiments were conducted on two study sites character-
ized by different heterogeneous land covers. The sites were
Reunion Island study site located in the Indian Ocean and
another site in the southwest of France. Their experimental
and analysis results have demonstrated the significance of the
proposed approach and the validity to combine the spatial and
spectral information for land cover classification.
The authors in [23] proposed a sparse kernel logistic
regression approach and an incremental learning technique
for import vector machines (IVM) for sequential classifi-
cation of hyperspectral data. Their approach included the
addition of new training samples and the deletion of non-
informative training samples to improve the classification
accuracy while maintaining memory and run-time efficien-
cies. The incremental learning strategy enables an efficient
update of the classifier model without a full re-training from
scratch to allow it to handle large data sets. Remote sensing
datasets were used to validate the performance of the incre-
mental IVM. The experiments aimed to classify 16 classes.
The performance of the IVM was also compared to the
FIGURE 1. Framework for FLTL remote sensing data management [4]. SVM for classification accuracy. Their experimental results
demonstrated that the IVM and SVM performed comparably
The authors in [14] proposed a large-scale crop mapping in terms of classification performance. However, the number
from multisource remote sensing images in Google Earth of import vectors was lower when compared to the number
Engine. There are three stages in their approach: (1) Har- of support vectors and remains constant or only slightly
monic analysis on NDVI data combined with spectral features increases with an increasing number of training samples.
obtained from satellites (Landsat-8 and Sentinel-2); (2) Uti- The authors in [24] proposed machine learning techniques
lizing prior constraints of crop distribution and dominance; for crop classification using temporal multispectral satellite
and (3) Information processing with Google Earth Engine. images. In their approach, several machine learning mod-
Their experiments used three crop types (wheat, rapeseed, els were investigated and applied to crop classification of
and corn) to evaluate their approach based on regression Sentinel-2 satellite image data. The selected study area was
tree classification techniques. Their results demonstrated an the region of Andhra Pradesh in India. The machine models in
overall accuracy of 84.25%. Their work also showed that their study included SVM, random forest, RNN with LSTM
the distribution of the crops in the region of their study was and RNN with GRU.
related to agricultural climate, topography and cultivation Their results showed that the SVM produced the highest
practices. The authors in [21] proposed an approach to ana- classification performance of 95.9% with the ground sur-
lyze crop fields evolution by utilizing spatial, spectral and veyed crop areas. The authors in [25] proposed a system
temporal S2-SITS data. Their approach consisted of three for the classification of rice seed varieties using RGB and
major stages: (1) Building a vegetation map by combin- hyperspectral images. The spatial and spectral features were
ing the spatial and spectral data with temporal NDVI data; extracted from the RGB images and hyperspectral image data
(2) Constructing a NDVI time series for a crop field and cubes. The high dimensional spectral feature sets were further
defining an adaptive regression model with a multilayer per- reduced using LDA [72]. Their work compared four combi-
ceptron neural network (MLP-NN); and (3) Extracting and nations of the spatial and spectral features: (1) Spatial only;
analysing the spatial-temporal information from the NDVI (2) Spectral only; (3) Combination of spatial and spectral
time series. The performance of their approach was validated features; and (4) Combination of LDA features from spectral
by experiments carried out on S2-SITS data acquired over an data and spatial features. The random forest classifier with
area located in Barrax, Spain. the four schemes were used to perform the classification.

VOLUME 9, 2021 36703

 K. L.-M. Ang, J. K. P. Seng: Big Data and Machine Learning With Hyperspectral Information in Agriculture

TABLE 3. Summary of representative works for machine learning techniques for hyperspectral data analytics in agriculture.

The performances of the proposed approaches were evaluated good classification performance and improve discrimination
on a large dataset of 90 rice seed varieties with 96 seeds ability to eliminate the impure species from rice seed samples.
per variety. The experimental results showed that the com- The authors in [26] presented the research work for the
bination of spatial features and spectral features could give classification of glycyrrhiza by utilizing NIR hyperspectral

36704 VOLUME 9, 2021

K. L.-M. Ang, J. K. P. Seng: Big Data and Machine Learning With Hyperspectral Information in Agriculture

imaging. The study used seed samples from three glycyrrhiza G and the discriminator D. The CycleGAN consists of two
varieties which were collected from four origins and two generators G and F. In their experiments, they analyzed
planting patterns. The authors used spectral information col- healthy barley leaves and leaves which were inoculated by
lected from 288 bands (948 nm to 2512 nm). The classifier powdery mildew. Their experiments showed that their pre-
was developed using the SVM and PLS-DA (Partial Least dictive model was able to forecast the disease spread from
Squares Discriminant Analysis) models. Their experiments the image time-series. The authors in [31] focused on the
showed that the SVM model gave classification accuracies of prediction of sorghum biomass prediction utilizing remote
93%. Their work demonstrated that NIR hyperspectral imag- sensing data with high spatial and temporal resolutions. The
ing with model discriminant analysis could be used for the authors proposed two approaches to perform the biomass pre-
identification of different glycyrrhiza varieties, origins and diction: (1) Nonlinear regression models to predict biomass
planting patterns. The authors in [27] utilized machine learn- directly from remote sensing data based on features from
ing methods for banana disease detection. The authors used LiDAR point clouds and hyperspectral data. Two nonlinear
hyperspectral images with spectral wavelengths ranging from regression models support vector regression (SVR) and mul-
364 nm to 1031 nm with a spectral resolution of 4.55 nm. tilayer perceptron (MLP) were developed. The authors used
Three classes were considered for disease classification: the parameter settings for SVR and MLP as described in [38];
(1) Dead; (2) Dying; and (3) Healthy. Their approach utilized and (2) Agricultural Production Systems Simulator using
morphological techniques from image processing to extract remote sensing data to parametrize the crop model, and then
the spatial and spectral features from the banana leaf samples simulate the biomass. Evaluations were performed for both
at both early and late stages. The SVM was used for the approaches to demonstrate the usefulness of the approaches.
classification task. Their experimental results demonstrated The authors in [32] proposed a self-training method and
that the hyperspectral images analysis classifier which was utilized a spatial majority filtering technique to locate the
trained by using the samples from banana leaves at late unlabeled samples that could assist in the SVM classifier
infected stages could be better used to predict the disease in training. The approach utilizes the assumption that the class
the earlier infected banana leaves compared to utilizing the labels of neighboring pixels are reliable and the authors
raw spectral information. proposed a majority voting-based algorithm. The perfor-
The authors in [28] presented a novel spectral–temporal mance of the algorithm is improved by considering the spec-
response surface (STRS) approach by utilizing Bayesian tral similarity between a center and its surrounding pixels.
theory to interpolate spectral information into multispec- The authors performed experimental results with agricultural
tral imagery. They also compared their approach with two datasets (including Indian Pines and Salinas) and confirmed
earlier methods (direct interpolation and direct interpola- the effectiveness of the approach for improving the classifica-
tion with spectral dimension imputation) for constructing tion accuracy in cases when the number of labelled samples is
the STRS. Their experimental results showed that the pro- limited. The authors in [33] demonstrated that spectral images
posed Bayesian STRS approach outperformed the two earlier of crops could be used to for nutrient deficiencies detection.
approaches. The Bayesian STRS gave correlations of 0.83 Their approach used multispectral cameras mounted on UAV
with leaf area index (LAI) and 0.77 with canopy chlorophyll to predict the vine water status using neural network models.
measurements compared to correlation values of 0.27 for LAI In their investigation, they computed the Normalized Differ-
and 0.09 for canopy chlorophyll measurements for the direct ence Vegetation Index (NDVI) from the spectral image data
interpolated STRS. The authors in [29] proposed an extreme for soil and plant classification. They utilized the multilayer
learning machine (ELM) classifier for mapping agricultural perceptron (MLP) to different spectral bands to predict the
tillage practices from hyperspectral remote sensing imagery. relation between the information contained in the spectral
The ELM is a single hidden layer feed forward neural net- bands and the vine water status. Their experimental results
work. The authors implemented the kernel version of the showed that plant stresses such as nutrient components could
ELM termed as the kernel ELM (KELM). A spatial convo- be predicted with an accuracy of 0.68 to 0.87.
lution filter was adopted to generate the spatial and spec- The authors in [34] proposed an approach using the
tral features by incorporating information from surrounding extreme learning machine (ELM) for soybean classification
pixels, which were used as the inputs into the KELM. The from remote sensing hyperspectral images. In their approach,
authors conducted the experiments on airborne hyperspectral the spectral data is transformed into a hyper spherical rep-
images and their experimental results showed that the KELM resentation and an image gradient is computed. The clas-
could outperform other traditional approaches like SVM and sification was performed by feedforward networks trained
random forest. with two methods: (1) ELM; and (2) Optimally Pruned
The authors in [30] proposed an approach to predict the ELM (OP-ELM). In the ELM approach, the training con-
spread of powdery mildew on barley leaves by utilizing sisted of random generation of the hidden layer weights
hyperspectral image data. The authors used the cycle- followed by solving a linear system of equations by least
consistent adversarial networks (CycleGAN) which is a spe- squares for the estimation of the output layer weights. The
cial type of a generative adversarial network (GAN). The authors used several classes (Perdiz, Monsoy 8544, Monsoy
GAN consists of two neural networks termed as the generator 9010, Kaiabi and Tabarana) in their evaluation of datasets.

VOLUME 9, 2021 36705

 K. L.-M. Ang, J. K. P. Seng: Big Data and Machine Learning With Hyperspectral Information in Agriculture

Their experimental results showed that the best results were learning. Several deep learning approaches have been pro-
obtained with 70 bands which gave significant improvement posed for solving problems including image classification
over previous results reported in the literature. Furthermore, in agriculture. This subsection presents a review of some
the OP-ELM gave improved results over other state-of-the- recent representative studies on deep learning techniques for
art methods using only the information from one spectral multispectral and hyperspectral data analytics in agriculture.
band. The authors in [35] provided a study of pixel-based A summary of the representative works is shown in Table 4.
and object-based image analysis with machine learning algo- The authors in [39] presented a technical tutorial on the state
rithms for the classification of agricultural landscapes using of the art of deep learning approaches for remote sensing data.
SPOT-5 HRG imagery. The authors performed comparisons There are different approaches that have been proposed for
using three supervised machine learning algorithms (decision deep learning networks such as CNNs (convolutional neural
tree (DT), random forest (RF), and support vector machine networks), DBNs (deep belief networks), AEs (autoencoders)
(SVM)). Their experiments showed that all the three clas- and SCs (sparse coders). The CNN [40] is a multilayer net-
sifiers were able to depict the broad land cover types with work architecture composed of several stages for hierarchical
acceptable accuracies. One finding was that the RF and SVM representation and feature extraction. Each stage consists of
classifiers were able to give better predictions of riparian, three layers: (1) convolutional layer; (2) nonlinearity layer;
wetland and crop land cover types compared to the DT classi- and (3) pooling layer. The deep structure of CNNs allows the
fier which had more errors for these classes. Another finding network model to function as highly abstract feature detectors
was that the object-based analysis required more computa- and to map the input features into representations that can
tional time compared to the pixel-based analysis. improve the performance of the subsequent classification.
The authors in [36] proposed a machine learning approach The DBN [41] is a generative model that contain many layers
based on hyperspectral remote sensing and agricultural of hidden variables. The DBN is trained one layer at a time
factors (topography, soil, vegetation and meteorology) in an unsupervised manner by restricted Boltzmann machines
for modelling alpine grassland forage phosphorus. Their (RBMs). The AE [42] is a symmetrical neural network that is
approach utilized the correlation factors (CFs) and correlation used to learn the features from a data set in an unsupervised
bands (CBs) based on fifteen variables and four types of manner by minimizing the reconstruction error between the
spectral transformations (original spectral (OR), log spectral input data at the encoding layer and its reconstruction at the
(1/R), first derivative (FD) and continuum removal spectral decoding layer. The SC [43] is an unsupervised approach
(CR)). The authors used three classifier models (artificial for learning sets of overcomplete bases to represent data
neural network (ANN), support vector machine (SVM) and efficiently to find a set of basis vectors which can be used
random forest (RF)) in their approach for their experimental to represent an input vector as a linear combination of these
evaluation. Their results showed that the FD and CR spectral basis vectors.
models could retrieve more feature bands located in the NIR The authors in [67] presented an overview on spa-
and SWIR regions than the Log (1/R) and OR spectral models tial and spectral information fusion approaches and tech-
for the forage phosphorus estimation. Their work also showed niques for hyperspectral image classification. In their work,
that the combination of IBs and other factors (longitude and the authors grouped spatial-spectral information fusion
monthly mean temperature) increased the accuracy of the approaches into three categories: (1) segmentation-based
forage estimation when compared with the models that used approaches where objects are used for classification; (2) fea-
IBs alone. The FD-IBs + SVM model gave the optimum ture fusion approaches; and (3) decision fusion approaches
forage model and could account for 88% of the variation of where information from several classifiers are combined to
forage phosphorus in alpine grassland. achieve the final classification strategy. The authors reviewed
This sub-section has demonstrated the potential of deploy- different techniques in these categories. The performances
ing machine learning techniques for hyperspectral data ana- of various fusion methods were evaluated for classification
lytics in agriculture. The representative works which have accuracy and running time on popular hyperspectral datasets
been discussed show a wide variety of agriculture applica- including Indian Pines and Salinas. The results showed that
tions (e.g. crop mapping, prediction of plant diseases and the feature fusion methods could provide superior classifi-
stresses, classification of species, canopy measurements, etc.) cation accuracy compared to other methods at the cost of
which would benefit by the combination of machine learning requiring more computational and processing time.
techniques with hyperspectral data analytics. Some popu- The authors in [44] proposed a deep learning approach
lar machine learning approaches which have demonstrated for semantic segmentation termed as DeepLab to extract the
potential for agriculture applications include the SVM, IVM, spatial features of hyperspectral images. The first principal
MLP, ELM, discriminant analysis, random forest, etc. components were used as the label image for the DeepLab
training. Normalization was performed using the z-score on
D. DEEP LEARNING TECHNIQUES FOR HYPERSPECTRAL the original spectral bands and the extracted spatial features.
DATA ANALYTICS IN AGRICULTURE The spectral and spatial information were combined using a
In recent years, deep learning approaches have demonstrated weighted fusion rule and passed into a SVM for classifica-
significant improvements in the area of advanced machine tion. The proposed approach had two significant advantages

36706 VOLUME 9, 2021

K. L.-M. Ang, J. K. P. Seng: Big Data and Machine Learning With Hyperspectral Information in Agriculture

TABLE 4. Summary of representative works for deep learning techniques for hyperspectral data analytics in agriculture.

VOLUME 9, 2021 36707

 K. L.-M. Ang, J. K. P. Seng: Big Data and Machine Learning With Hyperspectral Information in Agriculture

when compared with other deep learning approaches: (HSIC). There are two stages in their approach for
(1) The spectral features are extracted at multiple scales; and spectral–spatial HSIC. The first stage extracts the spatial
(2) The approach avoids reduction of the spatial resolution. features of HSI pixel-to-pixel at multiple scales and avoids
The work was validated and demonstrated the superiority of the reduction of spatial resolution. This is followed by the
the DeepLab feature extraction method particularly for small weighted fusion of the spatial and spectral features. In the
scale classes which contains limited number of pixels. Other second stage, these fused features are input into the SVM for
examples of studies for using deep learning for hyperspec- the final classification. The performance of their framework
tral data analytics in agriculture can be found in [73]–[75] was tested on two well-known public HSI datasets includ-
and [77]. ing the Indian Pines dataset which lies in a predominantly
The authors in [45] proposed a deep learning feature agricultural region and the University of Pavia dataset and
extraction and classification of spectral-spatial HSI using a compared with some conventional deep learning techniques.
cross domain CNN model for classification. Their approach Their results revealed good classification performance and
used a guided filter to compute the filter output. The authors that the proposed framework outperformed other deep learn-
used three principal components from the HSI as the guided ing methods, especially for small scale classes.
image. The resultant spatial feature maps at different scales The authors in [49] proposed a fusion approach for the
were combined to generate the hyperspectral data cube con- identification of drug crops from remote sensing images.
taining the spatial features. The spatial feature vectors of each Their data-driven approach to characterize these drug crops
pixel were reshaped to form a two-dimensional image which takes into account the complementary information from the
was passed into the CNN for classification. The experimental NIR channel and false-colour image representations. The
results showed that the approach gave good classification different CNN architectures were applied to distinct image
accuracy and had a simple implementation while making full representations, which were able to represent complemen-
use of the available spatial features. tary characterizations of such crops. These representations
The authors in [46] proposed a hybrid CNN and trans- were then input to an ensemble of CNN classifiers using
former architecture for crop classification on multitem- multiple architectures. The approach was validated using
poral and multispectral data. In their research, a dataset a dataset containing Cannabis Sativa crops in a Brazil-
with 65 acquiring dates were collected from Sentinel-2 ian region called the Marijuana Polygon. Their proposed
A/B and Landsat-8 for a region in central California. Their approach gave high mean F-measure, accuracy and low
approach used two steps. The first step obtained scale- false detections, and demonstrated a promising approach
consistent feature and position features from the multitem- for machine-learning approaches for drug crops detection
poral sequence. In the second step, the encoder module was in remote sensing images. The authors in [50] proposed a
used to express the correlation of the sequence to obtain the seasonal land cover and crop classification approach using
depth characteristics of the sequence. The proposed CNN- the Deep CNN (DCNN) architecture. Their work investigated
transformer approach was evaluated on a dataset with a the pixel-based crops and land cover classification on sev-
crop matrix that included several crops (tomatoes, corn, rice, eral dates for the same agricultural season from the Sentinel
grapes, alfalfa, sunflower, clover, almonds, walnuts and spe- satellite. The experiments were performed for some major
cialty crops (watermelons, carrots, onions, peas). The classi- crops and land cover classification in Egypt. The architecture
fication results showed that the proposed CNN-transformer used 10 spectral bands from the Sentinel-2 satellite imagery
architecture resulted in a significant performance improve- during the winter season of 2016. The proposed architecture
ment compared with other traditional methods such as ran- was also compared with other techniques such as support
dom forest, SVM, and other deep learning (multitemporal vector machines (SVMs), random forests (RFs) and k-nearest
CNN and CNN-LSTM) models. neighbours (k-NNs). The results revealed that the DCNN
The authors in [47] proposed an approach for hyperspec- achieved about 89% average accuracy for major crops and
tral image classification using Hierarchical Stacked Sparse land cover classes.
Autoencoder (SSAE) networks to learn sparse feature rep- The authors in [51] proposed a deep learning framework
resentations. The SSAE networks were applied to extract with CNN and markov random fields (MRF) for spatial-
the spatial and spectral features. The ATL (active transfer spectral classification of hyperspectral images (HSI). Their
learning) sampling method was used to select a subset of approach can be summarised into two stages: (1) A CNN
the unlabeled samples for labelling and to add them to the model was built to learn the deep spectral features and the
training set at each iteration. The authors performed a com- classification of HSI and the class posterior probability dis-
prehensive evaluation on three popular hyperspectral data tribution was estimated. The input into the CNN was the pixel
sets including the Salinas Valley dataset which contains 204 vectors, thus the CNN is a pixel-classifier in the spectral
bands. Experimental results demonstrated that the proposed domain; and (2) The MRF-based multilevel logistic (MLL)
method gave promising performance compared with many prior encoded the spatial information to regularize the clas-
state-of-the-art approaches. sification result from CNN. The MRF-based loopy belief
The authors in [48] proposed a deep learning framework propagation (LBP) was used to learn the marginal probability
based on DeepLab for hyperspectral image classification distribution in HSI to derive the correlation for both the

36708 VOLUME 9, 2021

K. L.-M. Ang, J. K. P. Seng: Big Data and Machine Learning With Hyperspectral Information in Agriculture

spectral and spatial features. Their experiments used three grating module, SCOMS camera, and electric displacement
public datasets including University of Pavia dataset and platform. The authors used 146 hyperspectral images cubes
two agriculture related datasets (Indian Pines dataset and of 73 peanut samples before and after contamination by afla-
Salinas dataset). Their approach was compared with some toxin. Their CNN architecture consisted of five hidden lay-
state-of-the-art methods, and results revealed the good per- ers: (1) Input layer; (2) Convolution layer; (3) Sub-sampling
formance of their approach. The authors in [52] proposed an layer; (4) Convolution layer; and (5) Sub-sampling layer. The
approach for generating rice variety distribution maps using output layer was a fully connected layer. Their approach gave
deep CNN learning in spectral and temporal domains for recognition rates of 96% and 90% on pixel and kernel levels
Sentinel-2 data. In their work, the deep CNN network was respectively, and gave better results compared with traditional
applied towards separating rice varieties at the Coleambally classifiers such as KNN, SVM and BP-ANN. The authors
Irrigation Area, NSW, Australia, during the 2016-17 rice in [56] applied the deep learning algorithm based on CNN to
growing season. Five rice varieties (Reiziq, Sherpa, Topaz, classify agriculture and urban subclasses. The authors con-
YRM 70 and Langi) were investigated. Their experiments sidered two modalities, hyperspectral data and LiDAR data
investigated the separability of the rice varieties based on in their work. The hyperspectral data had the advantages of
the spectral and temporal patterns. The temporal curves for being able to identify the surface objects based on their mate-
two spectral indices NDVI and LSWI were charted over the rial composition. However, it has the disadvantages of failing
growing period. The performance of CNN was also compared the identification when two or more objects composed of the
with SVM. Their results showed that the deep CNN gave a same materials have different heights. On the other hand, the
classification accuracy of 92.87% compared to 57.49% with LiDAR data had the advantages of being able to discriminate
the SVM. Amongst the varieties, Sherpa gave the highest the objects of different heights. The complementary nature
producer accuracy of 98%. of both the data modalities are fused to increase the classifi-
The authors in [53] proposed a deep learning-based regres- cation accuracy. Their work used the dataset from National
sion approach to utilize hyperspectral data for the pre- Ecological Observatory Network (NEON) [68]. Using the
diction of cadmium residue in lettuce leaves. Their deep proposed methodology, a classified map was obtained with
learning approach consisted of stacked auto-encoders (SAE) an overall accuracy of 96% for the fused modalities.
and partial least squares support vector machine regression The authors in [57] proposed a framework for predicting
(LSSVR). Their approach was applied together with Vis- Ethiopian wheat fungal outbreaks using hyperspectral satel-
NIR HSI technique to obtain depth features for cadmium lite imagery and deep feature learning. The authors compared
prediction in lettuce leaf. In their approach, the Vis-NIR various deep learning models including Deep Neural Net-
hyperspectral images of 1120 lettuce leaf samples were col- works (DNNs), Recurrent Neural Networks (RNNs), Con-
lected from the region of lettuce leaf and pre-processed with volutional Neural Networks (CNNs) and Long Short-Term
spectral pre-treatment methods. The authors used several Memory Networks (LSTMs) to automatically learn the spec-
algorithms (Successive Projections Algorithm (SPA), Partial tral features. They evaluated all models with the following
Least Squares Regression (PLSR) and SAE) to locate the parameters (20-fold nested cross validation, minibatches of
optimum wavelengths. The LSSVR model was built based 16, dropout rate of 0.5, 40 histogram buckets, 16 filters of
on characteristic wavelengths. The results showed that the size 3 × 3, 1 unidirectional LSTM layer with 512 hidden cells
deep learning approach showed good potential for detecting and 64-unit fully connected layer). Their experimental results
heavy metal content in lettuce leaves. The authors in [54] demonstrated that the CNN and LSTM approach significantly
proposed a CNN model for classification of five varieties of outperformed that of traditional classifiers.
corn seedling cold damage recognition. Their approach aimed The authors in [58] proposed an approach for winter wheat
to extract spectral features in the Vis-NIR range to estimate yield estimation from multitemporal remote images using
the cold damage of corn seedlings. The pre-processing of CNN. In their approach, they applied histogram dimension-
spectral data was performed using application of Gaussian ality reduction and time series fusion to generate the input
low-pass filter and Savitzky-Golay smoothing method com- layer for the CNN. The CNN was built to extract the fea-
bined with its first-order derivative. The CNN modelling tures of winter wheat growth from multitemporal MODIS
using 3600 pixels were sampled from the region of interests. images for yield estimation in North China. It consisted of the
The CNN used a ten-layer model for classification accuracy input layer, seven convolution layers, seven activation layers,
and computational efficiency. Their results showed that the seven batch normalization layers, three dropout layers, two
proposed approach gave high correlation for different types full connection layers, and an output layer. Their work was
of corn seedlings given by the traditional chemical method implemented by TensorFlow and the results showed good
(W22 (41.8%), BxM (35%), B73 (25.6%), PH207 (20%) performance and that the estimated yield of winter wheat
and Mo17 (14%)), and demonstrated that spectral analysis based on time-series remote sensing images was highly cor-
based on CNN modelling could provide a useful technique related with statistical data (Pearson r value of 0.82), and
for detecting cold damage in corn seedlings. demonstrated that the CNN could provide a useful reference
The authors in [55] developed a hyperspectral imagery for estimating crop yield. The authors in [59] proposed a deep
system using CNN to detect aflatoxin in peanuts using a learning approach by combining subspace feature extraction

VOLUME 9, 2021 36709

 K. L.-M. Ang, J. K. P. Seng: Big Data and Machine Learning With Hyperspectral Information in Agriculture

and CNNs for hyperspectral image classification. There were whereas the non-agricultural included another four class cat-
two major steps in their approach: (1) Subspace-based feature egories (Residential, Sea-Lake, Highway, and Forest). The
extraction to reduce the dimensionality of the hyperspectral experimental results showed that the extra information used
images by calculating the orthonormal basis of correlation for the training data that were unfamiliar to the Greek data
matrix for each class; and (2) CNN hyperspectral image clas- decreased the performance of the CNN. The authors in [63]
sification using majority voting strategy applied to the output investigated approaches utilizing deep learning models for
of CNNs for each feature of certain classes. Experiments were classification of crop types from multi-spectral time series
conducted on two real hyperspectral data sets including the data. In this work, the authors proposed approaches using
Indian Pines dataset covering the agricultural Indian Pines convolutional, recurrent and hybrid neural networks for eval-
test site in Northwestern Indiana. Their results showed that uating the importance of spatial and temporal structures in
the proposed strategy gave a performance improvement com- the data. Their experiments were conducted on imagery from
pared to conventional feature extraction strategies. An overall Sentinel-2. Their results showed that the hybrid configura-
classification accuracy of 98.1% was obtained for the Indian tions which allocated most of the parameters (up to 90%) for
Pines dataset. modelling the temporal structure of the multi-spectral data
The authors in [60] proposed a novel Parallel Convolu- gave the best performance.
tional Neural Network (PCNN) architecture for the pixel- The authors in [64] applied deep learning methods for
wise identification and discrimination of crop types using the prediction of the severity of late blight in potato crops
AVIRIS-NG hyperspectral images. For band selection, two caused by Phytophthora infestans. Their work used a UAV
techniques PCA and back traversal of pre-trained ANN were to capture images of different phenotypes of potato crops
used to identify an optimal set of bands having higher inter- with a multispectral sensor. The authors performed com-
class separability and lower intra-class variability. To dis- parisons with other machine learning algorithms including
criminate different crop stages for the same crop type, two random forests, MLP and support vector regression. Their
different CNN models were trained separately using two sets results showed that the random forest and the CNN models
of crops. During the prediction phase, the results of both gave the best performance for the identification of infested
models were combined in parallel to decide the final class potato crops. The authors in [65] proposed a deep learning
label based on the highest probability. Their experimental method for spatial-spectral classification for hyperspectral
results showed that the PCNN achieved slightly higher per- images based on the single gate recurrent unit (GRU). The
formance than ANN on augmented test dataset consistently authors conducted experiments on the different input modes
after 5000 iterations with almost identical training parame- in GRU of spectral information and investigated different
ters. The PCNN achieved the best test accuracy of 99.1%, ways of fusing the spatial information. By comparing the
The authors in [61] aimed to investigate the possibility to different utilization patterns with several spatial information
separate one grapevine variety from an enlarged group of fusion methods, their approach demonstrated a higher per-
other varieties when the number of samples was significantly formance for accuracy and efficiency. Their experimental
increased. Their work was used to separate samples of one results on datasets revealed that their approach outperformed
variety from 63 other varieties. The SVM and CNN classi- other traditional and deep learning methods, and also had
fiers were applied to separate two varieties (Touriga Franca the advantages of extracting homogeneous discriminative
(TFvar) and Touriga Nacional (TNvar)) from all the remain- feature representations. The authors in [66] proposed a deep
ing varieties. The built classifiers used the one-vs-all binary metric learning (DML) neural network for the classification
type to indicate if a spectrum belonged to a certain variety or of hyperspectral images. Their work aimed to decrease the
not. Their work showed that it is possible to separate the leaf distances between same classes and increase the distances
spectra of TNvar or TFvar from the spectra of 62 other vari- between different classes by multilayers nonlinear projection.
eties. In the case of TNvar, the SVM gave better classification Their approach was different from other conventional metric
performance compared to the CNN. The SVM could classify learning methods where the proposed DML method had the
63% of the non-TNvar spectra and 81% of the TNvar spectra. capability to exploit the non-linear information between sam-
For TFvar, the CNN gave the best performance with the ples with multi-layers nonlinear transformation. The exper-
non-TFvar and the TFvar spectra with correct classification iments used three datasets (Indian Pines, Pavia University,
percentages of 91% and 93% respectively. and Salinas) to validate the proposed spatial-spectral DML
The authors in [62] utilized deep learning approaches method. Their experimental results showed that the proposed
to detecting agricultural and non-agricultural land. Their approach could achieve classification performance which
methodology was based on classification with CNNs and were comparable with other metric learning or deep models.
transfer learning using AlexNet. The area of study con- This sub-section has demonstrated the potential of deploy-
sisted of the Ionian islands in Greece. The study used two ing deep learning techniques for hyperspectral data analytics
datasets (EuroSAT and Demokritos) which were partitioned in agriculture. Several representative works which have been
into two categories (agricultural and non-agricultural). The discussed show that deep learning approaches significantly
agricultural category included four class categories (Annual outperformed that of traditional machine learning classifiers
Crop, Permanent Crop, Herbaceous Vegetation, and Pasture) for agriculture applications. The representative works which

36710 VOLUME 9, 2021

K. L.-M. Ang, J. K. P. Seng: Big Data and Machine Learning With Hyperspectral Information in Agriculture

FIGURE 2. 3D cube representation for Big hyperspectral data.

have been discussed show a wide variety of agriculture appli- application for agriculture. This section gives brief discus-
cations (e.g. semantic crop segmentation and classification, sions and explores the potential of ensemble machine learn-
land cover classification, drug crops identification, agricul- ing and scalable parallel discriminant analysis (SPDA) for
tural and non-agricultural land detection, grapevine identifi- agriculture information processing towards the application
cation, prediction of crop diseases, etc.) which would benefit of hyperspectral image classification. A similar approach to
by the combination of deep learning techniques with hyper- the proposed SPDA has been previously reported for human
spectral data analytics. Many studies employ the CNN deep emotion and sentiment classification from unstructured Big
learning model. Other deep learning approaches which have data [69]. However, the potential of ensemble machine learn-
demonstrated potential for agriculture applications include ing and scalable parallel discriminant analysis (EML-SPDA)
RNN, LSTM, DNN, DML, etc. has not been explored in agriculture information processing.
The approach utilizes a tree-based conquer and divide mech-
III. ENSEMBLE MACHINE LEARNING AND SCALABLE anism with an ensemble of classifiers. This part of the paper
PARALLEL DISCRIMINANT ANALYSIS FOR discusses the EML-SPDA to address Challenges (1) and (2)
HYPERSPECTRAL IMAGE CLASSIFICATION for Big hyperspectral data for agricultural systems. A differ-
The previous section (Section II) has given a comprehensive ence between the previous work and the proposed approach
overview of agriculture with Big data, machine learning and is that the work in [69] was targeted towards two-dimensional
deep learning for hyperspectral and multispectral information facial image data, whereas the proposed approach is targeted
processing. There are several challenges which need to be towards large volume three-dimensional (3-D) hyperspec-
further addressed to achieve the potential of Big data and tral spatial-spectral data cubes (i.e. Big hyperspectral data).
hyperspectral information processing in agriculture: (1) The The 3-D hyperspectral data cube structure requires a careful
need for efficient machine learning algorithms and classifiers, arrangement of the data information processing to preserve
and also to overcome the shortage of high-quality and labeled the spatial-spectral relationships and for the tree-based con-
training images (e.g. semi-supervised or weakly supervised quer and divide mechanism and parallel information process-
approaches); (2) The need for efficient and scalable compu- ing. The section first gives some discussions on the proposed
tational architectures for efficient information processing; (3) EML-SPDA approach and is then followed by details and
The need for standardization and ease of use for different discussions on experiments and data analytics to validate the
remote sensing formats and sensor resolutions particularly approach.
for non-expert users; and (4) The need for data management
systems to support the efficient storing and indexing of geo- A. DISCUSSIONS ON PROPOSED APPROACH
graphical metadata. Figure 2 shows the 3-D cube representation for Big hyper-
As discussed in Section II and illustrated in Tables 3 and 4, spectral data. The hyperspectral cube comprises of two spatial
hyperspectral image classification is a popular and important dimensions and one spectral dimension. The data in the cube

VOLUME 9, 2021 36711

 K. L.-M. Ang, J. K. P. Seng: Big Data and Machine Learning With Hyperspectral Information in Agriculture

FIGURE 3. Tree-based Conquer and Divide Mechanism.

is re-arranged (split) using a tree-based organization for the

conquer and divide mechanism for the parallel information
processing as shown in Figure 3. The mechanism first divides
the hyperspectral cube system into spatial-spectral localized
computational cells. The hyperspectral cube is first divided
into the horizonal planes called spatial-spectral planes and
each spatial-spectral plane is linearly separated into spatial-
spectral bands. The tree-based conquer and divide mech-
anism is then performed on these spatial-spectral bands.
The mechanism breaks the bands into smaller bands based
on multiple trees branched recursion. There are different
algorithms and techniques which can be applied to perform
the information processing using the proposed EML-SPDA
framework. For the hyperspectral image classification task,
we illustrate the conquer and divide approach using the linear
discriminant analysis (LDA) supervised machine learning
technique [72], [76], [78]. To perform the LDA using the
EML-SPDA approach, the 3-D hyperspectral cube is first
mapped into a two-dimensional array structure. Let X ∈
Rd×n = [X1 , X2 , . . . , Xk ] denote the data matrix partitioned
into k classes in which Xi ∈ Rd×n P denotes samples from the
i

ith class, i = 1,2,. . . , k, and n = ki=1 ni . Using the notations

of Sw , Sb , and St to denote the within-class scatter matrix,
between-class scatter matrix, and total scatter matrix respec-
tively, the LDA class separability criterion can be formulated
as
Tr GT Sb G


G = argmax
. (1) FIGURE 4. Algorithm for EML-SPDA LDA conquer and divide mechanism.
G Tr GT Sw G
Table 5 shows a summary of some notations used for the
EML-SPDA scheme. matrix R of size n × n and an orthogonal matrix Q of size
Figure 4 shows the algorithm to perform the conquer and m × n. The first split stage divides the d × n data matrix into
divide mechanism for the EML-SPDA LDA implementation even rows and odd rows containing two d/2×n sub-matrices.
using the RQ decomposition following a binary tree split- The second split stage further sub-divides into four sub-
ting and re-merging mechanism. The RQ decomposition is a matrices containing d/4×n elements. The RQ decomposition
counterpart to the well-known QR decomposition. The output is then performed on each of the sub-matrices to complete the
of a RQ decomposition for a m × n matrix is a diagonal splitting stage. For this EML-SPDA approach for LDA, on a

36712 VOLUME 9, 2021

K. L.-M. Ang, J. K. P. Seng: Big Data and Machine Learning With Hyperspectral Information in Agriculture

FIGURE 5. Performance accuracy on Indian Pines dataset for different classifiers.

TABLE 5. Summary of notations for EML-SPDA. Experiments: The first set of experiments demonstrates
the performance efficacy and the second set of experiments
demonstrates the speedup in computational times for EML-
SPDA which can be obtained with implementation on parallel
processing (in our case multicore) architectures. The exper-
iments aim to demonstrate the efficacy of the conquer-and-
divide mechanism for EML-SPDA on parallel architectures
using the binary tree row-based re-merging mechanisms.
Data: These set of experiments used the AVIRIS Indian
Pines dataset [70]. The Indian Pines dataset covers the
agricultural Indian Pines test site in Northwestern Indiana
multiprocessor computing platform, each RQ decomposition and was collected by the AVIRIS sensor. This dataset con-
can be allocated to be performed on a separate processing unit tains 16 classes or categories and is a cube size of 145 ×
to be computed in parallel. Note that Figure 4 only shows 145×220 with a spatial resolution of 20 m and a spectral
the splitting suitable for four computational processing units. range from 0.2 to 2.4 µm. Table 6 shows the class categories
Further stages of splitting can be performed to accommo- for the AVIRIS Indian Pines dataset.
date a computing hardware platform with a higher number Computational setup: These set of experiments used an
of processors. A significant advantage is that the number Intel i7 workstation with a 2.2-GHz CPU (4 cores) and 16 GB
of decompositions to be performed can be tailored to suit of RAM.
the computational capability (e.g. number of processors or Results & Discussion: Figure 5 shows the performance
cores) to achieve the meta-scalability information processing accuracy of EML-SPDA for the binary tree row-based con-
required for the architecture and platform. The re-merging quer and re-merging mechanisms using three different clas-
mechanism takes the separate RQ local outputs from the RQ sifiers (SVM, k-NN and ensemble trees) for the Indian Pines
splitting stages and together with the label of class vectors, C dataset. These classifiers were chosen to be representative of
combines the local outputs into a global output to obtain the the different classification approaches which are available.
transformation matrix, G for the LDA. Other classifiers (e.g. random forest classifiers, Bayesian
classifiers, logistic regression, etc.) could be used to perform
B. DISCUSSIONS ON EXPERIMENTS AND DATA the classification task. The random forest classifier is an
ANALYTICS example of an ensemble machine learning (EML) classifier.
This sub-section gives discussions on the experimental imple- Other examples of EML approaches are bagging, boosting
mentation and testing for the EML-SPDA and elaborates on and stacking. The ensemble tree approach used in the experi-
the datasets used, the computational setup and the results and ments employed adaptive boosted trees [82]. The SVM used
discussions. the Gaussian kernel, and the k-NN used a value of k = 10.

VOLUME 9, 2021 36713

 K. L.-M. Ang, J. K. P. Seng: Big Data and Machine Learning With Hyperspectral Information in Agriculture

FIGURE 6. Visual classification results for different samples/class.

FIGURE 7. Computational time on multicore architectures.

TABLE 6. AVIRIS Indian pines hyperspectral dataset and its class that the focus of the paper is more on the dimensionality
categories for agriculture.
reduction using the conquer-and-divide EML-SPDA LDA
mechanism, and less on experimenting with improved clas-
sifiers to improve the recognition performance. However,
we note that the EML-SPDA LDA performed comparably
in terms of classification accuracy with the methods and
techniques discussed in [71]. Furthermore, the results showed
improved accuracy as the number of samples used for training
was increased with a classification accuracy of 77.8% for
SVM. The results also showed that for the classifiers trained
using 20 samples/class or higher, the k-NN classifiers per-
formed comparably with the SVM. Using the lower complex-
ity k-NN classifiers compared with the more complex SVM
classifiers can give advantages trade-offs to reduce the imple-
mentation complexity at a slight reduction in performance
accuracy. Figure 6 shows some visual classification results
for the Indian Pines dataset using the SVM classifier with a
The classifiers were trained using a range of samples Gaussian kernel. Only the visual classification results for the
from 10 to 50 for each class. Amongst the classifiers, SVM classifier are shown because it was the best performing
the highest accuracy was obtained using the SVM. Note amongst the various classifiers. The leftmost columns show

36714 VOLUME 9, 2021

K. L.-M. Ang, J. K. P. Seng: Big Data and Machine Learning With Hyperspectral Information in Agriculture

FIGURE 8. Samples for ICONES hyperspectral dataset.

the ground truth results, and the columns moving towards the
right show the classification results for increasing number of
training samples/class.
An advantage of the EML-SPDA is the conquer-and-divide
mechanism for implementation speed-up on parallel com-
putational units. A further investigation was performed to
look at the computational time for the EML-SPDA algo-
rithm on multicore architectures for the different datasets.
The experiments were conducted on an Intel i7 workstation
with a 2.2-GHz CPU (4 cores) and 16 GB of RAM. The
comparison in Figure 7 shows the computational times for
different number of samples/class for the Indian Pines dataset
for running on one-core and four-core architectures. For the
dataset, the four-core splitting and re-merging architecture
gave a speedup of 1.22 times for the Indian Pines dataset and
demonstrating the usefulness of the proposed techniques. It is
expected that a higher speedup can be obtained on computa-
tional platforms with larger number of computational units
(e.g. GPU and massively parallel processors).
For a final investigation, we used a recently developed and FIGURE 9. Future work and challenges for Big data and hyperspectral
information processing in agriculture.
published large dataset termed as the ICONES Hyperspec-
tral Satellite Images Dataset (ICONES- HSI) [79]. To the
best of our knowledge, the ICONES-HSI dataset is the
largest hyperspectral (approximately 36GB) and most recent resulting in a data matrix of 20,160, 000 × 480. The dimen-
(published in 2019) dataset available for researchers. This sionality reduced data matrix was passed to two different
dataset contains 486 remote sensing patches of dimensions classifiers (SVM and ensemble tree) to perform the classi-
300 × 300 hyperspectral pixels which were generated from fication tasks which returned 98.8% and 94.4% recognition
the NASA JPL AVIRIS. The spectral radiance measure- rates respectively. Figure 9 shows a summary of future work
ment data is sampled in 224 contiguous spectral chan- and challenges for Big data and hyperspectral information
nels/bands between 365 and 2497 nm. The patches in the processing in agriculture.
dataset are classified into nine categories (Agriculture, Forest,
Desert, Urban, Snow, Mountain, Ocean, Wetland and Cloud). IV. CONCLUSION AND FUTURE WORK
Figure 8 shows some representative samples for the nine Big data and machine learning in remote sensing for agri-
categories. The spatial-spectral feature for a patch contains culture is very promising. This paper has provided a com-
300 × 300×224 pixel measurements. In our experiments, prehensive review of the research efforts in remote sensing
we did not use the last six patches for the Cloud category in agriculture using Big data and machine learning. There

VOLUME 9, 2021 36715

 K. L.-M. Ang, J. K. P. Seng: Big Data and Machine Learning With Hyperspectral Information in Agriculture

are several challenges which need to be further addressed [12] L.-M. Ang, K. P. Seng, A. M. Zungeru, and G. K. Ijemaru, ‘‘Big sensor
to achieve the potential of Big data and hyperspectral infor- data systems for smart cities,’’ IEEE Internet Things J., vol. 4, no. 5,
pp. 1259–1271, Oct. 2017.
mation processing in agriculture: (1) The need for efficient [13] L.-M. Ang and K. P. Seng, ‘‘Big sensor data applications in urban environ-
machine learning algorithms and classifiers, and also to over- ments,’’ Big Data Res., vol. 4, pp. 1–12, Jun. 2016.
come the shortage of high-quality and labeled training images [14] X. Liu, H. Zhai, Y. Shen, B. Lou, C. Jiang, T. Li, S. B. Hussain, and
G. Shen, ‘‘Large-scale crop mapping from multisource remote sensing
(e.g. semi-supervised or weakly supervised approaches); images in Google Earth engine,’’ IEEE J. Sel. Topics Appl. Earth Observ.
(2) The need for efficient and scalable computational archi- Remote Sens., vol. 13, no. 13, pp. 414–427, 2020.
tectures for rapid information processing; (3) The need for [15] J. Kim, S. Kim, C. Ju, and H. Il Son, ‘‘Unmanned aerial vehicles in agricul-
ture: A review of perspective of platform, control, and applications,’’ IEEE
standardization and ease of use for remote sensing formats Access, vol. 7, pp. 105100–105115, 2019.
and sensor resolutions particularly for non-expert users; and [16] N. L. Tsakiridis, J. B. Theocharis, and G. C. Zalidis, ‘‘An evolution-
(4) The need for data management systems to support the effi- ary fuzzy rule-based system applied to real-world Big Data-the GEO-
CRADLE and LUCAS soil spectral libraries,’’ in Proc. IEEE Int. Conf.
cient storing and indexing of geographical metadata. The lat- Fuzzy Syst. (FUZZ-IEEE), Jul. 2018, pp. 1–8.
ter part of the paper has proposed the EML-SPDA to address [17] J. Xia, B. Huang, Y. W. Yang, H. X. Cao, W. Zhang, L. Xu, Q. Wan,
Challenges (1) and (2) for Big hyperspectral data in agri- Y. Ke, W. Zhang, and D. Ge, ‘‘Hyperspectral identification and classifica-
tion of oilseed rape waterlogging stress levels using parallel computing,’’
cultural information processing. For Challenge (1), the LDA IEEE Access, vol. 6, pp. 57663–57675, 2018.
EML-SPDA can perform comparably with other state-of-the- [18] S. Reshma, S. Veni, and J. E. George, ‘‘Hyperspectral crop classification
art methods although these methods are not designed for using fusion of spectral, spatial features and vegetation indices: Approach
to the big data challenge,’’ in Proc. Int. Conf. Adv. Comput., Commun.
scalability and parallel processing for hyperspectral data. The Informat. (ICACCI), Sep. 2017, pp. 380–386.
experimental results have validated the performance of the [19] S. A. El_Rahman, ‘‘Big data analysis: Hyperspectral image processing
approach. For Challenge (2), the EML-SPDA has addressed for agriculture applications,’’ Int. J. Comput. Digit. Syst., vol. 5, no. 4,
pp. 225–234, Jul. 2016.
the challenge of traditional conquer-and-divide mechanism [20] D. Wang, F. Zheng, J. Lai, T. Yu, J. Li, and S. Guo, ‘‘A new parallel algo-
which breaks and recursively solves the subproblems of the rithm based on five-layer fifteen-level remote sensing data organization,’’
original, and finally combines the solutions to the subprob- Microcomput. Inf., vol. 1, pp. 1–5, Mar. 2012.
[21] Y. T. Solano-Correa, F. Bovolo, L. Bruzzone, and D. Fernandez-Prieto,
lems but does not guarantee the optimal solutions for discrim- ‘‘A method for the analysis of small crop fields in Sentinel-2 dense time
inative analytics. The ensemble parallelism machine learning series,’’ IEEE Trans. Geosci. Remote Sens., vol. 58, no. 3, pp. 2150–2164,
which can be used with many existing machine learning Mar. 2020.
[22] Y. J. E. Gbodjo, D. Ienco, and L. Leroux, ‘‘Toward spatio–spectral analysis
techniques has also been proposed for applications involving of Sentinel-2 time series data for land cover mapping,’’ IEEE Geosci.
Big hyperspectral classification or prediction. In the future, Remote Sens. Lett., vol. 7, no. 2, pp. 307–311, Feb. 2020.
we plan to extend our work by incorporating and re-designing [23] R. Roscher, B. Waske, and W. Forstner, ‘‘Incremental import vector
machines for classifying hyperspectral data,’’ IEEE Trans. Geosci. Remote
other data analytics into our proposed framework to further Sens., vol. 50, no. 9, pp. 3463–3473, Sep. 2012.
address the above challenges. [24] R. Koppaka and T.-S. Moh, ‘‘Machine learning in indian crop classifica-
tion of temporal multi-spectral satellite image,’’ in Proc. 14th Int. Conf.
Ubiquitous Inf. Manage. Commun. (IMCOM), Jan. 2020, pp. 1–8.
REFERENCES [25] S. D. Fabiyi, H. Vu, C. Tachtatzis, P. Murray, D. Harle, T. K. Dao,
[1] M. C. Hunter, R. G. Smith, M. E. Schipanski, L. W. Atwood, and I. Andonovic, J. Ren, and S. Marshall, ‘‘Varietal classification of rice
D. A. Mortensen, ‘‘Agriculture in 2050: Recalibrating targets for sustain- seeds using RGB and hyperspectral images,’’ IEEE Access, vol. 8,
able intensification,’’ BioScience, vol. 67, no. 4, pp. 386–391, Apr. 2017. pp. 22493–22505, 2020.
[2] S. Wolfert, L. Ge, C. Verdouw, and M. J. Bogaardt, ‘‘Big data in smart [26] Q. Han, Y. Li, and L. Yu, ‘‘Classification of glycyrrhiza seeds by near
farming–a review,’’ Agricult. Syst., vol. 153, pp. 69–80, May 2017. infrared hyperspectral imaging technology,’’ in Proc. Int. Conf. High Per-
[3] R. H. L. Ip, L. M. Ang, K. P. Seng, J. C. Broster, and J. E. Pratley, ‘‘Big data form. Big Data Intell. Syst. (HPBD&IS), May 2019, pp. 141–145.
and machine learning for crop protection,’’ Comput. Electron. Agricult., [27] W. Liao, D. Ochoa, L. Gao, B. Zhang, and W. Philips, ‘‘Morphological
vol. 151, pp. 376–383, Aug. 2018. analysis for banana disease detection in close range hyperspectral remote
[4] Y. Huang, Z.-X. Chen, T. Yu, X.-Z. Huang, and X.-F. Gu, ‘‘Agricultural sensing images,’’ in Proc. IEEE Int. Geosci. Remote Sens. Symp. (IGARSS),
remote sensing big data: Management and applications,’’ J. Integrative Jul. 2019, pp. 3697–3700.
Agricult., vol. 17, no. 9, pp. 1915–1931, Sep. 2018. [28] C. M. Gevaert, J. Suomalainen, J. Tang, and L. Kooistra, ‘‘Generation of
spectral–temporal response surfaces by combining multispectral satellite
[5] E. Alpaydin, Introduction to Machine Learning. Cambridge, MA, USA:
and hyperspectral UAV imagery for precision agriculture applications,’’
MIT Press, 2020.
IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 8, no. 6,
[6] K. Liakos, P. Busato, D. Moshou, S. Pearson, and D. Bochtis, ‘‘Machine
pp. 3140–3146, Jun. 2015.
learning in agriculture: A review,’’ Sensors, vol. 18, no. 8, p. 2674,
[29] D. Lee, ‘‘Mapping agricultural tillage practices using extreme learn-
Aug. 2018.
ing machine,’’ in Proc. 8th Int. Conf. Agro-Geoinformatics (Agro-
[7] Y. LeCun, Y. Bengio, and G. Hinton, ‘‘Deep learning,’’ Nature, vol. 521, Geoinformatics), Jul. 2019, pp. 1–4.
pp. 436–444, May 2015. [30] A. Forster, J. Behley, J. Behmann, and R. Roscher, ‘‘Hyperspectral plant
[8] A. Kamilaris, A. Kartakoullis, and F. X. Prenafeta-Boldú, ‘‘A review on the disease forecasting using generative adversarial networks,’’ in Proc. IEEE
practice of big data analysis in agriculture,’’ Comput. Electron. Agricult., Int. Geosci. Remote Sens. Symp. (IGARSS), Jul. 2019, pp. 1793–1796.
vol. 143, pp. 23–37, Dec. 2017. [31] A. Masjedi, J. Zhao, A. M. Thompson, K.-W. Yang, J. E. Flatt,
[9] N. Tantalaki, S. Souravlas, and M. Roumeliotis, ‘‘Data-driven decision M. M. Crawford, D. S. Ebert, M. R. Tuinstra, G. Hammer, and
making in precision agriculture: The rise of big data in agricultural sys- S. Chapman, ‘‘Sorghum biomass prediction using uav-based remote sens-
tems,’’ J. Agricult. Food Inf., vol. 20, no. 4, pp. 344–380, Oct. 2019. ing data and crop model simulation,’’ in Proc. IEEE Int. Geosci. Remote
[10] Y. Ma, H. Wu, L. Wang, B. Huang, R. Ranjan, A. Zomaya, and Sens. Symp. (IGARSS), Jul. 2018, pp. 7719–7722.
W. Jie, ‘‘Remote sensing big data computing: Challenges and opportuni- [32] D. Han, Q. Du, and N. H. Younan, ‘‘Semisupervised classification of hyper-
ties,’’ Future Gener. Comput. Syst., vol. 51, pp. 47–60, Oct. 2015. spectral remote sensing images with spatial majority voting,’’ in Proc.
[11] A. Kamilaris and F. X. Prenafeta-Boldú, ‘‘Deep learning in agriculture: A 9th IAPR Workshop Pattern Recogniton Remote Sens. (PRRS), Dec. 2016,
survey,’’ Comput. Electron. Agricult., vol. 147, pp. 70–90, Apr. 2018. pp. 1–4.

36716 VOLUME 9, 2021

K. L.-M. Ang, J. K. P. Seng: Big Data and Machine Learning With Hyperspectral Information in Agriculture

[33] T. Poblete, S. Ortega-Farías, M. A. Moreno, and M. Bardeen, ‘‘Artificial [54] W. Yang, C. Yang, Z. Hao, C. Xie, and M. Li, ‘‘Diagnosis of plant
neural network to predict vine water status spatial variability using mul- cold damage based on hyperspectral imaging and convolutional neural
tispectral information obtained from an unmanned aerial vehicle (UAV),’’ network,’’ IEEE Access, vol. 7, pp. 118239–118248, 2019.
Sensors, vol. 17, no. 11, p. 2488, 2017. [55] Z. Han and J. Gao, ‘‘Pixel-level aflatoxin detecting based on deep learn-
[34] R. Moreno, F. Corona, A. Lendasse, M. Graña, and L. S. Galvão, ‘‘Extreme ing and hyperspectral imaging,’’ Comput. Electron. Agricult., vol. 164,
learning machines for soybean classification in remote sensing hyperspec- Sep. 2019, Art. no. 104888.
tral images,’’ Neurocomputing, vol. 128, pp. 207–216, Mar. 2014. [56] S. N. Chaudhri, N. S. Rajput, K. P. Singh, and D. Singh, ‘‘Different modal-
[35] D. C. Duro, S. E. Franklin, and M. G. Dubé, ‘‘A comparison of pixel- ity based remote sensing data fusion approach for efficient classification
based and object-based image analysis with selected machine learning of agriculture and urban subclasses,’’ in Proc. IEEE Int. Geosci. Remote
algorithms for the classification of agricultural landscapes using SPOT-5 Sens. Symp. (IGARSS), Jul. 2019, pp. 5710–5713.
HRG imagery,’’ Remote Sens. Environ., vol. 118, pp. 259–272, Mar. 2012. [57] R. Pryzant, S. Ermon, and D. Lobell, ‘‘Monitoring ethiopian wheat fun-
[36] J. Gao, B. Meng, T. Liang, Q. Feng, J. Ge, J. Yin, C. Wu, X. Cui, gus with satellite imagery and deep feature learning,’’ in Proc. IEEE
M. Hou, J. Liu, and H. Xie, ‘‘Modeling alpine grassland forage phosphorus Conf. Comput. Vis. Pattern Recognit. Workshops (CVPRW), Jul. 2017,
based on hyperspectral remote sensing and a multi-factor machine learning pp. 39–47.
algorithm in the east of tibetan plateau, China,’’ ISPRS J. Photogramm. [58] H. Mu, L. Zhou, X. Dang, and B. Yuan, ‘‘Winter wheat yield estimation
Remote Sens., vol. 147, pp. 104–117, Jan. 2019. from multitemporal remote sensing images based on convolutional neural
[37] ENVI. (2009). ENVI Reference Guide: ENVI Version 4.7. [Online]. Avail- networks,’’ in Proc. 10th Int. Workshop Anal. Multitemporal Remote Sens.
able: https://www.l3harrisgeospatial.com/docs/using_envi_Home.html Images (MultiTemp), Aug. 2019, pp. 1–4.
[38] Z. Zhang, A. Masjedi, J. Zhao, and M. M. Crawford, ‘‘Prediction of [59] T. Alipourfard, H. Arefi, and S. Mahmoudi, ‘‘A novel deep learning
sorghum biomass based on image based features derived from time series framework by combination of subspace-based feature extraction and
of UAV images,’’ in Proc. IEEE Int. Geosci. Remote Sens. Symp. (IGARSS), convolutional neural networks for hyperspectral images classification,’’
Jul. 2017, pp. 6154–6157. in Proc. IEEE Int. Geosci. Remote Sens. Symp. (IGARSS), Jul. 2018,
[39] L. Zhang, L. Zhang, and B. Du, ‘‘Deep learning for remote sensing data: A pp. 4780–4783.
technical tutorial on the state of the art,’’ IEEE Geosci. Remote Sens. Mag., [60] H. Patel, N. Bhagia, T. Vyas, B. Bhattacharya, and K. Dave, ‘‘Crop iden-
vol. 4, no. 2, pp. 22–40, Jun. 2016. tification and discrimination using AVIRIS-NG hyperspectral data based
[40] A. Krizhevsky, I. Sutskever, and G. E. Hinton, ‘‘Imagenet classification on deep learning techniques,’’ in Proc. IEEE Int. Geosci. Remote Sens.
with deep convolutional neural networks,’’ in Proc. Adv. Neural Inf. Pro- Symp. (IGARSS), Jul. 2019, pp. 3728–3731.
cess. Syst., 2012, pp. 1097–1105. [61] A. M. Fernandes, A. B. Utkin, J. Eiras-Dias, J. Cunha, J. Silvestre, and
[41] G. E. Hinton, S. Osindero, and Y.-W. Teh, ‘‘A fast learning algorithm for P. Melo-Pinto, ‘‘Grapevine variety identification using ‘Big Data’ collected
deep belief nets,’’ Neural Comput., vol. 18, no. 7, pp. 1527–1554, Jul. 2006. with miniaturized spectrometer combined with support vector machines
[42] P. Vincent, H. Larochelle, Y. Bengio, and P.-A. Manzagol, ‘‘Extracting and and convolutional neural networks,’’ Comput. Electron. Agricult., vol. 163,
composing robust features with denoising autoencoders,’’ in Proc. 25th Int. Aug. 2019, Art. no. 104855.
Conf. Mach. Learn. (ICML), 2008, pp. 1096–1103. [62] E. Charou, G. Felekis, D. B. Stavroulopoulou, M. Koutsoukou,
[43] H. Lee, A. Battle, R. Raina, and A. Y. Ng, ‘‘Efficient sparse coding A. Panagiotopoulou, Y. Voutos, E. Bratsolis, P. Mylonas, and
algorithms,’’ in Proc. Adv. Neural Inf. Process. Syst., 2007, pp. 801–808. L. Likforman-Sulem, ‘‘Deep learning for agricultural land detection
[44] L.-C. Chen, G. Papandreou, I. Kokkinos, K. Murphy, and in insular areas,’’ in Proc. 10th Int. Conf. Inf., Intell., Syst. Appl. (IISA),
A. L. Yuille, ‘‘Semantic image segmentation with deep convolutional nets Jul. 2019, pp. 1–4.
and fully connected CRFs,’’ 2014, arXiv:1412.7062. [Online]. Available: [63] V. S. F. Garnot, L. Landrieu, S. Giordano, and N. Chehata, ‘‘Time-space
http://arxiv.org/abs/1412.7062 tradeoff in deep learning models for crop classification on satellite multi-
[45] Y. Guo, H. Cao, J. Bai, and Y. Bai, ‘‘High efficient deep feature extrac- spectral image time series,’’ in Proc. IEEE Int. Geosci. Remote Sens.
tion and classification of spectral-spatial hyperspectral image using cross Symp. (IGARSS), Jul. 2019, pp. 6247–6250.
domain convolutional neural networks,’’ IEEE J. Sel. Topics Appl. Earth [64] J. Duarte-Carvajalino, D. Alzate, A. Ramirez, J. Santa-Sepulveda,
Observ. Remote Sens., vol. 12, no. 1, pp. 345–356, Jan. 2019. A. Fajardo-Rojas, and M. Soto-Suárez, ‘‘Evaluating late blight
[46] Z. Li, G. Chen, and T. Zhang, ‘‘A CNN-transformer hybrid approach for severity in potato crops using unmanned aerial vehicles and machine
crop classification using multitemporal multisensor images,’’ IEEE J. Sel. learning algorithms,’’ Remote Sens., vol. 10, no. 10, p. 1513,
Topics Appl. Earth Observ. Remote Sens., vol. 13, no. 13, pp. 847–858, Sep. 2018.
2020. [65] E. Pan, X. Mei, Q. Wang, Y. Ma, and J. Ma, ‘‘Spectral-spatial classifica-
[47] C. Deng, Y. Xue, X. Liu, C. Li, and D. Tao, ‘‘Active transfer learning tion for hyperspectral image based on a single GRU,’’ Neurocomputing,
network: A unified deep joint spectral–spatial feature learning model for vol. 387, pp. 150–160, Apr. 2020.
hyperspectral image classification,’’ IEEE Trans. Geosci. Remote Sens., [66] X. Cao, Y. Ge, R. Li, J. Zhao, and L. Jiao, ‘‘Hyperspectral imagery classifi-
vol. 57, no. 3, pp. 1741–1754, Mar. 2019. cation with deep metric learning,’’ Neurocomputing, vol. 356, pp. 217–227,
[48] Z. Niu, W. Liu, J. Zhao, and G. Jiang, ‘‘DeepLab-based spatial feature Sep. 2019.
extraction for hyperspectral image classification,’’ IEEE Geosci. Remote [67] M. Imani and H. Ghassemian, ‘‘An overview on spectral and spatial
Sens. Lett., vol. 16, no. 2, pp. 251–255, Feb. 2019. information fusion for hyperspectral image classification: Current trends
[49] A. Ferreira, S. C. Felipussi, R. Pires, S. Avila, G. Santos, J. Lambert, and challenges,’’ Inf. Fusion, vol. 59, pp. 59–83, Jul. 2020.
J. Huang, and A. Rocha, ‘‘Eyes in the skies: A data-driven fusion approach [68] National Ecological Observatory Network (NEON). Data Products
to identifying drug crops from remote sensing images,’’ IEEE J. Sel. NEON.DP1.30010.001, NEON.DP3.30006.001, NEON.DP1.30003.001,
Topics Appl. Earth Observ.Remote Sens., vol. 12, no. 12, pp. 4773–4786, NEON.DP3.30024.001. Battelle, Boulder, CO, USA. Accessed:
Dec. 2019. Oct. 20, 2018. [Online]. Available: http://data.neonscience.org
[50] N. Laban, B. Abdellatif, H. M. Ebeid, H. A. Shedeed, and M. F. Tolba, [69] J. Kah Phooi Seng and K. Li-Minn Ang, ‘‘Multimodal emotion
‘‘Seasonal multi-temporal pixel based crop types and land cover classifi- and sentiment modeling from unstructured big data: Challenges,
cation for satellite images using convolutional neural networks,’’ in Proc. architecture, & techniques,’’ IEEE Access, vol. 7, pp. 90982–90998,
13th Int. Conf. Comput. Eng. Syst. (ICCES), Dec. 2018, pp. 21–26. 2019.
[51] C. Qing, J. Ruan, X. Xu, J. Ren, and J. Zabalza, ‘‘Spatial-spectral classifi- [70] M. F. Baumgardner, L. L. Biehl, and D. A. Landgrebe, ‘‘Band AVIRIS
cation of hyperspectral images: A deep learning framework with Markov hyperspectral image data set,’’ in Purdue University Research
random fields based modelling,’’ IET Image Process., vol. 13, no. 2, Repository, 2015, p. 220. [Online]. Available: https://purr.purdue.edu/
pp. 235–245, Feb. 2019. publications/1947/about?v=1
[52] Y. Guo, X. Jia, and D. Paull, ‘‘Mapping of rice varieties with Sentinel-2 [71] F. Leyuan, N. He, S. Li, J. Anotonio Plaza, and P. Javier, ‘‘A new spatial–
data via deep CNN learning in spectral and time domains,’’ in Proc. Digit. spectral feature extraction method for hyperspectral images using local
Image Comput., Techn. Appl. (DICTA), Dec. 2018, pp. 1–7. covariance matrix representation,’’ IEEE Trans. Geosci. Remote Sens.,
[53] Z. Xin, S. Jun, T. Yan, C. Quansheng, W. Xiaohong, and H. Yingying, vol. 56, no. 6, pp. 3534–3546, Jun. 2018.
‘‘A deep learning based regression method on hyperspectral data for rapid [72] Z. Fan, Y. Xu, and D. Zhang, ‘‘Local linear discriminant analysis frame-
prediction of cadmium residue in lettuce leaves,’’ Chemometric Intell. Lab. work using sample neighbors,’’ IEEE Trans. Neural Netw., vol. 22, no. 7,
Syst., vol. 200, May 2020, Art. no. 103996. pp. 1119–1132, Jul. 2011.

VOLUME 9, 2021 36717

 K. L.-M. Ang, J. K. P. Seng: Big Data and Machine Learning With Hyperspectral Information in Agriculture

[73] J. Feng, J. Chen, L. Liu, X. Cao, X. Zhang, L. Jiao, and T. Yu, ‘‘CNN- KENNETH LI-MINN ANG (Senior Member, IEEE) received the B.Eng.
based multilayer spatial–spectral feature fusion and sample augmentation and Ph.D. degrees from Edith Cowan University, Australia. He was an
with local and nonlocal constraints for hyperspectral image classification,’’ Associate Professor of Networked and Computer Systems with the School
IEEE J. Sel. Topics Appl. Earth Observ. Remote Sens., vol. 12, no. 4, of Information and Communication Technology (ICT), Griffith University.
pp. 1299–1313, Apr. 2019. He is currently a Professor with the School of Science and Engineering,
[74] N. He, L. Fang, S. Li, A. Plaza, and J. Plaza, ‘‘Remote sensing scene University of Sunshine Coast. He has published over 180 articles in journals
classification using multilayer stacked covariance pooling,’’ IEEE Trans. and international refereed conferences. His research interests include big
Geosci. Remote Sens., vol. 56, no. 12, pp. 6899–6910, Dec. 2018.
data analytics, multimedia Internet-of-Things, embedded systems, wireless
[75] X. Xu, W. Li, Q. Ran, Q. Du, L. Gao, and B. Zhang, ‘‘Multisource remote
multimedia sensor systems, reconfigurable computing, development of real-
sensing data classification based on convolutional neural network,’’ IEEE
Trans. Geosci. Remote Sens., vol. 56, no. 2, pp. 937–949, Feb. 2018. world computer systems, and machine learning. He is a Fellow of the Higher
[76] W. Li, F. Feng, H. Li, and Q. Du, ‘‘Discriminant analysis-based dimension Education Academy, U.K.
reduction for hyperspectral image classification: A survey of the most
recent advances and an experimental comparison of different techniques,’’
IEEE Geosci. Remote Sens. Mag., vol. 6, no. 1, pp. 15–34, Mar. 2018.
[77] J. Yang, Y. Q. Zhao, and J. C. W. Chan, ‘‘Learning and transferring
deep joint spectral–spatial features for hyperspectral classification,’’ IEEE
Trans. Geosci. Remote Sens., vol. 55, no. 8, pp. 4729–4742, Aug. 2017.
[78] J. K. P. Seng and K. L.-M. Ang, ‘‘Big feature data analytics: Split and JASMINE KAH PHOOI SENG (Member, IEEE) received the B.Eng. and
combine linear discriminant analysis (SC-LDA) for integration towards Ph.D. degrees from the University of Tasmania, Australia. She is currently
decision making analytics,’’ IEEE Access, vol. 5, pp. 14056–14065, 2017. an Adjunct Professor with the School of Engineering and Information Tech-
[79] O. Ben-Ahmed, T. Urruty, N. Richard, and C. Fernandez-Maloigne, nology, UNSW. Before returning to Australia, she was a Professor and the
‘‘Toward content-based hyperspectral remote sensing image retrieval (CB- Department Head of Computer Science and Networked System with Sunway
HRSIR): A preliminary study based on spectral sensitivity functions,’’ University. Before joining Sunway University, she was an Associate Profes-
Remote Sens., vol. 11, no. 5, p. 600, Mar. 2019.
sor with the School of Electrical and Electronic Engineering, Nottingham
[80] F. Luo, H. Huang, Y. Duan, J. Liu, and Y. Liao, ‘‘Local geometric structure
University. She has published over 230 papers in journals and international
feature for dimensionality reduction of hyperspectral imagery,’’ Remote
Sens., vol. 9, no. 8, p. 790, Aug. 2017. refereed conferences. She is the lead author of the book Multimodal Analytics
[81] L. Zhang and F. Luo, ‘‘Review on graph learning for dimensionality for Next-Generation Big Data Technologies and Applications. Her research
reduction of hyperspectral image,’’ Geo-Spatial Inf. Sci., vol. 23, no. 1, interests include data analytics, Big data, machine learning, Artificial Intel-
pp. 98–106, Jan. 2020. ligence (AI) and intelligent systems, Internet of Things (IoT), multimodal
[82] P. Viola and M. Jones, ‘‘Fast and robust classification using asymmetric signal processing, pervasive computing and sensor networks, HCI and affec-
adaboost and a detector cascade,’’ in Proc. Adv. Neural Inf. Process. Syst., vative computing, and mobile software development.
2002, pp. 1311–1318.

36718 VOLUME 9, 2021