This article has been accepted for publication in IEEE Access.

This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2024.3355017

Received ** *** ****, accepted ** *** ****, date of publication * *** ***, date of current version ** *** ****.
Digital Object Identifier **.****/ACCESS.2023.DOI

Revolutionizing Agriculture: Machine

and Deep Learning Solutions for
Enhanced Crop Quality and Weed
Control
SYED MUJTABA HASSAN RIZVI 1 ,ASMA NASEER2 ,SHAFIQ UR REHMAN3 ,SHEERAZ
AKRAM4 , and VOLKER GRUHN5
1
Department of Computer Science, National University of Computer and Emerging Sciences Lahore, Pakistan (e-mail: [email protected])
2
Department of Computer Science, National University of Computer and Emerging Sciences Lahore, Pakistan (e-mail: [email protected])
3
College of Computer and Information Sciences, Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh, Kingdom of Saudi Arabia (e-mail:
[email protected])
4
College of Computer and Information Sciences, Imam Mohammad Ibn Saud Islamic University (IMSIU), Riyadh, Kingdom of Saudi Arabia (e-mail:
[email protected])
5
Universität Duisburg-Essen, Essen Germany (e-mail: [email protected])
Corresponding author: SHAFIQ UR REHMAN (e-mail: [email protected])

ABSTRACT Agricultural systems are being revolutionized due to emerging technologies that aim to
make improvements in the traditional agriculture system. The major goal is not just to enhance agricultural
output per hectare but also to enhance crop quality while protecting the natural environment. Weeds pose a
significant threat to crops as they consume nutrients, water, and light, thereby reducing crop productivity.
Spraying the entire field uniformly to control weeds not only incurs high costs but also has adverse
environmental effects. To address the limitations of conventional weed control methods, in this research,
Machine Learning (ML) and Deep Learning (DL) based techniques are proposed to identify and categorize
weeds in crops. For ML-based techniques, several statistical and texture-based features are extracted,
including central image and Hu moments, mean absolute deviation, Shannon entropy, gray level co-
occurrence matrix (GLCM) and local binary patterns (LBP), contrast, energy, homogeneity, dissimilarity,
correlation, and summarized local binary pattern histogram. YOLOv8m is employed to identify weeds
and for weed classification, features extracted from two standard benchmark datasets, CottonWeedID15
and Earlycrop-weed are fed to Support Vector Machine (SVM), Random Forest, and Artificial Neural
Network (ANN) while employing Synthetic Minority Oversampling Technique (SMOTE) to balance
the classes. In addition to ML-based techniques, Deep learners such as VGG16, VGG19, Xception,
DenseNet121, DenseNet169, DenseNet201, and ConvNeXtBase are trained on raw data with balanced
classes for automated feature extraction and classification. Among the ML-based techniques, SVM with a
polynomial kernel achieves 99.5% accuracy on the early crop weed dataset, and Artificial neural network
attains 89% accuracy on the Cottonweedid15 dataset. Meanwhile, the combined employment of ConvNeXt
and Random Forest results in the highest accuracy among DLs, specifically 98% on the early crop weed
dataset and 90% on the Cottonweedid15 dataset. The high accuracy achieved underscores the practical
viability of these methods, offering a sustainable and cost-effective solution for modern agriculture.

INDEX TERMS ConvNeXtBase, DenseNet, Generative AI, Smart Agriculture, VGG, Xception,
YOLOv8.

I. INTRODUCTION the population, the demand of food is also increasing.

To meet this demand, agricultural production needs to be
VERY year, the world’s population is growing at a
E rate of 1.09%. By 2050, it is projected that the global
population will reach 9 billion. Due to the increase in
increased by 70% [1]. However, the agricultural sector faces
many challenges, such as lack of cultivable land, saline

VOLUME 4, 2023 1

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4
 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2024.3355017

Syed Mujtaba H R et al.:Revolutionizing Agriculture: Machine and Deep Learning Solutions for Enhanced Crop Quality and Weed Control

land, barren land, climate change, water scarcity, as well On the early-crop-weeds Dataset [6] and the Plant Seedlings
as weeds in crops. Artificial intelligence can play crucial Dataset , they enhanced the classification accuracy by 0.51%
role in mitigating these issues in agriculture with the aid to 0.89% by fine-tuning four DL models [7].
of computer vision, machine learning, and deep learnings
[2]. According to their leaf shape, weeds are divided into Major Research Contributions:
three primary groups: broad-leaf weeds, grasses, and sedges. Our research addresses the challenges in weed detection and
Weeds are non-essential plants that are found in different classification by focusing on the features that can produce
parts of the crop. These weeds not only damage the crop but upstanding ML and DL-based models. The research makes
also provide shelter and breeding grounds for various pests. significant contributions to the smart agriculture field by
As per European Crop Protection (ECPA), weeds and pests introducing novel feature-based approaches and generating
cause about a 40% loss in crop yields. Therefore, many a comparative insight into the effectiveness of features
methods have been devised to eliminate weeds from the for weed identification and classification, and proposing
crop to avoid damage. One of these methods is to remove solutions for removing weeds.
weeds from the crop by hand [3]. This method demands
• A meticulous effort has been made to annotate the
lots of hard work and time. Another method is to remove
"CottonWeedID15" dataset. The dataset contains im-
weeds with the help of specially designed mechanical
ages of various weeds commonly found in cotton
devices. These devices are moved between the rows of
fields. These images are scrupulously annotated with
crops. however, these devices are not workable on crops that
rectangular regions of interest (ROI) markings and are
are not grown in rows. One approach to eliminating weeds in
released with this research [8]. By providing this anno-
crops is to use chemical sprays. Our farmers spray uniformly
tated dataset, the research serves as a valuable resource
across the entire field to keep weeds at bay. Spraying in
for the research community for future investigations in
fields uniformly raises production costs and has negative
the field of weed identification and control.
environmental consequences. Fig 1 shows the conventional
• Our research makes a significant contribution by thor-
weed-control techniques. Artificial intelligence is able to
oughly investigating the efficacy of various statistical
make a weed control system with the help of computer
and texture features, encompassing simple moments,
vision, deep learning, and machine learning. Artificial
Hu moments, GLCM (Gray-Level Co-occurrence Ma-
intelligence has both economic and environmental benefits.
trix), and LBP (Local Binary Pattern), in addition to
The first stage in building an automated weed removal
exploring the potential of deep learning features. This
system is to correctly detect and recognize weeds by the
exploration of diverse feature sets is crucial for advanc-
automated weed removal system. [4]. Weed classification
ing the understanding of feature extraction methods and
and detection in crops are challenging problems because
computation in the context of weed detection.Models
most of the time, weeds and crops have the same color
and feature sets yielded results of more than 88% on
and texture. It is difficult to differentiate between them.
the testing set of both datasets.
Due to different sun angles, lighting varies on the surface
• U2Net is used in a novel way to remove the background
of weeds and crops, producing illumination and shadow,
from the images and a rigorous comparison is made to
which creates more difficulty in detection and classification.
evaluate the performance of the learners on images with
Image capture, pre-processing of pictures, feature extraction,
a background and without a background.
and weed detection and classification are the four main
• Our research contributes by disseminating a vital
phases of a typical weed detection system [5]. In recent
awareness message to the world’s population, high-
years, with the advancement of science, Technology, and
lighting the transformative potential of deep learning
artificial intelligence, both have experienced rapid growth.
technology in agriculture. By emphasizing the benefits
For solving classification and detection problems, many new
of adopting deep learning techniques, we aim to inspire
computer vision, machine learning, and deep learning algo-
and educate farmers about the potential improvements
rithms have been introduced which are only able to work
in crop yield and overall prosperity that can be achieved
due to the graphics processing unit. Deep learning models,
through the integration of advanced technologies. This
such as deep neural networks, demand high computation.
awareness initiative is a proactive step toward bridging
But with the help of transfer learning, we are able to reduce
the gap between technological advancements and prac-
computation. In transfer learning, we use already learned
tical implementation in the agricultural sector, fostering
weights that are transferred from another problem-related
a more informed and tech-savvy farming community in
domain. Transfer learning (also known as transferring deep
the world.
learning model’s weights) solely entails fine-tuning model
parameters using additional datasets in the target domain. The remaining sections of the article are structured as
Transfer learning is really helpful in achieving good results follows: Section II probes into related work and explores
with less computation. The authors of [6] discovered existing approaches relevant to the topic while section III
that optimizing DL models on agricultural datasets helps outlines the proposed methodology with Fig 2, detailing
decrease training epochs while enhancing model accuracy. the techniques. Section IV discusses the experimental setup
2 VOLUME 4, 2023

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4
 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2024.3355017

Syed Mujtaba H R et al.:Revolutionizing Agriculture: Machine and Deep Learning Solutions for Enhanced Crop Quality and Weed Control

along with results obtained from the multiple experiments. was achieved by DCGAN after 29,500 iterations. In a noisy
Finally, the findings and implications of the research are test set, the performance of inception-resnet was 89.06%.
thoroughly discussed in section V, and the conclusion is With real and synthetic images, the inception F1 score was
presented in Section VI. 98.63, and in a noisy test set, the performance of inception
was 87.05%.
Many researchers used image processing techniques for
weed identification. [10] provided a review of image
processing techniques. After collecting the dataset, then
researchers apply preprocessing on the dataset, and after
preprocessing, image enhancement algorithms are used. On
enhanced images segmentation algorithms are used, which
are threshold-based or learning-based. Then researchers
extract features from the binary image. Feature extraction is
based on morphology, spectral property, visual texture, and
spatial context. Then these features are fed in machine learn-
ing or deep learning models and classified the weed images.
Deep learning models such as GANS and CNN were also
FIGURE 1. Weed removal approaches. discussed. Agriculture’s massive dataset challenges require
deep learning to solve.
Turf grass is used in athletics grounds, lawns, golf courses,
II. RELATED WORK and many other areas. So [11] proposed methods to avoid
Machine learning and deep learning techniques are used in weeds in turf grass. They used a Sony DSC-HX1 camera
the detection and recognition of weeds in crops, producing for dataset collection. They took images from multiple golf
astonishing results in precision farming. [3] proposed courses (Riverview, Sun City, Tampa, and Miami). Hy-
two techniques for classification based on weed density. drocotyle spp., Hedyotis cormybosa, and Richardia scabra
They used three classes, with each class representing weed were the weeds used in their datasets. They did training
density. In their first technique, after creating a density- with one weed and with turf grass and also with multiple
based dataset, they converted each image to grayscale, then weeds with turf grass. For training, they used VGGnet,
reduced the image size to reduce computing time. Grey Googlenet, and Detectnet architectures.The F1 score values
level co-occurences matrix is calculated from the reduced- of VGGnet and Googlenet for Hydrocotyle spp. are 0.9990
size images, and features like correlation, contrast, homo- and 0.667 on validation dataset. The F1 score values of
geneity, and energy are extracted from each grey level co- VGGnet and Googlenet for Hedyotis cormybosa are 0.9950
occurences matrix. They trained a support vector machine and 0.7091 on validation dataset. The F1 score values of
model with a radial basis kernel and achieved a 10-fold VGGnet and Googlenet for Richardia scabra are 0.9911
cross-validation accuracy of 72.73%. They also conducted and 0.6667 on the validation dataset. For multiple species
a comparison of radial bases and linear kernels. The highest Googlenet F1 score was 0.72667 and VGGnet F1 score was
accuracy achieved with a linear kernel is 51.52%, which 96.33 on the validation dataset. Many classification and
is comparatively lower than a radial base kernel. Random detection experiments have been conducted in agriculture
forest achieved a 69.70% cross validation accuracy after as a result of the advent of deep learning algorithms. [12]
a 10 fold cross-validation with GLCM features. In their did a survey of deep learning techniques. They conducted
second method, they extracted the green channel from a survey of existing deep learning algorithms for weed
the RGB image of a density-based dataset. Then they identification and classification in various crops. Deep
calculated Mean, variance, kurtosis, and skew. They trained learning architectures used in research papers are VGGNet,
a support vector machine model with a radial basis kernel modified Xception, Inception-ResNet, MobileNet, DensNet,
and achieved a 10-fold cross-validation accuracy of 84.85%. ResNet-50 , VGG-16, VGG-19, Inception v3, SegNet-512,
To address dataset issues, the generative adversarial tech- SegNet-256, YOLO-v3, tiny YOLO-v3, single-shot detector,
niques of deep learning are critical for creating synthetic convolutional neural network, segnet, alexnet, deeplab-
images. [9] They generate synthetic images with traditional v3, U-net, artificial neural network, VGG-F, VGG-vd-16,
augmentation as well as with a deep convolutional gener- AlexNet, U-Net, SegNet, hybrid network, faster R-CNN,
ative adversarial network. In their work, they use transfer ESNet, Joint unsupervised learning deep cluster, LeNET,
learning to set the weights of a neural network. They took a etc. Various crops and weeds are used with various deep
neural network with ImageNet weights. In their experiment learning models. Furthermore, they discussed how GANS
with DCGAN, they used the PlantVillage dataset. The best and synthetic data can also play an important role in catering
FID score was achieved after 46,000 iterations, and the FID to the problems of complex patterns in agriculture. For
was 86.93% for synthetic tomato images, and for synthetic weed removal applications, High precision is required, But
black nightshade images, the FID score is 146.85, which achieving high precision in agriculture is still a challenging
VOLUME 4, 2023 3

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4
 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2024.3355017

Syed Mujtaba H R et al.:Revolutionizing Agriculture: Machine and Deep Learning Solutions for Enhanced Crop Quality and Weed Control

problem. [13], they did work with sugar beet fields and image classification. With an accuracy of 98.90, VGG16 was
four species of weeds, which are named as Pigweed, Lamb- the best performing classification model. In their research
squarters, Hare’s-ear mustard, and turnip weed (scientifi- work [17] they presented a new dataset which had 5187
cally known as. Amaranthus chlorostachys, Chenopodium coloured images, captured under different natural light
album,Conringia orientalis and Rapistrumrugosum). They conditions, that contained images of 15 different weed
used fourier transform and moment features with SVM and classes (Morning Glory, Carpetweed, Palmer Amaranth,
ANN. The ANN achieved a 99.50% accurate classification Waterhemp,Purslane, Nutsedge, Eclipta, Sicklepod, Spotted
of weed. The ANN exhibited an overall accuracy rate of Spurge, Ragweed, Goosegrass, Prickly Sida, Crabgrass,
92.92%. On the other hand, SVM overall accuracy was 95. Swinecress, and Spurred Anoda). This study also evaluates
93.33% SVM correctly classified 93.33% of weeds. 35 state-of-the-art deep learning models for multi-class
In [6], they generated Early-crop-weed dataset, which weed identification. In total 35 models are trained and
includes tomato and cotton crops along with two weed among all these models the top 5 models that performed
species (black nightshade and velvetleaf). They combined the best were ResNeXt101, RepVGG-B1, RepVGG-B2,
fine-tuned pre-trained convolutional networks ( Inception- ResNeXt50, and RepVGG-A2.
Resnet, Densenet, Xception, VGNets, and Mobilenet) with An improved YOLOv5 Convolutional Neural Network is
"traditional" machine learning classifiers (Logistic Regres- constructued for Solanum rostratum Dunal detection [18].
sion, Support Vector Machine and XGBoost). They used Solanum rostratum Dunal weed is classified as one of
transfer learning for training. Their results showed that the most harmful weeds in the US and China. Total 413
the fine-tuned Densenet with Support Vector Machine images of Solanum rostratum Dunal at different stages of
combination achieved a micro F1 score of 99.29%. Other growth were obtained using different devices. YOLOv5 is
architectures also achieved more than 95% accuracy. In combined with the Convolutional Block Attention Module
[14] they purposed several experimented approaches and (CBAM) to increase the extraction of relevant features
explained how to fine-tune parameters and extract deep while suppressing others. This combination is known as
features using deep learning, combining them with machine YOLO-CBAM. YOLO-CBAM is made up of four parts:
learning algorithms. They used four public datasets in their input, backbone, neck, and prediction. The model results
work named as flavia, swedish leaf, UCI leaf and plantvil- in a precision of 0.9036, recall of 0.9012 and an average
lage. They extracted features with deep neural networks precision of 0.9272. This research [19] was carried out
(AlexNet and VGG-16) and after extracting features, they to identify weeds in the fields of bell pepper. During
applied classic machine learning classifiers (LDA and SVM) preprocessing, lighting variations and noise were removed,
for classification. In their last experiment, they produced and data augmentation was applied to enhance quality
features with AlexNet and VGG 16. Then they combined and avoid overfitting. AlexNet, GoogLeNet, InceptionV3
the features which are produced by AlexNet and VGG16. and Xception are those Convolutional Neural Network
Then they used end-to-end RNN on these featues, and after architectures that were applied during this research. All the
training, produced classification results on test data. All their models provided results with 94.5% - 97.7% of accuracy.
experiment produced more than 90% classification accuracy. Overall, InceptionV3 provided the highest accuracy of
Broadleaf crops and weeds that also have broadleaf make it 97.7%. In weed identification tasks, speed, computation
more difficult to identify broadleaf weeds inside broadleaf time, accuracy, and memory are very noticeable things
crops [15]. They used wheat and weed species (cleavers, [20]. In their work, they focused on such things and used
crickweed, and shepherds purse). For weed detection, they lightweight, deep learning models for weed identification.
used CenterNet2, Faster R-CNN, TridentNet, VFNet, and Using the SLIC super pixel technique, images were divided
YOLO version 3. For weed classification, they used Alexnet, into 15336 segments: 3249 for soil, 7376 for soybeans,
DenseNet, ResNet, and VGGNet. On weed detection, 3520 for grass, and 1191 for broadleaf weeds. For weed
YOLO v3 achieved the highest F1 score on validation identification, they used mobilenetv2, resnet50, and three
as compared to other models, which is 0.65. VGGNet custom models. The 5-layer CNN design has the lowest
and DenseNet F1 scores are 1, which is higher than as latency and memory utilisation (1.78 GB and 22.245 ms,
compared to other models. In another work [16], they did respectively), as well as the highest detection accuracy
a comparative performance analysis of 3 image classification 97.7%.
models that were trained for classifying various species of In this research [21] optimization algorithms(Adagrad,
weed, as well as the detection model performance developed AdaDelta, Adaptive Moment Estimation (Adam), and
to detect and classifying weed species. The dataset contain Stochastic Gradient Descent (SGD)) were used with deep
462 RGB photos of early season weeds commonly found convolution neural networks (AlexNet, GoogLeNet, VG-
in corn and soybean crops (redroot pigweed, gigantic GNet, and ResNet). VGGNet is particularly designed for
ragweed, foxtail, and cocklebur). There are 181 images small convolution kernels to limit the number of neurons
of redroot pigweed, 173 images of gigantic ragweed,73 and number of parameters. ResNet (Residual Network) is
images of foxtail, and 35 photographs of cocklebur. They used to fix the degradation problem for deep networks by
used models named Resnet50, VGG16, and inception for using residual learning to train the deeper networks. For the
4 VOLUME 4, 2023

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4
 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2024.3355017

Syed Mujtaba H R et al.:Revolutionizing Agriculture: Machine and Deep Learning Solutions for Enhanced Crop Quality and Weed Control

best performing input image size, the classification accuracy grey level co-occurrence matrix, they extract features like
hierarchy, from lowest one to highest one, was VGGNet, contrast, energy, dissimilarity, area second moment (ASM)
GoogLeNet, AlexNet ResNet. The study [22] shows how and correlation. Distance, angle, and a number of levels
we can use Siamese neural networks to solve large dataset were the parameters used in the grey level co-occurrence
problems. The Siamese neural network with convolutional matrix for feature extraction. Points and radius were the
layers was used for training. The support dataset contains parameters that were used in the local binary pattern.
1,5,10,15, and 20 images of each type, while the query They experimented with different numbers of features, and
contains 40 images of each. Support datasets were used SVM produced the best results with 90% accuracy, on 55
to fine tune the SNN. Then it was evaluated on a testing features (LBP8-1; 5 LBP162; 14 LBP24-3; 15 contrast; 15
set, and further enhancement in accuracy was observed as dissimilarity; and 4 correlation).
the accuracy jumped to 70.1% and 70.0% from 67.5% and Various models and algorithms exhibit distinct accuracies
66.6% for the validation and testing data sets, respectively. when applied to diverse datasets, each necessitating varying
In this paper [23], the main purpose was to train Deep computational resources. It is imperative to delve into the
Convolutional Neural Networks (DCNNs). Four DCNNs intricacies of these models and algorithms, meticulously
(GoogLeNet, ShuffleNet, MobileNet, and VGGNet) were analyzing their features to gauge the requisite computa-
assessed to find and differentiate weeds that are grow- tional power. Developing methodologies to address weed-
ing in bermudagrass turf. Both VGGNet and ShuffleNet related challenges with efficient computations is paramount.
demonstrated exceptional overall accuracy in the validation Existing studies often overlook the nuanced impact of
process, with values equal to or greater than 0.999. SE- background areas on the accuracy of weed and crop
YOLOv5x was first time tested on lettuce crops and weeds identification under varying lighting conditions. A notable
dataset [24]. The dataset they used in their work had five research gap lies in understanding how the background
kinds of weeds and one lettuce crop. For classification, area influences the performance of different models. A
SVM, SE-YOLOv5x surpassed YOLOv5x, SSD (VGG), comprehensive exploration of this aspect is essential for
SSD (Mobilenetv2), Faster-RCNN (Resnet50), and Faster- advancing the precision and applicability of weed detection
RCNN (VGG) are used. SE-YOLOv5x demonstrates su- methodologies Overcoming variations in lighting conditions,
perior performance in the classification of lettuce and diverse weed species, and the demand for expansive an-
weeds. The aim of [25] was to recognize plants in UAV notated datasets is pivotal. Particularly in countries, where
images using the transformer’s architecture. The dataset was traditional agricultural approaches persist, there is a pressing
divided into 5 classes: weed, beet, off-type beet, parsley, and need to raise awareness about modern technologies. This
spinach. Each class contains 3200 to 4000 images, except not only promises increased yields and profitability but
off-type beets, which only have 653 samples. Random also advocates for environmentally sustainable practices by
rotations and flips were performed so the total dataset discouraging the excessive use of herbicides. Bridging this
contains 19265 images. EfficientNet B0, EfficientNet B1 awareness gap would contribute significantly to advancing
and ResNet 50 were the conventional neural networks that agricultural practices in the world.
were applied to the dataset and provided a F1-score of
98.7%, 98.9% and 99.2% respectively. A deep learning
model named the original generative adversarial network III. PROPOSED METHODOLOGY
designed by Ian Goodfellow [26] is proposed. In their
network, they used two learning models: one called the
generator, and the other called the discriminator. They
trained them in an adversarial process. Generators try to
fool discriminators, and discriminators try to classify fake
and real data. Finding a discriminator with the highest
classification efficiency and a generator that confuses the
discriminator the most is the method for training a GAN
model. This first architecture of the GAN model is the next
step towards augmentation. [27] This paper provides an
overview of GAN’s architectural evolution and its applica-
tion in agriculture. They evaluate how GAN’s architecture
plays a role in weed detection, postharvest detection of fruit
defects, plant phenotyping, plant health conditions, animal
farming, and aquaculture. Handcrafted features are used
with machine learning and automated features through deep
learning [28]. For knowing which features produce better
results. For handcrafted features, they used a local binary FIGURE 2. High level methodology diagram.
pattern and a grey level co-occurrence matrix. From the
VOLUME 4, 2023 5

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4
 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2024.3355017

Syed Mujtaba H R et al.:Revolutionizing Agriculture: Machine and Deep Learning Solutions for Enhanced Crop Quality and Weed Control

A. DATASETS
We are using two datasets in our work: one called early-
crop-weed [6] and the other CottonWeedID15 [17]. early-
crop-weed datasets have two weeds (velvet leaf and black
night shade) and two crops (cotton and tomato), whereas
CottonWeedID15 has fifteen weeds (carpetweeds, crabgrass,
eclipta, goosegrass, morningglory, nutsedge, palmerama-
ranth, prickly sida, purslane, ragweed, sicklepod, spotted
spurge, spurred anoda, swinecress, and waterhemp).Fig 3
and 4 shows both datasets are unbalanced.

FIGURE 5. Annotation of purslane class image from cottonweed dataset.

2) Grayscale and Resize

To reduce computation, we converted the color images to
grayscale. This allowed us to process only one channel
image. Since the dataset images had large dimensions, we
resized them to 224 by 224. By reducing the image size,
we were able to significantly reduce the computation time
FIGURE 3. early-crop-weed dataset class imbalance.
required for further processing.

3) Remove background using U2-Net

The U2-Net model is a deep learning model developed by
researchers from Hefei University of Technology in China.
It consists of 23 layers and is an improved version of the U-
Net model. The U2-Net architecture is specifically designed
to capture multi-scale contextual information and accurately
detect salient objects in images. In U2-Net, the image is
passed through an encoder. The encoder includes multiple
convolutional layers that extract features and reduce the
image dimensions. The encoded features are then passed
to the decoder layer, which consists of upsample layers.
These layers gradually increase the spatial dimensions while
preserving the learned features.U2-Net also utilizes skip
connections between the encoder and decoder, which help
FIGURE 4. CottonWeedID15 dataset class imbalance. in achieving accurate image segmentation. The output of
the decoder is a saliency map, which is a binary mask. The
saliency map helps in segmenting the image by highlighting
B. PREPROCESSING the regions of interest. Fig 6 shows background removed
1) Annotation through U2-Net.
For YOLO training, an annotated dataset is required; there-
fore, we annotated using LabelImg. We concentrated on
creating bounding boxes only around the areas where weed’s
leaves are present. We made sure to maximize the weed’s
leaf coverage while minimizing the soil area. The objective
of our bounding box was to minimize the inclusion of
soil and maximize the area covered by weed’s leaves.This
approach also helps in reducing land pollution since the
spray was intended for leaves and not for the soil. Fig 5 FIGURE 6. Background removed through U2-Net.
shows annotation of purslane class image.
6 VOLUME 4, 2023

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4
 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2024.3355017

Syed Mujtaba H R et al.:Revolutionizing Agriculture: Machine and Deep Learning Solutions for Enhanced Crop Quality and Weed Control

4) SMOTE Synthetic Minority Over-sampling Technique

N X
N
SMOTE is a data upsampling technique that is helpful X GLCM(i, j)
Homogeneity =
in addressing class imbalances. SMOTE aims to create i=1 j=1
1 + |i − j|
synthetic samples that lie along the line segment connecting
N X
N
the original minority class sample and its nearest neighbor. X
SMOTE selects a random sample image from the minority Contrast = (i − j)2 GLCM(i, j)
i=1 j=1
class and examines the most closely similar images using
k nearest neighbors. Then, using interpolation, SMOTE n
1X
generates a new image, and this process continues until all xi
Mean = x̄ =
n i=1
classes are not balanced.
v
u n
u 1 X
C. MANUAL FEATURES Standard deviation = s = t (xi − x̄)2
n − 1 i=1
In our first experiment which named as manual features with
background and without background, we used the pretrained n
U2Net to remove the background from the images. This 1 X
Variance = s2 = (xi − x̄)2
process helped isolate the interest area in the images Next, n − 1 i=1
we converted the color images to grayscale. For reducing n
computational complexity, we resized the images to a di- 1X
Mean absolute deviation = M AD = |xi − x̄|
mension of 224 pixels. For feature extraction, we utilized n i=1
the Grey Level Co-occurrence Matrix (GLCM) approach. n
We calculated the GLCM using different angles (0°, 90°, 1X
Mean squared deviation = M SD = (xi − x̄)2
45°, and 135°) and considered neighboring pixel distances n i=1
of 1, 3, and 5. These settings allowed us to capture various 1
Pn
texture patterns and spatial relationships in the images.From (xi − x̄)3
skew = nP i=1
3
the GLCMs, we derived several texture features including 1 n 2 2
n i=1 (xi − x̄)
energy, correlation, dissimilarity, homogeneity, contrast, and 1
Pn 4
entropy. And the summation of the local binary uniform n i=1 (xi − x̄)
kurt =
−3
pattern histogram from local binary pattern. Additionally, 1 n 2 2
P
n i=1 (xi − x̄)
we computed statistical features such as mean, standard n
deviation, variance, mean absolute deviation, contrast, skew-
X
Entropy = H = − P (xi ) log2 P (xi )
ness, kurtosis, entropy, and image moments. Furthermore, i=1
we calculated seven Hu moments [29]. Which are invariant n
image moments representing shape and geometric proper- Summation of local binary pattern histogram =
X
hi
ties.To account for edges and finer details in the images, i=1
we performed the calculation of image moments and Hu XX
moments twice. The first calculation was carried out on the Image moment = mp,q = xp y q · I(x, y)
original grayscale images, while for the second calculation, x y
we applied the Prewitt filter to extract edge gradient be- XX
Central moment = µp,q = (x − x̄)p (y − ȳ)q · I(x, y)
fore computing the image moments and Hu moments. We
x y
performed this experiment without background removal too   
for knowing the importance of background.Normalization −1 0 1 −1 −1 −1
is applied to these features before being fed into classifiers. Prewitt Filter in xy direction : −1 0 1  0 0 0
We utilized the Synthetic Minority Oversampling Technique −1 0 1 1 1 1
(SMOTE) to address class imbalance in the dataset. s 
2  2
∂I ∂I
N X
N Edge Gradient = +
Energy =
X
GLCM(i, j)2 ∂x ∂y
i=1 j=1 Hue moment1 = ἦταe20 + ἦτα02 ,
Hue moment2 = (ἦτα20 − ἦτα02 )2 + 4ἦτα211 ,
N X
N
Hue moment3 = (ἦτα30 − 3ἦτα12 )2 + (3ἦτα21 − ἦτα03 )2 ,
X (i − µ)(j − µ)GLCM(i, j) Hue moment4 = (ἦτα30 + ἦτα12 )2 + (ἦτα21 + ἦτα03 )2 ,
Correlation =
i=1 j=1
σ2 Hue moment5 = (ἦτα30 − 3ἦτα12 )(ἦτα30 + ἦτα12 )[(ἦτα30 +
ἦτα12 )2 − 3(ἦτα21 + ἦτα03 )2 ] + (3ἦτα21 − ἦτα03 )(ἦτα21 +
N X
N ἦτα03 )[3(ἦτα30 + ἦτα12 )2 − (ἦτα21 + ἦτα03 )2 ],
Hue moment6 = (ἦτα20 −ἦτα02 )[(ἦτα30 +ἦτα12 )2 −(ἦτα21 +
X
Disimilarity = |i − j|GLCM(i, j)
i=1 j=1
ἦτα03 )2 ] + 4ἦτα11 (ἦτα30 + ἦτα12 )(ἦτα21 + ἦτα03 ),
VOLUME 4, 2023 7

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4
 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2024.3355017

Syed Mujtaba H R et al.:Revolutionizing Agriculture: Machine and Deep Learning Solutions for Enhanced Crop Quality and Weed Control

Hue moment7 = (3ἦτα21 − ἦτα03 )(ἦτα30 + ἦτα12 )[(ἦτα30 + IV. EXPERIMENTS AND RESULTS
ἦτα12 )2 − 3(ἦτα21 + ἦτα03 )2 ] − (ἦτα30 − 3ἦτα12 )(ἦτα21 + 1) Manual features and classifiers
ἦτα03 )[3(ἦτα30 + ἦτα12 )2 − (ἦτα21 + ἦτα03 )2 ]. After extracting the manual features, we divided them into
where: three ratios. The training dataset ratio was 65%, 20% was for
• ἦταij denotes the central moment of order (i, j), validation and 15%was for testing. These ratios were used to
• The central moments are calculated using the formulas train and evaluate our classifiers: SVM, random forest, and
mentioned earlier for central moments. ANN. Among these classifiers, the ANN (Artificial Neural
Network) showed superior performance compared to SVM
D. DEEP LEARNING FEATURES and random forest. The testing accuracy achieved by the
Artificial Neural Network was 89.26 on CottonWeedID15
Deep learning features from images are extracted through
dataset, and on early-crop-weed dataset SVM showed su-
CNNs (Convolutional Neural Networks). The early layers
perior performance compared to ANN and random forest.
of CNNs extract features like corners, textures, and edges.
The testing accuracy achieved by the SVM with polynomial
Deeper layers of CNNs extract higher-level features, such
kernel was 99 on early-crop weed dataset. We utilized
as shapes, objects, and semantic representations. To address
Autokeras for the artificial neural network. Autokeras tests
class imbalance, we applied synthetic minority oversampling
100 different architectures and selects the best architecture
technique (SMOTE). After balancing the classes, we used
based on validation accuracy.Fig 13 shows the validation
transfer learning and utilized the ImageNet weights with
and trainning loss. The SVM model’s optimal parameters
some famous cnns models for feature extraction. Fig 7 and
were determined using grid search. After evaluating various
8 show the results of applying SMOTE on the early-crop-
options for C, gamma, and degree, the grid search identified
weed and CottonWeedID15 datasets, respectively.
the best combination as C = 0.1, degree = 3, gamma = 0.4,
and kernel = ’poly’. These parameters were chosen from a
range of possibilities: C values included 0.1, 0.2, 0.3, 0.5,
0.6, 0.7, 0.8, 0.9, 1, 10, and 100; gamma values included
0.1, 0.2, 0.3, 0.5, 0.6, 0.7, 0.8, and 0.9; and degree values
included 1, 2, 3, 4, 5, and 6. In random forest, we start
training with 2 trees and incrementally add 1 tree until we
reach a total of 300 trees. Then, we select the tree that
achieves the highest validation score and train the random
forest using that specific number of trees. Table 1 shows the
experiment and results on early-crop-weed dataset of manual
features with Artificial neural network classifiers. Table 2
shows the experiment and results on CottonWeedID15 datset
of manual features with Artificial neural network classifiers.
FIGURE 7. Class imbalance removed from the CottonWeedID15 dataset Fig 9 shows the comparative analysis of classifiers on both
using SMOTE. dataset. Table 3 shows the experiment and results on early-
crop-weed dataset of manual features with SVM and Rndom
forest classifiers.
Classes Experiment 1 Experiment 2
ECW+ANN+SMOTE SEG+ECW+ANN+SMOTE
minmax normalization minmax normalization
Blacknightsade F1 score 0.97 0.79
Cotton F1 score 1 0.95
Tomato F1 score 0.98 0.98
Velvetleaf F1 score 1 0.8

TABLE 1. Experiment and results on early-crop-weed dataset of manual

features with Artificial neural network classifiers

2) Deep learning features and classifier:

Various CNN architectures, including VGG16, VGG19,
FIGURE 8. Class imbalance removed from the early-crop-weed dataset using Xception, DenseNet-121, DenseNet-169, DenseNet-210,
SMOTE.
and ConvNeXt, were utilized for automated feature extrac-
tion. These architectures were initialized with pre-trained
weights from the ImageNet dataset. Following the extrac-
tion of automated features, a random forest algorithm was
8 VOLUME 4, 2023

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4
 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2024.3355017

Syed Mujtaba H R et al.:Revolutionizing Agriculture: Machine and Deep Learning Solutions for Enhanced Crop Quality and Weed Control

that specific number of trees.

3) YOLO v8
YOLOv8, belongs to You Only Look Once (YOLO) fam-
ily, represents a real-time object detection algorithm that
has demonstrated substantial advancements compared to its
previous versions. YOLOv8 has a Backbone consists of
convolutional layers that extract features from the input
image. YOLO v8 also utilizes SPPF layer and convolution
FIGURE 9. Experiment and results on the CottonWeedID15 dataset and layers, YOLO v8 processes features at different scales.
early-crop-weed dataset of manual features.
The Upsample layers enhanced the feature resolutions. For
Classes Experiment 1 Experiment 2
enhacing detection accuracy, YOLO v8 used C2f module
CWID15+ANN SCWID15+ANN for the integration of contextual information and feature.The
+SMOTE +SMOTE Detection module utilizes convolution layers and linear
+Manual features +Manual features
layers for bounding boxes and object classes. We utilized
Carpetweed F1 score 0.79 0.94
Crabgrass F1 score 0.97 0.96 YOLOv8-M for object detection. We initialize the learn-
Eclipta F1 score 0.87 0.86 ing rate to 0.01 and used stochastic gradient optimizer
Goosegrass F1 score 0.95 0.92 with a batch size of 1. Additionally, we initialize weight
Morningglory F1 score 0.54 0.55
Nutsedge F1 score 0.95 0.89
decay with 0.0005 to prevent the model from becoming
Palmer Amaranth F1 score 0.76 0.7 overly complex and momentum with 0.9.After 100 epochs,
Prickly Sida F1 score 0.96 0.93 utilizing YOLOv8-M, achieved an overall mean average
Purslane F1 score 0.86 0.87 precision of 89.Table 8 shows analysis of YOLOv8 results.
Ragweed F1 score 0.97 0.93
Sicklepod F1 score 0.91 0.94 Fig 12 shows YOLOv8 validation and training loss. Fig 14
Spotted Spurge F1 score 0.91 0.89 shows the YOLOv8 confussion matrix on cotton weedid 15
Spurred Anoda F1 score 0.98 0.97 dataset. Fig 15 shows precision recall curve. Fig 16 shows
Swinecress F1 score 0.99 0.96
Waterhemp F1 score 0.86 0.79
detetection of weed.

TABLE 2. Experiment and results on CottonWeedID15 dataset of manual

features with ANN

employed for classification. Notably, the features obtained

from the ConvNeXt architecture outperformed both manual
features and features extracted from other CNNs when used
with random forest. On the early-crop weed dataset, the
random forest model with ConvNeXt achieved a testing
accuracy of 98%, while on the CottonWeedID15 dataset, FIGURE 10. Analysis of CottonWeedID15 dataset results .
the accuracy reached 89%. In Fig 10 and 11, a comparative
analysis is presented, examining the performance of auto-
mated features across various architectures when combined
with the random forest. table 4 and 5 shows experiments
and results on erarly crop weed and segmenter early-crop-
weed dataset of deep learning features with random forest
classifiers.Table 6 and 7 shows experiments and results on
CottonWeedID15 and segmented CottonWeedID15 dataset
of deep learning features with random forest classifiers. In
random forest, we start training with 30 trees and incremen-
tally add 30 tree until we reach a total of 300 trees. Then,
we select the tree that achieves the highest validation score
and train the random forest on cotton weed id 15 dataset
using that specific number of trees. And on early-crop-weed
dataset, we start training with 10 trees and incrementally
add 10 tree until we reach a total of 300 trees. Then, we
select the tree that achieves the highest validation score and FIGURE 11. Analysis of early-crop-weed dataset results .

train the random forest on early-crop-weed dataset using

VOLUME 4, 2023 9

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4
 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2024.3355017

Syed Mujtaba H R et al.:Revolutionizing Agriculture: Machine and Deep Learning Solutions for Enhanced Crop Quality and Weed Control

classes Experiment 1 Experiment 2 Experiment 3 Experiment 4

ECW+RF+SMOTE SEG+ECW+RF+SMOTE ECW+SVM+SMOTE SEG+ECW+SVM+SMOTE
minmax minmax minmax minmax
normalization normalization normalization normalization
Blacknightsade F1 score 0.96 0.94 1 0.83
Cotton F1 score 1 0.95 0.99 0.92
Tomato F1 score 0.98 0.98 1 0.95
Velvetleaf F1 score 0.99 0.93 0.99 0.79
Best Parameters 19 trees 35 trees C=0.1 C=0.3
gamma=0.4, gamma=0.3,
kernel=poly kernel=poly
degree 3 degree 4

TABLE 3. Experiment and results on early-crop-weed dataset of manual features with SVM and Random forest classifiers

classes Experiment 1 Experiment 2 Experiment 3 Experiment 4 Experiment 5 Experiment 6 Experiment 7

ECW+RF ECW+RF ECW+RF SEG+ECW+RF ECW+RF ECW+RF ECW+RF
+SMOTE +SMOTE +SMOTE +SMOTE +SMOTE +SMOTE +SMOTE
VGG16 VGG19 Xception Densenet 121 Densenet 169 Densenet 201 Convnext Base
Blacknightsade F1 score 0.96 0.91 0.7 0.86 0.85 0.9 0.97
Cotton F1 score 0.93 0.95 0.75 0.89 0.83 0.96 0.96
Tomato F1 score 1 1 0.91 1 0.97 1 1
Velvetleaf F1 score 0.9 0.9 0.73 0.94 0.86 0.89 0.99
Best Parameters 200 trees 100 trees 160 tree 70 tree 200 trees 90 trees 130 trees

TABLE 4. Experiment and results on early-crop-weed dataset of deep learning features with Random forest classifiers

classes Experiment 1 Experiment 2 Experiment 3 Experiment 4 Experiment 5 Experiment 6 Experiment 7

SECW+RF SECW+RF SECW+RF SECW+RF SECW+RF SECW+RF SECW+RF
+SMOTE +SMOTE +SMOTE +SMOTE +SMOTE +SMOTE +SMOTE
VGG16 VGG19 Xception Densenet 121 Densenet 169 Densenet 201 Convnext Base
Blacknightsade F1 score 0.94 0.92 0.88 0.94 0.95 0.0.84 0.95
Cotton F1 score 0.93 0.88 0.9 0.88 0.87 0.88 0.95
Tomato F1 score 0.95 0.91 0.9 0.97 0.96 0.95 0.96
Velvetleaf F1 score 0.91 0.92 0.88 0.88 0.82 0.99 0.95
Best Parameters 200 trees 100 trees 160 tree 70 tree 200 trees 90 trees 130 trees

TABLE 5. Experiment and results on segmented early-crop-weed dataset of deep learning features with Rndom forest classifiers

FIGURE 12. YOLOv8 training and validation loss.

FIGURE 13. Training and validation loss of artificial neural network.

10 VOLUME 4, 2023

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4
 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2024.3355017

Syed Mujtaba H R et al.:Revolutionizing Agriculture: Machine and Deep Learning Solutions for Enhanced Crop Quality and Weed Control

classes Experiment 1 Experiment 2 Experiment 3 Experiment 4 Experiment 5 Experiment 6 Experiment 7

CWID15+RF CWID15+RF CWID15+RF CWID15+RF CWID15+RF CWID15+RF CWID15+RF
+SMOTE +SMOTE +SMOTE +SMOTE +SMOTE +SMOTE +SMOTE
VGG16 VGG19 Xception Densenet 121 Densenet 169 Densenet 201 Convnext Base
Carpetweed F1 score 0.7 0.77 0.46 0.64 0.62 0.69 0.91
Crabgrass F1 score 0.86 0.88 0.57 0.76 0.78 0.72 0.93
Eclipta F1 score 0.63 0.68 0.45 0.55 0.59 0.57 0.81
Goosegrass F1 score 0.63 0.68 0.45 0.55 0.59 0.57 0.81
Morningglory F1 score 0.68 0.77 0.36 0.65 0.6 0.62 0.96
Nutsedge F1 score 0.84 0.87 0.54 0.67 0.65 0.66 0.91
palmer Amaranth F1 score 0.6 0.62 0.42 0.56 0.55 0.53 0.84
prickly sida F1 score 0.72 0.77 0.6 0.7 0.71 0.68 0.86
purslane F1 score 0.76 0.76 0.5 0.69 0.68 0.71 0.84
Ragweed F1 score 0.8 0.77 0.6 0.74 0.79 0.78 0.97
Sicklepod F1 score 0.63 0.73 0.46 0.58 0.67 0.6 0.86
Spotted spurge F1 score 0.7 0.73 0.5 0.65 0.64 0.63 0.86
Spurred Anoda F1 score 0.84 0.83 0.71 0.8 0.79 0.74 0.95
Swinecress F1 score 0.85 0.85 0.75 0.82 0.82 0.83 0.98
Waterhemp F1 score 0.62 0.73 0.47 0.6 0.58 0.63 0.84
Best Parameters 90 trees 190 trees 210 tree 210 tree 300 trees 240 trees 300 trees

TABLE 6. Experiment and results on CottonWeedID15 dataset of deep learning features with Random forest classifiers

classes Experiment 1 Experiment 2 Experiment 3 Experiment 4 Experiment 5 Experiment 6 Experiment 7

SCWID15+RF SCWID15+RF SCWID15+RF SCWID15+RF SCWID15+RF SCWID15+RF SCWID15+RF
+SMOTE +SMOTE +SMOTE +SMOTE +SMOTE +SMOTE +SMOTE
VGG16 VGG19 Xception Densenet 121 Densenet 169 Densenet 201 Convnext Base
Carpetweed F1 score 0.61 0.61 0.41 0.59 0.55 0.52 0.73
Crabgrass F1 score 0.71 0.73 0.62 0.59 0.54 0.52 0.8
Eclipta F1 score 0.78 0.8 0.64 0.67 0.68 0.62 0.75
Goosegrass F1 score 0.71 0.74 0.49 0.65 0.62 0.61 0.8
Morningglory F1 score 0.74 0.69 0.62 0.67 0.65 0.63 0.68
Nutsedge F1 score 0.57 0.55 0.45 0.53 0.47 0.46 0.72
palmer Amaranth F1 score 0.78 0.75 0.64 0.7 0.64 0.61 0.84
prickly sida F1 score 0.67 0.67 0.56 0.58 0.57 0.5 0.77
purslane F1 score 0.79 0.78 0.73 0.73 0.75 0.75 0.78
Ragweed F1 score 0.71 0.76 0.55 0.58 0.52 0.54 0.85
Sicklepod F1 score 0.77 0.71 0.58 0.7 0.62 0.6 0.67
Spotted spurge F1 score 0.92 0.92 0.76 0.83 0.84 0.82 0.95
Spurred Anoda F1 score 0.92 0.88 0.78 0.83 0.77 0.74 0.79
Swinecress F1 score 0.58 0.53 0.4 0.54 0.41 0.4 0.74
Waterhemp F1 score 0.87 0.88 0.77 0.81 0.8 0.74 0.91
Best Parameters 90 trees 190 trees 210 tree 210 tree 300 trees 240 trees 300 trees

TABLE 7. Experiment and results on Segmented CottonWeedID15 dataset of deep learning features with Random forest classifiers

VOLUME 4, 2023 11

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4
 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2024.3355017

Syed Mujtaba H R et al.:Revolutionizing Agriculture: Machine and Deep Learning Solutions for Enhanced Crop Quality and Weed Control

Classes images instances Box(P) Recall(R) map 50

All 1563 1770 0.911 0.839 0.822
Carpetweed 1563 328 0.83 0.686 0.664
Crabgrass 1563 56 0.97 0.586 0.587
Eclipta 1563 95 0.79 0.758 0.722
Goosegrass 1563 68 0.948 0.838 0.827
Morningglory 1563 367 0.909 0.874 0.865
Nutseedge 1563 84 0.867 0.94 0.704
Palmer Amaranth 1563 209 0.986 0.952 0.918
PricklySida 1563 42 0.922 0.85 0.868
Purslane 1563 154 0.916 0.916 0.912
Ragweed 1563 40 0.96 0.605 0.866
Sicklepod 1563 73 0.973 0.98 0.691 FIGURE 16. Detection of weed using YOLO v8 .
SpottedSpurge 1563 72 0.986 0.957 0.945
SpurredAnoda 1563 20 0.824 0.8 0.755
Swinecress 1563 22 0.875 0.909 0.864
Waterhemp 1563 140 0.914 0.936 0.877 V. DISCUSSION
Do deep learning features yield more accurate results com-
TABLE 8. Analysis of YOLO v8 reults
pared to hand-extracted features? Yes, deep learning features
indeed produce more accurate results than hand-extracted
features, as demonstrated by our experiments. Despite the
utilization of Prewitt filters for edge and fine detail en-
hancement, ConvNext still outperformed manual features
in terms of accuracy. But with deep learning, we require
high computation compared to using manual features with
classifiers.
Is the synthetic minority oversampling technique (SMOTE)
effective for weed classification problems? Yes, the synthetic
minority oversampling technique (SMOTE) proves to be
effective for weed classification problems, especially when
dealing with unbalanced datasets. The use of Synthetic
Minority Over-sampling Technique (SMOTE) addressed the
class imbalance inherent in weed detection datasets. By
generating synthetic samples for the minority class, the
training set became balanced, preventing the model from
being biased toward the majority class. This led to im-
proved generalization and better performance on previously
underrepresented weed instances. With the implementation
of SMOTE, we were able to achieve 89% accuracy on the
FIGURE 14. YOLO v8 confussion matrix on CottonWeedID15 dataset. cotton weed ID 15 dataset and 99% accuracy on the early-
crop-weed dataset. Both of these datasets were unbalanced,
and SMOTE played a crucial role in attaining such high
accuracy.
Does YOLOv8 perform well in agricultural problems?
Yes, YOLOv8 is an extremely powerful state-of-the-art ob-
ject detection model. The model’s ability to handle complex
scenes and diverse weed types is a significant advantage,
showcasing its suitability for agricultural applications. It
performed exceptionally well in agricultural problems. The
implementation of YOLO v8 for weed detection yielded
promising results, achieving an overall mean average preci-
sion (mAP) of 89.
Agricultural countries, with their predominantly agrarian
economies, play a crucial role in the global agricultural
landscape. The application of advanced technologies, such
as deep learning, in the agricultural sector can have profound
implications for improving productivity, sustainability, and
FIGURE 15. YOLO v8 Precision recall curve on CottonWeedID15 dataset. crop yield. The integration of deep learning models, such as
YOLOv8, into agricultural practices holds the potential to
12 VOLUME 4, 2023

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4
 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2024.3355017

Syed Mujtaba H R et al.:Revolutionizing Agriculture: Machine and Deep Learning Solutions for Enhanced Crop Quality and Weed Control

address challenges related to crop monitoring, pest control, Code availability Code can be provided on request.
and resource optimization.
To further advance weed detection in crops, future research Authors’ contributions SM worked on algorithm im-
could focus on refining the model’s robustness to environ- plementation and wrote the initial draft of the article. AN
mental factors and expanding the dataset to encompass a floated the idea, supervised the implementation and con-
broader range of agricultural scenarios. Investigating trans- tributed to the article write-up. AK helped in implementa-
fer learning techniques and exploring the use of multi-sensor tion. SR verified and analysed the results and contributed in
data for more accurate weed identification are potential article write-up. SA improved the algorithm and the article
avenues for improvement.Current research predominantly write-up. VG funded the project and supervised.
relies on RGB images for weed detection. Exploring the
integration of multispectral data, such as infrared or hy- REFERENCES
perspectral imagery, could provide additional insights into [1] P. Radoglou-Grammatikis, P. Sarigiannidis, T. Lagkas, and I. Moscholios,
weed characteristics and improve the model’s accuracy, “A compilation of uav applications for precision agriculture,” Computer
Networks, vol. 172, p. 107148, 2020.
particularly in scenarios where visual cues alone may be [2] R. Lal, “Soil structure and sustainability,” Journal of sustainable agricul-
insufficient. ture, vol. 1, no. 4, pp. 67–92, 1991.
[3] T. Ashraf and Y. N. Khan, “Weed density classification in rice crop using
computer vision,” Computers and Electronics in Agriculture, vol. 175, p.
VI. CONCLUSION
105590, 2020.
Agriculture is facing weed challenges, and automated weed [4] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan,
control systems can assist farmers in crop production while P. Dollár, and C. L. Zitnick, “Microsoft coco: Common objects in context,”
also lowering production costs. A large image dataset is in European conference on computer vision. Springer, 2014, pp. 740–755.
[5] S. K. Seelan, S. Laguette, G. M. Casady, and G. A. Seielstad, “Remote
required for future work to meet the challenges of real sensing applications for precision agriculture: A learning community
time in agriculture. Moreover, deep convolutional generative approach,” Remote sensing of environment, vol. 88, no. 1-2, pp. 157–169,
adversarial networks should utilized for crop and weed 2003.
[6] B. Espejo-Garcia, N. Mylonas, L. Athanasakos, S. Fountas, and I. Vasi-
augmentation. This approach allows us to enhance the lakoglou, “Towards weeds identification assistance through transfer learn-
agricultural dataset. Deep learning architectures produced ing,” Computers and Electronics in Agriculture, vol. 171, p. 105306, 2020.
great results, but room for improvement still exists. Deep [7] T. M. Giselsson, R. N. Jørgensen, P. K. Jensen, M. Dyrmann, and H. S.
Midtiby, “A public image database for benchmark of plant seedling classi-
learning algorithms are becoming a new step in improving fication algorithms,” arXiv preprint arXiv:1711.05458, 2017.
crop yield and getting rid of these weeds more efficiently. [8] S. Mujtaba, “Annotated weed images,” 2023. [Online]. Avail-
People in agricultural countries are continuing to use tradi- able: https://www.kaggle.com/datasets/mujtabatszh/roi-yolov8-cwid15-
by-syed-mujtaba-hassan-rizvi
tional approaches due to lack of awareness of deep learn- [9] B. Espejo-Garcia, N. Mylonas, L. Athanasakos, E. Vali, and S. Foun-
ing technologies. However, they should be directed toward tas, “Combining generative adversarial networks and agricultural transfer
modern technology to improve agriculture and crop yield. learning for weeds identification,” Biosystems Engineering, vol. 204, pp.
79–89, 2021.
The key contributions of this research lie in advocating for [10] A. Wang, W. Zhang, and X. Wei, “A review on weed detection using
the utilization of advanced technologies, particularly deep ground-based machine vision and image processing techniques,” Comput-
learning, to overcome traditional agricultural constraints. ers and electronics in agriculture, vol. 158, pp. 226–240, 2019.
[11] J. Yu, S. M. Sharpe, A. W. Schumann, and N. S. Boyd, “Deep learning for
This shift has the potential to significantly impact the image-based weed detection in turfgrass,” European journal of agronomy,
agricultural community by fostering increased awareness vol. 104, pp. 78–84, 2019.
and adoption of modern techniques in the world, thereby [12] A. M. Hasan, F. Sohel, D. Diepeveen, H. Laga, and M. G. Jones, “A survey
of deep learning techniques for weed detection from images,” Computers
elevating agricultural practices and crop yields. The study and Electronics in Agriculture, vol. 184, p. 106067, 2021.
aims to catalyze a transformative impact within the commu- [13] A. Bakhshipour and A. Jafari, “Evaluation of support vector machine
nity of practice and the relevant industry by promoting the and artificial neural networks in weed detection using shape features,”
Computers and Electronics in Agriculture, vol. 145, pp. 153–160, 2018.
integration of state-of-the-art technologies for sustainable [14] A. Kaya, A. S. Keceli, C. Catal, H. Y. Yalic, H. Temucin, and B. Tekiner-
and efficient weed management in agriculture. dogan, “Analysis of transfer learning for deep neural network based plant
classification models,” Computers and electronics in agriculture, vol. 158,
pp. 20–29, 2019.
DECLARATIONS
[15] J. Zhuang, X. Li, M. Bagavathiannan, X. Jin, J. Yang, W. Meng, T. Li,
Funding: This work is supported and funded by the L. Li, Y. Wang, Y. Chen et al., “Evaluation of different deep convolutional
Deanship of Scientific Research at Imam Mohammad Ibn neural networks for detection of broadleaf weed seedlings in wheat,” Pest
Saud Islamic University (IMSIU) (grant number IMSIU- Management Science, 2022.
[16] A. Ahmad, D. Saraswat, V. Aggarwal, A. Etienne, and B. Hancock,
RP23042). “Performance of deep learning models for classifying and detecting com-
mon weeds in corn and soybean production systems,” Computers and
Data Availability: The dataset used in this research [8] Electronics in Agriculture, vol. 184, p. 106081, 2021.
[17] D. Chen, Y. Lu, Z. Li, and S. Young, “Performance evaluation of deep
is freely available. transfer learning on multi-class identification of common weed species
in cotton production systems,” Computers and Electronics in Agriculture,
Conflict of Interest: It is declared by the authors that vol. 198, p. 107091, 2022.
[18] Q. Wang, M. Cheng, S. Huang, Z. Cai, J. Zhang, and H. Yuan, “A deep
there is no conflict of interest. learning approach incorporating yolo v5 and attention mechanisms for
field real-time detection of the invasive weed solanum rostratum dunal

VOLUME 4, 2023 13

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4
 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2024.3355017

Syed Mujtaba H R et al.:Revolutionizing Agriculture: Machine and Deep Learning Solutions for Enhanced Crop Quality and Weed Control

seedlings,” Computers and Electronics in Agriculture, vol. 199, p. 107194, SECOND: DR. ASMA NASEER received the
2022. PhD degree in Computer Science from National
[19] A. Subeesh, S. Bhole, K. Singh, N. Chandel, Y. Rajwade, K. Rao, University of Computer and Emerging Science
S. Kumar, and D. Jat, “Deep convolutional neural network models for (NUCES), Lahore, Pakistan, in 2019 and the M.S.
weed detection in polyhouse grown bell peppers,” Artificial Intelligence degree in in Computer Science from NUCES,
in Agriculture, vol. 6, pp. 47–54, 2022. Lahore, Pakistan, in 2008. She is currently work-
[20] N. Razfar, J. True, R. Bassiouny, V. Venkatesh, and R. Kashef, “Weed de- ing as an Associate Professor at NUCES. Prior
tection in soybean crops using custom lightweight deep learning models,”
to joining NUCES, she was a faculty member
Journal of Agriculture and Food Research, vol. 8, p. 100308, 2022.
at University of Management and Technology
[21] J. Yang, M. Bagavathiannan, Y. Wang, Y. Chen, and J. Yu, “A comparative
evaluation of convolutional neural networks, training image sizes, and (UMT) in the Department of Computer Science
deep learning optimizers for weed detection in alfalfa,” Weed Technology, from 2010 till 2021. Dr. Asma is a dedicated Artificial Intelligence (AI) and
pp. 1–11, 2022. Machine Learning (ML) expert, with almost 15 years of experience. She
[22] P. J. Hennessy, T. J. Esau, A. W. Schumann, A. A. Farooque, Q. U. Zaman, has been awarded full scholarships from Higher Education Commission
and S. N. White, “Meta deep learning using minimal training images for (HEC) Pakistan and other local and foreign bodies for her postgraduate
weed classification in wild blueberry,” 2022. and doctoral studies.
[23] X. Jin, M. Bagavathiannan, A. Maity, Y. Chen, and J. Yu, “Deep learning
for detecting herbicide weed control spectrum in turfgrass,” Plant Meth-
ods, vol. 18, no. 1, pp. 1–11, 2022.
[24] J.-L. Zhang, W.-H. Su, H.-Y. Zhang, and Y. Peng, “Se-yolov5x: An opti-
mized model based on transfer learning and visual attention mechanism
for identifying and localizing weeds and vegetables,” Agronomy, vol. 12,
no. 9, p. 2061, 2022.
[25] R. Reedha, E. Dericquebourg, R. Canals, and A. Hafiane, “Transformer
neural network for weed and crop classification of high resolution uav THIRD: DR. SHAFIQ UR REHMAN received
images,” Remote Sensing, vol. 14, no. 3, p. 592, 2022. the MS degree in Computer Science from Dres-
[26] I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, den University of Technology, Dresden, Germany
S. Ozair, A. Courville, and Y. Bengio, “Generative adversarial nets,” stat, and Ph.D. degree in Computer Science from the
vol. 1050, p. 10, 2014. Department of Software Engineering, Duisburg-
[27] Y. Lu, D. Chen, E. Olaniyi, and Y. Huang, “Generative adversarial net- Essen University, Germany in 2020. He is an
works (gans) for image augmentation in agriculture: A systematic review,”
assistant professor at the College of Computer
Computers and Electronics in Agriculture, vol. 200, p. 107208, 2022.
[28] G. Sunil, Y. Zhang, C. Koparan, M. R. Ahmed, K. Howatt, and X. Sun,
and Information Sciences, Imam Mohammad Ibn
“Weed and crop species classification using computer vision and deep Saud Islamic University (IMSIU), Riyadh, King-
learning technologies in greenhouse conditions,” Journal of Agriculture dom of Saudi Arabia. He also worked as a
and Food Research, vol. 9, p. 100325, 2022. consultant (Requirements Engineer) in a well renowned international or-
[29] M.-K. Hu, “Visual pattern recognition by moment invariants,” IRE trans- ganizations in Germany. He has published several research papers in high-
actions on information theory, vol. 8, no. 2, pp. 179–187, 1962. ranked international conferences and ISI indexed journals. He is involved
in different international funded projects in the field of cyber-physical
systems and cybersecurity. His research interests include AI, cyber-physical
systems, cybersecurity and requirements engineering.

FIRST: SYED MUJTABA HASSAN RIZVI re- FORTH: SHEERAZ AKRAM received his Mas-
ceived the Masters degree in Computer Sci- ter of Science degree in computer science from
ence from National University of Computer and the Lahore University of Management Sciences
Emerging Science (NUCES), Lahore, Pakistan, in (LUMS), Lahore, Pakistan, and the Ph.D. degree
2023. His research interests lie in the areas of in software engineering from the National Univer-
deep learning and image processing. Currently, sity of Sciences and Technology (NUST), Islam-
Syed is working as an instructor at Fast NUCES, abad, Pakistan. He is currently with the Informa-
Lahore, Pakistan, where he teaches Artificial In- tion Systems Department, College of Computer
telligence to BS students. He is passionate about and Information Sciences, Imam Mohammad Ibn
teaching and mentoring students, and he is always Saud Islamic University, Riyadh, Saudi Arabia.
looking for new ways to make learning more engaging and effective. He is also associated with the Department of Computer Science, Faculty
Syed is a highly motivated and talented young researcher with a bright of Computer Science and Information Technology, Superior University,
future ahead of him. He is committed to using his skills and knowledge Lahore. He is also a Coordinator and a Senior Member of Intelligent
to make a positive impact on the world. Syed is also looking for fully Data Visual Computing Research (IDVCR). He completed his postdoctoral
funded scholarships to pursue a PhD in Computer Science. He is excited research training with the University of Pittsburgh, USA, and worked
to continue his research in deep learning and image processing, and he is on a project funded through Grant U01 HL137159. He has 17 years of
eager to contribute to the field of artificial intelligence. working at universities, which includes three years of international research
experience. His research interests include data science, medical image
processing, artificial intelligence in data science, machine learning, deep
learning, computer vision, and digital image processing.

14 VOLUME 4, 2023

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4
 This article has been accepted for publication in IEEE Access. This is the author's version which has not been fully edited and
content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2024.3355017

Syed Mujtaba H R et al.:Revolutionizing Agriculture: Machine and Deep Learning Solutions for Enhanced Crop Quality and Weed Control

FIFTH: PROF. DR. VOLKER GRUHN holds

the Chair for Software Engineering at the Uni-
versity of Duisburg-Essen, Germany since 2010.
He received the MS degree in Computer Science
from the University of Dortmund, Germany in
1987 and a Ph.D. degree in Computer Science
from the University of Dortmund, Germany in
1991. He then worked for the Fraunhofer Institute
for Software and System Technology. He has
published more than 300 research papers in ISI-
indexed journals and high-ranked international conferences. Also, he super-
vised several Ph.D. students. His research focuses on methods for industrial
software engineering, as well as the effects of digital transformation on
enterprises. He co-founded adesso AG in 1997 and is the supervisory board
chairman.

VOLUME 4, 2023 15

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4