AAGF-CNN: An Android Application for

Grading Fruits using CNN

Presentation By
M. Sai Manvitha (Reg.No - 198W1A0532)
P. Jaya Vineela (Reg.No - 198W1A0544)
P. Rishitha (Reg.No - 198W1A0546)

Under the Guidance of

Mr. U. Prabu
Assistant Professor, Department of Computer Science and Engineering

DEPARTMENT OF COMPUTER SCIENCE AND ENGINEERING

Velagapudi Ramakrishna Siddhartha
 • Abstract
• Problem Statement
• Introduction
Presentati
• Literature Survey
on
Outline • Process Flow Diagram
• Methodology
• Results and Analysis
• Conclusion and Future Work
• References
2
ABSTRACT
Traditional manual visual grading of fruits has been one of the important challenges faced
by the agricultural industry due to its laborious nature as well as inconsistency in
inspection and classification process. The task of fruit grading is vital because there is a
great demand for high quality fruits in the market. Earlier they used to grade the fruits
through Machine Learning and Spectrophotometry. In this proposed system, the grading
of fruits is done by using a Convolutional Neural Network (ConvNet/CNN) which is a
Deep Learning technique. It is an efficient and effective machine vision system based on
the deep learning techniques and it offers a non-destructive and cost-effective solution for
automating the visual inspection of fruit freshness, ripeness and appearance. The
proposed system captures the image with a camera and converts it to pixels and
preprocess the image into a deep learning model which displays whether the fruit is
healthy or defected. The real time system efficiently segments multiple instances of the
fruits from an image then grades the individual objects (fruits) accurately. The system was
trained and tested on two data sets (apples and bananas). The test results show that the
system can sort 89% bananas and 86% apples accurately when tested with real time
images.

Keywords:-Fruit grading, Visual inspection, Convolutional Neural Networks,Image

Preprocessing, Ripeness assessment.
Problem Statement
Fruits are rich in vitamins and minerals which reduces human diseases. Grading
of fruits is important because it helps people to have better quality fruits
enhancing food safety. Despite of advancement in computer vision, image
processing, recognition and advancement in computer technology, automatic
fruit classification is a challenging task. The primary parameters that play vital
role while classifying a fruit include the machine learning algorithm that is being
used, quality of images in the fruit database, fruit’s images’ shape and size and
fruit’s color. Secondary parameters that affect the classification are similar
characters of fruits like color, shape, size, etc. If both primary and secondary
parameters are not analyzed properly in the beginning then it may cause
problem during classification and may lead to less accuracy and unexpected
results. Many related works have been conducted in fruit classification using
different classification algorithms but those approaches still lack in some
aspects. The proposed system is trained with apple and banana dataset that
trains itself using color, size, texture parameters. The system is proposed to
capture image of fruit using mobile camera and predict grade of fruit in less
time and with more accuracy as possible.
Introduction
Fruits are key components of the human diet in terms of both nutrition
and economics [1]. In the agriculture sector, the demand for fruits
increases day by day. Traditional grading of fruits is time-consuming,
inconsistent, and tedious. Apple is often regarded as the most
renowned fruit globally, with strong market demand and an annual
output of 80 million tonnes [2]. Apple fruits are usually damaged during
the harvesting and shipping, which may produce visible bruising in
apples, lowering their market value and it affects the fruit quality [3],
[4]. Sensory evaluation [5-7] and physicochemical analysis [8-10] are
the most used approaches for detecting and assessing fruit bruises.
Manual grading of these fruits is a difficult task. In general, precise
apple bruise grading and the significance of early diagnosis and
grading cannot be emphasized [11]. Classifying the ripe fruits
physically is a costly process in agriculture, and errors caused by
humans can result in inaccurate outcomes [12]. Maintaining accuracy
and consistency in fruit grading is made more difficult by manual
Motivation
The fiber and vitamins present in fruits/vegetables are crucial for a
healthy body. Phytochemicals in fruits and vegetables help to protect
human beings from cancer and other diseases. They are essential to
maintain good health, but consumption of substandard fruits and
vegetables lead to adverse effects and poor health. Therefore,
thorough identification of edible and inedible fruits and vegetables
are to be known . At present, most existing fruit quality detecting
and grading system have the disadvantage of high cost and
complexity. So it is significant to develop high speed and low cost
fruit size detecting and grading system.
 Objectives and Scope
Objective:
1. To analyze the quality of fruit.
2. To grade the fruit based on the Ripeness , color and size.
3. To develop an efficient and affordable software which is beneficial
for fruit vendors.
Scope:
1. The scope is limited to fruits like Apple and Banana only.
2. Our mobile application is limited to only fruit vendors for our
purpose.
Place and Client of the Project
Place:
Kanuru , Vijayawada,
Krishna District

Client:
Paidipati Lakshmi,
Fruit Vendor
 i. Efficient Fruit Grading System Using Spectrophotometry and
Machine Learning Approaches
ii. Apple Bruise Grading Using Piecewise Nonlinear Curve Fitting
for Hyperspectral Imaging Data
iii. Evaluation of image processing technique as an expert system in
mulberry fruit grading based on ripeness level using artificial
neural networks (ANNs) and support vector machine (SVM)

6. iv. Machine Learning Driven Approach Towards the Quality

Assessment of Fresh Fruits Using Non-Invasive Sensing
v. Automatic Counting and Individual Size and Mass Estimation of

Literature Olive-Fruits Through Computer Vision Techniques

vi. Fractional fuzzy 2DLDA approach for pomegranate fruit grade

Survey
classification
vii. Evaluation indices of sour flavor for apple fruit and grading
standards
viii. Rician k-Factors-Based Sensor for Fruit Classification by
Maturity Stage
ix. A machine vision technique for Grading of Harvested Mangoes
Based on Maturity and Quality
x. A Spectroscopy-Based Approach for Automated Nondestructive
Maturity Grading of Peach Fruits
xi. Computer vision based date fruit grading system design and
implementation
xii. Rapid Color Grading for Fruit Quality Evaluation Using Direct
Color Mapping
Title:
Efficient Fruit Grading System Using Spectrophotometry and Machine Learning
Approaches [12]
Journal Name:
IEEE Sensors Journal, July,2021.
Methodology:
Physical classification of fruits is expensive and human error may be high. This work
gives a novel method to segregate fruits. Using computer vision fruit is detected first
and then it predicts the grade of the fruit. After that, fruit is analysed for ripeness
using spectroscopy sensor. The data obtained from sensor is used to estimate
Ripeness through ML model.
Advantages:
This model segregates fruits efficiently and has is fully automated, AI based and cost
effective.
Disadvantages:
When the model is tested on unseen real-time data, it got only 72% accuracy which
can be improved along with some features.
Title:
Apple Bruise Grading Using Piecewise Nonlinear Curve Fitting for
Hyperspectral Imaging Data [11]
Journal Name:
IEEE Access Journal, August ,2020.

Methodology:
Apple fruits can be easily damaged, and bruises occur on peels during
harvest, transportation and storage, which could decrease fruit quality.
In this work, apples were mechanically damaged by the same impact
forces, and the bruised regions were exposed to room temperature for
at most 120 min(30,60,90,120 minutes) and then ECOC-SVM is used to
grade the bruised apples.
Advantages:
This model compares various algorithms with ECOC-SVM to grade
apples and proves it as best with 97.33% accuracy.
Title:
Evaluation of image processing technique as an expert system in mulberry fruit
grading based on ripeness level using artificial neural networks (ANNs) and support
vector machine (SVM) [14]
Journal Name:
Postharvest Biology and Technology, March,2020.
Methodology:
White and red mulberry fruit were classified according to maturity stage using image
processing and artificial intelligence classification algorithms. First, mulberry image
segmentation was performed using the RGB color space. We classify fruits into three
unripe, ripe, and overripe categories. In the next step, color, geometric, and texture
features were extracted with two feature selection methods, namely CFS and CONS.
After the image processing step, feature extraction, and dimension reduction, ANN
and SVM were applied to classify each fruit as one of the six possible classes.
Advantages:
ANN model used with CONS subset method supports least number of inputs and
high accuracy and can be used for online applications.
machine vision can be used as an accurate, fast, and non-destructive method to
classify and grade mulberry fruit, reduce product loss and increase its marketability.
Title:
Machine Learning Driven Approach Towards the Quality Assessment of Fresh
Fruits Using Non-Invasive Sensing [15]
Journal Name:
IEEE Sensors Journal, February, 2020
Methodology:
This work presented non-invasive sensing technique for the quality
assessment of fresh of fruits by integrating machine learning ML with
terahertz (THz) waves. The classification was performed to determine the
moisture content (MC) in fruits slices using SVM, KNN and D-tree classifiers.
The results illustrated that the performance of SVM exceeded other
classifiers results using 10-fold validation and leave-one-observation-out-
cross-validation techniques.
Advantages:
The proposed technique demonstrates the strong potential in the discipline of
food and science technology by integrating ML with THz waves to assess real-
time information of fruits on different days at cellular level.
Title:
Automatic Counting and Individual Size and Mass Estimation of Olive-Fruits Through
Computer Vision Techniques [16]
Journal Name:
Computers and Electronics in Agriculture, 2019.
Methodology:
This research presents a new methodology aimed at supporting accurate automatic
olive-fruit grading by using computer vision techniques and feature modeling. An
image analysis algorithm, based on mathematical morphology, was designed to
individually segment olives and extract descriptive features. The image analysis
algorithm was conceived to deal with complex situations as fruit touching, or fruit
cut by the image borders.
Advantages:
The present paper proposes a new methodology for the automatic counting, and the
individual mass and size estimation, of olive-fruits in digital images.
Regarding estimation models, and by relying on external validation results, variety-
dependent models outperformed the independent ones when estimating olive mass.
Disadvantages:
It argues for the methodology could operate under a real scenario
Title:
Fractional fuzzy 2DLDA approach for pomegranate fruit grade classification [17]
Journal Name:
IEEE Access Journal March, 2019.
Methodology:
This research work presents a pomegranate fruit grading approach based on the two
dimensional fruit images. In this approach of feature extraction and classification,
four algorithms (2DLDA, FLDA, F2DLDA, FF2DLDA) are applied for feature
extraction both in unilateral and bilateral approach. Then the kernel support vector
machine is applied in order to test which feature extraction technique could give the
best recognition rate.
Advantages:
Out of four algorithms the proposed fractional fuzzy 2DLDA gives
best recognition rate
Disadvantages:
Horizontal and Vertical image feature extractions using conventional and fuzzy
2DLDA approaches are not implemented
Using another classifier viz. K-Nearest Neighbour, fuzzy C Means, neural networks
and all these techniques optimized with heuristic techniques can also be checked
Title:
Evaluation indices of sour flavor for apple fruit and grading standards [18]
Journal Name:
Journal of Integrative Agriculture ,2018
Methodology:
Apple is a typical fruit that is dominated by malic acid content among all
organic acids. The average content of malic acid in apples in the present
study accounted for 94.5% of ToA, and the content order of organic acids in
most apples was Mal>Oxa>Cit>Lac>Suc>Fum A large sample of apples (106
cultivars) and regression analysis were used to discover significant linear or
logarithmic relationships between five apple sour flavor indices, namely Mal,
ToA, AcV, TiA, and pH value. At present, the evaluation indices of sour flavor
for apple are generally expressed in the absolute value of titratable acid
content
Advantages:
This study provides a scientific basis for evaluating apple flavor and selecting
apple cultivars.
Title:
Rician k-Factors-Based Sensor for Fruit Classification by Maturity Stage [19]
Journal Name:
IEEE Sensors Journal, September, 2016
Methodology:
This paper has proposed a non-destructive, non-invasive sensor system for
classifying durian fruits according to the maturity stages. The design of the
sensor system has adopted the principle of wireless communications in which
the Rician k-factors were utilized to determine the fruit maturity. The
simulations were thus carried out using a smooth surface so as to minimize
the computational resources requirement.

Disadvantages:
Apart from the durians, the proposed sensor system could be applied to
determining the maturity stage of several other fruits.
Title:
A machine vision technique for Grading of Harvested Mangoes Based on Maturity and Quality
[20]
Journal Name:
IEEE Sensors Journal, August , 2016
Methodology:
This works on intelligent machine vision-based techniques for grading of mangoes in four
different categories. In this, video image is captured by a charge couple device camera placed
on the top of a conveyer belt carrying mangoes. For maturity prediction in terms of actual-
days-to-rot, support vector regression has been employed and for the estimation of quality
from the quality attributes, multi attribute decision making system has been adopted. Finally,
fuzzy incremental learning algorithm has been used for grading based on maturity and quality.
Advantages:
Proposed system predicts the actual-days-to-rot and estimates the mango quality with
considerable accuracy using machine vision. Average time to grade one mango is
approximately 0.4 second.
Disadvantages:
There is a problem in detecting the firmness from this vision based measurement. They have
considered only one side of mango image, a mechanical setup should be introduced to rotate
the mango within the conveyer belt to get the image of the other side.
Title:
A Spectroscopy-Based Approach for Automated Nondestructive Maturity Grading of
Peach Fruits [1]
Journal Name:
IEEE Sensors Journal, October, 2015
Methodology:
This paper presents an automated approach for peach fruit maturity grading that, by
exploiting fiber-optic spectroscopy-based sensors and multivariate processing
techniques, minimizes the operator intervention while reducing discharge and waste.
The use of a spectroscopic sensor complies with the so-called nondestructive
measurement method, Maturity grading is accomplished by retrieving estimates of
the fruit flesh firmness by means of multivariate retrieval techniques applied to the
reflectance spectra acquired with the spectrometer and by processing the retrieved
values within the framework of a maturity fuzzy classifier.
Advantages:
It is useful for industries and horticulture applications.
It detects the fruit flesh firmness without causing any damage to fruit.
Disadvantages:
Did not use Spectroscopic sensors, use of this sensors improves maturity grading
Title:
Computer vision based date fruit grading system design and implementation
[13]
Journal Name:
IEEE Sensors Journal, September, 2011
Methodology:
This paper has proposed a prototypical computer vision based date grading
and sorting system. The system uses RGB images of the date fruits. From
these images, it automatically extracts the external date quality features.
They defined a set of external quality features. Based on that it classifies the
date fruits into three quality categories(grade1,2 and 3)
Advantages:
This model helps the date growers in the harvesting season for grading and
sorting of dates. This model is consistent in grading and maintains uniformity
in sorting.
Disadvantages:
This system cannot accurately estimate the bruises on date fruits. The date
fruits were not graded properly, they were misclassified due to the limited
Title:
Rapid Color Grading for Fruit Quality Evaluation Using Direct Color Mapping [21]
Journal Name:
IEEE Sensors Journal, April, 2011
Methodology:
This paper has proposed an effective and user-friendly color mapping concept for automated
color grading that is well suited for commercial production. This color mapping method uses
preselected colors of interest specific to a given application to calculate a unique set of
coefficients for color space conversion. The three-dimensional RGB color space is converted
into a small set of color indices unique to the application. In contrast with more complex color
grading techniques, the proposed method makes it easy for a human operator to specify and
adjust color-preference settings Tomato and date maturity evaluation and date surface defect
detection are used to demonstrate the performance of this novel color mapping concept.
Advantages:
Selection and adjustment of color preferences is much easier when compared to other color
grading techniques. Color mapping works efficiently with higher order polynomials and non-
linear functions.
Disadvantages:
Color index histogram analysis can be done using median instead of using
Process Flow Diagram
Proposed System
Methodology
Module 1:
Data collection & pre-processing
• The dataset used for fruit grading is “Fruits-360” Referred from
“Efficient fruit grading system using spectrophotometry and
machine learning approaches” [12].
• This dataset contains 1581 images of fresh bananas, 2224 images of
rotten bananas, 1693 images of fresh apples and 2342 images of
rotten apples.
• All the images are in high resolution with white background used
for training. The test set of images have 381 images of fresh
bananas, 530 images of rotten bananas, 395 images of fresh apples
and 601 images of rotten apples.
• Image data generator function has been used for image pre-
processing for rescaling.
Methodology
Module 2:
Training the Model
• CNN model is developed for this system that contains
Convolutional layers with 20,20,3 as shape of layers and 3x3
as kernel size.
• It has 7 layers and ReLU activation function is used in all
layers.
• The output layer uses softmax activation function.
• The model is trained with Fruits-360 dataset and model
returns class as fresh or rotten for input image indicating
whether it is apple or banana. Then it is validated and
tested using test dataset.
Methodology
Module 3:
Android Application Development
• The trained CNN model is saved as .h5 file and then converted
to .tflite file so that it becomes easy to integrate to android
application.
• XML files for designing interface of application are created
and java code is written for integrating the model.
• The application opens camera of mobile directly and predicts
the class of fruit.
• The model generated is converted to .tflite format and
included in java files in the android studio. The interface is
designed using XML files. The application uses camera from
mobile and grades the fruits accordingly.
Algorithm/Code
2.Fruit freshness detection
Begin
Call flow_from_directory() for training data
Ŧ:=Read(images)
Ŧ:=augment(Ŧ)
Call flow_from_directory() for training data
Ɠ:=Read(images)
Ɠ:=augment(Ɠ)
Use sequential model from keras
Add conv2d layer with input shape(20,20,3)
Add MaxPool2D layer with pool size(2,2)
Algorithm/Code
Add conv2d layer
Add MaxPool2D layer with pool size(2,2)
Flatten to reduce dimensions
Add Dense layer
Add Dense layer
Add Dense layer (softmax activation function)
Compile Model
Train the model with images in Ŧ
Call flow_from_directory() for test data
ѭ :=Read(images)
ѭ :=augment(ѭ)
Validate and test the model with images in Ɠ and ѭ
End
Algorithm/Code
3. Integration of CNN model to android application
Begin
Save the CNN model in .tflite format.
¥=read(labels)
€=read(model.tflite)
Create new project named fruit grading app
Load tflite packages
Create new activity
Create application layout
Add € and ¥ to assets
Link m and l to application using java
Run the application.

End
Result and Analysis
The Fruit Grading application uses a CNN model trained on Apple and
Banana dataset and depicted an accuracy of 93.7%. The Figure below
shows Plot Accuracy and Plot Loss over Epochs for training and
validation data over the fruits dataset. The overall accuracy for 26
bananas is 86%. The overall accuracy for 40 apples is 89.6%.

Training and validation Accuracy Plot Training and validation Loss

Plot
Conclusion and future work
This project focuses on creating an automated application for
grading of apple and banana fruits using Convolutional Neural
Networks which is an efficient deep learning technique. It has been
designed for fruit vendors who can assure the quality of fruits to
customers. This application uses a CNN model as backend support
for classification which is then converted to tflite model and
integrated with android application. Based on the features of the
fruit, the trained deep learning model will classify the fruit into its
particular grade. The application worked fine on testing with real-
time dataset. ​This application has ample of scope in future as
Agriculture is one of the largest economic sectors and due to high
demand for quality food. Also, this project scope is limited to apples
and bananas only. One can develop methods for grading other fruits
and vegetables.
Plagiarism Report
Publication Status
Submitted to International Conference on Electronics and
Sustainable Communication Systems(ICESC 2022) and got the
acceptance letter.
References
1. Matteoli, S., Diani, M., Massai, R., Corsini, G. and Remorini, D., 2015. A spectroscopy
based approach for automated non-destructive maturity grading of peach fruits. IEEE
Sensors Journal, 15(10), pp.5455-5464.
2. Keresztes, J.C., Diels, E., Goodarzi, M., Nguyen-Do-Trong, N., Goos, P., Nicolai, B. and
Saeys, W., 2017. Glare based apple sorting and iterative algorithm for bruise region
detection using shortwave infrared hyperspectral imaging. Postharvest biology and
technology, 130, pp.103-115
3. Van Zeebroeck, M., Ramon, H., De Baerdemaeker, J., Nicola¨ı, B.M. and Tijskens, E.,
2007. Impact damage of apples during transport and handling. Postharvest biology and
technology, 45(2), pp.157-167.
4. Pan, X., Sun, L., Li, Y., Che, W., Ji, Y., Li, J., Li, J., Xie, X. and Xu, Y., 2019. Non-destructive
classification of apple bruising time based on visible and near-infrared hyperspectral
imaging. Journal of the Science of Food and Agriculture, 99(4), pp.1709-1718.
5. Barrett, D.M., Beaulieu, J.C. and Shewfelt, R., 2010. Color, flavor, texture, and nutritional
quality of fresh-cut fruits and vegetables: desirable levels, instrumental and sensory
measurement, and the effects of processing. Critical reviews in food science and
nutrition, 50(5), pp.369-389.
References
6. Bavay, C., Symoneaux, R., Maˆıtre, I., Kuznetsova, A., Brockhoff, P.B. and
Mehinagic, E., 2013. Importance of fruit variability in the assessment of apple
quality by sensory evaluation. Postharvest biology and technology, 77, pp.67-74.
7. Londhe, D.H., Nalawade, S.M., Pawar, G.S., Atkari, V.T. and Wandkar, S.V., 2013.
Grader: A review of different methods of grading for fruits and vegetables.
Agricultural Engineering International: CIGR Journal, 15(3), pp.217-230
8. Ahmadi, E., Ghassemzadeh, H.R., Sadeghi, M., Moghaddam, M. and Neshat,
S.Z., 2010. The effect of impact and fruit properties on the bruising of peach.
Journal of Food Engineering, 97(1), pp.110-117.
9. Silva, V., Figueiredo, A.R., Costa, J.J. and Guin´e, R.P.F., 2014. Experimental and
mathematical study of the discontinuous drying kinetics of pears. Journal of
Food Engineering, 134, pp.30-36.
10.Magwaza, L.S. and Tesfay, S.Z., 2015. A review of destructive and non-
destructive methods for determining avocado fruit maturity. Food and
bioprocess technology, 8(10), pp.1995-2011.
References
11. Tang, Y., Gao, S., Zhuang, J., Hou, C., He, Y., Chu, X., Miao, A. and Luo, S.,
2020. Apple bruise grading using piecewise nonlinear curve fitting for
hyperspectral imaging data. IEEE Access, 8, pp.147494-147506. 27
12. Chopra, H., Singh, H., Bamrah, M.S., Mahbubani, F., Verma, A., Hooda, N.,
Rana, P.S., Singla, R.K. and Singh, A.K., 2021. Efficient fruit grading system
using spectrophotometry and machine learning approaches. IEEE Sensors
Journal, 21(14), pp.16162-16169.
13. Al Ohali, Y., 2011. Computer vision based date fruit grading system: Design
and implementation. Journal of King Saud University-Computer and
Information Sciences, 23(1), pp.29-36.
14. Azarmdel, H., Jahanbakhshi, A., Mohtasebi, S.S. and Muñoz, A.R., 2020.
Evaluation of image processing technique as an expert system in mulberry
fruit grading based on ripeness level using artificial neural networks (ANNs)
and support vector machine (SVM). Postharvest Biology and Technology, 166,
p.111201.
15. Ren, A., Zahid, A., Zoha, A., Shah, S.A., Imran, M.A., Alomainy, A. and Abbasi,
Q.H., 2019. Machine learning driven approach towards the quality assessment
References
16.Ponce, J.M., Aquino, A., Millan, B. and Andújar, J.M., 2019. Automatic counting and
individual size and mass estimation of olive-fruits through computer vision
techniques. IEEE Access, 7, pp.59451-59465.
17.Gurubelli, Y., Ramanathan, M. and Ponnusamy, P., 2019. Fractional fuzzy 2DLDA
approach for pomegranate fruit grade classification. Computers and Electronics in
Agriculture, 162, pp.95-105.
18.Zhen, Y.A.N., ZHENG, L.J., NIE, J.Y., LI, Z.X. and Cheng, Y., 2018. Evaluation indices of
sour flavor for apple fruit and grading standards. Journal of integrative
agriculture, 17(5), pp.994-1002.
19.Leekul, P., Chivapreecha, S., Phongcharoenpanich, C. and Krairiksh, M., 2016. Rician k-
factors-based sensor for fruit classification by maturity stage. IEEE Sensors
Journal, 16(17), pp.6559-6565.
20.Nandi, C.S., Tudu, B. and Koley, C., 2016. A machine vision technique for grading of
harvested mangoes based on maturity and quality. IEEE sensors Journal, 16(16),
pp.6387-6396.
21.Lee, D.J., Archibald, J.K. and Xiong, G., 2010. Rapid color grading for fruit quality
evaluation using direct color mapping. IEEE transactions on automation science and
engineering, 8(2), pp.292-302.
Thank You