Kumar S. Krishna et al.

; International Journal of Advance Research, Ideas and Innovations in Technology

ISSN: 2454-132X
Impact factor: 6.078
(Volume 6, Issue 2)
Available online at: www.ijariit.com

Fruit quality detection using machine vision techniques

S. Krishna Kumar J. Kaviya
[email protected] [email protected]
Dr. Mahalingam College of Engineering and Technology, Dr. Mahalingam College of Engineering and Technology,
Pollachi, Tamil Nadu Pollachi, Tamil Nadu

G. Dilip Prakash K. Srinivasan

[email protected] [email protected]
Dr. Mahalingam College of Engineering and Technology, Dr. Mahalingam College of Engineering and Technology,
Pollachi, Tamil Nadu Pollachi, Tamil Nadu
ABSTRACT quality of fruit by using the classification algorithm of
supervised learning technique in a short span of time with high
Machine vision techniques are now widely used to detect the
accuracy. These features are used to classify the fruits into
quality of fruits. Image processing is usually the first step in
different categories like good, moderate and rotten fruits.
detecting the quality of fruits. The process starts by capturing
Machine vision technique uses conventional neural network
the image of the fruits using raspberry pi. Then, the image is
algorithm to predict whether the fruit quality is good, moderate
transmitted to the processing stage where it can extract the
or rotten.
fruits features like shape, size and color. These processes are
done using the image processing. It helps to identify and
The Convolutional Neural Network (CNN) is trained with
compare the fruit shape, size and color with the trained
good, moderate and rotten images of apple, orange and banana,
datasets. This is done during the training and testing stage. A
so that it can predict whether it is good or rotten. Artificial
diversity of methods for automatic separation of fruits is
neural network is the extension of many classification
developed. Artificial Neural Network is the one that helps to
techniques. It classifies the fruits by using the shape, color and
segregate the fruits based on the quality such as good,
size features provided at the time of training and also the
moderate and rotten fruit. The existing system can only
extracted features of fruit and provides the result by comparing
separate the fruits into good and rotten one with accuracy of
these features. Finally, the fruits are categorized as a good,
87.4% but our proposed system is capable of separating the
moderate and rotten one.
fruits into good, moderate and rotten one with accuracy of
94.12%.
2. RELATED WORK
2.1 Literature Survey
Keywords⸻ Machine Vision, Image Processing, 2.1.1 Fruit Quality Inspection using Image Processing: The
Convolutional Neural Network, Fruits Quality Detection image processing circumvents the problem of processing or
1. INTRODUCTION quantifying the photographic data mathematically. Several
In today’s world, everything in agriculture is getting automated applications of image processing technology have been
and manual interference in the system becomes non- developed for the agricultural operations. These applications
commercial solution as well as time consuming task. The involve implementation of camera-based hardware systems or
quality of fruit must be checked before it is used for making color scanners for inputting the images. Fruit classification and
food products. In agriculture, the quality of fruit is somewhat fruit disease identification can be seen as an instance of image
dependent on the water availability, soil type, proper usage of categorization.
fertilizers, etc. In old days, more manpower is required used
for selection of quality fruits and vegetable for the production 2.1.2 Image processing and machine learning for
purpose in industries. In recent years many automated systems automated fruit grading system: ANN was used to classify
were invented and that are used for identification of quality empire and golden delicious apples based on surface
fruits. The existing system finds the quality of fruits and characteristics of the apple images Textural and histogram
display the message as good or rotten fruit which uses K- features are extracted from the images at selected wavelengths.
means algorithm of supervised learning technique, as it takes Then, images of apples with surface characteristics were used
more time to predict the fruits quality with an average accuracy in classification applications with two cases two class
of 94.12%. Our proposed system is capable of finding the classification and five class classification. Effectiveness of
© 2020, www.IJARIIT.com All Rights Reserved Page |17
 Kumar S. Krishna et al.; International Journal of Advance Research, Ideas and Innovations in Technology
method depend on the correlation between measured feature Machine Vision technology that combines mechanics, optical
parameter and quality factor. instrumentation, electromagnetic sensing, digital video and
image processing technology. As an integrated mechanical-
2.1.3 Image Segmentation using K mean algorithm and optical-electronic-software system, machine vision has been
Graph based algorithm: The algorithm firstly uses k-means widely used for examining, monitoring and controlling a very
algorithm to split the original image into regions based on broad range of applications. It is the use of device for optical,
Euclidean color distance in l*a*b* to produce an over non-contact sensing to automatically receive and interrupt the
segmentation result. The objective of this work is to develop a image of a real sense in order to obtain information or control
general algorithm to effectively segment objects in images to machines or process image. The applications of machine vision
facilitate fruit defect detection. The dimension of feature techniques has been expanded to various areas such as medical
vectors depends on the numbers of color channel used gray- diagnostics, automatic manufacturing and surveillance, remote
level based k-means for segmenting images. L*a*b* or CIE- sensing, technical diagnostics, autonomous vehicle and in
Lab color space is used for k-means clustering. agriculture and food industry including the inspection of
quality and grading of fruits and vegetables.
2.1.4 Fruit Classification System using Computer Vision:
Color, textural and morphological feature are the most Image
commonly used to identify the disease, maturity and class of Acquisition
the fruits. The computer vision technique include clustering
and color based segmentation, artificial neural network and Pre processing
different classifiers based classification of disease. Using
digital method, the disease detection can be accurately, time
efficient and result in saves time. Different image processing Segmentation
techniques have been developed with help of MATLAB for
accurate fruit disease identification.
Labelling
2.1.5 Machine Vision Based Autonomous Fruit Inspection
and Sorting: Machine vision technology or image processing
is used for inspection and grade wise sorting of fruits. Feature Extraction
MATLAB algorithms like conversion to binary image, area
calculation and average pixel value is used. Arduino-Uno
microcontroller is used for sorting. HSV color space is used to Classification
carry out fruit segmentation. It captures the image of the fruit
and calculate percentage value of the color in order to classify Fig. 1: Steps involved in Machine Vision Technique
the grade of the fruit.
3.2 Image Acquisition
2.1.6 A Fruit Quality Management System Based on Image The first stage of any vision system is the image acquisition.
Processing: Fruit non-destructive detection is the process of After the image has been obtained, this techniques uses all
detecting fruits inside and outside quality without any damage. properties of elimination to increase features that can be
CMOS based camera is used for capturing the image. Fruit obtained from the image. The image will be displayed in three
color is detected according RCB values and fruits are sorted dimensional through camera lens which in sense the reflected
according to color and size. The image can be processed by light rays will hit on pixel sensor. Each pixel contains
using MATLAB software. The canny method differs from the information like intensity of light, information of reflected
other edge detection methods in that it uses two different light back from 3D sensor. The two dimensional is used to
thresholds to detect strong and weak edges in the output only if project the three-dimensional object.
they are connected to strong edge.
3.3 Pre processing
2.1.7 Development of ANN Based Efficient Fruit Pre-processing is in many cases concerned with taking one
Recognition Technique: Artificial neural network classifies array of pixels as input and producing another array of pixels
the fruit by comparing shape, color and size provided at the as output which in some way represents an improvement to the
time of training. MATLAB/SIMULINK software used to original array. This pre-processing step may remove noise,
obtain the result. l*a*b technique is used for color detection in improve the contrast of the image, remove blurring caused by
fruits. Calculate RGB parameters and then converted into hue. movement of the camera during image acquisition, it may
Color feature extraction algorithm is used to extract the color correct for geometrical distortions caused by the lens.
feature of the image. Convert a RGB image into HIS, HSV,
l*a*b and YbCbCr. 3.4 Segmentation
Another way of extracting and representing information from
3. METHODOLOGY an image is to group pixels together into regions of similarity.
3.1 Machine Vision Technique This process is commonly called segmentation. It is a process
It provides image based automatic inspection and analysis for to find the object and separate it from the background image.
applications like automatic inspection, process control and And this process can be done by using Grey Scaling of Pre-
robot guidance. The process includes planning the details of processing step. In
the requirements and project, and then creating a solution. 2D - we would group pixels together according to the rate of
During a run- time, the process starts with imaging followed by change of their intensity over a region.
automated analysis of the image and extraction of required 3D - we group together pixels according to the rate of change
information. of depth in the image, corresponding to pixels lying on the
same surface such as a plane, cylinder, sphere etc.

© 2020, www.IJARIIT.com All Rights Reserved Page |18

 Kumar S. Krishna et al.; International Journal of Advance Research, Ideas and Innovations in Technology
3.5 Labelling
Connected component labeling works by scanning an image,
pixel-by-pixel (from top to bottom and left to right) in order to
identify connected pixel regions. Connected component
labelling works on binary or graylevel images and different
measures of connectivity are possible. However, for the
following we assume binary input images and 8-connectivity.
The connected components labeling operator scans the image
by moving along a row until it comes to a
point p (where p denotes the pixel to be labeled at any stage in
the scanning process) for which V={1}. When this is true, it Fig. 2: Architecture of Convolutional Neural Network
examines the four neighbors of p which have already been
encountered in the scan. Based on this information, the 4. GRADIENT DESCENT ALGORITHM AND
labeling of p occurs as follows: ACTIVATION FUNCTIONS
• If all four neighbors are 0, assign a new label to p, else The gradient descent algorithm, along with the back-
• if only one neighbor has V={1}, assign its label to p, else propagation technique, is used to optimize the weights of the
• if more than one of the neighbors have V={1}, assign one of neural network. During the forward pass, the network uses the
the labels to p and make a note of the equivalences. weights to predict the output. The “cost” or error value i.e. the
After completing the scan, the equivalent label pairs are sorted difference between the actual and predicted output is back
into equivalence classes and a unique label is assigned to each propagated through the network and the gradients are used to
class. As a final step, a second scan is made through the image, update the weight matrices. During each step, the gradient
during which each label is replaced by the label assigned to its descent algorithm takes a small step in the direction which has
equivalence classes. the lowest slope. This is repeated several times until the global
minimum is reached, and thus, the network is optimized.
3.6 Feature Extraction Equation 1 and 2 show the computations to get the gradients
Feature extraction a type of dimensionality reduction that for each layer, from right to left. Denotes the error values of
efficiently represents interesting parts of an image as a nodes in layer l. denotes the weight matrix from layer l to
compact feature vector. This approach is useful when image
sizes are large and a reduced feature representation is required layer l+1. g is the activation function, denotes the input
to quickly complete tasks such as image matching and
values to layer l, and is the activation at layer l.
retrieval. Feature detection, feature extraction, and matching
are often combined to solve common problems such as object
detection and recognition, content-based image retrieval, face (1)
detection and recognition, and texture classification.

3.7 Classification (2)

Classification is a type of supervised learning. It specifies the
class to which data elements belong to and is best used when
the output has finite and discrete values. It predicts a class for
an input variable as well. Some of the key areas where
classification cases are being used:
• To find whether an email received is a spam or ham
• To identify customer segments
• To find if a bank loan is granted
• To identify if a kid will pass or fail in an examination

3.8 Convolutional Neural Network

Convolution neural network gets the input image and convert it Fig. 3: Illustration of Gradient Descent
into a 3-dimensional array of pixel values then a convolution
layer is applied on the image to get the basic features using Figure 3 shows the visualization of cost where the global
feature map and the max pooling is applied to the images to minimum is at the center. J (w) denotes the cost for the weights
further decrease the dimension of the convolution layer then all w. The steps taken towards reaching the minimum are
the pixel values are flattened and are input as a neural network highlighted in black.
to learn the features like shape, size and color. The neural
network learns the extracted features using gradient decent The activation function is used to introduce non-linearity in the
algorithm. After learning, the modal is capable of prediction network. Without non-linearity, the output can only be a linear
the objects sent as input. combination of the inputs. But the dataset may contain
sequences whose underlying function cannot be approximated
Figure 2 shows the architecture of convolutional to a linear combination of inputs. Thus, non-linearity is
neural network. It consists of three layers which are required to successfully learn such sequences. Common
input layer, hidden layer and the output layer. The activation functions include hyperbolic tangent shown in
input is given in the input layer and the process of equation 3 and rectified linear tangent is shown in equation 4.
comparison and separation are done in the hidden (3)
layer. After the completion of separation process, (4)
output will be obtained in the classification layer.

© 2020, www.IJARIIT.com All Rights Reserved Page |19

 Kumar S. Krishna et al.; International Journal of Advance Research, Ideas and Innovations in Technology
4.1 Smart Fruit Separator percentage while training the data. The classifier fit model is
The smart fruit separator is controlled by raspberry pi. The used to train the model. The fit model has to be provided with
fruit separator has a pi camera to the quality of fruits. The number of times the dataset to be trained(epochs). Then predict
raspberry pi is connected to a webserver and the information is method is used to predict the belonging class of the input fruit
communicated from server. Smart fruit separator consists of image and it will display the belonging class.
Microcontroller (Raspberry Pi 3), sensors and servo motor. Th
Pi camera will capture the image and give their input to As our project is based on IoT implementation, we have to run
raspberry pi and it controls the separator and sends data to the the trained model in the Raspberry Pi. But it is not possible to
web server. The separator module consists of servo motor run the trained model in the small memory sized
which help us to separate the good fruits and rotten fruits in the Microprocessor. Our trained model is to saved locally from the
separate baskets. Google Colab and then saved model it is to feeded to
Raspberry Pi python module.
Figure 4 shows the working of the smart fruit separator the
microcontroller is connected to the pi camera for predicting the The Raspberry Pi is connected to Pi camera and servo motor.
fruits quality and image from the camera is sent to the feeded When the python code starts Pi camera captures image and
model where a CNN modal predicts whether it’s a good 9fruit save the image in the specified location. The feeded model will
or rotten fruit and based on the response from server raspberry consider input from the specified location and then the predict
pi will decides whether the fruit should be moved to good method will predict the belonging class of the fruit and it will
basket or rotten basket with help of servo motor connected to send back to python code and servo motor will move the fruit
Raspberry Pi. the good or rotten basket depends on the fruit class obtained.

Start

Capture Image

Pre-processing
processing
CNN Model

Quality Assessment Fig. 5: Raspberry connected with pi camera

Assessment 4.3 Dataset
The convolution neural network approach is tested on example
Separator images of some of the good and rotten fruits like oranges,
apples and bananas.

Good fruit Rotten fruit

Endse
nf Fig. 6: Good Fruit Dataset for apple
Fig. 4: Block Diagram of smart fruit separator

4.2 Working of Smart Fruit Separator

Initially, the sample dataset for good and rotten fruits of Apple,
Orange, Banana are downloaded from Kaggle dataset website.
The downloaded set consist of nearly 11000 images. The next
process is to train model for the downloaded dataset using Fig. 7: Rotten Fruit Dataset for apple
Convolutional Neural Network of machine learning technique.
Here, we use Jupyter Notebook of python programming to
train our model. But our system provided with Central
Processing Unit (CPU) is slow for processing huge amount of
dataset which consume long time span to complete the training
process, so we are moving forward to Google Colab runtime
environment is used as platform for the purpose of training the
dataset. Google Colab provide Graphical Processing Unit Fig. 8: Good Fruit Dataset for banana
(GPU) which is used for faster processing huge amount of
data. The classification algorithm is developed with python
programming Tensorflow (keras).
The classifier module is used to change the images to required
shape and size. This process is carried over to reduce the error Fig. 9: Rotten Fruit Dataset for banana
© 2020, www.IJARIIT.com All Rights Reserved Page |20
 Kumar S. Krishna et al.; International Journal of Advance Research, Ideas and Innovations in Technology
and again it will send back to the raspberry pi.This kind of
system can be employed in juice plants, food protection
industries, fruits and vegetables farm, packaging fruits and
vegetables, etc..

Fig. 10: Good Fruit Dataset for orange

Fig. 13: List of directories

Fig. 11: Rotten Fruit Dataset for orange Figure 13 shows the list of directories which are available in
My Drive. ls is the command used to list the directories. mkdir
The above figures 6-11 shows the sample dataset of good and is used for making a new directory.
rotten fruits quality for Apple, Orange and Banana which are
used to compare the quality of fruits are given in form of
digital image.

4.4 Evaluation Metric

The modal accuracy of the testing data is 94.12% is
accomplished with help of Gradient Descent Algorithm and the Fig. 14 Training Dataset
accuracy is stable with modal evaluation methods like
Convolutional Neural Network. The modal can predict Figure 14 describes that the fruits are trained based on the size,
different types of fruits. shape and color. The training of good fruit dataset involves
obtaining the features (shape, size and color) of good fruit. The
training of rotten fruit dataset involves obtaining the features of
rotten fruits.

6. CONCLUSION AND FUTURE SCOPE

The smart fruit separator is automated in such a way that the
fruit is placed in the tray and the process of raspberry pi takes
Fig. 12: Accuracy calculated with sample dataset place here by obtaining the fruit image through the pi camera
connected to the raspberry pi. The image is transformed to
readable format of web server using python flask framework
Figure 12 explains how much accuracy was obtained with and it is given as the input to the training model. The sample
sample datasets. The input digital fruit sample is compared Kaggle dataset for banana, apple, orange are used as training
with the trained datasets with the properties like size, shape and testing model and its result obtained with an accuracy level
and color. The accuracy obtained is dependent on the above
of 94.6% using Convolutional Neural Network Algorithm. And
properties.
some Sensors like E-nose, gas sensors were used to senses the
fruit quality.
5. RESULTS AND DISCUSSION
The Various quality detection algorithms like k-mean In future, the quality detection of our smart fruit separator
clustering, Edge Detection algorithm, Artificial Neural should be compared with other mechanical and automated
Networks, Convolutional Neural Networks, Support Vector techniques and some new parameters or features can be added.
Machine has been used to detect the quality of fruits and their The proposed model is capable of detecting a quality of one
accuracy and performances have been compared. By fruit at a time and this can be scaled up to detecting of multiple
comparing with the existing proposed model, we have found fruits of different kinds at a same time. The efficiency in
another new technique for detecting the quality of fruits using detecting fruits quality accuracy level can be increased and
Convolutional Neural Network algorithm and combined with time consuming can be reduced to short span of time. The
IoT sensors like E-Nose are used to obtain high accuracy in an smart fruit separator can also be upgrade to the next level like
efficient way. fully automated and it can be controlled through the mobile
application.
This project proposes a new technique for quality detection of
fruits. The techniques is started by capturing the fruits image 7. REFERENCES
using pi camera of raspberry pi, and then the image is [1] Krithika Jayasankar, Karthika B, “Fruit freshness detection
converted to base64 format (String) given to the web server using raspberry pi” International Journal of Pure and
using python flask framework. The base64 is again reverted Applied Mathematics, 1685-1691 ISSN: 1314-339, pp:
back to image format and given as input to the trained model. 1685-1690, 2018.
The features are efficiently extracted from the sampled image. [2] Mandeep Kaur, Reecha Sharma, “ANN based Technique
The extracted features based on the parameters like color, size for Vegetable Quality Detection, IOSR Journal of
and shapes. The CNN technique is used for checking the Electronics and Communication Engineering”, ISSN: 2278-
quality. The quality is determined by using the extracted 8735, pp:62-70, 2015.
features of fruits and values provided to the neural network [3] S.M. Shirsath, Sumit S. Telang , “Fruit Quality Inspection
during training. The proposed technique accurately detects the using Image Processing”, Journal of Springer, ISSN:2395-
quality of fruits. The result will be obtained as good or rotten 4396, pp:1355-1359, 2015.
© 2020, www.IJARIIT.com All Rights Reserved Page |21
 Kumar S. Krishna et al.; International Journal of Advance Research, Ideas and Innovations in Technology
[4] Haisheng Gao, Fengmei Zhu and Jinxing Cai, ”A Review of Journal of Electrical, Electronics and Data
Non-destructive Detection for Fruit Quality ”, Journal of Communication(IJEEDC). ISSN:2320-2084, pp: 67-70,
Springer, IFIPAICT vol 317, pp:133-140, 2010. 2018 .
[5] Rashmi Pandey, Sapan Naik, Roma Marfatia, “Image [11] Raja Sekar L, Ambika N, Divya V, “Fruit Classification
Processing and Machine Learning for Automated Fruit System Using Computer Vision”, International Journal of
Grading System”, International Journal of Computer Trend in Research and Development, ISSN:2394-9333,
Applications, ISSN:0975-8887, pp:29-39, 2013. pp:22-26, 2018.
[6] Van Huy Phan, Byung Ryong Lee, ”An Image Segmentation [12] Jyoti A Kodagali, “A Role of Computer Vision in Fruits and
Approach for Fruit Defect Detection Using K-Mean Vegetables Among Various Horticulture Products of
Clustering And Graph-Based Algorithm.”, Journal of Agriculture Fields”, Science Direct Journal, ISSN:2214-
Springer, ISSN:2196-8896, pp:25-33, 2015. 3173, pp: 01-21, 2018.
[7] Baietto M, Wilson AD, “Electronic-Nose application for [13] Dakshayini Patil, “Fruit disease detection and
fruit identification and quality grading”. Journal of Sensors, classification using image processing”. International
ISSN:1424-8220, pp:899-931, 2015. Journal for Research in Engineering Application &
[8] Ms Rupali S. Jadhav ,S.S. Patil, “A Fruit Quality Management (IJREAM) . ISSN : 2454-9150, pp:128-131,
Management System based on Image Processing”, The 2018.
International Organsation of Scientific Research Journals, [14] Kyosuke Yamamoto, Wei Guo, Yosuke Yoshiko, “On
ISSN: 2278-8735, pp: 01-05, 2013. Plant Detection of Intact Tomato Fruits Using Image
[9] Kedar Patil, Shriniwas Kadan, Suraj Kale“Machine Vision Analysis and Machine Learning”, Journal of Sensors
Based Autonomous Fruit Inspection And Sorting”, ISSN:1424-8220,pp:12191-12206, 2014.
International Research Journal of Engineering and [15] Bhanu pratap, Navneeth agarwal, sunil joshi, “Development
Technology, ISSN:2395-0072, pp:413-417, 2013. of ANN based efficient fruit recognition technique”, The
[10] Saifalt Tamakuwala, Jenish, “Quality Identification of Global Journal of Computer Science, ISSN: 0975-4172,
Tomato Using Image Processing Techniques”, International pp:18-28, 2014.

BIOGRAPHY

S. Krishna Kumar
Student
Dr. Mahalingam College of Engineering and Technology, Pollachi, Tamil Nadu

J. Kaviya
Student
Dr. Mahalingam College of Engineering and Technology, Pollachi, Tamil Nadu

G. Dilip Prakash
Student
Dr. Mahalingam College of Engineering and Technology, Pollachi, Tamil Nadu

K. Srinivasan
Professor
Dr. Mahalingam College of Engineering and Technology, Pollachi, Tamil Nadu

© 2020, www.IJARIIT.com All Rights Reserved Page |22