2017 IEEE 3rd International Symposium on Robotics and Manufacturing Automation (ROMA)

Automated Fruit Grading System

Mohammed A. H. Ali, Kelvin Wong Thai
Faculty of Manufacturing Engineering Faculty of Manufacturing Engineering
Universiti Malaysia Pahang Universiti Malaysia Pahang
26600 Pekan-Pahang, Malaysia 26600 Pekan-Pahang, Malaysia
[email protected] [email protected]

Abstract— In recent years, automatic visual inspection of quality control in fruit processing industries. From the past
technology has become more potential and important to fruit few years, different techniques have been enhanced to grade
grading applications. This is due to that the quality of fruits are and evaluate the quality of fruits. These methods can help to
the important factor for the consumer and so essential for detect different physical properties of fruits and with certain
marketing a uniform high quality products. The automated fruits
quality factors. For example, the vision-based systems include
grading technique have been set up to reduce the production
costs and improve fruit quality and replace the manual technique CCD or CMOS sensors that are used to estimate the size and
for grading of fruits as manual inspection is facing problems in shape of fruits. It helps to predict the size of the fruits from its
maintaining consistency and uniformity. Two kinds of fruits have RGB image frame with the help of CCD camera. Software
been inspected in this project; namely are apple and mango. A plays an important role in this color classification system. The
prototype of an automated fruit grading system is designed and software system is almost designed in MATLAB to detect the
developed in this paper to detect the defects on of the surface of color and size of the fruits. Color is very important in the
fruits. The system is capturing the fruit’s image using camera sorting of fruits but due to the similarity of colors between
and the fruits are placed onto of a rotating desk. The image is some fruits, the size also helps in solving the problems.
transmitted then to the processing level where the grading is done
using MATLAB.
J. Ramprabhu et.al (2014) had enhanced the technique for
Keywords—Visual inspection, Fruits; Grading System, sorting and grading the fruit quality by using Pixel wise
MATLAB, Image Processing classification method called Gaussian Mixture Model (GMM)
to improve the accuracy, reliability, consistency and
quantitative information apart from handling large volumes of
I. INTRODUCTION fruits [1]. By referring to the results, the edge extraction is one
The quality of the fruits is important for the consumers and of the important key factor for size detecting. By using the
become the requirement from the suppliers to provide fruits most powerful edge-detection method (Canny method) that
with high standards quality. So, in the past few years, fruit uses a multi-stage algorithms, we are able to detect a wide
grading systems have established to fulfil the needs of the fruit range of edges in images. On the other hand, Ali M. et. al,
processing industry inspection. Besides that, the process of (2011, 2012) has presented a visual inspection system to test
fruits involves several steps that can generally be classified the lateral surface of cylindrical products such as pin by using
into grading, sorting, packaging, transporting and storage. The cameras and image processing [2,3]. In this work, two kinds of
grading are considered as the most important steps towards the cameras are used for inspection: lines scan camera that uses a
high standard of quality. single line of photosensitive element with a CCD sensor and
Fruits are almost graded manually which is an expensive and matrix camera system that can capture the images by
time consuming process and labors shortage will affect to the involving 14 a rectangular mosaic of pixels. The image
operation during peak seasons. It has become increasingly resolution in line scan camera is better compare to matrix
difficult to hire or train the person who are willing to handle camera system. Rupali et. al, (2013) has used image
the monotonous task of inspection. In the meanwhile, a cost processing that provide the solution for automated fruit size
effective and accurate grading can be performed with detecting and grading system to solve the non-destructive
automated grading system. quality evaluation of fruits [4]. It was designed over
Generally, the fruits quality depends on outer parameters MATLAB software to inspect the color and size of the fruits
(size, color intensity, shape, surface appearances) and inner and categorize them into four groups: red color with small in
parameters (sugar contents, acid contents) but color and size is size, red color with big in size, green color with small in size
the most important factor for grading and sorting of fruits. and green color with big in size.
Nowadays, the fruit grading system is accomplished based on
weight, color and size which are accessible in all fruit Different types of algorithms and classifier are available to
processing industries. extract feature of fruit characters to solve the problems for
The fruit grading system techniques using computer fruit detecting and grading system as suggested Seema K. et.
machine vision and image processing play the important role al (2016) [5]. Based on the results, there are four methods : a)

978-1-5386-2539-2/ 17 / $31.00 ©2017 IEEE

Authorized licensed use limited to: Birmingham City University. Downloaded on March 03,2024 at 12:29:55 UTC from IEEE Xplore. Restrictions apply.
fuzzy logic that utilize digital fuzzy image processing, content measurement. Image processing for the ripeness, size and
predicted analysis, and statistical analysis; b) artificial neural decays were done by Mousavi A. et. al, (2012)). Based on the
network that uses an algorithms to get the better result for experiment results, cherries are categorized based on the total
colour and morphology; c) K-nearest neighbors classifier soluble solids that is the index for ripeness. By using binary
(KNN) that identify the input data by comparing it with the images of the cherries, the algorithms based on size was
trained data and last d) color mapping that used to evaluate the expanded. Lastly, the reflected light in the images were
quality and maturity stage. Chandran K. et. al (2015) has removed to decrease the error rate in computing the average
proposed a system for sorting and color grading (GLCM color components of the fruits.
parameter) that is used for the classifications of citrus fruit [6].
Based on the results, contrast shows the amount of local By results we can say that the inspection of rotationally symmetric
variation present in the image. Thus, when i and j are with the products such as fruits still a complicated problem in automation
same value, then (i-j)=0, then contrast is zero. Moreover, technology since it has to be tested from all geometrical sides.
GLCM consider the relationship between two pixels at one
time, called neighbor pixel and reference. Khojastehnazhand II. GRADING SYSTEM DESIGN AND FABRICATION
et. al (2010) has done an experiment on lemon sorting system
based on color and size by using the system that able to The grading system in this project relies on the features
inspect the lemon in two stages : external and internal extracted from the image. Therefore, feature extraction plays
inspection [7]. Based on the results, the data about the color an important role in developing the system.
and size of lemon product that acquires during the sorting will
be compared to the database that stores the range of color and
size for the accepted quality of lemon; which is resulted by
deciding if there is defects on the lemon or not.

A CCD camera system is mounted on the conveyor belt to

collect images of the fruits by Nandi C. et. al (2014) [8]. With
the fuzzy rule based algorithms, fruits can be sorted into many
grades. Mis-gradation may also occur when different maturity
With
level having almost the same colour pattern. An experiment on
Defects
distributed network architecture to interface the camera unit to
a computer system through GigE LAN environment for
automated inspection and grading of fruits is done by Yogitha. Without
S et.al (2014) [9]. The computer mission vision system in this Defects
work involves the following process: an image acquisition, Fig.1. Flowchart of grading process
image processing and decision making. Besides, the grading
and sorting system is based on external parameters such as In order to design a system to grade the fruit, image
color, size and shape. V. Pavithra et. al (2015) has designed an acquisition and features classification is the important steps.
automatic non-destructive grading and sorting system of The design is first concerned with establishing a basic
cherry tomatoes based on maturity and quality [10]. The structure of a system. Before that, the CCD camera/ camera
algorithms involves two phases : a) product maturity and b) from computer is used to capture the image of the fruits to do
product quality. In the first phase, the grading was based on the classification of fruits. Fruit varieties are further classified
colour and in the second one was based on colour, texture and upon their appearance such as colour and size. Fruit
shape. Quality based classification has been performed using classification and fruit disease identification can be seen as an
KNN based SVM classification. instance of image categorization. To measure and calculate the
features from the image samples to distinguish between one
Image inspection of the pear by the software of Lab type of image from another, feature extraction is used. The
Windows/CVI is done by Zhao Y. et. al, (2009) [11]. The feature extraction process is done using the MATLAB image
system can be used to detect the external conditions such as processing toolbox.
size, shape color and surface decay. The system can also be
used for several kinds of fruit such as apples, peaches and for As mentioned earlier, two types of fruits are used in this
vegetable like cucumbers. Unay D. et. al (2005) has designed project; namely are apples and mangoes. The performance of
apple grading system using an artificial neural network-based the grading system depends on many factors that farmers use
segmentation. In this work, one-view images of apples are for measuring the fruit quality. These factors can be classified
captured using a monochrome digital camera; then Linear into two groups that are external quality factors and internal
Discriminant Classifier (LDC), separates the feature space into quality factors. The external quality factors can be defined as
two half-spaces by minimizing a criterion function and visual appearance of the fruit. Commonly used factors are
Nearest Neighbor Classifier (k-NN). Euclidean distance is size, shape, colour, surface defects and decay. The internal
used to find the nearest samples in the case of similarity quality factors are smell, taste, sweetness and sourness but in

Authorized licensed use limited to: Birmingham City University. Downloaded on March 03,2024 at 12:29:55 UTC from IEEE Xplore. Restrictions apply.
this project we will only concern on the external quality = 1.8cm x 1.3kg= 2.34kgcm= 2.34kgcm x
factors that is surface defects and decay.
= 0.23N.m
Automatic grading system is the implementation that begins
with the fruit sample being captured using camera with white
B. Hardware Development
background that image analysis can be applied to make
grading in MATLAB. Image processing in MATLAB is used
This system consists of mechanical part such as rotating
to extract the parameter of apple and mango in order to
desk that act as a place for inspection; electrical parts such as
prepare the input for classification. The features such as
DC motor, Arduino, computer and software such as image
surface defects or decay of fruits is used in this project.
processing in MATLAB. The automatic fruit grading system
is shown in Fig. 3 as 3D drawing and Fig. 4 after fabrications.
A. Inspection System Setup
In this project, camera from laptop is positioned to detect the
lateral surface of the fruit. The sample of fruits are brought to
The apparatus used for fruit classification in this project is
the rotating desk for grading. The DC motor is used to rotate
made by simple image processing equipment. In this project,
the desk which allow the camera to capture the lateral surface
the computer vision system will be set up to detect the lateral
of the fruits.
surface of the fruits shown in Fig. 2. The system was tested by
using a samples of apples and mangoes. Firstly, the fruit is
brought manually to the rotating desk which is connected to
the shaft of the 12V DC motor. The DC motor is then set by
Arduino to rotate 180° twice for each of the fruits in order to
be able to detect the defects of the whole lateral surface of
fruits. Then, the camera will capture the image and show the
analyzed image at the Graphical User Interface (GUI). We can
then observe the surface defects and decay of fruits through
GUI.

Fig.3. Hardware draw by Catia software

Fig.2. Automatic fruit grading system

The DC motor required for rotating desk holding fruits has to

be chosen based on the following considerations:

- Maximum weight of fruit sample is considered to be less or

equal to 1.0kg Fig.4 Prototype of the grading system

- Weight of rotational desk is = 0.3kg

- Torque of DC motor= 0.784N.m C. Graphical User Interface (GUI)

- Weight of fruit sample + weight of rotating desk Graphical user interface (GUI) is a type of user interface that
allows users to interact with electronic devices. GUI is used in
= 1.0kg + 0.3kg this project to show the defects of lateral surface of fruits in
two halves as in Fig. 5. The image of the fruits that captured
- Radius of DC-Motor = 1.8cm by the camera for each half will be shown at the box in GUI.
Then, the centers and radii of defects of apple and mango are
Torque needed to turn the rotating desk and sample of fruit analyzed.

Authorized licensed use limited to: Birmingham City University. Downloaded on March 03,2024 at 12:29:55 UTC from IEEE Xplore. Restrictions apply.
 First half Cam. View Defect

Centres and Radii of defects

Second half Cam. View Defect

Start
Fig.6. System setup with white background

The texture is important for image analysis because the

Fig.5. Graphical User Interface (GUI)
recognition of color by computer vision system is different if
the surfaces of the fruits have different textures. For example,
D. Classification of fruits
the texture for the area with defects on the surface of fruits
will be darker. Thus, the dark patches or holes present in the
In this project, the classification of the fruits are done based
image are filled using the function ‘imfill()’, and its
on the types of defects. First, two types of fruits are being
syntax is BW1 = imfill(BW1,’holes’) that fills holes
chosen for this project that are apple and mango. Then,
in binary image BW.
mangoes are divided into two groups that are anthracnose and
normal mango. Apples are divided into two groups also that
are bull-eye rot and normal apple. The camera will capture the To grade the fruits, some apples and mangoes are brought to
image of the surface of fruits and will analyze the types of the rotating desk. By observing the image analyzed by
defects of the fruits. MATLAB and shown in GUI, the dark patches or spots will
be occurred at the area of fruit’s defects, which almost has a
III. GRADING SYSTEM IMPLEMENTATION, RESULTS circular shape. In this project, the command ‘[centers,
AND DISCUSSION radii] = imfindcircles(image,[10 40])’ is
used on the image to find the defects which are almost in
Firstly, an image of the fruits is captured by using a circular shape with the search radius of [10 40] pixels. From
camera and is loaded into the MATLAB by using the function the GUI, if there is any dark patches or spots at the analyzed
‘videoinput’. This function reads the image from the image, then, there is defect or decay on the surface of fruits. In
camera. As if it is an colored input, it need to be converted addition, if there is defect on the surface of fruit, the GUI will
show the position of the defect and number of defect.
into grayscale by the function ‘rgb2gray(image)’ and the
Moreover, a red circle will appear on the defect area in the
syntax is I=rgb2gray(image) which converts true color
image if there is a defect.
image RGB into grayscale intensity image. Then, the image is
converted into binary which the image consists only two
Figs. 7 and 8 show the GUI for apple with defects and without
colors that are black and white. Canny edge detection method
defect in respectively.
is used in this project to detect the edge of the image of the
fruits and the corresponding syntax is BW =
From the image in Fig. 7, one can see that there are black
edge(I,’canny’) to extract the boundary. Since the patches on the surface of fruit for the first half side but there is
image is captured with white background, the background will no black patch on the surface of fruit for the second half side.
be having pixel value of 255 indicating white color. Fig. 6 Therefore, in the analyzed images, there are defects on the
show the system setup white background is used to capture the first half side of fruit but no defect has been occurred in
image of fruits. second half side of fruit. Additionally, there are two red circles
on the first half side of fruits means that there are two defects.

Authorized licensed use limited to: Birmingham City University. Downloaded on March 03,2024 at 12:29:55 UTC from IEEE Xplore. Restrictions apply.
 Fig.9. GUI for mango with defect
Fig.7. GUI for apple with defect
From Fig. 10, it is clear that there is no black patch for the
In similar way, no defects for both first half side and second both first half side and second half side of mango, which is
half side of apple have been occurred as shown in Fig. 8 resulted by no defect for both first half side and second half
side of mango after testing.

Fig.8. GUI for apple without defect Fig.10. GUI for mango1 without defect

Figs. 9 and 10 show the GUI for mango with defects and From all results in Figs. 7-10, we can see that the system is
without defect in respectively. able to detect the defects that have a spot with radius not less
than 10mm. The other small spots are considered as some
Fig. 9 shows the black patches occurred in the first half side changes in the color of fruits as the fruits can’t have just one
of mango however there is no black patch at the second half color in the whole surface. The position of defects is found in
side of mango. By result, there are defects on the first half side the image as the coordinate system of the image’s pixel not in
but no defect on the second half side of mango. The GUI real dimensions.
shows that there are five red circles on the first half side of
mango means that there are five defects while there is no red IV. CONCLUSION
circle at the second half side means that there is no defect.
In this paper, an automatic fruit grading system was
developed. This system consists of mechanical part such as
rotating desk that act as a place for inspection; electrical parts
such as DC motor, Arduino, computer and software such as
image processing in MATLAB. This automatic grading
system has been designed to meet the demands in grading
fruits’ operation compared to manual grading. The grading of
the fruits is based on the external quality factor based on

Authorized licensed use limited to: Birmingham City University. Downloaded on March 03,2024 at 12:29:55 UTC from IEEE Xplore. Restrictions apply.
surface defect and decay. This automatic inspection system [7] Khojastehnazhand, M., Omid, M., & Tabatabaeefar, a.
has saved time, effort and better accuracy than manual sorting. (2010). Development of a lemon sorting system based
This system starts with a DC motor that is programmed by on color and size. Plant Science, 4(April), 122–127.
Arduino to rotate 180° twice for each of the fruit. If there is a
defect on the inspected surface of fruit, a red circle will appear [8] Nandi, C. S., Tudu, B., & Koley, C. (2014). Machine
around the defect at the analyzed image. The other parameters vision based automatic fruit grading system using fuzzy
such as size, shape should also be included in this project in algorithm. International Conference on Control,
future research. These parameters will play valuable role for Instrumentation, Energy and Communication, CIEC
quality analysis process. In addition, since there will have 2014, 26–30.
some kind of fruits with same color such as tomato and apple,
so, there will be having some misclassification. Hence one [9] Yogitha, S., & Sakthivel, P. (2014). A distributed
feature can be added also namely texture while classifying computer machine vision system for automated
such kinds of fruits. inspection and grading of fruits. Proceeding of the IEEE
International Conference on Green Computing,
ACKNOWLEDGEMENT Communication and Electrical Engineering, ICGCCEE
2014, (0), 1–4.
The authors would like to express gratitude and
acknowledge to Ministry of High education and, Universiti [10] Pavithra, V., Pounroja, R., & Bama, B. S. (2015).
Malaysia Pahang (UMP) for continuous support under FRGS Machine vision based automatic sorting of cherry
grant (RDU160131). tomatoes. 2nd International Conference on Electronics
and Communication Systems, ICECS 2015, (Icecs),
REFERENCES 271–275.

[1] J.Ramprabhu and S.Nandhini (2014). Enhanced [11] Zhao, Y., Wang, D., & Qian, D. (2009). Machine Vision
Technique For Sorting And Grading The Fruit Quality Based Image Analysis for the Estimation of Pear
Using Msp430 Controller. International Journal of External Quality. 2009 Second International Conference
Advances in Engineering & Technology. Vol. 7, Issue 5, on Intelligent Computation Technology and
pp. 1483-1488. Automation, 629–632.

[2] Ali, M. A. H., Mailah, M., Tang, H. H., & Kazi, S. [12] Unay, D., & Gosselin, B. (2005). Artificial neural
(2012). Visual inspection of cylindrical product’s lateral network-based segmentation and apple grading by
surface using cameras and image processing. machine vision. Proceedings - International Conference
International Journal of Mathematical Models and on Image Processing, ICIP, 2(9813783), 630–633.
Methods in Applied Sciences, 6(2), 340–34
[13] Mousavi, A., Ahmadimoghaddam, P., Modares, A., &
[3] Ali, M., Mailah, M., Kazi, S., & Tang, H. H. (2011). Dolaty, H. (2012). Sorting and grading of cherries on the
Defects Detection of Cylindrical Object’s Surface using basis of ripeness, size and defects by using image
Vision System, 222–227. processing techniques. International Journal of
Agriculture and Crop Sciences (IJACS), 4(16), 1144–
[4] .Rupali S.Jadhav, & PROF. S.S.Patil. (2013). A Fruit 1149.
Quality Management System Based On Image
Processing . IOSR Journal of Electronics and
Communication Engineering (IOSR-JECE), 8(6), 01–
05.

[5] Seema, Kumar, A., & Gill, G. S. (2015). Automatic Fruit

Grading and Classification System Using Computer
Vision: A Review. Proceedings - 2015 2nd IEEE
International Conference on Advances in Computing
and Communication Engineering, ICACCE 2015, (I),
598–603.

[6] Chandan Kumar, Siddharth Chauhan, R. N. A. and H. M.

gurram. (2015). Classifications of Citrus Fruit Using
Image Processing -GLCM Parameters. In IEEE
International Conference on Communications and Signal
Processing (ICCSP), 1743–1747.

Authorized licensed use limited to: Birmingham City University. Downloaded on March 03,2024 at 12:29:55 UTC from IEEE Xplore. Restrictions apply.