Turkish Journal of Agriculture and Forestry

Volume 37 Number 1 Article 7

1-1-2013

Online tomato sorting based on shape, maturity, size, and surface

defects using machine vision
OMID OMIDI ARJENAKI

PARVIZ AHMADI MOGHADDAM

ASAD MODDARES MOTLAGH

Follow this and additional works at: https://journals.tubitak.gov.tr/agriculture

Part of the Agriculture Commons, and the Forest Sciences Commons

Recommended Citation
ARJENAKI, OMID OMIDI; MOGHADDAM, PARVIZ AHMADI; and MOTLAGH, ASAD MODDARES (2013)
"Online tomato sorting based on shape, maturity, size, and surface defects using machine vision," Turkish
Journal of Agriculture and Forestry: Vol. 37: No. 1, Article 7. https://doi.org/10.3906/tar-1201-10
Available at: https://journals.tubitak.gov.tr/agriculture/vol37/iss1/7

This Article is brought to you for free and open access by TÜBİTAK Academic Journals. It has been accepted for
inclusion in Turkish Journal of Agriculture and Forestry by an authorized editor of TÜBİTAK Academic Journals. For
more information, please contact [email protected].
 Turkish Journal of Agriculture and Forestry Turk J Agric For
(2013) 37: 62-68
http://journals.tubitak.gov.tr/agriculture/
© TÜBİTAK
Research Article doi:10.3906/tar-1201-10

Online tomato sorting based on shape, maturity, size, and surface defects using machine
vision
Omid Omidi ARJENAKI, Parviz Ahmadi MOGHADDAM*, Asad Moddares MOTLAGH
Department of Agricultural Machinery Engineering, Faculty of Agriculture, Urmia University, Urmia, Iran

Received: 09.01.2012 Accepted: 04.07.2012 Published Online: 15.01.2013 Printed: 15.02.2013

Abstract: Online sorting of tomatoes according to their features is an important postharvest procedure. The purpose of this research
was to develop an efficient machine vision-based experimental sorting system for tomatoes. Relevant sorting parameters included
shape (oblong and circular), size (small and large), maturity (color), and defects. The variables defining shape, maturity, and size of
the tomatoes were eccentricity, average of color components, and 2-D pixel area, respectively. Tomato defects include color disorders,
growth cracks, sunscald, and early blight. The sorting system involved the use of a CCD camera, a microcontroller, sensors, and a
computer. Images were analyzed with an algorithm that was developed using Visual Basic 2008. In order to evaluate the accuracy of the
algorithms and system performance, 210 tomato samples were used. Each detection algorithm was applied to all images. Data about the
type of each sample image, including healthy or defective, elongated or rounded, small or large, and color, were extracted. Results show
that defect detection, shape and size algorithm, and overall system accuracies were 84.4%, 90.9%, 94.5%, and 90%, respectively. System
sorting performance was estimated at 2517 tomatoes h–1 with 1 line.

Key words: Grading, machine vision, sorting, tomato

1. Introduction following problems emerge in quality control carried out

Tomato is one of the important products in human by humans: high labor costs, labor fatigue, inconsistency,
nutrition that is consumed by millions of people daily. In and low precision due to various factors such as variations
addition, this product has a special place among Iranian in ambient light intensity, differences in personal
families. According to FAO statistics, world tomato perception of quality, and scarcity of trained labor. Using
production was 314 million t in 2010, and Iran ranks sixth machine vision will contribute to the automation of sorting
in the world with 5 million t of tomato harvested. About and reduce the labor costs and number of employees
75% of tomatoes produced are consumed fresh, and so required. The best technique for quality evaluation and
appropriate appearance is very important. Immaturity fruit sorting is machine vision. Among the advantages
and ripening disorders in tomatoes are common defects of machine vision are nondestructiveness, accuracy, and
seen in markets. According to Velioglu et al. (1998), consistency. According to Yud et al. (2002), a machine
vulnerabilities and defects in tomatoes increase with vision system can accurately identify the internal and
overuse of pesticides and toxins and incorrect storage. external characteristics of agricultural products, including
One of the most important processes in packaging and the degree of maturity, defects, moisture, and nutrients.
product supply to the market is sorting. This operation The charge-coupled device (CCD) digital cameras used
requires different parameters for quick identification and in previous studies assessed the characteristics of color or
management. Parameters include maturity, color, shape, monochrome grades to determine the quality of products
size, and defects. According to Jarimopas and Jaisin illuminated by a light source. This technique was used
(2008), the efficiency and effectiveness of sorting governs by Lino et al. (2008) in a study that classified lemons
the quality standard of the packing lines and the product, and tomatoes according to color, defects, and volume.
which, in turn, determines the marketability of the product. Equatorial diameter was measured in millimeters, and
Accordingly, it is necessary to have a rapid, consistent, the surface area was expressed in pixels as a mean of
effective, and robust method for sorting. Manual sorting diameter optical evaluation. The correlation coefficient
is the most common method for sorting the fruits. The between these 2 parameters (equatorial diameter and
* Correspondence: [email protected]
62
 MOGHADDAM et al. / Turk J Agric For

surface area) was 0.89. Blasco et al. (2009) developed a with 2 receivers for sorted tomatoes. The conveyor speed
machine for singulating, inspecting, and sorting satsuma was 105 mm s–1, and conveyor speed could be increased
mandarin (Citrus unshiu) segments using morphological using industrial cameras. The conveyor was driven by a
features. Their system automatically identified pieces of 1.49-kW, 3-phase electric motor that was adjusted by an
skin and other raw material, separated whole segments inverter (LG Inverter SV-iG5). On the left side of the belt
from broken ones, and was able to correctly classify 93.2% (Figure 1) was a box with a CCD camera (Sony, Japan). It
of sound segments on conveyor belts at 600 mm s–1. A was equipped with a circular polarizing filter and mounted
practical application was demonstrated by Zhang et al. on top of the conveyor. There were 4 LED lamps of 220
(2009) in a study that developed a machine vision system V (500–700 nm wavelength) with 4 polarizing films on
to automatically sort cherry tomato according to maturity. the right and left sides (45° from horizontal) and above
Nine features were extracted from each image. Tomatoes (perpendicular to the surface) the box to provide uniform
were classified into 3 categories (unripe, half-ripe, and light intensity with minimum shadow and light reflection.
ripe). Images were captured in the RGB color space. The The camera, with a focal length of 40.6–406 mm, was
principle component analysis (PCA) results showed that mounted 53 cm above the belt and provided a resolution
ripe tomatoes were distinguished from immature and half- of 2M pixels (spatial resolution: 640 × 480). In this study,
ripe tomatoes. The machine was able to correctly classify tomatoes were classified into 2 categories (desirable and
93.2% of tomato samples. undesirable) based on maturity, defects, shape, and size
In industry today tomatoes are sorted manually, as (2-D area) (Table 1).
are satsuma, limes, pomegranate, and other fruits. The
objective of this research was to develop an efficient
automated sorting system for tomatoes based on the image Table 1. Minimum thresholds of expectation.
processing techniques that were effectively used with
Type of sorting Threshold
limes, pomegranate, and other products.
Defect 72
2. Materials and methods
Shape 0.722
2.1. Hardware and software design
Size 35,696 (11 ± 0.2 cm2)
The hardware included a conveyor, power drive with
inverter, light source, CCD camera, mechanical segregator, Maturity R = 45–104, G = 23–50, B = 26–46
control unit, and computer. The software consisted of
separate algorithms for shape, size, maturity, and defect
detection. The electric control unit (Figure 2) comprised a
2.2. Hardware and operation microcontroller (ATmega8) and an IR sensor (made in the
Figure 1 shows the experimental tomato sorting system. It Urmia University Agricultural Machinery Engineering
featured a black conveyor belt 25 cm wide and 500 cm long Department workhouse) with wavelength of 840 nm. A
computer (CPU speed: 2.8 MHz, dual core) was used for
CCD Camera signal processing and capturing/processing images. While
LED Lamp
Lighting
Box
Segregator
Control CCD
Unit IR Camera
Sensor
Computer Control Unit
BAD

Microcontroller Computer

Processing Unit
Conveyor
Belt
Segregator
Power Drive GOOD

Figure 1. An experimental machine vision system for sorting

tomatoes. Figure 2. Block diagram depicting functional units.

63
 MOGHADDAM et al. / Turk J Agric For

traveling on the conveyor, the tomatoes passed an IR

a L Maj k – a L Min k
2 2
sensor comprising a 3-IR diode and a receiver. The signal
E=2 2 2 (1)
produced by the sensor was sent to the microcontroller, L Maj
processed, and sent onwards through a USB port to the
computer. The software commanded the CCD camera
driver to take an image using the IR diode and receiver. Here, LMaj is the length of the major axis, and LMin is the
length of the minor axis (Gonzalez and Wood 2002).
2.3. Image preprocessing
In addition, the center of the tomato could be calculated
Algorithms for processing the images were developed in as the mean value of the X and Y coordinates of the tomato’s
Visual Basic 2008. In order to separate the tomato from points (van Assen et al. 2002). As E approaches zero, the
the background, images were converted to HSI space (hue, tomato becomes more circular.
saturation, intensity). Image pixels with color outside of
2.5. Determination of size, maturity, and defects
the specified HSI range were filtered and filled with black. The segmentation algorithm that was explained in the
The processing duration for each image was 0.7 s. The section on shape determination was used. However,
suitable ranges for HSI components were: 71–31 for H, images were not thresholded since the black area (tomato
0–1 for S, and 0–1 for I. The new image was converted to only) was not considered. The area value was the size
RGB space, and its pixel colors were filtered in (0, 25) for index. Tomato maturity was inspected by 3 experts. After
red and (0, 64) for the green and blue range. The image was determination of 50 mature and 50 immature tomatoes,
then transformed to grayscale and thresholded with Otsu’s the tomato images were captured, histograms of those
algorithm. The final image was of tomato (Figure 3). images were made, and the mean of color component was
2.4. Determination of shape calculated. This value was used as a base for identifying
The shape of the tomato can be identified from its curvature. mature tomato status in the algorithm.
Defects of tomatoes include color disorders, growth
Shapes were categorized into “rounded” or “oblong”. In
cracks, sunscald, and early blight. With the exception of
order to find the shape index of a tomato, its eccentricity
being crushed, defects in tomato cause a color change
was calculated. In mathematics, the eccentricity, denoted
in the tomato skin. Images captured from defects were
E or ε, can be thought of as a measure of how much the thresholded with Otsu’s algorithm, and images captured
2-D object (section) deviates from being circular. As the from immature tomatoes were thresholded with the
eccentricity of a circle is 0, the eccentricity of an ellipse simple image statistic (SIS) algorithm. In the final image,
is greater than 0 but less than 1 (Weisstein 2011). The the background and defects were black, and the intact
visualization of eccentricity requires the center of gravity section of the tomato was white. The identification index
coordinates and the width, height, and area of the image. It of defects was “fullness”. This index was calculated as the
was calculated using Eq. (1). ratio of object area to the multiplied value of the width and

a. b. c.

d. e. f.

Figure 3. Image processing steps: a) original image, b) green component, c) red component, d) G-R
component, e) thresholded image, f) final image.

64
 MOGHADDAM et al. / Turk J Agric For

height. If it equaled 1, the entire rectangle of the object was

CE =
/N ij
(2)
filled by the pixels of the object (no black areas), which is
true only for white rectangles. If it equaled 0.5, only half of
/N i

the bounding rectangle was filled by white pixels. As the

fullness value decreased, the defective area increased. where Nij is the number of class j tomatoes in the class i
By comparing the experimental results with the receiver, and Ni is the number of class i tomatoes in the
threshold values for size, color, defective area, and class i receiver.
eccentricities, which were typed into the software, the
signal for a relevant category (desired or undesired) of 3. Results
sorting was produced. The most appropriate conveyor belt speed for image
capturing was 10.5 cm s–1. Considering the ability of the IR
2.6. Performance test sensor to accurately sense all tomatoes and the evaluation
The influences of conveyor speed, tomato spacing on the of the camera’s visual field for the imaging of 1 tomato, the
belt, and light intensity on the total performance of the perfect spacing distance for feeding was 15 cm. Spacing
machine vision system were evaluated. The optimum distance consisted of the distance between the sensor
speed of the belt was determined by analyzing the images and the end of the camera’s visual field (10 cm) and the
that were captured at different belt speeds. The spacing minimum confidence distance (5 cm).
distance of tomatoes in feeding was related to IR-sensor
3.1. Minimum threshold values of expectation
sensitivity and the visual field of the camera. After tomato All data gained from sorting experiments for each sorting
sample images from the light and dark regions of the criterion were distributed normally (in accordance with
conveyor were captured, the tomato colors in both regions Kolmogorov–Smirnov analysis). Minimum of fullness,
were analyzed in terms of gray level profiles of RGB top quartile value of eccentricity, down quartile value of
components. 2-D area, and range between minimum and maximum of
In order to remove the reflected light from tomatoes, mean color were assumed as thresholds of expectation in
5 polarizing filters, 4 for lamps and 1 for the camera, were defect, shape, size, and maturity sorting type, respectively
used. Samples were imaged, and the gray-level profiles of (Table 1).
bright spots and other tomato sections were obtained at 3.2. Color analysis in maturity sorting
the same pixel distance and were analyzed. Images obtained from 50 samples of tomatoes, with the
2.7. Performance of the machine vision system component value of mean color related to each sample,
The sorting system was operated under conditions were collected for investigation. The averages of each
considered to be optimum with regard to belt speed and component value were calculated (Table 2).
tomato spacing. A total of 210 tomatoes (Red Cloud) of 3.3. Sorting system performance
various degrees of quality (good, defective, and immature; By applying the minimum thresholds of expectation,
rounded and oblong; different colors and sizes) were samples were sorted, and the sorting accuracy of the
randomly selected from markets by an expert. Initially, system for each type of sorting was calculated (Table 3).
sorting type (size, shape, maturity, and defects) was In this study, it was assumed that defects appeared only on
selected in the software. The tomato was then sorted the side of the tomato seen by the camera.
out into 2 categories (desirable and undesirable). The
number of correctly and incorrectly sorted tomatoes in 4. Discussion
each receiver was recorded. The mean value of missorting 4.1. Effects of polarizing filters
(error) and the throughput capacity of the sorting system The results obtained from gray-level profiles indicated that
were evaluated. The mean value of missorting was the color components of tomato in all analyzed distances
evaluated using the following equation: were uniformed by the use of polarizing filters, and the

Table 2. Result of study and analysis of tomatoes in sorting based on maturity.

Type of samples Average of R Average of G Average of B

All samples 85.12 43.9 38.3

Healthy samples 82.68 40.22 37.76

Defective samples 92.08 42.08 39.23

65
 MOGHADDAM et al. / Turk J Agric For

Table 3. Calculated accuracy of each sorting type.

Expert identification System identification

Sorting type Accuracy
Good Bad True False

Defect 68 32 85 15 85.00%

Shape 40 15 50 5 90.90%

Size 28 27 52 3 94.54%

Maturity 37 13 46 4 92.00%

red component was higher. However, in the absence of a 4.2. Color analysis in maturity sorting
polarizing filter, at certain points in the distance the gray Results (Table 2) showed that the average of R was the
level of color components was close together or equal maximum among all averages. The average of B was less
(Figure 4). As a result, these points were blacked and were than the others. The average values of all components in
identified as defects. The use of polarizing filters to remove defective samples were greater than in healthy samples
the effects of light reflection from the tomatoes was (57.8 for defective samples; 53.55 for healthy samples).
effective and necessary. It was very important, especially Two profiles of gray level in defective and healthy tomatoes
when the defects were on bright spots. Polarizing filters are shown in Figure 6.
were used in 3 different modes: 1) filter used only on In order to separate the tomato from the background,
camera, 2) filter used only on light sources, and 3) filter the G component of the image was subtracted from
used on both camera and light sources. Use of the filter in the R component. Because the difference between R
the third mode provided the best results. and G in the tomato was 30 times greater than in the
The results obtained from gray-level profiles indicated background and defects, as a result of subtracted images,
that along with sampling distance, gray levels of green (G) the tomato appeared more significant and specific than the
and black (B) were affected by light intensity (Figure 5). background. Therefore, the background was removed and
As the intensity of light became greater, the gray levels the defects were effectively identified.
of G and B became greater and moved closer to red (R). 4.3. Sorting system performance
Under these conditions, B variations were greater than The average of sorting accuracy was 90.61% (Table
those of G on the conveyor. The difference between R and 3). Zhang et al. (2009) reported 94.9% accuracy of
G for tomato separation from the background was almost identification of ripe tomatoes from immature and half-
constant under both light conditions on the conveyor. ripe tomatoes. Those authors sorted tomatoes based only
As a result, use of the difference between R and G for on maturity. The current study simultaneously considered
segmentation was more appropriate. maturity, defects, shape, and size in one algorithm as

250

a 140 b
200
120

150 100
Intensity

Intensity

80
100 60

Red 40
50 Green
20 Red
Blue Green
0 Blue
0
0 20 40 60 80 100 0 50 100
Number of pixels Number of pixels

Figure 4. Image profile of captured tomato: a) without polarized filter, b) with polarized filter.

66
 MOGHADDAM et al. / Turk J Agric For

200
300
b
a
250
150

200

Intensity
Intensity

150 100

100 Red Red

Green 50 Green
50 Blue Blue

0 0
0 50 100 0 50 100
Number of pixels Number of pixels
Figure 5. Image profile of captured tomato: a) in bright region, b) in dark region.

200
250 b
a
200 150
Intensity

Intensity

150
100

100
50 Red
50 Red Green
Green Blue
0 Blue 0
0 50 100 0 20 40 60 80 100
Number of pixels Number of pixels

Figure 6. Gray-level profiles for: a) defective tomato, b) healthy tomato.

sorting factors. For this reason, the accuracy obtained in values, the carrying time on the conveyor belt for 1 tomato
this study is slightly lower than that reported by Zhang et to be sorted was 1.43 s. As a result, the throughput capacity
al. (2009). of the system was 2517 tomatoes h–1.
Accuracy for size sorting was higher than overall In this study, an image processing technique was
accuracy and the accuracies for maturity, shape, and defect developed to sort tomatoes according to 4 quality criteria:
sorting. The reasons for this difference are manifold. One maturity, defects, shape, and size. The software developed
reason could be that immature tomatoes were assumed to in this study evaluated tomato shape by its eccentricity,
have defects. In this case, 4 thresholding methods were tomato size by its 2-D image area, tomato maturity by its
used. The Otsu method gave optimum values for healthy mean color, and tomato defect by its fullness parameter.
tomatoes; however, the SIS method and a value of 16 for An experimental sorting system equipped with machine
threshold resulted in immature and defective tomatoes vision was constructed to test the ability of the software
in thresholding, respectively. Only the Otsu method was to sort tomatoes under 3 operational conditions: belt
chosen for sorting defects. The second reason could be conveyor speed, tomato spacing, and light intensity. After
incorrect identification of tomatoes by shape and color optimum operating conditions were defined, the sorting
by experts. The third reason could be that there were only machine was used to separate tomato samples according
2 classes to be sorted in all types of sorting, while more to their shape, color, size, and defects. The evaluation of
classes were sorted in combined sorting, which suggests experimental data indicated that sorting accuracy changed
that less contamination was likely to occur during size with the quality criteria considered, but overall accuracy
sorting. Based on optimum belt speed and tomato spacing was remarkably high (90.61%).

67
 MOGHADDAM et al. / Turk J Agric For

References
Blasco J, Aleixos N, Cubero S, Gómez SJ, Moltó E (2009) Automatic Velioglu SY, Mazza G, Gao L, Omah BD (1998) Antioxidant activity
sorting of Satsuma (Citrus unshiu) segments using computer and total phenolics in selected fruits, vegetables, and grain
vision and morphological features. Comput Electron Agr 66: products. J Agr Food Chem 46: 4113–4117.
1–8. Weisstein E (2011) Eccentricity. A Wolfram Web Resource.
Gonzalez RC, Woods RE (2002) Digital Image Processing, 2nd ed. MathWorld, Wolfram Research Inc., available online at http://
Prentice Hall, Inc., New Jersey. mathworld.wolfram.com/Eccentricity.html.
Jarimopas B, Jaisin N (2008) An experimental machine vision system Yud RC, Kuanglin C, Moon KS (2002) Machine vision technology for
for sorting sweet tamarind. J Food Eng 89: 291–297. agricultural applications. Comput Electron Agr 28: 173–191.
Lino LAC, Sanches J, Dal FMI (2008) Image processing techniques Zhang Y, Yin X, Zou X, Zhao J (2009) On-line sorting maturity of
for lemons and tomatoes classification. Bragantia Campinas cherry tomato by machine vision. IFIP AICT 295: 2223–2229.
67: 785–789.
van Assen HC, Egmont PM, Reiber JC (2002) Accurate object
localization in gray level images using the center of gravity
measure: accuracy versus precision. IEEE T Image Process 11:
1379–1384.

68