Computer vision system based on conventional imaging for
Computer vision system based on conventional imaging for
Review
A R T I C L E I N F O A B S T R A C T
Keywords: Quality assessment of fresh fruit and vegetables is an arduous and complex process which needs very intensive
Contactless technology labour of correlation among sensory and subjective determinations and conventional destructive methods.
Fresh produce Consumers’ requests for fruit and vegetables with high quality in terms of appearance, nutritional value and
Image analysis
safety have prompted industries and researchers to develop rapid, precise and low-cost techniques for food
Packaging material
Quality assessment
analysis. Among innovative techniques, image analysis by computer vision systems based on conventional im
aging (CVS-CI) have proved to be effective and suitable for application at industrial level. This review sum
marizes developments on CVS-CI technology for the evaluation, along the entire distribution chain, of external
defects, colour changes and internal chemical-physical attributes of fresh fruit and vegetables, with or without
the presence of plastic packaging. The most interesting researches carried out during the last ten years on CVS-CI
employments are reported and discussed. The description of each application points out the performances ob
tained, the hardware components, the image processing techniques used to extract information from the images
acquired, the classification/regression models used to grade products and to estimate their quality traits. Finally,
future perspectives and possible new applications of CVS-CI in postharvest field are proposed.
* Corresponding author at: Institute of Sciences of Food Production, National Research Council of Italy (CNR), c/o CS-DAT, Via Michele Protano, 71121 Foggia,
Italy.
** Corresponding author.
E-mail addresses: [email protected] (M. Palumbo), [email protected] (M. Cefola).
https://doi.org/10.1016/j.postharvbio.2023.112332
Received 21 June 2022; Received in revised form 9 March 2023; Accepted 10 March 2023
Available online 31 March 2023
0925-5214/© 2023 The Authors. Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
M. Palumbo et al. Postharvest Biology and Technology 200 (2023) 112332
et al., 2015; Aaby et al., 2019); appearance and colour traits, flavour and customers. The use of CVS-CI, where feasible, can provide greater flex
soluble solids content, hedonic liking and volatiles in mango (Salina ibility to match cost (they exploit cameras that can be already available
s-Hernández et al., 2015; Sung et al., 2019); sensory (i.e. juiciness, on the market at relatively low cost), time constraints and environment
mealiness, etc.) and instrumental texture attributes in melon genotypes requirements at different points of the supply chain up to the final user.
(Bianchi et al., 2016; Farcuh et al., 2020). A careful design is required to integrate mechanical and optical
Although still widely used, analytical and destructive techniques are components for producing digital images that contain the relevant in
time-consuming and expensive, adversely impact the environment, may formation for the task at hand (Patel et al., 2012). CVS-CI typically uses
require sophisticated equipment and need careful sample preparation. high resolution charged coupled device (CCD) or complementary
Finally, they are unsuitable for application in industrial lines, where metal-oxide semiconductor (CMOS) digital cameras based on RGB
rapid, reliable, non-destructive, less expensive and less polluting colour for image acquisition. They involve the choice of a proper illu
methods for grading fruit and vegetables, assessing their quality and mination system, carefully selecting spectral distribution and spatial
detecting defects are required (Narendra and Amithkumar, 2019). geometry of the light sources. The position as long as the choice between
Recently researchers have focused on the use of contactless, non- front or back lighting, the type of lamps (incandescent, fluorescent, la
destructive, rapid and accurate, as well as non-polluting, techniques sers, led or infrared lamps), colour quality and uniform distribution of
for fruit and vegetables analysis to objectively assess sensory and the illumination are all important concerns when designing an efficient
compositional quality. They can be considered complementary along the and accurate vision system (Zhang et al., 2014). Finally, a personal
supply chain enabling time and cost saving, continuous and reliable computer normally collects data acquired by the sensors, performs some
monitoring and reduction of impact on environment (Chaudhry et al., basic processing (colour correction, segmentation, feature extraction),
2020). accomplishes features classification or parameters estimation by con
Computer vision systems (CVS) represent an innovative and con structing suitable models using statistical methods or machine learning
tactless non-destructive technology suitable for in-line grading and techniques (i.e. decision trees, regression trees, ensemble learning,
quality assessment of fruit and vegetables (Fan et al., 2020). random forest or convolutional neural network) (Fig. 1).
The most common CVS for the quality inspection of fresh produce is a In the last decades, many researchers have studied CVS-CI technol
traditional system based on RGB colour cameras that reproduce the ogy to automatically assess several and relevant attributes of fruit and
vision of human eyes using three monochromatic filters centred on red vegetables in quality control. Classifying and sorting are key factors in
(R), green (G) and blue (B) wavelengths at 700, 546 and 435 nm, quality assessment of fruit and vegetables during postharvest handling.
respectively (Lorente et al., 2012). CVS based on conventional imaging Computer vision is highly effective in grading foods free from defects
(CVS-CI) mimic human vision, acquiring and analysing images of the and with good appearance in terms of colour, texture, shape and weight.
visible surface of food to assess its quality (Bhargava and Bansal, 2021). Moreover, recent research works have demonstrated the possibility to
The CVS-CI can measure and detect many external quality traits (i.e. predict also internal quality parameters of products through elaborated
colour, shape, size and texture) and defects. CVS-CI automatically images acquired by a conventional vision system. During postharvest
extract from images the most discriminative features related to quality storage, ripening or senescence processes lead to an alteration of inner
and grade product using properly trained classification or regression quality of fruit and vegetables causing changes in the visual appearance
models. (colour or texture) (Xia et al., 2016). The existence of correlations be
Recently, with the advances in hardware, software and high- tween colour and physical or chemical attributes (i.e. vitamins, antiox
resolution cameras, multispectral and hyperspectral CVS have been idants, total phenols, titratable acidity, soluble solids content or
developed as efficient technologies for quality assessment of agricultural enzymatic activity) allows the estimation of internal fruit properties
products (Baranowski et al., 2012; Chen et al., 2017; Li et al., 2018; starting from CVS-CI images.
Sendin et al., 2018). The spectral imaging data acquired by the hyper In addition, a CVS-CI may be very useful also on high-convenience
spectral and multispectral CVS provide information about internal and fresh-cut products, where a high level of quality in terms of appear
external traits that the CVS-CI can difficultly evaluate and analyse ance, sensorial, and nutritional characteristics is required and where the
(Zhang et al., 2014). Such techniques provide consistent and accurate higher convenience often corresponds to a more rapid perishability
quality assessment but acquiring multispectral and hyperspectral images during the shelf-life (Watada et al., 1996; Barrett et al., 2010; Francis
produces a large amount of data, requires costly and complex devices for et al., 2012). For these products, another important aspect to consider is
acquisition and processing. Their application often requires specific the increasing consumers’ need to ascertain the real quality level inside
competences by users and poses relevant constraints on the acquisition the packaging. The use of CVS-CI can be helpful in monitoring fresh-cut
environment. These characteristics make them unfeasible for cost fruit quality along the whole supply chain. A relevant challenge to face
effective and pervasive application along the supply-chain to provide a in ensuring a correct quality assessment through the packaging material
continuous monitoring of the parameters of interest from harvest to final by CVS-CI is the separation of opaque and affected regions of bags from
2
M. Palumbo et al. Postharvest Biology and Technology 200 (2023) 112332
Table 1
Applications of computer vision system based on conventional imaging (CVS-CI) for the evaluation of external defects, colour changes and internal chemical-physical
attributes of fresh and packaged fruit and vegetables.
Products Objective Classification methods Main results as Accuracy, Achievements and Model Reference
Parameters
Evaluation of external defects
Apple Two types of classification: - Multi-layer perceptron SVM accuracy of: Moallem et al. (2017)
- 2 categories (healthy or defected) (MLP) - 92.5% for the recognition of the 2 categories
- 3 quality rank (first rank, second rank and - Support vector machine - 89.2% for the assessment of 3 quality ranks
rejected ones) (SVM)
- K-nearest neighbor (KNN)
Blueberry Classification in: - Linear discriminant analysis - Identification of fruit orientation in 96.8% of the Leiva-Valenzuela and
- 4 classes (good blueberries, shrivelled, (LDA) cases Aguilera (2013)
decayed and mechanically damaged berries) - support vector machine Performances of
- 2 fruit orientations (stem-end and calyx-end) (SVM) - 97% for the evaluation of fungally decayed
- 93.3% for shrivelled fruits
- 86% for mechanically damaged berries
Mango - Automatic defect identification Pre-processing by three - Rotten fruit with higher value than the threshold Sahu and Potdar
- Maturity detection algorithms - High fruit quality with lower value than the (2017)
threshold
Mango and Classification of mango and apple into 2 Graphical user interface - Defect or decay on the surface of fruits identified as Ali and Thai (2017)
apple classes (anthracnose or normal mango and (GUI) dark patches or spots
bull-eye rot or normal apple) - GUI showed the position of the defect and the total
number of defects detected
Orange Detection of several types of defects (insect Processing by a novel Performance of: Rong et al. (2017)
injury, wind scarring, thrips scarring, scale algorithm - 93.8% in stem end classification from defective
infestation, canker spot, dehiscent fruit, orange
copper burn, phytotoxicity) - 91.9% in detection of individual defects
- 97% in detection of defective samples
Automatic orange classification into 3 groups Back propagation neural - The grading accuracy of the method reached Chen et al. (2018)
(grade A, B and C) network 94.38%
- The grading accuracy of the grade A reached 100%
Eggplant Classification of 4 categories of healthy or K-nearest neighbour Accuracy of 88% Akter and Rahman
unhealthy eggplant (healthy, partially (2017)
defected, moderately defected and unhealthy)
Tomato Classification into defective or non-defective Artificial neural network Accuracy of Arakeri and
and ripe or unripe vegetable fruit - 100% in defective/non-defective task Lakshmana (2016)
- 96.47% ripe/unripe task
Fresh-cut Detection of 4 grades, from good to bad - K-nearest neighbour (KNN) Same results of BPNN and KNN models with a Huang et al. (2019)
spinach - Support vector machine classification accuracy of 85.42%
(SVM)
- Back-propagation artificial
neural network (BPNN)
3
M. Palumbo et al. Postharvest Biology and Technology 200 (2023) 112332
Table 1 (continued )
Products Objective Classification methods Main results as Accuracy, Achievements and Model Reference
Parameters
Evaluation of external defects
Borlotto beans Definition of an algorithm to measure the Multiple regression - Change of seed ground colour as the most effective Amodio et al. (2011a)
colour of red spots and seed ground colour feature
- A positive regression between the sensorial seed’s
score and a* value
- A negative regression between the sensorial score
and the Hue angle
Rocket leaves Classification of rocket leaves into 5 quality Random forest model Accuracy of Palumbo et al. (2021)
levels (QL) and discrimination of the - 95% in QL assessment
cultivation approach - 65–70% in the discrimination of the cultivation
approach
Tomato Identification of 4 grading categories and - Gray-level coccurrence SVM-RBF accuracy of Ireri et al. (2019)
development of a calyx detection algorithm matrices (GLCM) - 95% for the calyx detection
- Support vector machine- - 98% in the detection of defected regions
radial basis function (SVM- - 97% in the identification of 4 categories
RBF)
- Artificial neural network
(ANN)
- Random forest
Fresh-cut apple Browning evaluation on 3 fresh-cut apple Normalization Significant accuracy in the detection browning Subhashree et al.
varieties (2017)
Fresh-cut Quality assessment by colour detection from Standard regression - Highest correlation coefficients (from 0.90 to 0.92) Amodio et al. (2011b)
artichoke whole quarter surface and from the browned for L* measured on the whole quarter surface
areas - Lower correlation coefficients (from 0.77 to 0.91)
for L* and b* measured on brown areas
Fresh-cut Quality level assessment through plastic bags - Convolutional neural - Irrelevant performance loss due to the presence of Cavallo et al. (2018)
lettuce network packaging
- 3-nearest neighbours - 83% classification accuracy on packaged lettuce
method - 86% accuracy on product without packaging
fresh-cut Classification into 5 quality levels (from 5, Correlations between visual - Higher correlations between visual appearance and Pace et al. (2011)
nectarine excellent, to 1 inedible) appearance and colour colour parameters b* and Chroma measured by CVS-
features extracted CI (R2 = 0.76)
- Lower correlations obtained using a colorimeter (R2
= 0.57)
Fresh-cut Classification and prediction of browning - Support vector machine - Accuracy of 96% to predict the storage life of the Hongyang et al.
potato (SVM) product by the SVM model (2021)
- Partial least squares PLSR accuracy of
regression (PLSR) - 86% for L*
- 96% for a*
- 99% for b*
Fresh-cut Evaluation of 5 quality levels (QL) (from 5, Dendrogram based on the Best discrimination of the QLs in 3 quality classes Pace et al. (2015)
radicchio very good, to 1, very poor) during the cold Euclidean distance (high, middle and poor) by colour parameter a* over
storage the W and the percentage of W2
Evaluation of chemical-physical attributes
Banana Evaluation of peel browning during storage Non-linear mathematical No significant differences of correlation coefficients Nadafzadeh et al.
by laboratory measurement of polyphenol models genetic programming between predicted values (0.98) and measured ones (2018)
oxidase (PPO) and peroxidase (POD) enzyme (0.97) of PPO and POD enzymatic activity
activity
Grapes Estimation of - Root mean square error of RMSEC values of Xia et al. (2016)
- soluble solids content (SSC) - pH cross-validation (RMSECV) - 0.09 for SSC prediction
- Root mean square error of - 0.77 for pH prediction
calibration (RMSEC) RMSECV values from
- 0.09–0.10 for pH prediction
- 0.79–0.80 for the SSC prediction
Kiwifruit Estimation of - R to B (Central R/B) ratio - Negative correlation between central R/B value and (Li et al., 2022)
- titratable acidity (TA) - B to G (Central B/G) ratio TA, vitamin C contents and firmness
- vitamin C of the central site of kiwifruit - Positive correlation among central R/B value with
from photos taken by a smartphone camera soluble solids content, total soluble sugars and total
plate counts
Mandarin Estimation of Polynomial transformation - Good correlations (0.88 for TSS and 0.45 for pH) Hadimani and Mittal
- Total soluble solids (TSS) to convert RGB to between measured physicochemical properties and (2019)
- pH L* a* b* colour space the changing peel colour of the fruits
- Linear regression - High accuracy (R2 = 0.97) with PLSR
- Partial least square
regression (PLSR) models
Mango - Classification into 3 grades (green ripe, fully - Support vector To classify mango fruits into 3 grades, SVC accuracy Huang et al. (2018)
ripe and over ripe) classification (SVC) rates of
- Prediction of hardness and total soluble - Support vector regression - 98.75 for the training set
solids (TSS) (SVR) - 97.5% for the prediction set
High SVR correlation coefficients for
- hardness (0.9051 in training sets and 0.8897 in
prediction sets)
- TSS (0.9515 and 0.924 in training and prediction
sets, respectively)
(continued on next page)
4
M. Palumbo et al. Postharvest Biology and Technology 200 (2023) 112332
Table 1 (continued )
Products Objective Classification methods Main results as Accuracy, Achievements and Model Reference
Parameters
Evaluation of external defects
Orange pH estimation of 3 orange varieties - Hybrid artificial neural High accuracy of MLP (R2 = 0.95) in orange pH value Sabzi et al. (2020)
network-particle swarm prediction as mean value of the 3 orange varieties
optimization (ANN-PSO)
- Multilayer perceptron
(MLP) neural network
Plum Prediction of total soluble solids content Multi attribute decision Strong associations between total soluble solids Kaur et al. (2018)
making theory content with mean intensity of green colour (R2 =
0.997) and R/G ratio (R2 = 0.846)
Strawberry Prediction of titratable acidity (TA) - Threshold method for the Significant correlation between TA and the image Palumbo et al.
segmentation of strawberries data (Pearson correlation coeffient is about 1) (2022a)
- Correlation matrices based
on the Pearson correlation
coefficient
- Hierarchical clustering
procedure by Euclidean
distance and Ward’s method
Estimation of - Multiple linear regression SVM-R accuracy of (Basak et al., 2022)
- total soluble solids (TSS) (MLR) - 84.1% and 79.2% for TSS in training and testing test
- pH - Support vector machine - 78.8% and 72.6% for pH in training and testing test
regression (SVM-R) models
Fresh rocket Total chlorophyll content prediction Random forest regression - Accuracy of the random forest regression of R2 Cavallo et al. (2017)
leaves = 0.90
- Accuracy of a common SPAD-meter of R2 = 0.79
Pigmented Prediction of Multivariate model - Good correlation between AA and TP predicted Pace et al. (2013)
carrot - Antioxidant activity (AA) levels and the real AA and TP measurements (R2 =
- Total phenols (TP) 0.97 and R2 = 0.94, respectively) for both internal
and external parts of carrots
- Lower determination coefficients (R2 = 0.93 for AA
and R2 = 0.86 for TP) for the internal part
Fresh-cut Prediction of - Random forest regression Pearson’s linear correlation coefficient of Palumbo et al.
rocket leaves - Total chlorophyll content - Partial Least Square (PLS) - 0.84 and 0.91 to predict chlorophyll and ammonia (2022b)
- Ammonia content models content on packaged products
through the packaging - 0.86 for chlorophyll and 0.92 for ammonia on
unpackaged products
PLS models accuracy of
- 0.70 for destructive methods
- 0.77 for CVS-CI through packaging
- 0.80 for CVS-CI without packaging
the transparent area where the product is visible with acceptable fidelity fruit and vegetables at harvest or during the postharvest storage.
of visual appearance. This separation step requires robust and powerful Research works from 2011 to date were considered, reporting the
segmentation approaches (Cavallo et al., 2018). At the moment, methodology adopted and their most important findings on quality
research results have been reported about the application of prediction in terms of accuracy. Finally, a brief discussion on challenges
non-destructive technologies working at higher wavelengths than the and future research needs is also mentioned. Each cited work is sup
visible spectrum to quality assessment of fresh-cut products through ported by Table 1, where objectives, classification methods and the main
packaging material (Lara et al., 2013; Giovenzana et al., 2014). Few results are reported, and by Table 2, in which the main software and
researchers have reported interesting results about the use of CVS-CI technical aspects of CVS-CI adopted for the image analysis of fruit and
through packaging. A powerful improvement of applications of vegetables are described. Image analysis and classification can be per
CVS-CIs comes from the integration of machine learning techniques in formed by several kind of software available either as open source or
several steps of processing. These methodologies can increase the flex upon paying a fee. In Table 2, the literature is listed according to the
ibility of systems while simplifying their tuning and deployment. software used for the elaboration and classification steps of image
Moreover, in several fields machine learning paradigms (such as analysis. Matlab is the most used one probably because it offers to
deep-learning architectures) are providing unparalleled performances customers specific image processing applications of easy and rapid
widening the applicability of CVS-CI to solving relevant problems in insight. Finally, Table S1 shows additional information about method
food evaluation. ology, results evaluation and validation tests developed by the authors
To the best of our knowledge, a detailed summarization of recent cited in the review.
developments and applications of CVS-CI to evaluate external and in
ternal quality of fruit and vegetables is not available. The present 2. CVS-CI to estimate the visual appearance of fresh and
comprehensive review aims to report the most interesting researches packaged fruit and vegetables on the basis of external defect and
carried out on computer vision technology based on conventional im colour change
aging (CVS-CI) for non-invasive analysis of fresh produce. In detail, 37
articles published in the last decade on important international journals Recent literature reports several works carried out on the visual
and on proceedings of international congresses were reviewed and dis quality assessment of fruit and vegetables by the use of CVS-CI. Auto
cussed. The literature research was focussed on image analysis by con matic external inspection of fresh produce by the use of innovative CVSs,
ventional CVS for the assessment of quality traits of fresh and packaged on the basis of the morphological traits, is widely used in industrial lines
5
M. Palumbo et al. Postharvest Biology and Technology 200 (2023) 112332
Table 2
Technical aspects of computer vision system based on conventional imaging (CVS-CI) applied to fruit and vegetables by digital cameras using as software Matlab,
Microsoft Visual Basic, Phyton, R, or C+ +.
Background Lights Features extracted Reference
Matlab
Black 4 fluorescent lamps (60 cm long) at 35 cm above Colour: Mendoza and Aguilera
the sample and at an angle of 45◦ with the sample - brown spots as a percentage of the total area (% BSA) (2004)
- number of brown spots per cm2 of surface (NBS/cm2)
Ambient light Colour (a* and hue angle) Amodio et al. (2011a)
4 Fluorescent 15 W lamps - colour (lightness, redness, and yellowness component) Amodio et al. (2011b)
- browned area (Lbrown, abrown, bbrown)
- browning percentage
8 halogen lamps (divided along two rows placed Colour (L, a, b, Chroma, hue angle) Pace et al. (2011)
at the two sides of the imaged area) at a 45◦ angle
Eight halogen lamps (placed in two lines) Colour features (L, a, b, hue angle) Pace et al. (2013)
8 halogen lamps (divided along two rows placed Colour features: Pace et al. (2015)
at the two sides of the imaged area) at a 45◦ angle - white (W) and red (R) pixels, and their subdivision into two components (W1 or
R1, dark) and (W2 or R2, light)
- average value of L* , a* and b*
- percentage of L* , a* and b* with respect to the product’s surface
Darkness Colour features (R, G, B, L, a and b) Cho et al. (2016)
Eight halogen lamps (placed in two lines) Colour features Cavallo et al. (2017)
Four lamps that contained two fluorescent tubes Computation of: Sahu and Potdar (2017)
- image moment
- shoulder fullness
Eight halogen lamps (divided along two rows Colour features (complete histogram in the a* b* plane) Cavallo et al. (2018)
placed at the two sides of the imaged area)
Natural light - Colour Kaur et al. (2018)
- texture
- size
8 halogen lamps (divided along two rows placed 2 set of features: Cavallo et al. (2019)
at the two sides of the imaged area) at a 45◦ angle - by statistical measures evaluated over the whole foreground on the channels in the
CIELAB colour space
- by a centroid-based colour segmentation algorithm
4 white light LED strips Colour features (from L, a and b values citrus colour index, CCI) Hadimani and Mittal
(2019)
Strip LED Colour features (gray mean and standard deviation of each channel R, G and B) Hongyang et al. (2021)
LED, Fluorescent and Tungsten lamps 5 different types of features: Sabzi et al. (2020)
- 336 colour
- 80 texture
- 6 histogram
- 10 moments
- 20 shape
8 halogen lamps, placed along two rows at the Colour features (complete histogram in the a* b* plane of the foreground pixels) Palumbo et al. (2021)
two sides of the imaged area, at 45◦
2 led ramps with 120 lamps Colour features Palumbo et al. (2022a)
Eight halogen lamps (placed in two lines) Colour features (complete histogram of the foreground pixels, expressing the Palumbo et al. (2022b)
number of occurrences of each colour in the a* b* plane)
Dark Ambient light - 8 colour features (mean and standard deviation of RGB and hue (H) component) Moallem et al. (2017)
- 5 textural features (average of gray level cooccurrence matrices, contrast,
correlation, energy, homogeneity and entropy)
- 3 geometric features (defect ratio, defect perimeter and defect medial axes length)
Ambient light - Colour (mean, standard deviation, and range) Ireri et al. (2019)
- texture (contrast, correlation, energy, homogeneity and entropy)
- shape (centroid)
White 4 fluorescent lamps at 40 cm above the samples - Colour features (mean, variance, skewness, kurtosis) Prabha and Kumar (2015)
with 45◦ angle - size features (area, perimiter, lenght and width)
Ambient light Textural features Ali and Thai (2017)
Two types of halogen lamps 50 W 17 colour features (R, G, B, r, g, b, H, S, I, and 8 equal areas of hue range (0–360◦ , Nadafzadeh et al. (2018)
each of these areas was 45◦ )
40 LED lights 15 features (minimum, mean, maximum, standard deviation, mode and correlation Fashi et al. (2019)
coefficient, range of variations, skewness, elongation, entropy, variance, median,
harmonic mean, covariance, and contrast)
Ambient light - 6 colour features from RGB space Xia et al. (2016)
- 28 colour features from other colour spaces (HIS, NTSC, YCbCr, HSV and CMY)
- 18 colour features from arithmetically calculated images (the ratio and
normalized operations between the red, green, and blue channel images)
Blue LED lights - 18 colour features (R, G, B, H, S, V, L, a, b, Rδ, Gδ, Bδ, Hδ, Sδ, Vδ, Lδ, aδ, bδ) Huang et al. (2019)
Not defined Ambient light - 6 standard features (the mean, standard deviation and mean first and second Leiva-Valenzuela and
derivative along the boundaries of the region of interest) Aguilera (2013)
- 16 Fourier descriptors
- 413 local binary pattern features
- 469 Gabor features (frequency and orientation of textures for the differentiation of
classes)
- 27 features from the grayscale images
- 7 Hu moments
- 4 Flusser and Suk moments
(continued on next page)
6
M. Palumbo et al. Postharvest Biology and Technology 200 (2023) 112332
Table 2 (continued )
Background Lights Features extracted Reference
Matlab
4 LED and 2 fluorescent lamps 4 standard colour features (mean, maximum, minimum, and standard deviation) Mohammadi et al. (2015)
Fluorescent lights - colour features (colour mean, standard deviation and skewness) Arakeri and Lakshmana
- texture features (contrast, correlation, energy and homogeneity) (2016)
Fluorescent lamps - 4 colour features (normalized L* values, hue angle, browning index, and total Subhashree et al. (2017)
colour change).
- 3 textural features (entropy, contrast and homogeneity)
Incandescent lamps 4 features (fruit surface colour, size, surface defect and shape) Chen et al. (2018)
Light system - 6 color features (hue, intensity, saturation, L, a, and b) Huang et al. (2018)
- 4 texture features (consistency, angular second moment, entropy, and correlation)
Microsoft Visual Basic
Dark Ambient light - Size Akter and Rahman (2017)
- shape
- colour
- percentage of diseased area
PYTHON
Black Two strips of light-emitting diodes (LEDs) Colour features (RGB, HSV and HSL colour spaces) (Basak et al., 2022)
R
Not defined Cold white light Colour features (Li et al., 2022)
Cþ þ
Blue Six white fluorescent tubes Colour features Rong et al. (2017)
for grading and classification (Chopde et al., 2017). In food sorting features and the evaluation of colour change in whole and fresh-cut fruit
systems, it has been shown that CVS-CI can judge marketability and and vegetables.
edibility of foods on the basis of shape, size and the presence of external
defects on their surface, both at harvest and postharvest (Bagri and 3. Detection of external defects
Johari, 2015; Sa et al., 2016; Jana et al., 2017). Automated grading
speeds up the processing time, reducing errors occurring during manual 3.1. Fruit
classification. On the contrary, grading fruit and vegetables through
other appearance features, such as bruises, colour changes, rottenness or CVS-CI proved to be a useful tool for the improvement of blueberries
other defects (i.e. stem-end and calyx confusion with other types of quality during the postharvest storage (Leiva-Valenzuela and Aguilera,
defects) is not yet efficient and is not still applied on automatic work 2013). A pattern recognition methodology is developed to sort blueberry
lines in industry (Leiva-Valenzuela and Aguilera, 2013). into four classes and two fruit orientations through the extraction and
The colour of both raw and processed fruits is one of the most selection of visible features from images (Table 1). This methodology
important and the first perceived quality hint that determines con could also improve common commercial sorting systems whose classi
sumers’ acceptance. In food technology, colour is traditionally expressed fication is a grading only on the overall colour of berries, without
in the CIELab colour space that is device independent and in which recognizing specific defects on the fruit surface (Table S1). Among
distances between colours are closer to the evaluations made by human several classifiers applied, the best ones were the linear discriminant
vision. Measurements are usually performed using colorimeters. analysis (LDA) and the support vector machine (SVM), that successfully
Although these instruments are non-destructive, easy to use and to distinguish the blueberry’s orientation in 96.8% of the cases and were
calibrate, they need to touch the samples and they normally measure able to classify berries with good accuracy (Table 1). The proposed
only very small areas (limited to a few mm2), being sensitive to the statistical pattern recognition methodology is a useful and promising
choice of sampling points. For these reasons their use at industrial level single-step tool for in-line sorting and grading of blueberries with regard
and for in-line monitoring is limited (Goñi and Salvadori, 2017). to different defects and orientations.
In the last decades, digital image analysis has been largely used for An automated fruit grading system was designed to detect defects on
the assessment of food colour. An important difference between a CVS- the surface of mango and apples with the aim to achieve greater effi
CI and a traditional colorimeter is the degree of dependency on the ciency for manual classification (Table 1). In detail, for image acquisi
environment condition: colorimeters, touching the product, can reduce tion, a rotating desk with a 12 V motor was used to acquire images of the
to a minimum the influence of the light in the environment. CVS-CIs, entire surface of the products (Table 2). A graphical user interface (GUI)
that operate at variable distances from the observed scenes, need a was specifically designed to allow operators to interact with the imaging
more careful calibration in order to account for the influence of the light system and to simplify and speed up the grading operation. The GUI
present in the environment on the measurement of colours. Another showed the total number and the positions of defects (Table S1) (Ali and
relevant difference is the amount of colour information that the two Thai, 2017).
instruments can acquire and analyse. Indeed, a CVS-CI extracts quanti It was possible to effectively detect the defects on orange surface by
tative colour information with the resolution of the pixel of the digital the development of a new segmentation algorithm without additional
image whose geometrical correspondent region can be controlled by lightness correction on images acquired by the CVS-CI (Table 2) (Rong
properly choosing sensor resolution and optics. A CVS-CI can examine et al., 2017). For this purpose, about 1200 images of randomly selected
the whole visible surface of a product and provide distinct colour in samples were analysed to classify several types of surface defects
formation for every single pixel. Therefore, it can provide global sta (Table 1). The main steps of the image processing consisted of back
tistical colour information about the whole product but also ground removal, image binarization with a sliding comparison window,
semantically separate different regions of the product, describe the image subtraction, hole filling and identification and removal of
spatial distribution of colour and even analyse separately the different stem-end pixels (Table S1). The proposed method was able to correctly
objects present in the scene (Wu and Sun, 2013). detect 97% of defective oranges and had a performance rate of 91.9% in
Many research applications have demonstrated the speed, accuracy, the individual defect detection (Table 1).
effectiveness and consistency of CVS-CI in the assessment of colour Quality control plays an important role in apple-based industries.
7
M. Palumbo et al. Postharvest Biology and Technology 200 (2023) 112332
Recently, an apple grading computer vision algorithm was introduced grades related to tissue decay, from good to bad (Table 1). KNN, SVM,
into an ordinary machine vision system (Table 2) (Moallem et al., 2017). and back-propagation artificial neural network (BPNN) were applied
This algorithm firstly detected stem-end, by a combination of morpho and compared in predicting spinach freshness (Table S1). BPNN and
logical methods and a classifier based on mahalanobis distance, and KNN models achieved the same classification accuracy as reported in
calyx area, by applying K-means clustering on the Cb component in Table 1. Furthermore, the study also applied the E-nose technology to
YCbCr colour space, separating them from defective regions; then de obtain odour information of the spinach samples and a multisensory
fects segmentation was achieved using a multi-layer perceptron (MLP) data fusion approach based on machine vision and E-nose data to detect
neural network. Two types of classification were done analysing colour, the freshness of spinach during storage. As result, the BPNN model based
textural and geometric features (Table 2): two apple categories were on this multisensory data fusion widely improved the accuracy of
distinguished in the first ranking and 3 categories in the second one spinach freshness detection (93.75%).
(Table 1). A good accuracy was achieved for the calyx detection algo
rithm both when the stem-end was inside the image (94%) and when it 4. Evaluation of colour changes
was outside (81%), even if it was lower than previous one. Moreover,
SVM classifier was the best one for the recognition of 2 categories and 3 4.1. Fruit
quality ranks (Table 1).
A novel methodology, through the construction of three algorithms, Banana is a fruit widely addressed by image processing applications.
was proposed for automatic defect identification and maturity detection Its peel colour, considered the main quality parameter for traders and
of mango by the use of CVS-CI for practical application on industrial consumers, shows relevant changes from harvest to the end of the arti
lines (Table S1) (Sahu and Potdar, 2017). As results, the fruits were ficial ripening process. A typical over-ripe process, called senescent
graded into two classes (defected or not-defected and mature or spotting, consists of brown spots on the external peel. The banana
immature) based on the quality ratio and maturity, respectively. If the maturity stage is usually classified by a 7-point scale, from green to
value of the quality ratio was greater than the threshold value, the fruit yellow and a CVS-CI was implemented to grade the 7 ripening stages of
was rotten, while if the value was less than the threshold value, the fruit bananas using colour and image texture information, as well as the
was good. The proposed algorithm efficiently and accurately determined development of brown spots, extracted by image analysis (Table S1)
the quality of mango (Table 1) and a similar approach was used also for (Mendoza and Aguilera, 2004). Results showed that a simple classifi
maturity detection (Table S1). cation technique, based on L* , a* and b* parameters, brown area per
Finally, as for CVS-CI applications on fruits, a new automatic centage and contrast, was able to identify the 7 ripening stages of
approach was proposed for manual orange classification (Chen et al., bananas with an accuracy of 98% (Table 1).
2018). Oranges were divided into 3 groups (Table 1) for each of the A recent research showed that the correct physiological maturity
characteristic indexes: colour, size, defect and shape. Finally, the stage of bananas at harvest is as important as the assessment of the
extracted characteristic parameters were studied and classified by using ripening stage after harvest to ensure the quality during the storage in
a back propagation (BP) neural network (Table 1). The grading accuracy the ripening chambers (Prabha and Kumar, 2015). With this objective a
was very high (Table 1) demonstrating how this classification system, CVS-CI, described in Table 2, was implemented to extract colour and size
based on image analysis, could realize a real-time automatic oranges features of fresh banana fruit images to classify 3 main categories before
grading detection reducing the labour loss and improving the efficiency harvest, finding that the mean colour intensity and area features were
compared to the traditional method. more significant than other features for the assessment of maturity
stages (Table 1). The model developed in this study could be used to
3.2. Vegetables and fruit vegetables identify an automatic detection system for banana maturity assessment
directly in the field.
As regard the CVS-CI implementation on vegetables, an automatic, The accuracy of CVS-CI in comparison with the use of traditional
accurate and low-cost system was developed for quality inspection of colorimeters and sensory tests was demonstrated to evaluate banana
tomatoes in the agroindustry using image processing techniques to peel browning during storage (Cho et al., 2016). The browning degree
classify the fruit vegetables in terms of ripeness and defects (Arakeri and was estimated using the changes in RGB colour value and the changes in
Lakshmana, 2016). Several colour and textural features were extracted the CIE L* a* b* colour values (Table 2). The methodology demon
from the acquired images and the final classification was carried out strated that the G and CIE L* a* b* values obtained by image analysis
through an artificial neural network (ANN) (Table 1). The system was showed the highest correlation coefficient with a sensory test, while CIE
able to separate defective from non-defective and ripe from unripe to L* a* b* values measured by the colorimeter also showed high correla
matoes with very good performance (Table 1). tion coefficients, but comparatively lower than those detected by image
An eggplant grading system was developed to grade healthy or un analysis (Table 1).
healthy samples using an automated CVS-CI, described in Table 2 (Akter Additionally, CVS-CI was also used to study the physical properties of
and Rahman, 2017). The diseased areas of the eggplants were persimmon fruits (Mohammadi et al., 2015). The authors developed a
segmented by Otsu and binary transformation methods (Table 1 and S1). sorting algorithm to categorize fruits into 3 commercially maturity
For the final classification, performed by K-nearest neighbour (KNN) stages (Table 1) on the base of external colour able to eliminate the noise
method, features on the size, shape, colour and percentage of diseased and binarize the acquired image, remove the black stains and finally
areas were considered, distinguishing four categories of the product extract 4 standard colour features (Table S1). The best results for
(Table 1). Because of little problems of shadow and lightning created grading persimmon fruits were reached by the quadratic discriminant
during the image acquisition, the system was able to grade the eggplants analysis (QDA) classifier with an overall accuracy of 90.24% (Table 1).
with an accuracy of 88% and further investigation are being studied to Maturity indices are also important for trade regulation and mar
improve the final results. keting strategy. In order to provide marketing flexibility and to guar
antee the consumers’ acceptance of plums, an implementation of image
3.3. Fresh-cut fruit and vegetables processing techniques, based on JPEG images, was developed to classify
fruits into 4 categories related to different maturity stages (Kaur et al.,
Regarding the fresh-cut products, a CVS-CI was used to detect the 2018). The multi-attribute decision making (MADM) theory was used
freshness of fresh-cut spinaches stored in a plastic bag at 4 ◦ C for 12 for making decisions on the base of colour and textural features to
d (Huang et al., 2019). As they are extremely perishable, for both achieve the highest accuracy (Table 1). As relevant result, colour was the
physiological and microbial degradation, spinaches were divided into 4 dominant factor for grading plums according to maturity stage and the
8
M. Palumbo et al. Postharvest Biology and Technology 200 (2023) 112332
error percentage between the manual and the calculated values of length colour changes on the external surface of products characterized by a not
and width was lower than 2.4%, as reported in Table 1. uniform colour. An image analysis technique was applied to study colour
CVS-CI was recently applied to implement a method to classify changes and the visual appearance of “Borlotto” beans, commonly used
pomegranate arils on the base of colour and size using image processing in ready-to-use soup ingredients (Amodio et al., 2011a). These beans are
and artificial intelligence (Fashi et al., 2019). These features represent characterized by bright tiny red spots on the surface that, since may
the most important elements to consider for grading pomegranate undergo severe discolouration depending on the atmosphere composi
especially when consumed as a ready-to-eat product. In this study, the tion within the package, represent a good indicator of packaging per
arils were graded by experts based on size and appearance into 3 cate formance. Given the small and irregular dimensions of these spots, the
gories and the ANN, adaptive neuro fuzzy inference system (ANFIS) and traditional measurement approach may not be helpful. A new algorithm
response surface methodology (RSM) classifiers were compared (Table 1 was defined and was able to objectively measure both spots and back
and S1). The ANN model showed a higher accuracy than that obtained ground colours in order to study the degradation kinetics over time,
for the ANFIS and RSM model (Table 1), representing a valid method to reporting positive and negative regressions between the appearance
assess the quality of arils through image analysis. scores evaluated sensorially and the a* and hue angle values, respec
The image processing and machine learning techniques may be tively (Table 1).
combined to reduce the human intervention for the configuration and
setting of algorithms with the purpose of increasing their final perfor 4.3. Fresh-cut fruit and vegetables
mances. A simplified CVS-CI was applied to two white table grape cul
tivars (Victoria and Italia) during cold storage (at 5 and 10 ◦ C) for the As for fresh-cut produce, an algorithm for rapid colour detection by
assessment of 5 different quality levels (QL) (Table 1) (Cavallo et al., image analysis based on RGB scale from the whole surface and the
2019). In the proposed approach, a random forest classifier was trained browned areas of fresh-cut artichokes was developed in order to
and tested and a large set of potential features were fed into the system, objectively compare the efficacy of anti-browning agents, in addition to
leaving to the machine learning tool the task of selecting how many and a sensory evaluation through a subjective scale (from 5, excellent, to 1,
which attributes were best suitable for the classification task (Table S1). inedible) (Amodio et al., 2011b). For all colour parameters (L*, a*, b*)
The system was able to separate the highest QL5 and QL4 from the obtained by CVS-CI a standard regression was performed to find possible
others with high accuracies on cultivar Italia and Victoria, without correlations with sensory evaluation (Table S1). Significant correlations
significant differences between the compared storage temperatures between all colour features were observed and L* measured on the
(Table 1), providing to be suitable for a real context where often tem whole surfaces showed the highest correlation coefficients, while L* and
perature regimes are not constant. Moreover, the obtained results b* measured on the brown areas showed lower, but still significant,
demonstrated the possibility to develop classification models specific for correlations (Table 1). On the other hand, the a* value was the least
single cultivars by the use of CVS-CI for continuous monitoring of the correlated parameter with an appearance score (showing negative cor
quality along the supply chain. relation values).
An implementation of the image analysis through CVS-CI was pro
4.2. Vegetables and fruit vegetables posed to obtain a global evaluation of colour of fresh-cut nectarine from
a standard RGB image acquired by a 3 CCD digital camera (Pace et al.,
In a recent work, a low-cost tomato grading machine vision system 2011). A novelty was the development of a polynomial colour trans
based on RGB images and machine learning technology was introduced formation to correct the images before any further processing
to develop an efficient and accurate calyx detection algorithm and (Table S1). The correlation between the visual appearance, scored by a
classification approach to sort tomatoes into four different grading 5–1 sensory scale, and the colour parameters b* and Chroma measured
categories (Table 1) (Ireri et al., 2019). Colour features, grey level by the computer vision system was higher than that obtained using a
texture features (gray-level co-occurrence matrices, GLCM) and shape colorimeter (Table 1), demonstrating how CVS-CI was more effective
features were analysed comparing three recognition models: support than the standard colorimetric method and how it can be suitable for
vector machine - radial basis function (SVM-RBF), ANN and random real-time application in the processing industry.
forest (Table 1). Calyx detection was performed with an accuracy of 95% As additional task, CVS-CI can automatically set parameters which
by histogram thresholding based on the mean red and green values of were normally identified by operators and completely automate some of
the regions of interest. Defected regions were detected by an SVM-RBF the image processing steps, adapting itself to a specific analysed product
classifier based on L*a*b* colour features, achieving a very high over (Pace et al., 2015). For this purpose, an innovative CVS-CI, characterized
all accuracy. The SVM-RBF outperformed all the compared models with by an automatic procedure for the extraction of colour features, was
the highest accuracy in grading 4 tomato categories based on colour and developed to evaluate the quality levels (QLs) changes occurring on two
texture features (Table 1). The proposed machine vision system can be varieties of fresh-cut radicchio (Correlli and Botticelli) during the cold
used for quality inspection, marketing, and packaging of tomatoes and storage in air or in modified atmosphere packaging (Pace et al., 2015).
to detect defects along the supply chain of tomato management. Brightness and white colour of fresh-cut radicchio are considered
Recently, an innovative CVS-CI was used to automatically select the freshness indicators and, among these, colour changes after processing
relevant features for the classification of rocket leaves with reduced and during storage occur on the white parts that tend to become brown.
human intervention applying a random forest model (Palumbo et al., Results of this research proved that the average of the component
2021). In detail, three colour correction models (white balance, linear a* over the white pixels and the percentage of light white pixels over the
correction, and polynomial correction) were evaluated and compared to whole visible surface allowed the best discrimination of the QL in three
identify the best solution for consistent colour measurement in terms of quality classes whose true value was verified on the base of sensory
classification performance. The proposed CVS-CI was applied to fresh evaluation (Table 1).
rocket leaves produced with two different cultural practices, with the The CVS-CI, for image acquisition of fresh-cut apples, was used to
aim to (i) objectively attribute 5 quality levels (QL) during storage at non-destructively study browning through colour and textural features
10 ◦ C and (ii) non-destructively discriminate the cultivation approach, (Subhashree et al., 2017). Colour and textural features selected by im
using the colour information extracted from the digital images. The ages acquired through a CVS-CI (Table 2) showed good results to detect
CVS-CI achieved an accuracy of about 95% in QL attribution and of browning on apple slices by image analysis (Table 1), although further
about 65–70% in the discrimination of the cultivation approach research is needed to better understand the colour kinetics in this
(Table 1). fresh-cut fruit.
Finally, image analysis could represent an effective way to measure Moreover, colour changes occurring on fresh-cut potatoes during
9
M. Palumbo et al. Postharvest Biology and Technology 200 (2023) 112332
cold storage were examined by the use of CVS-CI to perform quantitative antioxidants and include pink, red, blue and purple pigments known as
analysis at different stages of browning (Hongyang et al., 2021). Several anthocyanins. During the ripening of some fruit and vegetables, a colour
methods were built to classify and predict the browning features of change in the superficial tissue from green to purple is due to an alter
fresh-cut potatoes (Table S1). The SVM was able to predict the storage ation of pigment concentration (i.e. accumulation of anthocyanins). This
life of the product with an accuracy of 96%, while the partial least colour modification suggests the use of colour analysis to detect the
squares regression (PLSR) model showed an accuracy of 86%, 96% and antioxidant activity and the total phenols content, of these fruit and
99% for L* , a* and b* , respectively (Table 1). vegetables (Lou et al., 2012).
Regarding the use of image analysis by CVS-CI for the quality Many applications were reported in recent research works about the
assessment of fresh-cut products through plastic bags, an automatic use of CVS-CI to estimate inner quality parameters of fruit and
CVS-CI applied to fresh-cut iceberg lettuce was recently developed vegetables.
(Cavallo et al., 2018). The main purpose was to achieve a careful se
lection of the bag area that was meaningful to have similar performances 6. Evaluation of chemical-physical attributes
of the analysis for both packaged and unpackaged products. In detail,
during cold storage, fresh-cut lettuce leaves were placed in poly 6.1. Fruit
propylene bags and sensory evaluated according to a 5-point rating
scale. Images were then acquired by a CVS-CI for each fresh-cut lettuce An experimental study was carried out to establish quantitative
sample both with and without the bag. All acquired images were sub models between features measured by the CVS-CI and soluble solids
sequently segmented using a convolutional neural network (CNN), content and pH of Kyoto grapes (Xia et al., 2016). Several colour features
identifying and selecting only pixels belonging to the fresh-cut lettuce were extracted from the mean and standard deviation of the pixel values
and unaffected by artifacts induced by the illumination. The classifica considering each RGB channel (Table 2). The root mean square error of
tion was made by a 3-nearest neighbours classifier applied on a cross-validation (RMSECV) was calculated on the base of all the pre
two-dimensional colour histogram on the a* and b* components in the diction residuals. Furthermore, the models were also evaluated by the
CIE L*a*b* space (Table 2). The performance loss due to the presence of root mean square error of calibration (RMSEC) (Table 1). As result,
packaging was not relevant (Table 1) and the proposed model was also RMSEC values were 0.097 and 0.773 on average for the SSC and pH
successfully tested on commercial bags of lamb’s lettuce. The CNN prediction, respectively, while the RMSECV values were from 0.099 to
segmentation method was able to separate the opaque and affected re 0.100 for pH and from 0.788 to 0.808 for the SSC prediction (Table 1).
gions from the product inside the commercial bag, showing its flexibility In a previously reported study, the CVS-CI applied on plums also
for the potential application along the supply chain. showed a good correlation between fruit acidity (expressed as total
soluble solids content) and the mean intensity of the green colour (R2 =
5. CVS-CI to estimate chemical-physical attributes in fresh and 0.997) and the R/G ratio (R2 = 0.846) (Table 1) (Kaur et al., 2018).
packaged fruit and vegetables The possibility of predicting polyphenol oxidase (PPO) and peroxi
dase (POD) enzyme activity on the banana peel browning during storage
Besides external attributes, CVSs-CI have already been used also for using the image analysis by CVS-CI was recently demonstrated (Nadaf
the assessment of inner quality, providing a valid contribution to the zadeh et al., 2018). Several colour features were extracted from the
evaluation of fruit and vegetables. The retention of green colour is images acquired, as detailed in Table 2, and two equations were ob
closely related to the total chlorophyll content, which degrades with the tained by the use of a genetic programming modelling in order to predict
postharvest senescence, inducing a yellowing of the visible surface of the PPO and POD activities during the browning process of banana peel. As
leaves. Besides total chlorophyll, ammonia content is also considered an a result, there were no significant differences between predicted and
indicator of freshness, when is detected in low amounts in the vegetal measured PPO and POD enzymatic activity, demonstrating the high
tissues (Tudela et al., 2013) and high concentrations of this component performance of the image analysis and of the derived models (Table 1).
may cause tissue damage with visible effects on the overall quality of the The integration of CVS-CI and colorimetric sensor array (CSA) may
product (darkening and browning of leaves) (Mastrandrea et al., 2016; be used for a non-destructive methodology for a rapid and accurate
Amodio et al., 2018). So, the possibility to predict these internal traits by evaluation of mango quality, hardness and total soluble solids (TSS)
CVS-CI on the base of colour changes occurring during storage allows us (Huang et al., 2018). In detail, three grades were defined by experts by
to use them as markers and objective parameters for quality level sensory evaluation while hardness and TSS of mango samples were
assessment of leafy vegetables. detected by both conventional and non-destructive methods. All data
The development of fruit colour during ripening is considered a were elaborated using PCA to reduce dimensionality, support vector
maturity index and when its intensity increases, the ripening attributes classification (SVC) model for qualitative grading of fruits, and support
also improve. Total soluble solids mainly contribute to the sweetness of vector regression (SVR) model to elaborate the relationship between the
fruits during ripening. It has been demonstrated that as the fruit colour conventional and CVS-CI data (Table S1). The SVC model showed high
development enhances, the total soluble solids content increases pro accuracy for the training and prediction sets, respectively, to classify
portionately (Kaur et al., 2018). On the contrary, the trend of the fruit mango fruits into 3 grades, while the SVR allowed good prediction of
acidity is reversed: it decreases with the enhancement of the colour of hardness and TSS (Table 1).
the fruit during ripening. Additionally, the CVS-CI was proposed to estimate the maturity
During the storage period, enzymatic browning generally occurs on index of Kinnow mandarins which was strictly related to internal
the cut surfaces of fresh-cut products for the interaction of phenolic physiology and external peel colour changes (Hadimani and Mittal,
components with polyphenol oxidase and peroxidase, resulting in brown 2019). Mandarins are usually graded from homogeneous deep green to
pigments. The notable effect of this physiological disorder on the visual typical deep orange/red colour by a numerical value called citrus colour
appearance of fruits could allow to predict the enzymatic activity index (CCI). In this work, a polynomial transformation based on camera
responsible for the browning by the use of image analysis (Cho et al., characterization method was proposed to reduce the complexity of
2016). colour space transformation from RGB to L*a*b* . Physicochemical pa
Antioxidant activity and total phenols content are two other chem rameters of TSS and pH of mandarins were measured and correlated
ical parameters closely related to the nutritional quality of fruit and with the changes in peel colour detected by CVS-CI (Table S1). Measured
vegetables. It has been demonstrated that total phenols may be the main physicochemical properties had a good correlation with the change in
contributor to the antioxidant activity of many fruit and vegetables (Li peel colour (Table 1) and a high R2 = 0.97 was achieved with partial
et al., 2012). Among the phenolic components, flavonoids are potent least square regression (PLSR), showing confidence in the developed
10
M. Palumbo et al. Postharvest Biology and Technology 200 (2023) 112332
CVS-CI based method. rocket leaves and colour values in the RGB and L*a*b* colour spaces
On oranges, an automatic CVS-CI able to assess the pH value from (Table 2) with a better performance compared to the SPAD-meter (i.e. R2
external visible-range peel images was presented (Sabzi et al., 2020). = 0.90 vs. R2 = 0.79) (Table 1).
The proposed system was applied to 3 varieties of orange (Bam, Blood
and Thomson): after image acquisition, 6 colour and texture discrimi 6.3. Fresh-cut fruit and vegetables
nant features for each orange sample were selected among the extracted
ones through the use of the hybrid artificial neural network - particle Recently, the combination of image processing by CVS-CI and the
swarm optimization (ANN-PSO) and, finally, estimated orange pH random forests model was proposed to solve the classification problem
regression values were provided by the multilayer perceptron (MLP) (visual quality level assessment) and the regression problem (prediction
neural network architecture (Table 1). Results showed that the proposed of senescence indicators as chlorophyll and ammonia content) on
method was able to predict the orange pH value from the external peel unpackaged and packaged rocket leaves (Table 2 and S1) (Palumbo
images with high accuracy (Table 1). The average time spent to estimate et al., 2022b). The experiment showed a not significant performance loss
pH with the proposed non-destructive method was 0.42 s, which might on packaged products with respect to unpackaged ones (Table 1). The
be reasonable for in-line industrial applications. same CVS-CI, exploiting the machine learning components, was able to
Discrimination of the ripening stage (half-red or red) of strawberries build effective models for both the problems just by changing the
(cv. Candonga) harvested at three different times by CVS-CI image training data. Moreover, three partial least square models were built to
analysis was satisfying (Palumbo et al., 2022a). After several subsequent predict the visual quality level of fresh-cut rocket leaves using as pre
steps of segmentation (Table S1), the pixel count of red and green area of dictors the total chlorophyll and the ammonia contents obtained by
images was performed to calculate their relative percentage area. destructive methods, CVS-CI through packaging and CVS-CI without
Among the chemical indicators of ripening, titratable acidity was well packaging. The predictors obtained by CVS on samples with and without
correlated to image analysis (Pearson correlation coefficient of about 1), package provide better performances compared to the ones estimated by
providing a suitable indicator for fast and non-destructive evaluation of destructive analysis (R2 in validation of 0.70, 0.77 and 0.80, respec
the ripening stage in strawberries (Table 1). tively) (Table 1).
A CVS-CI was developed also to evaluate the application of image These latter results confirm the ability of the CVS-CI to evaluate the
processing technique for total soluble solids and pH prediction on product also through the packaging, even working only on the regions of
strawberries (Basak et al., 2022). The channels of RGB, HSV and HSL the image that provide meaningful colour information about the prod
colour spaces were used as input variables for developing MLR and uct’s surface.
support vector machine regression (SVM-R) models (Table 1). The
SVM-R model, working on features in the HSV colour space, performed 7. Conclusions and future perspectives
better than MLR model for total soluble solids and pH prediction
(Table 1). High demands and attention of modern consumers on the quality of
Finally, a very interesting application of image analysis provides an their purchases require detailed information about external and internal
innovative and smart technology to predict the shelf life and the quality quality of fresh products. Research is working on rapid, precise and low-
of kiwifruit during cold storage by acquiring and calculating RGB value cost techniques for food analysis. Among the innovative techniques
extracted from photos taken by a smartphone camera (Table 2) (Li et al., qualified for this task, image analysis by computer vision systems (CVS)
2022). Results reported that the R to B ratio values (Central R/B) was have proved to be effective and suitable for application at industrial
negatively correlated with titratable acidity and vitamin C contents and level. Different types of CVS have been developed for quality inspection
firmness and positively correlated with soluble solids content, total of fruit and vegetables, based on conventional (visible), multispectral
soluble sugars and total plate counts (Table 1). The study demonstrated and hyperspectral imaging. The solutions based on conventional imag
how the extracted the RGB values with mobile phones may provide a ing (CVS-CI) use RGB colour cameras that are sensible to the visible
rapid and consistent evaluation of postharvest quality of kiwifruit, wavelengths of the electromagnetic spectrum. They are already widely
which can be more suitable for distributors and consumers to rapidly used to measure colour, size, shape, and to detect some external defects.
assess the quality and storage time of the product. Moreover, their potentiality in determining internal characteristics
related to the product nutritional quality have been proved, providing
6.2. Vegetables and fruit vegetables added value to any actor of the supply chain of fruit and vegetables.
Unfortunately, as limitation, the only internal quality traits that this
The relationships between antioxidant activity (AA) and total phe technology is able to evaluate are strictly related to external visible
nols (TP) with colour features of a local landrace of pigmented carrot, changes in terms of physical alteration or colour modification.
characterised by different colours (from yellow to purple), were This review has presented the most recent and significant de
explored by a CVS-CI able to predict the AA or TP contents (Pace et al., velopments in computer vision technology for the quality evaluation of
2013). The obtained regression models were able to successfully esti fresh fruit and vegetables. A large part of the presented studies used
mate the AA and the TP contents as their predicted levels showed a very CVS-CI to sort and grade fruit and vegetables on the basis of external
good correlation with the measured AA and TP content when both in defects and colour changes, improving the productivity at industrial
ternal and external parts of carrots were considered. When data included level and providing products with high and homogeneous quality to
only the internal part (with a not uniform pigmentation) lower deter consumers.
mination coefficients were obtained (i.e. R2 = 0.93 for AA and R2 = 0.86 Online sorting systems along the whole supply chain could allow the
for TP) (Table 1). inspection of large quantities of fruit and vegetables in a short time and
Moreover, a new contactless and non-invasive approach was pro could provide a good prediction of the external and internal quality of
posed for the prediction of total chlorophyll content of fresh rocket the batch of inspected products. Moreover, the continuous and pervasive
leaves through the use of CVS-CI in order to replace the common non- monitoring of quality products in addition of their physiological state
destructive SPAD-meter, a commercially available device that detects during postharvest, can enable longer transportation times in order to
the chlorophyll content in local small areas by touching the leaf, reduce food losses along the entire supply chain. There are still many
providing the information related to few square millimetres of leaf open challenges to realize quality assessment in a faster and accurate
surface (Cavallo et al., 2017). The classification was performed by way in industrial line and during postharvest phase: stem-end calyx
random forest regression, a machine learning technique suitable to recognition, accounting for the distribution of lightness on curve sur
model the correlation between total chlorophyll content of fresh-cut faces, whole surface inspection, new and simpler statistical
11
M. Palumbo et al. Postharvest Biology and Technology 200 (2023) 112332
12
M. Palumbo et al. Postharvest Biology and Technology 200 (2023) 112332
Fashi, M., Naderloo, L., Javadikia, H., 2019. The relationship between the appearance of Narendra, V.G., Amithkumar, V.G., 2019. An intelligent computer vision system for
pomegranate fruit and color and size of arils based on image processing. Postharvest vegetables and fruits quality inspection using soft computing techniques. Agric. Eng.
Biol. Technol. 154, 52–57. https://doi.org/10.1016/j.postharvbio.2019.04.017. Int.: CIGR J. 21 (3), 171–178.
Francis, G.A., Gallone, A., Nychas, G.J., Sofos, J.N., Colelli, G., Amodio, M.L., Spano, G., Pace, B., Cefola, M., Renna, F., Attolico, G., 2011. Relationship between visual
2012. Factors affecting quality and safety of fresh-cut produce. Crit. Rev. Food Sci. appearance and browning as evaluated by image analysis and chemical traits in
Nutr. 52 (7), 595–610. https://doi.org/10.1080/10408398.2010.503685. fresh-cut nectarines. Postharvest Biol. Technol. 61 (2–3), 178–183. https://doi.org/
François, I.M., Wins, H., Buysens, S., Godts, C., Van Pee, E., Nicolaï, B., De Proft, M., 10.1016/j.postharvbio.2011.03.005.
2008. Predicting sensory attributes of different chicory hybrids using physico- Pace, B., Cavallo, D.P., Cefola, M., Colella, R., Attolico, G., 2015. Adaptive self-
chemical measurements and visible/near infrared spectroscopy. Postharvest Biol. configuring computer vision system for quality evaluation of fresh-cut radicchio.
Technol. 49 (3), 366–373. https://doi.org/10.1016/j.postharvbio.2008.02.011. Innov. Food Sci. Emerg. Technol. 32, 200–207. https://doi.org/10.1016/j.
Giovenzana, V., Beghi, R., Buratti, S., Civelli, R., Guidetti, R., 2014. Monitoring of fresh- ifset.2015.10.001.
cut Valerianella locusta Laterr. shelf life by electronic nose and VIS–NIR Pace, B., Cefola, M., Renna, F., Renna, M., Serio, F., Attolico, G., 2013. Multiple
spectroscopy. Talanta 120, 368–375. https://doi.org/10.1016/j. regression models and computer vision systems to predict antioxidant activity and
talanta.2013.12.014. total phenols in pigmented carrots. J. Food Eng. 117 (1), 74–81. https://doi.org/
Goñi, S.M., Salvadori, V.O., 2017. Color measurement: comparison of colorimeter vs. 10.1016/j.jfoodeng.2013.02.005.
computer vision system. J. Food Meas. Charact. 11 (2), 538–547. https://doi.org/ Palumbo, M., Pace, B., Cefola, M., Montesano, F.F., Colelli, G., Attolico, G., 2022b. Non-
10.1007/s11694-016-9421-1. destructive and contactless estimation of chlorophyll and ammonia contents in
Hadimani, L., Mittal, N., 2019. Development of a computer vision system to estimate the packaged fresh-cut rocket leaves by a Computer Vision System. Postharvest Biol.
colour indices of Kinnow mandarins. J. Food Sci. Technol. 56 (4), 2305–2311. Technol. 189, 111910 https://doi.org/10.1016/j.postharvbio.2022.111910.
https://doi.org/10.1007/s13197-019-03641-9. Palumbo, M., Pace, B., Cefola, M., Montesano, F.F., Serio, F., Colelli, G., Attolico, G.,
Hongyang, T., Daming, H., Xingyi, H., Aheto, J.H., Yi, R., Yu, W., Ji, L., Shuai, N., 2021. Self-configuring CVS to discriminate rocket leaves according to cultivation
Mengqi, X., 2021. Detection of browning of fresh-cut potato chips based on machine practices and to correctly attribute visual quality level. Agronomy 11 (7), 1353.
vision and electronic nose. J. Food Process Eng. 44 (3), e13631 https://doi.org/ https://doi.org/10.3390/agronomy11071353.
10.1111/jfpe.13631. Palumbo, M., Cozzolino, R., Laurino, C., Malorni, L., Picariello, G., Siano, F.,
Huang, X., Lv, R., Wang, S., Aheto, J.H., Dai, C., 2018. Integration of computer vision Stocchero, M., Cefola, M., Corvino, A., Romaniello, R., Pace, B., 2022a. Rapid and
and colorimetric sensor array for nondestructive detection of mango quality. J. Food non-destructive techniques for the discrimination of ripening stages in candonga
Process Eng. 41 (8), e12873 https://doi.org/10.1111/jfpe.12873. strawberries. Foods 11 (11), 1534. https://doi.org/10.3390/foods11111534.
Huang, X., Yu, S., Xu, H., Aheto, J.H., Bonah, E., Ma, M., Wu, M., Zhang, X., 2019. Rapid Patel, K.K., Kar, A., Jha, S.N., Khan, M.A., 2012. Machine vision system: a tool for quality
and nondestructive detection of freshness quality of postharvest spinaches based on inspection of food and agricultural products. J. Food Sci. Technol. 49 (2), 123–141.
machine vision and electronic nose. J. Food Saf. 39 (6), e12708 https://doi.org/ https://doi.org/10.1007/s13197-011-0321-4.
10.1111/jfs.12708. Prabha, D.S., Kumar, J.S., 2015. Assessment of banana fruit maturity by image
Ireri, D., Belal, E., Okinda, C., Makange, N., Ji, C., 2019. A computer vision system for processing technique. J. Food Sci. Technol. 52 (3), 1316–1327. https://doi.org/
defect discrimination and grading in tomatoes using machine learning and image 10.1007/s13197-013-1188-3.
processing. AIA 2, 28–37. https://doi.org/10.1016/j.aiia.2019.06.001. Rong, D., Rao, X., Ying, Y., 2017. Computer vision detection of surface defect on oranges
Jana, S., Basak, S., Parekh, R., 2017. Automatic fruit recognition from natural images by means of a sliding comparison window local segmentation algorithm. Comp.
using color and texture features. In: IEEE (Ed.) Devices for Integrated Circuit Electron. Agric. 137, 59–68. https://doi.org/10.1016/j.compag.2017.02.027.
(DevIC). IEEE, Kalyani, India, pp. 620–624). IEEE. 10.1109/DEVIC.2017.8074025. Sa, I., Ge, Z., Dayoub, F., Upcroft, B., Perez, T., McCool, C., 2016. Deepfruits: a fruit
Kader, A.A., Rolle, R.S., 2004. The role of post-harvest management in assuring the detection system using deep neural networks. Sensors 16 (8), 1222. https://doi.org/
quality and safety of horticultural produce. Vol 152. Food & Agriculture Org. 10.3390/s16081222.
Kaur, H., Sawhney, B.K., Jawandha, S.K., 2018. Evaluation of plum fruit maturity by Sabzi, S., Javadikia, H., Arribas, J.I., 2020. A three-variety automatic and non-intrusive
image processing techniques. J. Food Sci. Technol. 55 (8), 3008–3015. https://doi. computer vision system for the estimation of orange fruit pH value. Measurement
org/10.1007/s13197-018-3220-0. 152, 107298. https://doi.org/10.1016/j.measurement.2019.107298.
Lara, M.A., Lleó, L., Diezma-Iglesias, B., Roger, J.M., Ruiz-Altisent, M., 2013. Monitoring Sahu, D., Potdar, R.M., 2017. Defect identification and maturity detection of mango
spinach shelf-life with hyperspectral image through packaging films. J. Food Eng. fruits using image analysis. Am. J. Artif. Intell. 1 (1), 5–14. https://doi.org/
119 (2), 353–361. https://doi.org/10.1016/j.jfoodeng.2013.06.005. 10.11648/j.ajai.20170101.12.
Leiva-Valenzuela, G.A., Aguilera, J.M., 2013. Automatic detection of orientation and Salinas-Hernández, R.M., González-Aguilar, G.A., Tiznado-Hernández, M.E., 2015.
diseases in blueberries using image analysis to improve their postharvest storage Utilization of physicochemical variables developed from changes in sensory
quality. Food Control 33 (1), 166–173. https://doi.org/10.1016/j. attributes and consumer acceptability to predict the shelf life of fresh-cut mango
foodcont.2013.02.025. fruit. J. Food Sci. Technol. 52 (1), 63–77. https://doi.org/10.1007/s13197-013-
Li, H., Deng, Z., Zhu, H., Hu, C., Liu, R., Young, J.C., Tsao, R., 2012. Highly pigmented 0992-0.
vegetables: Anthocyanin compositions and their role in antioxidant activities. Food Sendin, K., Manley, M., Williams, P.J., 2018. Classification of white maize defects with
Res. Int. 46 (1), 250–259. https://doi.org/10.1016/j.foodres.2011.12.014. multispectral imaging. Food Chem. 243, 311–318. https://doi.org/10.1016/j.
Li, H., Lv, S., Feng, L., Peng, P., Hu, L., Liu, Z., Hati, S., Bimal, C., Mo, H., 2022. foodchem.2017.09.133.
Smartphone-Based Image Analysis for Rapid Evaluation of Kiwifruit Quality during Stavang, J.A., Freitag, S., Foito, A., Verrall, S., Heide, O.M., Stewart, D., Sønsteby, A.,
Cold Storage. Foods 11 (14), 2113. https://doi.org/10.3390/foods11142113. 2015. Raspberry fruit quality changes during ripening and storage as assessed by
Li, J., Chen, L., Huang, W., 2018. Detection of early bruises on peaches (Amygdalus colour, sensory evaluation and chemical analyses. Sci. Hortic. 195, 216–225.
persica L.) using hyperspectral imaging coupled with improved watershed https://doi.org/10.1016/j.scienta.2015.08.045.
segmentation algorithm. Postharvest Biol. Technol. 135, 104–113. https://doi.org/ Subhashree, S.N., Sunoj, S., Xue, J., Bora, G.C., 2017. Quantification of browning in
10.1016/j.postharvbio.2017.09.007. apples using colour and textural features by image analysis. Food Qual. Saf. 1 (3),
Lorente, D., Aleixos, N., Gómez-Sanchis, J.U.A.N., Cubero, S., García-Navarrete, O.L., 221–226. https://doi.org/10.1093/fqsafe/fyx021.
Blasco, J., 2012. Recent advances and applications of hyperspectral imaging for fruit Sung, J., Suh, J.H., Chambers, A.H., Crane, J., Wang, Y., 2019. Relationship between
and vegetable quality assessment. Food Bioproc. Tech. 5 (4), 1121–1142. https:// sensory attributes and chemical composition of different mango cultivars. J. Agric.
doi.org/10.1007/s11947-011-0725-1. Food Chem. 67 (18), 5177–5188. https://doi.org/10.1021/acs.jafc.9b01018.
Lou, H., Hu, Y., Zhang, L., Sun, P., Lu, H., 2012. Nondestructive evaluation of the Tudela, J.A., Marín, A., Garrido, Y., Cantwell, M., Medina-Martínez, M.S., Gil, M.I., 2013.
changes of total flavonoid, total phenols, ABTS and DPPH radical scavenging Off-odour development in modified atmosphere packaged baby spinach is an
activities, and sugars during mulberry (Morus alba L.) fruits development by unresolved problem. Postharvest Biol. Technol. 75, 75–85. https://doi.org/10.1016/
chlorophyll fluorescence and RGB intensity values. LWT-Food Sci. Tech. 47 (1), j.postharvbio.2012.08.006.
19–24. https://doi.org/10.1016/j.lwt.2012.01.008. Watada, A.E., Ko, N.P., Minott, D.A., 1996. Factors affecting quality of fresh-cut
Mendoza, F., Aguilera, J.M., 2004. Application of image analysis for classification of horticultural products. Postharvest Biol. Technol. 9 (2), 115–125. https://doi.org/
ripening bananas. J. Food Sci. 69 (9), E471–E477. https://doi.org/10.1111/j.1365- 10.1016/S0925-5214(96)00041-5.
2621.2004.tb09932.x. Wu, D., Sun, D.W., 2013. Colour measurements by computer vision for food quality
Moallem, P., Serajoddin, A., Pourghassem, H., 2017. Computer vision-based apple control–a review. Trends Food Sci. Technol. 29 (1), 5–20. https://doi.org/10.1016/j.
grading for golden delicious apples based on surface features. Inf. Process. Agric. 4 tifs.2012.08.004.
(1), 33–40. https://doi.org/10.1016/j.inpa.2016.10.003. Xia, Z., Wu, D., Nie, P., He, Y., 2016. Non-invasive measurement of soluble solid content
Mohammadi, V., Kheiralipour, K., Ghasemi-Varnamkhasti, M., 2015. Detecting maturity and pH in Kyoho grapes using a computer vision technique. Anal. Methods 8 (15),
of persimmon fruit based on image processing technique. Sci. Hortic. 184, 123–128. 3242–3248. https://doi.org/10.1039/c5ay02694.
https://doi.org/10.1016/j.scienta.2014.12.037. Zhang, B., Huang, W., Li, J., Zhao, C., Fan, S., Wu, J., Liu, C., 2014. Principles,
Nadafzadeh, M., Mehdizadeh, S.A., Soltanikazemi, M., 2018. Development of computer developments and applications of computer vision for external quality inspection of
vision system to predict peroxidase and polyphenol oxidase enzymes to evaluate the fruits and vegetables: a review. Food Res. Int. 62, 326–343. https://doi.org/
process of banana peel browning using genetic programming modeling. Sci. Hortic. 10.1016/j.foodres.2014.03.012.
231, 201–209. https://doi.org/10.1016/j.scienta.2017.12.047.
13