sensors

Review
Road Condition Monitoring Using Smart Sensing and Artificial
Intelligence: A Review
Eshta Ranyal 1,2 , Ayan Sadhu 1, * and Kamal Jain 2

1 Department of Civil and Environmental Engineering, Western University, London, ON N6A 3K7, Canada;
[email protected]
2 Department of Civil Engineering, IIT Roorkee, Roorkee 247667, India; [email protected]
* Correspondence: [email protected]

Abstract: Road condition monitoring (RCM) has been a demanding strategic research area in main-
taining a large network of transport infrastructures. With advancements in computer vision and
data mining techniques along with high computing resources, several innovative pavement distress
evaluation systems have been developed in recent years. The majority of these technologies employ
next-generation distributed sensors and vision-based artificial intelligence (AI) methodologies to
evaluate, classify and localize pavement distresses using the measured data. This paper presents an
exhaustive and systematic literature review of these technologies in RCM that have been published
from 2017–2022 by utilizing next-generation sensors, including contact and noncontact measurements.
The various methodologies and innovative contributions of the existing literature reviewed in this
paper, together with their limitations, promise a futuristic insight for researchers and transport
infrastructure owners. The decisive role played by smart sensors and data acquisition platforms, such
as smartphones, drones, vehicles integrated with non-intrusive sensors, such as RGB, and thermal



cameras, lasers and GPR sensors in the performance of the system are also highlighted. In addition to


sensing, a discussion on the prevalent challenges in the development of AI technologies as well as
Citation: Ranyal, E.; Sadhu, A.; Jain,
potential areas for further exploration paves the way for an all-inclusive and well-directed futuristic
K. Road Condition Monitoring Using
research on RCM.
Smart Sensing and Artificial
Intelligence: A Review. Sensors 2022,
22, 3044. https://doi.org/
Keywords: road condition monitoring; pavement distress evaluation; pavement monitoring; smart
10.3390/s22083044 sensors; AI; deep learning; machine learning

Academic Editors: Nicola Giaquinto,

Francesco Adamo and
Maurizio Spadavecchia
1. Introduction
Received: 27 March 2022 Roadway infrastructure is susceptible to structural degradation on account of material
Accepted: 12 April 2022 deterioration primarily caused by heavy traffic, harsh weather conditions, aging, poor
Published: 15 April 2022 construction quality, and lack of appropriate maintenance. The success of a road transport
Publisher’s Note: MDPI stays neutral system is inherently dependent on the riding quality and comfort level of the commuters,
with regard to jurisdictional claims in for which timely detection of faults and ensuing maintenance is of utmost importance. The
published maps and institutional affil- lack of rapid and automated road monitoring methods is a major contributing factor to
iations. pavement damages and is a common issue in many countries, necessitating huge efforts in
this direction. Most developed countries witness a significant portion of aged pavements,
while some suffer from deteriorated road networks due to extreme climatic events. On the
other hand, developing countries suffer from pavement distress due to a radical increase
Copyright: © 2022 by the authors. in the number of vehicles and intensive usage. The manual observation and detection
Licensee MDPI, Basel, Switzerland.
methods, though in practice, are cumbersome and time-consuming, involving high costs.
This article is an open access article
During recent decades, researchers and engineers have prioritized safety and reduction in
distributed under the terms and
inspection costs in the scope of developing an intelligent and smart roadway infrastructure
conditions of the Creative Commons
system to expedite maintenance and corrective action. Therefore, the significance of cost-
Attribution (CC BY) license (https://
effective road condition monitoring (RCM) systems to warrant long-standing structural
creativecommons.org/licenses/by/
4.0/).
integrity and safety levels has been emphasized globally. Emerging RCM methods have

Sensors 2022, 22, 3044. https://doi.org/10.3390/s22083044 https://www.mdpi.com/journal/sensors

 Sensors 2022, 22, x FOR PEER REVIEW 2 of 28
Sensors 2022, 22, x FOR PEER REVIEW 2 of 28
Sensors 22, 3044
2022, 2022,
Sensors 22, x FOR PEER REVIEW 2 of 227of 28
Sensors 2022, 22, x FOR PEER REVIEW 2 of 28

RCM methods have the potential to rationalize periodic inspections and minimize the
RCM
the potential methods
RCMassociated
methods have
to rationalize
have the potential
theperiodic
potential to
to rationalize
inspections periodic
and minimize
rationalize periodic inspections
the costs
inspections and
and minimize
associated with the
minimize the
costs
RCM methods have with failing pavement
the potential structures.
to rationalize This paper
periodic aims
inspections to provide a systematic
and minimize the
costs
failing
costs associated
pavement
associated with
structures.
with failing
This
failing pavement
paper
pavement structures.
aims to provide
structures. This
This a paper aims
systematic
paper aims to
and
to provide a systematic
comprehensive
provide a systematic
and
costs comprehensive
associated withreview review
failingofof the
pavement latest advancement
structures. Thisin in AI‐assisted
paper aims to RCM RCM technology.
provide a systematic
and
andofcomprehensive
review the latest failure
comprehensive advancement
review ofinthe
the latest
latest advancement
AI-assisted RCM technology.
advancement in AI‐assisted
AI‐assisted RCM technology.
technology.
Pavement
and comprehensive can be
review of attributed
the latest to diverse
advancement factors,
in such as vehicle
AI‐assisted RCM loading, envi‐
technology.
Pavement
Pavement
Pavement failure
failure
failure can
can be
be attributed
canconstruction
be attributed
attributed to
to diversediverse
to and factors,
factors,
diverse suchsuch
factors, as
such as vehicle
vehicle
as of
vehicle loading,
loading, envi-
loading, envi‐
envi‐
ronmental
Pavement conditions,
failure can be attributed quality to and maintenance.
diverse factors, Some
such as the
vehicle most commonly
loading, envi‐
ronmental
ronmental
ronmental conditions,
conditions,
conditions, construction
construction
construction qualityquality
and
quality and maintenance.
maintenance.
maintenance. Some Some
Someof theof
of the
most
the most commonly
commonly
most commonly
occurring
ronmental pavement
conditions, distresses
construction are quality
listed inand Table 1. Vehicle Some
maintenance. loading of andmost
the environmental
commonly
occurring
occurring
occurring pavement
pavement
pavement distresses
distresses are listed
are listed
distresses in
in Table
in Table 1.
1. Vehicle
1. Vehicle loadingloading
and and environmental
environmental
conditions
occurring majorly contribute
pavement distresses toare
are
listed
surface
listed in
Table
disintegration,
Table 1.
Vehicle
Vehicle
loading
resulting
loading
and
in potholes,
and
environmental
which be‐
environmental
conditions
conditions
conditionsmajorly majorly
majorly contribute
contribute
contribute to
to surface
to surface disintegration,
disintegration,
surface resulting
disintegration, resulting
in potholes,
resulting in
in potholes,
which
potholes, which
become
which be‐
be‐
come severe
conditions as
majorlydisintegration
contribute moves
to surfacedown the layers
disintegration, with time.
resulting Cracking,
in potholes, which
whichis an‐
be‐
come
severe as severe
come common as
disintegration
severe asform disintegration
moves
disintegration downmoves
moves the down
layers
down the layers
with time.
the layers with time.
Cracking,
with time. Cracking,
which
Cracking, is which
another
which is an‐
is can
an‐
other
come severe as of pavement
disintegration moves failure,
down occurs
the in different
layers with shapes
time. and sizes,
Cracking, which
which is an‐
other
common common
other form
common form
formtoof
of pavement pavement
failure, occurs
ofvehicle
pavement failure, occurs
in different in different
inshapes and shapes
sizes, andwhich sizes,
canwhich
again can
again be attributed
other common formtoof pavementload,failure,
causing
failure,
occurs
damage
occurs
different
in areas shapes
in different with anand
shapes
sizes,
inadequate
and sizes,
which can
packing
which
again
be attributed
again be
be attributed
to vehicle
attributed to vehicle
load,
vehicle load,
causing
load, causing
damage
causing damage
in areas
damage in areas
with
in an
areas with an
inadequate
with an inadequate
packing
inadequate of can
packing
packing
of basebelayers and/or topoor drainage, or poorly constructed paving lane joints or packing
volume
baseagain
of base
oflayers
attributed
base layers
and/or
layers and/or
poor vehicle
poor
drainage, load,
drainage, causing
or
or poorly
or poorly damage in areas
constructed
constructed paving with
pavinglanean inadequate
lane
jointsjoints or
or volume volume
changes
of base in theand/or
layers subgrade.
and/or
poorThere
poor
drainage,
are other
drainage, or
poorly
formsconstructed
poorly of distressespaving
constructed that come
paving
lane
lane
joints
under
joints
or volume
pavement
or volume
changes
changes in
changes in
the
in the
the subgrade.
subgrade.
subgrade. ThereThere
are
There are
other
are other
forms
other forms
of
forms of distresses
distresses
of that
distresses that
come
that come
comeunder under pavement
pavement
under pavement
distortions,
changessuch in thesuch as shoving, rutting, corrugation, depressions, and bumps and sags,
distortions,
distortions,
distortions, as subgrade.
such
such as
shoving, Thererutting,
shoving,
rutting,
asparallel
shoving,
are other
corrugation,
rutting,
forms
corrugation, of distresses
depressions,
corrugation, and that
depressions,
depressions, bumps come
and
and and under
bumps pavement
and
sags, which
bumps sags,
and pave‐
sags,
which are
distortions, observed
such as shoving, to the direction
rutting, of traffic
corrugation, and occur
depressions, due andto a weakened
bumps and sags,
are observed
which areparallel to the direction of traffic andof occur due to aoccur
weakenedtopavement layer,
ment are observed
whichlayer, observed
poor
parallel
parallel
bonding
to
to the
betweenthe direction
direction
layers, or
traffic
traffic and
ofexcessive and occur due
moisture due
in to aasubgrade.
the
weakened
weakened pave‐
pave‐
Surface
which
poorment are
bonding betweenobserved parallel
layers, or to the
excessive direction
moistureof traffic and occur
in the subgrade. due to
Surface a weakened
deterioration pave‐
ment layer,
layer, poor
deterioration poor bonding
bonding between
between layers,
in thelayers,
or excessive
orraveling, moisture
moisture in
excessiveweathering the
inand subgrade.
the bleeding
subgrade.owingSurface
Surface
ment
is also layer, poor
observed
deterioration inis
is
also
the
also
observed
bonding
form of between
observed raveling,
in the
form of
layers, or raveling,
weathering
form of excessive moisture
and bleeding
weathering inand
owing the subgrade.
to bleeding
aging, vehicle
owing
to
Surfaceto
deterioration
aging, vehicle is also
loading,observed
poor in
mix, the form
poor of raveling,
compaction, weathering
dust, and and
moisture. bleeding
Often, owing
sealantsto
deterioration
loading,
aging, poor
vehicle is also
mix,loading, observed
poor compaction,
poor in
mix, the form of raveling,
dust, compaction,
poor and moisture. weathering
Often,
dust, and and
sealants
moisture. bleeding
and owing
patches,
Often, sealantsto
aging,
and vehicle
patches, loading,
which are poor mix,
preventative poor compaction,
maintenance dust,
actions, and
appear moisture.
as Often,
distresses sealants
over the
whichaging,
and vehiclewhich
are preventative loading, poor mix,actions,
maintenance poor compaction, dust, and
appear as distresses overmoisture.
the course Often, sealants
of time.
and patches,
coursepatches,
of time.which
are
are preventative
preventative maintenance
maintenance actions,
actions, appear
appear as as distresses
distresses over
over the
the
and
coursepatches, which are preventative maintenance actions, appear as distresses over the
course of of time.
time.
Tablecourse
1. Typesof oftime.
common pavement distress (Miller and Bellinger 2014).
Table 1. Types of common pavement distress (Miller and Bellinger 2014).
Table 1. Types of common pavement distress (Miller and Bellinger 2014).
Table 1. Types of common pavement distress (Miller and Bellinger 2014).
Distress Type Table 1. Types of commonSeverity
pavement distress (Miller and Bellinger 2014).
Levels Causes
Distress Type Severity Levels Causes
Distress
Distress Type
Type Severity
Severity Levels
Levels Causes
Causes
Distress Type CrackCrack
Severity Levels Causes
Crack
Crack
Crack
LowLow
(no(noor or
fewfewinterconnections)
interconnections)
Low
Low (no
(no or few
or(forming
few interconnections)
interconnections)
Alligator
Alligator Moderate
LowModerate
(no or (forming
few a apattern)
pattern)
interconnections) Repeated traffic
Repeated trafficloadings
loadings
Alligator
Alligator Moderate
Moderate (forming
(forming aa pattern)
pattern) Repeated
Repeated traffic
traffic loadings
loadings
Alligator High
High
Moderate (severely
(severely
(forming spalled)
spalled)
a pattern) Repeated traffic loadings
High
High (severely
(severely spalled)
spalled)
High (severely spalled)

Low: width ≤ 6 mm
Low:
Low: width
Low: ≤≤ 66≤mm
width
width 6 mm
mm
Block Moderate:
Low: width6 ≤mm
66mmmm< width ≤ 1919mm
< width≤≤
Repeated traffic loadings
Block
Block
Block Moderate: 6
Moderate:
Moderate: 6 mm
mm << width
width ≤ 19
19 mm
mm
mm Repeated
Repeated traffic
Repeated trafficloadings
loadings
traffic loadings
Block High: width
Moderate:
High: 6 mm
width > 19
> 19<mm
width
mm ≤ 19 mm Repeated traffic loadings
High: width > 19
High: width > 19 mm mm
High: width > 19 mm

Low: width ≤ 6 mm
Low:
Low: width
width ≤ 66 mm
Longitudinal Moderate:
Low:
Low: 6 ≤≤mm
width
width
mm
6≤mm
6<mmwidth ≤ 19 mm Poor joint construction or location
Longitudinal
Longitudinal
Longitudinal
Moderate:
Moderate: 6
6
Moderate:
mm
mm
6 mm
<< width
width ≤ 19
< width≤≤1919mm
mm
mm
Poor
Poor joint
joint
Poor
construction
jointconstruction
or location
construction or location
location
Longitudinal High: width
Moderate: 6 mm> 19 <mm width ≤ 19 mm Poor joint construction or location
High:
High: width
High: width
width > 19
> 19
> 19 mm
mmmm
High: width > 19 mm

Low: width ≤ 6 mm
Low:
Low: width
width6 ≤≤mm66 mm
mm
Transverse Moderate:
Low: width ≤ 6≤mm< width ≤ 19 mm Axial loading or temperature change
Transverse
Transverse Moderate:
Low:
Moderate: 6
width
6 mm
mm
6<mm
width
width ≤≤ 19
<mm 19 mm
mm Axial
Axial loading
loading or
or temperature
temperature change
change
Transverse
Transverse High: width
Moderate: 6 mm
Moderate: >6 19
mm <mm
width
< width≤≤1919mm
mm Axial loading
Axial loadingorortemperature change
temperature change
High:
High: width
width >
> 19
19 mm
High:
High: width
width > 19> 19 mm
mm
Pothole and Patch
Pothole
Pothole and
and Patch
Patch
Pothole and Patch
Sensors 2022, 22, 3044 3 of 27

Table 1. Cont.
Sensors 2022, 22, x FOR PEER REVIEW 3 of 28
Sensors 2022, 22, x FOR PEER REVIEW 3 of 28
Sensors 2022, 22,Distress Type
x FOR PEER REVIEW Severity Levels Causes 3 of 28
Sensors 2022, 22, x FOR PEER REVIEW 3 of 28
Sensors 2022, 22, x FOR PEER REVIEW Pothole and Patch 3 of 28
Sensors 2022, 22, x FOR PEER REVIEW 3 of 28
Sensors 2022, 22, x FOR PEER REVIEW 3 of 28

Minimum
Minimum diameter
diameterof 150
150mm
Minimum diameter ofof150 mm
mm
Minimum
Low: depth
Low: diameter
depth ≤ 25≤ mm
25 mmof 150 mm
Pothole
Pothole Minimum
Low: depthdiameter
≤ 25 mmof 150 mm Water
Waterinfiltration
infiltrationinto
into cracks
cracks
Pothole Minimum diameter <ofdepth Water infiltration into cracks
Pothole
Low: depth25
Moderate:
Moderate:
Minimum
Low: depth
Moderate:
≤25
diameter
25 ≤ mm
25mm
mm
25
mm
mm <mm of 150
< depth
depth150≤≤mm
≤5050mm
mm
50 mm
mm
Water infiltration into cracks
Pothole Minimum
High:
Low: depth
Moderate:
High: depth diameter
depth
25≤≤>mm >
25 50
50 mm mm of 150 mm
< depth ≤ 50 mm Water infiltration into cracks
Pothole Low:
High: depth
Moderate:
depth 25≤>mm25 mm
50mm mm < depth ≤ 50 mm Water
Pothole Low: depth
Moderate: 25 25
mm << depth Water infiltration
infiltration into
into cracks
cracks
Pothole High: depth
Moderate:
High: depth 25 > mm50 mm depth ≤≤ 5050 mm
mm Water infiltration into cracks
Moderate:
High: depth 25>>mm 50 mm
50 mm < depth ≤ 50 mm
High: depth > 50 mm
High: depth
Surface ≥ to 0.1 > 50mmm 2
Surface ≥ to≥0.1
Surface to m22m2
0.1
Surface
Low: ≥ to 0.1
rutting ≤ 6mmm
Patch Surface
Low: ≥rutting
to 0.1
rutting
Low: ≤ 6≤ mmm622 mm Lack of preventative maintenance
Patch
Patch Surface
Low:
Moderate:≥ to
rutting6 0.1
≤
mm 6 mmm
< rutting ≤ 19 mm Lack of of
Lack preventative
preventative maintenance
maintenance
Surface
Low: ≥ to 0.1
rutting
Moderate: ≤6 mm
6mmmm
<22 rutting
< rutting≤≤1919mm
≥ to66 0.1
mm
Patch Moderate: mm Lack of preventative maintenance
Patch Surface
Low:
High: rutting
Moderate:
rutting ≤
mm> 619mm
< rutting
mm ≤ 19 mm Lack of preventative maintenance
Patch High:
Low:
High: rutting
rutting
Moderate:
rutting 6 mm >
≤>619mm <19 mm
rutting
mm ≤ 19 mm Lack
Patch Low: rutting
Moderate: 66 mm≤>619 mm
<<mm rutting Lack of
of preventative
preventative maintenance
maintenance
Patch High:
High:
rutting
Moderate:
rutting mm > 19 rutting ≤≤ 19
mm 19 mm
mm Lack of preventative maintenance
Moderate:
High: rutting6 mm
Surface > 19 < mm rutting ≤ 19 mm
Deformations
High: rutting
Surface > 19 mm
Deformations
High:Surface
rutting
Surface Deformations
> 19 mm
Deformations
Surface Deformations
Surface
Surface Deformations
Deformations
Surface Deformations
Rutting Not applicable Heavy load, wheel path, poor mix
Rutting Not applicable Heavy load, wheel path, poor mix
Rutting
Rutting NotNot
applicable
applicable Heavy load,
Heavy load,wheel
wheelpath,
path, poor mix
poor mix
Rutting Not applicable Heavy load, wheel path, poor mix
Rutting
Rutting Not
Not applicable
applicable Heavy
Heavy load,
load, wheel
wheel path,
path, poor
poor mix
mix
Rutting Not applicable Heavy load, wheel path, poor mix

Plastic movement of pavement surface,

Shoving Not applicable Plastic movement of pavement surface,
Shoving Not applicable Plastic
weak movement
subgrade,
Plastic movement of
ofpavement
improper surface,
rolling
pavement
Shoving Not applicable Plastic
weak movement
subgrade, of pavement
improper surface,
rolling
Shoving
Shoving NotNot
applicable
applicable Plastic
weak movement
subgrade,
surface, weak of pavement
improper
subgrade, surface,
rolling
Shoving Not Plastic
weak movement
subgrade, of pavement
improper surface,
rolling
Shoving Not applicable
applicable Plastic movement
improper
weak rolling of pavement surface,
Shoving Not applicable weak subgrade,
subgrade, improper
improper rolling
rolling
weak subgrade, improper rolling
Surface Defects
Surface Defects
Surface Defects
Surface
Surface Defects
Defects
Surface
Surface Defects
Defects
Surface Defects
Bleeding Not applicable Excessive binder, low air‐void content
Bleeding Not applicable Excessive binder, low air‐void content
Bleeding Not applicable Excessive binder, low air‐void content
Bleeding Not applicable Excessive binder, low air‐void content
Bleeding
Bleeding
Bleeding Not applicable
NotNot applicable
applicable Excessive
Excessivebinder,
Excessive binder, low
binder, air‐void
low air-void
low content
air‐voidcontent
content
Bleeding Not applicable Excessive binder, low air‐void content

Weather, installation, aggregate

Ravelling Not applicable Weather, installation, aggregate
Ravelling Not applicable Weather, installation,
separation, mechanicalaggregate
dislodging
Ravelling Not applicable Weather, installation,
separation, mechanicalaggregate
dislodging
Ravelling Not applicable Weather,
separation,installation,
mechanical aggregate
dislodging
Ravelling Not applicable Weather,
separation,
Weather,installation,
mechanical
installation, aggregate
dislodging
aggregate
Ravelling
Ravelling NotNot
applicable
applicable Weather,
separation,installation,
mechanical aggregate
Ravelling Not applicable separation, mechanical dislodging
separation,mechanical dislodging
dislodging
separation, mechanical dislodging

Polished Aggregate with insufficient flakiness and

Polished Not applicable Aggregate with insufficient flakiness and
Polished
Aggregate Not applicable Aggregatefriction
excessive with insufficient
on the roadflakiness and
by vehicles
Polished
Aggregate Not applicable Aggregatefriction
excessive with insufficient
on the roadflakiness and
by vehicles
Polished
Aggregate Not applicable Aggregate
excessive with
frictioninsufficient
on the road flakiness
by and
vehicles
Polished
Aggregate Not Aggregate
excessive with
frictioninsufficient
on the road flakiness
by and
vehicles
Not applicable
Polished Aggregate
Aggregate with insufficient
with insufficient flakiness
flakiness and
Polished
Aggregate applicable excessive friction on
Aggregate
Aggregate
NotNot
applicable
applicable excessive friction
and excessive on theonroad
the
friction road by
by vehicles
the road vehicles
Aggregate excessive friction on the road by vehicles
by vehicles
A typical RCM method relies on the data acquisition system that involves 1D time‐
A typical RCM method relies on the data acquisition system that involves 1D time‐
seriesAdata, 2DRCM
typical visualmethod
data orrelies
3D depth
on thedata.
data Low‐cost
acquisition sensors,
systemsuch
that as accelerometers,
involves 1D time‐
seriesAdata, 2DRCM
typical visualmethod
data orrelies
3D depth
on thedata.
data Low‐cost
acquisition sensors,
systemsuch
that as accelerometers,
involves 1D time‐
gyroscopes,
seriesA data, magnetometers,
2D
typical visual
RCM data
method orand
3D
reliesGPS,
depth
on are
the often
data.
data employed
Low‐cost
acquisition to measure
sensors,
system such
that motion,
as rotation,
accelerometers,
involves 1D time‐
Adata,
seriesA typical
gyroscopes, 2DRCM
visualmethod
magnetometers,
data orrelies
and
3D on the
GPS,
depth aredata
often
data. acquisition
employed
Low‐cost system that as
to measure
sensors, such involves
motion, 1D time‐
rotation,
accelerometers,
velocity,typical
gyroscopes, RCM
orientation, method
and
magnetometers, relies
location, on the
which data
are acquisition
one‐dimensionalsystem
in that involves
nature. The 1D time‐
commonly
series
series data,
data,
velocity,
gyroscopes,
2D
2D visual
orientation,
data
visualand orand
or 3D GPS,
3D
datalocation,
magnetometers, and
depth
depth are
which
GPS,
often
data.
data.
areare
often
employed
Low‐cost
Low‐cost
employed
to measure
sensors,
sensors,
one‐dimensional to insuch asmotion,
such as
nature.
measure
rotation,
accelerometers,
accelerometers,
The commonly
motion, rotation,
series
used data,
sensors
velocity,
gyroscopes, 2Dforvisual
orientation, datalocation,
acquiring
and
magnetometers, orand
3D depth
digital images
which data.
are Low‐cost
the sensors,
high‐quality,
one‐dimensional such asmotion,
accelerometers,
high‐resolution
in nature. RGB
The commonlysen‐
gyroscopes,
used sensors
velocity, magnetometers,
for acquiring
orientation, and GPS,
digital
and location, GPS, are
images
which
often
areare
often employed
the employed
to
to measure
high‐quality,
one‐dimensional measure
nature.motion,
high‐resolution
in
rotation,
rotation,
RGB
The commonly sen‐
gyroscopes,
sors,
used which
sensors
velocity, magnetometers,
facilitate
for in
acquiring
orientation, and and
acquiring
digital
location, GPS, are
multiscale
images
which often
are the employed
low‐level to
high‐quality,
one‐dimensional measure
as well asin
high‐levelmotion,
high‐resolution
nature. The rotation,
feature
RGBextrac‐
commonlysen‐
velocity,
sors,
used whichorientation,
sensorsfacilitate
for and
in
acquiring location,
acquiring
digital which arelow‐level
multiscale
images one‐dimensional
the as well asin
high‐quality, nature. The
high‐level commonly
feature
high‐resolution RGBextrac‐
sen‐
velocity,
tions.
sors,
used which orientation,
However,
sensors for these
facilitate and
in location,
images
acquiring
acquiring digital which are
aremultiscale
images one‐dimensional
two‐dimensional
low‐level
are in as
nature
well as
the high‐quality, in
and nature.
high‐level The
cannot be commonly
used
feature
high‐resolution RGBfor the
extrac‐
sen‐
Sensors 2022, 22, 3044 4 of 27

A typical RCM method relies on the data acquisition system that involves 1D time-
series data, 2D visual data or 3D depth data. Low-cost sensors, such as accelerometers,
gyroscopes, magnetometers, and GPS, are often employed to measure motion, rotation,
velocity, orientation, and location, which are one-dimensional in nature. The commonly
used sensors for acquiring digital images are the high-quality, high-resolution RGB sensors,
which facilitate in acquiring multiscale low-level as well as high-level feature extractions.
However, these images are two-dimensional in nature and cannot be used for the character-
istic evaluation of the features. The depth consideration of such images allows multi-fold
parametric attributes facilitating the examination of intrinsic characteristics and can be
acquired by using thermal imaging sensors, LiDAR, laser sensors and GPR, to name a few.
Table 2 provides a comparative evaluation of the next-generation sensors that have evolved
the entire data acquisition process for RCM. The convenient deployment of these sensors
on the ground and aerial platforms (i.e., drones) provides them with an edge in terms
of scalability and employability. Table 3 illustrates the merits and limitations of different
platforms on which sensors are commonly deployed.

Table 2. A comparative evaluation of the smart sensors used in RCM.

Key Variables Camera Laser GPR Thermal Vibration

Radio waves to Based on the
explore change in
3D construction of Accelerometers,
underground temperature of
Technology 2D imaging image using gyroscope, and
surface; creates 3D surrounding
reflection GPS readings
image of objects using
sub-surface infrared waves
Collection of depth
Complex
Collection of 3D images and Collection of heat Readings are
Processing image-processing
point cloud simulation data variation of surface directly used
algorithms
required
Real-Time Processor Cannot be used in
Yes Yes Yes
Application dependent real-time detection
Only after
While approaching While approaching While approaching While approaching
Sensing Time experiencing
distress distress distress distress
distress
Detection only
Characterization of Based on shape Based on heat
Based on 3D image Based on 3D image along wheel path
Distress and size maps
as 1D parameters
Sensitive to Not sensitive to
illuminance levels, Not sensitive to Not sensitive to light effect, but
Light Sensitivity None
light source light effect light effect surface
position temperatures
Algorithm Highly susceptible
Accuracy High High High
dependent to errors
Varying low to High-resolution Depends on Needs
Resolution -
high images frequency improvement
Delayed due to
Data collection and Data collection is
large data
analysis is fast; fast and can be Poor as data
processing; Data collection and
Processing Time response time is collected at speeds processing is
however, data analysis is fast
processor as high as 100 required
collection is
dependent km/h
automated
Cost Economical High Highly expensive Very expensive Low
Data Type 2D, 3D 3D 3D 2D, 3D 1D
nsors 2022, 22, x FOR PEER REVIEW 5 of 28
Sensors 2022, 22, x FOR PEER REVIEW 5 of 28

Table 3. List of sensor platforms available for data acquisition.

Table 3. List of sensor platforms available for data acquisition.
Sensors 2022, 22, 3044 Sensor Platform Advantages Limitations 5 of 27
Sensors 2022, 22, x FOR PEER Sensor
Unmanned
REVIEW Platform
Aerial Vehicle Advantages Limitations
5 of 28

Unmanned Aerial Vehicle

Table 3. List of sensor platforms availableLarge
for data acquisition.
FOV.
Table 3. List of sensor platforms available for data acquisition.
Large
High FOV.
Resolution.
Sensor Platform
Sensor Platform Advantages
Advantages Limitations Limitations
High Resolution.
In‐depth, detailed Payload and
Unmanned Aerial Vehicle
Unmanned Aerial Vehicle In‐depth, detailed
data. memory and
Payload
Large FOV. data.of deployment
Ease memory
restrictions.
High Resolution.
Ease
and of deployment
accessibility in restrictions.
Legislative
Large FOV.
In‐depth, detailed Payload and
High and accessibility
hazardous
Resolution. areas. in Legislative
restrictions.
data. memory Payload and memory
In-depth, hazardous
detailed data.
Flexibility
Ease of deployment forareas.
quick
restrictions. restrictions.
restrictions.
Ease
and of deployment in and
Flexibility for quick
inspections.
accessibility Legislative Legislative restrictions.
accessibility in hazardous
hazardous areas. areas.
restrictions.
inspections.
Flexibility
Flexibility for quick inspections.
for quick
inspections.
Ground Vehicle
Ground Vehicle
Ground Vehicle
Ground Vehicle

Long span availability.

Longof
Long spanArray span availability.Small FOV.
sensors.
availability.
Long
Arrayspan availability.
Array
of sensors. of sensors.
High‐resolutionFOV. Less
Small Small FOV.
cost‐effective.
Array of sensors.
High‐resolution Less
High‐resolution Small
cost‐effective. FOV.Less
Lessdependency cost-effective.
cost‐effective.
imagery. High
High-resolution
imagery. imagery.
High dependency High dependency on
Highly
imagery.
Highly dense and onHigh dependency
manpower.
Highlydense
dense and occluded
and on manpower. manpower.
terrain. Highly dense
occluded terrain.
occluded terrain. and on manpower.
occluded terrain.

Smartphone
Smartphone
Smartphone Low‐resolution
Smartphone Lightweight
imagery.
technology. Low‐resolution
Lightweight Limited by RGB
Economically
Lightweight viable.
technology. Low‐resolution
imagery.
Low-resolution imagery.
Lightweight
technology. data.
Economically viable. imagery.
Limited
Limited by RGB
by RGBdata.
technology. viable.
Economically
Limited
data. by RGB
Economically viable.
data.
Advances in next‐generation sensors [1] and data‐driven technologies [2–4] have
transformed data collection and interpretation, contributing to extensive research on
RCM. Recent years have seen the widespread use of vision‐based approaches to provide
cost‐effective
Advances solutions in identifying and sensors
in next‐generation localizing [1]
pavement anomalies. Diverse
and data‐driven meth‐
technologies [2–4] have
ods have Advances
been presented, in from
ranging next-generation
traditional image sensors
processing[1]techniques
and data-driven vision‐ technologies
totechnologies [2–4] have
Advances in next‐generation sensors [1] and data‐driven
transformed data collection and interpretation, contributing to extensive research [2–4] have
on
transformed
based AI approaches. The data collection
archetypal and interpretation,
vision‐based RCM is commonly contributing
carried out either to extensive research on RCM.
transformed
RCM. Recentimage
by employing
dataprocessing
years collection
have seen theandwidespread
interpretation,
such use
contributing
of vision‐based toapproaches
extensive research
to provide on
Recent years havetechniques
seen the(IPTs),
widespread as thresholding, or by using
use of vision-based ma‐
approaches to provide cost-
RCM. Recent
cost‐effective
chine learning years
solutions
(ML) have seen the
in identifying
algorithms for local widespread
featureand use
localizing
extraction. ofpavement
In the vision‐based
process approaches
anomalies.
of implementing to provide
Diverse meth‐
effective solutions in identifying and localizing pavement anomalies. Diverse methods
cost‐effective
IPTs,
ods haveencouraged solutions
by grayscale
been presented, in identifying
and
ranging from and
texture‐based localizing
methods,
traditional Ref.pavement
image [5] processing
performed anomalies.
asphalt
techniquesDiverse meth‐
to vision‐
pavement have been
pothole presented,
detection and ranging from
segmentation using atraditional
waveletimage image
energy fieldprocessing
to highlight techniques to vision-based
ods
based have been presented,
AI approaches. The ranging
archetypal from traditional
vision‐based RCM processing
is commonly techniques
carried to either
out vision‐
the potholeAI approaches.
region. The proposed Themethod
archetypal
exhibitedvision-based RCM is commonly
satisfactory performance with an carried out either by
bybased
overall
AI
employing approaches.
accuracy image
of nearly
The
processing archetypal
techniques
87%, processing
and successfully
vision‐based
(IPTs),(IPTs),RCM
such
differentiated theas
is commonly
thresholding,
potholes
carried out either
from the or by using ma‐ machine
employing image techniques such as thresholding, or by using
by
chine employing
learning
cracks, patches, image
(ML)
greasy processing
algorithms
dirt, shadows, for
andtechniques
local feature
maintenance (IPTs), such as
extraction.
hole covers, In
and thresholding,
the process
precisely seg‐of or by using ma‐
implementing
learning (ML) algorithms for local feature extraction. In the process of implementing
chine
mented
IPTs, learning
encouraged (ML)
the pothole. by
IPTs, encouraged
algorithms
However,
grayscale for local
the overall
and
by grayscale
feature
methodology
texture‐based extraction.
wasmethods,
and texture-based
Inmethods,
time consuming the process
Ref. re‐ of implementing
and performed
[5] asphalt asphalt
Ref. [5] performed
quired encouraged
IPTs, tedious processing.
by grayscale and texture‐based methods, Ref. [5] performed asphalt
pavement pavement
pothole detection and segmentation
pothole detection using a wavelet
and segmentation energy field
using a wavelet energy to field
highlight
to highlight the
pavement
the pothole pothole
region. detection
The and
proposed segmentation
method using
exhibited a wavelet
satisfactory energy field
performance
pothole region. The proposed method exhibited satisfactory performance to highlight
with an an overall
with
the pothole
overall accuracyregion.
accuracy The
ofofnearly proposed
nearly 87%, method exhibited
andsuccessfully
87%, and satisfactory
successfullydifferentiated
differentiatedthethe performance
potholes
potholes fromfrom withthean patches,
the cracks,
overallpatches,
cracks, accuracy of nearly
greasygreasy
dirt, shadows, 87%,
andand
dirt, shadows, successfully
and maintenance
maintenance holedifferentiated
hole and
covers, covers,theand
potholes
preciselypreciselyfrom
segmented the
seg‐the pothole.
cracks, the
mented patches, greasy
pothole. dirt,
However, shadows,
the and
overall maintenance
methodology hole
was covers,
time and precisely
consuming
However, the overall methodology was time consuming and required tedious processing.and seg‐
re‐
mented
quired the pothole.
tedious [6], However,
processing.
Ref. thetooverall
in his attempt classifymethodology wasdeveloped
a distressed area, time consuming
a uniqueand re‐
two-stage RCM
quired tedious processing.
system. In the first stage, the artificial bee colony (ABC) algorithm was used to obtain a
threshold value to segment the image into distressed and non-distressed sections, followed
by feature extraction. The second stage involved training on a single-layer artificial neural
network (ANN). Although the proposed method successfully classified an area of distress,
there was a redundant and computationally intensive step to segment the images using the
Sensors 2022, 22, 3044 6 of 27

ABC algorithm, as the neural network alone can be efficiently trained to segment the images
and extract features for classification. Ref. [7] proposed an unsupervised and automated
pavement crack detection based on the analysis of photometric information from two-layer
(intensity, depth) images. The method is based on histograms and Otsu’s thresholding. The
results showed that the proposed method could be used for fast and approximate crack
estimations, especially in cases of a low signal-to-noise ratio. Despite the limitations of
the conventional IPTs in terms of handling the background complexities and illumination
complications, continuous research has been carried out using IPTs to enhance and amplify
the results. Recently, Ref. [8] proposed a three-phased crack detection system based on IPTs.
The approach involved contrast enhancement followed by the application of a discrete
wavelet algorithm for effective detection and, finally, Jerman enhancement to enhance crack
detection. Comparisons with existing methods showed the effectiveness of the proposed
technique to validate the recognition of surface cracks.
To improve the accuracy of existing RCM methods, ML approaches, such as KNN,
SVM, and k-means clustering, were integrated with IPTs. Ref. [9] used ML-based pat-
tern recognition methods with computer vision methods on street view images for the
assessment of pavements. A series of IPTs followed by SVM [10] classification successfully
identified the severity and location of pavement defects in the image. A comparison with
the U-Net DL model on the same dataset suggested inferior performance due to scarce
data. Ref. [11] presented a model based on the library of SVM (LIBSVM) to distinguish
the potholes from the cracks in concrete pavements. The proposed model is effective in
segmenting potholes with a high F1 score of 98.7%. A comparison with the Otsu, edge
detection, k-means, and watershed methods re-establishes the processing efficiency of the
method on cement concrete pavement potholes.
Ref. [12] adopted a k-means clustering algorithm followed by Otsu’s thresholding to
segment cracks in asphalt pavements. The proposed algorithm provided a satisfactory re-
sult in localizing and detecting pavement cracks. Ref. [13] developed a texture-independent,
tile-based image-processing algorithm to detect pavement cracks and classify them into
longitudinal and transverse cracks on 2D and 3D pavement images. The lengths of the
cracks were measured using curve fitting and an orientation axis. The method suggested
promising results when tested on 130 images of Portland cement concrete and asphalt
concrete surfaces, and took less than 20s per image to generate results. Ref. [14] adopted an
integrated approach, establishing IPTs in association with ML algorithms for automated
road crack detection and classification. Heuristic segmentation and denoising algorithms
were applied, followed by image-processing practices for enhancement and feature ex-
traction, and the application of a multi-stage hybrid model based on ensemble learning
for classification. A comparative study with existing ML approaches underscored the
competitive performance of the proposed work.
In the past few years, there have been several review papers published in the field
of RCM. For example, Ref. [15] performed an extensive review on the RCM methods
focusing largely on the types of pavement distresses and the detection equipment used. A
quantitative analysis of the work conducted in the past on individual pavement distresses,
such as cracks, potholes, surface deformations and defects, and the macrostate of the
road, such as road roughness, skid resistance, substructure quality and road surroundings,
were performed. Different equipment, such as a camera, laser, accelerometer, acoustic,
pressure sensor, radar, ultrasonic, deflectometer, friction tester for distress detection and
data collection, are emphasized and compared in the study. The paper presents a detailed
discussion on the various distresses and detection techniques; however, it does not focus
on the algorithms used for RCM. Another substantive review by [16] highlighted the
advancement of deep learning (DL) methodologies in accurate pavement crack detection.
The DL techniques were catalogued under three headings: classification, object detection
and segmentation. The performance metrics employed to evaluate these methods on
benchmark datasets in addition to traditional and DL-based crack-detection methods on 3D
data were also reviewed. In another review paper, Ref. [17] assessed the various computer-
Sensors 2022, 22, 3044 7 of 27

vision techniques employed in the detection of cracks and potholes in pavements. The
techniques were categorized under DL, non-DL, depth camera and tire pressure/vibration
methods. Varying inputs, such as RGB images, laser and thermal images for vision-based
techniques, were discussed and evaluated. It was deduced that major detection techniques
were oriented around computer vision approaches, though they failed to produce good
results. RGB images owe their limitations to lighting conditions, texture variation and
background complexity; however, laser and thermal imaging overcome these limitations,
but suffer due to environmental and cost factors as well. Limitations in terms of the system
size, computational requirements and real-time applicability were also highlighted in the
study. Unlike DL methods, [18] in their review, summarized the research carried out on
digital images in RCM based on IPTs and acquisition devices. The image acquisition devices,
along with their characteristics, were compared, and various image technologies applied in
pavement detection were expounded. Concurrently, the IPT for specific problems, such
as on-site pavement cracks, pavement texture, rutting and potholes, was elaborated. The
authors concluded that the salient factors in determining the detection equipment should be
based on cost, efficiency, clarity of image details and type of pavement, while ML methods
outperform the traditional IPT in RCM.
Over the last few years, vision-based approaches have augmented RCM solutions
and remarkably advanced the development of pavement monitoring and analysis. It
is imperative to note that there are other existing reviews [15–18] addressing pavement
distress studies, as discussed above. These review articles largely emphasize the types of
pavement distresses, the detection equipment used, the sensor technologies available, and
various computer vision techniques employed in the detection of cracks and potholes in
pavements; however, they do not focus on the data-driven algorithms used for distress
detection. The review on pavement-defect detection methods based on DL by [16] provided
a substantive overview of the subject, but has a restricted scope. The proposed review in this
paper aims to provide an all-inclusive review by focusing on the next-generation sensing
technologies and associated AI-based RCM methods, by elucidating the methodologies and
challenges in current developments, as well as recognizing the prevailing research voids
for further research studies. Therefore, the criteria for consideration of research articles in
this paper are the evaluation of the existing sensor-based and AI approaches deployed on
different platforms, namely UAVs, ground vehicles and smartphones for RCM. AI is a data-
driven amalgamation of various ML and DL algorithms inherently dependent on sensors
and data acquisition to provide solutions to real-world problems. These algorithms are a
subset of the entire AI domain and have been shifted from handcrafted feature extraction-
based ML methods to automated DL methods. To help researchers and engineers better
understand the application of AI methodologies in pavement monitoring and analysis,
the current review summarizes the recent work that has been established from the year
2017 to 2022. The AI approaches are classified under sensor-based methodologies, along
with the application of machine learning and deep learning algorithms for RCM. An
exhaustive review of the DL methodologies is presented that builds on classification,
segmentation and detection, employing next-generation sensors integrated with different
data acquisition platforms.
This paper is organized as follows. Section 2 presents a brief discussion of the reviewed
articles and their limitations as a comprehensive and systematic review. It elaborates
on AI technology in general, categorized under next-generation sensors deployed on
data acquisition systems and compares the related work in terms of the methodology
adopted and results achieved. Section 3 presents the potential future research directions.
Conclusions and challenges in using DL approaches in RCM are finally presented in
Section 4.

2. Smart Sensing of Pavement Distress Data

Smart sensing techniques provide high-resolution information on pavements. Faster
installation and flexibility in deploying noncontact sensors on various mobile data ac-
 adopted and results achieved. Section 3 presents the potential future research direc
Conclusions and challenges in using DL approaches in RCM are finally presented in
tion 4.

Sensors 2022, 22, 3044 2. Smart Sensing of Pavement Distress Data 8 of 27

Smart sensing techniques provide high‐resolution information on pavements. F
installation and flexibility in deploying noncontact sensors on various mobile data a
sition platforms
quisition platforms facilitate convenient
facilitate convenient ways to
ways to inspect inspect pavements,
pavements, overcoming overcoming
space spac
environmental constraints. Every platform has its advantages and
and environmental constraints. Every platform has its advantages and limitations and limitations and
complements plements eachusability.
each other’s other’s usability. A schematic
A schematic representation
representation of the sensors
of the sensors and data a
and data
sition platforms
acquisition platforms is shownisinshown
Figurein1.Figure 1.

Figure 1. A schematic
Figure representation
1. A schematic of next-generation
representation sensors and their
of next‐generation platforms.
sensors and their platforms.

Smart sensors primarily result in 1D (i.e., vibration time series) and 2D/3D (i.e., vision)
Smart sensors primarily result in 1D (i.e., vibration time series) and 2D/3D (i.
RCM data, which were analyzed using various signal processing, machine learning, and
sion) RCM data, which were analyzed using various signal processing, machine lear
AI techniques in the literature, as presented below.
and AI techniques in the literature, as presented below.
2.1. Vibration Data-Based RCM
2.1. Vibration Data‐Based RCM
Smartphones are installed with diverse sensors, such as gyroscopes, accelerometers,
and GPS receivers,Smartphones arepowerful
and thus act as installed measuring
with diverse sensors,
tools. such
Hence, as gyroscopes,
capitalizing on theaccelerom
and GPS receivers, and thus act as powerful measuring tools. Hence,
in-built vibration sensors to assist RCM has taken center stage in numerous research works. capitalizing o
in‐built vibration sensors to assist RCM has taken center stage
Vibration-based methods for detecting road anomalies rely on 1D data acquired from these in numerous res
works.
sensors and find highVibration‐based methods
applicability in the for detecting
real-time road
detection of anomalies
pavement rely on
distress due1Dtodata acq
low cost, fewerfrom these requirements
memory sensors and find
andhigh applicability
immunity to lightin the real‐time detection of pavemen
conditions.
tress smartphone
Ref. [19] used due to low cost, fewer memory
accelerometers requirements
to measure and immunity
acceleration to light
data. Basic signalconditions
Ref. [19]
processing and pattern used smartphone
recognition methods accelerometers to measure
were used to correlate theacceleration
measured datadata. Basic s
processing
with the international and pattern
roughness indexrecognition methods
(IRI), validating that were used
variance in to correlate the
acceleration mea-measured
surements can be used to determine IRI using smartphone sensors. Ref. [20] proposed
an asphalt pavement quality monitoring system using data collected from accelerometer
sensors of smartphones by treating the data as a multi-dimensional time series classifica-
tion problem. The proposed approach attained a good classification accuracy. Ref. [21]
compared a standard inertial profiler with a smartphone-based application, Roadroid, to
establish the usability of smartphone sensors in estimating the ride quality of the concrete
pavement. It was observed that Roadroid significantly underestimated the IRI values,
though a linear correlation existed between the IRI measurements obtained from both
methods. In another attempt, Ref. [22] proposed an AI-assisted smartphone-based data-
Sensors 2022, 22, 3044 9 of 27

driven technique to detect vibration-induced road anomalies using vibration sensors and
the detection of pavement patch defects using smartphone images. The results showed that
vibration-based methods were quite successful. However, they failed to efficiently cover
the entire roadway and detect non-vibration-induced pavement anomalies, which were
then complemented by a vision-based method.
Ref. [23] presented an AI-based low-cost RCM system by detecting pavement potholes
using smartphone sensors and onboard diagnostic devices. The method is based on the
observed patterns of the vehicle’s interaction with the road surface and confirms the usage
of smartphone sensors in low-cost pothole detection. Ref. [24] provided a cost-effective
pavement evaluation system for low-volume roads using smartphone-based roughness
data. Compared to conventional roughness measurement methods, it was found that a
good correlation existed between the results, when compared to the results from the Class
III-type roughness measurement equipment, justifying sufficient accuracy to determine
the presence of pavement distresses. Ref. [25] developed an RCM technique by using 3D
pavement data to train ML models on low-cost vehicle-mounted smartphone sensor data.
The model provided distress values as outputs, which can be used for estimating IRI. The
applicability of the technique could be justified by a high peak in the correlation between
IRI estimations produced from numerous runs along the same route. In another study,
Ref. [26] proposed a method to determine the IRI of a pavement surface using conventional
vehicles and smartphones by using the grey box model algorithm and the quarter-car
vehicle model. The results indicate that the IRI of the pavement can be determined with
reasonable accuracy using smartphones.
In a novel attempt, Ref. [27] proposed a vibration-based RCM system by considering
factors, such as vehicle suspension and a direct reconstruction of the pavement profile, to
compute IRI rather than using correlation-based procedures. Field testing of the method
proves its potential for RCM with reasonable accuracy and efficiency. Ref. [28] used low-
cost smartphone accelerometer data by developing a prototype smartphone application
to measure the IRI, by incorporating vehicle speed and vehicle type. The results showed
a strong correlation with the measured IRI data collected using profiler vans. Ref. [29]
proposed a low-cost method employing smartphone sensors for pavement roughness
measurement by collecting the vertical vibration values when a vehicle is driven over a
pavement. The smartphone-based acceleration values are converted to the IRI by using
the quarter-car simulation model. The high correlation values promise a model that can
be used in smartphone-based RCM. Ref. [30] proposed a road condition tool (RCT) to
assess road pavement defects based on a telemetric data crowdsourced by smartphone
users by analyzing the vehicle motion dynamics. The results obtained, in comparison with
reference data from highly specialized equipment, confirm the proposed RCT solution [31]
established an algorithm to calculate the IRI from the acceleration values obtained using
smartphone sensors. The algorithm identified the physical parameters of a quarter actual
vehicle model and established a relation between the acceleration, IRI and the profile
elevation. It was observed that a consideration of the dynamic characteristics of vehicles
improved the proposed method’s measurement accuracy.
Thus, vibration-based methods find immense applicability in RCM. They provide
a comparative measure of the degraded pavement condition that is memory friendly
and suitable for the real-time detection of road anomalies, but are vulnerable to errors
due to noise and signal sensitivity from other vehicle sensors. Moreover, the vibration
effects caused by any road obstacle can be perceived as a road anomaly by these sensors.
Additionally, it is important to note that vibration-based sensors only take into account
data along the wheel path.

2.2. Vision Data-Based RCM Using Machine Learning

The exploitation of ML methods allows for the effective identification, classification
and analysis of pavement structure conditions. Generally, ML approaches in pavement
engineering comprise support vector machines (SVMs) and artificial neural networks
Sensors 2022, 22, 3044 10 of 27

(ANNs). The following section briefly summarizes the conventional ML approaches used
in recent years in RCM using different data acquisition platforms.

2.2.1. UAV-Based Data Acquisition

Multispectral image-based ML algorithms were employed by [32,33] on drone-acquired
images to distinguish undamaged pavement from damaged pavement. SVM, ANN, and
Random Forest (RF) algorithms were evaluated and compared both on RGB and multi-
spectral images. It was deduced that, amongst the three algorithms, RF performs best with
a high accuracy on multispectral images. In addition to the feature set, it was deduced
that the spatial resolution of the pavement imagery is also a conclusive factor in the per-
formance of the classifier. Ref. [34] developed an RCM system based on drone images
using image-based methods to identify three types of cracks in pavements. Decent results
were achieved by the proposed combination method developed by combining Canny edge
detection and Otsu thresholding. Ref. [35] proposed the usage of low-flying UAV acquired
multispectral imagery in combination with CNN extracted spatial features to assess the
aging and damage conditions of pavements with the help of an SVM classifier. A multi-
scale semantic segmentation algorithm was carried out on the UAV pavement imagery
for classification. Three principal components obtained, after dimensionality reduction
in the multispectral imagery, were combined with deep abstract spatial features from the
CNN and fed as inputs to the SVM classifier for pavement surface classification into cracks,
potholes, early aging, middle aging and late aging. However, the dimensionality reduction
in multispectral imagery demands additional processing. Moreover, low-flying UAVs may
not be a practical approach for dense-traffic roadways.

2.2.2. Smartphone-Based Data Acquisition

Ref. [36] carried out an exhaustive review on road surface anomaly detection using
smartphone sensors, while highlighting various issues and challenges in the current ap-
proaches, and identified research gap areas in the domain. The factors affecting anomaly
detection, such as smartphone orientation, mounting location, vehicle suspension and
speed dependency, were also investigated. The existing approaches were compared using
five primary criteria: sensor data collection, pre-processing, processing, post-processing,
and performance evaluations. The major focus of the review was on data collection and pro-
cessing, rather than result-oriented methodologies, to perform anomaly detection. Ref. [37]
performed a smartphone-based pavement analysis to classify the road surface conditions
using an accelerometer, gyroscope and GPS data. Multiclass supervised ML techniques
were applied using features from all three axes of the smartphone sensors. The classification
was focused on three main classes- smooth roads, potholes, and deep transverse cracks.
The results confirmed a higher performance than models using single-axis features.
A comparative analysis between the vibration-based method using a smartphone
accelerometer and gyroscope and the vision-based method using video processing was
performed by [38]. It was observed that vision-based methods outperform the vibration
method. However, the latter is sufficient for routine monitoring purposes, while the vision-
based method was apt for detailed analysis. Ref. [39] studied and experimented with an
ML-based method for the identification and classification of a bottom-up cracked pavement
based on vibro-acoustic signatures from roadside sensors. Different ML classifiers, such
as MLP, CNN, RFC, and SVM, were used and compared, showing great accuracies for a
specific vibro-acoustic signature on concealed cracked road pavements. Ref. [40] studied
the vibration-based sensors and GPS embedded in smartphones for automated pothole
detection. Data was collected using a customized mobile application installed in dedicated
vehicles. A series of image processing techniques were applied, and the extracted features
were fed to various ML classifiers to identify the best approach. It was observed that the
time and frequency domain features outperformed when identifying potholes, and the RF
classifier exhibited the best performance amongst other classifiers, which was evaluated
against various road types to check its robustness.
Sensors 2022, 22, 3044 11 of 27

2.2.3. Ground Robot-Based 3D Data Acquisition

In contrast to the traditional 2D and manual methods, Ref. [41] proposed a 3D laser-
based pavement scanning and automatic defect detection approach to identify micro
(cracks) and macro (deformation) defects. A sparse processing algorithm to extract crack
candidate points and deformations support points were designed. The algorithm success-
fully obtained an accurate location and the classification information of defects with a
detection accuracy above 98%. Ref. [42] defined an ML-based system for the acquisition
and intelligent classification of potholes and cracks in pavement surfaces. A Kinect device
was used to obtain 3D point-cloud structures of pavements and retrieve their physical
properties. The system could successfully classify cracks and potholes, along with the
computing of the physical properties of pavement depressions, such as the length, width
and depth. Ref. [43] created a method for detecting road damage by using a laser scanner
mounted on a mobile mapping system to generate 3D pavement images. An attempt was
made to segment the pavement using a series of ML techniques. Ref. [44], in their work,
developed a computationally efficient pothole-detection algorithm based on disparity maps
and disparity transformation algorithms. The detection of potholes was finally achieved by
a comparison between actual and modelled disparity maps. The point clouds of potholes
were extracted thereafter. The authors created three datasets for stereo vision-based pothole
detection, and the algorithm achieved an overall accuracy of 98.7%.
Ref. [45] proposed a fully automated LiDAR-based, two-fold RCM assessment over
highways and airport runways using a mobile mapping system. Firstly, a fully automated
algorithm was proposed to detect and locate pavement distress from the 3D point cloud,
followed by characterization by severity in terms of the depth below the road surface, sur-
face area at different depths and filling volume of the detected potholes. The accuracy of the
point clouds was determined to be ±1–2 cm and elucidates the capability of LiDAR-based
mobile mapping systems to precisely detect locations with pavement distress of varying
severity and different causes. However, the entire setup of the mobile mapping system
turns out to be an expensive affair. Ref. [46] developed a pavement evaluation method us-
ing mobile mapping systems (MMSs) for automatic detection by spatial frequency analysis
of 3D point-cloud data acquired by laser scanners mounted on MMSs. Pavement distresses,
such as rutting, flatness and potholes, were detected, along with the crack percentage.
The choice of platform and sensors is totally dependent on the specification of a task at
hand, taking into consideration the scope, frequency of operation, area in question, extent
of detail, budget and strategic importance. A comparative summary of each platform’s
usability in the field of pavement distress evaluations is listed in Table 4.

Table 4. A comparative summary of the data acquisition platforms used for RCM.

Platforms Advantages Limitations

• Well-matched for
aerial reconnaissance. • Unsuitable for RCM in dense
• Unhindered large field of view. traffic roadways.
UAVs • Allow for navigation through • Limited by weather
difficult terrains. conditions, such as wind
• Facilitate safe and speed and harsh climate.
quick inspections.

• Lightweight sensors.
• Ease of employability due to • Limited by image resolution.
their size. • Suffer noisy data due to
Smartphones
• Stand-alone (hand-held device) external factors.
data acquisition method and can
be easily installed on vehicles.
Sensors 2022, 22, 3044 12 of 27

Table 4. Cont.

Platforms Advantages Limitations

• Extensive usability in dense • Limited by a small field

traffic areas. of view.
• Scalable platforms for • Poor cost-effectiveness in
Ground robots
multi-array sensors. terms of long hours of
• Find widespread serviceability operation and resources.
in RCM.

2.3. Vision Data-Based RCM Using Deep Learning

DL methods have become the most extensively used computational approach in the
field of civil engineering and intelligent transportation systems [47,48], attaining outstand-
ing results in RCM. One of the benefits of DL is its ability to learn large quantities of
data. With the rapid development of DL techniques, deep convolutional neural networks
(DCNNs) have gained paramount significance in carrying out vision-based tasks. In com-
parison to the conventional handcrafted feature-based techniques, the DL-based techniques
learn multi-level image features in detail, which are more descriptive than the handcrafted
features. A typical CNN architecture is created using several layers, such as the input layer,
hidden layers comprising convolution, pooling and activation functions resulting in feature
maps, and the output layer. A deep CNN is characterized by an architecture composed of
many layers. Auxiliary layers, such as dropout and batch normalization, are also incorpo-
rated within the aforesaid layers as per the necessity of purpose. A schematic representation
of a CNN is shown in Figure 2. The first layer of a CNN is the input layer, which is a 2D
representation of the input image. The convolution layer performs an element-by-element
dot product between a subarray of an input array and a receptive field, followed by a
summation of the multiplied values to which the bias is added. This layer is responsible
for the reduction in the input data size and subsequent reduction in the computational cost.
Batch normalization is performed to re-center and rescale the inputs by normalizing the
data distribution. It plays a significant role in smoothening the loss function by optimizing
the model. The activation function imparts nonlinearity to the model for semblance to
complex real-world problems. The pooling layer is another significant aspect of the CNN
responsible for down-sampling as it reduces the spatial size of an input array. A dropout
layer can be added to the CNN as an optimization layer to handle complex models that
might be prone to overfitting. All these layers, excluding the input layer, comprise the
Sensorshidden layers
2022, 22, x FOR PEER of the CNN whose output is fed to fully connected layers resulting 13
REVIEW inofthe
28
classification and detection post-application of a suitable activation function.

Figure 2. A schematicFigure
representation of a CNN.
2. A schematic representation of a CNN.

In general, vision‐based DL methods undertake three major pattern recognition

tasks: classification, object detection and image segmentation. Classification determines
the presence of objects in an image or video. It refers to training DL models to find the
classes (pavement distress types) that are present. Classification is useful at the binary
level of decision making, whether an image contains the desired object/anomaly or not. A
distinct task from the classification is localization, which determines the position of the
Sensors 2022, 22, 3044 13 of 27

In general, vision-based DL methods undertake three major pattern recognition tasks:

classification, object detection and image segmentation. Classification determines the
presence of objects in an image or video. It refers to training DL models to find the classes
(pavement distress types) that are present. Classification is useful at the binary level of
decision making, whether an image contains the desired object/anomaly or not. A distinct
task from the classification is localization, which determines the position of the classified
object. Object detection combines classification and localization to determine the objects
that are present and specify where they are in the image. It applies classification to distinct
objects and uses bounding boxes to show the detection and location of an object. Image
segmentation divides an image into regions, extracting potentially meaningful areas for
further processing, dependent on its shape and border, such as classification and object
detection. The goal of image segmentation is to highlight foreground elements and make it
easier to evaluate them. Image segmentation provides pixel-by-pixel details of an object,
making it different from classification and object detection. Thus, object classification
identifies the category of objects in the image, object detection identifies the category
and location of objects with rectangular bounding boxes, and segmentation predicts the
categories of each pixel and distinguishes the object instances.
In this paper, the literature review in the following section is catalogued under DL
as classification, segmentation and distress detection, and further sub-categorized under
data acquisition platforms. It is important to emphasize here that DL-based classification
and segmentation methodologies in recent years were restricted to data acquired using
ground-based sensors, while UAV and smartphone-based data acquisition platforms find
major applicability in areas related to distress detection in RCM, rather than classification
and segmentation.

2.3.1. Classifications in RCM Using DL

In contrast to Image Processing Techniques (IPTs), Ref. [49] implemented a vision-
based sliding window approach, using deep neural convolutional networks to detect
cracks in concrete structures. The high-resolution images were sliced into 40 K images of
a 256 × 256 pixel resolution. The method successfully identified the presence of a crack
on high-resolution images of the concrete surface and performed well when compared to
other IPT algorithms, such as Canny and Sobel edge detection. However, the proposed
approach deems to be a computationally expensive approach as it involves redundant
scanning through the entire image. A domain adaptation approach by [50] to detect cracks
in pavement images provided a simplified vision-based pavement crack detection system.
The overall methodology used a truncated VGG-16 DCNN as a deep feature generator,
pre-trained on the ImageNet database, to vectorize the labeled pavement images. The cross-
domain image classification, a deep transfer learning approach, was an exclusive attempt at
automated detection on surfaced pavement images owing to varied surface characteristics.
A comparison with existing ML models elucidated the high performance of neural networks
in RCM. Ref. [51] presented the German Asphalt Pavement distress dataset for training and
evaluation of existing DL models. The authors went a step further by evaluating an image
processing toolbox for road pavement surface-crack distresses with shallow and deep
neural networks. The authors summarized the results by confirming satisfying detection
results by DL approaches, in comparison to conventional computer-vision approaches.
Furthermore, the effectiveness of traditional regularization techniques, including dropout,
batch normalization, max-norm regularization and weight decay, were assessed.
An autonomous real-time road crack and pothole detection algorithm was proposed
by [52], deployable on a GPU-based conventional processing board with an associated
camera. The approach was based on a deep neural network architecture integrated with a
pre-processing method to ensure real-time performance. The authors assured that the pro-
posed system could be installed as a plug-and-play module in various autonomous robots
and self-driving cars. Ref. [53] proposed the application of CNN on Ground Penetrating
Radar images for automatically classifying pavement subgrade defects. Two CNNs called
Sensors 2022, 22, 3044 14 of 27

multi-stage CNN and cascade CNN were established to accomplish the tasks that achieved
high accuracies. A comparison with ML algorithms further confirmed the CNNs’ robust
performance. However, despite the 3D images, the physical characteristics associated with
the defects were not calculated. To maximize the evaluation accuracy, an automated pave-
ment evaluation system was proposed by [54]. Two-channel images obtained by overlaying
gray-scale (2D) and range (depth) images were fed to a CNN network for image classifica-
tion. Though the accuracy metrics were not specified, the authors claimed the superiority
of the method over conventional ML techniques. Ref. [55] presented an enlarged dataset
GAPs v2 as an extension of the GAPs dataset by adding 500 images and providing more
refined annotations. Additionally, a CIFAR-like subset of 50 k images was also created for
easy comparison and evaluation. Extensive experiments on the extended GAPs dataset
were performed using existing DL architectures to understand the role of context in distress
detection. Ref. [56] proposed an image-based system for the classification of a speed bump
and pothole using a five-layer simple convolutional neural network. The proposed network
achieved a classification accuracy of 97.7%.
Ref. [57] proposed the application of CNNs for pothole detection. A pre-pooling CNN
was designed, which added a pre-pooling layer before the first convolution layer to improve
the precision of pothole detection. The results demonstrated that the optimized pre-pooling
CNN had 98.95% recognition precision on testing data. The stability and comparative
study revealed the robustness of the CNN model in real-world situations, such as varying
light conditions and pavement materials. Ref. [58] suggested an edge AI-based framework
named the vehicular ad hoc network (VANET) for road anomaly detection, such as a
pothole, bump, and cracks. Pre-trained DL models, VGG-11 and ResNet-18, were used
parallelly for the automated classification and detection of anomalies. The dataset employed
was collected from various online sources, which was augmented for better results, and it
was suggested that the model’s performance superseded the other techniques used for the
detection and classification of road anomalies. Ref. [59] proposed a method for automatic
pavement evaluation by the identification, classification and quantification of multiple
urban flexible pavement distresses through the application of CNNs. The process involved
the concatenation of two CNNs, one that performs identifications with reasonable precision,
recall, and F1 score, followed by the quantification of the severity of the predicted distress.
Ref. [60] applied a three-layer DCNN to implement crack detection in asphalt pavements by
resizing images and classifying them as crack and non-crack. The DCNN was evaluated on
two benchmarked datasets and a self-collected dataset, with high accuracy on all datasets.
Ref. [61] proposed a fusion-based pavement damage-detection CNN model for the
classification of damages into nine pavement categories using EfficientNet-B4. The fusion
was conducted through multi-source sensor information, resulting in a thermal–RGB fused
image. It was observed that the model performed better on fused images, in comparison to
RGB and thermal images independently. However, in comparison, the prediction perfor-
mance was found to be better on RGB images for road marking, shadow, and manholes.
Ref. [62] proposed an automatic crack classification system by implementing transfer learn-
ing on pre-trained networks. ResNet101, GoogleNet and AlexNet were used for crack
classification. It was observed that GoogleNet was the fastest to reach maximum accuracy,
followed by ResNet101. Ref. [63] proposed a pothole classification system using a fusion
of image processing and DL methodology. The process involved the detection of objects
other than potholes in pavement images, which were then discarded and the remaining
image used for edge detection. Finally, the image post-edge detection was classified using
a YOLO classifier to confirm the presence of a pothole.

2.3.2. Segmentation in RCM Using DL

Ref. [64] presented a Feature Pyramid and Hierarchical Boosting Network for pave-
ment crack segmentation and a novel crack measurement technique. The crack detection
was formulated as a pixel-wise binary classification task. The method, along with other
ML and DL algorithms, was compared and evaluated in relation to five datasets to demon-
Sensors 2022, 22, 3044 15 of 27

strate the superiority and generalizability of the method. The authors also presented a
pixel-annotated crack dataset of asphalt pavements named Crack500. Ref. [65] presented a
road crack segmentation method based on generative adversarial networks (GANs) based
on U-Net [66]. The GAN [67] is a special type of neural network architecture used to
discriminate between real and fake images, until it is unable to distinguish the real from
the fake. The evaluation of the model was performed on three datasets, of which two were
public, and one was customized. The model performed moderately on the public dataset
with a maximum F1 score of 77.3% and did not obtain good segmentation on a custom
dataset. Ref. [68] implemented two DL networks, an improved CNN with structured
prediction and an FCN, to detect cracks on pavements with different severity levels. The
classification of the cracks was based on crack severity, with an F1 score of less than 70% for
both methods. Moreover, the Laser Crack Measurement System software was used, which
employed laser intensity to compute the depth of the cracks. The depth factor played a role
in determining the severity level during the annotation process.
Ref. [69] proposed a conditional Wasserstein Generative Adaptive Network (cW-
GAN) [70] based method, ConnCrack, to the inspect cracks on road surfaces using a
cost-effective commercial grade sports camera. This method performed pixel-level crack
detection and provided a novel algorithm based on a depth-first search to determine the
optimal crack connectivity map. A pixel-level annotated dataset, EdmCrack600, with
600 images was created for public use by the authors. However, in this study, the pixel-
level masks were not transformed to the physical properties, such as the width or length of
the cracks. Additionally, the proposed method was a data-starving model with training
on three datasets (ImageNet, Crack Forest Dataset, and EdmCrack600). Ref. [71] proposed
CrackU-net, a crack extraction method from pavement images, regardless of noise levels,
background conditions and image quality. The method was based on deep neural networks
for pixel-wise crack detection. CrackU-net was trained on images collected by high-speed
vehicle-mounted cameras and smartphones and exhibited a high accuracy of 99%.
Ref. [48] proposed a segmentation RCNN model based on Faster RCNN [72] and
FCN for RCM. While Faster RCNN performed the feature extraction, classification and
localization of distresses, FCN performed a pixel-wise semantic segmentation to provide
morphological information on each distress. The result provided a real-time pavement
inspection technique, with an average computation time of 34 ms/frame and overall pre-
cision and recall of 91.5% and 90.5%, respectively. Ref. [73] proposed a deep generative
adversarial network, named CrackGAN, for pavement crack detection. The network pro-
motes crack-patch-only supervised generative adversarial learning for end-to-end training
and solves the “All Black” issue existing in FCN-based pixel-level crack detection. The
model was trained on small image patches, but can handle all-size images by feeding a
bigger-size crack image into an asymmetric U-shape generator. The proposed approach was
validated using four crack datasets and achieved good performance in terms of efficiency
and accuracy.
An attempt to automate pavement distress classification based on pixel-level seg-
mentation was performed by [74] using U-Net and Resnet architecture. A dataset was
prepared from over a 350 km stretch of asphalt pavements in China along with data aug-
mentation, resulting in over 10,000 images of six types of distresses. Binary classification
(distress or non-distress) was performed, resulting in a high accuracy of 97%, while the
multi-classification accuracy was fairly low. Ref. [75] proposed the Mask RCNN [76] ap-
proach for the pixel-wise segmentation of the Region of Interest in pavements for pothole
detection and area calculations. The database was manually collected and annotated with
an overall accuracy of 90% for a computed area of a pothole. Ref. [77] proposed a novel DL
segmentation approach based on synthetic training data generation for the segmentation
of cracks in road pavement images. The synthetic data repository was created using three
publicly available datasets: KITTI and Cityscapes for the images of road scenes, and Crack
Forest Dataset, as a source of pixel-level labeled cracks. Segmentation was implemented
using Mask R-CNN and U-Net, showing good results.
Sensors 2022, 22, 3044 16 of 27

A grid-based crack detection model was proposed by [78] using a segmentation CNN
approach. Each image was divided into three grids of different sizes, namely 10 × 10,
20 × 20 and 30 × 30, as inputs to the segmentation model to classify crack or no crack.
Grids with cracks were segmented into binary images using the entropy thresholds, which
were then used to calculate the crack widths. Recently, Ref. [79] performed an extensive
study by incorporating stereo vision and DL to efficiently perform crack and pothole
segmentation using an automated pixel-level detection framework for RCM. The multi-
view stereo imaging system was used to establish the datasets containing color images,
depth images and color-depth overlapped images. A modified U-Net was implemented
and tested on asphalt roads with millimeter-level accuracy. The 3D crack segmentation
model outperformed the other models in terms of the inference speed and segmentation
accuracy. Additionally, a highly accurate automated pothole volume measurement method
based on a segmentation map was proposed to evaluate pothole severity.

2.3.3. Damage Detection in RCM Using DL

Ref. [80] proposed the use of the Faster R-CNN algorithm on a customized dataset for
damaged pavement detection, which was divided into six distinct categories: lateral crack,
longitudinal crack, pothole and separation, alligator crack, well cover without damage and
damage around the well cover. The proposed CNN architecture showed high accuracy and
stability in locating and identifying the cracks in the dataset. Banking on the advantages of
automatic feature extraction by CNNs, Ref. [81] presented a framework based on DL to
automate the roadway condition assessment. A CNN was designed to classify four types
of pavement cracks, namely, longitudinal, transverse, alligator and potholes, and the crack
detection dataset was created from images taken from search engines. The overall accuracy
of the proposed CNN was 76%, which was justified as reasonable when compared to the
methodologies limited to binary classifications.
Ref. [82] analyzed the feasibility and accuracy of thermal imaging in the field of pot-
hole detection. The work involved the acquisition of thermal and visual images of potholes
under various conditions and the application of augmentation techniques, followed by
the implementation of convolutional neural networks on the images. A comparison be-
tween the customized and pre-trained models revealed the best accuracy of 97.08% with
ResNet101. Ref. [83] implemented a novel method for pothole detection by integrating
images with mobile mapping point-cloud data. This method first detected the location
and 2D edge of a candidate pothole in the image. The point clouds were then used to
differentiate between potholes and patches by calculating the mean depth of the candidate
pothole. The geometric accuracy of the pothole extraction was evaluated using a simulation
experiment, showing that the mean size accuracy was ~1.5–2.7 cm.
Ref. [84] utilized existing DL methods to carry out a hotspot analysis on urban road
networks, highlighting pavement distress types and associated severities in terms of reme-
dial actions that were either necessary or not. Efforts were also made to outline approaches
to using the analysis for the continuous monitoring of pavement health. Damage detection
models were accurately able to highlight the location and assess distress severity. However,
the severity analysis was limited by human bias carried out during data annotation, rather
than post-detection employing mathematical calculations. To assess the workability of DL
models on Google Street View images, Ref. [85] implemented two classical DL frameworks
for pavement condition assessment on a customized labeled dataset. The images were man-
ually annotated, and the distresses were classified into nine categories, namely, reflective,
transverse, block, longitudinal, alligator, sealed transverse, sealed longitudinal, and lane
longitudinal cracking, along with potholes.
Ref. [86] developed stereo-vision techniques for the automatic identification of pot-
holes. Transfer learning-based DL methods for segmentation (Mask R-CNN) and localiza-
tion (YOLOv2) were also used for pothole detection, suggesting good precision results and
real-time applicability, respectively. Ref. [87] proposed a 2D vision-based pothole detection
method using two-stage CNNs that focused on the discriminative regions in the road
Sensors 2022, 22, 3044 17 of 27

instead of the global context. The approach was based on using two CNN subnetworks, a
localization network to find the regions likely to contain potholes based on heat maps and
a second CNN for classification, which feeds on the candidate regions from the localiza-
tion network. The experiments on the pothole dataset showed that the proposed method
achieved high precision and recall, and outperformed most existing methods. However,
this approach turned out to be expensive as the localization network added prior to the
classification increased the computational time.
To classify the road surface characteristics and destabilizations produced by pavement
distresses, and man-made obstructions, Ref. [88] proposed a DL approach to identify the
kinds of road surfaces and distinguish the stability events produced by potholes from
the stability events produced by other man-made structures or driver actions. A mobile
application was used to record the vibration and GPS information. DL models, such as
CNN, Long Short-Term Memory (LSTM), and Reservoir Computing were trained to identify
the types of road surfaces and different stability events. The models performed well with the
best accuracy of 85% for road surface classification and 93% for stability event classification
by the CNN. Ref. [89] extensively evaluated the performance of eight DL models on the
RDD2020 dataset [90], and presented a modified dataset based on the same. The study
recognized the single-shot detector (SSD-Inception V2), and faster R-CNN-Inception V2, as
the best road-damage-detection approach in terms of accuracy and image processing time.
Ref. [91] proposed a solution to detect dry and wet potholes using smartphone sensors
and camera images. Twelve different sensor values were collected and manually annotated
to perform pothole detection using ML algorithms. The captured images were examined
for potholes by applying Mask R-CNN and U-Net segmentation algorithms. Both sensor
and camera-based methodologies provided promising results. Ref. [92] proposed a RCM
system that can be integrated with road-damage acquisition systems and presented an
asphalt pavement dataset with over 45,000 instances of various distress types. RetinaNet
architecture with VGG16 as the backbone was used, promising a better performance than
other object detection models. The light memory footprint allows the model to have an
easy integration into mobile systems.
Ref. [93] presented an RCM method based on Faster R-CNN to recognize and locate
cracks, potholes, oil bleeding and dot surface autonomously. A total of 20 Faster R-CNNs
were trained and tested on 6498 pavement images, with the performance of the optimal one
having accuracy rates, recall rates and location errors of 90.4%, 89.1% and 6.521 pixels. In
comparison to the CNN and K-value method, the optimal Faster R-CNN located pavement
distresses with more precision [94] proposed a methodology for automatic pavement
image distress detection and classification using CNN models and a low-cost vehicle-
mounted high-definition camera. The pavement distress types were categorized as linear or
longitudinal crack, network crack, fatigue crack or pothole, patch, and pavement marking.
The detection rate and classification accuracy of the proposed approach with the trained
CNN model reached 83.8% over the test set. A sensitivity analysis was also carried out for
evaluating the different regularization techniques and data generation strategies.
Ref. [95] developed a DL model, RoadID, to detect multiple pavement distresses. The
distress dataset contained over 44 k images, 14 k damages, and 8 distress types—crack,
net, pothole, patch crack, patch net, patch pothole, hinged joint, and manhole. The model
successfully located and identified the pavement damages with a mAP of 85.94%. Ref. [96]
proposed a pothole detection method using a modified VGG16 (MVGG16) network as a
backbone for the Faster R-CNN. The key feature of the MVGG16 network was its ability to
capture more contextual information by increasing the dilation rate of the convolution. A
comparison of the performance of YOLOv5 and Faster R-CNN with different backbones
proved that the MVGG16 network as the backbone of the Faster R-CNN provides a better
mean precision and shorter inference time than using other backbones. Ref. [97] performed
a classification of the pavement cracks into longitudinal, transverse, alligator cracks and
potholes using the DL model YOLO v3 for the localization and detection of cracks with a
precision of 0.7 and an average IoU of 50.39%.
Sensors 2022, 22, 3044 18 of 27

In view of inadequate large-scale data in civil engineering projects, Ref. [90] presented
RDD2020 by extending the existing RDD2018 dataset using GAN-based data augmentation.
PG-GAN and Poisson blending were used to generate realistic road damage images, which
fared well in the visual Turing test. The pothole dataset was increased to 2000 images,
and DL methodologies performed reasonably well on the augmented dataset. Ref. [98]
proposed a DL methodology to automatically localize diseased areas in pavements using
a novel approach named the Iteratively Optimized Patch Label Inference Network. The
method involved training image patches obtained by slicing high-resolution images and
inferring the patch labels using an algorithm based on the label of the high-resolution
image. The authors also released a novel, large-scale RCM dataset, CQU-BPDD, involving
various pavement diseases.
Ref. [99] implemented a highly accurate pavement defect and cleanness inspection sys-
tem using a DL-based framework in the pavement-sweeping robot Panthera. A lightweight
DCNN model was developed and trained on 6000 pavement defect and garbage images,
which took approximately 132.2 milliseconds for detecting both pavement defects and
garbage. Moreover, the geotagging of the pavement defects helped in mapping the defects.
In their work, Ref. [100] suggested that pothole detection categorized on specific road
types produced better results. The proposed model first classified the road surface as
asphalt, unpaved and paved using a CNN algorithm, followed by three YOLO v3 models,
respectively, for different road surfaces to detect potholes. Public datasets were used for
training, and augmentation was performed to improve the model accuracy. The authors
went a step further by geotagging the images during detection and updating a database to
mark the potholes on maps’ pothole tracking. The geotagged images were used to identify
the street and keep an account of distress conditions in that area.
Ref. [101] presented a CNN-based pothole detection method on a small dataset and
evaluated the performance of Faster RCNN with YOLOv3. As per the results, Faster
RCNN performed better than its counterpart. The accuracy of Faster RCNN was further
improved by integrating it with feature extraction using VGG16. The dataset was also
increased using augmentation techniques to achieve a better performance. Ref. [102]
compared and evaluated various DL object-detection techniques for damaged pavements
to detect potholes. The Kaggle and RDD2020 datasets were used for comparison, and it
was concluded that YOLOv4 performed the best with a mAP of 0.535. Ref. [103] proposed
a CNN-based approach for the automated detection of pavement distresses using images
collected from smartphones. The distresses were classified into six categories [90]. The
system demonstrated a practical efficiency and detection rate with more than 80% accuracy.
Ref. [104] implemented the YOLOv3 DL algorithm for pothole detection with a mAP of
65.65% on 416 × 416 pixel images. The detected potholes were logged, and their locations
translated for visualization on Google Maps using Google API.
Ref. [105] studied three YOLO object-detection frameworks to evaluate the best per-
formance for real-time pothole detection. The experiment was conducted on a dataset
of 665 images of 720 × 720 resolution using YOLOv4, YOLOv4-tiny, and YOLv5. It was
confirmed that YOLOv4-tiny was the best fit for pothole detection with a mAP of 78.7%.
Ref. [106] implemented a single-stage object detector called RetinaNet to detect and classify
fatigue, longitudinal, and transverse cracks having different severity levels using an auto-
mated road-scanning vehicle with two cameras. The authors declared that the accuracy of
detection and classification was highly dependent on the amount and quality of training
images, and the detection error exponentially decreased with an increase in the number of
training images. The trained network model achieved a detection accuracy of 84.9% on
images annotated with a crack type and severity level, while achieving an accuracy of 89.1%
when only considering the crack type. Ref. [107] proposed a novel Faster R-CNN approach
to localize and classify the cracks and damages in pavements. The Faster R-CNN approach
used pre-trained models, such as VGG16 and ResNet152. A dataset of 3533 images was
manually collected and categorized into alligator crack, linear crack, nonlinear crack, dam-
Sensors 2022, 22, 3044 19 of 27

age and non-crack image. The overall accuracy of the model with VGG16 was observed to
be 87%, while with ResNet152 was 90%.

UAV-Based Data Acquisition

Ref. [108] designed a distributed platform for pavement damage detection using
drones and a multi-agent architecture PANGEA (Platform for Automatic coNstruction of
orGanizations of intElligent Agents). The YOLOv4 classifier was customized to achieve
promising results with an accuracy of 95%. The images acquired by drones used in the
dataset have been published for use by the scientific community. Ref. [109] implemented a
real-time drone-based DL model to detect the cracks and potholes in pavement images. The
method involved identifying the yellow lane of the road for drone-flight autonomous navi-
gation, while concurrently performing real-time road crack and pothole detection using the
robot operating system within the UAV. The CNN achieved an F1 score of 85.78 and 94.04
for cracks and potholes, respectively. Ref. [110] studied drone-flight settings for optimal
pavement image quality and provided an open dataset unmanned aerial pavement dataset
(UAPD) for distress analysis with six distress types. Three object-detection algorithms were
used to train and test the UAPD. The prediction performances of the three algorithms were
compared to YOLOv3 as the best, having a mAP of 56.62% and the ability to recognize
cracks in different environments, including shadows, trees, and pavement markings.

Smartphone-Based Data Acquisition

Ref. [111] prepared a large-scale road damage dataset with eight damage categories
using a smartphone installed on a car. The training and evaluation of a lightweight
CNN-based damage detection model on the proposed data set were undertaken, with an
assessment of accuracy and runtime speed on a GPU server as well as a smartphone. Finally,
the road damages were classified into eight types by applying the proposed object detection
method. The road damage dataset is a breakthrough dataset that standardizes and provides
a platform to researchers for the comparison and evaluation of state-of-the-art DL models.
However, the model used for road damage detection does not provide a high accuracy
for different classes of road damage. Ref. [112] proposed an IoT-based end-to-end system
named PotSpot for monitoring and mapping potholes. An Android application was built
to perform CNN-based pothole detection integrated with Google Maps API for a real-time
pothole-marked map. The performance of the model was evaluated on a real-world road
image dataset with an accuracy of 97.5%. Ref. [113] implemented a self-driving car model
that can avoid potholes using a vision-based CNN approach. The end-to-end approach
does not detect potholes as bounding boxes, but rather avoids potholes by predicting the
vehicle parameters for driving decisions and the automatic control of the car. The game
simulator was used to collect driving data and perform training and testing.
Thus, vision-based classification, segmentation and detection have largely contributed
to RCM, allowing distress detection, monitoring and analysis. Each of these DL methodolo-
gies has its own strengths and limitations, as summarized in Table 5. It is important to note
that data is an important consideration in determining the performance of DL models; thus,
various data augmentation techniques have been implemented to address class imbalance
issues and increase the dataset size for better training and evaluation. To facilitate the
research community in future endeavors, a list of openly available data sources has been
compiled, as illustrated in Table 6.
Sensors 2022, 22, 3044 20 of 27

Table 5. The advantages and limitations of DL methodologies in RCM.

DL Methodologies Advantages Limitations Accuracy

• Demands training on large
volumes of data.
• Better than conventional ML • Very high-resolution images
approaches in terms of subjected to stitched patches • Ranges from 90–97%
Classification
performance. with distresses. Thus, the
results are discontinuous
and have ambiguous
structural semantics.

• Demands training on large

volumes of data.
• Performs pixel-level • Requires post-processing
classifications. algorithms to extract exact
and smooth shapes from • Ranges from 70–99%.
• Pixel-wise class assignment
pixelated outlines. • Higher accuracies
Segmentation allows an in-depth analysis
• Results are prone to noises. observed in single-class
of an image.
• Most of the studies seldom segmentation problems.
• Helps in determining the
morphology of the distress. focus on studying the
physical characteristics
associated with the defects,
such as width and length.

• High accuracies in pavement

distress detection.
• Provide classification as well
as localization of defects. • Ranges from 70–97%.
• Demands training on large
• Allow the mapping • Higher accuracies
volumes of data.
of defects. observed in single-class
• Physical characteristics of
Detection • With technologies, such as object detection, when
pavement distresses remain
depth measurement systems compared to
a gap, when limited to 2D
using LiDAR, and laser, and multi-classification and
data evaluation.
point clouds, the detection.
measurement of physical
characteristics of pavement
distress is possible.

Table 6. List of the open-source databases on pavement images.

Reference Name of Database Type Number and Type of Images

[51] GAPs Asphalt 1968 grayscale
[90] RDD2019 Asphalt 26,336 RGB
[41] Crack500 Asphalt 500 RGB
[41] GAPs384 Asphalt 384 grayscale
[114] CrackTree200 Asphalt 200 grayscale
[115] Crack Forest Dataset Asphalt 118 grayscale
[116] AEL Asphalt 58 grayscale
[117] Deep Crack Concrete, asphalt 537 RGB
[55] GAPs v2 Asphalt 2468 grayscale
[116] AigleRN Asphalt 38 grayscale
[85] Pavement Image Dataset - 7237 RGB
Sensors 2022, 22, 3044 21 of 27

Table 6. Cont.

Reference Name of Database Type Number and Type of Images

[108] - Asphalt 1362 RGB
[44] - Asphalt 630 RGB
[95] RoadID Asphalt 44,532 RGB
[110] UAPD Asphalt 3151 RGB
[98] CQU-BPDD Asphalt 60,059 RGB
[69] EdmCrack600 - 600 RGB
[92] Road Surface Damage Asphalt 18,345 RGB

3. Future Research Directions

Endeavors have been made to bridge the gaps and successfully implement DL algo-
rithms in RCM using data acquired from sensors. In addition to the breakthrough results
attained by researchers in this field, there exist several challenges that are listed below:
• Quality and quantity are often considered as the keys to a balanced dataset and
contribute to high performance in DL models. However, in spite of data augmentation
techniques resulting in large pavement datasets [91], the performance of different
DL models is not satisfactory. Thus arises a need to quantify the size of multi-class
samples in a dataset and identify appropriate augmentation algorithms and methods
to ensure a balanced, all-encompassing dataset.
• Standardization of distress types provides a comprehensive evaluation platform to
identify the best approach. However, it is observed that there exists no standard
nomenclature for pavement distress annotations in the publicly available datasets,
resulting in an ineffectual pavement distress analogy.
• DL models demand large storage space. The size of high-performing models and their
computational requirements pose a challenge to their usability in real-time scenarios.
To streamline the process above and beyond the existing challenges, the following
directive actions need to be brought into effect.
(i) Real-time processing: on-board processing on UAVs and other platforms demand a
lightweight though efficient detection model for the real-time assessment of pavements.
There is a need to develop computationally reasonable and cost-effective models that can
be deployed on sensor platforms to provide prompt results and avoid processing delays.
(ii) Data standardization: with large amounts of work being conducted in the de-
velopment of a pavement management system, there is a need to standardize the data
requirements to benefit cumulatively from the contributions made by the research com-
munity in the field. Data regulations can ensure a global collaboration towards a common
goal, resulting in a robust solution to the problem.
(iii) Characterization of distresses: the autonomous detection of distresses has paved
the way beyond the limitations faced by conventional systems. However, a step further
in this direction that quantifies the extent of damage can help agencies to prioritize the
regions demanding immediate attention. Thus, the physical characterization of distresses
can streamline the evaluation system by working towards a cohesive solution.
(iv) Crowdsourcing platform: with data no longer being a limitation and ease of access
to technology, a crowdsourcing platform can be developed for reporting distresses. This
will add to the collective database and a robust system that can recognize and report the
different distress types.
(v) Multidisciplinary research: civil engineers, computer scientists and data analysts
can come together by contributing their expert domain knowledge to build an exhaustive
distress evaluation system. A proper understanding of the pavement structures and
their distresses can contribute to high-quality data resources and help in designing high-
performance DL models.
Sensors 2022, 22, 3044 22 of 27

(vi) High computing resources: DL models are data hungry and need high-performance
computing resources. Cloud Computing and Edge Computing resources, including
Internet-of-Things sensors, 5G and 6G, can be instantly provisioned with the ability to scale
up and down. This will also allow systematic computing on a long-term basis devoid of
memory and compute resource limitations.
(vii) Integrated system: innovation in system integration is another area on which
research needs to be focused. The end-users are mostly government agencies and can
benefit more from a package that delivers an end-to-end solution.

4. Conclusions
In this paper, the existing research work in RCM using next-generation sensors and
AI methodologies was extensively reviewed and compared. The existing approaches are
evaluated based on the data acquisition platforms under various AI approaches, especially
DL for classification, segmentation and object detection. Data acquisition systems, which
are a combination of non-intrusive sensors and their platforms, are at the center of the
RCM system and involve the collection of 1D data, 2D visual data or 3D depth data. Every
platform has its own advantages and limitations and complements each other’s usability.
For the tasks involving a simple classification of distresses, RGB sensors are a good option,
while for a detailed study involving the various characteristics of distresses, LiDAR, laser,
thermal or GPR sensors can be used. It is also necessary to appreciate the value and
importance of data, along with the need to understand with clarity the definition of data.
Whether the problem deals with classification, detection, or localization in RCM, at the
core of every DL vision algorithm is a large collection of labeled images. Thus, another
important step is the data annotation of collected data as ground truths, a crucial step in
determining the accuracy of any DL model.
Developing DL models that accurately identify and characterize pavement distress
is a challenging task. Overall, the CNN-based DL classification models show very high
accuracy in detecting the presence or absence of multiple pavement distresses, and are
better than conventional computer vision approaches in terms of performance; however,
they fail to specify the location of distresses and thus lose their applicability in real-time
scenarios. Over the course of time, CNN algorithms involving intelligent techniques to
extract pavement distress features, such as transfer learning, pre-pooling layers, parallel
CNNs, multi-stage CNNs, multi-source sensors, and IPTs integrated with CNNs, have been
adopted to improve the existing performance of the DL classifiers.
Segmentation, which performs pixel-level classification employing CNNs, is another
approach used in pavement distress analysis and can be used to extract the morphological
characteristics of the distresses. However, not much work has been conducted in retrieving
the physical characteristics of pavement damages, attributed to noisy post segmentation.
Different approaches, such as connectivity maps and GANs, are being used to enhance
segmentation outputs. Extensive work is being conducted in this area to precisely measure
distress dimensions in terms of pixels; however, measurement accuracy, a decisive factor in
determining the distress severity, is a contentious issue faced by many researchers.
Distress detection that involves classification and localization shows good results with
2D data and promises detailed analysis with 3D data, but not much work related to the
measurements of distresses has been undertaken in this area. The detection of pavement
damages helps in localizing the defects and has led many researchers to overlay this infor-
mation on maps for navigation purposes. Innovative methods, such as thermal imaging,
point cloud data, stereo-vision analysis, location-aware CNNs, mobile applications and
UAV-based models for real-time assessment, to name a few, have been applied by many
researchers to enhance the performance of existing CNN models. It has also been observed
that CNN models have achieved a high performance in localizing distresses as a single
class, when compared to localizing distresses under various classes. The performance of
DL algorithms with advanced contributions from the scientific world and standardizations
will continue to validate the use of DL approaches in the field of RCM. CNNs are currently
Sensors 2022, 22, 3044 23 of 27

the most reliable method to monitor the pavements for distresses, and the further integra-
tion with innovative technologies, affordable sensors and platforms shall encompass the
futuristic, fully automated RCM systems.

Author Contributions: Conceptualization, A.S. and E.R.; investigation, A.S. and E.R.; writing—
original draft preparation, E.R.; writing—review and editing, A.S. and K.J.; supervision, A.S. and K.J.;
funding acquisition, A.S., K.J. and E.R. All authors have read and agreed to the published version of
the manuscript.
Funding: This research was funded by MITACS-Shastri Indo Canadian Institute (MITACS-SICI)
partnership and Ontario Ministry of Colleges and Universities’ Early Researcher Award.
Institutional Review Board Statement: Not applicable.
Informed Consent Statement: Not applicable.
Data Availability Statement: Table 6 provides the link to all the datasets that are publicly available.
Acknowledgments: The authors would like to thank the MITACS-Shastri Indo Canadian Institute
(MITACS-SICI) partnership for providing financial support to conduct this research through the
Globalink Research Award. This research is also supported by the Ontario Ministry of Colleges and
Universities’ Early Researcher Award funding provided to the corresponding author.
Conflicts of Interest: The authors declare no conflict of interest.

Abbreviations

AI Artificial Intelligence
DCNN Deep Convolutional Neural Network
DL Deep Learning
FCN Fully Convolutional Network
FOV Field of View
GAN Generative Adversarial Network
GPR Ground Penetrating Radar
GPS Global Positioning System
IoU Intersection over Union
IPT Image Processing Technique
IRI International Roughness Index
LiDAR Light Detection and Ranging
mAP Mean Average Precision
ML Machine Learning
RCM Road Condition Monitoring
RDD Road Damage Dataset
R-CNN Region Convolutional Neural Network
RF Random Forest
SVM Support Vector Machine
UAV Unmanned Aerial vehicle
YOLO You Only Look Once

References
1. Sony, S.; Laventure, S.; Sadhu, A. A literature review of next-generation smart sensing technology in structural health monitoring.
Struct. Control Health Monit. 2019, 26, e2321. [CrossRef]
2. Barbosh, M.; Singh, P.; Sadhu, A. Empirical mode decomposition and its variants: A review with applications in structural health
monitoring. Smart Mater. Struct. 2020, 29, 093001. [CrossRef]
3. Sony, S.; Dunphy, K.; Sadhu, A.; Capretz, M. A systematic review of convolutional neural network-based structural condition
assessment techniques. Eng. Struct. 2021, 226, 111347. [CrossRef]
4. Dunphy, K.; Sadhu, A. Recent Developments in Structural Health Monitoring and Assessment—Opportunities and Challenges; World
Scientific: Singapore, 2022.
5. Wang, P.; Hu, Y.; Dai, Y.; Tian, M. Asphalt Pavement Pothole Detection and Segmentation Based on Wavelet Energy Field. Math.
Probl. Eng. 2017, 2017, 1604130. [CrossRef]
Sensors 2022, 22, 3044 24 of 27

6. Banharnsakun, A. Hybrid ABC-ANN for pavement surface distress detection and classification. Int. J. Mach. Learn. Cybern. 2017,
8, 699–710. [CrossRef]
7. Akagic, A.; Buza, E.; Omanovic, S.; Karabegovic, A. Pavement crack detection using Otsu thresholding for image segmentation.
In Proceedings of the 2018 41st International Convention on Information and Communication Technology, Electronics and
Microelectronics (MIPRO), Opatija, Croatia, 21–25 May 2018; pp. 1092–1097. [CrossRef]
8. Taifi, K.; Taifi, N.; Azougaghe, E.-S.; Safi, S. An Automatic Detection by Classification of Cracked Pixels or Noncracked Pixels in
Road Surface. Math. Probl. Eng. 2021, 2021, 3151460. [CrossRef]
9. Chacra, A.D.; Zelek, J. Municipal Infrastructure Anomaly and Defect Detection. In Proceedings of the 26th European Signal
Processing Conference (EUSIPCO 2018), Rome, Italy, 3–7 September 2018.
10. Vapnik, V. Statistical Learning Theory; Wiley: New York, NY, USA, 1998.
11. Gao, M.; Wang, X.; Zhu, S.; Guan, P. Detection and Segmentation of Cement Concrete Pavement Pothole Based on Image
Processing Technology. Math. Probl. Eng. 2020, 2020, 1360832. [CrossRef]
12. Mubashshira, S.; Azam, M.; Ahsan, M. An Unsupervised Approach for Road Surface Crack Detection. In Proceedings of the 2020
IEEE Region 10 Symposium (TENSYMP), Dhaka, Bangladesh, 5–7 June 2020.
13. Safaei, N.; Smadi, O.; Masoud, A.; Safaei, B. An Automatic Image Processing Algorithm Based on Crack Pixel Density for
Pavement Crack Detection and Classification. Int. J. Pavement Res. Technol. 2021, 15, 159–172. [CrossRef]
14. Ahmadi, A.; Khalesi, S.; Golroo, A. An integrated machine learning model for automatic road crack detection and classification in
urban areas. Int. J. Pavement Eng. 2021. [CrossRef]
15. Coenen, T.B.; Golroo, A. A review on automated pavement distress detection methods. Cogent Eng. 2017, 4, 1374822. [CrossRef]
16. Cao, W.; Liu, Q.; He, Z. Review of Pavement Defect Detection Methods. IEEE Access 2020, 8, 14531–14544. [CrossRef]
17. Dib, J.; Sirlantzis, K.; Howells, G. A Review on Negative Road Anomaly Detection Methods. IEEE Access 2020, 8, 57298–57316.
[CrossRef]
18. Du, Z.; Yuan, J.; Xiao, F.; Hettiarachchi, C. Application of image technology on pavement distress detection: A review. Measurement
2021, 184, 109900. [CrossRef]
19. Aleadelat, W.; Saha, P.; Ksaibati, K. Development of serviceability prediction model for county paved roads. Int. J. Pavement Eng.
2016, 19, 526–533. [CrossRef]
20. Souza, V.M. Asphalt pavement classification using smartphone accelerometer and Complexity Invariant Distance. Eng. Appl.
Artif. Intell. 2018, 74, 198–211. [CrossRef]
21. Fortunatus, M.; Onyango, L.; Fomunung, I.; Owino, J. Use of a Smart Phone based Application to Measure Roughness of
Polyurethane Stabilized Concrete Pavement. Civ. Eng. Res. J. 2018, 4, 555645. [CrossRef]
22. Christodoulou, S.; Kyriakou, C.; Hadjidemetriou, G. Pavement Patch Defects Detection and Classification Using Smartphones,
Vibration Signals and Video Images. In Advanced Computing Strategies for Engineering; Elsevier: Amsterdam, The Netherlands,
2019; pp. 365–380.
23. Kyriakou, C.; Christodoulou, S.E.; Dimitriou, L. Smartphone-Based Pothole Detection Utilizing Artificial Neural Networks. J.
Infrastruct. Syst. 2019, 25, 04019019. [CrossRef]
24. Sandamal, R.M.K.; Pasindu, H.R. Applicability of smartphone-based roughness data for rural road pavement condition evaluation.
Int. J. Pavement Eng. 2022, 23, 663–672. [CrossRef]
25. Chatterjee, A.; Tsai, Y.-C. Training and Testing of Smartphone-Based Pavement Condition Estimation Models Using 3D Pavement
Data. J. Comput. Civ. Eng. 2020, 34, 04020043. [CrossRef]
26. Shohel Rana, D. Vibration Based Smart Pavement Monitoring System Using Vehicle Dynamics and Smartphone; enDAQ: Woburn, MA,
USA, 2020.
27. Rana, S. Vibration based pavement roughness monitoring system using vehicle dynamics and smartphone with estimated vehicle
parameters. Results Eng. 2021, 12, 100294. [CrossRef]
28. Alqaydi, S.; Zeiada, W.; Llort, D.; Elwakil, A. Using Smart Phones to Assessment Road Roughness in the UAE. In Proceedings of
the 11th Annual International Conference on Industrial Engineering and Operations Management, Singapore, 7–11 March 2021.
29. Lekshmipathy, J.; Velayudhan, S. Pavement Condition Assessment Using Smartphone Accelerometers. Int. J. Pavement Res.
Technol. 2021. Available online: https://www.ijert.org/pavement-condition-assessment-using-smartphone-accelerometers
(accessed on 26 March 2022).
30. Staniek, M. Road pavement condition diagnostics using smartphone-based data crowdsourcing in smart cities. J. Traffic Transp.
Eng. (Engl. Ed.) 2021, 8, 554–567. [CrossRef]
31. Zhang, Z.; Zhang, H.; Xu, S.; Lv, W. Pavement roughness evaluation method based on the theoretical relationship between
acceleration measured by smartphone and IRI. Int. J. Pavement Eng. 2021. [CrossRef]
32. Pan, Y.; Zhang, X.; Sun, M.; Zhao, Q. Object-Based and Supervised Detection of Potholes and Cracks from The Pavement Images
Acquired By UAV. ISPRS-Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2017, 42, 209–217. [CrossRef]
33. Pan, Y.; Zhang, X.; Cervone, G.; Yang, L. Detection of Asphalt Pavement Potholes and Cracks Based on the Unmanned Aerial
Vehicle Multispectral Imagery. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 2018, 11, 3701–3712. [CrossRef]
34. Zhang, L.; Xu, W.; Zhu, L.; Yuan, X.; Zhang, C. Study on Pavement Defect Detection Based on Image Processing Utilizing UAV. J.
Physics Conf. Ser. 2019, 1168, 042011. [CrossRef]
Sensors 2022, 22, 3044 25 of 27

35. Pan, Y.; Chen, X.; Sun, Q.; Zhang, X. Monitoring Asphalt Pavement Aging and Damage Conditions from Low-Altitude UAV
Imagery Based on a CNN Approach. Can. J. Remote Sens. 2021, 47, 432–449. [CrossRef]
36. Sattar, S.; Li, S.; Chapman, M. Road Surface Monitoring Using Smartphone Sensors: A Review. Sensors 2018, 18, 3845. [CrossRef]
37. Basavaraju, A.; Du, J.; Zhou, F.; Ji, J. A Machine Learning Approach to Road Surface Anomaly Assessment Using Smartphone
Sensors. IEEE Sensors, J. 2019, 20, 2635–2647. [CrossRef]
38. Lekshmipathy, J.; Samuel, N.M.; Velayudhan, S. Vibration vs. vision: Best approach for automated pavement distress detection.
Int. J. Pavement Res. Technol. 2020, 13, 402–410. [CrossRef]
39. Praticò, F.G.; Fedele, R.; Naumov, V.; Sauer, T. Detection and Monitoring of Bottom-Up Cracks in Road Pavement Using a
Machine-Learning Approach. Algorithms 2020, 13, 81. [CrossRef]
40. Wu, C.; Wang, Z.; Hu, S.; Lepine, J.; Na, X.; Ainalis, D.; Stettler, M. An Automated Machine-Learning Approach for Road Pothole
Detection Using Smartphone Sensor Data. Sensors 2020, 20, 5564. [CrossRef] [PubMed]
41. Zhang, K.; Cheng, H.D.; Zhang, B. Unified Approach to Pavement Crack and Sealed Crack Detection Using Preclassification
Based on Transfer Learning. J. Comput. Civ. Eng. 2018, 32, 04018001. [CrossRef]
42. Huincalef, R.; Urrutia, G.; Ingravallo, G.; Martínez, D.C. An Approach to Automated Recognition of Pavement Deterioration
Through Machine Learning. Commun. Comput. Inf. Sci. 2019, 995, 124–140. [CrossRef]
43. van der Horst, B.B.; Lindenbergh, R.C.; Puister, S.W.J. Mobile Laser Scan Data for Road Surface Damage Detection. ISPRS Int.
Arch. Photogramm. Remote Sens. Spat. Inf. Sci. 2019, 42, 1141–1148. [CrossRef]
44. Fan, R.; Ozgunalp, U.; Hosking, B.; Liu, M.; Pitas, I. Pothole Detection Based on Disparity Transformation and Road Surface
Modeling. IEEE Trans. Image Process. 2020, 29, 897–908. [CrossRef] [PubMed]
45. Ravi, R.; Bullock, D.; Habib, A. Highway and airport runway pavement inspection using mobile lidar. ISPRS-Int. Arch.
Photogramm. Remote Sens. Spat. Inf. Sci. 2020, 43, 349–354. [CrossRef]
46. Kawamura, H.; Nagata, Y.; Tokuno, T.; Ishida, T.; Mizutani, T.; Yamashita, J. Pavement Evaluation Method Using MMS. In RILEM
Bookseries; Springer Science and Business Media B.V.: Berlin/Heidelberg, Germany, 2022; Volume 27, pp. 1455–1463. [CrossRef]
47. Zhu, W.; Wu, J.; Fu, T.; Wang, J.; Zhang, J.; Shangguan, Q. Dynamic prediction of traffic incident duration on urban expressways:
A deep learning approach based on LSTM and MLP. J. Intell. Connect. Veh. 2021, 4, 80–91. [CrossRef]
48. Liu, C.; Li, J.; Gao, J.; Gao, Z.; Chen, Z. Combination of pixel-wise and region-based deep learning for pavement inspection and
segmentation. Int. J. Pavement Eng. 2021. [CrossRef]
49. Cha, Y.-J.; Choi, W.; Büyüköztürk, O. Deep Learning-Based Crack Damage Detection Using Convolutional Neural Networks.
Comput. Civ. Infrastruct. Eng. 2017, 32, 361–378. [CrossRef]
50. Gopalakrishnan, K.; Khaitan, S.K.; Choudhary, A.; Agrawal, A. Deep Convolutional Neural Networks with transfer learning for
computer vision-based data-driven pavement distress detection. Constr. Build. Mater. 2017, 157, 322–330. [CrossRef]
51. Eisenbach, M.; Stricker, R.; Seichter, D.; Amende, K.; Debes, K.; Sesselmann, M.; Ebersbach, D.; Stoeckert, U.; Gross, H.-M. How to
get pavement distress detection ready for deep learning? A systematic approach. In Proceedings of the 2017 International Joint
Conference on Neural Networks IJCNN, Anchorage, AK, USA, 14–19 May 2017; pp. 2039–2047. [CrossRef]
52. Murshed, M.; Institute of Electrical and Electronics Engineers and International Association for Pattern Recognition. Proceedings
of the 2018 International Conference on Digital Image Computing: Techniques and Applications (DICTA), Canberra, Australia, 10
December–13 December 2018.
53. Tong, Z.; Gao, J.; Zhang, H. Innovative method for recognizing subgrade defects based on a convolutional neural network. Constr.
Build. Mater. 2018, 169, 69–82. [CrossRef]
54. Mokhtari, S.; Yun, H.-B. A deep learning approach for pavement evaluation using 2D and 3D imaging systems. In Proceedings of
the 18th International Conference Road Safety on Five Continents (RS5C 2018), Jeju Island, Korea, 16–18 May 2018.
55. Stricker, R.; Eisenbach, M.; Sesselmann, M.; Debes, K.; Gross, H.-M. Improving Visual Road Condition Assessment by Extensive
Experiments on the Extended GAPs Dataset. In Proceedings of the 2019 International Joint Conference on Neural Networks
(IJCNN), Budapest, Hungary, 14–19 July 2019; pp. 1–8. [CrossRef]
56. Thomas, A.; Harikrishnan, P.M.; Nisha, J.S.; Gopi, V.P.; Palanisamy, P. Pothole and Speed Bump Classification Using a Five-Layer
Simple Convolutional Neural Network. In Advances in Intelligent Systems and Computing; Springer: Singapore, 2021; Volume 1245,
pp. 491–499. [CrossRef]
57. Ye, W.; Jiang, W.; Tong, Z.; Yuan, D.; Xiao, J. Convolutional neural network for pothole detection in asphalt pavement. Road Mater.
Pavement Des. 2019, 22, 42–58. [CrossRef]
58. Bibi, R.; Saeed, Y.; Zeb, A.; Ghazal, T.M.; Rahman, T.; Said, R.A.; Abbas, S.; Ahmad, M.; Khan, M.A. Edge AI-Based Automated
Detection and Classification of Road Anomalies in VANET Using Deep Learning. Comput. Intell. Neurosci. 2021, 2021, 6262194.
[CrossRef] [PubMed]
59. Llopis-Castelló, D.; Paredes, R.; Parreño-Lara, M.; García-Segura, T.; Pellicer, E. Automatic Classification and Quantification of
Basic Distresses on Urban Flexible Pavement through Convolutional Neural Networks. J. Transp. Eng. Part B Pavements 2021, 147,
04021063. [CrossRef]
60. Faudzi, M.J.A.A.; Osman, M.K.; Yusof, N.A.M.; Ahmad, K.A.; Ahmad, F.; Idris, M.; Raof, R.A.A.; Hazlyna, H.N. Detection of
Crack on Asphalt Pavement using Deep Convolutional Neural Network. J. Physics Conf. Ser. 2021, 1755, 012048. [CrossRef]
61. Chen, C.; Chandra, S.; Han, Y.; Seo, H. Deep Learning-Based Thermal Image Analysis for Pavement Defect Detection and
Classification Considering Complex Pavement Conditions. Remote Sens. 2021, 14, 106. [CrossRef]
Sensors 2022, 22, 3044 26 of 27

62. Jana, S.; Thangam, S.; Kishore, A.; Kumar, V.S.; Vandana, S. Transfer learning based deep convolutional neural network model for
pavement crack detection from images. Int. J. Nonlinear Anal. Appl. 2022, 13, 1209–1223. [CrossRef]
63. Baek, J.-W.; Chung, K. Pothole Classification Model Using Edge Detection in Road Image. Appl. Sci. 2020, 10, 6662. [CrossRef]
64. Yang, F.; Zhang, L.; Yu, S.; Prokhorov, D.; Mei, X.; Ling, H. IEEE Transactions on Intelligent Transportation Systems, under Review.
1 Feature Pyramid and Hierarchical Boosting Network for Pavement Crack Detection. Available online: https://github.com/
fyangneil/pavement-crack-detection (accessed on 26 March 2022).
65. Gao, Z.; Peng, B.; Li, T.; Gou, C. Generative Adversarial Networks for Road Crack Image Segmentation; Generative Adversarial
Networks for Road Crack Image Segmentation. 2019. Available online: http://www.ieee.org/publications (accessed on
26 March 2022).
66. Ronneberger, O.; Fischer, P.; Brox, T. U-Net: Convolutional Networks for Biomedical Image Segmentation. Available online:
http://arxiv.org/abs/1505.04597 (accessed on 1 May 2015).
67. Goodfellow, I.; Pouget-Abadie, J.; Mirza, M.; Xu, B.; Warde-Farley, D.; Ozair, S.; Courville, A.; Bengio, Y. Generative Adversarial
Networks. Commun. ACM 2020, 63, 139–144. [CrossRef]
68. Jung, W.M.; Naveed, F.; Hu, B.; Wang, J.; Li, N. Exploitation of deep learning in the automatic detection of cracks on paved roads.
Geomatica 2019, 73, 29–44. [CrossRef]
69. Mei, Q.; Gül, M. A cost effective solution for pavement crack inspection using cameras and deep neural networks. Constr. Build.
Mater. 2020, 256, 119397. [CrossRef]
70. Arjovsky, M.; Chintala, S.; Bottou, L. Wasserstein Gan. Available online: https://arxiv.org/abs/1701.07875 (accessed on
1 March 2017).
71. Huyan, J.; Li, W.; Tighe, S.; Xu, Z.; Zhai, J. CrackU-net: A novel deep convolutional neural network for pixelwise pavement crack
detection. Struct. Control Health Monit. 2020, 27, e2551. [CrossRef]
72. Ren, S.; He, K.; Girshick, R.; Sun, J. Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks. Available
online: https://github.com/ (accessed on 26 March 2022).
73. Zhang, K.; Zhang, Y.; Cheng, H.-D. CrackGAN: Pavement Crack Detection Using Partially Accurate Ground Truths Based on
Generative Adversarial Learning. IEEE Trans. Intell. Transp. Syst. 2020, 22, 1306–1319. [CrossRef]
74. Li, D.; Duan, Z.; Hu, X.; Zhang, D. Pixel-Level Recognition of Pavement Distresses Based on U-Net. Adv. Mater. Sci. Eng. 2021,
2021, 5586615. [CrossRef]
75. Arjapure, S.; Kalbande, D. Deep Learning Model for Pothole Detection and Area Computation. In Proceedings of the 2021
International Conference on Communication information and Computing Technology (ICCICT), Mumbai, India, 25–27 June 2021;
pp. 1–6. [CrossRef]
76. He, K.; Gkioxari, G.; Dollár, P.; Girshick, R. Mask R-CNN. Available online: http://arxiv.org/abs/1703.06870 (accessed on 1
March 2017).
77. Kanaeva, I.A.; Ivanova, J.A. Road pavement crack detection using deep learning with synthetic data. IOP Conf. Series Mater. Sci.
Eng. 2021, 1019, 012036. [CrossRef]
78. Xiao, L.; Li, W.; Huyan, J.; Sun, Z.; Tighe, S. Crack grid detection and calculation based on convolutional neural network. Can. J.
Civ. Eng. 2021, 48, 1192–1205. [CrossRef]
79. Guan, J.; Yang, X.; Ding, L.; Cheng, X.; Lee, V.C.; Jin, C. Automated pixel-level pavement distress detection based on stereo vision
and deep learning. Autom. Constr. 2021, 129, 103788. [CrossRef]
80. Li, J.Q.; Zhao, X.F.; Li, H.W. Method for detecting road pavement damage based on deep learning. Health Monit. Struct. Biol. Syst.
XIII 2019, 10972, 81. [CrossRef]
81. Aslan, O.D.; Gultepe, E.; Ramaji, I.J.; Kermanshachi, S. Using Artifical Intelligence for Automating Pavement Condition
Assessment. In Proceedings of the International Conference on Smart Infrastructure and Construction 2019 (ICSIC) Driving
Data-Informed Decision-Making, Cambridge, UK, 8–10 July 2019; pp. 337–341. [CrossRef]
82. Aparna; Bhatia, Y.; Rai, R.; Gupta, V.; Aggarwal, N.; Akula, A. Convolutional neural networks based potholes detection using
thermal imaging. J. King Saud Univ.-Comput. Inf. Sci. 2019, 34, 578–588. [CrossRef]
83. Wu, H.; Yao, L.; Xu, Z.; Li, Y.; Ao, X.; Chen, Q.; Li, Z.; Meng, B. Road pothole extraction and safety evaluation by integration of
point cloud and images derived from mobile mapping sensors. Adv. Eng. Inform. 2019, 42, 100936. [CrossRef]
84. Roberts, R.; Giancontieri, G.; Inzerillo, L.; Di Mino, G. Towards Low-Cost Pavement Condition Health Monitoring and Analysis
Using Deep Learning. Appl. Sci. 2020, 10, 319. [CrossRef]
85. Majidifard, H.; Jin, P.; Adu-Gyamfi, Y.; Buttlar, W.G. Pavement Image Datasets: A New Benchmark Dataset to Classify and
Densify Pavement Distresses. Transp. Res. Rec. J. Transp. Res. Board 2020, 2674, 328–339. [CrossRef]
86. Dhiman, A.; Klette, R. Pothole Detection Using Computer Vision and Learning. IEEE Trans. Intell. Transp. Syst. 2020, 21,
3536–3550. [CrossRef]
87. Chen, H.; Yao, M.; Gu, Q. Pothole detection using location-aware convolutional neural networks. Int. J. Mach. Learn. Cybern. 2020,
11, 899–911. [CrossRef]
88. Varona, B.; Monteserin, A.; Teyseyre, A. A deep learning approach to automatic road surface monitoring and pothole detection.
Pers. Ubiquitous Comput. 2020, 24, 519–534. [CrossRef]
89. Cao, M.-T.; Tran, Q.-V.; Nguyen, N.-M.; Chang, K.-T. Survey on performance of deep learning models for detecting road damages
using multiple dashcam image resources. Adv. Eng. Inform. 2020, 46, 101182. [CrossRef]
Sensors 2022, 22, 3044 27 of 27

90. Maeda, H.; Kashiyama, T.; Sekimoto, Y.; Seto, T.; Omata, H. Generative adversarial network for road damage detection. Comput.
Civ. Infrastruct. Eng. 2021, 36, 47–60. [CrossRef]
91. IEEE Region 10 and Institute of Electrical and Electronics Engineers. In Proceedings of the 2020 IEEE Region 10 Conference
(TENCON), Osaka, Japan, 16–19 November 2020.
92. Ochoa-Ruiz, G.; Angulo-Murillo, A.A.; Ochoa-Zezzatti, A.; Aguilar-Lobo, L.M.; Vega-Fernández, J.A.; Natraj, S. An Asphalt
Damage Dataset and Detection System Based on RetinaNet for Road Conditions Assessment. Appl. Sci. 2020, 10, 3974. [CrossRef]
93. Song, L.; Wang, X. Faster region convolutional neural network for automated pavement distress detection. Road Mater. Pavement
Des. 2021, 22, 23–41. [CrossRef]
94. Zhang, C.; Nateghinia, E.; Miranda-Moreno, L.F.; Sun, L. Pavement distress detection using convolutional neural network (CNN):
A case study in Montreal, Canada. Int. J. Transp. Sci. Technol. 2021. [CrossRef]
95. Li, Y.; Liu, C.; Shen, Y.; Cao, J.; Yu, S.; Du, Y. RoadID: A Dedicated Deep Convolutional Neural Network for Multipavement
Distress Detection. J. Transp. Eng. Part B Pavements 2021, 147, 04021057. [CrossRef]
96. Ahmed, K.R. Smart Pothole Detection Using Deep Learning Based on Dilated Convolution. Sensors 2021, 21, 8406. [CrossRef]
[PubMed]
97. Opara, J.N. Defect detection on asphalt pavement by deeplearning. Int. J. GEOMATE 2021, 21, 87–94. [CrossRef]
98. Tang, W.; Huang, S.; Zhao, Q.; Li, R.; Huangfu, L. An Iteratively Optimized Patch Label Inference Network for Automatic
Pavement Distress Detection. IEEE Trans. Intell. Transp. Syst. 2021. [CrossRef]
99. Ramalingam, B.; Hayat, A.; Elara, M.; Gómez, B.F.; Yi, L.; Pathmakumar, T.; Rayguru, M.; Subramanian, S. Deep Learning Based
Pavement Inspection Using Self-Reconfigurable Robot. Sensors 2021, 21, 2595. [CrossRef]
100. Agrawal, R.; Chhadva, Y.; Addagarla, S.; Chaudhari, S. Road Surface Classification and Subsequent Pothole Detection Using
Deep Learning. In Proceedings of the 2021 2nd International Conference for Emerging Technology (INCET), Belagavi, India,
21–23 May 2021. [CrossRef]
101. Pei, L.; Li, S.; Sun, Z.; Li, W.; Gao, Y.; Chen, Y. Detecting potholes in asphalt pavement under small-sample conditions based on
improved faster region-based convolution neural networks. Can. J. Civ. Eng. 2022, 49, 265–273. [CrossRef]
102. Darapaneni, N.; Reddy, N.S.; Urkude, A.; Paduri, A.R.; Satpute, A.A.; Yogi, A.; Natesan, D.K.; Surve, S.; Srivastava, U. Pothole
Detection Using Advanced Neural Networks. In Proceedings of the 2021 IEEE 12th Annual Information Technology, Electronics
and Mobile Communication Conference (IEMCON), Vancouver, BC, Canada, 27–30 October 2021; pp. 567–572. [CrossRef]
103. Omarov, B.; Kizdarbekova, M.; Omarov, B.; Omarov, N. Development of an Automatic Road Damage Detection System to Ensure
the Safety of Tourists. Commun. Comput. Inf. Sci. 2021, 1393, 404–413. [CrossRef]
104. Yik, Y.K.; Alias, N.E.; Yusof, Y.; Isaak, S. A Real-time Pothole Detection Based on Deep Learning Approach. J. Physics Conf. Ser.
2021, 1828, 012001. [CrossRef]
105. Ha, J.; Park, K.; Kim, M. A Development of Road Crack Detection System Using Deep Learning-based Segmentation and Object
Detection. J. Soc. e-Bus. Stud. 2021, 26, 93–106. [CrossRef]
106. Tran, V.P.; Tran, T.S.; Lee, H.J.; Kim, K.D.; Baek, J.; Nguyen, T.T. One stage detector (RetinaNet)-based crack detection for asphalt
pavements considering pavement distresses and surface objects. J. Civ. Struct. Health Monit. 2020, 11, 205–222. [CrossRef]
107. Aravindkumar, S.; Varalakshmi, P.; Alagappan, C. Automatic Road Surface Crack Detection Using Deep Learning Techniques. In
Artificial Intelligence and Technologies; Springer: Singapore, 2022; pp. 37–44. [CrossRef]
108. Silva, L.A.; Blas, H.S.S.; García, D.P.; Mendes, A.S.; González, G.V. An Architectural Multi-Agent System for a Pavement
Monitoring System with Pothole Recognition in UAV Images. Sensors 2020, 20, 6205. [CrossRef] [PubMed]
109. Hassan, S.-A.; Rahim, T.; Shin, S.-Y. An Improved Deep Convolutional Neural Network-Based Autonomous Road Inspection
Scheme Using Unmanned Aerial Vehicles. Electronics 2021, 10, 2764. [CrossRef]
110. Zhu, J.; Zhong, J.; Ma, T.; Huang, X.; Zhang, W.; Zhou, Y. Pavement distress detection using convolutional neural networks with
images captured via UAV. Autom. Constr. 2022, 133, 103991. [CrossRef]
111. Maeda, H.; Sekimoto, Y.; Seto, T.; Kashiyama, T.; Omata, H. Road Damage Detection and Classification Using Deep Neural
Networks with Smartphone Images. Comput. Civ. Infrastruct. Eng. 2018, 33, 1127–1141. [CrossRef]
112. Patra, S.; Middya, A.I.; Roy, S. PotSpot: Participatory sensing based monitoring system for pothole detection using deep learning.
Multimed. Tools Appl. 2021, 80, 25171–25195. [CrossRef]
113. Rosyid, H.A.; Burhan, O.S.; Zaeni, I.A.E.; Pee, A.N.C. Comparison of Deep Learning Models in Pothole Avoidance for Self-Driving
Car. In Proceedings of the 2021 7th International Conference on Electrical, Electronics and Information Engineering (ICEEIE),
Malang, Indonesia, 2 October 2021; pp. 466–471. [CrossRef]
114. Zou, Q.; Cao, Y.; Li, Q.; Mao, Q.; Wang, S. CrackTree: Automatic crack detection from pavement images. Pattern Recognit. Lett.
2012, 33, 227–238. [CrossRef]
115. Shi, Y.; Cui, L.; Qi, Z.; Meng, F.; Chen, Z. Automatic Road Crack Detection Using Random Structured Forests. IEEE Trans. Intell.
Transp. Syst. 2016, 17, 3434–3445. [CrossRef]
116. Amhaz, R.; Chambon, S.; Idier, J.; Baltazart, V. Automatic Crack Detection on Two-Dimensional Pavement Images: An Algorithm
Based on Minimal Path Selection. IEEE Trans. Intell. Transp. Syst. 2016, 17, 2718–2729. [CrossRef]
117. Liu, Y.; Yao, J.; Lu, X.; Xie, R.; Li, L. DeepCrack: A deep hierarchical feature learning architecture for crack segmentation.
Neurocomputing 2019, 338, 139–153. [CrossRef]