ORIGINAL RESEARCH

published: 02 July 2018

doi: 10.3389/fbuil.2018.00031

Utilizing UAV and 3D Computer

Vision for Visual Inspection of a
Large Gravity Dam
Ali Khaloo 1 , David Lattanzi 1*, Adam Jachimowicz 2 and Charles Devaney 3
1
Department of Civil, Environmental, and Infrastructure Engineering, George Mason University, Fairfax, VA, United States,
2
Argonne National Laboratory, Washington, DC, United States, 3 Quadrocopter, Las Vegas, NV, United States

Dams are a critical infrastructure system for many communities, but they are also one of
the most challenging to inspect. Dams are typically very large and complex structures,
and the result is that inspections are often time-intensive and require expensive,
specialized equipment and training to provide inspectors with comprehensive access
to the structure. The scale and nature of dam inspections also introduce additional
safety risks to the inspectors. Unmanned aerial vehicles (UAV) have the potential
to address many of these challenges, particularly when used as a data acquisition
platform for photogrammetric three-dimensional (3D) reconstruction and analysis, though
the nature of both UAV and modern photogrammetric methods necessitates careful
planning and coordination for integration. This paper presents a case study on one
such integration at the Brighton Dam, a large-scale concrete gravity dam in Maryland,
Edited by:
Dryver R. Huston, USA. A combination of multiple UAV platforms and multi-scale photogrammetry was
University of Vermont, United States used to create two comprehensive and high-resolution 3D point clouds of the dam and
Reviewed by: surrounding environment at intervals. These models were then assessed for their overall
Vasilis Sarhosis,
Newcastle University, United Kingdom
quality, as well as their ability to resolve flaws and defects that were artificially applied
Martina Colombo, to the structure between inspection intervals. The results indicate that the integrated
Politecnico di Milano, Italy
process is capable of generating models that accurately render a variety of defect
*Correspondence:
types with sub-millimeter accuracy. Recommendations for mission planning and imaging
David Lattanzi
[email protected] specifications are provided as well.
Keywords: infrastructure inspection, computer vision, structure from motion, dam inspection, 3D scene
Specialty section: reconstruction, aerial robots, structural health monitoring, unmanned aerial vehicles
This article was submitted to
Structural Sensing,
a section of the journal
Frontiers in Built Environment
1. INTRODUCTION
Received: 18 April 2018 Dams provide vital service and protection for many communities, however, their complex nature
Accepted: 12 June 2018 poses many challenges for an efficient and reliable inspection. These infrastructure systems can
Published: 02 July 2018
experience a range of problems during their service life, and improper operation and maintenance
Citation: result in major issues. Furthermore, dams can be deteriorated due to weathering, alkali-aggregate
Khaloo A, Lattanzi D, Jachimowicz A
reaction (AAR, or alkali-silica reaction, ASR), freezing and thawing, or other chemical reactions.
and Devaney C (2018) Utilizing UAV
and 3D Computer Vision for Visual
The Federal Emergency Management Agency (FEMA) publishes the Guidelines for Dam Safety
Inspection of a Large Gravity Dam. (U.S. Dept. Of Homeland Security, Federal Emergency Management Agency, 2004), and suggests
Front. Built Environ. 4:31. that formal inspections occur at least every 5 years. They also recommend informal, intermediate
doi: 10.3389/fbuil.2018.00031 and special inspections as needed. The Maryland Department of Environment also recommends

Frontiers in Built Environment | www.frontiersin.org 1 July 2018 | Volume 4 | Article 31

Khaloo et al. UAV Dam Inspection

that owners inspect their dams after extreme rainfall and formally somewhat complimentary approach is to use photogrammetric
once every 5 years (Maryland Department of Environment, methods to extract 3D geometries from large sets of two-
1996). According to the American Society of Civil Engineers’ dimensional (2D) digital images. In either case, the result is
(ASCE) 2017 Infrastructure Report Card (ASCE, 2017) the a scale-accurate, high-resolution virtual model of a structure
average age of the 90,580 dams in the United States is 56 years, and its surrounding areas. These digital models capture current
with 17% rated as high-hazard potential dams necessitating conditions of the entire structure that can be used for archival and
additional inspections. analytical purposes (Ghahremani et al., 2016; Jafari et al., 2017;
The conventional standard of practice requires a detailed D’Altri et al., 2018). However, both 3D imaging approaches suffer
visual inspection not just of the primary structure, but of from the same access challenges that hinder conventional visual
the subsystems and the surrounding watershed as well. Dam inspections.
inspections often require expensive and specialized equipment Unmanned aerial vehicles (UAV) are a disruptive innovation
to provide inspectors access to difficult to reach regions of the (Christensen, 1997) with potential to transform traditional dam
structure and the costs for these inspections typically range into inspection methodologies by expanding the capabilities of 3D
the hundreds of thousands of dollars, comprising an outsized imaging in these environments. While UAV have been in use for
portion of annual operating budgets. The scale and nature of some time, their recent popularity is in part due to reductions
dam inspections also introduce additional safety risks to the in hardware costs, improvements to software interfaces, and to
inspectors. Typically, a limited number of photographs, and the expanded range of sensor payload options (Turner et al.,
occasionally videos, are captured to provide a visual record of the 2012). The portability, mobility, and low cost of UAV can mitigate
current state of the structure. By themselves, these recordings are the need for expensive inspection access equipment and reduce
not ideal data products, as reviewing them can be tedious and the safety risks to inspectors (Khaloo et al., 2018). Furthermore, UAV
lack of spatial context can prove disorienting to data analysts and serve as an almost ideal data collection platform for modern 3D
engineers. reconstruction techniques. Critically, the nature of both UAV
Among conventional nondestructive evaluation (NDE) and 3D reconstruction methods necessitate careful planning and
techniques, Impact-Echo has been widely used to investigate the coordination to properly integrate and tailor these technologies
condition of concrete and extent of cracking in concrete dams for dam inspection.
(Sack and Olson, 1995). This stress-wave propagation method is
one of the most promising NDE approaches due to the fact that it 1.1. Prior Work on Modern Dam Inspection
only requires to access one side of a test member, and is capable of The reduced accessibility of dams, both for uptake needs and for
determining member thickness as well as flaw depth while being their strategic nature, and the large amount of time needed for an
less sensitive to the heterogeneity of concrete in comparison to inspection by traditional methods do not facilitate direct visual
other methods (e.g., ultrasonic) (Malhotra and Carino, 2003). inspection. Therefore, novel methods that integrate modern
In the work by Olson and Sack (1995), the Spectral Analysis remote sensing tools, robotics and computer vision techniques
of Surface Waves (SASW) method was utilized for evaluation have been investigated in the past few years. In the work by Ridao
of surface damage and determining the depth of freeze/thaw et al. (2010), an Autonomous Underwater Vehicle (AUV) was
defects in a large concrete dam located on the Muskegon River in designed to collect images from a hydroelectric dam which was
western Michigan. For monitoring of a large hydraulic structure later used to generate photomosaic (with approximate resolution
affected by ASR, an Ultrasonic Pulse Velocity (UPV) method of 1 pixel/mm) of the inspected area to help with the visual
was used, which enabled the identification of a major tensile inspection. González-Aguilera et al. (2008) studied the viability
crack (Rivard et al., 2010). UPV is an effective way for detecting of utilizing TLS systems to generate 3D models of a large concrete
internal cracking and other defects as well as changes in concrete dam and further assessed its capability for structural monitoring.
such as deterioration due to aggressive chemical environment However, due to the limited access for data acquisition, the final
and freezing and thawing (Malhotra and Carino, 2003). In the model lacked the necessary completeness to capture the entire
study by Colombo and Comi (2016), a bi-phase damage model of dam. Berberan et al. (2011) studied using TLS for deformation
an existing concrete arch dam subjected to ASR was generated by monitoring of the downstream face of the Cabril Dam in Portugal
taking into account the effects of both temperature and humidity using 3D models generated at two different times for comparison
on the hydraulic structure through a heat diffusion analysis purposes. Although both of the aforementioned studies proved
and a moisture diffusion analysis, respectively. Although these the value of 3D modeling using TLS for dam inspection, they
NDE methods can be used to further evaluate the condition of were not able to capture the overall geometry and resolve the
the concrete structures, periodic visual inspection is still the fine-scale (1-mm) details needed for accurate visual inspection.
dominant method to assess the structural integrity of dams There have also been a series of recent studies on the use
and appurtenant structures (U.S. Dept. Of Homeland Security, of UAV for dam inspection. In the study by González-Jorge
Federal Emergency Management Agency, 2004). et al. (2014), photogrammetric 3D modeling using UAV acquired
Recently, there have been advancements in the use of 3D imagery was tested for the monitoring of dam breakwaters.
imaging systems for capturing the in-situ 3D state of civil Recently, researchers have focused on using camera-equipped
infrastructure systems (Fathi et al., 2015). The most widely UAV to facilitate dam visual inspection through 3D modeling
used technology for generating 3D models, or point clouds, is (Henriques and Roque, 2015; Ridolfi et al., 2017). In the work
Terrestrial Laser Scanning (TLS). A lower-cost alternative and by Ming-Der et al. (2016), the capability of rotary UAV platform

Frontiers in Built Environment | www.frontiersin.org 2 July 2018 | Volume 4 | Article 31

Khaloo et al. UAV Dam Inspection

for image data collection and later 3D model generation was system, as well as an evaluation of the imaging specifications
investigated for landslide dam monitoring. A full-scale test necessary to render small-scale inspection details. It is also the
structure was built and two sets of image data was collected first time, to the authors’ best knowledge, that a multi-scale
before and after damaging it, in order to mimic dam failure photogrammetric 3D reconstruction technique has been used
due to severe flooding. The generated 3D data was later used to to capture the overall geometry of a large-scale complex gravity
identify the reduced volume upon dam failure. In a similar effort dam, capable of resolving fine structural flaws on the order of 1
by Zekkos et al. (2016), the image data of a collapsed uniform mm using multiple UAV platforms for data acquisition.
earthfill dam, captured using a rotary UAV, was used to generate The details of the data collection procedure are presented
a 3D model which later utilized for cross-sectional analysis of first, followed by reporting of the operations on two inspection
the failure region. Hallermann et al. (2015) used 1600 high- mission days. This is followed by a presentation of the data
resolution aerial images to generate a 3D model of the Rappbode processing and analysis methodologies. An evaluation of the
Dam, the highest dam in Germany, as well as an orthomosaic results of the methodological approach for the Brighton Dam
image of the downstream face. In the study by Oliveira et al. inspection is included as well. The paper concludes with a
(2014), an orthomosaic image of the downstream face of a summation of the findings of the research team and avenues for
concrete dam, located in Portugal, was generated using 99 UAV- future work in this domain.
acquired images. Later, a defect map was manually generated
by overlaying the marked anomalies on the orthomosaic image 2. DATA COLLECTION
in AutoCAD. In the recent work by Buffi et al. (2017), UAV-
based photogrammetry was used as a new tool for surveyors 2.1. Logistics and Planning
to generate a complete 3D model of the Rideacoli Dam in The goal of the field testing was to collect two comprehensive
Italy. The generated model was compared against conventional sets of digital images to be converted into 3D point clouds,
techniques such as total stations, TLS and Global Positioning using a combination of DSfM and rigid registration techniques.
System (GPS) to assess the overall geometry captured through The point clouds needed to have the point density—analogous
3D photogrammetric approach. The evaluation results showed an to resolution—necessary to resolve inspection details on the
accuracy within 2 cm, but the surface resolution (point density) millimeter scale while comprehensively capturing the overall
of the model was estimated to be an average of one point every spatial context of the dam. To achieve this, image acquisition
1 cm2 which may be inadequate for detailed visual inspection of in terms of camera positioning, the number of captured images,
smaller scale damages such as cracks. adjacent image overlapping, and image quality were all carefully
considered.
1.2. Contributions of This Work A variant of the DSfM process previously developed by the
While UAV have previously been used to generate 3D research team was chosen to generate 3D point clouds (Khaloo
reconstructions of dams, a multi-UAV approach has yet to be and Lattanzi, 2016; Khaloo et al., 2018). This process, referred
evaluated. Furthermore, the efficacy of such an approach for to as Hierarchical Point Cloud Generation (HPCG), is designed
representing small scale inspection details has not been evaluated, to integrate images captured at a wide range of standoff distances
rather only large volumetric analyses have been undertaken in from a structure into one complete model. From a data collection
prior works. To address these needs, this paper presents a case standpoint, images were collected in networks of similar standoff
study on integrating multiple UAV platforms and 3D computer distances. The images from each network were then integrated
vision for the visual inspection of a large gravity dam in the and merged into a global, multi-scale point cloud model, as
United States. The results of this case study indicate how these presented in the later sections of the paper.
technologies can be integrated and evaluated for suitability in Both fixed-wing and multi-rotor UAV platforms with
inspection scenarios, and provide guidance to future efforts mounted cameras were used to acquire image networks. The
with regards to the imaging and flight operations specifications concept was to use the fixed wing UAV to capture the global
necessary to attain a desired level of 3D model quality. geometry of both the upstream and downstream faces of the
In this study, a combination of multiple UAV platforms dam, and for the multi-rotor UAV to generate a series of oblique
and photogrammetric approaches was used to create two image networks from varying standoff distances. UAV flight
comprehensive 3D point clouds of a dam and surrounding planning that resulted in images with greater than 80% overlap
environment that were then assessed for their relative quality between adjacent photos was desired to minimize the ground
and ability to render artificially applied defects. The Brighton sample distance (GSD) and consequently maximize the spatial
Dam, located in Brookeville, Maryland and managed by the resolution. The minimum standoff distance from the dam was
Washington Suburban Sanitary Commission (WSSC), was held to approximately 2.5 m. This corresponded to a pixel size
selected as the subject of this study (Figure 1). The dam was put of approximately 0.0024 (mm/pixel) and a GSD of 0.6 mm in
into service in 1944, and is representative of a large-scale gravity the plane of the dam façade for the lowest resolution sensor used
dam in-service across the United States. during the project.
The key technical contributions of this work include the These images networks were collected over the course of two
development and assessment of a multi-UAV system for distinct days, in order to simulate variances in field conditions
generating massive, dense, and comprehensive 3D point clouds between inspection intervals and provide a basis for temporal
(contain more than one billion points) of the targeted dam analysis between the subsequent models. Data collection on the

Frontiers in Built Environment | www.frontiersin.org 3 July 2018 | Volume 4 | Article 31

Khaloo et al. UAV Dam Inspection

FIGURE 1 | The Brighton dam.

first day was designed to comprehensively image the entire dam safe operations were feasible, the team began to target dates for
and environs using a combination of three different UAV. Data data collection.
collection on the second day focused on an isolated section of the The performance of UAV inspection imaging is primarily
dam that was selected for the defect analysis portion of the study, dependent on operating conditions that do not impede piloting of
as will be discussed later. the UAV itself. In particular, cold weather was a key consideration
A custom-built fixed wing mounted with a Sony Alpha Series when identifying mission dates. The team determined that the
5100 camera (24.3-megapixel) with a Sony E-PZ 16–50 mm lens battery life of the selected UAV would not operate sufficiently
was used to capture a series of nadir angle shots that covered the in temperatures below 5 degrees Celsius. Rain, snow, and high
dam and the surrounding area. The utilized UAV’s airframe was winds were also weather phenomena that dictated the ability
the Super Sky Surfer fixed wing expanded polyolefin (EPO) foam to operate the UAV safely. Secondary to selecting mission days
frame. It was modified in order to custom fit various components, with viable UAV operating conditions, radiometric conditions
such as the autopilot, GPS module, airspeed sensor, camera that optimized the consistency of UAV imagery were preferred.
payload, motor and batteries. The communication between the High contrast lighting, typically due to bright sun conditions,
UAV and ground control station (GCS) was done through radio can create strong shadows that degrade the performance of the
telemetry. DSfM process (Remondino et al., 2014; Khaloo and Lattanzi,
Two DJI Inspire 1 aircrafts mounted with 12-megapixel (MP) 2016). Having considerable changes in exposure, white balance
cameras were used to separately capture oblique imagery of the and lighting can cause inaccuracies at both image matching and
downstream and upstream portions of the dam. On the second dense multi-view reconstruction stages (see section 3).
day of data collection, in addition to capturing the entire dam Recently, researchers (Gaiani et al., 2016) have developed
structure, the mission was to focus on a specific region of the methods to lessen the impact of radiometric factors through
dam, Bay #5 of the downstream facade with preinstalled targets, series of 2D image processing algorithms such as color
which required maneuvering a UAV in a confined space. For enhancement, image denoising, and image content enrichment
this task, a DJI Phantom 4 Pro with a 16-MP camera, a smaller prior to initiating the 3D scene reconstruction. However, due
quadcopter aircraft, was selected due to its better maneuverability to the relatively high quality of the acquired images, these pre-
and collision avoidance features relative to the other available processing steps were not implemented in this study.
aircraft.
The logistics of flying multiple UAV over the two mission 2.2. Day One Operation
days, along with the complexities of the dam environment, Mission planner software was used to develop a flight plan for
meant that proper planning was essential. After the UAV pilots the fixed wing nadir imaging that would allow the aircraft and
confirmed the test site was not in a restricted airspace, and that camera to operate autonomously. A lawnmower pattern for an

Frontiers in Built Environment | www.frontiersin.org 4 July 2018 | Volume 4 | Article 31

Khaloo et al. UAV Dam Inspection

area of 174,502 m2 (Figure 2) was selected, at an elevation of 75 recovery of both the 3D geometry (structure) of a scene and
m. A camera intervalometer frequency of 1.69 seconds/image was the camera pose (motion) using a sparse set of correspondences
specified, resulting in 558 nadir images (Figure 3). Total flight between image features, and has been shown to produce results
time for the fixed wing UAV was 26 min. comparable to laser scanners (Seitz et al., 2006; Remondino et al.,
The two rotary UAV were piloted manually, with a second 2014).
operator controlling the camera and gimbal in order to guarantee Generally speaking, the first step in the SfM process is to
the specified 80% overlap between images and maintain standoff automatically detect keypoint feature descriptors (pixel locations
distances, specified to maximize the quality of the DSfM that are highly distinctive) such as the Scale Invariant Feature
reconstructions (Figure 4). Transform (SIFT) (Lowe, 2004) in each input image. Next,
In total 2,020 images of the structure were captured using all feature descriptors are matched between pairs of images by
three aircrafts. finding a correspondence in the second image using a nearest
neighbor similarity search (Muja and Lowe, 2014) to construct
2.3. Day Two Operation the relationship of feature points between image pairs, called
The second day of operations also focused on Bay #5, on the tracks.
downstream face of the dam. The goals of this day’s operations Because corresponding points in two images are subject to
were to generate a point cloud for comparisons with the point the epipolar constraints (describe by the fundamental matrix),
cloud generated through Day One operations, as well as to filtering the matches by enforcing these constraints removes
assess the quality of the UAV point cloud generation process for false correspondences (Hartley and Zisserman, 2003). By using
resolving small-scale structural defects. the normalized eight-point algorithm (Hartley, 1997) in tandem
Prior to UAV flight and imaging inside Bay #5, a series with the RANSAC (RANdom SAmple Consensus) (Fischler and
of controlled and simulated defects with known dimensions Bolles, 1981) paradigm, it is possible to minimize the number of
were applied to the dam structure, serving as a benchmark for wrong matches across images.
reconstruction accuracy and analytical testing. Overall, three Next, an initial image pair with a large number of matched
different types of defects were applied: linear markings with features and a long separation distance is selected and their
controlled thickness (Type 1), square surface area markings camera parameters are estimated using the 5-point algorithm
(Type 2), and spherical volumes (Type 3). (Nistér, 2004), followed by triangulation of the matched
Type 1 defects were designed to simulate cracking and crack- features using the polynomial method (Hartley and Sturm,
like defects. Line thicknesses of 0.7 mm, 1 mm, and 3 mm widths 1997). Subsequently, new images are added by using the
were applied, with lines varying in length from 12.7 to 152.4 correspondences between 3D points and image features through
mm for each width. The sets of lines were applied using both the Perspective n-Point (PnP) algorithm with RANSAC and
black and white chalk on the sides of Bay #5 (Figures 5A,B). Gauss-Newton optimization (Lepetit et al., 2009).
Type 2 defects were designed to simulate localized area defects After the orientation of each image, bundle adjustment (a
such as concrete efflorescence, or staining due to corrosion. nonlinear least-squares problem) is performed to minimize
These defects took the form of a series of square chalk markings the sum of re-projection errors using the Levenberg-Marquardt
(Figure 5A) with dimensions varying from 645 to 1,6129 mm2 . algorithm (Wu et al., 2011). In this process intrinsic camera
Type 3 defects simulated volumetric changes, such as concrete parameters matrix, K, along with the pose of each particular
spall off. As physically removing portions of the dam structure camera described by rotation, R, and the position of its optical
was not permissible, volumes were instead temporarily added center, C, as well as the positions of the 3D points X are optimized
to the structure. Three Styrofoam hemispheres, with diameters simultaneously.
of 127, 203.2, and 304.8 mm (Figures 5C,D) were painted and XX
textured to have the visual appearance of concrete, and placed on kxij − (Kj Rj (Xi − Cj ))k2 → min (1)
the lateral brace of Bay #5. Kj ,Rj ,Xi ,Cj
j i∈j

2.4. Laser Scanning Where i ∈ j indicates that the point Xi is visible in image j,
For comparative purposes, a phase-shift based Faro Focus3D laser and xij denotes the projection of 3D points Xi onto image j.
scanner was used to collect data from downstream face of the This procedure is repeated until an orientation is available for all
dam. The quality of the data collection was set to 6x in order images within each imaging network. The result of this pipeline is
to reduce the noise in the scan data and thus increases the scan a relatively sparse set of 3D points, due to only utilizing extracted
quality. Due to placement limitations, it was not possible to feature points in the 3D reconstruction. For this study, Agisoft
capture the upstream face of the dam with the scanner. A total of PhotoScan software (Agisoft, 2014) was used to register images
9 scans was collected and merged using the Faro SCENE software. and generate sparse 3D point clouds.
In order to densify the reconstructed models and produce a
3. 3D POINT CLOUD GENERATION model dense enough to capture small geometric changes, multi-
view stereo algorithms (Seitz et al., 2006) are used to capture
From the available algorithms for image-based 3D reconstruction information from all pixels in the input 2D images. In this work,
techniques, in this work a variant of the Structure-from-Motion the Semi-Global Matching (SGM) algorithm (Hirschmüller,
(SfM) process was chosen. SfM is based on the simultaneous 2008) was used to perform pairwise dense matching, as it has

Frontiers in Built Environment | www.frontiersin.org 5 July 2018 | Volume 4 | Article 31

Khaloo et al. UAV Dam Inspection

FIGURE 2 | Flight path for fixed wing UAV.

FIGURE 3 | Orthomosaic generated using the fixed wing imagery dataset.

Frontiers in Built Environment | www.frontiersin.org 6 July 2018 | Volume 4 | Article 31

Khaloo et al. UAV Dam Inspection

FIGURE 4 | Camera poses using oblique imagery dataset captured by rotary UAVs (downstream face).

FIGURE 5 | Controlled simulated damages: (A) 2D image of Type 1 and 2 defects; (B) 3D point cloud rendering of Type 1 and 2 defects; (C) 2D image of Type 3
volumetric defects; (D) 3D point cloud rendering of Type 3 volumetric defects.

Frontiers in Built Environment | www.frontiersin.org 7 July 2018 | Volume 4 | Article 31

Khaloo et al. UAV Dam Inspection

been shown to provide a high level of point density relative to that allows for higher resolution and emphasis in critical regions
other methods utilized for densification (Remondino et al., 2014; of structures. This approach also increases the rate of image
Khaloo and Lattanzi, 2016). The SGM algorithm uses a pixel- registration during the SfM process, thereby improving model
wise matching cost for compensating radiometric differences accuracy, resolution and completeness.
of registered images within the depth map estimation. Later,
individual depth maps are merged together through depth map 3.1. Brighton Dam Model Generation
fusion technique to generate a single, globally consistent 3D For this process to work properly there must be a global point
representation (Fuhrmann and Goesele, 2011). In this work, cloud that captures the overall geometry of the structure, and
SURE (SUrface REconstruction from imagery) (Rothermel et al., into which the other point clouds are merged through ICP. In
2012) software was used, which implements an adaptation of this work, this geometry network was captured using the fixed
the SGM algorithm that reduces processing time with low wing UAV. The second key criterion is that the standoff distances
memory consumption. This software is based on Libtsgm, which between merged point clouds cannot vary excessively. This was
prepares a core functionality for image rectification, dense accomplished by capturing images at varying standoff distances
matching, image triangulation and a user interface for C/C++ (range from 2.5 to 10 m).
programming.
The HPCG approach used in this study is designed for 4. CASE STUDY RESULTS
seamlessly matching and integrating images with different scales,
viewpoints, and cameras into one single reconstruction (Khaloo The images collected on Day One and processed using the
and Lattanzi, 2016). The HPCG process begins by first generating HPCG technique yielded a point cloud of 1,469,690,005 points
3D point clouds separately for each image network, using the (Figure 6). The Day Two mission resulted in a model with
aforementioned photogrammetric process. The point clouds 997,799,119 points.
generated for each network are then merged together into a single
model using the Iterative Closest Point (ICP) algorithm (Besl and 4.1. Point Cloud Quality Analysis
McKay, 1992). The ICP approach refines the alignment assuming Four metrics were used to assess the generated point clouds:
an initial coarse registration of the 3D models is provided. This (i) local noise level, (ii) local point density, (iii) dimensionality
initial alignment can be performed through pair-wise matching descriptors, and (iv) ability to resolve the controlled synthetic
of 10 manually selected point correspondences in order to reduce defects applied on mission Day Two.
the inherent sensitivity of the ICP algorithm regarding the initial The noise level in a point cloud is defined as the residual
positions of the 3D models. between each point and the best fitting plane computed on its
The rigid transformation between two sets of corresponding
3D point sets X = {x1 , x2 , ... , xN } and Y = {y1 , y2 , ..., yN }
extracted through utilizing kd-trees (Friedman et al., 1977; Muja
and Lowe, 2014), can be formulated as the solution of the least-
squares problem:

N
X
(R, t) = arg min k(RXn + t) − yn k2 (2)
R,t
n=1

Where rotation matrix R and translation vector t can be derived

by arranging the point in two 3 × N matrices X̄ and Ȳ that have
x̄n and ȳn as columns:

N N
1 X 1 X
x̄n = xn − xn , ȳn = yn − yn (3)
N N
n=1 n=1

By computing the Singular Value P Decomposition (SVD) of the

3×3 covariance matrix X̄ Ȳ T , U V T = SVD(X̄ Ȳ T ), the optimal
R and t are given by:
 
1 N N
 UT , 1 X 1 X
R=V 1 t= yn − R xn
N N
det(VU T ) n=1 n=1
(4)
These rotations and translations are then used to align the
FIGURE 6 | Generated 3D Point cloud of the Brighton Dam: (A) Downstream
individual point clouds for each network and form the complete face; (B) Upstream face.
global model. The result is a multi-scale/resolution 3D model

Frontiers in Built Environment | www.frontiersin.org 8 July 2018 | Volume 4 | Article 31

Khaloo et al. UAV Dam Inspection

local k-sized nearest neighbours (k-NN). Digital image noise, The second metric, local point density, is a characteristic
radiometric parameters, nonconformity of the neighbourhood analogous to pixel resolution in 2D digital images. It corresponds
of a point, imperfect data registration, and DSfM reconstruction to the ability of a 3D point cloud to resolve structural defects
inaccuracies can all affect this characteristic. and small-scale details. It also provides insight in the variations
This noise residual is defined as the lowest valued eigenvalue in point cloud quality that stem from the multi-UAV imaging
of the covariance matrix for a local neighborhood around a point approach. The local density ρ at a point pi is defined as
in a cloud. For the k points that form the neighborhood of a 3D ρ = 4 k+1 3 , where rk−NN is the radius of the spherical
3 π rk−NN
point pi in a point cloud, the 3×3 covariance matrix, C, is defined
neighbourhood of the k closest neighbours of a 3D point. In this
as (Hoppe et al., 1992):
study, rk−NN was set to 0.62 cm to achieve an enclosing volume
k k of 1 cm3 .
1X 1X
C3×3 = (pi − p̄)T (pi − p̄); p̄ = pi (5) For a given 3D point pi and its k closest neighbourhood, the
k k derived eigenvalues (λ2 ≥ λ1 ≥ λ0 ) using the described method
i=1 i=1
can directly be exploited to estimate the order/disorder of 3D
where p̄ is the arithmetic mean within pi ’s neighborhood
points within the local neighbourhood (Weinmann et al., 2014).
(Npi ) and C represents a symmetric positive-definite matrix.
In order to achieve this, a series of local features were defined to
By performing Singular Value Decomposition (SVD) on the
characterize the geometrical representation of pi and its k-NN as
covariance matrix, it is possible to compute the eigenvectors
linear (1D), planar (2D), or volumetric (3D) features.
V (v2 , v1 and v0 ) and their corresponding eigenvalues λ (λ2 ≥
Dimensionality features represent by linearity Lλ , planarity
λ1 ≥ λ0 ). Within this approach v0 approximates the point’s pi
Pλ , and scattering Sλ can be defined as:
normal, while λ0 quantitatively describes the variation along the
normal vector to provide an estimation of the local noise level
e2 − e1
(i.e., roughness). In addition, the normalized surface variation Lλ = (6)
or change of curvature (σ ) can be defined as λ2 +λλ10 +λ0 , which is e2
invariant under rescaling (Pauly et al., 2002). For this study a e1 − e0
Pλ = (7)
value of k=30 nearest neighbour points was empirically specified e2
for roughness calculations (Figure 7). Utilizing k-NN rather than e0
Sλ = (8)
the fixed distance neighbourhood (FDN) avoids the problem of e2
point density variation within point clouds (Samet, 2006; Khaloo 6λ = e2 + e1 + e0 (9)
and Lattanzi, 2017).
Although surface variation (σ ) can be used to estimate the Where e2 , e1 and e0 are the normalized eigenvalues
change in curvature, as defined by Pauly et al. (2002), for derived by dividing the three eigenvalues λ2 , λ1 and λ0
local shape analysis other fundamental quantities defined in by their sum 6λ . By choosing e2 as the denominator, the
differential geometry such as local Gaussian, mean and principal dimensionality features can be considered as the probabilities
curvature values were taken into account (Crosilla et al., 2009). (since Lλ + Pλ + Sλ = 1) of each point to be labelled as 1D,

FIGURE 7 | Estimated noise level (lowest eigenvalue λ0 ) for Bay#5; low to high noise level ranges from dark to light intensity.

Frontiers in Built Environment | www.frontiersin.org 9 July 2018 | Volume 4 | Article 31

Khaloo et al. UAV Dam Inspection

2D, or 3D (Weinmann et al., 2014). These features can be quality analysis for the Bay #5 region of the dam are shown in
used as metrics to evaluate the quality of the point clouds Figure 9 and Table 2.
in reconstructing geometrically accurate models. Figure 8 These results highlight the advantages of using a UAV for
illustrates a representation of the aforementioned dimensionality point cloud generation. Both HPCG models had over twice
features for Bay#5. Since this part of the dam consists of the density and mean roughness values less than half those
planar surfaces, it was expected to have higher values of Pλ in of the laser scanner. These performance improvements can be
comparison with other dimensional features, indicating with a attributed to the ability of the UAV to capture high-resolution
high probability that the majority of the points can be labeled as images at the higher elevations of the bay, whereas the laser
2D features (Figure 8). Furthermore, the sum of the eigenvalues scanner could only scan those areas from the ground. Compared
6λ can represent the total amount of variation in all three to the overall point cloud models, the results for the HPCG
directions, an additional evaluation of the level of noise in the models are more similar for Bay #5.
3D point clouds (Weinmann et al., 2014). Table 3 presents the analysis of the local dimensionality
The last evaluative metric was to measure the defects applied features in the Bay#5 region of the model. These results
to Bay #5 during the second mission day, in the resulting 3D indicate that, by utilizing the multi-scale HPCG approach in
point cloud. Both Type 1 and 2 simulated damages were manually tandem with smaller quadcopter to capture image data from
measured from the point cloud models and the results was inside of the targeted part of the dam, it was possible to
compared against the field measurements. In order to lessen reconstruct the planar surfaces with a higher accuracy (higher
the error, the Euclidean distance measurement for each pairwise planarity mean value). The Generalized Extreme Value (GEV)
point selection was repeated 10 times and the average values probability density function (PDF) was determined to be the
were compared against the ground-truth values measured in the most accurate representation of the dimensionality features
field. distribution characteristics and the presented mean values are
Volumetric defects (Type 3) were measured by using the direct based on the estimated GEV parameters.
cloud-to-cloud (C2C) distance estimation technique (Girardeau- In addition, similar to both mean roughness and change of
Montaut et al., 2005) to track and quantify geometrical changes curvature values, the total variation metric also illustrates a lower
in the already registered 3D point clouds. Upon finding the noise level in the image-based model generated using the data
closest point correspondences in the two registered clouds, the from April 2017.
Hausdorff distance (Huttenlocher et al., 1993) was utilized to find In order to assess how image resolution affected model quality,
the distance between points in the first dataset and their closest the images captured via UAV during the April mission were
points in the second dataset. Given two finite point sets P = downsampled to 50, 25, and 12.5% of the original image size
{p1 , p2 , ..., pn } and Q = {q1 , q2 , ..., qm }, the two-sided Hausdorff using the bicubic image interpolation. The downsampled images
distance H(P, Q) is defined as: were then used to regenerate point clouds of Bay #5. The resulting
model assessments are shown in Table 4.
H(P, Q) = max(max min kp − qk, max min kq − pk) (10) With a loss of image resolution, the density of the models
p∈P q∈Q q∈Q p∈P
was reduced, as expected. This reduction did not scale linearly
Note that this notation of distance is purely geometric and does with the number of pixels, but quadratically. Notably, the average
not make any assumptions on the uniformity of the point cloud cloud roughness increased with a reduction in image resolution,
density. By using the Hausdorff distance method, it was possible as can be seen in the images disparity maps (Figure 10). This
to automatically estimate the radius of the hemispheres used suggests that the reduced number of feature points used for
in experimental testing, which was then used to estimate their SfM reconstruction along with excessive noise within the depth
volumes. This measurement process was not intended to serve as map estimation and fusion (Figure 10) affected the accuracy and
a general-purpose volumetric analysis technique. quality of the resulting dense 3D point clouds.
Figure 11 illustrates the changes in curvature values (σ ) for
4.2. Point Cloud Analysis Results models generated using different images resolutions. By using
The results of the point cloud quality analysis are shown lower resolution images, the surface variation became more
in Table 1. Model density from the January UAV flights was evenly distributed in the generated 3D models, which indicates
substantially higher than those generated through TLS or during higher local noise level. For instance, the curvature values for the
the April flights. The differences between the two HPCG models model based on the full resolution (16 MP) images were more
is likely due to the change in sensor resolution between mission skewed toward lower values; in contrast, the model based on the
days and changes in the flight protocols, as the April flights downsampled (2 MP) images had a relatively even distribution
focused on Bay #5. Average roughness and curvature values across curvature values, consequently resulting in a higher mean
for the three models are all of a similar order of magnitude, value (Table 4).
indicating relatively similar 3D geometric accuracy. As can be To further compare the quality of the generated 3D models,
seen, the point cloud generated by the laser scanner consists the dimensionality features were again utilized (Table 5). A
of significantly fewer points than the models generated through decrease in image resolution corresponded to a reduction in
HPCG. This was due to the limited options for scanner planarity values, which suggests reduced geometric accuracy. The
placement, particularly with respect to the higher elevation increase in the total variation also illustrated the excessive noise
regions of the dam facade and the upstream facade. Point cloud levels in the models generated using lower resolution images.

Frontiers in Built Environment | www.frontiersin.org 10 July 2018 | Volume 4 | Article 31

Khaloo et al. UAV Dam Inspection

FIGURE 8 | Illustration of the three dimensionality features for Bay#5. Color bar represents the normalized value of each descriptor.

TABLE 1 | Point cloud quality analysis.

3D model Number of points Local point density Mean roughness Mean change
(points/cm3 ) (mm) of curvature

TLS (April 2017) 63,688,823 26.97 2.22 0.026

HPCG (January 2017) 1,469,690,005 53.94 1.47 0.029
HPCG(April 2017) 997,799,119 29.58 1.01 0.023

FIGURE 9 | Local point density for Bay#5; low to high density ranges from dark to light intensity: (A) Day One dataset; (B) Day Two dataset.

TABLE 2 | Point cloud quality analysis for Bay #5.

3D model Number of points Local point density Mean roughness Mean change
(points/cm3 ) (mm) of curvature

TLS (April 2017) 28,356,650 20.81 2.21 0.036

HPCG (January 2017) 108,286,272 56.19 0.92 0.026
HPCG (April 2017) 159,579,984 69.31 0.58 0.018

It is notable that there was a correlation between the local such as edges, where an increase in local noise levels are most
noise level indicators (such as total variation and roughness) likely to occur (see Figures 7, 8). The previous analysis reported
and the scattering (Sλ ) dimensionality feature. Scattering can in Table 3 also indicated the same relation between these
be used to label the points located on higher order surface metrics.

Frontiers in Built Environment | www.frontiersin.org 11 July 2018 | Volume 4 | Article 31

Khaloo et al. UAV Dam Inspection

TABLE 3 | Point cloud quality analysis for Bay #5 using mean values of dimensionality features.

3D model Number of points Linearity Planarity Scattering Total variation

TLS (April 2017) 28,356,650 0.1864 0.5702 0.1071 1.0180e-04

HPCG (January 2017) 108,286,272 0.3113 0.6875 0.0425 2.2326e-05
HPCG (April 2017) 159,579,984 0.2559 0.7533 0.0359 1.1052e-05

TABLE 4 | Bay #5 3D point cloud generation using various image resolutions.

Image resolution Number of points Local point density Mean roughness Mean change
(megapixel) (points/cm3 ) (mm) of curvature

16 MP 159,579,984 69.31 0.58 0.018

8 MP 49,111,922 17.78 1.34 0.032
4 MP 12,820,420 4.71 2.41 0.047
2 MP 3,841,665 1.38 4.33 0.104

FIGURE 10 | (A) Original input image; (B) depth map reconstructed for the 16 MP image; (C) depth map reconstructed for the 8 MP image; (D) depth map
reconstructed for the 2 MP image. Each depth value encodes the distance from the camera center to the geometry.

Figure 12 shows the planarity (Pλ ) feature for the generated measurements were possible. However, as image resolution
models based on different image resolutions. Using lower decreased, measurement accuracy decreased. For models using
resolution images resulted in a lack of accuracy in generating the 4 MP or less in resolution, most flaws were not resolved at
relatively planer surfaces in Bay#5. all in the models and so measurements were not possible.
None of the models were able to reconstruct the 0.7 mm
4.3. Evaluation of Flaw Resolving flaws.
Capabilities These results suggest that a point density somewhere between
Tables 6, 7 summarize the comparisons between the ground- 18 points/m3 and 69 points/m3 was necessary to guarantee the
truth dimensions of the Type 1 defects and the dimensions reconstruction of Type 1 flaws in the model. During these tests,
measured in the point cloud. For the 1 and 3 mm defect the standoff distance from the UAV to the defects was held at
widths, the point clouds generated using 16 MP images were approximated 2.5 m. Given the 16 MP image resolution and the
able to resolve all of the flaws to a degree that highly accurate sensor dimensions of the DJI Phantom 4 camera, each pixel in the

Frontiers in Built Environment | www.frontiersin.org 12 July 2018 | Volume 4 | Article 31

Khaloo et al. UAV Dam Inspection

FIGURE 11 | Comparison of curvature values in models generated using different image resolutions.

TABLE 5 | Bay #5 model analysis using mean values of dimensionality features.

Image resolution Number of points Linearity Planarity Scattering Total variation

(megapixel)

16 MP 159,579,984 0.2559 0.7533 0.0359 1.1052e-05

8 MP 49,111,922 0.2325 0.7272 0.0399 3.6032e-05
4 MP 12,820,420 0.2426 0.6962 0.0666 1.3787e-04
2 MP 3,841,665 0.2458 0.5869 0.1645 5.2111e-04

FIGURE 12 | Comparison of planarity values in models generated using different image resolutions.

Frontiers in Built Environment | www.frontiersin.org 13 July 2018 | Volume 4 | Article 31

Khaloo et al. UAV Dam Inspection

TABLE 6 | Measurement accuracy of 1 mm width type 1 defects at various image images corresponded to approximately 1 mm in the plane of the
resolutions. defects.
Ground truth defect length Length measured in point cloud (mm) The measurements for Type 2 defects are shown in Table 8.
(mm) For these defects, the models derived from either the 16 MP
16 MP 8 MP 4 MP 2 MP (69 points/m3 ) or the 8 MP (18 points/m3 ) images were able
to accurately resolve the flaws. However the 4 MP and 2 MP
12.7 13 – – –
image models were not able to consistently resolve defects. This
25.4 27 23 – –
corresponds to a minimum pixel size of 1.5 mm in the plane of
76.2 73 72 – –
the defect.
152.4 150.2 142 – –
The measurement results for the Type 3 volumetric defects are
shown in Figure 13 and Table 9. For this test, all levels of image
resolution were able to generate models that captured all three
TABLE 7 | Measurement accuracy of 3 mm width type 1 defects at various image
volumetric changes. However, the reduction in image resolution
resolutions. resulted in systematic under prediction of volume measurements.
Ground truth defect length Length measured in point cloud (mm) 4.4. Limitations and Sources of Error
(mm)
16 MP 8 MP 4 MP 2 MP The process used to generate the point clouds has two key
potential sources of error. The first is the possibility of
12.7 13 10 – – misalignment of cameras at the image matching stage. The
25.4 27 23 – – second is possible misalignment of individual networks models
76.2 73 72 68 – during ICP registration. The quality and comprehensiveness
152.4 151 142 139 – of the point clouds was also impacted by the inability of the
UAV to access certain regions of the structure. Specifically,

TABLE 8 | Measurement accuracy of type 2 defects at various image resolutions. TABLE 9 | Measurement accuracy of Type 3 volumetric defects at various image
resolutions.
Ground truth defect Size measured in point cloud (mm × mm)
length (mm × mm) Ground truth defect Volume measured in point cloud (mm3 )
16 MP 8 MP 4 MP 2 MP length (mm3 )
16 MP 8 MP 4 MP 2 MP
25.4 × 25.4 26 × 25 26 × 24 – –
536,265 430,145 367,809 190,852 166,519
50.8 × 50.8 51 × 50 52 × 51 – –
2,196,540 2,113,300 2,032,190 18,552,990 1,526,810
76.2 × 76.2 76 × 76 76 × 76 62×70 –
7,413,330 7,210,900 7,068,580 6,65,2890 6,253,830
101.6 × 101.6 100 × 100 97 × 100 98×93 –
127 × 127 127 × 127 125 × 125 121 × 125 – Mean error 8.8% 14.5% 30.1% 38.4%

FIGURE 13 | Volumetric 3D change analysis.

Frontiers in Built Environment | www.frontiersin.org 14 July 2018 | Volume 4 | Article 31

Khaloo et al. UAV Dam Inspection

certain interior regions and underwater sections of the dam these specifications resulted in point clouds capable of rendering
were not imaged, and thus not included in the point cloud. It millimeter scale details, while lower density models generated
is recognized that both of these areas would be critical for any with larger pixel sizes were often unable to resolve the artificial
inspection and assessment, and other methods of 3D imaging, defects.
such as handheld photogrammetry or laser scanning could be Future work seeks to study how to merge other sources
used in these circumstances. Furthermore, photogrammetric of 3D point clouds, such as laser scanning and hand-
reconstruction techniques are sensitive to severe changes in held photogrammetry, with the model generated via UAV.
lighting and occlusions. In this case study, UAV flight operations Furthermore, using AUV as a platform to collect imagery dataset
were restricted to minimize this effect, and further study on the from underwater sections of the dam can certainly improve
impact of radiometric changes on model quality are warranted. the overall completeness of the generated 3D model and has
Lastly, it is worth noting that the large number of points the potential for future investigation. Additionally, the process
generated through this process (numbered in the billions) should be validated for a variety of infrastructure material
inhibited rendering and visualization. Out-of-core memory types, and under varying radiometric conditions, to assess the
processes that decouple rendering efforts from the scale of the reliability and consistency of the presented UAV inspection
data in order to overcome memory limitations, are recommended approach.
to allow higher rendering frame rates and higher rendering
quality during user interactions. AUTHOR CONTRIBUTIONS

5. CONCLUSIONS AK, DL, AJ, and CD conceived and designed the experiments.
AK developed the algorithms, performed the experiments, and
This study highlights the potential of using a combination of analyzed the data. All authors reviewed and accepted the final
UAV and photogrammetry for the inspection and assessment of version.
dam infrastructure. Ultimately, the goal was to generate models
with sufficient density and quality to resolve a variety of critical FUNDING
inspection details at the millimeter scale. The mission protocols
specified to achieve this were flight path planning that guaranteed This material is based upon the work supported by the National
sufficient image overlap at multiple standoff distances, as well Science Foundation (NSF) under Grant No. CMMI-1433765.
as a minimum standoff distance that corresponded to a pixel Any opinions, findings, and conclusions, or recommendations
size of 1 mm, to guarantee reconstruction of defects at that expressed in this publication are those of the authors and do not
scale. The assessment of the resulting models indicates that necessarily reflect the views of the NSF.

REFERENCES Fischler, M. A., and Bolles, R. C. (1981). Random sample consensus: a paradigm for
model fitting with applications to image analysis and automated cartography.
Agisoft (2014). Agisoft Photoscan. Professional Edition. Commun. ACM 24, 381–395. doi: 10.1145/358669.358692
ASCE (2017). 2017 Infrastructure Report Card. Technical report, American Society Friedman, J. H., Bentley, J. L., and Finkel, R. A. (1977). An algorithm for finding
of Civil Engineers, Reston, VA. best matches in logarithmic expected time. ACM Trans. Math. Softw. 3, 209–
Berberan, A., Ferreira, I., Portela, E., Oliveira, S., Oliveira, A., and Baptista, B. 226. doi: 10.1145/355744.355745
(2011). “Overview on terrestrial laser scanning as a tool for dam surveillance,” Fuhrmann, S., and Goesele, M. (2011). Fusion of depth maps with multiple scales.
in 6th International Dam Engineering Conference, (Lisboa: LNEC). ACM Trans. Graph. 30, 148. doi: 10.1145/2070781.2024182
Besl, P. J., and McKay, N. D. (1992). “Method for registration of 3-d shapes,” in Gaiani, M., Remondino, F., Apollonio, F. I., and Ballabeni, A. (2016).
Robotics-DL Tentative, ed P. S. Schenkar (Boston MA: International Society for An advanced pre-processing pipeline to improve automated
Optics and Photonics), 586–606. photogrammetric reconstructions of architectural scenes. Remote Sens.
Buffi, G., Manciola, P., Grassi, S., Barberini, M., and Gambi, A. (2017). Survey 8:178. doi: 10.3390/rs8030178
of the ridracoli dam: Uav–based photogrammetry and traditional topographic Ghahremani, K., Khaloo, A., and Lattanzi, D. (2016). “Automated 3d image-based
techniques in the inspection of vertical structures. Geomat. Nat. Hazards Risk section loss detection for finite element model updating,” in 33rd International
8, 1562–1579. doi: 10.1080/19475705.2017.1362039 Symposium on Automation and Robotics in Construction, (Auburn, AL).
Christensen, C. (1997). The Innovator’s Dilemma: When New Technologies Cause Girardeau-Montaut, D., Roux, M., Marc, R., and Thibault, G. (2005). “Change
Great Firms to Fail. Boston, MA: Harvard Business Review Press. detection on points cloud data acquired with a ground laser scanner,” in ISPRS
Colombo, M., and Comi, C. (2016). “Damage analyses of concrete dams subject to WG III/3, III/4, V/3 Workshop Laser Scanning 2005 (Enschede), 1–6.
alkali-silica reaction,” in Conference on Italian Concrete Days, (Rome: Springer), González-Aguilera, D., Gómez-Lahoz, J., and Sánchez, J. (2008). A new approach
481–495. for structural monitoring of large dams with a three-dimensional laser scanner.
Crosilla, F., Visintini, D., and Sepic, F. (2009). Reliable automatic classification and Sensors 8, 5866–5883. doi: 10.3390/s8095866
segmentation of laser point clouds by statistical analysis of surface curvature González-Jorge, H., Puente, I., Roca, D., Martínez-Sánchez, J., Conde, B., and
values. Appl. Geomat. 1, 17–30. doi: 10.1007/s12518-009-0002-4 Arias, P. (2014). Uav photogrammetry application to the monitoring of
D’Altri, A. M., Milani, G., de Miranda, S., Castellazzi, G., and Sarhosis, V. (2018). rubble mound breakwaters. J. Perform. Construct. Facilit. 30, 04014194.
Stability analysis of leaning historic masonry structures. Automat. Construct. doi: 10.1061/(ASCE)CF.1943-5509.0000702
92, 199–213. doi: 10.1016/j.autcon.2018.04.003 Hallermann, N., Morgenthal, G., and Rodehorst, V. (2015). “Unmanned aerial
Fathi, H., Dai, F., and Lourakis, M. (2015). Automated as-built 3d reconstruction systems (uas)–case studies of vision based monitoring of ageing structures,”
of civil infrastructure using computer vision: achievements, opportunities, and in Proceedings of the International Symposium Non-Destructive Testing In Civil
challenges. Adv. Eng. Inform. 29, 149–161. doi: 10.1016/j.aei.2015.01.012 Engineering, (Berlin), 15–17.

Frontiers in Built Environment | www.frontiersin.org 15 July 2018 | Volume 4 | Article 31

Khaloo et al. UAV Dam Inspection

Hartley, R., and Zisserman, A. (2003). Multiple View Geometry in Computer Vision. Pauly, M., Gross, M., and Kobbelt, L. P. (2002). “Efficient simplification of point-
Cambridge, UK: Cambridge University Press. sampled surfaces,” in Proceedings of the Conference on Visualization’02, (IEEE
Hartley, R. I. (1997). In defense of the eight-point algorithm. IEEE Trans. Pattern Computer Society), 163–170.
Anal. Mach. Intell. 19, 580–593. doi: 10.1109/34.601246 Remondino, F., Spera, M. G., Nocerino, E., Menna, F., and Nex, F. (2014). State
Hartley, R. I., and Sturm, P. (1997). Triangulation. Comput. Vis. Image Understand. of the art in high density image matching. Photogram. Record 29, 144–166.
68, 146–157. doi: 10.1006/cviu.1997.0547 doi: 10.1111/phor.12063
Henriques, M. J., and Roque, D. (2015). “Unmanned aerial vehicles (uav) as a Ridao, P., Carreras, M., Ribas, D., and Garcia, R. (2010). Visual inspection of
support to visual inspections of concrete dams” in Second International Dam hydroelectric dams using an autonomous underwater vehicle. J. Field Robot.
World Conference (Lisbon: Laboratorio Nacional De Engenharia Civil). 27, 759–778. doi: 10.1002/rob.20351
Hirschmüller, H. (2008). Stereo processing by semiglobal matching and Ridolfi, E., Buffi, G., Venturi, S., and Manciola, P. (2017). Accuracy analysis of a
mutual information. IEEE Trans. Pattern Anal. Mach. Intel. 30, 328–341. dam model from drone surveys. Sensors 17:1777. doi: 10.3390/s17081777
doi: 10.1109/TPAMI.2007.1166 Rivard, P., Ballivy, G., Gravel, C., and Saint-Pierre, F. (2010). Monitoring of an
Hoppe, H., DeRose, T., Duchamp, T., McDonald, J., and Stuetzle, W. (1992). hydraulic structure affected by asr: a case study. Cement Concrete Res. 40,
Surface Reconstruction from Unorganized Points, New York, NY: ACM. 676–680. doi: 10.1016/j.cemconres.2009.09.010
Huttenlocher, D. P., Klanderman, G. A., and Rucklidge, W. J. (1993). Comparing Rothermel, M., Wenzel, K., Fritsch, D., and Haala, N. (2012). “Sure:
images using the hausdorff distance. IEEE Trans. Pattern Anal. Mach. Intel. 15, Photogrammetric surface reconstruction from imagery,” in Proceedings LC3D
850–863. doi: 10.1109/34.232073 Workshop (Berlin).
Jafari, B., Khaloo, A., and Lattanzi, D. (2017). Deformation tracking in 3d point Sack, D. A., and Olson, L. D. (1995). Advanced ndt methods for evaluating
clouds via statistical sampling of direct cloud-to-cloud distances. J. Nondestruct. concrete bridges and other structures. NDT e Int. 28, 349–357.
Eval. 36, 65. doi: 10.1007/s10921-017-0444-2 doi: 10.1016/0963-8695(95)00045-3
Khaloo, A., and Lattanzi, D. (2016). Hierarchical dense structure-from-motion Samet, H. (2006). Foundations of Multidimensional and Metric Data Structures. San
reconstructions for infrastructure condition assessment. J. Comput. Civil Eng. Francisco, CA: Morgan Kaufmann.
31:04016047. doi: 10.1061/(ASCE)CP.1943-5487.0000616 Seitz, S. M., Curless, B., Diebel, J., Scharstein, D., and Szeliski, R. (2006). “A
Khaloo, A., and Lattanzi, D. (2017). Robust normal estimation and region growing comparison and evaluation of multi-view stereo reconstruction algorithms,”
segmentation of infrastructure 3d point cloud models. Adv. Eng. Inform. 34, in Computer Vision and Pattern Recognition, 2006 IEEE Computer Society
1–16. doi: 10.1016/j.aei.2017.07.002 Conference on, (New York, NY: IEEE).
Khaloo, A., Lattanzi, D., Cunningham, K., Dell’Andrea, R., and Riley, M. Turner, D., Lucieer, A., and Watson, C. (2012). An automated technique for
(2018). Unmanned aerial vehicle inspection of the placer river trail bridge generating georectified mosaics from ultra-high resolution unmanned aerial
through image-based 3d modelling. Struct. Infrastruct. Eng. 14, 124–136. vehicle (uav) imagery, based on structure from motion (sfm) point clouds.
doi: 10.1080/15732479.2017.1330891 Remote Sens. 4, 1392–1410. doi: 10.3390/rs4051392
Lepetit, V., Moreno-Noguer, F., and Fua, P. (2009). Epnp: An accurate o (n) U.S. Dept. Of Homeland Security, Federal Emergency Management Agency
solution to the pnp problem. Int. J. Comput. Vis. 81, 155–166. (2004). Federal Guidelines for Dam Safety. Federal Emergency Management
Lowe, D. G. (2004). Distinctive image features from scale-invariant keypoints. Agency, Washington, DC.
Int. J. Comput. Vis. 60, 91–110. doi: 10.1023/B:VISI.0000029664. Weinmann, M., Jutzi, B., and Mallet, C. (2014). Semantic 3d scene interpretation:
99615.94 a framework combining optimal neighborhood size selection with relevant
Malhotra, V. M., and Carino, N. J. (2003). Handbook on Nondestructive Testing of features. ISPRS Ann. Photogram. Remote Sens. Spatial Inf. Sci. 2, 181.
Concrete Second Edition. Boca Raton, FL: CRC Press. doi: 10.5194/isprsannals-II-3-181-2014
Maryland Department of Environment (1996). Maryland Dam Safety Manual. Wu, C., Agarwal, S., Curless, B., and Seitz, S. M. (2011). “Multicore bundle
Maryland Department of Environment, Baltimore, MD. adjustment,” in Computer Vision and Pattern Recognition (CVPR), 2011 IEEE
Ming-Der, Y., Huang, K.-S., and Hui-Ping, T. (2016). Monitoring and Conference on, (Providence, RI: IEEE).
measurement of an artificial landslide dam using uav images and image- Zekkos, D., Manousakis, J., Greenwood, W., and Lynch, J. (2016). “Immediate
based modeling (mcsse). DEStech Trans. Comput. Sci. Eng. 357–362. uav-enabled infrastructure reconnaissance following recent natural disasters:
doi: 10.12783/dtcse/mcsse2016/10996 case histories from greece,” in International Conference on Natural Hazards and
Muja, M., and Lowe, D. G. (2014). Scalable nearest neighbor algorithms for Infrastructure.
high dimensional data. IEEE Trans. Pattern Anal. Mach. Intel. 36, 2227–2240.
doi: 10.1109/TPAMI.2014.2321376 Conflict of Interest Statement: The authors declare that the research was
Nistér, D. (2004). An efficient solution to the five-point relative pose problem. IEEE conducted in the absence of any commercial or financial relationships that could
Trans. Pattern Anal. Mach. Intel. 26, 756–770. doi: 10.1109/TPAMI.2004.17 be construed as a potential conflict of interest.
Oliveira, A., Oliveira, J. F., Pereira, J. M., De Araújo, B. R., and Boavida, J.
(2014). 3d modelling of laser scanned and photogrammetric data for digital Copyright © 2018 Khaloo, Lattanzi, Jachimowicz and Devaney. This is an open-
documentation: the mosteiro da batalha case study. J. Real-time Image Process. access article distributed under the terms of the Creative Commons Attribution
9, 673–688. doi: 10.1007/s11554-012-0242-0 License (CC BY). The use, distribution or reproduction in other forums is permitted,
Olson, L. D., and Sack, D. A. (1995). “Nondestructive evaluation of concrete dams provided the original author(s) and the copyright owner(s) are credited and that the
and other structures,” in Nondestructive Evaluation of Aging Structures and original publication in this journal is cited, in accordance with accepted academic
Dams, eds S. Nazarian and L. D. Olson (Oakland, CA: International Society practice. No use, distribution or reproduction is permitted which does not comply
for Optics and Photonics), 113–125. with these terms.

Frontiers in Built Environment | www.frontiersin.org 16 July 2018 | Volume 4 | Article 31