https://doi.org/10.36375/prepare_u.iei.

a226

Implementation of Computer Vision technique for Crack

Monitoring in Concrete Structure
Nikhil Kanani*, Paresh V. Patel* and Harsh Kapadiya**
*Department of Civil Engineering, ** Department of Electronics and Instrumentation
Engineering
Institute of Engineering, Nirma University
Sarkhej-Gandinagar Higway, Ahmedabad – 38248
{Corresponding author’s email: [email protected]}
Abstract - Assessment of structural health is strength and warn against possible failure.
essential for safe and efficient functioning of Physical inspection of concrete structures and
built environment. Physical inspection of crack monitoring help in assessment of
structures for its health monitoring is time strength. In alternate to physical inspection,
consuming, costly and risky. Advances in
image capturing and processing techniques as
images of critical locations can be captured by
well as numerical simulation tools have made cameras and processing of images gives data
computer vision a cost effective and accurate about current condition of structures. With the
alternative for structural health assessment. proper interpretation of collected data and
Evolution of convolution neural network (CNN) artificial intelligence, residual strength of
has reduced human effort and made it easy to structural elements can be predicted and if
develop algorithms for identification of required, they can be strengthened for
structure defects. One of the primary defects increased life span of structure. This technique
in concrete is crack. Concrete cracking occurs doesn’t need any equipment rather than visual
due to many reasons like shrinkage, heaving,
premature drying, excessive loading etc. and
input equipment so it is easier and economical
it leads to reduction in strength of structures. to identify cracks. Generally concrete
This paper presents a computer vision system contribute major strength in compression and
developed for crack monitoring of concrete steel reinforcement is used for satisfying
cubes subjected to compressive loading. tension strength requirement of structure.
Camera is used to capture real time images Artificial intelligence (AI) is a broad term for
when concrete cubes are subjected to loading. the simulation of human intelligence in
Images are further processed using CNN to computers. Artificial Intelligence is the capacity
obtain various features of cracks like of computers to think and behave in the same
numbers, location, length, area etc. Present
Computer vision system is developed using
way as humans do. Machine learning and deep
LabVIEW and implemented using tensor learning are two subcategories of AI. Machine
processing unit (TPU) for better computational Learning is the branch of Artificial Intelligence
efficiency. The outcome of present system that allows a machine to learn on its own
demonstrated better and accurate real time without having to be specifically programmed.
monitoring of cracking when concrete is There are a variety of machine learning
subjected to loading. Proposed computer algorithms that evaluate the data and
vision system can be implemented for generate a function to predict an output based
structural health monitors of real-life civil on the new inputs.
engineering structures like buildings and
bridges.
In 2003, Hung and Voloshin [1] developed a
fast and simple (FAS) detection algorithm
Keywords - Artificial intelligence; convolution based on digital image correlation (DIC) for
neural network; computer vision; digital measurement of the surface deformation of
image processing; concrete crack detection; planar objects. The concept of finite element
structural component health monitoring. method (FEM) was applied by Sun et al.[20] to
INTRODUCTION determine the complete, two-dimensional

C oncrete is one of the widely used displacement field during the image correlation
construction material, made of various process on digital images. They used both
cementitious materials and aggregates. There numerical studies and a real experiment to
can be several causes for failure of concrete. verify the proposed formulation and showed
Cracking in concrete indicate deterioration in that the image correlation with the finite
element formulation is computationally
https://doi.org/10.36375/prepare_u.iei.a226

efficient, accurate, and robust. In 2006, ninth of the 607,380 bridges in the U.S. were
Besnard [12] et al. introduced the concept of structurally deficient and required a $20.5
multiscale approach on top of FEM based DIC billion annual investment for fixing the
method to generate meaningful solution for a problems by 2028. While in developing
fine texture and large initial displacement countries like China and India, more civil
measurement. In 2011, Nguyen et al. [13] infrastructure is being built. To efficiently
developed a new automated method of monitor and maintain such a large number of
fracture identification and quantification based existing civil infrastructure is critical yet
on standard DIC approach. An automatic crack challenging for both safety and economic
detection system was proposed by Zhang et al. reasons.
[14] by employing a coarse-to-fine This paper presents application of computer
methodology that also included concepts like vision technique for crack identification in
Region of Aggregation (ROA) and Region of concrete cubes subjected to compressive
Belief (ROB) for segmentation and localization loading. The developed system includes
of cracks. Chambon and Moliard [19] proposed camera for image capturing, CNN for crack
a new approach to image-based crack identification from captured images
detection, GaMM, based on a multi-scale implemented through LabView and TPU. The
extraction and Markovian segmentation, which presented system can be easily implemented
reduced the percentage of false positives in for real time monitoring of concrete structures.
comparison to morphological methods that
combine thresholding and refinement by CRACK DETECTION METHODOLOGY IN PROPOSED SYSTEM
morphological analysis. Xie et al. [15] Deep learning refers to the process by which
demonstrated the potential for Deep Learning computers can imitate human behaviour. They
based pavement crack detection by applying make use of neural networks, which are multi-
ConvNets on a dataset of 500 images of size layered systems inspired by the human brain's
3264  2448, collected using a low-cost structure. One of the most popular image
smartphone. Ying and Salari [16] proposed a recognition algorithms used in deep learning is
beamlet transform-based technique for the convolutional neural network. It takes an
pavement crack detection and classification, image as input and processes it so that
which was more robust in extracting linear different aspects of the image can be
features in the presence of noise. Oliveira and distinguished from one another. They use
Correia [17] proposed an integrated system, artificial neurons, which are mathematical
CrackIT, for automatic detection and functions that measure the weighted number
characterization of cracks in flexible pavement of multiple inputs and outputs an activation
surfaces using a combination of unsupervised value and are modelled after neurons in the
learning (clustering) followed by supervised human brain.
learning (classification), thus eliminating the The conventional image processing method
need for manually labelling the samples. It was does not work well in the detection of cracks in
noted that although CrackIT was able to detect concrete structures as the intensity range of
multiple cracks in the same image, it had cracks and non-cracks in the concrete block is
difficulty in dealing with cracks less than 2 mm almost similar. Further, as the load increases
width. Based on the fact that crack pixels in on concrete specimen, the area of the crack
pavement images had distinct grayscale increases and it merges to become a longer
intensities compared to their surrounding non- crack so, area-based filtering of the non-crack
crack pixels, Cheng et al. [18] proposed a block is quite challenging. CNN improves this
pavement crack detection algorithm based on task by combining the output of different
fuzzy logic. Feng et al.[21] proposed a deep trained convolutional networks in a fusion
active learning strategy for civil infrastructure multi-Layer perceptron and the features
defect detection and classification, where they present in the feature map extracted by the
used a deep residual network (ResNet) to train convolution layer are summarized by the
a small set of images with defect labels and pooling operation.
use this low-accuracy defect detector to filter A. Convolutional Neural Network
out many non-defect images. Convolutional Neural networks, also known as
The ageing civil infrastructure (e.g., tunnels ConvNet or simply CNN, are a deep learning
and bridges) is a common problem in many algorithm, which mainly finds its applications in
developed countries such as the United States visual imagery tasks. It can take an image as
and Japan. According to (ASCE 2013), one an input and can identify and differentiate
https://doi.org/10.36375/prepare_u.iei.a226

various features from it. The Convolutional detection in concrete. A dataset of 3600
Neural networks were inspired by the human images containing cracks and non-cracks with
brain and have a structure analogous to the a dimension of 256  256 pixels was prepared.
neuron connectivity pattern in human brains. The primary network used was a single-
They are composed of multiple layers of channel convolutional neural network (SCNN)
artificial neurons, which are mathematical in which feature extraction was done using
functions that calculate a weighted sum of intermediate max-pooling layers. The model
inputs and outputs a possibility score or was trained with a learning rate of 0.0005 and
activation value. achieved an accuracy of 90.5%. For making
1) Architecture of CNN: The convolutional the model robust, data augmentation was
neural networks have many layers and try to performed using random rotations, shifts and
extract different features from the input image. zooming. This led to a drop in accuracy to
The starting layers try to find basic features 82.25%. For optimizing this model, they
and the complexity increases as we go deeper introduced a second channel, which had
into the network. Each layer has three shallow network structure and skip
dimensions, height, width and depth. connections, thus making the model to be a
2) Convolutional Layers: The convolutional Dual-channel Convolutional Neural Network.
layers try to find the high-level features from The new model was trained with 6400 images
an input image such as edges. These are the at a learning rate of 0.0005 and achieved an
starting layers of a convolutional network and accuracy of 92.25%. The average time taken
there can be more than one convolutional by each epoch was 159.031 seconds.
layer in the network. They start by extracting
the low-level features such as edges, color, EDGE COMPUTATION
etc. and as we go deeper to other Edge computing is changing the way data is
convolutional layers, the network adapts to being
extract more complex features.
3) Pooling Layer: The pooling layer tries to
reduce the spatial size of the convolved
feature to reduce the computational power
required. It also extracts the more dominant
features, that are invariant to rotation and
position. Pooling is of two types, Max Pooling,
and Average Pooling. handled, processed and then delivered from
4) Max Pooling: It returns the maximum value many devices all over the world. The
from the portion of the image covered by the tremendous growth of internet-connected
filter. devices – the IoT – alongside new applications
5) Average Pooling: It returns the average that needs real-time computing power,
value. continues to drive edge-computing systems.
6) Classification Layer: The classification layer, Faster networking technologies, like 5G
also known as the Fully Connected Layer, are wireless, are allowing edge computing systems
the final layers of the convolutional network in to accelerate the creation or support of real-
which the output of the pooling layer acts as time applications, like video processing and
the input and uses a SoftMax function to analytics, self-driving cars, AI and robotics.
calculate the final possibility probabilities of Edge computing simply provides information
the input. Complete CNN architecture is shown processing close to the edge that is data
in Fig. 1. gathered through IoT devices are passed
through the edge computing devices to act like
a gateway where data are interpreted quickly
before being pass through the internet to be
sent to a server or cloud storage for further
processing if necessary or just for data-
keeping. To achieve high-speed data
transmission, google has launched hardware
Fig. 1 Architecture of Convolutional Neural Network called Edge TPU (Tensor Processing Unit) that
can be connected by just using a USB
Kumar and Ghosh [2], used a Ducal Channel connection. TPU is an AI accelerator
Convolutional Neural Network model for crack application-specific integrated circuit (ASIC)
https://doi.org/10.36375/prepare_u.iei.a226

developed specifically for neural network TPU option is selected, then the re-trained
machine learning in the TensorFlow ML library. inception v3 edge TPU compatible model is
used for crack detection purposes. The GUI
PROPOSED COMPUTER VISION SYSTEM FOR CONCRETE also has an option using the crack detection
CRACK MONITORING model in live stream mode in which the images
Fig. 2 shows a complete block diagram of of the concrete block are captured by the
proposed computer vision system camera in real-time and sent to the crack
implemented at Heavy Structures Laboratory detection model along with the real-time load
of Nirma University for compression testing of applied on the concrete block. If live stream
concrete cubes. In this system, for crack mode is not selected then images and load cell
detection model, the previously re-trained data are read from a saved folder on the
Inception v3 model is used along with a server device. LabVIEW analyses different
LabVIEW GUI especially developed for crack characteristics of cracks like the number of
cracks and area of cracks and plots a graph of
Load vs Number of Cracks, Load vs Crack
Length, Load vs Area of Cracks and Load vs
Time. Fig. 4 presents screenshot of proposed
system showing image information like no. of
cracks. Along with crack information, preview
of images is given so that the portion of the
cube to be focused can be changed.

detection. It follows the same methodology but

with a few changes. The input image is read by
the LabVIEW and is
Fig 2. Concept Block Diagram

processed in LabVIEW itself. The input image is

first converted into a greyscale image and then
to a binary image. Contours are found in the
binary image using LabVIEW only. Using the
bounding box coordinates of the found
Fig. 4 Concrete Crack Detection Model Result
contours, different regions of interest are
extracted from the greyscale image. All these
Various graphs produced in the system for
steps are performed on LabVIEW platform.
crack analytics like Load v/s Number of cracks,
After extracting the contours, each contour is
Load v/s Crack length, Load v/s area of crack
now passed to the CNN-based re-trained
and Load v/s time are presented in Fig. 5.
Inception v3 model to classify them as crack or
non-crack and to obtain a prediction score for
each contour. The final output is again
displayed in the LabVIEW GUI with all the
contours found and classified as crack or non-
crack. The complete flowchart of crack
detection using Control panel unit(CPU)+tensor
penal unit(TPU) with NI Labview, python and
Tensorflow is shown in Fig. 3.
Fig. 3 Flowchart of crack detection performed using a
custom CNN model on a CPU + TPU with NI LabVIEW,
Python and TensorFlow
Fig. 5 a) Load vs Number of Cracks, b) Load vs Crack
The LabVIEW Graphical User Interface (GUI) Length, c) Load vs Area of Cracks and d) Load vs Time
developed in this system has the facility to
select whether the model is to be run on CPU Above data obtained from proposed Computer
alone or CPU + TPU. Based on the selection, if Vision System helps in assessment of
CPU alone is selected the re-trained inception structural health of concrete structures.
v3 model with python is used, and if the CPU +
https://doi.org/10.36375/prepare_u.iei.a226

CONCLUDING REMARKS Infrastructure Engineering, 9(6), 567–577.

https://doi.org/10.1080/15732479.2011.593891
The computer vision system proposed in this
[7] Rabah, M., Elhattab, A., & Fayad, A. (2013). Automatic
paper perform repetitive and monotonous concrete cracks detection and mapping of terrestrial
tasks of concrete structure ealt monitoring at a laser scan data. NRIAG Journal of Astronomy and
faster rate and simplifies the work of manual Geophysics, 2(2), 250–255.
https://doi.org/10.1016/j.nrjag.2013.12.002
inspection. Several tasks of health monitoring
[8] Ying, L., & Salari, E. (2010). Beamlet Transform-Based
can be automated without the need for human Technique for Pavement Crack Detection and
intervention. Computer vision systems that Classification. Computer-Aided Civil and Infrastructure
have been trained very well will commit zero Engineering, 25(8), 572–580.
https://doi.org/10.1111/j.1467-8667.2010.00674.x
mistakes in crack detection and result in faster
[9] Rabah, M., Elhattab, A., & Fayad, A. (2013). Automatic
delivery of high-quality outcomes. concrete cracks detection and mapping of terrestrial
When run with LabVIEW in CPU (central laser scan data. NRIAG Journal of Astronomy and
processing unit) + TPU (tensor processing unit) Geophysics, 2(2), 250–255.
https://doi.org/10.1016/j.nrjag.2013.12.002
mode, the best accuracy is achieved with a re-
[10] Mane, A., & Tade, S. (n.d.). Concrete Structure
trained Inception v3 model, updated to Edge Analysis and Classification using Image Processing. In
TPU compatible model, with an accuracy of International Journal of Advanced Computer Research.
93.20 percent. A LabVIEW user interface is [11] Asifullah Khan, Anabia Sohail, Umme Zahoora, and
Aqsa Saeed Qureshi, “A Survey of the Recent
developed in this system. The input images are
Architectures of Deep Convolutional Neural Networks”
first processed by LabVIEW to extract different in Artificial Intelligence Review
contours, and then each contour is analyzed by [12] Besnard G, Hild F, Roux S. “finite-element”
a re-trained Inception v3 model to determine displacement fields analysis from digital images:
application to portevin–le châtelier bands. Exp Mech
whether they are crack or non-crack. LabView
2006;46 (6):789–803.
also included a feature called Camera Mode [13] Nguyen TL, Hall SA, Vacher P, Viggiani G. Fracture
(live-stream crack detection), which uses a mechanisms in soft rock: identification and
camera's live feed to detect cracks in concrete quantification of evolving displacement discontinuities
by extended digital image correlation. Tectonophysics
blocks. Load vs. Number of Cracks, Load vs.
2011;503(1):117–28.
Length of Cracks, and Load vs. Area of Cracks [14] Chen J, Zhang X, Zhan N, Hu X. Deformation
graphs are also available from this system, measurement across crack using two-step extended
which are very useful for health monitoring. digital image correlation method. Opt Lasers Eng
2010;48 (11):1126–31.
Google Coral Edge TPU is used in this system
[15] D. Xie, L. Zhang, L. Bai, Deep learning in visual
to boost the computational power. Based on computing and signal processing, Appl. Comput. Intell.
the comparison of the computational time Soft Comput. 2017 (2017) e1320780,
required to process an image using a CPU and https://doi.org/10.1155/2017/1320780.
[16] L. Ying, E. Salari, Beamlet transform-based technique
a CPU + TPU combination, it is observed that
for pavement crack detection and classification,
the use of TPU greatly reduced the Comput.-Aided Civil Infrastruct. Eng. 25 (2010) 572–
computational time. 580, https://doi.org/10.1111/j.1467-
8667.2010.00674.x.
[17] H. Oliveira, P.L. Correia, Automatic Road Crack
REFERENCES
Detection and Characterization, IEEE Trans. Intell.
[1] Hung P. C, Voloshin A. In-plane strain measurement by
Transp. Syst. 14 (2013) 155–168,
digital image correlation. J Braz Soc Mech Sci Eng
https://doi.org/10.1109/ TITS.2012.2208630
2003;25(3):215–2
[18] Chou, J. C., O’Neill, W. A. & Cheng, H. D. (1994),
[2] Babloo Kumar and Sayantari Ghosh, “Detection of
Pavement distress classification using neural networks,
Concrete Cracks Using Dualchannel Deep
systems, man, and cybernetics, IEEE International
Convolutional Network”, in 11th ICCCNT 2020.
Conference, 1, 397– 401.
[3] Gopalakrishnan, K., Khaitan, S. K., Choudhary, A., &
[19] S. Chambon, P. Subirats, and J. Dumoulin,
Agrawal, A. (2017). Deep Convolutional Neural
"Introduction of a Wavelet Transform Based on 2D
Networks with transfer learning for computer vision-
Matched Filter in a Markov Random Field for Fine
based data-driven pavement distress detection.
Structure Extraction: Application on Road Crack
Construction and Building Materials, 157, 322–330.
Detection," in Proceedings of the IS&T/SPIE Electronic
[4] Talab, A. M. A., Huang, Z., Xi, F., & Haiming, L. (2016).
Imaging Science and Technology Jose, USA, 2009, pp.
Detection crack in image using Otsu method and
72510A-72510A-12
multiple filtering in image processing techniques.
[20] Sun Y, Pang JH, Wong CK, Su F. Finite element
Optik, 127(3), 1030–1033.
formulation for a digital image correlation method.
https://doi.org/10.1016/j.ijleo.2015.09.147
Appl Opt 2005;44(34):7357–63
[5] Bhattacharjee, S., & Deb, D. (2016). Automatic
[21] C. Feng, M.-Y. Liu, T.-Y. Lee, Deep Active Learning for
detection and classification of damage zone(s) for
Civil Infrastructure Defect Detection and Classification,
incorporating in digital image correlation technique.
in: 2017 International Workshop on Computing in Civil
Optics and Lasers in Engineering, 82, 14–21.
Engineering. Presented at the 2017 International
https://doi.org/10.1016/j.optlaseng.2016.01.015
Workshop on Computing in Civil Engineering (IWCCE
[6] Lee, B. Y., Kim, Y. Y., Yi, S. T., & Kim, J. K. (2013).
2017), American Society of Civil Engineers (ASCE),
Automated image processing technique for detecting
Seattle, Washington, 2017
and analysing concrete surface cracks. Structure and