See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/356223402

Detecting Hook Attachments of a Safety Harness Using Inertial Measurement

Unit Sensors

Conference Paper · November 2021

DOI: 10.22260/ISARC2021/0079

CITATIONS READS

2 719

3 authors, including:

Namgyun Kim
University of Dayton
16 PUBLICATIONS 124 CITATIONS

SEE PROFILE

All content following this page was uploaded by Namgyun Kim on 15 November 2021.

The user has requested enhancement of the downloaded file.

 38th International Symposium on Automation and Robotics in Construction (ISARC 2021)

Detecting Hook Attachments of a Safety Harness Using

Inertial Measurement Unit Sensors
Hoonyong Leea, Namgyun Kima, and Changbum Ryan Ahnb*

a
Department of Architecture, College of Architecture, Texas A&M University, Texas, USA
b
* Department of Architecture and Architectural Engineering, Seoul National University, Seoul, South Korea
E-mail: [email protected], [email protected], [email protected]

Abstract warning of the improper use of a safety harness would

Construction workers are required to wear a contribute to change workers’ safety behaviors. One of
safety harness while working at height, and safety the safety manager’s tasks is to frequently monitor
managers need to ensure that a safety hook is workers and site conditions to get real-time data through
attached to proper anchorage points to prevent falls direct observation and interaction with workers [6].
from height. However, it is difficult for the managers Although a safety manager should identify workers who
to monitor all the worker’s hook attachments do not properly use a safety harness in a hazardous zone
continuously and remotely in dynamic workplace (e.g., a roof and top floor), it would be a challenge to
environments. This study developed an approach to monitor all the workers who are working at height from
detect an individual worker’s hook attachments by the ground level, continuously and remotely.
assessing the relative movements between the hook In recent years, advances in sensing technology and
and the worker’s body. An Inertial Measurement machine learning algorithms have enabled safety
Unit sensor was attached to the hook and the body managers to monitor the activity and physical status of
strap to monitor the relative movements. The workers in real-time [7]. Previous studies have applied
collected IMU data was transformed into image data these technical improvements to develop detection
by Markov Transition Field. The detection algorithm systems for the use of a safety harness. A previous study
was developed based on the convolution neural [8] developed an approach that uses an image
networks that classify the worker’s postures, classification algorithm to detect whether a worker wears
activities, and hook attachments simultaneously, and a safety harness. In this study images of workers taken
the developed detection system provided classification with a monocular camera were used, but the quality of
accuracies of 86.40%, 86.97%, and 96.58, respectively. image data is affected by environmental factors (e.g.,
The results validated that the relative movement weather and light) that would degrade the detection
between the hook and the worker’s body is a key performance. In another previous study [9], based on the
feature for hook attachment detection. distance between the hook and the lifeline, it was
detected whether the worker attached the safety hook to
Keywords – the lifeline. This approach used Bluetooth Low Energy
construction safety; safety harness; hook beacons to measure the distance based on the worker’s
attachment detection; wearable computing location. If the distance is less than the threshold, the
safety hook is considered to be attached to the lifeline.
However, a close distance between the hook and the
1 Introduction lifeline does not always guarantee the connection.
Falls from height (FFH) have been identified as a In this context, this study aims to develop a novel
significant source of fatal accidents at construction sites detection approach for the proper use of a safety harness.
[1]. In order to protect workers from FFH, some The sensing sources of the developed approach were
prevention measures have been proposed [2]. For safety hook and worker’s bodily movements, measured
example, the use of a safety harness is required while by wearable Inertial Measurement Unit (IMU) sensors.
working at height. However, workers could often be In this current study, a distinct body movement pattern
reluctant to use a safety harness because of non- according to postures and activities and a distinct hook
compliance and restrictions to movement [3]. Although movement pattern according to attachments were
construction worker education and training are effective assessed. The developed system simultaneously detects
ways to address the reluctance, these are not always the worker’s posture, activity, and hook attachment.
effective entirely [4,5]. Real-time monitoring and Therefore, the main contribution of this study is to detect

583
 38th International Symposium on Automation and Robotics in Construction (ISARC 2021)

the proper use of a safety harness while undertaking hazards that generated abnormal gait patterns [17].
various construction tasks at the workplace. Because IMU-based monitoring systems directly record
the worker’s bodily movement, their performances are
less affected by environmental factors (e.g., light or
2 Background weather). However, the bodily movements could be
different for each worker and may vary depending on the
2.1 Wearable Sensors in Construction worker’s physical status, which may cause performance
Worker Safety variations depending on training data.
Various sensor technologies have been used to
improve the safety of workers on construction sites. For 2.2 Monitoring Use of Safety Harness in
example, Real-Time Locating Systems (RTLS) have Construction
been implemented by Radio-Frequency Identification A previous study [8] developed an approach to detect
(RFID) and Bluetooth Low Energy (BLE) technologies whether workers are wearing their safety harnesses using
[10]. These systems consist of a transmitter and several an image classification algorithm. The developed
receivers. The transmitter is attached to a worker while approach has two phases: (1) worker presence detection
transmitting radio signals with an identification number, and (2) safety harness identification. Although this
and the receivers are attached to a moving object, PPE, approach provided 99% and 80% precision performance
or located in hazardous areas. RTLS measures the current on phases 1 and 2, respectively, this approach did not
location of the worker based on the distance between the detect hook attachments. Even if a worker wears a
transmitter and the receiver. One parameter for harness, the worker may not properly use the safety
calculating the distance is the Received Signal Strength harness. For example, the safety hook would be attached
Indicator (RSSI), which measures the attenuated power to the worker’s body or placed on the ground. Therefore,
at the receiver. For construction safety management, it is necessary to monitor not only wearing a safety
RTLS has been used to warn workers when entering harness but also properly using the safety harness.
hazardous zones (e.g., a roof and top floor) or Another previous study [9] developed a system
approaching dangerous moving objects (e.g., heavy detecting the proper use of a safety harness using BLE
machines) [11]. RTLS can warn workers of danger even technologies. This system detected whether the safety
in blind spots because radio signals can penetrate or hook is attached to the lifeline hook according to the
reflect from some obstacles to reach the receivers in non- distance between the lifeline hook and the worker who
line of sight environments [12]. Additionally, the needs to attach the safety hook. Once the worker attaches
transmitter can be attached to PPE, such as a safety the safety hook to the lifeline hook, the worker’s location
helmet or harness to monitor whether individual workers would be identical to the lifeline hook. A BLE receiver
wear PPE in the workplace [9]. However, since RFID and was attached to the worker’s safety hook and a BLE
BLE beacons have a limited coverage area and signal beacon was attached to the lifeline hook. The distance
propagation can be affected by environmental factors, the between these BLE devices was calculated based on
accuracy can decrease as the distance between beacons RSSI. Another BLE beacon was located in the hazardous
increases. zone where the worker must attach the safety hook to the
Physical response measurement systems have been lifeline. The third BLE beacon was placed at an interval
also implemented to improve worker safety management. of 2m, where the working began at height. Due to the
IMU sensors have frequently been used to assess workers’ limited coverage area of the BLE beacons, a distance
physical changes while undertaking construction tasks. A between 1 and 2 m was required between beacons.
typical IMU sensor consists of an accelerometer and a Although this system was validated in a field experiment,
gyroscope. The IMU sensor is attached to the worker’s this approach had some practical limitations. Multiple
body part and measures the movement of the body part BLE beacons are required to cover the space, and those
in three-axis acceleration and angular velocity. Most beacons need to be relocated when the working
construction tasks require physical demands without environment changes. Also, the distance-based detection
sufficient rest, which can lead to work-related approach would produce false detection if the worker is
musculoskeletal disorders (WMSDs) [13]. Therefore, working near the lifeline without attaching the safety
measuring a worker’s physical response to repetitive and hook to the lifeline.
prolonged construction tasks would help prevent In this context, this study developed a new detection
overexertion injuries. Measured bodily movements were system for the proper use of a safety harness. This study
used to detect awkward postures [14,15], excessive load measured the hook and the bodily movements using IMU
carrying that produced distinct patterns of bodily sensors attached to the hook and body strap and found
movements [16]. Gait kinematics were also measured by that the hook movement is affected by both the worker’s
IMU sensors to assess exposure to slip, trip, and fall (STF) bodily movement and hook attachment points (e.g.,

584
 38th International Symposium on Automation and Robotics in Construction (ISARC 2021)

attaching to a rigid structure and the worker’s body or acceleration and angular velocity data along three axes at
placing on the ground). Therefore, the hook attachment a 50 Hz sampling rate. Figure 2 shows the IMU data
can be detected by assessing the relative movements collected while moving bricks by kneeling—(a), (b), and
between the worker’s body and the safety hook. (c) —and standing—(d), (e), and (f). For each hook
attachment point, the hook IMU data show a distinct
pattern, whereas the back-worn IMU data show a very
3 Methodology similar pattern for the same posture. Moreover, different
postures generate different patterns of hook IMU data
3.1 Data Collection even for an identical hook attachment point, (see Figure
2(b) and (d)). Therefore, the hook IMU data depended on
Five subjects participated in the experiment to collect
IMU data of the safety hook and bodily movements while both the attachment point and the bodily movement
performing different activities: (1) walking, (2) moving related to activity and posture.
bricks, and (3) using a drill machine. While performing This study assessed the unique relative movement
the activities, the safety hook was attached at several between the hook and the subject’s body to detect hook
points: (1) attached to scaffolding, (2) attached to the attachments in various postures and activities.
body strap (chest), and (3) placed on the ground. Also,
moving bricks and using a drill machine were performed
by two postures: (1) standing and (2) kneeling. Therefore,
18 cases of relative movements between the hook and the
body were collected from each subject (2 postures, 3
activities, and 3 attachment points). Figure 1 shows an
example of moving bricks while standing with a
scaffolding attachment. The subjects performed each
activity repeatedly for 3 minutes. They did not change
their locations while moving bricks and using a drill
machine, but randomly changed locations when walking.
While the hook was always attached to the chest for a
body strap attachment, the subjects attached the safety
hook to various parts of the scaffolding.
During the experiment, an IMU sensor was attached
to the safety hook and the body strap (back), indicated by Figure 1. Moving bricks with a scaffolding
blue circles in Figure 1. The IMU sensors collected attachment

Figure 2. Collected IMU data while moving bricks: (a) keeling with a scaffolding attachment; (b) kneeling with
a body attachment; (c) kneeling with the hook on the ground; (d) standing with a scaffolding attachment; (e)
standing with a body attachment; (f) standing with the hook on the ground

3.2 Preprocessing order low-pass filter with a 10 Hz cut-off frequency to

remove high-frequency noise. The denoised IMU data
The collected IMU data were first filtered by a fifth- were sampled by a 3-second moving window with a 2-

585
 38th International Symposium on Automation and Robotics in Construction (ISARC 2021)

second overlap. A previous study has demonstrated that that better classification accuracies on the posture and
transforming time-series data into image data using activity help to improve the classification of hook
Markov Transition Fields and extracting features by attachment points.
Convolutional Neural Networks (CNN) provides more
stable and better classification results than raw data [18].
This study also transformed each IMU sample to an MTF
that generated a 128×128×6 tensor for each IMU sample,
where 128×128 represents the size of the image data, and
6 represents the number of channels composed of 3-axis
acceleration and angular velocity data. Figure 3 shows
examples of transformed image data for hook and back-
worn IMU data. Both image data were simultaneously
used to detect hook attachments.

Figure 4. Model structure

Table 1. Model structures of F1 and F2

Layer Output Shape
Input Layer 128×128×6
Convolution 128×128×32
Max Pooling 42×42×32
Batch Normalization 42×42×32
Convolution 42×42×64
Figure 3. Transformed image data: (a) hook IMU Batch Normalization 42×42×64
data and (b) back-worn IMU data Max Pooling 21×21×64
Dropout 21×21×64
3.3 Model Structure Batch Normalization 21×21×64
CNNs were used to build a model to classify postures,
activities, and hook attachment points from the Table 2. Model structures of F3 and F4
transformed image data. Figure 4 shows the model Output Shape
structure consisting of three classifiers, C1, C2, and C3, Layer
F3 F4
for the posture, activity, and hook attachment point,
Input Layer 21×21×64 21×21×128
respectively. From the input data, four feature extractors,
Convolution 21×21×128
F1, F2, F3, and F4 extract features for each classifier. F1
Batch Normalization 21×21×128
and F2 extract features from the image data collected
Max Pooling 10×10×128
from the back and the hook, respectively. The extracted
Dropout 10×10×128
features by F1 are used to classify the worker’s postures
Batch Normalization 10×10×128
by C1 and the features are also inputted to the next feature
Convolution 10×10×256
extractor, F3. The extracted features by F3 are inputted to
Batch Normalization 10×10×256
C2 to classify the worker’s activity. Each set of features
Max Pooling 5×5×256
extracted by F1 and F2 is concatenated and inputted to F4.
Dropout 5×5×256
Also, each set of features extracted by F2 and F4 is
Batch Normalization 5×5×256
concatenated and inputted to C3. Therefore, the
developed model detects the worker’s posture, activity,
Table 3. Model structures of C1, C2, and C3
and hook attachment at the same time.
Tables 1 and 2 summarize the model structures of the Output Shape
feature extractors. Since the input data of F4 is the Layer
C1 C2 C3
concatenated features of F1 and F2, the size of the input Input Layer 21×21×64 5×5×256 5×5×512
layer of F4 is twice that of F3. Table 3 shows the model Flatten 28224 6400 12800
structures of the classifiers. The output shape of the last Dense 256
dense layer means that each classifier classifies three Dense 512
different classes. Dropout 512
During the training, the model is trained to reduce the Batch Normalization 512
combined classification loss of C1, C2, and C3, meaning Dense 3

586
 38th International Symposium on Automation and Robotics in Construction (ISARC 2021)

4 Results
70% of the total data were randomly selected as the
training data (9,345 samples), and the remaining 30% of
the total data were used as the testing data (4,005
samples). The developed model was trained for 6,000
epochs in a 10-batch size. The classification results
provided 86.40% of posture, 86.97% of activity, and
96.58% of hook attachment accuracies, respectively.
While the back-worn IMU data were only used for the
posture and activity classifications, the developed
approach utilized both hook and back-worn IMU data to
classify the hook attachment point. Therefore, the
classification accuracy for the hook attachment was Figure 5. Training curves for (a) postures, (b)
higher than that for the posture and activity activities, and (c) hook attachments
classifications.
Figure 5 shows the training curves for each
classification accuracy. In this study, the approach was
designed to reduce overfitting by applying kernel
regularization, dropout layers, and batch normalization
layers. However, the developed model was slightly
overfitted for the posture classification as compared to
the activity and hook attachment classifications. One
reason for this overfitting issue could be related to the
number of features extracted by each feature extractor
because too many features may fit the training dataset but
fail to be generalized to the test dataset. For the posture
classification, the number of features extracted by F1 was
28,224 while F3 extracted 6,400 features and F4 extracted
12,800 features.
The developed approach provided a relatively lower
performance on the posture and activity detections than
the hook attachment. Figure 6 shows the confusion
matrix for each classification result. In the posture
classification, the developed model misclassified some
Figure 6. Confusion matrix of classification
cases of kneeling and standing because similar bodily
results
movements could occur between kneeling and standing.
For example, while using a drill, subjects often did not
bend their backs when both kneeling and standing. 5 Discussion
Conversely, while moving bricks, the subjects gradually
bent their backs as they were being exhausted when both 5.1 Methodological Contribution
kneeling and standing. In the activity classification, some
cases of moving bricks and using a drill machine were The previous study [9] detected the proper use of a
misclassified. For each posture, the two different safety harness based on the distance between the safety
activities were performed by moving arms mainly, which hook and the lifeline. Therefore, the previous approach
could generate invariant back movements for activity. In would identify lifeline attachment whenever the worker
this case, similar patterns of the IMU data could be is closed to the lifeline regardless of the hook attachment.
collected from the back, thereby reducing the overall However, the current study detected the proper use of a
performance of the activity classification. safety harness based on the relative movement between
the hook and the worker’s body. Since this approach
directly detects the hook attachment, the developed
approach could monitor the proper use of a safety harness
wherever they are working.
Additionally, the previous approach would require
further development to monitor multiple workers because
the distance between the lifeline and the safety hook is

587
 38th International Symposium on Automation and Robotics in Construction (ISARC 2021)

the calculated distance of a pair of BLE beacons. high-risk workers to FFH.

Therefore, if multiple workers are working at height, However, the performance of this learning-based
more BLE beacons are required for each worker. The approach would be affected by the quality of the training
previous approach would need to detect each worker’s data and have practical limitations as the training data
pair of beacons and filter the signals receiving from other needs to be collected from each worker. Therefore, a
beacons. However, in this current study, hook subject-independent approach needs to be developed for
attachments were detected based on the IMU data further study.
collected from an individual worker’s safety harness.
Therefore, the developed system is able to monitor the
safety hook attachment individually, allowing to monitor References
multiple workers simultaneously without further [1] Dong X. S., Brown S., and Brooks R.D. Trends of
methodological improvement. fatal falls in the U.S. construction industry. In
Proceedings of the 21st Congress of the
5.2 Practical Application International Ergonomics Association, pages 309-
The developed system detected hook attachments by 313, Online, 2021.
utilizing attachable IMU sensors to the existing safety [2] Chi C. F., Chang T. C., and Ting H. I. Accident
harness. This implementation would make it easy to patterns and prevention measures for fatal
deploy this detection system to construction sites without occupational falls in the construction industry.
additional devices. In addition, a LED bulb can be Applied Ergonomics. 36: 391–400, 2005
attached to a safety harness and indicate the status of the [3] Zhang M. and Fang D. A cognitive analysis of why
hook attachment. This application would empower safety Chinese scaffolders do not use safety harnesses in
managers to monitor the proper use of the safety harness construction. Construction Management
from a distance without any communication networks. Economics. 31: 207–222, 2013.
The developed system could monitor construction [4] Teizer J., Cheng T., and Fang Y. Location tracking
workers for a long period without causing intrusive and data visualization technology to advance
associated with wearing additional sensors by attaching construction ironworkers’ education and training in
IMU sensors to the safety harness. The application of the safety and productivity. Automation in
developed system could allow safety managers to Construction. 35: 53–68, 2013.
identify workers at risk of FFH as repeated improper use [5] Clevenger C., Glick S. G., and del Puerto . L.
of the safety harness can be a precursor to FFH. This Interoperable learning leveraging building
high-risk worker identification would serve as objective information modeling (BIM) in construction
data for worker education and training that could education. International Journal of Construction
effectively change the safety behavior of high-risk Education and Research. 8: 101–118, 2012.
workers. [6] Toole T.M. Construction site safety roles. Journal
of Construction Engineering and Management. 128:
203–210, 2002.
6 Conclusion [7] Schneider S.P. Musculoskeletal injuries in
construction: a review of the literature. Applied
The developed approach provided an accuracy of Occupational Environmental Hygiene. 16:1056–
96.58% for hook attachment detection, and the approach 1064, 2001.
provided a consistent performance on different activities [8] Fang W., Ding L., Luo H., Love P. E. D. Falls from
and postures. The detection approach also provided heights: a computer vision-based approach for
classification accuracies of 86.40% and 86.97% for the safety harness detection. Automatation in
postures and the activity, respectively. The hook Construction. 91: 53–61, 2018.
movement is affected not only by the attachment point [9] Gómez-de-Gabriel J. M., Fernández-Madrigal J. A.,
but also by the worker’s posture and activity. These López-Arquillos A., and Rubio-Romero J.C.
results validated that the relative movement between the Monitoring harness use in construction with BLE
hook and the worker’s body is a key feature for hook beacons, Measurement. 131: 329–340, 2019.
attachment detection. [10] Peng L., Qingbin L., Qixiang F., and Xiangyou G.
The developed system, in this study, was Real-time monitoring system for workers’
implemented by two IMU sensors that can be attachable behaviour analysis on a large-dam construction site.
to existing safety harnesses, allowing this system to be International Journal of Distributed Sensor
extended as the construction environment evolves in Networks. 2013: 509423, 2013.
practice. By applying the developed detection system to [11] Park J., Yang X., Cho Y. K., and Seo J. Improving
construction sites, it would be possible to reduce FFH and dynamic proximity sensing and processing for
to increase construction safety by reliably identifying

588
 38th International Symposium on Automation and Robotics in Construction (ISARC 2021)

smart work-zone safety. Automation in

Construction. 84: 111–120, 2017.
[12] Lee H. S., Lee K. P., Park M., Baek Y., and Lee S.
RFID-based real-time locating system for
construction safety management. Journal of
Computing in Civil Engineering. 26: 366–377,
2012.
[13] Yu Y., Li H., Yang X., Kong L., Luo X., Wong A.
Y. L. An automatic and non-invasive physical
fatigue assessment method for construction workers.
Automation in Construction. 103: 1–12, 2019.
[14] Chen J., Qiu J., Ahn C. Construction worker’s
awkward posture recognition through supervised
motion tensor decomposition. Automation in
Construction. 77: 67–81, 2017.
[15] Nath N.D., Akhavian R., Behzadan A. H.,
Ergonomic analysis of construction worker’s body
postures using wearable mobile sensors. Applied
Ergonomics. 62: 107–117, 2017.
[16] Lee H., Yang K., Kim N., and Ahn C. R. Detecting
excessive load-carrying tasks using a deep learning
network with a Gramian Angular Field. Automation
in Construction. 120: 103390, 2020.
[17] Yang K. and Ahn C. R. Inferring workplace safety
hazards from the spatial patterns of workers’
wearable data. Advanced in Engineering
Informatics. 41: 100924, 2019.
[18] Wang Z., and Oates T. Imaging Time-Series to
Improve Classification and Imputation. In
Proceedings of the Twenty-Fourth International
Joint Conference on Artificial Intelligence, pages
3939–3945, Buenos Aires, Argentina, 2015.

589

View publication stats