(IJACSA) International Journal of Advanced Computer Science and Applications,

Vol. 15, No. 1, 2024

Anomaly Detection in Structural Health Monitoring

with Ensemble Learning and Reinforcement Learning
Nan Huang*
Jiangsu University of Science and Technology, Zhenjiang 212000, Jiangsu, China

Abstract—This research introduces a novel approach for anomaly detection. Variations in this data may result from
improving the analysis of Structural Health Monitoring (SHM) diverse factors such as weather, vehicle overloads, accidents,
data in civil engineering. SHM data, essential for assessing the or unexpected events. It is crucial to recognize that not all
integrity of infrastructures like bridges, often contains anomalies indicate structural issues; some may stem from
inaccuracies because of sensor errors, environmental factors, and sensor errors, calibration issues, noise, or transmission
transmission glitches. These inaccuracies can severely hinder problems. To tackle these anomalies, solutions can be applied
identifying structural patterns, detecting damages, and at both hardware and software levels. While hardware solutions
evaluating overall conditions. Our method combines advanced like using wired data channels, extra sensors, or self-validating
techniques from machine learning, including dilated
sensors are effective, they are often costly. Therefore, there is a
convolutional neural networks (CNNs), an enhanced differential
equation (DE) model, and reinforcement learning (RL), to
growing preference for advanced data preprocessing
effectively identify and filter out these irregularities in SHM techniques specifically designed for anomaly detection.
data. At the heart of our approach lies the use of CNNs, which SHM faces the challenge of data imbalance, where class
extract key features from the SHM data. These features are then instances vary significantly in number. To tackle this issue, two
processed to classify the data accurately. We address the approaches are used: data-centric and algorithm-based. Data-
challenge of imbalanced datasets, common in SHM, through a centric strategies, such as under-sampling, over-sampling, and
RL-driven method that treats the training procedure as a
hybrid methods, aim to balance class distribution. Notably, the
sequence of choices, with the network learning to distinguish
synthetic minority oversampling technique (SMOTE) [4]
between less and more common data patterns. To further refine
our method, we integrate a novel mutation operator within the
creates new minority class instances by linear interpolation,
DE framework. This operator identifies key clusters in the data, while NearMiss [5] under-samples the majority class using a
guiding the backpropagation process for more effective learning. nearest neighbor algorithm. However, over-sampling can lead
Our approach was rigorously tested on a dataset from a large to overfitting, and under-sampling may lose critical
cable-stayed bridge in China, provided by the IPC-SHM information. Algorithmic approaches focus on emphasizing the
community. The results of our experiments highlight the underrepresented class. These include modifying ensemble
effectiveness of our approach, demonstrating an Accuracy of learning, altering decision thresholds, and employing cost-
0.8601 and an F-measure of 0.8540, outperforming other sensitive learning strategies. Cost-sensitive methods treat
methods compared in our study. This underscores the potential classification as cost minimization, assigning higher
of our method in enhancing the accuracy and reliability of SHM misclassification costs to minority cases. Ensemble methods
data analysis in civil infrastructure monitoring. combine multiple classifiers for a final decision, and threshold
adjustment methods tweak the decision threshold during
Keywords—Structural health monitoring; Anomaly detection; testing. These techniques aim to effectively balance accuracy
reinforcement learning; differential equation; imbalanced and information retention in SHM data classification [6].
classification
Furthermore, the incorporation of deep learning
I. INTRODUCTION methodologies can serve as an avenue to address the challenge
SHM is a key method for overseeing civil infrastructure, of imbalanced classification [7, 8]. Deep Reinforcement
offering insights into structural loads, performance, responses, Learning (DRL) emerges as a promising solution to handle
and future behavior predictions. SHM's widespread adoption imbalanced data due to its distinct attributes. By employing a
has led to a significant increase in data generation; for reward mechanism, DRL can assign augmented importance to
example, China's Sutong Bridge, with its 785 sensors, produces the minority class, either by imposing stricter penalties for
2.5 TB of data annually [1, 2]. Analyzing this vast amount of misclassifying instances from the minority class or by offering
SHM data is challenging due to various anomalies caused by greater rewards for accurately identifying them. This approach
sensor errors, system failures, environmental factors, and more. actively counters the bias that conventional techniques display
These issues, compounded by data from significant events like towards the majority class. The advantages of DRL extend
earthquakes or accidents, can jeopardize the accuracy of beyond the mere balancing of class distribution. It also enriches
structural analysis and the predictive power of SHM systems the visibility of crucial patterns, particularly those associated
[3]. with the minority class, by effectively filtering out noisy data.
DRL's ability to unearth significant yet often overlooked
Implementing sensor-driven SHM methods generates large features within the data contributes to the development of a
amounts of sequential data, complicating manual analysis and more robust and efficient model [9].

121 | P a g e
www.ijacsa.thesai.org
 (IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 15, No. 1, 2024

The initial weight configuration in neural networks is 2) The approach integrates a unique reward system that
crucial for training in SHM prediction. Traditional training, reinforces accurate decisions while penalizing incorrect ones.
often using gradient-driven algorithms like backpropagation, By allocating enhanced rewards to the less represented class,
typically starts with randomly assigned weights. However, the we directly address the challenge of data skewness,
initial weight selection greatly impacts the training's efficiency
encouraging the algorithm to appropriately focus on lesser-
and outcome. Careful consideration of the initialization
strategy is essential for effective training and accurate SHM known data. This strategic maneuver contributes to a more fair
prediction. One effective approach is population-based and balanced classification procedure.
training, where the best solution from a range of generated 3) To extract deeper insights from images and refine the
models is chosen as the starting point for the neural network. classification decision-making process, we employ a fusion of
This method helps avoid the common issue of getting stuck in CNN models. This approach enhances the representation of
local optima, prevalent in standard training methods. Notably, features, resulting in improved accuracy and robustness in
simple evolutionary algorithms have shown effectiveness on classification efforts.
par with stochastic gradient descent in neural network training 4) We have developed an enhanced DE algorithm to
[10]. initialize weights in the proposed model efficiently. This tactic
DE [11] is a popular population-based optimization aids in identifying a promising region for initiating the BP
algorithm widely used in solving optimization problems, algorithm within the model.
particularly effective for weight initialization in machine
learning. DE offers several advantages: it ensures a broad The structure of this document is as follows: Section II
exploration of the solution space, preventing entrapment in details a review of relevant literature, and Section III provides
local optima and leading to better weight configurations. It an overview of the key dataset utilized in this study. Section IV
updates weights iteratively based on the difference between delves into the proposed strategy, elucidating the core
current and target solutions, promoting faster convergence and methodology in depth. Section V unfolds the empirical
improved performance. DE is also resilient to noise in fitness outcomes and their subsequent dissection. Concluding
assessments, adeptly handling data uncertainties during weight observations and potential avenues for future inquiry are
initialization and providing stable initial weight settings. encapsulated in Section VI.
Furthermore, DE's flexibility and adjustability in weight II. LITERATURE REVIEW
initialization permit tailoring to particular problem areas, like
establishing weight limits or integrating previous insights. This Artificial Intelligence (AI) techniques bring forth the
versatility improves DE's capability to initiate weights that are capability to uncover patterns within time-series data with no
aptly matched to the distinct learning challenges being prior comprehension of the underlying structural architecture.
addressed. These methodologies involve exploring either the time or
frequency domain of the data, extracting pertinent
This study investigates a novel approach combining a RL- characteristics through statistical evaluations, or employing
based training algorithm with an advanced DE technique, signal processing tools like the Fourier and wavelet transforms,
specifically designed for SHM of bridges. It focuses on as well as the Hilbert-Huang and Shapelet transforms. On the
detecting anomalies in time-series sensor data from a major other hand, deep learning (DL) algorithms possess the ability
cable-stayed bridge in China. The data is divided into seven to autonomously extract significant attributes by interpreting
categories: normal, trend, square, missing, minor, drift, and time-frequency data as visual inputs within a CNN framework.
outlier, with 'normal' being the most frequent. To overcome the However, it is important to acknowledge that DL-centric
issue of class imbalance in the dataset, the research introduces approaches, while potent, demand substantial computational
a framework that treats classification as a series of strategic resources and cause meticulous fine-tuning of hyperparameters
decisions. In each iteration, an agent assesses a training sample [12].
(environmental state) and makes classification decisions,
earning rewards or penalties based on the outcome. Classes In order to tackle irregularities within SHM data, Pan et al.
with fewer samples are assigned higher rewards, encouraging [13] presented an approach rooted in transfer learning. They
accurate identification of less common anomalies. employed a deep neural network to discern and rectify aberrant
Additionally, the study integrates a unique mutation operator data, enhancing the accuracy of bridge evaluations. Samudra et
based on clustering principles within the DE framework to al. [12] devised a comprehensible framework rooted in
improve the backpropagation (BP) process. This operator decision trees, employing random forest classifiers to
identifies dominant clusters in the DE population and categorize acceleration data in the realm of SHM. This
implements a novel approach for creating potential solutions. approach, boasting a remarkable 98% accuracy, emerges as an
The key contributions of this research lie in its innovative economically viable avenue for gauging infrastructure state. Li
approach to class imbalance, decision-making process in et al. [14] outlined a strategy to elevate the efficacy of anomaly
classification, and enhanced training methodology through detection within bridge SHM systems. Employing strategies
integrating RL, DE, and BP processes: like data augmentation, feature dimension reduction, and a
two-stage deep convolutional neural network, they achieved an
1) We present an innovative RL-based method specifically elevated level of recognition accuracy. Tang et al. [3]
designed to address the inherent challenges of imbalanced presented an innovative anomaly detection method catering to
classification in SHM. SHM, employing a CNN that transforms time series data into
visual representations. This approach achieves precise

122 | P a g e
www.ijacsa.thesai.org
 (IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 15, No. 1, 2024

identification of diverse pattern anomalies, scaling effectively distilled from the bridge's measured data, is concisely
and bolstering accuracy. Ye et al. [15] proposed a technique summarized in Table I.
rooted in deep learning for identifying data anomalies within
SHM systems. By deploying time-frequency analysis and Besides the irregularities mentioned, it is vital to
CNNs, they translated SHM data into RGB images, acknowledge that acceleration sensors, particularly those
subsequently classified through a Google network. Green et al. affixed to structures with potential vulnerabilities, adeptly
[16] introduced a new way to use Bayesian techniques in the detect a broad range of atypical patterns. These include offsets,
analysis of inclinometer data for SHM. This method allows for characterized by sudden, noticeable jumps in response, and
the detection of anomalies, forecasting, and quantifying gains, marked by a slow, consistent increase in response over
uncertainties, leading to better risk assessment and cost time. Furthermore, the sensors can identify precision
reduction. deterioration, where the response shows erratic fluctuations,
and complete failures, which result in a response akin to the
Moreover, the framework has the potential to be applied in randomness of white noise in the frequency domain.
various engineering fields beyond inclinometers. Boccagna et Recognizing and interpreting these additional types of
al. [17] suggested an AI approach for monitoring structural deviations are crucial for comprehensive structural health
health in almost real-time, using unsupervised deep learning. monitoring, as they can provide early warning signs of more
By preprocessing data and utilizing artificial neural network significant issues or impending failures.
autoencoders, the technique effectively identifies anomalies,
surpassing current methods and demonstrating encouraging The dataset under study contains a thorough record of
outcomes. Lei et al. [18] proposed a residual attention network acceleration data over a month, meticulously gathered from 38
(RAN) to detect abnormal data in measured structures. The strategically placed accelerometers across the bridge. For
RAN incorporates attention mechanisms and residual learning detailed analysis, this data is segmented into individual hourly
to enhance classification accuracy and efficiency. It achieved time series, leading to an extensive collection of 28,272 such
exceptional performance and generalization on datasets from series. This figure is calculated considering the number of
sensors, the days in the month, and the daily time cycle. Given
an arch bridge and a cable-stayed bridge, surpassing existing
models in terms of multi-classification and accuracy. Yang and that the accelerometers recorded at a rate of 20 Hz, the total
Nagarajaiah [19] introduced a principled, independent data volume reaches an astonishing data points, a
component analysis approach to reduce faulty data during data multiplication of the number of time series, seconds in an hour,
transmission; then, they achieved reliable data transfer and and the sampling rate. This vast dataset offers a rich source for
image restoration using compression sensing methods [20]. in-depth analysis, allowing for the examination of minute
Yang and Nagarajaiah [21] employed the principal component changes and patterns over time, providing a comprehensive
pursuit method to detect and minimize burst noise in ambient understanding of the bridge's dynamic behavior under various
vibration response. They also introduced a data management conditions.
and processing framework based on sparsity rank and low-rank The subsequent classification task systematically organizes
techniques. Park et al. [22] utilized transmission errors and these 28,272 time series responses into seven distinct
ensemble empirical mode decomposition to identify anomalies categories. This includes the ‘normal’ set and six other types of
such as gear teeth spalls and cracks in rotating machinery. anomalies: trend, square, missing, minor, drift, and outlier. A
detailed chart in Table I itemizes the precise distribution of
III. DATASET DESCRIPTION time series across each category within this dataset. This
In this study, our primary focus centers on the detection of categorization is crucial for identifying the predominant
specific deviations - including trends, squares, omissions, anomalies and understanding their relative occurrences. It aids
minor variances, drifts, and outlier - within the acceleration in developing targeted strategies for monitoring and
time series derived from a lengthy cable-stayed bridge in maintenance, ensuring focused attention on the most critical or
China. The IPC-SHM community [23] provides access to frequently occurring issues, enhancing the overall efficiency
curated data from this bridge. An overview of these anomalies, and effectiveness of the structural health monitoring process.
TABLE I. DESCRIPTION OF THE ANOMALIES

Class Description Count

The time-domain response showcases balance, while multiple resonance peaks can be observed in the frequency-domain
Normal 13575
response.
A discernible trajectory is apparent in the time-domain response, and a distinctive peak value is identified in the frequency-
Trend 5778
domain response.
Square The time-domain response mirrors a square wave. 2996
Missing The bulk of the time-domain response is absent. 2942
Minor Compared to standard category data, the time-domain response exhibits a notably reduced magnitude. 1775
Drift The time-domain response varies unpredictably, either exhibiting arbitrary shifts or increasingly diverging over time. 679
Outlier The time-domain response contains singular or multiple pronounced protrusions. 527

123 | P a g e
www.ijacsa.thesai.org
 (IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 15, No. 1, 2024

through this landscape, leading to quicker and more effective

IV. MODEL ARCHITECTURE convergence. Furthermore, this approach also contributes to the
Fig. 1 illustrates the intricacies of our innovatively robustness of the model, making it less susceptible to the
developed model, meticulously engineered to boost the challenges posed by different data distributions and
efficiency of anomaly detection. This model is specifically complexities inherent in various learning tasks. Our method
tailored to tackle issues like uneven distribution of classes and does not just improve the efficiency of the learning process, but
the critical importance of accurately setting initial weights. By also broadens its applicability and effectiveness across a range
integrating the DE algorithm with PPO, our model adeptly of scenarios.
circumvents common hurdles encountered in standard models. Additionally, the RL element in our framework is
Conventional methods often miss a systematic strategy for thoughtfully structured to significantly favor the accurate
establishing initial weights, resulting in reduced learning identification of the minority class, underscoring these crucial
speeds and a propensity to settle on less-than-ideal outcomes. predictions. This represents a significant advancement over
Our technique leverages DE to provide a diverse spectrum of conventional supervised learning approaches, which often
initial weights. This variety helps the model to bypass smaller struggle with insufficient data representation across various
optima and more efficiently converge on more comprehensive classes. The dynamic nature of policy learning in RL
solutions. Expanding on this, the utilization of DE in our encourages a fairer approach to decision-making, leading to
method is not just about broadening the range of initial enhanced strategies for identifying underrepresented
weights, but also about introducing a dynamic and adaptive categories. The flexibility of RL in our setup sets it apart from
way of initializing these weights. This adaptability is crucial in orthodox methodologies, equipping it with the essential
complex models where the landscape of solutions is vast and capabilities to efficiently address the typical challenges found
varied. By starting from a more helpful position in the solution in conventional classification techniques employed in anomaly
space, our method enhances the model’s ability to navigate detection.

Fig. 1. Our model pipeline encompasses a sequence of procedures.

A. Pre-training Phase incorporates the power of a clustering technique and an

Deep models depend heavily on the initialization of deep innovative fitness function to optimize its performance. In our
network weights. If the initial values are not accurate, it can improved DE algorithm, we utilize a mutation and updating
lead to convergence issues in the model. The first stage in this scheme based on clustering to enhance the optimization
paper is to set the CNN and feed-forward neural network performance. Complex architectures rely significantly on the
weights. We offer a more effective DE approach, which initial setup of deep network parameters. Inaccurate starting

124 | P a g e
www.ijacsa.thesai.org
 (IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 15, No. 1, 2024

points may cause the model to struggle with convergence. The optimization approach that meticulously navigates through the
initial step in our study involves configuring the weights for the search space, assessing the effectiveness of solutions against a
CNN and the feed-forward neural networks. We propose an series of established patterns. In practical application, GPBA
enhanced DE method that integrates the effectiveness of a changes the population by meticulously choosing and
clustering algorithm and a novel fitness function to boost its substituting individuals according to their performance
efficiency. In our refined DE technique, we employ a mutation indicators. By employing these patterns as navigational aids,
and refresh strategy centered on clustering to improve the GPBA refines its quest for the best solutions, often resulting in
optimization process. Extending this, our approach takes into a more streamlined and efficient path to the global optimum.
account the intricacies of deep learning architectures, ensuring This method’s real strength lies in its structured approach to the
that the weight initialization is not only randomly, but exploration of the search space. By using gradient patterns, the
strategically influenced by the underlying data structure. The algorithm can intelligently predict the direction in which
clustering-based mutation strategy allows for a more targeted improvements can be made, rather than relying on random or
and data-driven adjustment of weights, which is particularly exhaustive search methods. This predictive capability is helpful
useful in navigating the high-dimensional spaces typical in in complex optimization scenarios, characterized by vast
deep learning. This method aids in avoiding local minima and search spaces and elusive optimal solutions. It allows the
accelerates the convergence of the model. algorithm to bypass fewer promising regions of the search
space, focusing its efforts on areas more likely to yield fruitful
The mutation mechanism, influenced by studies referenced results. Furthermore, the GPBA’s adaptability to different
in [24], identifies a promising region in the exploration optimization problems adds to its versatility. Whether the task
domain. Employing the k-means clustering technique, the involves continuous or discrete variables, linear or non-linear
existing group P is segregated into k segments, each relationships, the GPBA can be tailored to suit the specific
representing a unique portion of the exploration zone. A characteristics of the problem. This adaptability is achieved
random integer chosen from the interval √ dictates the through the customization of its pattern-based search
cluster count. The cluster deemed most optimal possesses the mechanisms, which can suit various problem structures and
lowest mean fitness value across its gathered samples post- complexities. Besides its efficiency in finding solutions, the
clustering. Expanding on this, our method enhances the search GPBA also offers improved computational speed compared to
strategy within the algorithm. By dividing the population into more traditional optimization methods. This is beneficial in
clusters, we can pinpoint specific regions in the search space real-time applications or scenarios where time is a critical
that hold potential for better solutions. This clustering not only factor. The algorithm’s ability to quickly converge to an
focuses the search but also adds a layer of precision in optimal solution without sacrificing accuracy makes it an
identifying promising areas, thereby increasing the efficiency attractive choice for a wide range of optimization tasks.
of the mutation process. Additionally, the number of clusters is
dynamically determined based on the population size, allowing The process unfolds as follows:
for a flexible and adaptable approach to clustering. This
 Selection: To initiate the algorithm, generate k random
adaptability is crucial in dealing with diverse problems and
individuals that will function as the initial points.
varying sizes of search spaces. The concept of assessing the
perfection of a cluster based on its average fitness introduces a  Generation: Produce a set of solutions using
competitive element among the clusters, driving the algorithm mutation based on clustering and denote it as .
to favor areas of the search space that show higher potential for
optimal solutions. Furthermore, our approach refines the  Replacement: Choose solutions randomly from the
selection process within each cluster. After identifying the current population to form set .
most optimal cluster, we focus on fine-tuning the solutions  Update: Select the top M solutions from the combined
within this cluster, leveraging the collective intelligence of the groups and B to create a new group . The
group. This targeted mutation within the most promising refreshed population is derived by merging members of
cluster ensures the algorithm does not just wander aimlessly set P not included in B with those from set (
across the entire search space but makes informed, strategic
moves towards areas more likely to yield superior results.
B. DRL
The clustering-based approach outlines the proposed
mutation: DRL stands as a formidable approach in the domain of
deep learning. Within this framework, an intelligent agent
⃗⃗⃗⃗⃗⃗⃗ ⃗⃗⃗⃗⃗⃗⃗⃗⃗ ⃗⃗⃗⃗⃗ ⃗⃗⃗⃗⃗ (1) engages dynamically with its environment, aiming to maximize
its cumulative rewards. This flexible and adaptive learning
where, ⃗⃗⃗⃗⃗ and ⃗⃗⃗⃗⃗ represent two candidate solutions mechanism empowers the agent to make a series of decisions,
randomly selected from the current population while ⃗⃗⃗⃗⃗⃗⃗⃗⃗ often in the face of uncertainty, which has profound
corresponds to the best solution within the promising region. It applications across a wide spectrum of domains, including but
is important to note that ⃗⃗⃗⃗⃗⃗⃗⃗⃗ may not be the best solution for not limited to robotics, healthcare, and finance [26]. The
the entire population. prowess of DRL becomes evident in tasks that require
sequential decision-making and the ability to adapt to
Following the creation of M new solutions via mutation unforeseen and evolving circumstances. Its capacity to handle
grounded in clustering, the current population undergoes an complex activities that unfold over time, adjusting its strategies
update under GPBA [25]. The GPBA is an innovative

125 | P a g e
www.ijacsa.thesai.org
 (IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 15, No. 1, 2024

and responses as needed, underscores its versatility and broad During the learning stage, as the network experiences state
applicability in addressing real-world challenges. DRL’s ability , it generates a state-specific action. Subsequently, the system
to learn from interactions with the environment, optimize provides a reward r and transitions to the next state . These
decision-making processes, and navigate through dynamic components are combined into a set , subsequently
scenarios positions it as a valuable tool for a wide range of stored in memory M. Groups of such sets, termed Batches B,
applications, making it a compelling area of research and are selected for performing gradient descent. The method for
development in artificial intelligence. calculating loss is detailed in Eq. (7).
In categorization-related tasks, a major challenge lies in =∑ (7)
handling datasets with imbalanced distributions, where one
category is markedly more dominant than others. This Here, θ symbolizes the model's weights, while indicates
disproportion might cause skewed educational outcomes, as the approximated objective for the Q function, evaluated as the
standard classification approaches often lean towards the summation of the reward linked with the state-action pair and
predominant group, leading to subpar identification of the less the reduced maximum Q value in future instances, as
represented categories. Under such conditions, DRL stands out illustrated in Eq. (8).
as a superior strategy for educating neural networks over (8)
conventional approaches. DRL addresses the problem of
lopsided categorization by employing a system based on It is important to recognize that the Q value assigned to the
rewards [27]. Through carefully allocating incentives, it shifts terminal state is initialized at zero. The gradient's magnitude
the agent's attention towards instances belonging to the for the loss function during the i-th iteration is ascertainable
underrepresented categories, thus improving the detection of through Eq. (9).
these rarer classes. The reward-centric model of DRL promotes
a comprehensive decision-making process, prioritizing the ∑
discovery and classification of rare events or infrequently (9)
occurring categories. Through the execution of a gradient descent iteration on the
In the realm of DRL, the primary goal of the agent is to loss function, adjustments are made to the model’s weights
select actions that optimize prospective benefits. The under Eq. (10). This modification endeavors to lessen the
aggregation of rewards for forthcoming situations, symbolized discrepancy, where α denotes the learning rate dictating the
by the reward value, gradually decreases over time, influenced extent of advancement within the optimization procedure.
by the discount rate γ, as illustrated in Eq. (2). In this formula, = (10)
T corresponds to the concluding time-step of an episode [28].
1) Problem formulation: Within this paper, the
∑ (2)
application of the RL algorithm is directed towards the field of
where, represents the cumulative reward starting from
SHM. The ensuing explanation delineates the method’s
time , and denotes the reward received at time . Q-
values, representing the quality of state-action interactions, functioning and interpreting each component:
denote the anticipated outcome of policy upon executing  State : This matches the image captured at the
action within state . This is computed as depicted in Eq. (3). temporal interval t. Here, an image refers to a graphical
(3) representation of the time series data. This image is
composed of plots or graphs that visually depict the
The most optimal action-value function, represented as the various anomalies identified in the acceleration time
highest anticipated reward among all approaches after series of the bridge.
witnessing state and performing action , is calculated as
depicted in Eq. (4).  Action : The categorization executed on the image is
regarded as an action. This signifies a choice carried out
(4) by the network, grounded in its prevailing
comprehension of the objective.
The function carries out the Bellman equation [29], which
states that the supreme anticipated outcome for a particular  Action : A reward is furnished for every
maneuver is the sum of the benefits from the present maneuver categorization, designed to steer the network towards
and the utmost anticipated outcome from forthcoming accurate categorization. The formulation of this
maneuvers in the next instance. This concept is exemplified in remuneration process is expressed as:
Eq. (5).
= (5)
{ (11)
The computation of the ideal action-value function is
methodically executed using the Bellman equation, as
illustrated in Eq. (6). In this context, { } , and
= (6) { }
indicates the minority classes. Accurate or erroneous
classification of a case from the prevalent category leads to an

126 | P a g e
www.ijacsa.thesai.org
 (IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 15, No. 1, 2024

incentive or penalty of +λ or -λ, respectively. This outlined of incorporating dilated convolution, RL, and DE methods
method compels the network to focus on accurately identifying becomes clear. Our model manifested a notable error rate
instances from the less frequent class by allocating a higher reduction, approximately 5.35%, in comparison to its
absolute value to the reward. Concurrently, the incorporation counterparts. This outcome accentuates the critical impact that
of the normal class and the flexible reward parameter within the amalgamation of dilated convolution, RL, and DE
the range of 0<λ<1 adds complexity to the reward scheme. strategies has in boosting the model's performance, thus
This allows for refined adjustment of the network's focus solidifying their role as catalysts in the evolution of deep
between the more and less prevalent classes. learning models.
V. EMPIRICAL EVALUATION In Fig. 2, we present the receiver operating characteristic
(ROC) curves corresponding to the methodologies outlined in
In the meticulous assessment stage, a detailed and Table II. The area under the curve (AUC) serves as a pivotal
exhaustive analysis was carried out, contrasting our suggested metric for quantifying the performance of classifiers. An AUC
model with six distinct deep-learning contenders, namely score of 1 signifies impeccable discrimination ability, while a
TransAnoNet [13], AnoSegNet [14], WaveletCNN [15], GAN- score of 0.5 suggests performance no better than random
VAE [30], CNN-MIAD [31], and VibroCNN. This evaluation guessing.
aimed to provide an all-encompassing insight into the strengths
of our model vis-à-vis established methods. Moreover, we It is worth highlighting that our proposed model emerges as
delved into different versions of our model by introducing the leader in this analysis, boasting a notable AUC value of
three alternative variants for evaluation. The initial variant, 0.60. This solid outcome highlights its remarkable proficiency
termed "Proposed without dilated convolution," was based on a in accurately differentiating between favorable and unfavorable
similar foundational architecture to our original model, yet did results, reinforcing the credibility of our approach as a potent
not incorporate dilated convolution. The subsequent variant, predictive tool. Additionally, the "Proposed without RL"
designated as "Proposed without RL," excluded the approach also demonstrates strong performance, achieving an
reinforcement learning component from the classification AUC of 0.57, further affirming its ability to discern between
procedure. The third altered version, named "Proposed without positive and negative instances. In contrast, WaveletCNN and
DE," employed random initialization for the weights. We TransAnoNet, which achieve AUC scores of 0.46 and 0.49,
appraised these models using standard performance indicators, respectively, offer less impressive performance. VibroCNN,
focusing specifically on measures like the F-measure and the GAN-VAE, and CNN-MIAD display even less favorable
geometric mean due to their proven effectiveness in tackling outcomes, with AUC values ranging from 0.43 to 0.45.
imbalanced datasets. The findings, detailed in Table II, Particularly, VibroCNN's meager AUC of 0.43, only slightly
resoundingly affirm the preeminence of our proposed model surpassing random chance, highlights its underwhelming
over all competing models, including those previously performance. The ROC analysis vividly illustrates the varying
recognized as industry standards like AnoSegNet and degrees of performance among the evaluated methodologies.
TransAnoNet. Across every evaluative criterion, our model The exceptional predictive prowess demonstrated by our
demonstrated consistent superiority over its rivals. Noteworthy proposed method, whether in its standalone form or when
accomplishments involve a marked error reduction exceeding coupled with RL, underscores the potency of our approach.
9% in the F-measure and surpassing 8% in the G-means Furthermore, it establishes a robust foundation for future
indices. These notable advancements highlight the efficacy of enhancements and promising applications in the realm of
our model in surmounting the difficulties associated with predictive modeling, charting a path toward even more
imbalanced datasets and its adeptness at furnishing more effective methodologies in the prediction domain. This
accurate forecasts. In juxtaposing our model with the modified remarkable performance positions our model as a key player in
iterations "Proposed without dilated convolution," "Proposed the field of predictive analytics.
without RL," and "Proposed without DE," the indispensability
TABLE II. EFFICIENCY INDICATORS OF THE SUGGESTED SYSTEM COMPARED TO RIVAL ADVANCED NETWORKS FOR SHM

Accuracy F-measure G-means

TransAnoNet 0.8104±0.0156 0.7802±0.1053 0.8102±0.0203
AnoSegNet 0.8001±0.2156 0.7371±0.0268 0.7902±0.1236
WaveletCNN 0.8005±0.2130 0.7202±0.0268 0.7906±0.2156
GAN-VAE 0.6801±0.0526 0.5402±0.1236 0.6405±0.0256
CNN-MIAD 0.7703±0.1563 0.6602±0.1036 0.7403±0.1265
VibroCNN 0.6704±0.0501 0.5501±0.0652 0.6462±0.0052
Proposed without dilated convolution 0.7904 ± 0.0517 0.7615 ± 0.1623 0.8014 ± 0.3622
Proposed without RL 0.8315 ± 0.0243 0.8200 ± 0.0417 0.8315 ± 0.0621
Proposed without DE 0.8425 ± 0.0123 0.8345 ± 0.0120 0.8436 ± 0.0505
Proposed 0.8601 ± 0.0384 0.8540 ± 0.0297 0.8760 ± 0.0123

127 | P a g e
www.ijacsa.thesai.org
 (IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 15, No. 1, 2024

Fig. 2. AUC chart for the suggested approach and alternative comparative techniques.

Fig. 3 showcases the confusion matrices for the proposed learning and enhancement of its predictive capabilities over
model, providing a detailed representation of its classification time. It is significant to observe that the most pronounced
performance across different categories. From the matrix, we decrease in error happens during the early training stages,
can observe the number of correct predictions (true positives) slowly leveling off as the number of epochs increases. This
along the diagonal for each class, which are as follows: 13,271 trend indicates that with ongoing training, the rate of error
for ‘Normal’, 5,363 for ‘Trend’, 2,608 for ‘Square’, 2,750 for reduction lessens, signifying a point where further error
‘Missing’, 1,571 for ‘Minor’, 539 for ‘Drift’, and 409 for minimization from extended training becomes less impactful.
‘Outlier’. These figures suggest the model is most proficient at Near the 425th epoch, a clear steadying of the error rate is
identifying the ‘Normal’ class and least proficient at observed, consistently hovering around a value of
identifying ‘Outlier’ instances, which could be because of their approximately 4.2962 in subsequent epochs. This leveling off
lower occurrence in the dataset. The off-diagonal numbers of error rates suggests that continued training beyond this
represent the instances where the model misclassified the juncture is unlikely to result in notable enhancements in the
inputs. For example, there are 76 instances where ‘Normal’ model's forecasting accuracy. This stage may signal that the
was incorrectly classified as ‘Missing’, and 80 instances where model has attained a state of convergence, reaching an
‘Square’ was mislabeled as ‘Normal’. Such misclassifications accuracy level where additional fine-tuning might not bring
can diagnose and improve the model’s performance, possibly considerable improvements. Alternatively, this stabilization
by providing it with more representative training data or could also suggest the emergence of overfitting concerns,
refining its feature detection capabilities. especially if the model's performance on validation or test
datasets ceases to improve. This insight into the error dynamics
Fig. 4 illustrates the evolution of error dynamics within the across epochs not only showcases the model's learning journey
proposed model across 500 epochs. Commencing at an initial but also provides valuable guidance for fine-tuning training
value of 12, the error undergoes a consistent descent as epochs duration and preventing potential overfitting scenarios.
unfold. This sustained decline signifies the model's progressive

128 | P a g e
www.ijacsa.thesai.org
 (IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 15, No. 1, 2024

Fig. 3. Comparative confusion matrices for the proposed model.

Fig. 4. Comparative diagram of error dynamics.

A. Impact of the Reward Function extracted from the analysis indicate that the framework
The allocation of rewards to both the more common and achieves its peak effectiveness when λ is established at 0.7, as
less frequent categories for accurate and inaccurate observed across all assessed performance indicators. This
classifications is denoted by +1 and ±λ. The particular observation suggests that the ideal λ magnitude lies within the
magnitude of λ is determined by the ratio of frequent range of zero to one. It's important to recognize that although
occurrences relative to rare events. As this ratio rises, it is modulating λ to reduce the impact of the dominant group is
expected that the ideal magnitude of λ will diminish essential, configuring it too low might adversely affect the
proportionally. In order to thoroughly investigate the influence overall effectiveness of the entire structure. The evidence
of λ, we executed an extensive assessment of the suggested clearly indicates that the choice of λ markedly affects the
structure employing diverse λ magnitudes, varying from 0 to 1 success of the structure. The suitable λ magnitude depends on
in steps of 0.1. Concurrently, the incentive for the more the comparative occurrences of more frequent and infrequent
frequent category stayed unchanged. The detailed results are events, highlighting the need for careful determination to
illustrated in Fig. 5. When adjusting λ to 0, the effect of the achieve the best results. This study underscores the intricate
dominant group turns negligible. interplay between λ and the framework's success, advocating
for a balanced choice of λ to strike a harmonious equilibrium
Conversely, with a value of λ = 1, both the more common between the two categories and foster effective results.
and less common groups carry equivalent weight. The insights

129 | P a g e
www.ijacsa.thesai.org
 (IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 15, No. 1, 2024

Fig. 5. Evaluation of the performance metrics of the proposed system under various settings of the parameter λ.

B. Effect of Loss Function more favorable outcomes for the underrepresented class.
The landscape of strategies available to combat the Notably, CL emerges as a standout loss function, showcasing
complexities arising from data imbalances in machine learning its prowess in scenarios where imbalanced data prevails. By
models is vast and diverse. It spans an array of techniques, skillfully adjusting the weights of the loss function, CL
ranging from the fine-tuning of data augmentation methods to demonstrates its ability to prioritize intricate samples over
the meticulous selection of aptly suited loss functions. The simpler ones, thereby enhancing its adaptability in the face of
deliberate choice of an appropriate loss function plays a central challenging data distributions.
role in ensuring the model's capacity to glean valuable insights Our rigorous experimentation and analysis of these diverse
from the underrepresented class embedded within the dataset. loss functions are presented in meticulous detail in Table III.
In our quest to unravel the nuances of the varying impacts of The outcomes unequivocally affirm the supremacy of CL over
distinct loss functions, we embarked on a comprehensive TL, leading to a substantial 3.72% reduction in the error rate
exploration of five distinct contenders: WCE [32], BCE [33], concerning accuracy and an impressive 3.58% decrease in the
DL [34], TL [35], and CL [36]. F-measure. Nevertheless, it is crucial to underscore that, when
Among these contenders, both BCE and WCE have benchmarked against the performance of our proposed model,
established themselves as widely adopted loss functions, CL exhibits a 1.5% deficit. These findings underscore the
treating positive and negative samples with equal significance. paramount significance of making a judicious selection of an
However, it's imperative to recognize that these functions appropriate loss function when navigating the intricacies of
might not be optimally configured to cater to datasets imbalanced data. Furthermore, they shine a spotlight on the
characterized by pronounced imbalances that accentuate the commendable performance of our model in effectively
minority class. In stark contrast, DL and TL exhibit superior addressing this prevalent and challenging issue in machine
performance when confronted with skewed datasets, delivering learning.
TABLE III. PERFORMANCE EVALUATION METRICS OF THE PROPOSED MODEL AGAINST VARIOUS LOSS FUNCTIONS IN SHM

Accuracy F-measure G-means

WCE 0.7303± 0.0269 0.7105± 0.1204 0.7412± 0.1120
BCE 0.7963± 0.0626 0.7536± 0.1203 0.7963± 0.0103
DL 0.7923± 0.0365 0.7821± 0.0056 0.8103± 0.1123
TL 0.8236± 0.2126 0.8023± 0.0145 0.8352± 0.1035
CL 0.8563± 0.0035 0.8325± 0.0032 0.8523± 0.0039

130 | P a g e
www.ijacsa.thesai.org
 (IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 15, No. 1, 2024

peak functionality while maintaining a delicate balance

C. Effect of CNNs between thorough feature extraction and operational efficiency.
The architecture encompasses an array of CNNs that Our comprehensive experiments, as illustrated in Fig. 6,
concurrently derive feature vectors from images. The quantity unequivocally demonstrate that three CNN feature extractors
of CNNs utilized for feature extraction greatly influences the yield the model's best performance. Interestingly, as the
model's efficiency. Inadequate number of CNNs results in number of CNNs increases, the model's performance declines,
inadequate feature extraction, whereas too much may lead to with six or seven extractors being less effective than using just
problems such as overfitting or superfluousness. Both one. This observed pattern highlights the existence of an
scenarios can diminish the model's overall utility. Therefore, optimal number of CNN feature extractors, maximizing the
carefully selecting the ideal number of CNN feature extractors model's ability to capture relevant and discriminative features,
is crucial. To identify this optimal number, we conducted a ultimately leading to enhanced overall performance. Selecting
thorough and systematic analysis, evaluating the model's three CNN feature extractors strikes a harmonious balance,
performance across a range of 1 to 7 CNN feature extractors. optimizing the model's ability to extract essential information
Our aim was to pinpoint the point where the model achieves from input imagery, boosting its efficiency and effectiveness.

Fig. 6. Plotting the performance indicators of the suggested model while altering the quantity of convolutional feature extraction layers.

D. Discussion vastly different issues compared to a highway bridge spanning

The proposed model signifies a significant advancement in a saline estuary. Each structure reacts to external influences in
the landscape of anomaly detection methods within the domain a nuanced manner, whether it is the ceaseless battering of
of SHM. By incorporating dilated convolutional and DE and waves, vehicular loads, or the immense pressure of contained
RL techniques, this model showcases impressive predictive water. Therefore, while the anomalies identified in the Chinese
accuracy. These strides in technological innovation are bridge dataset offer invaluable insights, they might only scratch
especially pertinent considering the current challenges that the the surface of potential structural concerns when considering
field of civil infrastructure encounters on a global scale. the full spectrum of potential issues.

However, it is essential to subject the model to critical Furthermore, alongside the diversity in structures, the
examination within a broader context of its applicability. While environments in which they exist introduce an additional layer
the initial results are promising, they are inherently tied to data of complexity. The health of a structure isn't solely a reflection
originating from a singular architectural marvel – a long-span of its construction but also a result of its interactions with the
cable-stayed bridge situated in China. While an in-depth focus environment [37]. From corrosion due to saline exposure to
on a specific dataset can yield valuable insights, it also presents vibrations induced by seismic activities, the array of external
the potential risk of confining the model to a narrow scope. stressors is extensive. This raises legitimate concerns about
Civil engineering marvels around the world encompass an whether the proposed model, primarily trained on the Chinese
immense range – from complex metro rail networks navigating bridge dataset, can seamlessly adapt to the myriad challenges
urban mazes to towering skyscrapers reaching for the skies. that structures worldwide encounter. To address these
Each of these structures is the culmination of distinct concerns, several solutions can be implemented:
combinations of design, materials, and environmental factors,  Diverse Data Collection: Expanding the training dataset
leading to unique challenges in structural health. For example, to include data from structures in different
a dam nestled within mountainous terrain would encounter environmental conditions and geographic locations.

131 | P a g e
www.ijacsa.thesai.org
 (IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 15, No. 1, 2024

This would enhance the model's ability to generalize construction projects and newly developed materials.
across a wide range of scenarios [38]. This will ensure that the model stays current and can
recognize anomalies associated with new construction
 Environmental Conditioning: Integrating environmental methodologies.
factors into the model, allowing it to learn how different
environmental conditions affect structural health. This  Collaborative Data Sharing: Establishing a
could involve adding parameters that account for local collaborative network with other civil engineering
climate, pollution levels, and other relevant projects and research institutions for sharing data and
environmental data. insights. This collective approach can significantly
broaden the spectrum of scenarios the model is exposed
 Transfer Learning: Applying transfer learning to, enhancing its ability to identify a wide range of
techniques to adapt the model trained on the Chinese anomalies.
bridge dataset to other structures. This approach
involves fine-tuning the model with smaller datasets  Predictive Analytics: Incorporating predictive analytics
from different structures, enabling it to adjust to new to forecast potential structural issues based on current
environments with minimal data. trends and construction practices. This proactive
approach can help in early identification and prevention
 Regular Model Updates: Continuously updating the of structural failures.
model with new data collected from various structures
over time. This would ensure that the model stays  Cross-Disciplinary Integration: Integrating knowledge
relevant and effective in predicting structural health from other fields such as materials science,
under changing environmental conditions [3]. meteorology, and environmental engineering. This
interdisciplinary approach can provide a more
 Hybrid Modeling Approaches: Combining the strengths comprehensive understanding of how various factors
of different modeling techniques, such as physics-based might contribute to new types of structural anomalies.
models and data-driven models [39]. This hybrid
approach can leverage the accuracy of physics-based  Regular Sensitivity Analysis and Testing: Performing
models in well-understood scenarios and the flexibility sensitivity analyses and stress tests under a variety of
of data-driven models in complex, variable conditions. conditions to evaluate the model's effectiveness in
detecting anomalies in different materials and
 Real-time Environmental Monitoring: Integrating real- construction methods.
time environmental monitoring systems to provide
continuous input to the model. This would allow the  Expert Involvement and Feedback Loops: Engaging
model to adjust its predictions based on current industry experts in regular reviews of the model's
environmental conditions. performance, ensuring that practical, real-world insights
are incorporated. Establishing feedback loops can also
 Stress Testing and Simulations: Conducting stress tests aid in continuous improvement of the model.
and simulations under various environmental conditions
to validate and improve the model's accuracy in As we chart our course ahead, multiple paths invite
different scenarios. investigation. Firstly, testing the proposed model against a
range of SHM datasets that include different types of
As the field of civil engineering advances, embracing new structures, such as high-rise buildings, bridges, tunnels, and
materials and groundbreaking construction techniques, the historical monuments, could reveal its extensive applicability
characteristics of potential structural irregularities are likely to [41]. Employing transfer learning techniques to adapt pre-
transform. A cutting-edge SHM system must be proficient in existing models to these varied scenarios could be key in
identifying established problems and adept at signaling new, rapidly broadening the model's utility without necessitating
unexplored issues [40]. This capacity forms a crucial extensive data gathering from each new structure type. In
benchmark that the proposed model must meet. The addition to these approaches, several other strategies could be
implications are significant; failing to detect a key anomaly can beneficial:
result in catastrophic events, loss of human lives, and severe
economic consequences. To enhance the proposed model's  Cross-Functional Collaboration: Engaging with experts
capability in this dynamic field, several approaches can be from different fields within civil engineering and data
considered: science to gain insights into specific structural
characteristics and data processing techniques. This
 Incorporation of Advanced Learning Algorithms: collaboration could enhance the model's accuracy and
Utilizing machine learning and artificial intelligence relevance across various structures.
algorithms that are capable of identifying patterns and
anomalies not only from past data but also adapting to  Real-Time Data Integration: Incorporating real-time
new trends. Techniques like unsupervised learning or monitoring data into the model to continually update
deep learning can be particularly effective in and refine its predictive capabilities. This could include
recognizing unforeseen issues. data from sensors monitoring weather conditions,
material fatigue, and other relevant parameters.
 Continuous Model Updating and Training: Regularly
updating the model with the latest data from ongoing

132 | P a g e
www.ijacsa.thesai.org
 (IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 15, No. 1, 2024

 Customizable Model Parameters: Developing the model into current workflows, enhancing its practicality and
with customizable parameters that can be adjusted adoption.
according to the specific requirements of different
structures. This flexibility would allow for tailored  Regular Benchmarking and Validation: Regularly
applications, enhancing the model's effectiveness across comparing the model's performance with other state-of-
diverse structural contexts. the-art anomaly detection systems in the field to
validate its effectiveness and identify areas for
 Scalability and Efficiency Improvements: Optimizing improvement.
the model for scalability and computational efficiency
to handle large datasets and enable its deployment in  Sustainability and Environmental Impact Assessment:
large-scale projects, such as city-wide infrastructure Considering the environmental impact and
monitoring. sustainability of the model, especially in terms of its
energy consumption and the materials required for
 Community Engagement and Feedback: Involving sensor deployment and maintenance.
community feedback, especially from those who live or
work in or near monitored structures, to provide  Training and Education for Stakeholders: Providing
ground-level insights into the model's performance and comprehensive training and educational resources for
impact. engineers, technicians, and stakeholders to understand
and effectively use the model in their operations.
 Robust Validation and Testing: Conducting rigorous
Finally, it is important to delve deeper into the specific
validation and testing under various conditions and
mechanisms and algorithms used in state-of-the-art techniques
scenarios to ensure the model's reliability and accuracy,
for handling imbalanced datasets in RL [43]. This involves
particularly in critical and emergency situations.
examining different approaches, such as oversampling, under
 Policy and Regulatory Alignment: Ensuring that the sampling, synthetic data generation, cost-sensitive learning,
model aligns with existing policies, standards, and and novel reward shaping strategies [44, 45]. By contrasting
regulatory requirements related to structural health and these methods with our own, we can identify unique
safety, to facilitate its acceptance and implementation. advantages or shortcomings in both theoretical and practical
applications. Investigating how these techniques perform in
Furthermore, the ever-changing characteristics of civil diverse RL environments, ranging from simulated tasks to real-
structures require that our SHM systems adapt and improve world applications, will provide a more holistic understanding
constantly. Implementing online learning paradigms in the of their adaptability and robustness. It would also be beneficial
proposed model would enable it to dynamically adjust to to explore the integration of our technique with other advanced
evolving structural health patterns. This could be achieved by machine learning strategies like deep learning, transfer
continuously feeding the model with live data and allowing it learning, and meta-learning, to enhance its performance in
to learn and update its parameters in real-time. The integration handling imbalanced datasets. Such an in-depth analysis will
of diverse data sources, such as vibrations, strains, temperature not only fortify our research but also pave the way for future
changes, acoustic emissions, and even visual data from innovations in the field, fostering a more effective approach to
inspections, would significantly enrich the model's predictive tackling the challenges posed by imbalanced datasets in
accuracy [42]. Several additional steps can be taken to enhance reinforcement learning [46].
the model's utility and efficiency:
VI. CONCLUSION
 Edge Computing Implementation: Developing the
model for deployment in edge computing environments This study introduced a groundbreaking model
where data processing occurs closer to the data source. meticulously crafted to confront the intricate challenges
This reduces latency and can be crucial for timely associated with anomaly classification within SHM data. The
decision-making, especially in emergency scenarios. proposed model harnessed a strategic fusion of dilated
convolutional, RL, and DE techniques to achieve a high level
 User-Friendly Interface Development: Creating of accuracy in its results. At its core, the model utilized a group
intuitive user interfaces for the model that enable of CNNs to extract essential feature vectors from input images
engineers and maintenance personnel to easily interpret concurrently. These extracted features were seamlessly
and act upon the data and predictions provided by the integrated into downstream processes, bolstering the model's
system. prowess in identifying complex patterns present in SHM data.
 Automated Alert and Reporting System: Integrating an The efficacy of the proposed model was rigorously validated
automated system that generates alerts and detailed through experimentation on an imbalanced dataset obtained
reports when anomalies are detected, thereby from a long-span cable-stayed bridge in China-sourced from
facilitating prompt and informed responses from the the IPC-SHM community. Handling imbalanced datasets poses
relevant authorities or maintenance teams. distinct challenges in training classifiers, as the overrepresented
class often exerts a disproportionate influence on the learning
 Interoperability with Existing Systems: Ensuring that process, leading to suboptimal performance for the
the model is compatible with existing infrastructure underrepresented class. To effectively address this concern, a
management systems and can be seamlessly integrated novel approach was employed, integrating RL principles to
formulate the training procedure as a series of interconnected

133 | P a g e
www.ijacsa.thesai.org
 (IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 15, No. 1, 2024

decisions. Within this framework, the dataset samples assumed [13] Q. Pan, Y. Bao, and H. Li, "Transfer learning-based data anomaly
the role of states, while the model operated as the agent, detection for structural health monitoring," Structural Health
Monitoring, p. 14759217221142174, 2023.
receiving appropriate rewards or penalties based on accurate or
[14] S. Li, L. Jin, Y. Qiu, M. Zhang, and J. Wang, "Signal anomaly detection
incorrect classifications, respectively. This adaptive strategy of bridge SHM system based on two-stage deep convolutional neural
enabled the model to place a heightened focus on the networks," Structural Engineering International, vol. 33, no. 1, pp. 74-
underrepresented class, thereby enhancing classification 83, 2023.
outcomes. An innovative contribution to the training [15] X. Ye, P. Wu, A. Liu, X. Zhan, Z. Wang, and Y. Zhao, "A Deep
methodology was introduced by incorporating a mutation Learning-based Method for Automatic Abnormal Data Detection: Case
operator grounded in clustering principles within the Study for Bridge Structural Health Monitoring," International Journal of
Structural Stability and Dynamics, p. 2350131, 2023.
framework of DE. This approach initiated the BP process by
[16] D. K. Green and A. Jaspan, "Applied Bayesian Structural Health
identifying a prominent cluster within the existing DE Monitoring: inclinometer data anomaly detection and forecasting," arXiv
population. Subsequently, a novel update strategy was preprint arXiv:2307.00305, 2023.
implemented to generate potential solutions, adding a layer of [17] R. Boccagna, M. Bottini, M. Petracca, A. Amelio, and G. Camata,
sophistication to the training process. The experimental results "Unsupervised Deep Learning for Structural Health Monitoring," Big
underscored the superior performance of the proposed model in Data and Cognitive Computing, vol. 7, no. 2, p. 99, 2023.
the detection of multi pattern anomalies within SHM data, [18] X. Lei, Y. Xia, A. Wang, X. Jian, H. Zhong, and L. Sun, "Mutual
showcasing remarkable accuracy. Through the adept information based anomaly detection of monitoring data with attention
amalgamation of dilated convolutional, RL, and DE mechanism and residual learning," Mechanical Systems and Signal
Processing, vol. 182, p. 109607, 2023.
techniques, the model exhibited its potential as an advanced
[19] Y. Yang and S. Nagarajaiah, "Data compression of structural seismic
tool for anomaly detection within SHM systems. This responses via principled independent component analysis," Journal of
capability is of utmost importance in safeguarding the Structural Engineering, vol. 140, no. 7, p. 04014032, 2014.
structural integrity and safety of critical infrastructures, [20] Y. Yang and S. Nagarajaiah, "Robust data transmission and recovery of
including vital components like bridges. images by compressed sensing for structural health diagnosis,"
Structural Control and Health Monitoring, vol. 24, no. 1, p. e1856, 2017.
REFERENCES [21] Y. Yang and S. Nagarajaiah, "Blind denoising of structural vibration
[1] J. P. Lynch, C. R. Farrar, and J. E. Michaels, "Structural health responses with outliers via principal component pursuit," Structural
monitoring: technological advances to practical implementations Control and Health Monitoring, vol. 21, no. 6, pp. 962-978, 2014.
[scanning the issue]," Proceedings of the IEEE, vol. 104, no. 8, pp. [22] S. Park, S. Kim, and J.-H. Choi, "Gear fault diagnosis using transmission
1508-1512, 2016. error and ensemble empirical mode decomposition," Mechanical
[2] W.-H. Hu, S. Said, R. G. Rohrmann, Á. Cunha, and J. Teng, Systems and Signal Processing, vol. 108, pp. 262-275, 2018.
"Continuous dynamic monitoring of a prestressed concrete bridge based [23] Y. Bao, J. Li, T. Nagayama, Y. Xu, B. F. Spencer Jr, and H. Li, "The 1st
on strain, inclination and crack measurements over a 14-year span," international project competition for structural health monitoring (IPC-
Structural Health Monitoring, vol. 17, no. 5, pp. 1073-1094, 2018. SHM, 2020): A summary and benchmark problem," Structural Health
[3] Z. Tang, Z. Chen, Y. Bao, and H. Li, "Convolutional neural network‐ Monitoring, vol. 20, no. 4, pp. 2229-2239, 2021.
based data anomaly detection method using multiple information for [24] J. Parra, L. Trujillo, and P. Melin, "Hybrid back-propagation training
structural health monitoring," Structural Control and Health Monitoring, with evolutionary strategies," Soft Computing, vol. 18, no. 8, pp. 1603-
vol. 26, no. 1, p. e2296, 2019. 1614, 2014.
[4] H. Han, W.-Y. Wang, and B.-H. Mao, "Borderline-SMOTE: a new over- [25] K. Deb, "A population-based algorithm-generator for real-parameter
sampling method in imbalanced data sets learning," in International optimization," Soft Computing, vol. 9, pp. 236-253, 2005.
conference on intelligent computing, 2005: Springer, pp. 878-887. [26] V. François-Lavet, P. Henderson, R. Islam, M. G. Bellemare, and J.
[5] I. Mani and I. Zhang, "kNN approach to unbalanced data distributions: a Pineau, "An introduction to deep reinforcement learning," Foundations
case study involving information extraction," in Proceedings of and Trends® in Machine Learning, vol. 11, no. 3-4, pp. 219-354, 2018.
workshop on learning from imbalanced datasets, 2003, vol. 126: ICML, [27] M. Bahadori, M. Soltani, M. Soleimani, and M. Bahadori, "Statistical
pp. 1-7. Modeling in Healthcare: Shaping the Future of Medical Research and
[6] A. Fernández, S. García, M. Galar, R. C. Prati, B. Krawczyk, and F. Healthcare Delivery," in AI and IoT-Based Technologies for Precision
Herrera, Learning from imbalanced data sets. Springer, 2018. Medicine: IGI Global, 2023, pp. 431-446.
[7] S. Wang, W. Liu, J. Wu, L. Cao, Q. Meng, and P. J. Kennedy, "Training [28] S. Danaei et al., "Myocarditis Diagnosis: A Method using Mutual
deep neural networks on imbalanced data sets," in 2016 international Learning-Based ABC and Reinforcement Learning," in 2022 IEEE 22nd
joint conference on neural networks (IJCNN), 2016: IEEE, pp. 4368- International Symposium on Computational Intelligence and Informatics
4374. and 8th IEEE International Conference on Recent Achievements in
[8] C. Huang, Y. Li, C. C. Loy, and X. Tang, "Learning deep representation Mechatronics, Automation, Computer Science and Robotics (CINTI-
for imbalanced classification," in Proceedings of the IEEE conference on MACRo), 2022: IEEE, pp. 000265-000270.
computer vision and pattern recognition, 2016, pp. 5375-5384. [29] E. Barron and H. Ishii, "The Bellman equation for minimizing the
[9] E. Lin, Q. Chen, and X. Qi, "Deep reinforcement learning for maximum cost," NONLINEAR ANAL. THEORY METHODS
imbalanced classification," Applied Intelligence, vol. 50, pp. 2488-2502, APPLIC., vol. 13, no. 9, pp. 1067-1090, 1989.
2020. [30] J. Mao, H. Wang, and B. F. Spencer Jr, "Toward data anomaly detection
[10] S. V. Moravvej, S. J. Mousavirad, D. Oliva, and F. Mohammadi, "A for automated structural health monitoring: Exploiting generative
Novel Plagiarism Detection Approach Combining BERT-based Word adversarial nets and autoencoders," Structural Health Monitoring, vol.
Embedding, Attention-based LSTMs and an Improved Differential 20, no. 4, pp. 1609-1626, 2021.
Evolution Algorithm," arXiv preprint arXiv:2305.02374, 2023. [31] M. Zhao, A. Sadhu, and M. Capretz, "Multiclass anomaly detection in
[11] T. Eltaeib and A. Mahmood, "Differential evolution: A survey and imbalanced structural health monitoring data using convolutional neural
analysis," Applied Sciences, vol. 8, no. 10, p. 1945, 2018. network," Journal of Infrastructure Preservation and Resilience, vol. 3,
no. 1, p. 10, 2022.
[12] S. Samudra, M. Barbosh, and A. Sadhu, "Machine Learning-Assisted
Improved Anomaly Detection for Structural Health Monitoring," [32] Ö. Özdemir and E. B. Sönmez, "Weighted cross-entropy for unbalanced
Sensors, vol. 23, no. 7, p. 3365, 2023. data with application on covid x-ray images," in 2020 Innovations in

134 | P a g e
www.ijacsa.thesai.org
 (IJACSA) International Journal of Advanced Computer Science and Applications,
Vol. 15, No. 1, 2024

Intelligent Systems and Applications Conference (ASYU), 2020: IEEE, [39] C. Bigoni, "Numerical methods for structural anomaly detection using
pp. 1-6. model order reduction and data-driven techniques," EPFL, 2020.
[33] F. Huang, J. Li, and X. Zhu, "Balanced Symmetric Cross Entropy for [40] Y. Bao, Z. Tang, H. Li, and Y. Zhang, "Computer vision and deep
Large Scale Imbalanced and Noisy Data," arXiv preprint learning–based data anomaly detection method for structural health
arXiv:2007.01618, 2020. monitoring," Structural Health Monitoring, vol. 18, no. 2, pp. 401-421,
[34] X. Li, X. Sun, Y. Meng, J. Liang, F. Wu, and J. Li, "Dice loss for data- 2019.
imbalanced NLP tasks," arXiv preprint arXiv:1911.02855, 2019. [41] Y. Zhang, Z. Tang, and R. Yang, "Data anomaly detection for structural
[35] S. S. M. Salehi, D. Erdogmus, and A. Gholipour, "Tversky loss function health monitoring by multi-view representation based on local binary
for image segmentation using 3D fully convolutional deep networks," in patterns," Measurement, vol. 202, p. 111804, 2022.
Machine Learning in Medical Imaging: 8th International Workshop, [42] X. Xu et al., "Anomaly detection for large span bridges during
MLMI 2017, Held in Conjunction with MICCAI 2017, Quebec City, operational phase using structural health monitoring data," Smart
QC, Canada, September 10, 2017, Proceedings 8, 2017: Springer, pp. Materials and Structures, vol. 29, no. 4, p. 045029, 2020.
379-387. [43] S. Susan and A. Kumar, "The balancing trick: Optimized sampling of
[36] S. A. Taghanaki et al., "Combo loss: Handling input and output imbalanced datasets—A brief survey of the recent State of the Art,"
imbalance in multi-organ segmentation," Computerized Medical Engineering Reports, vol. 3, no. 4, p. e12298, 2021.
Imaging and Graphics, vol. 75, pp. 24-33, 2019. [44] K. M. Hasib et al., "A survey of methods for managing the classification
[37] A. Moallemi, A. Burrello, D. Brunelli, and L. Benini, "Model-based vs. and solution of data imbalance problem," arXiv preprint
data-driven approaches for anomaly detection in structural health arXiv:2012.11870, 2020.
monitoring: A case study," in 2021 IEEE International Instrumentation [45] D. Ramyachitra and P. Manikandan, "Imbalanced dataset classification
and Measurement Technology Conference (I2MTC), 2021: IEEE, pp. 1- and solutions: a review," International Journal of Computing and
6. Business Research (IJCBR), vol. 5, no. 4, pp. 1-29, 2014.
[38] Y.-M. Zhang, H. Wang, H.-P. Wan, J.-X. Mao, and Y.-C. Xu, "Anomaly [46] J. M. Johnson and T. M. Khoshgoftaar, "Survey on deep learning with
detection of structural health monitoring data using the maximum class imbalance," Journal of Big Data, vol. 6, no. 1, pp. 1-54, 2019.
likelihood estimation-based Bayesian dynamic linear model," Structural
Health Monitoring, vol. 20, no. 6, pp. 2936-2952, 2021.

135 | P a g e
www.ijacsa.thesai.org
 © 2024. This work is licensed under
http://creativecommons.org/licenses/by/4.0/ (the “License”). Notwithstanding
the ProQuest Terms and Conditions, you may use this content in accordance
with the terms of the License.