Robotics and Computer–Integrated Manufacturing 94 (2025) 102944

Contents lists available at ScienceDirect

Robotics and Computer-Integrated Manufacturing

journal homepage: www.elsevier.com/locate/rcim

Digital twin-based architecture for wire arc additive manufacturing using

OPC UA
Mohammad Mahruf Mahdi a , Mahdi Sadeqi Bajestani b , Sang Do Noh c , Duck Bong Kim d,*
a
Department of Electrical and Computer Engineering, Tennessee Technological University, TN 38505, USA
b
Department of Mechanical Engineering, Tennessee Technological University, TN 38505, USA
c
Department of Industrial Engineering, Sungkyunkwan University, 2066 Seobu-ro, Jangan-gu, Suwon, 16419, Republic of Korea
d
Department of Manufacturing and Engineering Technology, Tennessee Technological University, TN 38505, USA

A R T I C L E I N F O A B S T R A C T

Keywords: This paper presents a digital twin (DT)-based architecture for wire arc additive manufacturing (WAAM) utilizing
Digital twin Open Platform Communications Unified Architecture (OPC UA) for enhanced communication, security, and real-
Wire Arc additive manufacturing time control. DT is explored at both enterprise management and individual asset scales, providing a compre­
OPC UA
hensive framework for process optimization. The proposed architecture integrates advanced 3D visualization,
Remote/Real-time Control and Inspection
real-time defect prediction using convolutional neural networks (CNNs), and structured data management. A
practical case study involving a 6-degree-of-freedom (DOF) industrial robotic arm demonstrates the application
of the architecture in a WAAM deposition scenario. The architecture’s effectiveness is evaluated, focusing on
anomaly detection, joint angle accuracy, and communication reliability, highlighting the integration of computer
vision and cloud-based data storage. The results indicate significant improvements in defect detection, process
monitoring, and real-time interaction between the physical entity and the DT, underscoring the potential of the
proposed DT architecture.

1. Introduction data. It can be classified into three categories: powder bed system (PBS),
powder feed system (PFS), and wire feed system (WFS) [6]. Wire arc
Smart manufacturing has contributed to industry advancement by additive manufacturing (WAAM), a sub-category of WFS, is a wire-based
visualization of the processes and assets through employing digital twins direct energy deposition (DED) approach that uses an electric arc for
(DTs), which are dynamic digital replicas of physical assets, processes, fusion melting of wire feed-stock to deposit on a previous layer [7].
and systems and monitor their life cycle [1]. Realization of DTs occurs WAAM benefits from low capital cost, open architecture, part size
on two scales: enterprise management scale and individual asset scale. flexibility, high deposition rate, and applicability in metal big area ad­
The former closely matches smart manufacturing in developing replicas ditive manufacturing (mBAAM) [8,9]. Therefore, it has a wide range of
of the processes through conceptual and generic frameworks, while the applications in different industries, including but not limited to auto­
latter demonstrates the complex physics of a discrete system [2]. DTs’ motive, aerospace, and shipbuilding [10].
functionalities include product design and development, maintenance, Despite its benefits, WAAM fabricated parts are prone to mechanical
monitoring and control, decision support, and defect detection [3]. defects such as deformation, residual stress, porosity, cracks, and
Therefore, they are widely used in aerospace, energy, automotive, ma­ delamination. Nevertheless, they can be minimized through heat treat­
rine, petroleum, agriculture, healthcare, mining, and advanced ment, interpass cold rolling and cooling, peening, and ultrasonic impact
manufacturing [4]. In particular, DTs have highly impacted metal ad­ treatment [11]. In addition, WAAM has a stochastic nature, and its
ditive manufacturing (AM) by optimizing process parameters, variabilities can include process development, path design and pro­
decreasing computational costs, analyzing big data, detecting and gramming, material quality and performance, process modeling, and
monitoring process anomalies, and multi-scale modeling [5]. product quality assurance, all of which accumulatively lead to the curse
Metal AM denotes the layer-upon-layer process of joining metals or of dimensionality [12]. DTs can help mitigate such challenges by visu­
alloys to fabricate a part employing three-dimensional (3D) modeling alization and monitoring the manufacturing process, predicting

* Corresponding author.
E-mail address: [email protected] (D.B. Kim).

https://doi.org/10.1016/j.rcim.2024.102944
Received 28 September 2024; Received in revised form 25 November 2024; Accepted 27 December 2024
Available online 3 January 2025
0736-5845/© 2024 Elsevier Ltd. All rights are reserved, including those for text and data mining, AI training, and similar technologies.
M.M. Mahdi et al. Robotics and Computer-Integrated Manufacturing 94 (2025) 102944

production performance, and simulation of product properties [13]. reported the 3D visualization for multidimensional information displays
Although numerous research has been conducted to enhance WAAM such as geometry, physics, rules, and behavior of DT applications in
through automated computer vision (e.g., defect detection and product workshops. Li et al. [30] conducted a thorough review of digital twin­
geometry), real-time remote control and inspection through DTs have ning and visualization of WAAM and provided a route to duplicate the
not been explored [14–16]. This can facilitate the maintenance process, realistic system.
improve the process substantially, and reduce the time and cost of Bidirectionality has also recently become a topic of interest.
manufacturing [17]. Research conducted by Knapp et al. [31] in developing a DT of melt pool
The implementation of DT for WAAM is a data-intensive process in in AM processes and Pantelidakis et al. [32] in modeling fused deposi­
which data can be in the form of smart sensor data, images, analog and tion process demonstrated that bidirectionality in the data flow is
digital signals, and inventory records. Hence, appropriate data man­ mandatory for the autonomous operation of DT models. Geng et al. [33]
agement by a well-defined information model and database and its used the socket-client framework for bidirectional mapping of DT ar­
streamlined communication among users for effective decision-making chitecture for SM to cyber-physical space. Hu et al. [34] presented a DT
is of prime importance [18]. Unmanaged data and inhibited communi­ epistemology that considered bidirectional information flow for
cation will increase the process uncertainty. This can be addressed by real-time optimization of the visual templates. Bonnard et al. [35] used
employing Open Platform Communication Unified Architecture (OPC bidirectional functionality to ensure compatibility between the steps of
UA) as the communication protocol. It is a standardized their designed digital thread. However, these studies were limited to
client-server-based machine-to-machine communication mechanism modeling and simulation and were not validated through case scenarios.
that supports object-oriented information modeling [19]. OPC UA pro­ In addition, monitoring, inspection, and control through DTs have
vides a solution for semantic interoperability, thus making it been considered a viable approach by the community since AM pro­
platform-independent. Additionally, it ensures real-time, reliable cesses are vulnerable to defects induced by process complexities and
communication between the DTs and the physical entities with robust uncertainties. Liu et al. [36] suggested remote inspection and feedback
security measures [20,21]. To this date, applications and implementa­ control of a cloud-based smart HMI-based cyber-physical machine tool.
tions of digital twins in additive manufacturing have been studied by a Urbas et al. [37] developed a model-based defining system for remote
number of researchers [22,23], however, the main characteristics of the inspection of manufactured components using Augmented Reality (AR).
DT concept which are bidirectional information flow and real-timeness Linn et al. [38] developed a remote inspection module using Virtual
have been neglected. In addition, this study, for the first time, presents a Reality (VR) for industrial equipment and difficult-to-reach regions of a
framework that integrates OPC UA, CNN-based anomaly detection, and plant. Geng et al. [33] created an adaptable DT for remote machining
structured data management into the digital twin concept. process control using a VR-based, immersive, two-way human-computer
This paper aims to develop a novel DT-based architecture by interaction technique. Stan et al. [39] proposed the DT for a robotic
applying OPC UA in WAAM. The proposed architecture presents a DT deburring work cell that uses a web platform as a part of the DT to
and a real-time remote inspection for defect detection and prediction monitor the cell using the RESTful API architecture remotely. Recently,
using convolutional neural networks (CNNs) and machine learning Mu et al. [40] presented an adaptive online simulation model for pre­
(ML). It also benefits from cross-platform communication and structured dicting distortion fields in metal AM, utilizing a diffusion model archi­
data management in the form of an information model through OPC UA. tecture for distortion process modelling. In addition, despite the
The proposed architecture is evaluated by a WAAM deposition case promising opportunities of VR and AR for remote inspection and control,
scenario using a six-degree-of-freedom (DOF) industrial robotic arm. as explored by Liu et al. [36] and Geng et al. [33], these technologies are
The remainder of the paper is structured as follows: Section 2 reviews not yet fully integrated into DT systems for real-time monitoring and
the related studies of key concepts in understanding DT-based WAAM; control.
Section 3 introduces the conceptual DT architecture for WAAM So far, although substantial progress has been made in the 3D visu­
employing OPC UA; Section 4 demonstrates the DT for WAAM; Section 5 alization of DTs, the integration of this feature with bidirectional in­
discusses the results and pinpoints the possible future investigations; formation flow for real-time operational control remains underexplored.
and Section 6 concludes the research. Research should bridge this gap by creating more immersive and
interactive remote inspection tools that leverage the full potential of VR
2. Literature review and background and AR, enabling precise and efficient control over complex
manufacturing processes.
DT is an elaborate concept with many functionalities, characteristics,
and enabling technologies [24]. The proposed architecture is an exten­ 2.2. CNN and ML
sion of smart manufacturing; hence, the literature on relevant func­
tionalities of the DT (i.e., 3D visualization, bidirectional information CNN and ML have also shown promising results in mitigating the
flow, monitoring, control, and inspection) and enabling technologies (i. anomalies of WAAM-formed parts. Anomaly detection in the metal AM
e., CNN and ML) and the communication protocol employed in this process has been investigated by Qin et al. [41], where the detection
research, i.e. OPC UA, are discussed in this Section. process was divided into image-based, sensor signal-based, acoustic
emission-based, and optical emission-based approaches. The use of
2.1. Digital twin CNN, long short-term memory (LSTM), support vector machine (SVM),
and multilayer perceptron (MLP), among others, were reported as the
3D synchronized visualization is an effective solution to accurately prediction instrument. Kim et al. [14] used 1D voltage waveform con­
and intuitively present multidimensional DT information to the users. verted to images to predict normal and abnormal beads in WAAM using
Fan et al. [25] designed a flexible manufacturing system that used dig­ CNN-based real-time monitoring and achieved 96 % accuracy. Cho et al.
ital mock-ups to present multi-source heterogeneous information for [15] used 2D melt-pool image data to detect WAAM anomalies in
user interaction. Zhang et al. [26] developed a rapid construction of a real-time and concluded that MobileNetV2 has the best performance
discrete DT for a workshop system and standalone software for a 3D with 98 % accuracy. Shin et al. [42] proposed multi-score transfer
visualized model. Duan et al. [27] introduced a DT-based collaborative learning for detecting the balling effect in WAAM, and an accuracy of
robot arm monitoring system where 3D visualization of the DT was 90.3 % was achieved. WAAM process monitoring and defect detection
achieved using the Web Graphics Library (WebGL). Sanabria et al. [28] system that integrated an intelligent algorithm was proposed by Li et al.
summarized implementations of 3D simulation and visualization envi­ [43], who used SVM to detect noises in welding signals indicating po­
ronments such as AzureDT in real-life DT applications. Jia et al. [29] tential defects [39]. Yaseer et al. [44] implemented MLP and random

2
M.M. Mahdi et al. Robotics and Computer-Integrated Manufacturing 94 (2025) 102944

forest (RF) to model surface roughness in WAAM that used a weaving addressed to some extent, as evidenced by the work of Martinelli et al.
path. Studies by Qin et al. [41] and Kim et al. [14] demonstrate the [49] and Ivanova et al. [52], but there is still a need for more compre­
potential of these technologies but often rely on limited datasets or hensive and scalable security frameworks that can be easily integrated
specific conditions. into existing industrial systems. Future research should prioritize
Therefore, employing ML algorithms for predictive maintenance and developing standardized protocols and best practices to implement
anomaly detection can improve the accuracy and responsiveness of DT robust security measures in industrial environments, including
systems. Integration with standardized information models like Auto­ advanced encryption algorithms, multi-factor authentication, and clear
mationML can facilitate interoperability and data exchange across authorization protocols. Secure communication channels such as Secure
different platforms. Developing modular architectures that support OPC UA over Transport Layer Security (TLS) should be developed to
scalable and flexible deployment in cloud environments will further protect data in transit. These measures will collectively ensure the
enhance the system’s capability to manage multiple DTs and their protection of sensitive data and maintain the integrity and reliability of
physical counterparts efficiently. Notable efforts to apply ML to anomaly DT systems in industrial applications. Moreover, while information
detection in WAAM processes are still in their infancy. Further research models in OPC UA for representing WAAM processes, as discussed by
is needed to develop more generalized and robust ML models that can Shin et al. [45] and Buchgeher et al. [46], are important, there is a lack
handle diverse and complex real-world scenarios, incorporating varied of universally accepted standards. Developing standardized information
data sources and enhancing prediction accuracy. models that can be easily adopted across different platforms and in­
dustries is crucial for ensuring interoperability and efficient data
2.3. OPC UA exchange.

Information models play an important role in representing the 3. The proposed architecture
physical entities in a digital space. They are used to represent the logical
structure and the real-time status of each critical component of the The proposed architecture for WAAM DT is shown in Fig. 1, which
WAAM process in a well-defined manner, and the information modeling allows for real-time inspection and optimization of the manufacturing
aspect of OPC UA has been extensively studied. For instance, Shin et al. process. It consists of the physical entity, middleware, and the digital
[45] proposed the OPC UA-compliant interface for the information ex­ twin. The Physical entity comprises the robotic arm, the smart sensors,
change of Predictive Model Markup Language (PMML) models by and the data acquisition unit (DAU); the middleware consists of the OPC
designing the mapping rule and information models for data transfer. UA communication protocol, big data, and cloud computation, while the
Buchgeher et al. [46] reported the migration of Engel Austria machine DT includes CNN, 3D visualization approaches, and computer vision.
software stack to OPC UA-based communication where the information
model was derived from the existing code base and corresponding 3.1. Physical entity
implementation was automatically generated with human-readable
documentation. Müller et al. [47] proposed the knowledge modeling The physical entity in the proposed architecture is comprised of three
for self-organized reconfiguration management for Cyber-Physical Sys­ main components: the robotic arm, the smart sensors, and the DAU. The
tem (CPS), where the knowledge modeling was realized through Unified first component, the robotic arm, must be equipped with sophisticated
Modeling Language (UML) information models integrated into the sys­ sensors and actuators that provide real-time motion feedback. Essential
tem using XML files. And Martins et al. [48] developed a novel infor­ components include accelerometers, encoders, and high-performance
mation model that extended CNC Companion specifications with new actuators, which deliver data on the arm’s position, orientation, and
industry-relevant datatypes, developed through a generic C# abstract movement. In addition, force and torque sensors enable the arm to
class. detect and respond to external forces, facilitating sensitive interactions
Security has been a specific consideration in the creation of OPC UA, with its environment. Cameras and machine vision algorithms play a
and it offers several security modes (no security, integrity only, or crucial role, allowing the arm to identify, track, and navigate objects,
integrity and confidentiality) and security rules that define crypto­ enhancing its ability to interact with its surroundings. These technolo­
graphic encryption and signing methods to protect communication. gies collectively enable the robotic arm to achieve high accuracy and
Martinelli et al. [49] demonstrated a framework that used the OPC UA dexterity in its movements.
architecture to enable continuous control over action execution in the The second component, the smart sensors, detect physical charac­
industrial setting where connectivity security was implemented by teristics and evaluate data to generate actionable insights. In the WAAM
attribute-based access control. In another study, Hildebrandt et al. [50] setup, smart sensors are essential for tracking parameters like temper­
carried out a successful supply chain attack employing an OPC UA server ature and material properties. By performing local data analysis, these
along with a Siemens S7–1500 PLC as a client, which was discovered sensors reduce reliance on external systems, leading to more efficient
using the proposed detection approach utilizing a one-class classifier, and flexible production processes. Local data analysis refers to the
achieving a detection rate of up to 89.5 %. Honda et al. [51] proposed a process of analyzing data directly at the sensor level rather than relying
technique for monitoring without communication loads that involved on external systems for processing. The approach begins with data
making the decryption technique autonomous and using the decrypted collection, where smart sensors continuously monitor parameters like
data to enable flexible collaboration with other systems, such as Security process parameters and material properties among others. The raw data
Information and Event Management, which could be installed on both undergoes preprocessing steps to filter noise, normalize values, and
the OPC UA client and the server to monitor the encrypted data. Ivanova convert it into usable formats. Feature extraction follows, where the
et al. [52] examined the potential for cyberattacks and the primary sensor identifies specific data attributes critical for tracking anomalies
dangers to the safety of the OPC UA-based CPS. They developed proven or changes in process conditions. Analyzing these features involves
guidelines for improving security, including operating in “Security applying lightweight algorithms or embedded ML models—known as
Mode,” selecting cryptographic algorithms, and using user authentica­ TinyML—that are designed for edge devices, which enable the sensors to
tion, among others. Gilles et al. [53] suggested combining the authen­ detect and classify major manufacturing defects. Event-driven process­
tication of distributed system components, such as remote devices, with ing techniques are also used to trigger alerts or adjust settings if certain
legacy devices that cannot be upgraded. Their experiment and conse­ thresholds are breached, enhancing operational responsiveness. These
quent risk assessment emphasized the protection of the clear-text nonce methods, collectively referred to as edge computing, empower sensors to
and the group keys of the OPC UA PubSub security groups. perform actions autonomously based on real-time data. For instance,
On the other hand, security in OPC UA implementations has been smart sensors continuously collect real-time data on parameters such as

3
M.M. Mahdi et al. Robotics and Computer-Integrated Manufacturing 94 (2025) 102944

Fig. 1. The proposed architecture of WAAM DT using OPC UA.

temperature, humidity, and material properties, making autonomous preprocessing capabilities such as filtering and averaging. Real-time
decisions and performing local data analysis. This minimizes the need data acquisition allows the DAU to continuously receive and interpret
for constant external intervention. data from various sensors, providing a comprehensive view of the AM
Furthermore, smart sensors facilitate a connected and interoperable process. The choice of storage method—whether local or cloud-base­
manufacturing system by seamlessly integrating various devices or d—is critical for structuring system architecture, managing data, and
centralized monitoring systems. Smart sensors are used in WAAM to ensuring security. Local storage offers rapid data access, which is
gather real-time data and enable intelligent decision-making, thereby beneficial for applications requiring minimal latency, especially in time-
monitoring and improving the AM process. Wireless communication sensitive processes. This setup ensures data remains within the facility,
protocols for connection, embedded software and firmware for inde­ enhancing control over access and facilitating compliance with stringent
pendent decision-making, and microcontrollers and processors for on­ security policies. On the other hand, cloud-based storage, such as Azure
board data processing are the key enabling technologies of smart Blob Storage mentioned in this study, provides scalability and remote
sensors. First, smart sensors incorporate on-sensor microcontrollers or access, allowing for easy integration with other cloud services. This
processors that may run algorithms for data processing and judgment. approach is advantageous for multi-site operations or when large vol­
Second, wireless communication methods like Bluetooth, Wi-Fi, or IoT- umes of data require archiving, analysis, or sharing across locations.
specific protocols are frequently used by smart sensors to ensure smooth Additionally, cloud storage offers built-in security features, including
connectivity. Third, embedded software and firmware provide smart encryption, secure APIs, and managed access. For the WAAM DAU, a
sensors their intelligence and allow them to carry out operations like hybrid approach could combine the low-latency benefits of local storage
data interpretation, analysis, and filtering. Finally, smart sensors have for immediate data processing with cloud storage for long-term analysis
energy-efficient power management algorithms built in, which enables and backup. Balancing these storage options ensures that the system
them to function for longer periods of time without always needing new remains efficient, adaptable, and secure, with the ability to support both
batteries. These sensors provide a thorough understanding of the AM real-time monitoring and extended data processing needs, all while
process. Smart sensors can self-calibrate and frequently have self- addressing specific WAAM operational requirements.
monitoring features, which guarantee accurate and consistent data Multi-sensor integration enables the DAU to combine data from
over time. Implementing smart sensors involves selecting appropriate multiple sources—such as temperature, voltage, and current sen­
sensors for WAAM’s specific requirements, establishing communication sors—offering a holistic system understanding. High sampling rates are
protocols, and integrating them into the overall control and monitoring critical for accurately capturing the rapidly changing parameters in
system. Overcoming obstacles like data security and integration dynamic systems like WAAM. By leveraging ML models, such as con­
complexity is crucial to fully leverage smart sensors, thereby enhancing volutional neural networks (CNNs) and anomaly detection algorithms,
the efficiency and quality of the WAAM process. the DAU can transform raw data into actionable insights, identifying
The third component, the DAU, is crucial in industrial operations and patterns and deviations from optimal conditions. Real-time monitoring
serves as the central system for gathering, analyzing, and storing real- capabilities enable the DAU to detect anomalies immediately, allowing
time data from various sensors and actuators. This unit monitors and for responsive adjustments to maintain consistent quality. For optimi­
optimizes critical parameters such as temperature, arc properties, and zation, data from the DAU feeds into predictive models that forecast
material deposition rates. Key characteristics include real-time data process performance, such as deposition rate stability or energy effi­
acquisition, multi-sensor integration, high sampling rates, and data ciency. For instance, using edge computing, the DAU processes data

4
M.M. Mahdi et al. Robotics and Computer-Integrated Manufacturing 94 (2025) 102944

locally, reducing latency and enabling instantaneous feedback to in WAAM [57]. Real-time analysis of manufacturing processes, facili­
adaptively modify parameters like feed rate or temperature based on tated by CNNs, enables the detection of abnormalities like weld bead
real-time observations. Additionally, the DAU’s data contributes to irregularities or surface flaws, minimizing waste and eliminating the
historical analysis for trend identification, supporting predictive main­ need for post-production inspections. Moreover, CNNs contribute to
tenance and reducing machine downtime by alerting operators to Process Optimization by analyzing manufacturing images to provide
impending issues before they impact production. By integrating with insights into optimal machine settings, which can then be used by
cloud-based storage, data processed by the DAU can be archived for automated control systems or human operators to enhance product
long-term analysis, facilitating improvements in future WAAM projects. quality and efficiency. Furthermore, CNNs can be trained to identify
The DAU’s data preprocessing capabilities ensure the accuracy and trends indicative of impending equipment failure, enabling predictive
consistency of the collected data, which is vital for effective process maintenance measures to be taken, thereby reducing downtime and
monitoring and optimization. prolonging equipment lifespan. CNNs empower adaptive control of
WAAM processes by automatically adjusting settings based on real-time
3.2. Middleware image data, environmental factors, and desired product specifications,
ensuring superior management and optimization of product quality.
The middleware of the proposed architecture is structured based on In addition to its role in process optimization and quality control, 3D
OPC UA with the additional support of big data analytics and cloud visualization also serves as a powerful educational tool for training en­
computation. OPC UA fundamentally establishes the communication gineers and operators. Through immersive VR and AR environments,
between the physical entity and the DT. Physical entities are often called operators can interact with the production process and understand how
legacy devices, which are driven by proprietary software and are diffi­ different factors influence the final product’s quality. This dynamic
cult to interface with software applications associated with DTs. As such, learning approach accelerates learning and encourages experimentation
OPC UA is required to bridge the gap between legacy devices and with new concepts in a risk-free virtual environment. The core func­
contemporary applications. Instituting communication between legacy tionality of 3D visualization lies in enhancing process visibility, enabling
devices and software at an enterprise level raises security issues. Data- operators to interpret data effectively and make informed decisions. By
intensive communications such as WAAM are often prone to cyber- providing an immersive, real-time representation of the production
attacks such as sniffing, tampering, spoofing, flooding, and denial of environment, 3D visualization serves as a powerful tool for under­
service (DoS), among others [54]. As such, OPC UA provides a secure standing the relationship between operational parameters and final
channel through its many services, such as certificates, private keys, product quality. Among other applications, 3D visualization assists op­
session management, encryption, and many more. In the client-server erators in identifying early-stage anomalies, such as deviations in part
communication setup of this study, the large volume of data handling geometry or unexpected surface defects, which might not be easily
often becomes computational and storage intensive. Hence, big data detectable through conventional monitoring. This visual feedback sup­
analytics can be used to resolve the issue by employing NoSQL database ports anomaly detection and prompts timely corrective actions, thus
and distributed file system. The computational complexity of the WAAM indirectly contributing to process optimization. Additionally, visuali­
process can be addressed by cloud computing leveraging the flexibility zation tools allow operators to assess the impact of parameter changes,
and scalability of virtualized resources through platform as a service such as adjustments to robot trajectories or welding speeds, on the
(PaaS) and software as a service (SaaS) [55]. overall process, facilitating iterative improvements.
The OPC UA client-server communication is where multiple OPC UA While optimization itself is typically achieved through data analysis
clients and servers can interact and fulfill dual roles, allowing them to and control systems, 3D visualization significantly aids the interpreta­
communicate with other components in the system. These components, tion of such analyses. It translates complex datasets into intuitive visual
acting as OPC UA applications, share common functionalities within the formats, enabling operators and engineers to assess trends, detect ir­
OPC UA framework. Each application is defined by details like its regularities, and validate changes before applying them to the physical
application description, which includes identifiers, certificates, names, process. This integration of 3D visualization with control systems and
and the application type. The application type specifies whether the data analysis ensures that monitoring insights are effectively leveraged
application acts as a client, server, both, or a discovery server. A dis­ to optimize production processes, bridging the gap between observation
covery server is responsible for locating all registered servers within the and actionable process improvements. However, integrating 3D visual­
system. The OPC UA information model allows for developing simple ization with WAAM poses several challenges, including the need for
and complex models, utilizing object-oriented techniques with features robust computing resources to handle large volumes of real-time data
like inheritance and type hierarchies. Pauker et al. highlight key prin­ and ensure data fidelity and quality in the 3D model. Specialized soft­
ciples guiding OPC UA information modeling, including the use of nodes ware solutions are required to seamlessly integrate this technology with
to establish diverse connections of information and the absence of strict existing WAAM systems, often necessitating significant adjustments to
modeling limitations, ensuring flexibility in design [56]. The re­ current operations. Additionally, comprehensive training for staff
sponsibility for information modeling always rests with the server side, members is essential to ensure proficient utilization of the technology
where nodes in the server address space represent objects, their defini­ and accurate interpretation of visual data.
tions, and their relationships. Moreover, computer vision systems in WAAM leverage sophisticated
cameras and advanced image processing techniques to analyze complex
3.3. Digital twin visual data in real time. The successful integration of multiple enabling
technologies is crucial for implementing computer vision in WAAM,
In WAAM, a digital twin (DT) refers to a suite of tools aimed at each contributing to enhancing the system’s overall functionality. High-
enhancing production accuracy, efficiency, and quality assurance. Key resolution cameras and sensors play a vital role in capturing minute
components include CNN for anomaly detection, 3D visualization for an details of the production process, enduring harsh environments common
in-depth representation of the WAAM process, and computer vision to in metal processing, and adapting to various illumination conditions.
recognize part parameters accurately. CNNs are vital in identifying and These components are essential for ensuring the efficacy and accuracy of
predicting irregularities during production and are trained on extensive the computer vision system in WAAM. Computer vision in WAAM fa­
datasets encompassing various WAAM scenarios. Employing state-of- cilitates defect identification by detecting anomalies in real-time, which
the-art ML techniques like CNNs holds promise for revolutionizing allows external systems or operators to take corrective actions, thereby
manufacturing processes such as WAAM. CNNs excel in handling image reducing waste and enhancing final product quality. Moreover, it en­
data, making them valuable for tasks like Quality Control and Inspection ables process optimization by providing data-driven insights for fine-

5
M.M. Mahdi et al. Robotics and Computer-Integrated Manufacturing 94 (2025) 102944

tuning manufacturing settings tailored to different materials and ge­ guarantees adaptive and optimized WAAM operations.
ometries. It also aids in identifying defects that contribute to equipment
wear or failures, providing critical insights that can be integrated with 4. Development and demonstration
predictive maintenance systems to minimize downtime and prolong
equipment life. A demonstration of the proposed architecture is presented in this
Real-time bidirectional communication across the 3D model and the section. A gas tungsten arc welding (GTAW) based WAAM is considered
physical entity is enabled by the dynamic feedback element of the DT the case scenario. As shown in Fig. 2, the system includes three main
architecture for WAAM. The DT mainly uses processed insights from sectors: physical entity, middleware, and DT. A brief description of each
computer vision, 3D visualization, and CNN-based anomaly detection is followed.
algorithms to provide feedback. Among these insights are anomalies
found, part parameters optimized, and robot movements modified. The 4.1. Physical entity
feedback control block receives this information and dynamically
transforms it into useful changes for the physical object. In particular, The physical entity comprises the physical robot data module and the
CNNs identify production inconsistencies, including surface imperfec­ process signature module. The former includes a Miller Dynasty 400
tions or irregular weld beads, and communicate this knowledge to the GTAW power supply, a standard wire feeder, and a six-axis FANUC
feedback system. A thorough spatial depiction of the manufacturing ArcMate 120iC robot arm with an R-3iB controller. The robotic arm and
environment is also provided by the 3D visualization component, which the power supply controls regulate the welding current, wire feed rate,
also helps to spot any spatial irregularities or discrepancies. The com­ and speed. The FANUC ArcMate 120iC is a six-degree-of-freedom in­
puter vision element adds to the feedback loop by capturing parameter dustrial robot, and to replicate its movement during the WAAM process,
deviations and part signatures in real time. To align the physical system certain parameters need to be accessed, namely, joint velocity, joint
with the intended production quality, this combined data from the DT is torque, joint angle, and joint position. When transformed into the 3D
analyzed to produce updates on machine settings, robot motions, or cartesian coordinate system, such parameters give the position of the
other process parameters. The OPC UA Client is used by the feedback robotic arm’s tool center point (TCP). FANUC Software Development Kit
loop to send updates, like modified robot trajectories or parameter (SDK) version 9.4 is used to access the real-time robot movement
changes, to the physical entity. By ensuring that identified problems are parameter. On the other hand, the teach pendant was used in conjunc­
fixed quickly, these real-time updates improve production precision and tion with the robot controller to control the welding process.
effectiveness while reducing waste. By bridging the gap between virtual The second module acquires process signatures. A Weldvis WL2-
insights and actual implementation, this dynamic feedback mechanism H7ML-M35 high dynamic range (HDR) camera at a sampling rate of
turns the DT from a passive monitoring tool into an active system that 50 frames per second (fps) is integrated with the robotic arm to observe

Fig. 2. Demonstration case of WAAM DT using Azure Blob Storage implementing anomaly detection, part parameter detection, and real-time joint movement.

6
M.M. Mahdi et al. Robotics and Computer-Integrated Manufacturing 94 (2025) 102944

the geometry of the weld pool, bead, metal transfer, and arc creation safeguards data at rest using both Microsoft-managed and
[58]. Each experiment produced an mp4 file, subsequently divided into customer-managed keys. Secure network connections are emphasized,
separate .jpg files while maintaining the original sampling rate. Each with HTTPS recommended for data in transit. Service endpoints and
image was cropped to eliminate noise and imperfections, focusing solely Network Security Groups (NSGs) bolster network security by limiting
on the melt-pool area. The frames were then resized to 224×224 pixels traffic to the Azure backbone and providing rule-based control over
based on the average coordinates determined by the You Only Look traffic.
Once (YOLO) object recognition algorithm, ensuring consistency across These communications among the various entities using Azure Blob
all frames. Data labeling was performed by two experts, classifying each Storage are in Fig. 4. First, the raw images of the WAAM process and the
frame into one of three categories: normal, abnormal, or unclassified. In robot motion are collected from the HDR camera and the 3D web
normal conditions, the bead geometry is uniform; in abnormal cases, cameras. These are then automatically uploaded to the “image­
defects such as balling or thinning are observed, and unclassified frames container” of the storage account through the connection string. Using
include mixed normal and abnormal images, preheating, and processing the “imagecontainer” blobs, anomaly detection is performed locally, and
initiation or termination frames. the results and performance plots are exported to the “cnn” container.
Furthermore, from the raw robot motion images, the joint angles are
calculated using the ArUco markers [60] locally and exported in
4.2. Middleware (Azure Blob Storage)
real-time as a .csv file in the “statuscontainer”. In addition, the
connection status of the robot is updated in the “statuscontainer”. To
Communication between the physical entity and DT occurs through a
visualize the part parameters, we obtain the raw image data from the
cloud storage system known as Microsoft Azure Blob storage. It is a filing
“imagecontainer”. Followed by the images are processed using the
system in the cloud environment that can store binary, text-based, and
digital image processing technique to obtain part parameters. These part
application installer-type data, broadly encompassing documents,
parameter values, and the processed video file are then uploaded to the
media files, and many more. The Azure Blob storage generally comprises
“partparameter” container. In the Unity visualization interface, we get
a hierarchical structure of storage accounts, containers, and blobs. Data
the data from the containers in real-time to display the values.
in the storage account can be accessed by other software, in our case,
Unity and Python, using the unique connection string or the connection
key. The storage account generally consists of several containers that are 4.3. Digital twin
used to group the blobs. The container can be considered analogous to
the filing system of any computer operating system. In our case, as The DT is the integrated approach that monitors, controls, computes,
shown in Fig. 3, the containers consist of four containers: the CNN and simulates the WAAM process which involves 3D visualization of the
anomaly detection (“cnn”), the raw images (“imagecontainer”), and the physical robot and the CNN prediction of the anomalies. In this
robot motion (“statuscontainer”) and parameters (“partparameter”). In demonstration, it is divided into 3D visualization, anomaly detection,
addition, Azure Blob Storage offers blob Access Tiers catering to diverse and part/parameter detection.
data access patterns. In this paper, Hot Access Block blobs were utilized The virtualization of a robotic arm begins with precise 3D modeling
for our operations. to create an accurate DT. This process involves producing a detailed CAD
In addition, Azure Blob Storage prioritizes data security through model in SolidWorks, which describes the robot’s linkages and joint
various mechanisms to ensure confidentiality, integrity, and availabil­ locations. The focus of this study is on displaying the CAD drawing
ity. Authorization and authentication are enforced through Shared Ac­ within a game engine environment. Direct incorporation of CAD designs
cess Signatures (SAS), protecting account key secrecy and enabling into game engines requires format conversion. Initially, the CAD file (.
restricted and time-limited access. Integration with Azure AD simplifies STP) generated in SolidWorks is transformed into a Parasolid file (.x_t)
centralized identity management and Role-Based Access Control (RBAC) format using the same software, allowing the robot’s joints and links to
[59]. Encryption, including Storage Service Encryption (SSE), be converted into separate entities rather than a single solid body. This

Fig. 3. Storage architecture of Azure cloud account.

7
M.M. Mahdi et al. Robotics and Computer-Integrated Manufacturing 94 (2025) 102944

Fig. 4. Data communication using Azure Blob Storage.

Parasolid file is then converted into a computer graphics format (.blend) tends to move if the Base is rotated.
using CAD Exchanger, which facilitates its import into Blender. Within A three-camera setup along the three cardinal axes of the robotic arm
Blender, specific details such as parent-child relationships, pivot points, was exercised, as shown in Fig. 7. Since the robot has six DOFs, we have
and tool center points are defined to ensure proper articulation and used six 4 × 4 ArUco markers ranging from marker identification
movement. Finally, the graphics file is exported in the .fbx format, which number of 0 to 5 for joints 1 to 6, respectively. The markers were
can be seamlessly incorporated into the Unity engine for the accurate positioned so that once a certain joint of the robotic arm is rotated, the
digital representation of the physical robotic arm, as shown in Fig. 5. marker is rotated with it, which is captured by the axial cameras to be
Fig. 6 shows the configurations and the defined axes of the robotic converted to the respective joint angles. To obtain the joint angle of the
arm joints. The parent-child connection is established using Blender markers, a Python script was developed that detects each marker’s
software, making the Base act as the parent while the subsequent link­ center and corner points for determining the relative rotations. The
ages act as the offspring. The Base is also denoted as the parent to all relative rotation of the markers in degrees, along with the marker
subsequent links. For instance, Axis 1 is considered a parent to Axis 2 identifier, is then exported to a .csv file format and uploaded to the cloud
and a child of the Base. The links that make up this parent-child rela­ platform to be accessed by the Unity interface.
tionship are organized into a hierarchical tree structure generated by To perform the anomaly detection, three CNN models – AlexNet,
Blender software. The idea of creating this connection is to manage ConvNeXt, and Swin Transformer – were employed, as detailed in Fig. 8.
mobility. For instance, if Axis 1 is rotated, all linkages from Axis 2 to 6 AlexNet, known for its GPU-accelerated training capabilities, features
will also rotate because Axis 1 is declared a parent to Axis 2 to 6, but the five convolutional layers with max pooling and three fully connected
Base will not be affected by this change. Conversely, the complete model layers, incorporating ReLU activation functions and dropout techniques
to mitigate overfitting [61]. In addition, ConvNeXt adopts a vision
transformer (ViT) architecture, utilizing patchify layers for image divi­
sion into flattened sequential patches, facilitating translational equiv­
ariance and scalability. ConvNeXt enhances performance by
implementing larger kernel sizes and the Gaussian Error Linear Unit
(GELU) activation function, resulting in fewer activation functions
required overall [62]. Swin Transformer, another ViT variant, utilizes a
hierarchical approach by splitting images into shifted, non-overlapping
windows to capture long-range dependencies and complex spatial re­
lationships, offering improved anomaly detection capabilities [63].
The data division process was structured to ensure the training,
testing, and validation of the CNN and the transformer models. Initially,
Fig. 5. Format conversion of CAD drawing.

8
M.M. Mahdi et al. Robotics and Computer-Integrated Manufacturing 94 (2025) 102944

Fig. 6. Robotic arm joints showing (a) the parent-child configuration and (b) the six axes.

Fig. 7. Camera setup to detect real-time joint angles using ArUco digital markers.

the training dataset comprised 7933 images, while the testing dataset a smaller dataset, we finalized the datasets to include 688 images for
contained 2752 images, providing a comprehensive basis for model training, 252 images for validation, and 301 images for testing. This
evaluation. Further, the training dataset was split into a 70:30 ratio for adjustment allowed for assessing the model’s generalization capability
learning and validation purposes, following standard ML practices to with a reduced dataset, which is crucial for scenarios where data
ensure a balanced evaluation of model performance. The data pre­ availability is constrained, such as in real-world WAAM processes.
processing steps in this regard included data cropping, labeling, and Although feature selection techniques were not explicitly applied, the
augmentation. Data augmentation techniques, including height shifting, approach implicitly optimized the feature extraction process by
width shifting, and zooming, were applied to address class imbalance leveraging CNN architectures, which are inherently designed to identify
and increase the number of training images. For example, the training and learn relevant features automatically. By evaluating the models on a
dataset was augmented from an original 7933 images to 63,291, smaller, well-prepared dataset, it was ensured that the study provides
significantly improving the models’ capacity to learn from diverse data insights into their performance in both resource-rich and constrained
variations. However, to streamline computations and optimize resource environments.
usage, while also observing the model’s ability to perform effectively on Pretraining and transfer learning strategies are employed to adapt

9
M.M. Mahdi et al. Robotics and Computer-Integrated Manufacturing 94 (2025) 102944

Fig. 8. Functional diagram of anomaly detection process using CNN and ViT.

pre-trained models, trained on datasets like ImageNet [64], for anomaly effectiveness of transfer learning in adapting existing models to highly
detection in the WAAM process, ensuring robust performance and effi­ specialized domains like WAAM.
cient utilization of available data. The source domain for the CNN and The choice of hyperparameters for the CNN and transformer models
transformer models transfer learning used in this study is the ImageNet as outlined in Table 1 used in this study was specifically tailored to meet
dataset, a comprehensive collection of diverse images that serves as a the requirements of anomaly detection in the WAAM process. Although
robust foundation for feature extraction through pre-training. In CNNs and transformers generally involve a broad range of tunable
contrast, the target domain is a specialized WAAM process dataset hyperparameters, the study focused on key parameters that had a sig­
focused on detecting anomalies in manufactured parts. This dataset nificant influence on model performance while ensuring computational
captures unique textures, shapes, and defect patterns that are specific to efficiency. Hyperparameters such as image size, patch size, depth,
the WAAM process. Transfer learning was applied based on the assess­
ment of feature transferability between the two domains. Both domains Table 1
share visual characteristics, such as textures and structural shapes, Selected hyperparameters for the CNN and ViT models.
which enable convolutional layers pre-trained on ImageNet to recognize
SwinTransformerV2 AlexNet ConvNext
features like basic shapes and edges that are directly relevant to
WAAM-specific anomaly detection tasks. Image size 224×224 224×224 224×224
Patch size 4×4 4×4
This similarity ensures that the features learned during pre-training
–
Depth 24 – 27
on ImageNet can be effectively transferred to the WAAM dataset, Embedding Dimension 128 – 1024
enhancing model performance despite the smaller size of the WAAM- Attention Heads 8 – –
specific dataset. The WAAM dataset comprises 1241 annotated im­ Window size 7×7 3×3 7×7
Dropout 0 0.5
ages, which, although smaller than the extensive ImageNet dataset, is –
Classes 2 2 2
sufficient for fine-tuning the pre-trained models. By employing transfer Activation GeLU ReLU GeLU
learning, the study leverages the extensive feature knowledge embedded Learning Rate 0.0001 0.0001 0.0001
in the pre-trained models, achieving robust anomaly detection while Optimizer ADAM ADAM ADAM
conserving computational resources and addressing the challenge of Batch Size 32 32 32
Epochs 5 5 5
limited WAAM-specific labeled data. This approach underscores the

10
M.M. Mahdi et al. Robotics and Computer-Integrated Manufacturing 94 (2025) 102944

embedding dimension, dropout, activation function, and optimizer were process underwent preprocessing steps, including resizing, grayscale
carefully selected to optimize the architecture of each model while conversion, Gaussian blur, and contrast filtering, to highlight regions of
addressing the specific demands of WAAM anomaly detection. interest for parameter detection. Techniques such as Canny edge
For all models, an image size of 224×224 pixels was chosen as it detection and Hough Line Transformation were employed for arc shape
offers a balance between computational efficiency and sufficient detail and feed angle detection, while binary thresholding was used for arc
to accurately capture anomalies in WAAM images. For transformer- length determination. Parameter values were then exported for further
based models like SwinTransformerV2 and ConvNeXt, a patch size of analysis, facilitating comprehensive monitoring and optimization of the
4 × 4 enabled effective feature extraction without imposing excessive WAAM process.
computational demands. Depth and embedding dimensions were
tailored to the architectures, with SwinTransformerV2 using a depth of 5. Results and discussion
24 and an embedding dimension of 128, while ConvNeXt employed a
depth of 27 and an embedding dimension of 1024, both designed for The proposed architecture enhances the WAAM process by inte­
effective complex feature learning. SwinTransformerV2 also incorpo­ grating various components. OPC UA facilitates communication be­
rated attention heads and window sizes (8 heads, 7 × 7 windows) to tween physical entities and the DAU. The middleware layer, using OPC
capture intricate spatial relationships. UA, ensures seamless data exchange and system integrity. Cloud
Regularization was addressed through dropout, with a 0.5 rate computation and big data storage accompany scalability and data
applied to AlexNet, and ConvNeXt to enhance generalization, while management. The DT leverages CNN for anomaly detection, 3D visual­
SwinTransformerV2, owing to its robust architecture, did not require ization for model synchronization, and computer vision for part
dropout. Activation functions were selected to complement each model: parameter detection. In the case study, the data is acquired from the
GeLU for SwinTransformerV2 and ConvNeXt, and ReLU for AlexNet, robotic arm, and the DT is used to visualize the process and detect
ensuring efficient non-linear computations. anomalies. Communication with Azure Blob Storage allows for real-time
And to detect the part parameters, three key parameters, arc shape, data storage and retrieval, enabling functionalities like robot joint angle
feed angle, and arc length, were investigated for their contributions to detection. In this section, the challenges faced during the implementa­
the characteristics of deposited parts in WAAM. The summary of the tion and the anomaly detection results are discussed.
process is depicted in Fig. 9. Arc shape plays a vital role in ensuring
uniform heat distribution and material deposition, which directly im­ 5.1. CNN anomaly detection
pacts the durability and integrity of manufactured components. Simi­
larly, feed angle influences heat dispersal, melting, and cooling rates, all The performance metrics are summarized in Table 2 in terms of ac­
of which affect the microstructure and mechanical properties. Mean­ curacy, precision, recall, and F1 score. The SwinTransformerV2 model
while, arc length controls heat input, interlayer fusion, and process demonstrates excellent performance with high accuracy (above 93 % on
stability, which is crucial for achieving consistent deposition quality and all datasets) and perfect recall (1.0), indicating its ability to detect
dimensional accuracy. Real-time video captured during the WAAM anomalies effectively while minimizing false positives. Despite slightly
higher accuracy than SwinTransformerV2, AlexNet maintains consistent
performance across all metrics, achieving perfect recall. This suggests its
effectiveness and reliability in anomaly detection. ConvNext performs
competitively with high accuracy (above 96 % on all datasets) and
perfect recall.
In Fig. 10, the confusion matrices of the models on all three datasets
are demonstrated. They are crucial for guiding model development,
optimization, and decision-making processes within classification tasks
[15]. In this study, it was found that the models had most of the pre­
dictions correct, as suggested by the true positives (sensitivity) and the
true negatives (specificity) along the principal diagonal. Additionally,
most misclassifications occur for the false positives ranging from 9 to 36
samples based on models and datasets. The false negative mostly re­
mains zero except for the Swin Transformer case.
Receiver Operating Characteristic (ROC) curves visualize a model’s
diagnostic by plotting the true positive rate against the false positive rate
across a spectrum of classification thresholds [65]. The Area Under the
Curve (AUC) metric quantitatively measures the models’ overall class
discrimination ability, with values closer to 1 signifying superior per­
formance. Fig. 11 shows the ROC curves of the various models over the
datasets. It is observed overall the AUC ranges from 0.8 to 0.92. The
ConvNeXT model shows an AUC of 0.92, reflecting better predictive
performance.

5.2. Joint angle measurement

In this study, a real-time joint angle detection and visualization

system within a Unity DT framework was developed. Leveraging
OpenCV’s capabilities, the system identifies and tracks Aruco markers
attached to physical joints, transmitting the extracted joint angles to the
Unity environment via Azure Blob Storage. This integration facilitates
the creation of interactive DTs that accurately reflect the physical
movements and configurations of robotic systems. However, challenges
Fig. 9. Steps depicting the part parameters detection. such as flickering joint movements, limited camera field of view, and

11
M.M. Mahdi et al. Robotics and Computer-Integrated Manufacturing 94 (2025) 102944

Table 2
Performance metrics of CNN and ViT models.
SwinTransformerV2 AlexNet ConvNext

Train Validation Test Train Validation Test Train Validation Test

Accuracy 0.94 0.93 0.93 0.95 0.96 0.94 0.96 0.96 0.96
Precision 0.920354 0.902062 0.9018405 0.934831 0.941489 0.907975 0.949772 0.941489 0.942675
Recall 1.000 0.988701 0.9932432 1.000 1.000 1.000 1.000 1.000 1.000
F1 Score 0.958525 0.943396 0.9453376 0.966318 0.969863 0.951768 0.974239 0.969863 0.970492

Fig. 10. Confusion matrices (CM) of CNN and ViT over the defined dataset.

time delays between physical and virtual movements were encountered, tracking, where the chances of losing track of a marker are minimized as
impacting the smooth portrayal of the physical entity within Unity. As multiple cameras cover the same axis. However, this configuration
shown in Fig. 12, a one-camera-per-axis setup [66] and a introduced a new challenge: a limited field of view for each camera,
two-camera-per-axis setup [67] were considered for the joint angle which restricts the motion capture capability of the DT. When markers
measurement. In the one-camera setup, significant issues were identi­ move out of the combined field of view of both cameras, it results in gaps
fied, including flickering, false detection of Aruco markers, and or abrupt changes in the detected joint angles. To address these issues,
considerable latency. Flickering occurred due to inconsistencies in several technical adjustments are recommended. For the one-camera
marker detection, often caused by lighting variations, motion blur, and setup, enhancing the camera’s setup with consistent lighting and
partial occlusion of markers. False detection happens when the camera higher image quality can improve detection stability. Additionally,
misidentifies or loses track of the markers, a problem intensified by the calibrating detection parameters and employing data smoothing algo­
limitations of the Logitech C920x HD Pro Webcam’s image quality and rithms, such as Kalman filters or moving averages, can reduce flickering.
the rapid movements or environmental changes in the physical setup. To address latency, buffering mechanisms, and redundancy protocols
The significant latency arose from the delays in transmitting data from can ensure more stable and reliable data transmission. For the two-
the physical entity to the DT, where the use of Azure Blob Storage can camera setup, strategically optimizing the cameras’ placement to
introduce additional lag due to data upload and retrieval times. maximize their combined field of view can help capture a broader range
In contrast, the two-camera setup showed a reduced incidence of of motions. Furthermore, integrating additional cameras or using
flickering and false detection due to improved redundancy in marker higher-resolution cameras can improve the overall detection capability.

12
M.M. Mahdi et al. Robotics and Computer-Integrated Manufacturing 94 (2025) 102944

Fig. 11. ROC and AUC of CNN and ViT over the defined dataset.

Fig. 12. Joint angle measurement visualization in Unity.

13
M.M. Mahdi et al. Robotics and Computer-Integrated Manufacturing 94 (2025) 102944

Finally, refining the transformation process within Unity to accurately using confusion matrices and ROC curves. ConvNeXt demonstrated the
translate the received local Euler angles can ensure that the DT move­ best predictive performance with an AUC of 0.92. The joint angle
ments closely match those of the physical counterpart, achieving a more detection system, integrated into the Unity DT framework, provided
realistic and immersive digital representation. real-time visualization of the robotic arm movements. However, chal­
lenges such as flickering movements, limited camera field of view, and
5.3. Future direction data transmission delays were identified. To mitigate these issues, en­
hancements in camera setup, data smoothing algorithms, and reliable
The current work has laid the groundwork for real-time joint angle data transmission methods were suggested.
detection and visualization through a DT. In the future, several potential Overall, This study offers a significant advancement in WAAM by
directions can enhance this foundation. The future research directions of addressing several critical challenges. First, it provides real-time moni­
this study are as follows: toring and control capabilities, enabling real-time anomaly detection.
Second, the integration of CNNs for defect detection and prediction
• Implementing standardized information models such as Automa­ enhances the quality and reliability of the WAAM process and products.
tionML or Open Automation Description (OpenAD) promotes inter­ Third, the use of OPC UA ensures secure and efficient communication
operability with different software tools and facilitates seamless data between the physical entities and their digital counterparts. Finally, the
exchange across diverse applications. 3D visualization component provides a comprehensive understanding of
• Leveraging big data analytics frameworks like Apache Spark or the WAAM process, facilitating decision-making and troubleshooting.
Hadoop to analyze historical sensor data can identify trends, predict By leveraging these technologies, our DT architecture has the potential
maintenance needs, and optimize robot performance. to revolutionize WAAM, leading to increased productivity, reduced
• Integrating more robust ML and AI techniques, such as explainable costs, and improved product quality. In addition, this work underscores
AI and generative AI, for anomaly detection and filtering will help the importance of integrating advanced data management and ML
identify and correct outliers in the captured data, ensuring the ac­ techniques to optimize AM processes, paving the way for smarter
curacy and reliability of the DT. manufacturing systems.
• Exploring real-time communication protocols like WebSockets or
UDP could lower latency compared to Azure Blob Storage. CRediT authorship contribution statement
• Implementing data compression techniques to reduce bandwidth
usage, particularly with multiple cameras or high-resolution images, Mohammad Mahruf Mahdi: Writing – original draft, Visualization,
can improve transmission efficiency and reduce latency. Validation, Software, Methodology, Formal analysis, Data curation.
• Enhancing the DT representation with physics engines in Unity can Mahdi Sadeqi Bajestani: Writing – review & editing, Methodology,
create a more realistic simulation of the robot movements. Formal analysis, Conceptualization. Sang Do Noh: Writing – review &
• Developing a user interface in Unity for real-time visualization, editing, Methodology, Investigation, Funding acquisition, Conceptuali­
manipulation, and analysis of the DT will facilitate tasks like human- zation. Duck Bong Kim: Writing – review & editing, Supervision,
in-the-loop control, visualization of historical data, and real-time Project administration, Methodology, Investigation, Conceptualization.
monitoring of sensor data.
• Integrating the system with robot control software frameworks like
Robot Operating Systems (ROS) can provide a more holistic control Declaration of competing interest
and monitoring platform.
• Designing the system for modularity will allow easier integration The authors declare that they have no known competing financial
with various robots, cameras, sensors, and additional welding pa­ interests or personal relationships that could have appeared to influence
rameters, making it adaptable and extensible for diverse industrial the work reported in this paper.
and research applications.
Acknowledgments
By focusing on these areas, the DT for WAAM can be significantly
improved, offering a robust, accurate, versatile, secure, and This research was supported by Brain Pool program funded by the
information-rich system for advanced robotics, welding process opti­ Ministry of Science and ICT through the National Research Foundation
mization, predictive maintenance, and remote monitoring. of Korea (RS-2023-00262501). This research was supported by the MSIT
(Ministry of Science, ICT), Korea, under the Global Research Support
6. Conclusion Program in the Digital Field program (RS-2024-00411653) supervised
by the IITP(Institute for Information & Communications Technology
This research presents a DT-based architecture for WAAM, Planning & Evaluation).
leveraging OPC UA for secure and efficient communication. The pro­
posed architecture incorporates advanced 3D visualization, real-time Data availability
defect prediction using CNN, and structured data management,
enhancing the monitoring and optimization capabilities of WAAM. The Data will be made available on request.
DT system operates on both enterprise management and individual asset
scales, facilitating real-time inspection, simulation-based planning, and References
sensor-based control. A practical case study using a 6-DOF industrial
[1] M. Liu, S. Fang, H. Dong, C. Xu, Review of digital twin about concepts,
robotic arm demonstrates the architecture’s effectiveness in a single- technologies, and industrial applications, J. Manuf. Syst. 58 (2020) 346.
layer bead deposition scenario. Key components of the system include [2] A. Phua, C.H.J. Davies, G.W. Delaney, A digital twin hierarchy for metal additive
CNN-based anomaly detection, computer vision for part parameter manufacturing, Comput. Ind. 140 (2022).
[3] Y.H. Son, G. Kim, H.C. Kim, C. Jun, S.D. Noh, Past, present, and future research of
detection, and communication with Azure Blob Storage for real-time
digital twin for smart manufacturing, J. Comput. Des. Eng. 9 (2023) 1.
data management. [4] M.R. Enders, N. Hoßbach, Dimensions of digital twin applications-a literature
The anomaly detection module, utilizing CNN models like Swin­ review, AMCIS 2019 Proceedings 25 (2019) 20.
TransformerV2, AlexNet, and ConvNeXt, showed high accuracy and [5] T. Mukherjee, T. DebRoy, A digital twin for rapid qualification of 3D printed
metallic components, Appl. Mater. Today 14 (2019) 59–65.
recall in detecting defects. The models’ performance metrics, such as [6] W.E. Frazier, Metal additive manufacturing: a review, J. Mater. Eng. Perform. 23
accuracy, precision, recall, and F1 score, were thoroughly evaluated (2014) 1917.

14
M.M. Mahdi et al. Robotics and Computer-Integrated Manufacturing 94 (2025) 102944

[7] C.R. Cunningham, J.M. Flynn, A. Shokrani, V. Dhokia, S.T. Newman, Invited [36] C. Liu, H. Vengayil, Y. Lu, X. Xu, A cyber-physical machine tools platform using
review article: strategies and processes for high quality wire arc additive OPC UA and MTConnect, J. Manuf. Syst. 51 (2019) 61.
manufacturing, Addit. Manuf. 22 (2018) 672. [37] U. Urbas, R. Vrabič, N. Vukašinović, Displaying product manufacturing
[8] S.W. Williams, F. Martina, A.C. Addison, J. Ding, G. Pardal, P. Colegrove, Wire + information in augmented reality for inspection, Procedia CIRP 81 (2019) 832.
Arc additive manufacturing, Mater. Sci. Technol. 32 (2016) 641. [38] C. Linn, S. Bender, J. Prosser, K. Schmitt, D. Werth, Virtual remote inspection—A
[9] C. Greer, A. Nycz, M. Noakes, B. Richardson, B. Post, T. Kurfess, L. Love, new concept for virtual reality enhanced real-time maintenance, 23rd International
Introduction to the design rules for metal big area additive manufacturing, Addit. Conference on Virtual System & Multimedia (VSMM) (2017) 1–6.
Manuf. 27 (2019) 159–166. [39] L. Stan, A.F. Nicolescu, C. Pupăză, G. Jiga, Digital Twin and web services for
[10] S. Jadhav, G.H. Jeong, M.S. Bajestani, S. Islam, H. Lee, Y.T. Cho, D.B. Kim, robotic deburring in intelligent manufacturing, J. Intell. Manuf. 34 (2022) 2765.
Investigation of surface roughness, microstructure, and mechanical properties of [40] H. Mu, F. He, L. Yuan, H. Hatamian, P. Commins, Z. Pan, Online distortion
overhead structures fabricated by wire arc additive manufacturing, Int. J. Adv. simulation using generative machine learning models: a step toward digital twin of
Manuf. Technol. 131 (2024) 1–21. metallic additive manufacturing, J. Ind. Inf. Integr. 38 (2024) 100563.
[11] B. Wu, Z. Pan, D. Ding, D. Cuiuri, H. Li, J. Xu, J. Norrish, A review of the wire arc [41] J. Qin, F. Hu, Y. Liu, P. Witherell, C.C. Wang, D.W. Rosen, T.W. Simpson, Y. Lu,
additive manufacturing of metals: properties, defects and quality improvement, Q. Tang, Research and application of machine learning for additive manufacturing,
J. Manuf. Process. 35 (2018) 127. Addit. Manuf. 52 (2022) 102691.
[12] D. Bong Kim, G. Shao, G. Jo, A digital twin implementation architecture for wire + [42] S. Shin, S. Hong, S. Jadhav, D.B. Kim, Detecting balling defects using multisource
arc additive manufacturing based on ISO 23247 34, Manuf. Lett. (2022) 1–5. transfer learning in wire arc additive manufacturing, J. Comput. Des. Eng. 10
[13] H. Mu, F. He, L. Yuan, P. Commins, H. Wang, Z. Pan, Toward a smart wire arc (2023) 1423–1442.
additive manufacturing system: a review on current developments and a [43] Y. Li, J. Polden, Z. Pan, J. Cui, C. Xia, F. He, H. Mu, H. Li, L. Wang, A defect
framework of digital twin, J. Manuf. Syst. 67 (2023) 174. detection system for wire arc additive manufacturing using incremental learning,
[14] E. Kim, D. Lee, G. Seo, D. Kim, S. Shin, Development of a CNN-based real-time J. Ind. Inform. Integr. 27 (2022) 100291.
monitoring algorithm for additively manufactured molybdenum, Sens. Actuat. A: [44] A. Yaseer, H. Chen, Machine learning based layer roughness modeling in robotic
Phys. 352 (2023) 114205. additive manufacturing, J. Manuf. Process. 70 (2021) 543–552.
[15] H. Cho, S. Shin, G. Seo, D.B. Kim, D. Lee, Real-time anomaly detection using [45] S. Shin, An OPC UA-compliant interface of data analytics models for interoperable
convolutional neural network in wire arc additive manufacturing: molybdenum manufacturing intelligence, IEEE Trans. Ind. Inf. 17 (2021) 3588.
material, J. Mater. Process. Technol. 302 (2022) 117495. [46] G. Buchgeher, B. Dorninger, C. Klammer, A. Walchshofer, A. Kern, Migrating
[16] M.S. So, G.J. Seo, D.B. Kim, J. Shin, Prediction of metal additively manufactured cyber-physical systems to OPC UA, Procedia Comput. Sci. 200 (2022) 276.
surface roughness using deep neural network, Sensors 22 (2022) 7955. [47] T. Müller, S. Kamm, A. Löcklin, D. White, M. Mellinger, N. Jazdi, M. Weyrich,
[17] C. Linn, S. Bender, J. Prosser, K. Schmitt, D. Werth, Virtual remote inspection-a Architecture and knowledge modelling for self-organized reconfiguration
new concept for virtual reality enhanced real-time maintenance, 23rd International management of cyber-physical production systems, Int. J. Comput. Integr. Manuf.
Conference on Virtual System & Multimedia (VSMM) (2017) 1-6. 36 (2022) 1842.
[18] D.B. Kim, M.S. Bajestani, J.Y. Lee, S. Shin, S.D. Noh, Introduction of human-in-the- [48] A. Martins, J. Lucas, H. Costelha, C. Neves, CNC machines integration in smart
loop in smart manufacturing (H-SM), Int. J. Precis. Eng. Manuf.-Smart Technol. 2 factories using OPC UA, J. Ind. Inf. Integr. 34 (2023) 100482.
(2024) 209–214. [49] F. Martinelli, O. Osliak, P. Mori, A. Saracino, Improving security in industry 4.0 by
[19] Y. Lu, X. Xu, L. Wang, Smart manufacturing process and system automation – a extending OPC-UA with usage control, Int. Conf. Availab. Reliab. Security 15
critical review of the standards and envisioned scenarios, J. Manuf. Syst. 56 (2020) (2020) 1–10.
312. [50] M. Hildebrandt, K. Lamshöft, J. Dittmann, T. Neubert, C. Vielhauer, Information
[20] X. Zhang, S. Lim, C. Lee, W.S. Song, Y.C. Kim, M. Yu, S.H. Hong, N.H. Yoo, M. Wei, hiding in industrial control systems: an OPC UA based supply chain attack and its
Integration of 5G and OPC UA for Smart Manufacturing of the Future, in: 2023 detection, Proceedings of the 2020 ACM Workshop on Information Hiding and
IEEE/SICE International Symposium on System Integration (SII), 2023, pp. 1–6. Multimedia Security (2020) 115–120.
[21] M. Kröll, C. Cseh, Implementation model for digital retrofit for sustainable [51] T. Honda, Y. Shimazawa, T. Hamaguchi, Y. Hashimoto, Monitoring encrypted
production, Procedia Comput. Sci. 217 (2023) 486–494. communication with OPC UA, Int. J. Serv. Knowl. Manag. 6 (2022) 2.
[22] T. Shen, B. Li, Digital twins in additive manufacturing: a state-of-the-art review, [52] T. Ivanova, Y. Belev, I. Batchkova, Cybersecurity of OPC ua based cyber-physical
Int. J. Adv. Manuf. Technol. 131 (2024) 63–92. systems, Industry 4.0 6 (2021) 204–207.
[23] L. Jin, X. Zhai, K. Wang, K. Zhang, D. Wu, A. Nazir, J. Jiang, W. Liao, Big data, [53] O. Gilles, D.G. Pérez, P. Brameret, V. Lacroix, Securing IIot communications using
machine learning, and digital twin assisted additive manufacturing: a review, OPC UA pubsub and trusted platform modules, J. Syst. Archit. 134 (2023) 102797.
Mater. Des. 244 (2024) 113086. [54] Ö. Aslan, S.S. Aktuğ, M. Ozkan-Okay, A.A. Yilmaz, E. Akin, A comprehensive
[24] D.B. Kim, M.S. Bajestani, G. Shao, A. Jones, S.D. Noh, Conceptual architecture of review of cyber security vulnerabilities, threats, attacks, and solutions, Electronics
digital twin with human-in-the-loop-based smart manufacturing, ASME 12 (2023) 1333.
International Mechanical Engineering Congress and Exposition 87608 (2023). [55] L. Wang, X.V. Wang, Cloud-based Cyber-Physical Systems in Manufacturing,
V003T03A076. Springer, 2018.
[25] Y. Fan, J. Yang, J. Chen, P. Hu, X. Wang, J. Xu, B. Zhou, A digital-twin visualized [56] F. Pauker, T. Frühwirth, B. Kittl, W. Kastner, A systematic approach to OPC UA
architecture for flexible manufacturing system, J. Manuf. Syst. 60 (2021) 176. information model design, Procedia CIRP 57 (2016) 321–326.
[26] Y. Zhang, C. Zhang, J. Yan, C. Yang, Z. Liu, Rapid construction method of [57] F. He, L. Yuan, H. Mu, M. Ros, D. Ding, Z. Pan, H. Li, Research and application of
equipment model for discrete manufacturing digital twin workshop system, Robot. artificial intelligence techniques for wire arc additive manufacturing: a state-of-
Comput. Integrated Manuf. 75 (2022) 102309. the-art review, Robot. Comput. Integrated Manuf. 82 (2023) 102525.
[27] J. Duan, X. Gong, Q. Zhang, J. Qin, A digital twin–driven monitoring framework [58] D.B. Kim, H. Chong, M.M. Mahdi, S. Shin, Data-fused and concatenated-ensemble
for dual-robot collaborative manipulation, Int. J. Adv. Manuf. Technol. 125 (2023) learning for in-situ anomaly detection in wire and arc-based direct energy
4579. deposition, J. Manuf. Process. 112 (2024) 273–289.
[28] D.M. Botín-Sanabria, A. Mihaita, R.E. Peimbert-García, M.A. Ramírez-Moreno, R. [59] T.C. da Silva, Open-Source framework for digital forensics investigations, Master’s
A. Ramírez-Mendoza, J.D.J. Lozoya-Santos, Digital twin technology challenges and thesis, Universidade do Porto (Portugal) (2024).
applications: a comprehensive review, Remote Sens. (Basel) 14 (2022). [60] A. Poroykov, P. Kalugin, S. Shitov, I. Lapitskaya, Modeling ArUco markers images
[29] W. Jia, W. Wang, Z. Zhang, From simple digital twin to complex digital twin part II: for accuracy analysis of their 3D pose estimation, international conference on
multi-scenario applications of digital twin shop floor, Adv. Eng. Inf. 56 (2023). computer graphics and machine vision, (GraphiCon) 30 (2020) 14(1)–14(7).
[30] H. Li, X. Shi, B. Wu, D.R. Corradi, Z. Pan, H. Li, Wire arc additive manufacturing: a [61] A. Krizhevsky, I. Sutskever, G.E. Hinton, Imagenet classification with deep
review on digital twinning and visualization process, J. Manuf. Process. 116 (2024) convolutional neural networks, Communications of the ACM 60 (2017) 84–90.
293–305. [62] Z. Liu, H. Mao, C. Wu, C. Feichtenhofer, T. Darrell, S. Xie, A convnet for the 2020s,
[31] G.L. Knapp, T. Mukherjee, J.S. Zuback, H.L. Wei, T.A. Palmer, A. De, T. DebRoy, Proceedings of the IEEE/CVF conference on computer vision and pattern
Building blocks for a digital twin of additive manufacturing, Acta Mater. 135 recognition 10 (2022) 11976–11986.
(2017) 390–399. [63] Z. Liu, Y. Lin, Y. Cao, H. Hu, Y. Wei, Z. Zhang, S. Lin, B. Guo, Swin transformer:
[32] M. Pantelidakis, K. Mykoniatis, J. Liu, G. Harris, A digital twin ecosystem for hierarchical vision transformer using shifted windows 9 (2021) 10012–10022.
additive manufacturing using a real-time development platform, Int. J. Adv. [64] J. Deng, W. Dong, R. Socher, L. Li, K. Li, L. Fei-Fei, Imagenet: a large-scale
Manuf. Technol. 120 (2022) 6547. hierarchical image database, IEEE Conference on Computer Vision and Pattern
[33] R. Geng, M. Li, Z. Hu, Z. Han, R. Zheng, Digital Twin in smart manufacturing: Recognition 3 (2009) 248–255.
remote control and virtual machining using VR and AR Technologies, Struct. [65] P. Akbari, F. Ogoke, N. Kao, K. Meidani, C. Yeh, W. Lee, A.B. Farimani,
Multidiscip. Optim. 65 (2022) 321. MeltpoolNet: melt pool characteristic prediction in metal additive manufacturing
[34] F. Hu, Mutual information-enhanced digital twin promotes vision-guided robotic using machine learning, Addit. Manuf. 55 (2022) 102817.
grasping, Adv. Eng. Inform. 52 (2022) 101562. [66] M. Mahdi, 1 Camera Setup, YouTube, 2024. https://www.youtube.com/watch?
[35] R. Bonnard, J. Hascoët, P. Mognol, E. Zancul, A.J. Alvares, Hierarchical object- v=3FVvp6m14sg. accessed November 20, 2024.
oriented model (HOOM) for additive manufacturing digital thread, J. Manuf. Syst. [67] M. Mahdi, 2 Camera Setup, YouTube, 2024. https://www.youtube.com/watch?
50 (2019) 36–52. v=q-IM1mj6YPU. accessed November 20, 2024.

15