MANAS Journal of Engineering

ISSN 1694-7398 | e-ISSN 1694-7398

Volume 12, Issue 1, (2024) Pages 1-28

MJEN https://doi.org/10.51354/mjen.1197753

A Systematic Survey of Machine Learning and Deep Learning

Models Used in Industrial Internet of Things Security
Ersin Enes Eryılmaz1,*, Sedat Akleylek2, Yankı Ertek3, Erdal Kılıç4
1 Department of Computational Science, Samsun, Türkiye, [email protected], ORCID: 0000-0003-1163-970X
2 Department of Computer Engineering, Samsun, Türkiye, [email protected], ORCID: 0000-0001-7005-6489
3 Renaissance System and Technology Solutions, Ankara, Türkiye, [email protected], ORCID: 0000-0001-
7005-6489
4 Department of Computer Engineering, Samsun, Türkiye, [email protected], ORCID: 0000-0003-1585-0991

ABSTRACT ARTICLE INFO

IIoT “Industrial Internet of Things” refers to a subset of Internet of Things technology designed Review article
for industrial processes and industrial environments. IIoT aims to make manufacturing facilities,
energy systems, transportation networks, and other industrial systems smarter, more efficient Received: 01.11.2022
and connected. IIoT aims to reduce costs, increase productivity, and support more sustainable Accepted: 16.02.2024
operations by making industrial processes more efficient. In this context, the use of IIoT is Keywords:
increasing in production, energy, healthcare, transportation, and other sectors. IoT has become industrial internet of
one of the fastest-growing and expanding areas in the history of information technology. things,
Billions of devices communicate with the Internet of Things with almost no human intervention. IIoT security,
IIoT consists of sophisticated analysis and processing structures that handle data generated by deep learning,
internet-connected machines. IIoT devices vary from sensors to complex industrial robots. machine learning,
Security measures such as patch management, access control, network monitoring, Industry 4.0.
authentication, service isolation, encryption, unauthorized entry detection, and application
*Corresponding
security are implemented for IIoT networks and devices. However, these methods inherently author
contain security vulnerabilities. As deep learning (DL) and machine learning (ML) models have
significantly advanced in recent years, they have also begun to be employed in advanced
security methods for IoT systems. The primary objective of this systematic survey is to address
research questions by discussing the advantages and disadvantages of DL and ML algorithms
used in IoT security. The purpose and details of the models, dataset characteristics, performance
measures, and approaches they are compared to are covered. In the final section, the
shortcomings of the reviewed manuscripts are identified, and open issues in the literature are
discussed.

network use short-range technologies such as Bluetooth,

1. Introduction Zigbee, and Wi-Fi. These technologies will naturally be used
by default as long as these networks exist. IoT revenues will
The internet environment is changing at an incredible speed.
exceed $1,5 trillion by 2030. China, North America, and
The internet is not just about smartphones or laptops; it has Europe have 73% of IoT global revenue [2]. With the
gone beyond internet-connected devices. Physical devices expansion of the IoT ecosystem, security concerns are also
communicate with each other or large systems through the
increasing. Considering the IoT architecture brings together
Internet of Things (IoT). Users can complete their work in a
multiple pieces of sensing and communication. Integrating
short time with devices connected with IoT. As the future of
devices is not only a complex task but also demonstrates that
IoT looks so promising, it will be an integral part of every IoT networks and devices are a system that requires constant
device, from home appliances to security devices. attention [3], [4].
While the number of IoT devices worldwide was 15.14 billion
IoT can be divided into three main groups. Consumer IoT can
in 2023, it will reach 29.42 billion devices in 2030 and will
be considered end-user applications, smartphones, smart
increase 2.3 times [1] because it is cheaper and more
watches, wearable devices, and internet-connected home
accessible. Almost 75% of devices connected to the IoT
devices. Large infrastructures for enterprises are referred to as

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg

 E. E. Eryılmaz, S. Akleylek, Y. Ertek, E. Kılıç / MANAS Journal of Engineering 12(1) (2024) 1-28 2

Commercial IoT, while controllers, actuators, sensors, recurrent neural networks [27], long short-term memory [28],
industrial assets, remote telemetry, monitoring, and gated recurrent unit [20], [29], autoencoder [30], generative
management systems are classified as Industrial IoT. In this adversarial network [31], restricted Boltzmann machines [32]
survey, Industrial IoT (IIoT) will be discussed [4]. The IIoT is and deep belief networks [33] are used.
a new, fully connected, efficient vertical model for intelligent
systems and is vulnerable to cyber threats. Malicious actors 1.1. Related Surveys
can exploit some vulnerabilities and risks due to the
In this subsection, current manuscripts compiling recently
misapplication of security standards [6].
published or highly cited ML and DL-based models for IIoT
Automation and intelligent computing services such as
industrial systems, critical infrastructure devices, embedded security are reviewed. Some of these survey manuscripts used
devices, and modern systems have come together with the systematic literature survey method, and some consisted
of summarizing the papers. A systematic literature survey is
production engineering thanks to the internet. However,
the distinction and examination of papers prepared to answer
standardization with IIoT brings many new challenges,
research questions according to predetermined selection and
including legal and social aspects of security, and privacy. In
particular, the increasing diversity of IoT network and IoT elimination criteria related to a selected topic. Table 1
device presence requires highly scalable solutions for data summarizes the characteristics of the reviewed surveys.
IIoT security research activity is geographically dispersed, the
communication, naming, information management,
most popular broadcast locations, and fog computing for IIoT
addressing and service delivery. Many IoT devices still have
security threats [34]. IIoT security requirements refer to the
limited capabilities that require low-cost, low-power, fully
geographical distribution of scientific publications, popular
networked architectures compatible with standard
communication methods [7]. publication areas, and distribution over the years. In addition,
It is a well-known fact that IoT is an ecosystem where data is the future of fog computing in the industrial field is discussed
and proposed four-layer IIoT security architecture [35].
transmitted and requires some privileged features to manage
Firstly, security analysis includes MAC, ucode, IP, and EPC.
large amounts of data. At this point, ML and DL models
Analysis of the network layer is also available in capillary
collect and analyze data with artificial intelligence (AI). The
networks (HomePlug, BLE, Bluetooth, RFID, NFC, IrDA,
security of devices can be ensured by making predictions with
DL and ML models from the data produced by IoT INSTEON, EnOcean, ANT+, WirelessHART, UWB, ZigBee,
ecosystems. Using the AI concept in security ensures a regular Thread, and ISA 110.11a, etc.). Secondly, their coverage and
functionality ranges are mainnet in the background (3G, LR-
data flow between IoT devices and proper management
WAN, Ethernet, WiMax, WLAN) and backbone network
without human error. Thus, AI has become necessary in the
(DASH7, LoRaWAN, NB-Fi, NB-IoT, SigFox, NWAVE, and
growth of the IoT industry.
The communication protocol used in wearable technologies RPMA). At the processing layer, security analysis resides in
and industrial applications Bluetooth low energy: BLE has end-to-end data protection. Finally, application layers work on
HTTP, MQTT, CoAP, SOAP, XMPP, REST, DDS, and
been seen in many attacks where it is vulnerable to attacks.
AMQP protocols [36]. Protocol-based and data-based attacks
Since the packets transmitted with BLE consist of plain text
show that traditional IoT attack prevention tools are no longer
content, it has been seen to contain security vulnerabilities in
effective. Artificial intelligence methods, blockchain, and
user authentication and reconnection of two paired devices
[4], [5]. elliptic curve encryption seem to be new effective methods for
The increasing benefits of internet-connected devices have securing IoT networks [37]. IoT security threats and
countermeasures, common points, and differences between
also brought challenges related to security issues. With the
IoT and IIoT are defined. A literature review of different
widespread use of IoT devices, security problems have also
security approaches specific to IIoT [38]. Blockchain, AI
increased, and anomalies have occurred in IoT networks.
algorithms, consensus mechanisms, storage and
Anomalies in the IoT network and systems are detected by
intrusion detection systems. Work on IDS has been ongoing communication perspectives on smart supply chains, and
since Anderson's network security monitoring work [8]. Since Industry 4.0 are explained [39]. A comprehensive analysis of
attacks against IIoT systems and solutions to these attacks, as
Anderson's technical report, manuscripts have continued for
suggested in the latest literature, is presented [40]. DL and ML
different intrusion detection systems based on various
methods and blockchain integration for the IoT perception,
methods [9]-[15]. There are different approaches for detecting
network, and application layers are discussed [41]. Reviews
anomalies in IoT networks with DL and ML models [16]-[19].
The ML and DL algorithms used to detect anomalies in IIoT various DL techniques and their uses in different industries,
security recognize malicious network traffic by comparing it including CNN, AE, and RNN. DL use cases for intelligent
IoT technologies are summarized [42]. A systematic literature
to benign network traffic. In the papers, support vector
review specifically addressing DL and ML algorithms
machine [20], Bayes networks [21], decision trees [22], k-
commonly used in IoT network security is proposed, but does
nearest neighbors [23], random forest [24], and k-means [ 25],
machine learning algorithms are preferred. As deep learning not focus on IIoT [43]. A systematic survey of how deep
algorithms generally convolutional neural networks [26], learning approaches detect IoT network and system security
and large-scale attacks is studied [44]. An anomaly-based

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg

 E. E. Eryılmaz, S. Akleylek, Y. Ertek, E. Kılıç / MANAS Journal of Engineering 12(1) (2024) 1-28 3

systematic survey with ML and DL together; however, the have been summarized about ML or DL-based approaches to
datasets are not exhaustive [45] and [46]. 40 manuscripts were IoT security solutions on between 2017 and 2022 [119].
summarized in databases such as Google Scholar, Academia, One hundred five manuscripts were examined through
Science Direct, and IEEE with the keywords IoT, cyber different elimination and purification steps with the research
security, cyber security frameworks, and cyber security questions' queries. The use of DL has been claimed to be a
approaches. No information is provided about ML and DL- permanent and reasonable approach to IoT security. M. A. Al-
based algorithms and the databases used [115]. ML and DL- Garadi et al. explained DL and ML methods with dataset
based solutions for privacy threats in IoT systems were details, but it is not a systematic survey [41]. R. Ahmad & I.
analyzed with dataset features without a systematic survey Alsmadi gave a systematic review of manuscripts conducted
[116]. A detailed analysis of the IDS developed in the IoT in the years 2019-2020 specialized in IoT security, which
environment was performed and a new smart IDS was explains the ML and DL methods with dataset detail and is not
proposed, which was tested on the NS3 simulator using fuzzy an IIoT-specific review. Our manuscript differs from other
CNN by extracting features with information gain. This manuscripts in that it consists of systematically conducted
manuscript can be considered as a non-systematic detailed manuscripts with detailed datasets where IIoT-specific ML
survey that includes experiments and analysis [117]. Many and DL approaches were experimented on between 2019 and
manuscripts have been summarized about ML or DL-based 2023 [43], [119].
approaches to IoT security solutions [118]. Many manuscripts

Table 1. Deep Learning and Machine Learning Based Survey Papers for IIOT Security

Survey Article Title Journal Name Year Systematic ML and DL Anomaly Dataset
survey? together? based? detail?
[34] Towards a systematic survey of ACM Digital Library 2019 ✓ X ✓ X
industrial IoT security Proceedings of the
requirements: research method and Workshop on Fog
quantitative analysis Computing and the IoT
[35] A Systematic Survey of Industrial IEEE Communications 2020 ✓ X X X
Internet of Things Security: Surveys & Tutorials
Requirements and Fog Computing
Opportunities
[36] Recent Technologies, Security MDPI Sensors 2021 X X X X
Countermeasure and Ongoing
Challenges of Industrial Internet of
Things (IIoT): A Survey
[37] Security trends in Internet of SpringerLink SN 2021 X X X X
Things: A survey Applied Sciences
[38] Challenges and Opportunities in IEEE Transactions on 2021 X X X X
Securing the Industrial Internet of Industrial Informatics
Things
[39] Deep reinforcement learning for ScienceDirect /Elsevier 2021 X ✓ X X
blockchain in industrial IoT: A Computer Networks
survey
[40] Cyber Threats to Industrial IoT: A MDPI IoT 2021 X ✓ X X
Survey on Attacks and
Countermeasures
[41] A Survey of Machine and Deep IEEE Communications 2020 X ✓ ✓ ✓
Learning Methods for Internet of Surveys & Tutorials
Things (IoT) Security
[42] Deep Learning in the Industrial IEEE Internet of Things 2021 X X ✓ X
Internet of Things: Potentials, Journal
Challenges, and Emerging
Applications
[43] Machine learning approaches to ScienceDirect /Elsevier 2021 ✓ ✓ ✓ ✓
IoT security: A systematic Internet of Things
literature review
[44] A systematic review on Deep ScienceDirect/ Elsevier 2021 ✓ X ✓ ✓
Learning approaches for IoT Computer Science
security Review

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg

 E. E. Eryılmaz, S. Akleylek, Y. Ertek, E. Kılıç / MANAS Journal of Engineering 12(1) (2024) 1-28 4

[45] A Brief Review on Internet of MDPI Electronics 2022 ✓ ✓ ✓ X

Things, Industry 4.0 and
Cybersecurity
[46] State-of-the-art survey of artificial ScienceDirect/ Elsevier 2022 ✓ ✓ ✓ X
intelligent techniques for IoT Computer Networks
security
[115] Cybersecurity Risk Analysis in the MDPI Electronics 2023 ✓ X ✓ X
IoT: A Systematic Review
[116] A Survey of Machine and Deep MDPI Sensors 2023 X ✓ X ✓
Learning Methods for Privacy
Protection in the Internet of Things
[117] A Comprehensive Survey on Hindawi Computational 2023 X X ✓ ✓
Machine Learning-Based Intrusion Intelligence and
Detection Systems for Secure Neuroscience
Communication in Internet of
Things
[118] Internet of Things (IoT) Security SpringerLink Mobile 2023 X ✓ X X
Intelligence: A Comprehensive Networks and
Overview, Machine Learning Applications
Solutions and Research Directions
[119] Intelligent approaches toward ScienceDirect/ Elsevier 2023 ✓ ✓ ✓ ✓
intrusion detection systems for Journal of Network and
Industrial Internet of Things: A Computer Applications
systematic comprehensive review
Our A Systematic Survey of Machine - - ✓ ✓ ✓ ✓
paper Learning and Deep Learning
Models Used in Industrial Internet
of Things Security

1.2. Motivation, Scope and Contribution of Manuscript

In this systematic literature survey, we focused on the schemes The contributions of this survey are as follows:
for anomaly-based attack detection in the IIoT network. This • Research strategies with seven different academic
survey differs from the other survey manuscripts in Table 1 in databases were scanned.
that it is systematic, includes ML and DL models, is based on • Article scans were made systematically.
anomalies found in IIoT networks, includes details of the • Survey articles and literature were searched.
datasets used, and focuses on the framework of industrial • Frequently used abbreviations and metrics in the
internet of things. The general content and scope of this article and in the literature are explained in detail.
systematic review, which is prepared with the aim of • Manuscripts with machine learning and deep learning
providing detailed information to researchers working in the models have been researched.
field of IIoT security, are as follows:
• The main idea, advantages, and disadvantages of the
• The proposed schemes in the selected approaches are
proposed models are explained.
examined, and the information on these schemes is
• The benign and malignant numbers and features of the
briefly summarized.
• Manuscripts that propose models developed to detect dataset used in the models were extracted.
anomaly-based attacks in industrial IoT networks • The usage purposes, tasks, and performance results of
and reduce these attacks are systematically selected the models are given.
and eliminated according to specific criteria. • The models against which the proposed schemes are
• It is stated which ML and DL models are used in the compared, and open research problems are discussed.
approaches.
• Manuscripts with performance metrics have been 1.3. Organization
interpreted. In this systematic survey, to make the technical information
• Details of the datasets presented in the training and outlined in the manuscript more understandable, IoT
testing phases of the proposed models are given. architecture, IIoT concept, vulnerabilities in IIoT devices,
• The methods compared with the performance metrics some attacks against IIoT devices, and ML and DL models are
reached by the setup of the models are shown. briefly explained in Section 2. Section 3 explains the research
• In the final section, the deficiencies of the examined questions and objectives, search strategy, search process, and
manuscripts are outlined. Evaluation of what filtering criteria. The approaches taken in the selected
situations these deficiencies may lead to is presented. manuscripts are summarized in Section 4. Section 5 answers

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg

 E. E. Eryılmaz, S. Akleylek, Y. Ertek, E. Kılıç / MANAS Journal of Engineering 12(1) (2024) 1-28 5

the research questions that a systematic review should answer, manuscripts examined are emphasized. Finally, our
summaries of the models, advantages, and disadvantages, and manuscript briefly addresses general and open issues, offering
details of the datasets used. In Section 6, DL and ML models, a comprehensive overview of the broader challenges. Table 2
datasets, and their properties are evaluated, an overview of the shows the abbreviations and expansions frequently used in
models is presented, and the deficiencies encountered in the this survey.

Table 2. Abbreviations and Expansions

Abbreviation Expansion Abbreviation Expansion

IDS Intrusion Detection System MLP Multilayer Perceptron
DT Decision Tree AI Artificial Intelligence
NB Naive Bayes RNN Recurrent Neural Network
AE Autoencoder DAE Denoising Autoencoder
RBM Restricted Boltzmann Machines SAE Stacked Autoencoder
KNN K-Nearest Neighbors RF Random Forest
FDI False Data Injection GAN Generative Adversarial Networks
DoS Denial of Service LSTM Long Short-Term Memory
UDP User Datagram Protocol CART Classification and Regression Tree
DBN Deep Belief Network LR Linear Regression
BN Bayesian Network RT Random Tree
DDoS Distributed Denial of Service SDN Software Defined Networking
SVM Support Vector Machine ICMP Internet Control Message Protocol
NN Neural Networks ANN Artificial Neural Networks
MitM Man-in-the-middle TCP Transmission Control Protocol
CNN Convolutional Neural Networks ROC Receiver Operating Characteristic Curve
ACC Accuracy DR Detection Rate
TN True Negative AUC Area Under the ROC Curve
PRE Precision TRT Training Time
TP True Positive TET Testing Time
REC Recall F1 F1-Score
FP False Positive LL Log Loss
SPC Specificity G-mean Geometric mean
FN False Negative RI Rand Index
TPR True Positive Rate SR Speedup Ratio
SNS Sensivity ER Encryption Time
FPR False Positive Rate IMF Intrinsic Mode Function
FAR False Alarm Rate GCM Gradient Compression Mechanism
FNR False Negative Rate CCQ Clustering Center Quality
PoR Proof of Reliance KBL Kernel Based Learning
PoW Proof of Work RTU Remote Terminal Unit
FL Federated Learning PSO Particle Swarm Optimization
SSO Swallow Swarm Optimization CFS Cloud Service System
RS Random Subspace SHOCFS Secure High-Order Clustering with Fast Search
CEEMDAN Complete Ensemble Empirical Mode IABC Improved Artificial Bee Colony
Decomposition with Adaptive Noise
SCADA Supervisory Control and Data Acquisition SHODS3O-CFS Safe High-Order Optimum Density Selection in a Hybrid
Cloud Environment
AB Adaboost GXGBoost Genetic-Based Extreme Gradient Boosting
VIF Variance Isolation Forest MQTT Message Queuing Telemetry Transport

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg

 E. E. Eryılmaz, S. Akleylek, Y. Ertek, E. Kılıç / MANAS Journal of Engineering 12(1) (2024) 1-28 6

[35]. IoT consists of less scope, while IIoT consists of systems

2. Preliminaries that receive data from sensors, analyze it, and transfer it to the
cloud [38].
In this section, basic definitions and background are given. At
the same time, deep learning and machine learning models are
2.1.1. IoT and IIoT Layers
briefly explained.
IoT is a 3-layer structure: application, network, and
2.1. IoT, IIoT and Attack Types perception layer [38]. The perception layer provides the
outside world communication of IoT devices. It houses the
With IoT, using other machines to talk to other machines on actuators and sensors that generate data. At the perception
behalf of humans, the concepts of ubiquity apply. In the age layer, attacks such as physical attacks, impersonation, and
of IoT, where people communicate with objects and objects DoS are carried out. Some manuscripts explain the perception
communicate with each other, there are connectivity layer as the physical layer and the network layer as the
dimensions for everything and everyone, anytime, anywhere. communication layer [44].
Objects have identities and virtual personalities in the internet IoT devices are connected to the internet environment
of the future [47]. IoT is a network where physical devices can through the network layer. The network layer forms a bridge
communicate via the internet. IoT is to be connected to various between the perception and application layers. IoT networks
devices that make use of different communication models reach the internet with wired and wireless communication
from human to human, human to machine, or a machine to technologies at the network layer. At the network layer,
machine [48]. attacks such as DoS, MitM, and routing attacks are carried out.
In recent years, IoT has been used in automotive, energy, On the other hand, the application layer consists of the
health, manufacturing, water, finance, etc. It has entered a perception and network layer communication data and IoT
wide range of industry sectors, including IIoT. With machine applications. Therefore, it can be difficult to ensure security
learning, the IIoT will advance the fourth industrial due to the possibility of software changes creating different
revolution. While IIoT facilitates data collection in an bugs. As a result, application layer attacks like malicious code
industrial environment, the collected data are used for training injection, data leakage, and denial of service (DoS) are carried
algorithms with the help of ML and especially DL. out.
The industrial use of IoT technologies has emerged with the In the early stages of IoT-related research, three layers were
concept of Industry 4.0. IoT networks consist of structures that introduced. It has three layers: perception, network and
monitor, analyze and change data without human intervention. application layers. Three-layer architecture defines the main
SCADA systems also consist of several smart devices that idea of the Internet of Things, but it is not sufficient for IoT
monitor and control machines in industries for years [49]. IIoT research. New research describes more multilayered
standardization has emerged as a technology developed on architectures. IoT has five layers, the middleware, and the
SCADA with scalability, resolution, and data analytics. Using business layers, as well as the IoT detection, network,
AI methods, IIoT can create new security measures from data perception and application layers [87]. Middleware is a
collected from the cloud. [50]. system and software that uses data collected by the perception
IIoT is considered to be a subset of IoT. IoT typically layer and runs primarily on servers serving the upper layers.
encompasses retail and lifestyle consumer devices. IoT These software and services are part of training a new
usually consists of single device structures such as smart computationally demanding machine learning model.
television, smart phone, wearable devices, home automation, Application and business layers provide software for the end
and display systems. IIoT technologies, on the other hand, are user [86], [87]. Middleware and network layers are vulnerable
potent systems formed by the combination of more and to attacks such as MitM and DoS. In addition, attacks such as
advanced IoT devices such as smart factories, smart city, SQL injection, session hijacking, and buffer overflow occur at
smart grids, innovative vehicles, robotics. While the plans for the business and application layers. Some manuscripts have
IoT are between 2-5 years, 30-year frameworks are considered named the layers differently. These are called perception,
for IIoT systems. The IoT is sensitive to water, dust, and application, business, transport and preprocessing layers
power fluctuations and is highly mobile. IIoT is also suitable [114].
for operation in extreme situations, and its mobility is low.
While the IoT prioritizes critical operations, IIoT systems
2.1.2. IIoT Attacks and Countermeasures
have to synchronize in milliseconds. Since IIoT is built on
smart logistics, smart cities, and smart manufacturing Because IIoT is a natural evolution of IoT, there are similar
processes, it has to rely on broader security measures than IoT. security challenges and specific security concerns to protect
At the same time, security solutions that apply to the IoT also critical industrial control systems.
apply to the IIoT. The confidentiality, integrity, and There are always security vulnerabilities for devices
availability (CIA) triad is an elementary information security connected to the Internet. If security vulnerabilities are not
and includes security requirements and objectives. Solutions detected and fixed, devices turn into zombie and robot
for Industry 4.0 should be evaluated within this framework devices. Without security solutions, IoT devices turn into a

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg

 E. E. Eryılmaz, S. Akleylek, Y. Ertek, E. Kılıç / MANAS Journal of Engineering 12(1) (2024) 1-28 7

botnet. Large-scale attacks such as TCP timeouts and keeping machine (SVM), masking technique, and NOS middleware
HTTP connections open on web servers slowly consume the [60]-[65]. At the network layer, there are RFID spoofing,
server's resources and ultimately cause it to stop responding to traffic analysis attacks, routing information attacks,
legitimate requests. Other large-scale volume-based attacks unauthorized access, sinkhole attack, selective routing,
include SNMP, DDoS, TCP SYN packet, UDP flood, ICMP wormhole attack, MitM, Sybil attacks, DoS/DDoS attacks,
flood, slowdown, ping of death, zero-day attacks, known web replay attacks [57], [58], [66]. For attacks on the network
server exploits, scrambling attack, OpenBSD, and layer, have privacy-protecting traffic obfuscation framework,
amplification attack [51], [52]. The first purpose is to block SRAM-based PUF, hash chain authentication, cluster-based
IoT traffic and make it inaccessible to regular users. intrusion detection system, trust-aware protocol, secure
There are two main attack techniques, anomaly, and MQTT: cross-device authentication, beacon encryption,
signature-based. Signature-based attacks can be defined as EDoS Server: SDN-based IoT framework and machine
exploitation or knowledge-based attacks, and anomaly-based learning models [67]-[77]. At the application layer, there are
attacks can be defined as behavior-based attacks [53]. malware attacks such as viruses, worms, trojans, spyware, and
Signature-based techniques rely on existing threats to identify adware [57], [66]. The most well-known of these are the Mirai
attacks. Anomaly-based systems detect attacks based on botnet and Jeep hack attacks. Lightweight framework for
traffic patterns [54]. Systems that detect signature-based attacks on the application layer; high-level synthesis (HLS),
attacks work well for attacks, but updating the signature and malware image classification; there are prevention
database takes time. As datasets grow, it will become harder methods such as the lightweight neural network framework
to compare input. This method cannot detect Zero-day attacks [78]-[81]. There are also data attacks such as unauthorized
[55]. Anomaly detection systems block malicious traffic. access, data inconsistency, and data breaches. Chaos-based
Anomaly-based systems can detect unknown attack types and schema against data attacks; blockchain architecture,
zero-day attacks. However, too many false positives are blockchain-based ABE; privacy protection ABE, two-factor
encountered with anomaly prevention systems [56]. authentication; measures and methods such as DPP, ISDD,
Physical attacks such as RF interference or jamming, and machine learning [82]-[87]. As it can be seen, many
tampering, fake node injection, malicious code injection, prevention methods have been proposed for IoT attack types,
permanent denial of service (PDoS), sleep denial attacks, and and many of these proposed methods include machine
side channel attacks are made in the perception layer [57]- learning methods. Table 3 presents IIoT attack types and
[59]. Against these attacks, there are techniques such as PUF- suggested measures.
based Authentication, CUTE Mote, PAuthKey, support vector

Table 3. IIoT Attack Types and Recommended Measures

Attack Type IIoT Layer/Attack Recommended Measures Papers

Target
Side channel attack, RF interference or jamming, fake Perception Layer PUF-based authentication, CUTE Mote, PAuthKey, [60]-[65]
node injection, tampering, permanent denial of service machine learning methods, masking technique, NOS
(PDoS), malicious code injection, sleep denial attack. middleware.

RFID spoofing, traffic analysis attack, routing information Network Layer Privacy-protecting traffic obfuscation framework, [67]-[77]
attacks unauthorized access, Sinkhole-attack, selective SRAM-based PUF, hash chain authentication,
routing, wormhole-attack, MitM, Sybil-attack, DoS/DDoS clustering-based intrusion detection system, trust-aware
attacks, replay-attack. protocol secure MQTT; cross-device authentication,
digital signature, and encryption (signcryption), EDoS
Server; SDN-based IoT framework, machine learning
methods.

Malware attacks like viruses, worms, trojans, spyware and Application and Lightweight framework; high-level synthesis (HLS), [78]-[81]
adware, Mirai botnet, and jeep hack. Business Layer lightweight NN, malware image classification.

Data inconsistency, unauthorized access, and data breach. Middleware layer The chaos-based scheme, blockchain architecture, [82]-[87]
and Data Attack blockchain-based ABE; privacy protection ABE, two-
factor authentication; DPP, ISDD, and machine learning
methods.

2.2. Machine Learning and Deep Learning Methods increasing their accuracy. For example, SVM, BN, DT, KNN,
RF, and K-Means are machine learning, CNN, RNN, LSTM,
ML is a branch of AI and computer science that imitates how
GRU, GAN, RBM, DBN, and AE are deep learning
humans learn, focusing on using data and algorithms and
algorithms [112]. In addition to these, there are ensemble

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg

 E. E. Eryılmaz, S. Akleylek, Y. Ertek, E. Kılıç / MANAS Journal of Engineering 12(1) (2024) 1-28 8

learning (EL) and transfer learning methods. At the same time, 3.1. Research Questions and Purposes
algorithms such as ABC, PSO, and SSO as machine learning
This systematic literature survey examines ML, and DL-based
methods based on biological intelligence are also used in IIoT
IDS approaches developed to prevent or detect attacks on IIoT
attack detection and prevention.
devices and systems. To achieve this goal, the focus has been
on which ML and DL models are used to distinguish between
3. Research Method
benign network traffic and malignant network traffic. In
This section refers the method applied when selecting papers addition, the performance criteria used to evaluate the models
specific to ML and DL-based IIoT security and the numerical and the preferred datasets for training and testing the models
results found. At the same time, the research questions and are reviewed. For this systematic review to reach its goal, the
objectives, query sentences and areas, criteria for selecting research questions (RQs) and the purposes of these questions
and screening from the remaining manuscripts, and the are shown in Table 4.
general flow of the research method are given in the tables.

Table 4. Research Questions and Purposes

Research Research Questions Purposes

Question
Number
RQ1 In IIoT security, what performance metrics or measures are evaluated in Evaluating the proposed machine learning and deep learning
ML and DL models? models in IIoT security with their performance metrics and
defining the most used performance metrics.

RQ2 In terms of IIoT security, What are the malign and benign data types To reveal which datasets are preferred for training and testing of
found in the datasets used in the ML and DL models, and what are the ML and DL models used in IIoT security and to learn the
features of the datasets? properties of these datasets.

RQ3 Which ML and DL approaches are used in IIoT security, and what are To identify the tasks of the ML and DL models used in the
the application fields of the models? proposed schemes to protect IIoT devices and systems from
attacks and to measure the models' performances.

3.2. Research Strategy

In order to find articles that can be examined in this systematic filtering, querying, and advanced search in search engines.
literature survey, research is conducted in seven basic These databases do not allow searching by query clauses, they
academic databases (Web of Science: WoS, Scopus, IEEE only offer advanced search. Therefore, SpringerLink and
Xplore, ScienceDirect: Elsevier, Hindawi, Wiley Online Google Scholar databases were not used in this manuscript as
Library, MDPI) accepted by the scientific community. they were not systematically searched.
These academic databases are preferred because they have The research questions shown in Table 4 were transformed
search engines that can be searched in detail to obtain the into the necessary queries to conduct research in the seven
manuscripts to be examined. However, it has been observed databases described above, with Table 5. Table 5 indicates the
that the other academic databases, SpringerLink and Google query sentences used to search seven databases and in which
Scholar websites, have limited ability to perform detailed areas they were made.

Table 5. Query Sentences and Fields

Database Query Sentence Query Area

Web of ALL=(("industrial internet of things security" or "iiot security" or "industrial iot security") All metadata
Science (WoS) and ("machine learning" or "deep learning"))

Scopus TITLE-ABS-KEY ( ( "industrial internet of things security" OR "iiot security" OR Title, abstract and keywords
"industrial iot security" ) AND ( "machine learning" OR "deep learning" ) )

IEEE Xplore ("All Metadata": industrial internet of things security or industrial iot security) AND ("All All metadata
Metadata": deep learning or machine learning)

ScienceDirect ("industrial internet of things security” OR "IIot Security" OR "industrial iot security”) Title, abstract and keywords
(Elsevier) AND ("Deep Learning" OR " Machine Learning ")

Hindawi ("industrial internet of things" OR "IIot Security" OR "industrial iot security") AND ("Deep All metadata
Learning" OR "Machine Learning")

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg

 E. E. Eryılmaz, S. Akleylek, Y. Ertek, E. Kılıç / MANAS Journal of Engineering 12(1) (2024) 1-28 9

Wiley Online ("industrial internet of things security" OR "IIot Security" OR "industrial iot security") All metadata
Library AND ("Deep Learning" OR " Machine Learning")

MDPI Keywords = ("industrial internet of things security" OR "IIot Security") AND ("Deep Title and keywords
Learning" OR "Machine Learning")

3.3. Search Process and Filtering Criteria anomaly detection (EC2) in IIoT network security do not
disclose the datasets used (EC3) and do not cover ML and DL
The criteria determined for selection and elimination among
models for IIoT security (EC4) are discarded. 252 papers were
the manuscripts obtained as a result of the query sentences in
obtained from seven different databases with the help of query
Table 5 are given in Table 6.
clauses in Table 5 and selection criteria in Table 6. The
This systematic review included manuscripts published in
remaining papers were analyzed using the elimination criteria
2019-2023 (SC2). The reason for choosing this date range is
in Table 6, resulting in the examination of 25 different papers
that the manuscripts published before 2019 have been
for this survey. During the analysis, it was preferred that the
performed today. Then, among the journal manuscripts
article was new and had been cited more. At the same time,
written in English (SC1) in this date range, articles published
the content of the remaining articles after the elimination
in Q1 or Q2 level journals (SC3) and manuscripts using ML
criteria was read and the remaining articles were selected
and DL models in IIoT security (SC4) are listed. However,
accordingly. When all selection and elimination processes are
publications in the conference, editorial notes, books, and
carried out, 25 articles containing the answers to the research
preprint stages were eliminated. Replicated manuscripts
questions in Table 4 along with their analysis processes are
(EC1), which are literature searches or reviews and are also
examined in detail within the scope of this systematic survey
found in other academic databases, are eliminated. In the
research. Figure 1 shows the general flow of the research
continuation of the review, manuscripts that do not deal with
method developed to select the articles to be reviewed.

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg

 E. E. Eryılmaz, S. Akleylek, Y. Ertek, E. Kılıç / MANAS Journal of Engineering 12(1) (2024) 1-28 10

Figure 1. General Flow of the Research Method

Table 6. Selection and Elimination Criteria

Selection Selection Criteria (SC) Elimination Elimination Criteria (EC)

Criteria Criteria
Number Number
SC1 Articles published in English in a journal. EC1 Whether the article is a survey or a literature search.

SC2 Articles published in 2019-2023. EC2 The focus of work on IIoT network anomaly detection.

SC3 Have an article-type manuscript published in a Q1 or Q2 EC3 Articles lacks reference to the datasets employed in the
level journal. research.

SC4 Articles using ML and DL models in IIoT security. EC4 Articles do not include ML and DL models.

4. Descriptions of Manuscripts in the Literature

In this section, manuscripts on deep learning and machine verify the transactions of malicious nodes. The model has
learning used to ensure IIoT security in the literature are been tested with the Bot-IoT dataset [90].
reviewed. The AE algorithm is used for false data injection (FDI)
Researchers used a modified PoW algorithm PoR, which is attack detection, and the DAE algorithm is used for noise
computationally more challenging, to identify malicious IIoT removal of corrupted data. It also performed significantly
devices based on blockchain-powered deep learning and

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg

 E. E. Eryılmaz, S. Akleylek, Y. Ertek, E. Kılıç / MANAS Journal of Engineering 12(1) (2024) 1-28 11

better than the SVM model. A distributed dataset of sensor accuracy, 0,05% false positive rate (FPR), 0,22 ms total
readings was used for hydraulic system monitoring [91]. training time for 3738 samples, and 0,1 ms total test time for
A new random hybrid deep network (HDRaNN) is tested 1602 samples were measured. The proposed model is
on DS2OS and UNSW-NB15 datasets. HDRaNN has compared with the RSKNN model [97].
classified 16 types of cyber-attacks used for DS2OS and AMCNN-LSTM with gradient compression based on Top-
UNSW-NB15 with 98% and 99% higher accuracy, k selection is used to detect anomalies accurately, while the
respectively. The model achieves its best performance for the model is used to train the FL scheme in anomaly detection.
optimum learning rate and a certain number of epochs. The AMCNN with LSTM model accuracy is 96.85% for the power
results were evaluated with 10-fold cross-validation for the demand dataset [98].
datasets. The HDRaNN model is run for 150 epochs. The Feature selection is made by training the original dataset in
simulation is run at five learning rates; 0,005 – 0,01 – 0,75 – the first stage. Then the previously trained data is tested. It is
1,00 and 1,50 [92]. then combined with the original sample set with a subset of
The KDL CUP99 used in GRU and SVDD log anomaly other instances of the same classifier. Finally, Kernel-Based
detection model is preprocessed by PCA to remove Learning (KBL) has been proposed, which clusters the
unnecessary features and increase productivity in the high- controversial samples according to their distance from the
dimensional original dataset. Then, the advanced GRU-based center. The proposed method on 3000 malign and 5000 benign
algorithm with the SVDD algorithm for modeling the network datasets yielded 86.08% accuracy and 0.8655 (KBL) G-mean,
log shows that it is better than traditional methods in detecting 80.69 accuracies, and 0.7843 (random) G-mean [99].
the anomaly according to the analysis of many experimental The features were normalized with the min-max technique
results on the dataset [93]. in a single preprocessing step. PCA was used to reduce the
Different security attacks like spying, wrong setup, DoS, size and extract the best features. Training, testing times,
malicious control, malicious operation, probing, and scanning confusion matrix of the models, and computational
are remarked. ML algorithms are applied to the DS2OS complexity are given. The OCSVM model has been added to
dataset against attacks. To predict attacks, a RaNN-based the proposed framework to detect unprecedented attacks. The
random neural network model is suggested. Various OCSVM algorithm showed a detection accuracy of 86,14% in
evaluation criteria such as F1 measurement, accuracy, recall, attacks that were not seen before in the natural gas pipeline
and precision were used for the RaNN model. RaNN approach dataset and 94,53% in attacks that were not seen before in the
achieved 99,2% accuracy, 99,20% F1 score, 99,13% recall, SWaT dataset. The total training time for the SWaT dataset is
99,11% accuracy in 34,51 seconds. The detection accuracy is 1200 seconds, and the model testing time is 0,03 ms for each
5,65% better than other algorithms compared [94]. sample, with a total of 2,98 seconds. The total training time
A deep random neural (DRaNN) based model for IDS in for the Gas Pipeline dataset is 1115 seconds, and a model test
IIoT was estimated on the UNSW-NB15 dataset. The DRaNN time of 0,02 ms for each sample, with a total of 1,1 seconds
model has successfully classified nine different attack types [100].
with low FPR and high accuracy of 99,54%. The results are 7 ML methods and 1 DL model were evaluated with the
compared with other DL-based IDS models. In addition, the dataset TON_IoT containing telemetry data, operating system
proposed model achieved a high intrusion detection rate with logs, and network traffic. The ML and DL frameworks used
99,41% DR [95]. are LR, RF, LDA, CART, KNN, NB, SVM, and LSTM
IIoT attack models are updated and validated with the algorithms, and all models have been cross-validated by four
collaborative data generator DNN. The approach using times. The TON_IoT dataset consists of 7 different datasets:
SCADA data is compared with DNN and SVM (sigmoid) refrigerator sensor, GPS tracking, remote garage door,
models. In terms of performance in the proposed noisy thermostat, smart light detection, weather, and Modbus
environment, it gave better results than other models datasets. These datasets feature nine types of cyber-attacks
available. Classification performances are also reported for (Ransomware, scanning, backdoor, DoS, XSS, DDoS,
the dataset with different levels of noise added, ranging from password cracking attack, data injection and MitM). After the
1% to 50% noise. It was classified with 95.42% accuracy preprocessing and normalization steps, the datasets are trained
without noise and 92.91% accuracy with semi-noise. It is with AI based model. LSTM model for refrigerator sensor
classified as 17.85% Log Loss without noise (binary cross 100% accuracy, accuracy, all models for garage door 100%
entropy) and 21.59% Log Loss with semi-noise [96]. accuracy, kNN algorithm for GPS tracking 88% accuracy,
An RS learning method and an RT combination were used CART algorithm for Modbus 98% accuracy, LSTM for smart
to detect SCADA attacks using network traffic from the motion detection 59% accuracy, kNN for thermostat and
SCADA IIoT platform. All 15 different datasets in SCADA except for the CART algorithms, all other models achieved
consist of thousands of different attacks. Datasets are 66% accuracy. For the weather dataset, the CART algorithm
randomly sampled at a rate of 1% to reduce the impact of a reached 87% accuracy. A new experiment result was made by
small sample size. With Binary Classification, 96,71% combining the entire dataset, and the CART algorithm for

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg

 E. E. Eryılmaz, S. Akleylek, Y. Ertek, E. Kılıç / MANAS Journal of Engineering 12(1) (2024) 1-28 12

binary classification gave 88% accuracy, and again for the Job Safety Analysis (JSA) was conducted to identify factors
multi-classification model, the CART algorithm gave 77% that cause worker accidents and injuries. With smart PPE,
accuracy [101]. notifications from electronic devices are transmitted to
The paper proposes a new anomaly detection approach operators, and ThingsBoard, an open-source IoT platform,
based on centralized data collection and forwarding design provides communication between active sensors for data
that can successfully cooperate in using adaptable processing and IoT management. Device connectivity is
CEEMDAN feature with a single, smart optimization for IIoT provided via industry IoT protocols (HTTP, MQTT, CoAP),
small data. The swarm intelligence algorithm is used with the supporting cloud and on-premises deployments. CNN has
IABC OCSVM classifier to detect different anomalies. The been realized with the ThingsBoard platform. The cross-
recommended IABC-OCSVM model has high performance. validation CNN has an accuracy of 92,05% [106].
The dataset was collected from sensors in an oil field in China. The dataset for IoT and IIoT applications called the open-
These sensors contain engine speed, electrical parameters, and source Edge-IIoTset was proposed, and tests have been
flow and pressure information. WIA-PA transmits data to carried out on the dataset with ML and DL-based models
Remote Terminal Unit: RTU and RTU transmit data to a [107].
higher monitoring center via ModBus and TCP. There are EDIMA, an IoT botnet detection solution, is proposed. A
109672 IIoT data, 225 data strings, and 100 abnormal data new two-stage Machine Learning (ML) based detector
strings. OCSVM is optimized using traditional ABC and PSO developed for IoT bot detection uses supervised ML
algorithms under five different attack powers. The training algorithms and an Autocorrelation function for bulk traffic
accuracy of the ABC-OCSVM model is 95,1%, and the test classification. As a result, EDIMA has a high detection rate,
classification accuracy is 89%. The IABC-OCSVM model low bot detection delays, and low RAM consumption in
reaches average training accuracy of 94,5% and test accuracy detecting IoT bots [108].
of 89,8% [102]. LSTM, CNN, and RNN deep learning methods based on a
IIoT cloud computing risks privacy disclosure by feature selection method based on LightGBM, and DDQN and
outsourcing users. There is the SHOCFS technique to solve DQN Deep Reinforcement Learning models were used [109].
this problem. With the SHOCFS method, the most suitable IIoT threat detection was performed with the Cu-
density peaks are determined, and the model's speed is tried to LSTMGRU + Cu-BLSTM hybrid model, and high accuracy
increase. Swallow swarm optimization (SSO) enables the was achieved with a low false positive rate. The proposed
selection of optimal density peaks of clustering models. A model was compared with the Cu-DNNLSTM and Cu-
clustering algorithm is proposed to find optimal density points DNNGRU models [110].
with the hybrid cloud SHODS3O-CFS model. In the Ensemble models RF-PCCIF and RF-IFPCC methods were
SHODS3O-CFS model, the overlapping peaks of the cluster used to improve IDS performances on Bot-IoT and NF-
can be reduced. Clustering center quality (CCQ), Rand index UNSW-NB15-v2 dataset [120].
(RI), speedup-ratio (SR), and encryption time performance 23 features were selected with a feature selection based on
metrics were used. It achieved a higher mean RI of 93.4%, correlation; SVM and Decision Tree classification models and
compared to 29.68% and 17% of the proposed manuscript. NSL-KDD dataset are used to analyze network intrusion and
The dataset is taken from the 5567 home energy consumption attack detection performance [121].
data warehouse participating in the UK Power Network Synchronous optimization of parameters and architectures
meeting for the low carbon London project, and the dataset is by genetic algorithms with convolutional neural networks
available on the Kaggle website [103]. blocks (SOPA-GA-CNN) on five intrusion detection datasets
IoT-Flock developed as an open source, a benign and in IIoT, including secure water treatment (SWaT), water
malignant health dataset is created for IoT devices. Six distribution (WADI), Gas Pipeline, BoT-IoT and Power
machine learning models were used to detect cyber-attacks System Attack Dataset for the intrusion detection has been
and protect the health system from attacks. The RF algorithm implemented [122].
showed the best performance with 99,7% accuracy, 99,79% The residual neural network (P-ResNet) model was
sensitivity, 99,51% accuracy, and 99,65% F1 score [104]. implemented by combining seven IoT sensors (e.g.,
Feature selection with Fisher score and genetic-based fridge_sensor, GPS_tracker_sensor, motion_light_sensor,
extreme gradient boosting model was used to detect IoT garage_door_sensor, modbus_sensor, thermostat_sensor, and
attacks. GXGBoost achieved 99.96% accuracy on the N- weather_sensors) TON_IoT datasets [123].
BaIoT dataset with 10-fold cross-validation. The dataset The main idea and focus of the examined approaches and
malicious Mirai and the Bashlite class are instantiated in the the advantages and disadvantages of the models proposed in
Benign class dimension [105]. these approaches are given in Table 7.

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg

 E. E. Eryılmaz, S. Akleylek, Y. Ertek, E. Kılıç / MANAS Journal of Engineering 12(1) (2024) 1-28 13

Table 7. An Overview of Suggested Approaches in Manuscripts

Paper Main Idea Advantages Disadvantages

[90] A new modified PoW algorithm PoR, which is With the improved PoW algorithm, PoR, the Not applied in a real environment.
computationally more difficult, to identify malicious operations of malignant nodes are made Untested in different deep learning
IIoT devices based on blockchain-powered deep learning difficult. and machine learning models.
[91] A new method of false data injection (FDI) attack Proposed framework can detect other types The denoising autoencoder needs to
detection using automatic encoders (AE) is introduced. of attacks without any updates be trained for all attack types. When
Also, corrupted data is cleaned using denoising AEs It is the first manuscript to recommend using the Autoencoder is supervised
(DAEs). DAE is very efficient in recovering clean data DAEs to clean up corrupted (hacked) data. learning, it does not need to be
constantly trained.
[92] A new hybrid and random deep learning model Various performance metrics Not applied in a real environment.
(HDRaNN) DS2OS and UNSW-NB15 have been tested HDRaNN is compared to key detection Untested in machine learning
with two different datasets. patterns. models
[93] Gated Recurrent Unit and Support Vector Domain Compared with many deep learning models. Not applied in a real environment.
Definition log anomaly detection model has been No known performance measures
proposed. The model has been tested on the were used.
KDLLCUP99 dataset.
[94] Attacks are classified with ML models. The best model Attacks are detected, and classified with high Not applied in a real environment.
is RaNN based random NN model. success rates such as 99,11% accuracy, Untested in different machine
99,13% precision and 99,20% F1 score. learning models.
[95] The DRaNN-based model was estimated on the UNSW- The attacks were detected with 99,54% Not applied in a real environment.
NB15 dataset. accuracy and 99,41% detection rate, with a Untested in different ML and DL
high success rate of 0,76% false positives. models.
[96] A downsampling encoder-based collaborative data Real IIoT dataset. Accuracy and loss rates are given
generator trained using an adaptive algorithm is The data were classified by adding noisy only as performance criteria, and
proposed. data to the test dataset. other known criteria are not used.
[97] RS learning method and RT combination were used to Creating a detection engine based on If a feature exists, the best feature is
detect SCADA attacks using network traffic from the industrial protocols and a high DR of limited to random selection.
SCADA IIoT platform. 96,71% Excessive execution time
[98] A deep learning-based federated learning tool to detect It uses convolutional neural network units The federated learning model is
communication-efficient, new anomalies to detect time based on the attention mechanism, thus vulnerable for loss of malign
series data in IIoT avoiding memory loss and gradient anomaly attacks
distribution problems.
[99] ML-based KBL selection method is proposed for defense Extracted from the malware dataset as static Untested in different deep learning
against hostile attacks in an IIoT environment. features with the Androguard tool. models.
Feature selection method is used,
which is not used very much in the
literature.
[100] A new two-stage community deep learning model and Resistant to unstable datasets where the Complex architecture
attack correlation scheme is proposed for unstable numbers of malignant and benign datasets
industrial control system data using the OCSVM model are not close to each other.
to detect unprecedented attacks. Capable of detecting never-before-seen
attacks
[101] A new dataset (TON_IoT) is proposed, which includes Variety of benign and malign events for Advanced parameter optimization is
Telemetry data of IoT and IIoT services, traffic of IoT different IoT or IIoT devices. required to optimize hyper
network, and operating systems logs. It is designed based Contains heterogeneous data sources. parameters and obtain better results.
on integrating IoT/IIoT systems with three layers of Fog,
Edge, and Cloud.
[102] CEEMDAN feature and swarm intelligence algorithm The real-world dataset in China oil IIoT The dataset is not public and not
ABC-based IABC-OCSVM model. system. detailed.
Attacks under five different attack power No evaluation was made with
have been detected. different performance criteria.
No comparison with deep learning
models.
[103] With IIoT cloud computing, the SHODS3O-CFS A safe optimized clustering method is No evaluation has been made with
algorithm, which is a new SHOCFS technique, is proposed to obtain optimal density peaks. other ML and DL algorithms.
recommended for users outsourcing privacy disclosure Not tested in a real environment.
risk.
[104] Developing a benign and malignant IoT use case with An open source software has been created Deep learning models are not used.
IoT-Flock, which creates open source IoT data, and for IoT healthcare environments that capture
traffic generation and evaluation of IoT health dataset data in the context-aware MQTT and COAP
with ML techniques. categories.

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg

 E. E. Eryılmaz, S. Akleylek, Y. Ertek, E. Kılıç / MANAS Journal of Engineering 12(1) (2024) 1-28 14

[105] IoT botnet network attacks are detected by feature A high detection rate (average accuracy Hard to verify that parameters reach
selection with Fisher score and GXGBoost algorithm and results in 99,96%) the global optimum,
identify the most relevant features. Sensitivity and randomness of the
genetic algorithm used for the initial
population
[106] A smart helmet 5.0 CNN model that monitors The model was evaluated with ML and DL Except for the accuracy
environmental conditions and performs real-time risk algorithms. performance metric, no other
assessment performance metric is used.
[107] The dataset for IoT and IIoT applications called the Data was collected from more than 10 IoT Realistic but not real environment
open-source Edge-IIoTset has been proposed, and tests devices, and 61 new features were extracted
have been carried out on the dataset with ML and DL- from 1176 features. Performance was
based models. evaluated with ML and DL algorithms.
[108] EDIMA, an IoT botnet detection solution, is proposed. A A high detection rate, Low bot detection Difficulty of retraining the model,
new two-stage Machine Learning (ML) based detector delays, and low RAM consumption in Deep learning models are not used.
developed for IoT bot detection uses supervised ML detecting IoT bots. Not tested in a real environment.
algorithms and an Autocorrelation function for bulk
traffic classification.
[109] For IIoT, LSTM, CNN, and RNN, deep learning methods Both deep learning methods and deep Machine learning models are not
based on a feature selection method based on LightGBM, reinforcement learning models were used. used.
DDQN, and DQN Deep Reinforcement Learning models Not tested in a real environment.
were used.
[110] For the IIoT environment, a hybrid DL, SDN-enabled The model is programmable and expandable Not tested in a real environment,
approach is proposed to detect threats and intrusions. on iiot data devices. Machine learning models are not
Open flow switches are used in SDN used.
[120] Ensemble models RF-PCCIF and RF-IFPCC methods Pearson Correlation Coefficient (PCC) Not used deep learning models,
Isolation Forest (IF) to reduce computational Not tested in a real environment.
cost and prediction time
[121] Correlation based features selection SVM and DT Correlation features selection Not tested in a real environment,
methods Deep learning models are not used.

[122] Synchronous optimization of parameters and On five intrusion detection datasets in iiot, Not tested in a real environment,
architectures by genetic algorithms with convolutional including secure water treatment (swat), Machine learning models are not
neural networks blocks (SOPA-GA-CNN) water distribution (WADI), Gas Pipeline, bot- used.
iot and Power System Attack Dataset for the
intrusion detection
[123] Residual neural network (P-ResNet) model with seven Combining seven iot sensors Not tested in a real environment.
IoT sensors dataset

These criteria are given in Table 8. Table 8 summarizes the

5. Results definitions of performance criteria, their mathematical
equations, if any, and in which manuscripts they are used.
In the results section, the determining research questions are According to Table 8, it is seen that the criteria of F-1 score,
answered. precision, accuracy, recall, FPR, DR and FAR and are widely
preferred for the evaluation of the models. Since these criteria
5.1. RQ1: In IIoT security, what performance metrics or were used, TP, FP, TN and FN values were measured in each
measures are evaluated in ML and DL models? manuscript. The total time taken for training (TRT) and testing
(TET) models is also frequently used in manuscripts. Other
The performance of the ML and DL models used in the criteria are used in the evaluation of the models in accordance
proposed schemes was evaluated by means of various criteria. with the purpose of the proposed models.

Table 8. Performance Metrics Used in the Evaluation of Machine Learning and Deep Learning Algorithms

Performance Performance Description Mathematical Equation Articles Used

Metrics
TP A correctly predicted situation is correct - All
TN An incorrectly predicted situation is correct - All
FP False of a positively predicted situation - All
FN False of a negatively predicted situation - All
ACC Percentage of correctly classified sample data out of 𝑇𝑃 + 𝑇𝑁 [90], [91], [92], [94],
all classified sample data. 𝑥100 [95], [96], [97], [98],
𝑇𝑃 + 𝑇𝑁 + 𝐹𝑃 + 𝐹𝑁
[99], [100], [101],
[102], [104], [105],
[106], [107], [108],
[109], [110], [120],

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg

 E. E. Eryılmaz, S. Akleylek, Y. Ertek, E. Kılıç / MANAS Journal of Engineering 12(1) (2024) 1-28 15

[121], [122] , [123]

PRE Percentage of how accurately we guessed from all 𝑇𝑃 [90], [92], [94], [99],
classes 𝑥100 [100], [101], [104],
𝑇𝑃 + 𝐹𝑃
[105]], [107], [108],
[109], [110], [120] ,
[122], [123]
REC Percentage of how accurately we guessed from all 𝑇𝑃 [90], [92], [94], [99],
positive classes 𝑥100 [100], [101], [104],
𝑇𝑃 + 𝐹𝑁
[105]], [107], [108],
[109], [110], [120],
[123]
SPC (TNR) Ratio of true negative samples. 𝑇𝑁 [99], [110]
𝑥100
𝑇𝑁 + 𝐹𝑃

F1 Harmonic mean of precision and recall measures. 2𝑥𝑃𝑅𝐸𝑥𝑅𝐸𝐶 [92], [94], [99],
Low recall or high precision (or vice versa) 𝑥100 [100], [101], [104],
𝑃𝑅𝐸 + 𝑅𝐸𝐶
[105]], [107], [108],
[109], [120] , [122],
[123]
SNS (TPR) Rate of positive samples correctly classified as 𝑇𝑃 [99], [110]
positive. 𝑥100
𝑇𝑃 + 𝐹𝑁
FPR/FAR Rate of negative samples falsely classified as 𝐹𝑃 [91], [93], [95], [97],
positive. 𝑥100 [110]
𝐹𝑃 + 𝑇𝑁
G-Mean Geometric mean of Specificity and Sensitivity √𝑆𝑃𝐶𝑥𝑆𝑁𝑆 [99]
MCC Matthews correlation coefficient 𝑇𝑃. 𝑇𝑁 − 𝐹𝑃. 𝐹𝑁 [110]
√(𝑇𝑃 + 𝐹𝑃)(𝑇𝑃 + 𝐹𝑁)(𝑇𝑁 + 𝐹𝑃)(𝑇𝑁 + 𝐹𝑁)
FNR Rate of positive samples falsely classified as 𝐹𝑁 [110]
negative. 𝑥100
𝑇𝑃 + 𝐹𝑁
FDR False Discovery Rate 𝐹𝑃 [110]
𝑥100
𝑇𝑃 + 𝐹𝑃
ROC The curve obtained by plotting FPR versus TPR, as - [91], [92], [104],
the threshold data values vary over a range. [110], [120], [123]
MSE Mean square error - [91], [92], [104]
AUC ROC area under the curve. - [91], [110], [120],
[123]
RMSE Root mean square error 1 [98]
𝑚 2
1 2
[ ∑(|𝑦𝑖 − 𝑦^𝑝 |) ]
𝑛
𝑖=1
RI The measure of the exact clustering results versus 𝑇𝑃 + 𝑇𝑁 [102]
the actual clustering results of the clustering 𝑥100
𝑇𝑃 + 𝑇𝑁 + 𝐹𝑃 + 𝐹𝑁
algorithm.
DR Rate of correctly detected positive samples among 𝑇𝑃 [91], [93], [95],
total positive samples. 𝑥100 [108]
𝑇𝑃 + 𝐹𝑁
ER The rate of how often the model misclassifies. 𝐹𝑃 + 𝐹𝑁 [103]
𝑥100
𝑇𝑃 + 𝑇𝑁 + 𝐹𝑃 + 𝐹𝑁
TRT Total time spent training the model. - [93], [98], [99],
[102], [103], [120],
[121], [123]
TET Total time spent testing the model. - [96], [99], [102],
[103], [107], [110],
[120], [122], [123]
ATR Average time spent training the model. - [105]
ATE Average time spent testing the model. - [105]
ET The encryption time of the model. - [105]
Log Loss: LL The log loss is found by subtracting the performance 𝑀 [94]
results of the model from the expected results. − ∑ 𝑦𝑜, 𝑐 𝑙𝑜𝑔(𝑝𝑜, 𝑐)
Lower log loss is better performance. 𝑐=1
CCQ Distance between clustering centers produced 𝐶
[103]
𝑖 𝑖 2
√∑‖𝑣𝑖𝑑𝑒𝑎𝑙 −𝑣 ‖
𝑖=1

SR Speedup ratio - [103]

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg

 E. E. Eryılmaz, S. Akleylek, Y. Ertek, E. Kılıç / MANAS Journal of Engineering 12(1) (2024) 1-28 16

5.2. RQ2: In Terms of IIoT Security, what are the Malign and time, total bytes per source IP, incoming connections per
Benign Data Types Found in the Datasets Used in the ML source IP [90]. The DS2OS dataset has eight classes and 13
and DL Models, and the Features of What are the Datasets? features [92], [94], and the UNSW-NB15 dataset] has ten
classes and 49 features [92], [95. The KDD CUP99 (NSL-
There are various types and numbers of datasets used for the KDD) dataset includes DOS, R2L, U2R, and Probe attack
manuscripts reviewed. With ML and DL algorithms, models types with 42 features [93]. Datasets containing 15 different
are trained and tested on data sets. The datasets used in the datasets in the SCADA network were sampled at a rate of 1%.
models are selected by the purpose of the proposed schemes Detailed information about the features was not given [96],
to ensure IIoT security. If the developed approaches are used [97]. There are time series datasets consisting of four real-
to detect which attack types, datasets containing examples of world data (Engine, Power Demand, ECG, Space Shuttle)
those attack types are recommended for train and testing the combined from various sensors. These datasets have normal
models. Table 9 shows the datasets used to train and test the subsequences and abnormal subsequences. No detailed
models or the datasets created by the authors for use in papers. information was given about the features [98]. In the
The datasets encompass various types of malignant and manuscript found [99], the Android Malware dataset
benign samples, and pertinent information about the statistical recommended and the number and types of features used were
properties of these samples, as well as the manuscripts in not given [111]. There are 17 features in the Pipeline dataset,
which they were utilized. The number of features, classes, and 51 features, and 31 scenarios in the Swat dataset [110]. There
dimensions of the dataset is also given. However, detailed are 52 features within the attack types (ransomware, scanning,
information about the datasets used in the manuscripts are not backdoor, DoS, XSS, DDoS, password cracking attack, data
given in the articles in which they are used [91], [103], [104]. injection, and MitM) [101]. The articles do not have dataset
For this reason, the details of these datasets are not available details and feature information [102]-[104]. The N-BaIoT
in Table 9. Data types and attack types are not given for dataset has 115 features derived from malignant and benign
malign and benign [96]-[98], [102], [106]. data [105], [110]. The number of features is not specified in
The Bot-IoT [113] dataset contains 14 features. These are the the dataset created for Smart Kask 5.0 [106]. Edge-IIoTset
numeric expression of feature status, the minimum duration of dataset is generated from various IoT devices and proposes 61
total records, the standard deviation of total records, number new features [107]. IoT-NSS-BPR uses IoT-23 dataset, and
of inbound connections per destination IP, the average UNSW IoT dataset. Dataset types are malware samples,
duration of total records, highest period of total records, total malware traffic pcap files, and aggregate IoT traffic pcap files
bytes per destination IP, the sequence number of the Argus [108]. Real dataset of the natural gas pipeline transportation
agent, per unit time packets from source to destination, network publicly released by the U.S. Department of Energy’s
packets from destination to source, packets from source to Oak Ridge National Laboratory [109].
destination, packets from destination to source per unit of

Table 9. Datasets Used in Models and Properties

Dataset Type and Number of Malign Data Type and Number Total Data Numbers Number of Articles
of Benign Data Features/ Classes/ Using
Dimensions
Bot-IoT: is a dataset - UDP DoS and DDoS: - UDP: 7225 - Malign: 73360900 14 features [91],
containing detailed network 39624597 - ICMP: 9 - Benign: 9543 [120],
information of benign and - Service scanning: - TCP: 1750 - Total: 73370443 [122]
malignant data traffic and 1463364 - RARP: 1
various network attacks. - HTTP DoS and DDoS: - ARP: 468
49477 - IGMP: 2
- TCP DoS and DDoS: - IPV6-ICMP: 88
31863600
- OS fingerprint:
358275
- Keylogging: 1469
- Data theft: 118
DS2OS: It includes 13 -Spying: 532 Normal: 347935 -Malign total: 10017 13 features and 8 [92],
features and 7 malign and 1 -DoS: 5780 -Benign total: 347935 classes [94]
benign data -Malicious Control: 889 -Total: 357952
-Wrong setup: 122
-Scan: 1547
-Malicious Operation: 805
-Data type probing: 342

UNSW-NB15: 9 malign, 1 -Fuzzers: 24246 Normal: 93000 -Malign total:164673 49 features and 10 [92],

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg

 E. E. Eryılmaz, S. Akleylek, Y. Ertek, E. Kılıç / MANAS Journal of Engineering 12(1) (2024) 1-28 17

benign data produced by the -Backdoor: 2329 -Benign total: 93000 classes [95],
Australian Cyber Security -Analysis: 2677 -Total: 257673 [120]
Center's Cyber Range -Reconnaissance: 13987
Laboratory -Exploits: 44525
-Generic: 58871
-DoS: 16353
-Shellcode: 1511
-Worms: 174
KDD CUP 99 -DOS:2000 Normal: 2000 -Malign total: 4000 42 features [93],
NSL-KDD -R2L:1000 -Benign total: 2000 [121]
-U2R:500 -Total: 6000
-PROBE:1500
SCADA 28 attack scenarios 9 normal event 28 total scenarios - [96],
scenarios [97]
Power Demand Abnormal substring: 6 Normal substring: Normal substring: 45 1 Dimension [98]
45 Abnormal substring: 6
Total substring: 51
Original sequence:1
Space Shuttle Abnormal substring: 8 Normal substring: Normal substring: 20 1 Dimension [98]
20 Abnormal substring: 8
Total substring: 28
Original sequence:3
ECG Abnormal substring: 1 Normal substring: Normal substring: 215 1 Dimension [98]
215 Abnormal substring: 1
Total substring: 216
Original sequence:1
Engine Abnormal substring: 152 Normal substring: Normal substring: 240 12 Dimension [98]
240 Abnormal substring: 152
Total substring: 392
Original sequence:30
Android Malware dataset 3000 malwares 5000 benign Total 8000 - [99]
suggested by the authors
[111]
Pipeline 60048 (21,86%) attack examples 214580 (78,14%) 274628 total samples 17 features [100] ,
- Malicious state command injection normal samples [122]
(MSCI)
-Naive malignant response injection
(NMRI)
-Reconnaissance (Recon)
-Complex malignant response
injection (CMRI)
-DoS
-Malign function code injection
(MFCI)
- Malignant parameter command
injection (MPCI)
Swat (safe water treatment) 12,1% attacks 87,9% normal Total: 449920 samples 51 features 31 [100] ,
scenario [122]
TON_IoT Total: 162932 Benign 35000 for Malicious:162932 - Refrigerator [101],
- XSS all datasets Benign: 245000 sensor:7 [123]
- scanning Total benign: Total: 407932 - GPS tracking:7
- data injection 245000 - Garage door:7
- DoS, - Thermostat:7
- MitM - Intelligent light
- DDoS, detection:7
- ransomware - Weather:8
- backdoor - Modbus:9
- password cracking attack Total Features: 52

Oil field dataset in China 100 abnormal data strings 200 normal data 300 data strings - [102]
[102] strings
N-BaIoT Mirai: 3668402 Benign: 555932 Malignant: 4700458 115 features [105],
Bashlite: 1032056 Benign: 555932 [110]
Total: 5256390
Dataset created by the - - 11755 samples in total 12 scenarios [106]
authors [106] for Smart
Helmet 5.0
Edge-IoTset Backdoor: 24862 Normal: Normal: 11223940 New 61 features [107]
DDoS_HTIP: 229022 11223940 Attack: 9728708 with high

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg

 E. E. Eryılmaz, S. Akleylek, Y. Ertek, E. Kılıç / MANAS Journal of Engineering 12(1) (2024) 1-28 18

DDoS_ICMP: 2914354 Total: 20952648 correlations from

DDoS_TCP: 2020120 1176 found
DDoS_UDP: 3201626 features
Fingerprinting: 1001
MITM: 1229
Password: 1053385
Port_Scanning: 22564
Ransomware: 10925
SQL_injectioion: 51203
Uploading: 37634
Vulnerability_scanner: 145869
XSS: 15915
IoT-NSS-BPR, IoT-23 IoT-NSS-BPR: 23 live IoT malware - - Best 8 features [108]
dataset, UNSW IoT dataset samples,
UNSW IoT dataset :28 different
uninfected IoT devices collected at a
gateway.
U.S. Department of Energy’s NMRI 2763 Normal: 161156 Normal: 161156 26 features and [109]
Oak Ridge National CMRI 15466 Attack: 32396 one label
Laboratory natural gas MSCI 78 Total: 193552
pipeline transportation MPCI 7637
network MFCI 573
DoS 1837
Recon 6805

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg

 E. E. Eryılmaz, S. Akleylek, Y. Ertek, E. Kılıç / MANAS Journal of Engineering 12(1) (2024) 1-28 19

generate conflicting attack data, and this generated data was

5.3. RQ3: which ML and Dl Approaches are Used in IIoT classified [96].
Security, and What are the Application Fields of the A reliable ensemble learning model with a combination of
Models? the SCADA network RS learning method and RT has been
tested on 15 different datasets. The model, whose
Table 10 summarizes the usage areas of the models, the classification accuracy and model complexity was balanced,
datasets they are trained and tested with, the performances performed well compared to other cutting-edge approaches
they show as a result of the experiments, and the information [97].
about which models they are compared with. Firstly, the FL model is developed to collaboratively train
When Table 10 is examined, a feedforward multilayer anomaly detection on decentralized edge devices. Secondly,
multiclass neural network with Microsoft Azure Machine the attention mechanism CNN-LSTM model is proposed for
Learning Studio is used with the Bot-IoT dataset with various the correct detection of anomalies. The AMCNN-LSTM
hyperparameters. An advanced intrusion detection method scheme uses CNN units based on the attention mechanism to
using a deep learning model together with blockchain is capture important detailed features, thus avoiding memory
proposed for malignant IIoT devices. High-performance loss and gradient distribution problems. Thirdly, in order to
results have been achieved with this model [90]. increase communication efficiency, anomaly detection has
It is aimed to detect false data injection attacks with an been made in the industrial area with the model that
automatic encoder algorithm that is easy to train and learns compresses the gradients based on Top-k feature selection
hidden complex relationships. When SVM and AE were [98].
compared, AE gave more successful results. A DAE was used It has been tested with feature selection methods such as
to get rid of the effects of the attack on the data [91]. random, L1, Euclidean, and KBL. High performance was
HDRaNN, proposed for cyber-attack detection in IIoT uses obtained with the KBL selection method SVM algorithm [99].
implementations of HDRaNN and MLP. The HDRaNN OCSVM was used to detect previously unseen attacks,
includes input, hidden, and output layers. Performance creating a boundary around normal samples and reporting
measurements were made on two separate datasets such as others as never-seen attacks. The proposed model is a complex
UNSW-NB15 and DS2OS. With the HDRaNN model, attacks deep neural network consisting of partially or fully connected
are classified with an accuracy of 98% and over 99% for the layers that detect IoT attacks [100].
UNSW-NB15 and DS2OS datasets. HDRaNN model has 4-fold cross-validation of LR, RF, LDA, CART, KNN, NB,
been compared with RNN, DBN, DAE, and RBM deep and SVM models were evaluated on the newly proposed
learning models [90]. TON_IoT dataset. 80% of the data and 20% of the data are
A log anomaly and malignancy detection model based on allocated to the test dataset to train/validate ML methods.
GRU and Support Vector Domain Definition algorithms Classification results are given for the TON_IoT dataset
framework is proposed. Numerous experiments and analyses refrigerator sensor, GPS tracking, garage door, thermostat,
of experimental results on the KDL CUP99 dataset have smart light detection, weather, and Modbus datasets with
shown that the advanced GRU-based algorithm is better than different models. As a result of the estimation made by
traditional deep learning models in detecting an anomaly. The combining the whole dataset, the CART algorithm for binary
highest DR was measured at 99,6%, and the smallest FAR at classification reached the most successful result with 88%
0,01%. Five types of anomalies (DoS, R2L, U2R, PROBE, accuracy, and again for the multi-classification model, the
and mixed) were detected with five algorithms (GRU-SVDD, CART algorithm achieved the most successful result with
BGRU-MLP, LSTM, LSTM-RNN, PCA-SVM) [93]. 77% accuracy. Training and testing times for binary
RaNN deep learning model evaluated accuracy, precision, classification are high for LSTM, SVM, and KNN models and
precision, and F1 score performance metrics on the DS2OS low for LR, LDA, RF, CART, and NB models. Training and
dataset. The RaNN model is compared with SVM, DT, and testing times for multiclassification are high for LSTM, SVM,
ANN models. RaNN model accuracy is compared with the LR, and KNN models and low for LDA, RF, CART, and NB
accuracy of previous intrusion detection models [94]. models [101].
Intrusion detection was performed on a model UNSW- A new IABC-OCSVM anomaly attacks classification
NB15 dataset based on a DRaNN model for intrusion scheme is proposed for the IIoT small dataset that can
detection in IIoT. Feature transformation and normalization skillfully cooperate in CEEMDAN model feature use
were performed in the preprocessing step. With the DRaNN compatible with the smart optimizer OCSVM classifier. With
model intrusion detection system, the data is classified as CEEMDAN decomposition, energy entropies are measured
normal or attack [95]. with IMF components. Multi-scale analysis of the IIoT dataset
The down sampler-based data generator for SCADA is performed. The IABC-OCSVM model created with
attacks detection is alternatively updated and validated using Gaussian mutation was found to have 94.5% training
a deepNN splitter during training. A GAN was developed to

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg

 E. E. Eryılmaz, S. Akleylek, Y. Ertek, E. Kılıç / MANAS Journal of Engineering 12(1) (2024) 1-28 20

accuracy, 89.8% test accuracy, and 0.0081 seconds test time Normal centralized DT, RF, KNN, SVM, DNN model PRE,
[102]. REC, and F1 have 100% and federated 2-class IID and Non-
SHODS3O-CFS clustering algorithm and the most IID ACC:100% performance. Edge-IIoTset, produced by ten
appropriate density selection in the hybrid cloud are different IoT devices, was evaluated together with two
suggested. The SHODS3O-CFS algorithm gave clustering different ML-based IDS with the centralized and federated
center accuracy (RI) of 87.7% for 50 data objects, while mode in 7 different layers [107].
PPHOCFS achieved lower RI results of 62.7% and SHOCFS The EDIMA model has been proposed. EDIMA consists of
76.6%. The SHODS3O-CFS algorithm achieved 95,2% RI for a traffic parser, feature extractor, ML-based bot detector,
250 data objects. The PPHOCFS and SHOCFS methods, on policy engine, ML model constructor, and a malware PCAP
the other hand, yielded lower clustering accuracy of 66% and database. RF algorithms ACC, PRE, REC, and F1 have 100%
81,2% RI, respectively [103]. performance [108].
Benign and malignant data in pcap format with IoT-Flock LightGBM feature selection method, PPO2 interface, and
software were converted into CSV format with the python CNN, RNN, LSTM, DDQN, and DQN model were used.
program. The categorical properties of the dataset, such as the Deep Reinforcement Learning model DDQN has a 97,74 F1-
protocol type (MQTT and COAP), have been replaced with score [109].
numeric values using the Label Encoder to facilitate further The hybrid model (Cu-LSTGRU + Cu-BLSTM), Cu-DNN-
processing. Missing data is filled with 0. The most important LSTM, and Cu-DNN-GRU were evaluated, and (Cu-
ten features consist of TCP and MQTT data by feature LSTMGRU + Cu-BLSTM) gave the highest performance
selection with the LR algorithm. The dataset was tested with result with an F1-score rate of 99.47%. Model GRU-RNN has
NB, KNN, RF, AB, LR, and DT algorithms. Confusion been compared with Autoencoder (EDSA) and Multi-CNN
matrix, ROC-AUC, F1 score, precision, accuracy, recall, and [110].
values of each algorithm are given. The RF model showed the RF-PCCIF and RF-IFPCC have 99.98% and 99.99% Acc
best performance with 99,70% accuracy, 99,79% recall, and prediction time of 6.18 sec and 6.25 sec, respectively, on
99,51% accuracy and 99,65% F1 score [104]. Bot-IoT. The two models also achieve 99.30% and 99.18%
Improved GXGBoost algorithm to well classify IIoT accuracy and prediction time scores of 6.71 sec and 6.87 sec
network attacks. Several trials have been conducted on the on NF-UNSW-NB15-v2, respectively [120].
public N-BaIoT dataset of IIoT devices. GXGBoost achieved Quadratic SVM has 99.7% accuracy, prediction speed is
99.96% accuracy on the N-BaIoT dataset using only three 1100 s and training time is 465.28 s. Fine Tree has 99.4%
features out of 115 features [105]. accuracy, prediction speed is 570.000 sec and training time is
An intelligent helmet prototype is presented that monitors 11.029 seconds [121].
environmental conditions and works in near real-time risk (SOPA-GA-CNN) has 98.1 F1 Score with gas pipeline
assessment. The dataset consisting of 11755 examples and 12 dataset [122].
different attack-type scenarios is evaluated by ML and DL. P-ResNet has a performance of 87% accuracy, 88%
The cross-validation CNN model for business risk analysis precision, 86% recall, 86% F1 Score, 83% ROC AUC, TRT:
yielded 92,05% accuracy. The CNN approach is evaluated by 24401.586s, TET: 3.014s [123].
comparing it with NB, SVM, and NN [106].

Table 10. An Overview of the Models

Papers Models/Methods Used and Their Tasks Datasets and Uses Performance Compared Models
or Approaches
[90] A feedforward multilayer multiclass Bot-IoT: the dataset is split - Overall ACC: 95,9% -
neural network with various 6:4 into training and test - Average ACC: 98,36%
hyperparameters is used with Microsoft data. - Micro average PRE: 95,9%
Azure Machine Learning Studio to - Micro-average REC: 95,9%
simulate the deep learning model. - Macro averaged REC: 58,18%
[91] The Auto-encoder algorithm is used to The dataset includes a total MSE training loss: 3.99e-7 -SVM RBF Kernel
reveal false data injection attacks. of 15 sensor data. MSE validation loss: 4.37e-7 -SVM Linear
Clean corrupted data (AE) performed (volumetric flow, pressure, AE ACC: 97,65% Kernel
better with the support vector machine engine, temperature, cooling, SVM ACC: 85,1% -SVM Gaussian
(SVM) algorithm in terms of ROC. vibration, and power). AE DR: 100% Kernel
Pump, coolant, valve, and accumulator SVM DR: 88,55%
values are measured. AE FAR: 6,42%
SVM FAR: 16,3%
DAE MSE: 0,0064
AE MSE: 0,1
AE TRT:1 min
SVM TRT:15 min

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg

 E. E. Eryılmaz, S. Akleylek, Y. Ertek, E. Kılıç / MANAS Journal of Engineering 12(1) (2024) 1-28 21

[92] HDRaNN model has been used for DS2OS is used for training ACC: %98,56 RNN, DBN, DAE,
cyber-attack detection in IIoT. and testing. Attack PRE: %98,25 RBM
distributions are given in REC: %98,36
detail. A confusion matrix F1: %98,3
was created. LL: %36,24
AUC-ROC: %91,28
UNSW-NB15 is used for ACC: 99,19%
training and testing. Attack PRE: 99,07%
distributions are given in REC: 98,98%
detail. A confusion matrix F1: 99,02%
was created. LL: 12,23%
AUC-ROC: 98,82%
[93] A log anomaly detection model based 10% of the KDL CUP 99 DR: 99,6% BGRU-MLP,
on GRU and Support Vector Domain dataset is trained. FAR: 0,01% LSTM, PCA-SVM
Definition algorithms framework and LSTM-RNN
[94] Detecting attacks in DS2OS dataset Intrusion detection was ACC: 99,2% SVM, DT, ANN
with a new lightweight random neural performed by dividing the PRE: 99,08%
network model DS2OS dataset 8:2 train and REC: 99,16%
test data F1: 99,04%
TET: 34,51 ms
[95] Intrusion detection was performed on UNSW-NB15 is used for ACC: 99,54% BLSTM RNN,
the UNSW-NB15 dataset with DRaNN 75% training and 25% DR: 99,41% Adaboost, CNN
based model. testing. Attack distributions FPR: 0,76% and WDLSTM,
are given in detail. DL, FFDNN, DNN,
DBN
[96] The down sampler-based data generator SCADA: 36000 samples, Noiseless ACC: 95,42% DNN, SVM
for SCADA attack detection is half of which benign traffic Semi-noisy ACC: 92,91%
alternatively updated and validated and half of malign attack GAN ACC: 95,55%
using a deepNN splitter during training. traffic GAN LL: 47,55%
Developing and classifying a GAN to TRT: 2,58h
generate conflicting attack data
[97] An improved ensemble learning model SCADA 15 datasets and Binary Classification ACC: 96,71% RSKNN
is proposed to detect SCADA thousands different attacks. FPR: 0,05%
cyberattacks based on the combination Datasets are randomly TRT: 0,22
of RS learning method and RT. sampled at a rate of 1%. TET: 0,1
[98] AMCNN-LSTM model based on the Engine, Space Shuttle, ECG, For Power Demand, AMCNN-LSTM SVM, SAE, GRU,
attention mechanism is proposed. Power Demand ACC: 96,85% CNN with LSTM
RMSE: <5% and LSTM
AMCNN-LSTM time with GCM: 25min
AMCNN-LSTM time without GCM:
90min
[99] In the malware literature, the KBL Android Malware dataset ACC: 86,08% DNN, SVM, RF,
selection method has a 6% performance G-Mean: 86,55% Bayes
improvement over random selection. AUC: 95,8%
SVM ACC: 98,5%
[100] The proposed IDS consist of two Pipeline dataset created by ACC: 96,2% DT, SVM, K-
unsupervised SAEs, feature extraction Mississippi State University PRE: 96,17% Means, NB,
using PCA and a Decision Tree REC: 96,2% AIKNN, LSTM
classification and using OCSVM to F1: 96,18%
detect previously unseen attacks TRT: 1200s
TET: 2,98s
Swat (safe water treatment) PRE: 99,99% DT, LADS-ADS,
dataset created by Singapore REC: 99,99% DNN, ID CNN,
Technological University F1: 99,98% MADGAN, Tabor,
TRT: 1115s LSTM, ST-ED
TET: 1,1s
[101] A new dataset (TON_IoT) is proposed Refrigerator sensor For LSTM; LR, RF, LDA,
for the next generation IoT and IIoT ACC, PRE, REC and F1: 100% CART, KNN, NB,
dataset for data-driven IDS. On the TRT:190,493 SVM, LSTM
TON_IoT dataset, LR, RF, LDA, TET:3,705
CART, KNN, NB, and SVM models GPS tracking For KNN;
were evaluated with 4-fold cross- ACC: 88%
validation. All algorithms classification PRE: 89%
results are given on seven different REC: 88%
datasets, TON_IoT dataset, refrigerator F1: 88%
sensor, GPS tracking, garage door, TRT: 0,08
thermostat, smart light detection, TET: 1,508
weather, and Modbus datasets. In Garage door For all algorithms
addition, for the combined_TON_IoT ACC, PRE, REC and F1: 100%
dataset, which is the combination of all NB TRT: 0,01sec

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg

 E. E. Eryılmaz, S. Akleylek, Y. Ertek, E. Kılıç / MANAS Journal of Engineering 12(1) (2024) 1-28 22

data sets, all algorithms are evaluated TET: 0,02sec

with binary and multi-classification Thermostat For NB;
models, and attack types are classified. ACC: 66%
PRE: 44%
REC: 66%
F1: 53%
TRT: 0,009
TET: 0,002
Intelligent light detection For LSTM;
ACC: 59%
PRE: 35%
REC: 59%
F1: 44%
TRT: 63,132
TET: 3,73
Weather For CART;
ACC: 87%
PRE: 88%
REC: 87%
F1: 87%
TRT: 0,258
TET: 0,03
Modbus For CART;
ACC: 98%
PRE: 99%
REC: 98%
F1: 99%
TRT: 0,367
TET: 0,01
[102] An improved ABC algorithm IABC- Pressure, engine speed, flow Training ACC: 94,5% EEMD and
OCSVM, based on an arrow-variable and some electrical Test ACC: 89,8% CEEMDAN
Gaussian mutation with CEEMDAN parameters TET: 0,0081s ABC-OCSVM,
decomposition compatible with the PSO-OCSVM
intelligent optimizer OCSVM classifier
[103] SHOCFS was used for speed Incorporating daily weather RI: 95,2% PPHOCFS and
improvement and detection of optimum changes into energy use, ER: 778,80ms SHOCFS
density peaks, SSO was used to select clustering center quality SR: 36,86
optimum density points of the collected from data in CCQ: 0,401
clustering model, and the SHODS3O- England, Wales and Scotland
CFS method was suggested in a hybrid is evaluated on clustering
cloud. The SHODS3O-CFS model center quality, encryption
reduces overlapping peaks in the time, accuracy, speed-up rate
cluster and increases security in the performance measures.
hybrid cloud.
[104] Data in MQTT and COAX categories From environmental ACC: %99,51% NB, KNN, RF, AB,
from environment monitoring sensors monitoring sensors (air- PRE: 99,7% LR
and patient monitoring sensors were humidity, air-temperature, REC: 99,79%
created with IoT-Flock software. The co, fire, smoke, barometer, F1: 99,65%
created dataset was evaluated with NB, solar radiation sensors) and AUC: 100%
KNN, RF, AB, LR, and DT algorithms. patient monitoring sensors
The model that gave the best results (remote electrocardiogram
was the RF algorithm. (ECG) monitoring, galvanic
skin response (GSR) sensor),
infusion pump pulse
oximetry (SPO2),
nose/mouth air flow sensor,
blood pressure monitor
sensor, glucose meter,
electromyography (EMG)
sensor, body temperature
sensor
[105] GXGBoost performed several The N-BaIoT dataset consists ACC: 99,96% DNN, DT, KNN,
experiments on the public N-BaIoT of the malignant Mirai, PRE: 99,95% DAE, SVM, VIF
dataset for efficient classification Bashlite and Benign datasets. REC: 99,95%
F1: 99,95%
ATR: 545,040 sec
ATE: 4,208 sec
[106] CNN model ThingsBoard tool. It consists of a dataset of ACC: 92,05% NN, NB, SVM
ThingsBoard. CNN algorithm works 11,755 examples and 12
independently with an alarm system in different scenarios.

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg

 E. E. Eryılmaz, S. Akleylek, Y. Ertek, E. Kılıç / MANAS Journal of Engineering 12(1) (2024) 1-28 23

simulation.
[107] DT, RF, SVM, KNN, DNN centralized Edge-IIoTset, produced by Normal centralized DT, RF, KNN, SVM, DT, RF, SVM,
model and 2-class (binary 10 different IoT devices, was DNN model PRE, REC, F1: 100% KNN, DNN and
classification), 6-class (multi- evaluated together with 2 federated 2-class IID and Non-IID Federated DL
classification), and 15-class (multi- different ML-based IDS with ACC:100%, etc. [107] models
classification) federated DL approach. centralized and federated
mode in 7 different layers.
[108] Supervised ML algorithms (NB, SVM, Top 8 features selected to RF ACC, PRE, REC, F1:100% NB, SVM, RF
RF model) and Autocorrelation train ML classifiers
Function
[109] GBM's feature selection algorithm, and 26 features are removed and For DDQN ACC: 99,05% CNN, RNN,
PPO2 interface of the Stable baseline to only 3 features are used PRE: 98,42% LSTM, DDQN,
implement model training has been without reducing REC: 97,08% DQN
used. DRL-IDS intrusion detection performance. F1: 97,74%
agent is tested on the training and
validation sets.
[110] Hybrid model (Cu-LSTMGRU + Cu- N-BaIoT hosts malware, Cu-LSTMGRU + Cu-BLSTM ACC: Cu-DNN–LSTM
BLSTM) 10-fold cross-validation namely Bashlite and Mirai. It 99,45% and Cu-DNN–
multiclass, GPU-Enabled, Compared consists of 8 attacks and 115 PRE: 99,34% GRU,
with hybrid algorithms, Cuda- features. 49500 normal IIoT REC: 98,49% GRU-RNN,
DNNLSTM and Cuda-DNNGRU data. F1: 99,47% Autoencoder
FNR, FDR: 0.002 (EDSA)
FOR: 0,004 Multi-CNN
FPR: 0,003
TPR: 99,33%
TNR: 99,13%
MCC: 99,13%
TET: 9,79ms
[120] RF-PCCIF and RF-IFPCC Ensemble Bot-IoT with 15 selected Bot-IoT ACC: Information gain
model features and with NF- RF-PCCIF: 99,98% and gain ratio, Chi-
UNSW-NB15-v2 with 24 RF-IFPCC: 99,99% square, CNN, ET
features UNSW-NB15-v2 ACC:
RF-PCCIF: 99,3%
RF-IFPCC: 99,18%
TRT: 145.24s
[121] Linear SVM, Quadratic SVM, Fine NSL-KDD Linear SVM ACC: 99.3% Linear SVM,
Tree, Medium Tree Quadratic SVM ACC: 99.7% Quadratic SVM,
Fine Tree ACC: 99.4% , TRT: 11.029s Fine Tree, Medium
Medium Tree ACC: 95.9% Tree
[122] synchronous optimisation of Secure water treatment Gas pipeline: SVM, RNN,
parameters and architectures by genetic (SWaT), water distribution ACC: 99,04% LSTM, NB,
algorithms with convolutional neural (WADI), Gas Pipeline, BoT- PRE: 98,14% BiLSTM, CNN,
networks blocks (SOPA-GA-CNN) IoT and Power System REC: 98,07% VCDL, Deep-IFS
Attack Dataset F1: 98,1%
[123] Residual neural network (P-ResNet) Seven IoT sensors (e.g., P-ResNet LSTM, NN, CNN,
fridge_sensor, ACC: 87% RNN, FCN, LeNet,
GPS_tracker_sensor, PRE: 88% IncepNet,
motion_light_sensor, REC: 86% MCDCNN
garage_door_sensor, F1: 86%
modbus_sensor, ROC AUC: 83%
thermostat_sensor, and TRT: 24401.586s
weather_sensors) TET: 3.014s

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg

 E. E. Eryılmaz, S. Akleylek, Y. Ertek, E. Kılıç / MANAS Journal of Engineering 12(1) (2024) 1-28 24

of the proposed models in the real-world environment is

6. Conclusion, Discussions and Open Research questioned.
Problems • A manuscript has not been tested in other machine
learning and deep learning models, except for
In this manuscript, a systematic literature survey used in biologically inspired intelligence-based methods, and
Industrial Internet of Things security was done, and studies on has been evaluated with performance measures that are
ML and DL models used to detect anomaly-based attacks in not often used. The results obtained in different models
IIoT networks were examined. The examined approaches are with known performance metrics will become important
obtained from the Web of Science, Scopus, IEEE Xplore, for the evaluation of the manuscript [103].
ScienceDirect, Hindawi, Wiley Online Library, and MDPI • Some studies did not provide any conclusions regarding
academic databases using the query sentences in Table 5. training or testing times, such as the ML and DL models
Among the papers revealed as a result of the queries, 25 of them
used in other reviewed manuscripts. An analysis of the
were selected and summarized according to the selection and
resource consumption of IIoT devices with insufficient
elimination criteria given in Table 6, with the publication years
resources cannot be made. Therefore, the efficiency of
between 2019 and 2023. A systematic literature survey used in
Industrial Internet of Things security was done, and approaches that include models without resource
manuscripts on ML and DL models used to detect anomaly- consumption and training-test time analysis in a real
based attacks in IIoT networks were examined. The examined IIoT environment is unknown [90]-[93], [95], [99].
approaches are obtained from the Web of Science, Scopus, The source codes of the ML and DL models used in the
IEEE Xplore, ScienceDirect, Hindawi, Wiley Online Library, manuscripts examined were not shared in a public
and MDPI academic databases using the query sentences in environment, except for the manuscript [103]. Therefore,
Table 5. Among the papers revealed as a result of the queries, these models are not known to be established, as shown in
25 of them were selected and summarized according to the manuscripts. In addition, by sharing the source codes of the
selection and elimination criteria given in Table 6, with the approaches, other researchers examining the codes will
publication years between 2019 and 2023. contribute more to the literature in their future manuscripts.
When the reviewed manuscripts are evaluated, it is concluded In this survey, firstly, short and concise explanations about
that many manuscripts have different deficiencies. These the approaches proposed in the selected articles are given. Then,
deficiencies are summarized as follows: the main ideas, advantages and disadvantages found in these
• There are extra security measures in the blockchain to manuscripts are summarized in Table 7. Second, the criteria
make it harder for malicious nodes to verify transactions used to evaluate the performance of the ML and DL models
and connect to other devices, but it has not been used in the approaches are shown in Table 8. Third, various
compared to other DL and ML algorithms [90]. information about the datasets used in the testing and training
• Except for a few manuscripts, their datasets have not processes of the models are presented in Table 9. Fourth, the
been publicly shared [99], [103], [105]. Therefore, the ML and DL approach used in the proposed approaches to IIoT
performance results of the proposed approaches are security are given in Table 10. Table 10 summarizes the usage
controversial. Additionally, if the datasets are shared areas of the models, the datasets they use, the training-test result
publicly, other researchers will be able to evaluate the performances, and the information about which models they are
usability of these datasets and improve the datasets. compared with. In the evaluation part, the shortcomings of the
• Some datasets do not include detailed counts of manuscripts examined are given. In the conclusion part, the
malignant and benign data types [91], [96]-[98], [103], manuscript is summarized, and open research problems are
[104]. The lack of these details prevents obtaining briefly explained.
sufficient information about the datasets. IIoT leverages a variety of existing and emerging
• Most of the models proposed in the manuscripts have technologies such as communication networks, sensing
not been tested in a real-life. Therefore, the performance technologies, and high-performance processing platforms to
values of these approaches in an environment where build its entire ecosystem. As a result, IIoT security and privacy
they are actually used cannot be estimated. Manuscripts concerns don't just focus on monolithic technology issues.
should be tested in a real environment, and their There is an integrated heterogeneous environment from the
performance should be measured. physical security of connected devices to the communication
• The vulnerabilities of the proposed approaches against security of networks, from data security to IIoT application
various types of attacks were not addressed in the security. It covers a wide variety of IIoT ecosystems, consisting
reviewed manuscripts. Apart from a manuscript [100], of various security protocols, defense schemes, and many
other systems proposed to be secure against certain types standards of IIoT structure. Most models have traditional
of attacks can be used as schemes, frameworks, models, methods of protecting and defending data communications. It is
or parts. The status of security levels against different or debatable whether these traditional mechanisms deployed are
unknown attack types is unknown. That is, the usability

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg

 E. E. Eryılmaz, S. Akleylek, Y. Ertek, E. Kılıç / MANAS Journal of Engineering 12(1) (2024) 1-28 25

still sufficient to protect the latest IIoT technologies; this section • For an accurate assessment of the energy consumption
also discusses the overt security and privacy issues of IIoT. and computational complexity of the proposed
• Classes with fewer datasets will give less successful approaches, on which platforms the datasets are created
results in the real environment or a new dataset, as they and tested, training, testing, real-time response, and
will cause data to be overfitting [98], [102]. The execution times are not explicitly given [90], [92], [93],
imbalance of datasets, that is, very different numbers of [95], [99], [103], [104], [106].
benign and malignant datasets will also create • Zero-day attacks are a type of security vulnerability that
complexity and invalidate learning for different data and is exploited the day a vulnerability is discovered or
real environments [90], [92]-[95], [100]. before an update is available by the developer.
• Except for some manuscripts, other datasets are old and Dynamically changing zero-day attacks can cause
outdated [93], [107]. Therefore, it is difficult to find a unknown malicious behavior to be detected [46].
suitable benchmark dataset to apply ML and DL models • False positives will cause economic worsening that will
in IIoT security. However, most of the datasets used are affect the relevant services and production areas.
not publicly available or the datasets are too small, Whenever a false positive is found, especially medical,
especially for deep learning models [102]. industrial units will have to stop production. False
• While machine learning models are successful in some negatives are even more problematic. It is the
datasets, deep learning models give more successful appearance of a condition as negative as a result of a test
results in others [101]. Some approaches do not make when it actually is. As a result of misinterpretation of
comparisons between ML and DL models. In addition, data due to unforeseen conditions, not only economic
some articles do not apply preprocessing and feature but also human losses will occur [98], [100], [102],
selection steps for datasets [91]. Therefore, too many [104], [106]. Such cases are still important problems to
features are obtained. Feature selection and feature be solved.
extraction are very important in terms of performance As a result, in this systematic survey, detailed information
and complexity, especially for ML models. The about open research problems in the literature and models
performance of ML models can be increased by consisting of deep learning and machine learning algorithms
selecting the feature. to find anomalies in IIoT networks and reduce these anomalies
• Several authors working on the same dataset did not are given.
compare the results of the manuscripts [90], [99], [91].
Some articles do not include dataset details and feature REFERENCES
information [96]-[99], [102]-[104], [106]. [1] L.S. Vailshery, Number of Internet of Things (IoT) connected devices
• Anomaly detection, which is mainly used, may not be worldwide from 2019 to 2023, with forecasts from 2022 to 2030,
applied in the same way in all areas. For example, while https://www.statista.com/statistics/1183457/iot-connected-devices-
worldwide/ , Statista, Last accessed: October 31, 2021.
temperature change is very important in the field of [2] M. Hatton, The IoT in 2030: Which applications account for the biggest
industrial medicine, it may not be that important for a chunk of the $1.5 trillion opportunity? TransformaInsights,
smart factory. Therefore, anomaly detection should not https://www.kisa.link/PsHW, Last accessed: October 31, 2021..
[3] F. Meneghello, et al., IoT: Internet of Threats? A Survey of Practical
be applied to all areas in the same way [98], [100], [102], Security Vulnerabilities in Real IoT Devices, IEEE Internet of Things
[106]. Journal, vol. 6, no. 5, pp. 8182–8201, 2019.
• Normal data may be close to the cluster containing the [4] C. Xenofontos, et al. Consumer, commercial and industrial iot (in)
anomaly data, and anomaly data may be close to the security: attack taxonomy and case studies. IEEE Internet of Things
Journal, 2021.
cluster containing the normal data [46]. In such cases, [5] D. Antonioli, et al., Blurtooth: Exploiting cross-transport key
anomaly detection becomes very difficult. Normal data derivation in bluetooth classic and bluetooth low energy, arXiv preprint
may change according to time and space and appear as arXiv:2009.11776, 2020.
[6] L. L. Dhirani, E. Armstrong, and T. Newe, Industrial IoT, Cyber
an anomaly. In these cases, it may be necessary to Threats, and Standards Landscape: Evaluation and Roadmap. Sensors,
change the hyperparameters used in the ML and DL 21(11), 3901, 2021
models. [7] A. R. Sadeghi, C. Wachsmann, & M. Waidner, Security and privacy
challenges in industrial internet of things. In 2015 52nd
• The DL and ML models used in IIoT security focus only ACM/EDAC/IEEE Design Automation Conference (DAC) (pp. 1-6).
on the accuracy performance metric in some articles IEEE, June 2015.
[96], [102], [106]. Instead, manuscripts including [8] J. P. Anderson, Computer security threat monitoring and surveillance,
Technical Report, James P. Anderson Company, 1980
precision, recall, and F1 score performance criteria [9] B. B. Zarpelão, et al, A survey of intrusion detection in Internet of
should be conducted to better understand the Things, Journal of Network and Computer Applications, Volume 84,
manuscripts. In some cases, performance criteria such as Pages 25-37, ISSN 1084-8045,
log loss, speedup ratio, g-mean, rand-index, and https://doi.org/10.1016/j.jnca.2017.02.009, 2017
[10] E. Hodo, et al, Threat analysis of IoT networks using artificial neural
specificity are used, which are not used much in the network intrusion detection system. In 2016 International Symposium
literature [92], [102].

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg

 E. E. Eryılmaz, S. Akleylek, Y. Ertek, E. Kılıç / MANAS Journal of Engineering 12(1) (2024) 1-28 26

on Networks, Computers and Communications (ISNCC) (pp. 1-6). [31] T. Salimans, et al. Improved techniques for training gans. Advances in
IEEE, May 2016. neural information processing systems, 29, 2234-2242. 2016.
[11] E. Anthi, et al, A supervised intrusion detection system for smart home [32] U. Fiore, et al. Network anomaly detection with the restricted
IoT devices. IEEE Internet of Things Journal, 6(5), 9042-9053, 2019. Boltzmann machine. Neurocomputing, 122, 13-23. 2013.
[12] S. Raza, L. Wallgren, & T. Voigt, SVELTE: Real-time intrusion [33] Y. Zhang, P. Li, & X. Wang, Intrusion detection for IoT based on
detection in the Internet of Things. Ad hoc networks, 11(8), 2661-2674, improved genetic algorithm and deep belief network. IEEE Access, 7,
2013. 31711-31722. 2019.
[13] V. Kumar, A. K. Das, & D. Sinha, UIDS: A unified intrusion detection [34] K. Tange, et al. Towards a systematic survey of industrial IoT security
system for IoT environment. Evolutionary Intelligence, 14(1), 47-59, requirements: research method and quantitative analysis, Proceedings
2021. of the Workshop on Fog Computing and the IoT, 2019.
[14] M. Eskandari, et al, Passban IDS: An intelligent anomaly-based [35] K. Tange, et al, A Systematic Survey of Industrial Internet of Things
intrusion detection system for IoT edge devices. IEEE Internet of Security: Requirements and Fog Computing Opportunities, in IEEE
Things Journal, 7(8), 6882-6897, 2020. Communications Surveys & Tutorials, vol. 22, no. 4, pp. 2489-2520,
[15] E. Aydogan, et al. A central intrusion detection system for rpl-based Fourthquarter 2020.
industrial internet of things. In 2019 15th IEEE International [36] T. Soo Fun, & A. Samsudin, Recent Technologies, Security
Workshop on Factory Communication Systems (WFCS) (pp. 1-5). Countermeasure and Ongoing Challenges of Industrial Internet of
IEEE, May 2019. Things (IIoT): A Survey. Sensors, 21(19), 6647. 2021.
[16] M. Zolanvari, et al., Machine learning-based network vulnerability [37] S. Bhatt, & P.R. Ragiri, Security trends in Internet of Things: A survey.
analysis of industrial Internet of Things. IEEE Internet of Things SN Applied Sciences, 3(1), 1-14. 2021.
Journal, 6(4), 6822-6834, 2019. [38] M. Serror, et al, Challenges and Opportunities in Securing the
[17] J. B. Awotunde, C. Chakraborty, & A. E. Adeniyi, Intrusion Detection Industrial Internet of Things, IEEE Transactions on Industrial
in Industrial Internet of Things Network-Based on Deep Learning Informatics, vol. 17, no. 5, pp. 2985-2996, doi:
Model with Rule-Based Feature Selection. Wireless Communications 10.1109/TII.2020.3023507, May 2021.
and Mobile Computing, 2021. [39] Y. Wu, et al. Deep reinforcement learning for blockchain in industrial
[18] A. H. Muna, N. Moustafa & E. Sitnikova, Identification of malicious IoT: A survey. Computer Networks, 191, 108004. 2021.
activities in industrial internet of things based on deep learning [40] K. Tsiknas, et al, Cyber Threats to Industrial IoT: A Survey on Attacks
models. Journal of Information security and applications, 41, 1-11, and Countermeasures. IoT, 2(1), 163-188, 2021.
2018. [41] M. A. Al-Garadi, et al, A Survey of Machine and Deep Learning
[19] G. E. I. Selim, et al. Anomaly events classification and detection Methods for Internet of Things (IoT) Security, IEEE Communications
system in critical industrial internet of things infrastructure using Surveys & Tutorials, vol. 22, no. 3, pp. 1646-1685, 2020.
machine learning algorithms. Multimedia Tools and Applications, [42] R. A. Khalil, et al. Deep Learning in the Industrial Internet of Things:
80(8), 12619-12640, 2021. Potentials, Challenges, and Emerging Applications, IEEE Internet of
[20] A. F. M. Agarap, A neural network architecture combining gated Things Journal, vol. 8, no. 14, pp. 11016-11040, 15 July15, 2021.
recurrent unit (GRU) and support vector machine (SVM) for intrusion [43] R. Ahmad & I. Alsmadi, Machine learning approaches to IoT security:
detection in network traffic data. In Proceedings of the 2018 10th A systematic literature review. Internet of Things, 100365. 2021.
international conference on machine learning and computing (pp. 26- [44] L. Aversano, et al. A systematic review on Deep Learning approaches
30). 2018, February. for IoT security. Computer Science Review, 40, 100389. 2021
[21] S. Aljawarneh, M. Aldwairi, & M. B. Yassein. Anomaly-based [45] Rudenko, R., Pires, I. M., Oliveira, P., Barroso, J., & Reis, A. (2022).
intrusion detection system through feature selection analysis and A Brief Review on Internet of Things, Industry 4.0 and Cybersecurity.
building hybrid efficient model. Journal of Computational Science, 25, Electronics, 11(11), 1742.
152-160. 2018. [46] Ahanger, T. A., Aljumah, A., & Atiquzzaman, M. (2022). State-of-the-
[22] L. Breiman, et al, Classification and regression trees. Routledge. 2017. art survey of artificial intelligent techniques for IoT security. Computer
[23] L. Li, H. Zhang, H. Peng, & Y. Yang, Nearest neighbors based density Networks, 108771.
peaks approach to intrusion detection. Chaos, Solitons & Fractals, 110, [47] L. Tan and N. Wang, Future internet: The Internet of Things, 2010 3rd
33-40. 2018. International Conference on Advanced Computer Theory and
[24] A. L. Buczak & E. Guven. A survey of data mining and machine Engineering (ICACTE), pp. V5-376-V5-380, 2010
learning methods for cyber security intrusion detection. IEEE [48] F. A. Alaba, et al, Internet of Things security: A survey, J. Netw.
Communications surveys & tutorials, 18(2), 1153-1176. 2015. Comput. Appl., 88, 10–28, 2017.
[25] A. P. Muniyandi, R. Rajeswari, & R. Rajaram, Network anomaly [49] H. Boyes, et al. The industrial internet of things (IIoT): An analysis
detection by cascading k-Means clustering and C4. 5 decision tree framework. Computers in industry, 101, 1-12. 2018.
algorithm. Procedia Engineering, 30, 174-182. 2012. [50] J. Sengupta, S. Ruj & S. D. Bit, A comprehensive survey on attacks,
[26] R. Vinayakumar, K. P. Soman, & P. Poornachandran, Applying security issues and blockchain solutions for IoT and IIoT. Journal of
convolutional neural network for network intrusion detection. In 2017 Network and Computer Applications, 149, 102481. 2020.
International Conference on Advances in Computing, [51] U. Saxena, J. S Sodhi, & Y. Singh. An Analysis of DDoS Attacks in a
Communications and Informatics (ICACCI) (pp. 1222-1228). IEEE. Smart Home Networks. In 2020 10th International Conference on
September, 2017. Cloud Computing, Data Science & Engineering (Confluence) (pp. 272-
[27] A. A. Diro, & N. Chilamkurti. Distributed attack detection scheme 276). IEEE. January 2020.
using deep learning approach for Internet of Things. Future Generation [52] S. Alzahrani and L. Hong, Generation of DDoS attack dataset for
Computer Systems, 82, 761-768. 2018. effective IDS development and evaluation, J. Inf. Secur. 9 (4), 225–
[28] J. Kim, et al. Long short term memory recurrent neural network 241, 2018.
classifier for intrusion detection. In 2016 International Conference on [53] Y. Gu, et al, Semi-supervised K-means DDoS detection method using
Platform Technology and Service (PlatCon) (pp. 1-5). IEEE. (2016, hybrid feature selection algorithm, IEEE Access 7, 64351–64365,
February). 2019.
[29] P. Liu, X. Qiu, & X. Huang, X. Recurrent neural network for text [54] Y.N. Soe, et al, DDoS attack detection based on simple ANN with
classification with multi-task learning. arXiv preprint SMOTE for IoT environment, in: 2019 Fourth International Conference
arXiv:1605.05101. 2016. on Informatics and Computing (ICIC), pp. 1–5, 2019.
[30] M. Yousefi-Azar, et al. Autoencoder-based feature learning for cyber [55] N. Chaabouni, et al, Network intrusion detection for iot security based
security applications. In 2017 International joint conference on neural on learning techniques, IEEE Commun. Surv. Tutor. 21 (3), 2671–
networks (IJCNN) (pp. 3854-3861). IEEE. (2017, May). 2701, 2019.

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg

 E. E. Eryılmaz, S. Akleylek, Y. Ertek, E. Kılıç / MANAS Journal of Engineering 12(1) (2024) 1-28 27

[56] P. García-Teodoro, et al, Anomaly-based network intrusion detection: [79] S.T.C. Konigsmark, D. Chen, M.D.F. Wong, Information dispersion
techniques, systems and challenges, Comput. Secur. 28 (1), 18–28, for trojan defense through high-level synthesis, 2016 53nd
2009. ACM/EDAC/IEEE Design Automation Conference (DAC), 2016.
[57] I. Andrea, C. Chrysostomou, G. Hadjichristofi Internet of things: [80] H. Naeem, et al, A light-weight malware static visual analysis for iot
security vulnerabilities and challenges, 2015 IEEE Symposium on infrastructure, International Conference on Artificial Intelligence and
Computers and Communication (ISCC), ), pp. 180-187, 2015. Big Data (ICAIBD), 2018.
[58] M.M. Ahemd, M.A. Shah, A. Wahid, Iot security: a layered approach [81] J. Su, et al, Lightweight classification of iot malware based on image
for attacks and defenses, 2017 International Conference on recognition, IEEE 42nd Annual Computer Software and Applications
Communication Technologies (ComTech), pp. 104-110, 2017. Conference (COMPSAC), vol. 02, 2018.
[59] M. R. Bartolacci, et al, Personal denial of service (PDOS) attacks: A [82] T. Song, et al, A privacy preserving communication protocol for iot
discussion and exploration of a new category of cyber crime. Journal applications in smart homes, IEEE Internet Things J., 4 (6), 2017.
of Digital Forensics, Security and Law, 9(1), 2. 2014. [83] C. Machado, A.A.M. Frhlich, Iot data integrity verification for cyber-
[60] M.N. Aman, et al, A light-weight mutual authentication protocol for physical systems using blockchain, 2018 IEEE 21st International
iot systems, GLOBECOM 2017 - 2017 IEEE Global Communications Symposium on Real-Time Distributed Computing (ISORC), 2018.
Conference, pp. 1-6, 2017. [84] Y. Rahulamathavan, et al, Privacy-preserving blockchain based iot
[61] T. Gomes, et al, Cute mote, a customizable and trustable end-device ecosystem using attribute-based encryption, IEEE International
for the internet of things, IEEE Sens. J., 17 (20), pp. 6816-6824, 2017. Conference on Advanced Networks and Telecommunications Systems
[62] P. Porambage, et al, Pauthkey: a pervasive authentication protocol and (ANTS), 2017.
key establishment scheme for wireless sensor networks in distributed [85] D. Zheng, et al, Efficient and privacy-preserving medical data sharing
iot applications, Int. J. Distributed Sens. Netw., 10 (7), 2014. in internet of things with limited computing power, IEEE Access, 6,
[63] X. Hei, et al, Defending resource depletion attacks on implantable 2018.
medical devices, 2010 IEEE Global Telecommunications Conference [86] P. Gope, B. Sikdar, Lightweight and privacy-preserving two-factor
GLOBECOM 2010, pp. 1-5. 2010. authentication scheme for iot devices, IEEE Internet Things J., 2018.
[64] J. Choi and Y. Kim, An improved lea block encryption algorithm to [87] J. Sengupta, et al, End to end secure anonymous communication for
prevent side-channel attack in the iot system 2016 Asia-Pacific Signal secure directed diffusion in iot, Proceedings of the 20th International
and Information Processing Association Annual Summit and Conference on Distributed Computing and Networking, ICDCN '19,
Conference (APSIPA), pp. 1-4, 2016. 2019.
[65] S. Sicari, et al, Reato: reacting to denial of service attacks in the [88] F. Li, et al, System statistics learning-based IoT security: Feasibility
internet of things, Comput. Network., 137, pp. 37-48, 2018. and suitability, IEEE Internet Things J., vol. 6, no. 4, pp. 6396-6403,
[66] P. Varga, et al, Security threats and issues in automation iot, 2017 Aug. 2019.
IEEE 13th International Workshop on Factory Communication [89] Magaia, Naercio, et al. Industrial Internet-of-Things Security
Systems (WFCS), pp. 1-6, 2017. Enhanced with Deep Learning Approaches for Smart Cities. IEEE
[67] J. Liu, et al, Epic: a differential privacy framework to defend smart Internet of Things Journal 8.8, 2020
homes against internet traffic analysis, IEEE Internet Things J., 5 (2), [90] Sharma, M., Pant, S., Kumar Sharma, D., Datta Gupta, K., Vashishth,
2018. V., & Chhabra, A. Enabling security for the Industrial Internet of
[68] U. Guin, et al, A secure low-cost edge device authentication scheme for Things using deep learning, blockchain, and coalitions. Transactions
the internet of things, 31st International Conference on VLSI Design on Emerging Telecommunications Technologies, 32(7), e4137. 2021.
and 17th International Conference on Embedded Systems (VLSID). [91] M. M. N. Aboelwafa, et al, A Machine-Learning-Based Technique for
2018. False Data Injection Attacks Detection in Industrial IoT, in IEEE
[69] G. Glissa, et al, A secure routing protocol based on rpl for internet of Internet of Things Journal, vol. 7, no. 9, pp. 8462-8471, Sept. 2020.
things, IEEE Global Communications Conference (GLOBECOM), [92] Z. E. Huma et al., A Hybrid Deep Random Neural Network for
2016. Cyberattack Detection in the Industrial Internet of Things, in IEEE
[70] C. Pu and S. Hajjar, Mitigating forwarding misbehaviors in rpl-based Access, vol. 9, pp. 55595-55605, 2021.
low power and lossy networks, 2018 15th IEEE Annual Consumer [93] S. Liu, et al, Network Log Anomaly Detection Based on GRU and
Communications Networking Conference (CCNC), 2018. SVDD, 2019 IEEE Intl Conf on Parallel & Distributed Processing with
[71] C. Cervantes, et al, Detection of sinkhole attacks for supporting secure Applications, Big Data & Cloud Computing, Sustainable Computing
routing on 6lowpan for internet of things, 2015 IFIP/IEEE & Communications, Social Computing & Networking
International Symposium on Integrated Network Management (IM), (ISPA/BDCloud/SocialCom/SustainCom), pp. 1244-1249, 2019.
2015. [94] S. Latif, et al, A Novel Attack Detection Scheme for the Industrial
[72] P. Shukla, Ml-ids: A machine learning approach to detect wormhole Internet of Things Using a Lightweight Random Neural Network," in
attacks in internet of things, Intelligent Systems Conference IEEE Access, vol. 8, pp. 89337-89350, 2020.
(IntelliSys), 2017. [95] S. Latif, et al, DRaNN: A Deep Random Neural Network Model for
[73] D. Airehrour, J.A. Gutierrez & S.K. Ray, Sectrust-rpl: a secure trust- Intrusion Detection in Industrial IoT, 2020 International Conference
aware rpl routing protocol for internet of things, Future Gener. on UK-China Emerging Technologies (UCET), pp. 1-4, 2020.
Comput. Syst., 2019. [96] M. M. Hassan, M. R. Hassan, S. Huda and V. H. C. de Albuquerque, A
[74] M. Singh, et al, Secure mqtt for internet of things (iot), 5th International Robust Deep-Learning-Enabled Trust-Boundary Protection for
Conference on Communication Systems and Network Technologies, Adversarial Industrial IoT Environment, in IEEE Internet of Things
2015. Journal, vol. 8, no. 12, pp. 9611-9621, 15 June15, 2021.
[75] Y. Ashibani, Q.H. Mahmoud, An efficient and secure scheme for smart [97] M. M. Hassan, A. Gumaei, S. Huda and A. Almogren, Increasing the
home communication using identity-based signcryption, 2017 IEEE Trustworthiness in the Industrial IoT Networks Through a Reliable
36th International Performance Computing and Communications Cyberattack Detection Model, in IEEE Transactions on Industrial
Conference (IPCCC), 2017. Informatics, vol. 16, no. 9, pp. 6154-6162, Sept. 2020.
[76] V. Adat, B.B. Gupta, A ddos attack mitigation framework for internet [98] Y. Liu et al., Deep Anomaly Detection for Time-Series Data in
of things, 2017 International Conference on Communication and Industrial IoT: A Communication-Efficient On-Device Federated
Signal Processing (ICCSP), 2017. Learning Approach, in IEEE Internet of Things Journal, vol. 8, no. 8,
[77] D. Yin, et al, A ddos attack detection and mitigation with software- pp. 6348-6358, 15 April15, 2021.
defined internet of things framework, IEEE Access, 6, 2018. [99] M. Khoda, T. Imam, J. Kamruzzaman, I. Gondal and A. Rahman,
[78] C. Liu, P. Cronin, C. Yang, A mutual auditing framework to protect iot Robust Malware Defense in Industrial IoT Applications Using
against hardware trojans, 2016 21st Asia and South Pacific Design Machine Learning With Selective Adversarial Samples, in IEEE
Automation Conference (ASP-DAC), 2016.

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg

 E. E. Eryılmaz, S. Akleylek, Y. Ertek, E. Kılıç / MANAS Journal of Engineering 12(1) (2024) 1-28 28

Transactions on Industry Applications, vol. 56, no. 4, pp. 4415-4424, [120] Mohy-Eddine, M., Guezzaz, A., Benkirane, S., Azrour, M., &
July-Aug. 2020. Farhaoui, Y. (2023). An Ensemble Learning Based Intrusion Detection
[100] A. N. Jahromi, H. Karimipour, A. Dehghantanha and K. -K. R. Choo, Model for Industrial IoT Security. Big Data Mining and Analytics,
Toward Detection and Attribution of Cyber-Attacks in IoT-Enabled 6(3), 273-287.
Cyber–Physical Systems, in IEEE Internet of Things Journal, vol. 8, [121] Alshahrani, H., Khan, A., Rizwan, M., Reshan, M. S. A., Sulaiman, A.,
no. 17, pp. 13712-13722, 1 Sept.1, 2021. & Shaikh, A. (2023). Intrusion Detection Framework for Industrial
[101] A. Alsaedi, N. Moustafa, Z. Tari, A. Mahmood and A. Anwar, Internet of Things Using Software Defined Network. Sustainability,
TON_IoT Telemetry Dataset: A New Generation Dataset of IoT and 15(11), 9001.
IIoT for Data-Driven Intrusion Detection Systems, in IEEE Access, [122] Huang, J. C., Zeng, G. Q., Geng, G. G., Weng, J., & Lu, K. D. (2023).
vol. 8, pp. 165130-165150, 2020. SOPA‐GA‐CNN: Synchronous optimisation of parameters and
[102] J. Zhao, et al, Anomaly Detection Collaborating Adaptive CEEMDAN architectures by genetic algorithms with convolutional neural network
Feature Exploitation with Intelligent Optimizing Classification for blocks for securing Industrial Internet‐of‐Things. IET Cyber‐Systems
IIoT Sparse Data. Wireless Communications and Mobile Computing, and Robotics, 5(1), e12085.
2021. [123] Mehedi, S. T., Anwar, A., Rahman, Z., Ahmed, K., & Islam, R. (2022).
[103] T. Primya & G. Subashini, Swarm intelligence‐based secure high‐order Dependable intrusion detection system for IoT: A deep transfer
optimal density selection for industrial internet‐of‐things (IIoT) data on learning based approach. IEEE Transactions on Industrial Informatics,
cloud environment. International Journal of Communication Systems, 19(1), 1006-1017.
34(17), e4976, 2021.
[104] F. Hussain et al, A Framework for Malicious Traffic Detection in IoT
Healthcare Environment. Sensors, 21(9), 3025. 2021.
[105] M. Alqahtani et al, IoT botnet attack detection based on optimized
extreme gradient boosting and feature selection. Sensors, 20(21), 6336,
2020.
[106] I. Campero-Jurado et al. Smart Helmet 5.0 for industrial internet of
things using artificial intelligence. Sensors, 20(21), 6241. 2020
[107] M. A. Ferrag, O. Friha, D. Hamouda, L. Maglaras and H. Janicke,
"Edge-IIoTset: A New Comprehensive Realistic Cyber Security
Dataset of IoT and IIoT Applications for Centralized and Federated
Learning," in IEEE Access, vol. 10, pp. 40281-40306, 2022, doi:
10.1109/ACCESS.2022.3165809.
[108] Kumar, A., Shridhar, M., Swaminathan, S., & Lim, T. J. Machine
learning-based early detection of IoT botnets using network-edge
traffic. Computers & Security, 117, 102693. 2022
[109] Tharewal, S., Ashfaque, M. W., Banu, S. S., Uma, P., Hassen, S. M.,
& Shabaz, M. Intrusion detection system for industrial Internet of
Things based on deep reinforcement learning. Wireless
Communications and Mobile Computing, 2022.
[110] Javeed, D., Gao, T., Khan, M. T., & Shoukat, D. A hybrid intelligent
framework to combat sophisticated threats in secure industries.
Sensors, 22(4), 1582. 2022.
[111] D. Arp, et al. Drebin: Effective and explainable detection of android
malware in your pocket. In Ndss (Vol. 14, pp. 23-26), February 2014.
[112] H. Satilmiş & S. Akleylek, A review of machine learning and deep
learning models used for IoT security. Bilişim Teknolojileri Dergisi,
14(4), 457-481. 2021.
[113] N. Koroniotis, et al. Towards the development of realistic botnet
dataset in the internet of things for network forensic analytics: Bot-iot
dataset. Future Generation Computer Systems, 100, 779-796. 2019.
[114] Sethi, P., & Sarangi, S. R. (2017). Internet of things: architectures,
protocols, and applications. Journal of electrical and computer
engineering, 2017.
[115] AlSalem, T. S., Almaiah, M. A., & Lutfi, A. (2023). Cybersecurity
Risk Analysis in the IoT: A Systematic Review. Electronics, 12(18),
3958.
[116] Rodríguez, E., Otero, B., & Canal, R. (2023). A survey of machine and
deep learning methods for privacy protection in the Internet of Things.
Sensors, 23(3), 1252.
[117] Santhosh Kumar, S. V. N., Selvi, M., & Kannan, A. (2023). A
comprehensive survey on machine learning-based intrusion detection
systems for secure communication in internet of things. Computational
Intelligence and Neuroscience, 2023.
[118] Sarker, I. H., Khan, A. I., Abushark, Y. B., & Alsolami, F. (2023).
Internet of things (iot) security intelligence: a comprehensive
overview, machine learning solutions and research directions. Mobile
Networks and Applications, 28(1), 296-312.
[119] Nuaimi, M., Fourati, L. C., & Hamed, B. B. (2023). Intelligent
approaches toward intrusion detection systems for Industrial Internet
of Things: A systematic comprehensive review. Journal of Network
and Computer Applications, 103637.

MJEN MANAS Journal of Engineering, Volume 12 (Issue 1) © 2024 www.journals.manas.edu.kg