Stacked Ensemble Learning Based Approach for

Anomaly Detection in IoT Environment

Neel Gandhi
Department of Information and Communication Technology
Pandit Deendayal Energy University
Gandhinagar, Gujarat
[email protected]

Abstract—Machine learning based anomaly detection systems have proved to be promising solutions in the sector of IoT for
have gained prominence in field of Internet of Things (IoT) due preserving privacy [3]. Machine learning algorithms integrated
to their effectiveness in dealing with security and privacy issues. with IoT have found applications in the areas of surveillance,
2021 2nd International Conference on Range Technology (ICORT) | 978-1-6654-4956-4/21/$31.00 ©2021 IEEE | DOI: 10.1109/ICORT52730.2021.9581549

Internet of things has manifold applications possible with aid of

effective integration of sensors, databases, machines and services security, monitoring, and control. Machine learning has ability
at work. Due to increasing use of IoT-based architectures, attacks to provide strong security mechanism for defending IoT sys-
and anomaly detection has become a crucial part for functioning tems against undesirable attacks caused by attackers to exploit
of IoT. The basic goal of an anomaly detection system is to given IoT system. Attack detection systems fall under two
verify whether the behavior of the system is normal or unfaithful categories namely signature-based systems and anomaly-based
actions are being taken by system. Anomaly detection systems
are used for detection of attacks and anomalies right from denial- systems. Signature-based systems were found helpful only
of-service to malicious operations that may cause disruption to incase of known attacks whereas anomaly detection systems
IoT-based systems. A variety of machine learning algorithms have were able to identify unknown attacks using machine learning
been used for the purpose of anomaly and attack detection. In algorithms. Anomaly detection systems provided better levels
this paper, we analyzed different machine learning algorithms of security in comparison to traditional signature-based meth-
and compared it against our proposed Stacked ensemble learning
model. Evaluation metrics used for comparison of various ma- ods. Machine learning algorithms became extensively been
chine learning algorithms against our proposed stacked ensemble used for detection of attacks in IoT systems to prevent attacks.
learning model include F1 score, precision,accuracy, recall, and In the given paper, we propose machine learning algorithm
area under ROC curve. The proposed system is found to have based on stacked ensemble machine learning model for the
an accuracy of 99.8% that is superior in comparison to most process of anomaly detection in IoT sensor environment.
traditional machine learning algorithms. The proposed stacked
ensemble learning model could be effectively used for improving The paper proposes anomaly detection system that would be
the existing anomaly detection system. effective in detection of vulnerabilities and provide secure and
Index Terms—Internet of Things,Anomaly detection,Stacked reliable IoT infrastructure against a variety of cyberattacks.
Ensemble Learning, Machine Learning,Cybersecurity The main contribution of the paper include :-
1) The paper proposes stacked ensemble machine learning
I. I NTRODUCTION approach for purpose of anomaly detection tested on
Internet of things has grown exponentially over the years DS2OS traffic traces data set in virtual IoT environment
as a result of progress in Information and Communication 2) Secondly, a detailed comparative analysis of various ma-
Technology. Internet of things has assisted devices to exchange chine learning models against proposed machine learning
data with each other. Internet of things has evolved as tech- model are provided in order to measure performance and
nology for bringing a new world of interconnected devices. effectiveness of proposed machine learning model
Data-driven architectures used in internet of things along with 3) Proposed stacked ensemble model was evaluated on
efficient integration of machine learning have assisted different various evaluation metrics including recall, preci-
domains of human life. Still, IoT has concerns regarding sion,accuracy and F1 score
privacy and safety of the system due to being defenseless 4) Proposed stacked ensemble model was integrated with ex-
in cases of attacks [1]. Concerns of security and privacy isting IoT systems and proved to be effective in providing
might affect end-users by leakage or modification of critical significant security parameters and feasible performance
information affecting social life of end-user.Also, increasing for developing secure IoT environment against variety of
complexity of the data-driven architecture for IoT framework cyberattacks.
has resulted in creation of new horizons for attacks and
anomalies to exploit given IoT environment [2]. Hence, it II. R ELATED WORKS
is crucial to improve existing IoT system based architectures In the sphere of IoT, several researchers have sought to
in terms of user privacy and information security with the overcome the challenge of anomaly detection.Researchers
help of reliable machine learning methods. In the past couple have been successful in dealing with the problem of anomaly
of years, machine learning-based anomaly detection systems detection using machine learning and other related methods.

Authorized licensed use limited to: Vivekanand Education Society's Inst of Tech. Downloaded on September 23,2024 at 08:46:40 UTC from IEEE Xplore. Restrictions apply.
 Detector and firewall were developed by Pahl et al [4] for evaluated for its effectiveness on the basis of precision and
detection of anomaly in IoT microservices on IoT site. K- recall metrics. A deep belief network using single hidden
means and BIRCH based clustering methods [5] were im- layer restricted Boltzmann machine was used for unsupervised
plemented for different microservices in IoT environments. feature attribute reduction for the purpose of anomaly detec-
Also, cluster center and standard deviation were taken into tion by Alrawashdeh and Purdy [17].Hasan et al [18] used
consideration for online learning technique. The clustering machine learning approaches for Attack and anomaly detection
model was able to achieve accuracy of about 96.3 %. Dense in IoT sensors for IoT sites.The authors of the papers prepared
Random Neural Network (DRNN) based method was used detailed comparative analysis of various machine learning
for detection of security breaches in smart home system by algorithms and identified random forest as the most appro-
Gelenbe et al [6]. The paper provides description about Denial priate algorithm for solving problem of anomaly detection
of Sleep attack and Denial of Service attack on IoT site.The providing an accuracy of 99.4%. Reddy et al [19] developed
system was developed by Liu et al [7] that use light probe a deep neural network based approach for anomaly detection
routing mechanism for estimating trust of each neighboring providing assistance to applications of smart cities. Literature
node utilising the mechanism of on and off attack by de- survey proves that machine learning has the ability to handle
tector for identifying malicious network node and resulting the problem of anomaly detection.In the given paper,we have
in detection of anomaly. Fog-to-things architecture was used proposed stacked ensemble learning method for detection of
by Diro et al [8] for detection of attacks and anomaly. The anomalies that proved to be superior in comparison to other
authors of the paper provided a detailed comparative study for machine learning approaches.
shallow and deep neural networks for four classes of attacks
and obtain the accuracy of 98.27% and 96.75% for Deep III. DATASET D ESCRIPTION
and Shallow neural networks respectively. Digital watermark The data set used for the purpose of IoT traffic tracing was
were proposed by Usmonov et al [9] to handle the security collected from Distributed Smart Space Orchestration System
problems while developing embedded technologies in case of (DS2OS) that generates synthetic data using virtual IoT envi-
IoT . The method was helpful in preserving data at physical, ronment.The data set is publicly available on Kaggle [20],
logical, and virtual component level for a given IoT system. in given IoT architecture, microservices communicate with
Software including Wireshark for capturing network traffic and each other with use of Message Queuing Telemetry Transport
Weka for applying machine learning classifiers were used by (MQTT) protocol. The data set comprises 13 distinctive fea-
Anthi et al [10] for developing an intrusion detection system. tures including 347,935 Normal data and 10,017 anomalous
A comparative analysis of machine learning classifiers was data. All feature attributes were nominal by nature except
developed for the operations of denial of service, probing timestamp being discrete in nature and value attribute being
and network scanning. Anomaly detection systems were used continuous. Also, normality for given dataset was classified
in healthcare for applications of medical image analysis, big into normal and anomalous types where seven types of at-
data mining, biomedical signal analysis and predictive ana- tacks were classified into Malicious Control (M.C),Data Type
lytics by Ukil et al [11].Two-tier classification and Two-layer Probing (D.P),Denial of Service (DoS),Malicious Operation
dimension reduction were used for the purpose of intrusion (M.O),Scan(SC),Spying (SP),Normal(NL) and Wrong Setup
detection on NSL-KDD dataset by Pajouh et al [12]. User (W.S).Dataset Description in terms of frequency table and data
to Root and Remote to Local attack like malicious activi- distribution is demonstrated by TableI.
ties were identified and dimensionality reduction was done
using component analysis and linear discriminant analysis. TABLE I: Frequency table and data distribution table for all
The suspicious malicious behavior was detected using two- classes of attacks
tier classification models .Batch relevance-based approach was Attacks % of % of Frequency
used by D’Angelo et al. [13] for classification of feature Total Data Anomalous Data Count
using J-48 classification algorithm. The algorithm obtained Denial of Service 01.61% 57.70% 5780
accuracy of 94.1% in case of NSL-KDD dataset and 97.4% in Data Type Probing 00.09% 03.41% 342
case of Real Traffic Data. Extreme Learning Machines (ELM) Malicious Control 00.24% 08.87% 889
[14] based approach for classification of attacks on cloud Malicious Operation 00.22% 08.03% 805
architecture yielding netflow formatted data was analyzed for Scan 00.43% 15.44% 1547
three scenarios namely command and control, scanning, and
Spying 00.14% 05.31% 532
infected hosts. The accuracy results for the three scenarios
Wrong Setup 00.03% 01.21% 122
were found to 0.76,0.99 and 0.95, respectively. Artificial
neural network [15] based approach was used to deal with
challenges of IoT. During internet packet traces, denial-of- A. Data preprocessing
service attacks were considered.Categorization was done on Data preprocessing is an essential step before feeding
basis of threat and normal behavior. IoT-IDM-based smart the data into a machine learning model for classification.
device model was used in intrusion detection for investigating Exploratory data analysis(EDA) helps us to get meaningful
malicious activities by Nobakht et al [16]. The model was insights from given Distributed Smart Space Orchestration

Authorized licensed use limited to: Vivekanand Education Society's Inst of Tech. Downloaded on September 23,2024 at 08:46:40 UTC from IEEE Xplore. Restrictions apply.
 System (DS2OS) data set. Exploratory data analysis and learning pipeline was helpful for attack and anomaly detection
observations help us to uncover the missing values in DS2OS in IoT sensor environment as illustrated by Fig 1.
data set having 13 features and 357,952 samples of categorical
data. In the given data set, “Value” column and “Accessed
Node Type” column have missing data which are used for
generating anomalies for the process of data transfer in IoT
environment.“Accessed Node Type” feature having categorical
values contains 148 rows of not a number(NaN) that are
substituted by title “malicious” in order to detect anomalies
rising as result of data transfer process. Also,“Value” col-
umn contains abrupt data values including ”True”, ”False”,
”Twenty” and ”None” values that were substituted by 1,0,20
and 0 respectively. Data preprocessing steps were helpful for
preprocessing our given data set before feeding it to a machine
learning model.Timestamp feature was found to be discrete in
nature and not considered during preprocessing step. Further,
preprocessed categorical data was transformed into appropriate
numerical data with use of Label Encoding. The data set was
helpful in classifying the normality of data into 8 classes for
carrying out the process of multi-class classification.

IV. M ETHODOLOGY
The proposed framework is integration of a number of steps
right from data acquisition to evaluation of proposed machine
learning model.The section provides detailed description of
complete machine learning framework in terms of machine
learning pipeline utilised for the process of detecting anomaly
in IoT sensor environment. The section also discusses about
proposed stacked ensemble machine learning approach used
for the purpose of anomaly detection along with all the com-
ponents involved in stacked ensemble machine learning.The
section includes detailed description regarding Machine learn-
ing pipeline and Stacked ensemble machine learning along
with proper mathematical formulation.

A. Machine Learning pipeline

The framework developed for the process of anomaly Fig. 1: Machine Learning pipeline
detection in IoT sensor environment comprised of several
steps including data acquisition,data preprocessing,data trans- B. Proposed Stacked ensemble learning model
formation,training machine learning model and evaluating
machine learning model against various evaluation metrics.The The proposed stacked ensemble learning model was devel-
process of data acquisition comprises data acquired from oped using Random Forest,Artificial neural network and Sup-
several sensors present in IoT environment in variety of data port vector machines as base learner at level 0.Further,Logistic
types including nominal, discrete and continuous.Further,data Regression was used as meta-learner at level 1 for detection
is preprocessed using various data cleaning,visualisation and of anomaly with significant accuracy.
other preprocessing technique to handle problem of miss- Stacked ensemble machine learning approach works on
ing values and get maximum insights from the given data basis of aggregating weak learners to form powerful classi-
set.The preprocessed data was then transformed using label fication model to handle the problem of anomaly detection
encoding for converting label into feature vector.The proposed in IoT sensor environment.The proposed stacked ensemble
stacked ensemble learning method was applied to given data model consists of two levels namely level 0 and level 1.Level
set. The data set was split in ratio of 80:20 for training 0 comprises heterogeneous weak learners including support
and testing data set.Finally,the model made corresponding vector machine(SVM),artificial neural network(ANN) and ran-
predictions for detection of anomalies in IoT environment dom forest(RF),whereas level 1 has Logistic regression(LR)
and examined against various evaluation metrics including model as meta learning classifier.Given dataset D consists of
accuracy,precision,recall and F1 score.The proposed machine following samples. dj = (xj , yj ) where xj indicate anomaly

Authorized licensed use limited to: Vivekanand Education Society's Inst of Tech. Downloaded on September 23,2024 at 08:46:40 UTC from IEEE Xplore. Restrictions apply.
 detection features in given dataset, and yj generates corre- and M represents machine learning methods used as base
sponding prediction (normal or anomalous). j ∈ [1, M ], M learners
represents the total number of samples present in a given 3) Data obtained from level 1 was fitted on Logistic regres-
dataset. Base learners at level 0 include SVM, ANN, and RF sion model(meta-learner)
are denoted as Lt (t = 1, 2, 3). Initially, Dataset is separated 4) Finally,predictions were made using the meta learning
into two distinct subsets; one is used to train base learning based classifier on the test data set.
techniques to level-0 classifiers, while the other is used to Stacked ensemble machine learning model proved to be
make predictions. effective in comparison to other machine learning methods
in terms of performance and computational cost.
hjt = Lt (D − dj ) ∀j = 1, 2, · · · , M; ∀t = 1, 2, 3 (1)
V. R ESULT
Predictions (zit ) made by trained classifiers:
The proposed stacked ensemble learning method is evalu-
zjt = hjt (xj ) (2) ated on basis of accuracy,precision,recall and F1-score. Accu-
racy is a measure of correctly predicted samples by machine
A new dataset is created by combining the outputs of level-0 learning model to total number of samples.Accuracy helps to
classifiers with their real categorization. measure the performance of the model in terms of correctly
predicted samples.
D0 = ((zjt , zjt , · · · , zjt ) , yj ) (3)
T ruepositive + T ruenegative
After that, they’re fed to level 1 to teach the meta-learner. Accuracy = (5)
T otalsamples
As a result, LR can put together the classification results of
Precision is calculated by dividing the number of accurately
base learners to generate a final prediction for new cases:
predicted samples by the total number of true positive and
Yx = LR (g1 (x), g2 (x), g3 (x), g4 (x)) (4) false positive samples.
T ruepositive
Stacked ensemble machine learning model can be illustrated P recision = (6)
by the Fig 2. T ruepositive + F alsepositive
Recall is calculated by dividing the number of accurately
predicted samples by the total number of true positive and
false negative samples.Recall is also referred to as sensitivity.
T ruepositive
Recall = (7)
T ruepositive + F alsenegative
F1-score is a measure of harmonic mean derived from Pre-
cision and recall. F1 score takes into consideration Precision
and recall for estimation given machine learning model per-
formance.
2 ∗ P recision ∗ Recall
F 1Score = (8)
P recision + Recall
For proposed stacked ensemble learning method,evaluation
metrics based on precision, recall and f1-score is illustrated
by Table II.

TABLE II: Evaluation metrics for different types of attacks

precision recall f1-score
Fig. 2: Proposed Stacked ensemble learning model DoS attack(DoS) 0.98 0.65 0.78
Scan(SC) 1.00 1.00 1.00
The steps involved in process of detecting anomaly using Malicious Control(M.C) 1.00 1.00 1.00
stacked ensemble method are:- Malicious Operation(M.O) 1.00 1.00 1.00
1) Initially,all heterogeneous machine learning models were Spying(SP) 1.00 1.00 1.00
configured at level 0.Weak heterogeneous learners includ- Data Probing(D.P) 1.00 1.00 1.00
ing SVM,RF and ANN were trained using the training Wrong SetUp(W.S) 1.00 1.00 1.00
data set and K fold cross validation was performed at Normal(NL) 0.99 1.00 1.00
level 0 for making corresponding predictions
2) Heterogenous weak learners were able to generate a Receiver Operating Characteristic(ROC) helps to identify
matrix of N X M where N represents cross-validated the performance of a machine learning model.The aim of
predicted values from each of the machine learning model machine learning model is to maximize the true positive rate at

Authorized licensed use limited to: Vivekanand Education Society's Inst of Tech. Downloaded on September 23,2024 at 08:46:40 UTC from IEEE Xplore. Restrictions apply.
 (a) GaussianNB (b) SVM (c) ANN Classifier (d) LDA

(e) Random Forest (f) Logistic Regression (g) Decision Tree Classifier (h) Boosting
Classifier

(i) Bagging (j) Proposed Stacked

ensemble learning model
Fig. 3: Comparative analysis for various machine learning algorithms on basis of ROC Curve

the same time reducing false positive rate.ROC is characterized vides summary of prediction results for our problem of multi-
by true positive rate and false positive rate on y-axis and x- class classification.
axis respectively.ROC curve for a number of machine learning
algorithms right from Gaussian Naive Bayes to Proposed
Stacked ensemble learning method is illustrated by Fig 3.
A comparative analysis for various machine learning algo-
rithms against our proposed model on the basis of accuracy is
illustrated in Table III.
TABLE III: Comparative analysis for various machine learn-
ing algorithms against our proposed model on the basis of
accuracy
Traditional classifier Accuracy
GaussianNB 88.71
SVM 97.01
ANN 98.65
LDA 96.50
Random Forest Classifier 98.98
Logistic Regression 96.68
Decision Tree Classifier 97.98
Boosting 98.50 Fig. 4: Confusion Matrix
Bagging 98.10
Proposed model 99.86 VI. C ONCLUSION
Internet of things uses data-driven architectures for in-
Confusion matrix for the proposed stacked ensemble learn- terconnected devices.Also,complex data-driven architectures
ing method is illustrated by the Fig 4.Confusion matrix pro- have created new horizon for troublesome flow of information

Authorized licensed use limited to: Vivekanand Education Society's Inst of Tech. Downloaded on September 23,2024 at 08:46:40 UTC from IEEE Xplore. Restrictions apply.
 leading to multiple types of cybersecurity attacks between [10] E. Anthi, L. Williams, and P. Burnap, “Pulse: an adaptive intrusion
multiple devices across wide area network. In the current detection for the internet of things,” 2018.
[11] A. Ukil, S. Bandyoapdhyay, C. Puri, and A. Pal, “IoT healthcare
scenario, preserving IoT infrastructures from various types analytics: The importance of anomaly detection,” in 2016 IEEE 30th
of cybersecurity attacks is essential to ensure a safe and international conference on advanced information networking and ap-
secure virtual environment for end-user. Also, compromising plications (AINA). IEEE, 2016, pp. 994–997.
[12] H. H. Pajouh, R. Javidan, R. Khayami, A. Dehghantanha, and K.-
on services of IoT with restriction might lead to slow down K. R. Choo, “A two-layer dimension reduction and two-tier classification
in the development process and creating chaos in technical model for anomaly-based intrusion detection in IoT backbone networks,”
society. For dealing with the problem of attack and anomaly IEEE Transactions on Emerging Topics in Computing, vol. 7, no. 2, pp.
314–323, 2016.
detection, in case of IoT networks, applying machine learning [13] G. D’angelo, F. Palmieri, M. Ficco, and S. Rampone, “An uncertainty-
approach for detecting known and unknown IoT attacks. Var- managing batch relevance-based approach to network anomaly detec-
ious machine learning methods have been used for detection tion,” Applied Soft Computing, vol. 36, pp. 408–418, 2015.
[14] R. Kozik, M. Choraś, M. Ficco, and F. Palmieri, “A scalable distributed
of majorly seven types of attacks and classify them in order machine learning approach for attack detection in edge computing
to avoid any damage to end-user. The paper proves that environments,” Journal of Parallel and Distributed Computing, vol. 119,
stacked ensemble machine learning could be effective in case pp. 18–26, 2018.
[15] E. Hodo, X. Bellekens, A. Hamilton, P.-L. Dubouilh, E. Iorkyase,
of dealing with network traffic traces.The paper uses DS2OS C. Tachtatzis, and R. Atkinson, “Threat analysis of IoT networks
traffic traces data set for the purpose of study. The proposed using artificial neural network intrusion detection system,” in 2016
stacked ensemble machine learning proved to be superior International Symposium on Networks, Computers and Communications
(ISNCC). IEEE, 2016, pp. 1–6.
compared to other traditional machine learning algorithms [16] M. Nobakht, V. Sivaraman, and R. Boreli, “A host-based intrusion detec-
for dealing with the problem of cyber attacks on IoT sensor tion and mitigation framework for smart home IoT using OpenFlow,”
networks. The future perspective of work would be applying in 2016 11th International conference on availability, reliability and
security (ARES). IEEE, 2016, pp. 147–156.
stacked ensemble machine learning to real world settings as [17] K. Alrawashdeh and C. Purdy, “Toward an online anomaly intrusion
given work was performed in virtual IoT environment.Also, detection system based on deep learning,” in 2016 15th IEEE inter-
anomalous data might raise some difficulties in case of real- national conference on machine learning and applications (ICMLA).
IEEE, 2016, pp. 195–200.
time data. Further,micro-services may behave distantly and [18] M. Hasan, M. M. Islam, M. I. I. Zarif, and M. M. A. Hashem, “Attack
cause deviation from normal behavior for certain services and anomaly detection in IoT sensors in IoT sites using machine learning
resulting in anomalies. The paper contributes to the field of approaches,” Internet of Things, vol. 7, p. 100059, 2019.
[19] D. K. Reddy, H. S. Behera, J. Nayak, P. Vijayakumar, B. Naik, and
IoT by leveraging the data-driven infrastructure with effective P. K. Singh, “Deep neural network based anomaly detection in internet
integration of machine learning methods that would be helpful of things network traffic tracking for the applications of future smart
in advancements of decision making in creating new horizons cities,” Transactions on Emerging Telecommunications Technologies, p.
e4121, 2020.
of data-driven architectures helpful in building secure and safe [20] “IoT traffic traces gathered in a the ds2os iot environment,” https://www.
IoT systems. kaggle.com/francoisxa/ds2ostraffictraces, accessed: 2021-04-01.

R EFERENCES
[1] X. Xiaohui, “Study on security problems and key technologies of the
internet of things,” in 2013 International conference on computational
and information sciences. IEEE, 2013, pp. 407–410.
[2] K. Zhao and L. Ge, “A survey on the internet of things security,” in
2013 Ninth international conference on computational intelligence and
security. IEEE, 2013, pp. 663–667.
[3] V. L. Thing, “Ieee 802.11 network anomaly detection and attack classi-
fication: A deep learning approach,” in 2017 IEEE Wireless Communi-
cations and Networking Conference (WCNC). IEEE, 2017, pp. 1–6.
[4] M.-O. Pahl and F.-X. Aubet, “All eyes on you: Distributed Multi-
Dimensional IoT microservice anomaly detection,” in 2018 14th In-
ternational Conference on Network and Service Management (CNSM).
IEEE, 2018, pp. 72–80.
[5] C. C. Aggarwal, S. Y. Philip, J. Han, and J. Wang, “A framework
for clustering evolving data streams,” in Proceedings 2003 VLDB
conference. Elsevier, 2003, pp. 81–92.
[6] E. Gelenbe and Y. Yin, “Deep learning with dense random neural
networks,” in International Conference on Man–Machine Interactions.
Springer, 2017, pp. 3–18.
[7] X. Liu, Y. Liu, A. Liu, and L. T. Yang, “Defending ON–OFF attacks
using light probing messages in smart sensors for industrial communi-
cation systems,” IEEE Transactions on Industrial Informatics, vol. 14,
no. 9, pp. 3801–3811, 2018.
[8] A. A. Diro and N. Chilamkurti, “Distributed attack detection scheme
using deep learning approach for Internet of Things,” Future Generation
Computer Systems, vol. 82, pp. 761–768, 2018.
[9] B. Usmonov, O. Evsutin, A. Iskhakov, A. Shelupanov, A. Iskhakova, and
R. Meshcheryakov, “The cybersecurity in development of IoT embedded
technologies,” in 2017 International Conference on Information Science
and Communications Technologies (ICISCT). IEEE, 2017, pp. 1–4.

Authorized licensed use limited to: Vivekanand Education Society's Inst of Tech. Downloaded on September 23,2024 at 08:46:40 UTC from IEEE Xplore. Restrictions apply.