1

Intrusion Detection System (IDS) using Various

Machine Learning Methods
Ratul Chowdhury1, Sayan Adhikary2, Subhajit Garai3, Soumyadip Roy4, Gourab Mukharjee5
1Assistant Professor, Department of Computer Science, Future Institute of Engineering and
Management, Sonarpur, West Bengal, India.
2-5BTech student, Dept. of Computer science engineering, Future institute of engineering and
management, Sonarpur, Kolkata, India.

---------------------------------------------------------------**-------------------------------------------------------------------

1. Abstract
In contemporary network environments, ensuring robust security against
malicious attacks poses significant challenges to Intrusion Detection
Systems (IDS). Despite their critical role in safeguarding network
integrity, IDS often encounter performance degradation due to evolving
attack methodologies and increasing network complexities. To mitigate
these challenges and bolster network security, innovative approaches
are required.
This study explores novel techniques for enhancing IDS performance by
leveraging advanced Machine Learning technologies. Through
meticulous preprocessing of an extensive Internet of Things (IoT)
dataset encompassing both normal network traffic and various anomaly
attack types, features directly correlated with the target column, denoting
different attack categories, are identified. The emphasis is placed on
discerning dependent features crucial for accurate intrusion detection.
Subsequently, diverse classification algorithms are employed to evaluate
their efficacy in accurately identifying and classifying network intrusions
based on the identified features. By comparing the performance of
different classifiers, the study aims to ascertain the most suitable
algorithm for robust and efficient intrusion detection in modern networks.
Furthermore, the study elucidates the trans formative potential of
advanced technologies in fortifying network security amidst evolving
Cyber threats. By harnessing innovative approaches, such as Machine
 2

Learning algorithms, IDS can adapt dynamically to emerging attack

vectors, thereby mitigating performance challenges and effectively
safeguarding critical digital infrastructures. Through continuous
refinement and adaptation, these advanced intrusion detection
methodologies instill confidence among network stakeholders, fostering
trust in the reliability and security of digital networks.

2. Introduction
The proliferation of interconnected devices and digital infrastructures has
heightened the importance of network security in safeguarding sensitive
information, critical assets, and organizational operations. As cyber
threats continue to evolve in sophistication and complexity, organizations
are faced with the formidable task of fortifying their networks against a
myriad of potential vulnerabilities and attacks. In this context, the role of
network security technologies, including Intrusion Detection Systems
(IDS), is paramount in mitigating risks, detecting anomalies, and
preserving the integrity of digital ecosystems. This section delves into
the multifaceted landscape of network security, exploring key challenges,
strategies, and technologies aimed at fortifying network defenses and
mitigating Cyber threats.
Challenges in Network Security:
Ensuring robust network security poses formidable challenges for
organizations in the face of evolving Cyber threats, burgeoning network
complexities, and the proliferation of interconnected devices. One of the
primary challenges lies in the dynamic nature of Cyber threats, which
continuously adapt and evolve in response to advancements in
technology and security measures. From sophisticated malware attacks
to stealthy infiltration attempts, organizations must contend with a
diverse array of threats that target vulnerabilities in network
infrastructure, software applications, and user endpoints.
 3

Moreover, the sheer scale and complexity of modern networks

exacerbate the challenge of network security, as organizations grapple
with managing vast networks comprising diverse devices, systems, and
applications. The proliferation of IoT devices, cloud services, and remote
work environments further complicates the network security landscape,
introducing additional attack vectors and potential points of vulnerability.
Additionally, the rise of insider threats, compliance requirements, and
regulatory mandates imposes additional burdens on organizations
seeking to fortify their network defenses and safeguard sensitive data.

Strategies for Enhancing Network Security:

In response to the evolving threat landscape, organizations must adopt a
proactive and multifaceted approach to network security, encompassing
a range of strategies and best practices. One key strategy is the
implementation of robust access controls and authentication
mechanisms to prevent unauthorized access to network resources and
sensitive data. By enforcing strong password policies, implementing
multi-factor authentication, and segmenting network access based on
user roles and privileges, organizations can significantly reduce the risk
of unauthorized breaches and insider threats.
Furthermore, the deployment of Intrusion Detection Systems (IDS) plays
a pivotal role in enhancing network security by continuously monitoring
network traffic for signs of malicious activity, policy violations, and
anomalous behavior. By leveraging advanced detection algorithms,
anomaly-based analysis, and real-time alerts, IDS systems enable
organizations to promptly detect and mitigate security incidents before
they escalate into full-fledged breaches. Additionally, organizations can
complement IDS deployments with Intrusion Prevention Systems (IPS)
that actively block and mitigate identified threats in real-time, thereby
bolstering network defenses and thwarting potential attacks.
 4

Technologies for Network Security:

In the realm of network security technologies, advancements in machine
learning, artificial intelligence, and behavioral analytics hold significant
promise for enhancing threat detection capabilities and bolstering
network resilience. Machine learning algorithms, for instance, can
analyze vast datasets of network traffic patterns to identify anomalous
behavior indicative of potential security threats. Similarly, artificial
intelligence-powered security solutions can autonomously detect,
analyze, and respond to emerging cyber threats in real-time, augmenting
the capabilities of human security teams and enabling rapid threat
mitigation.
Moreover, emerging technologies such as Software-Defined Networking
(SDN) and Zero Trust Architecture (ZTA) offer novel approaches to
network security by enabling granular control, visibility, and
segmentation of network traffic. SDN architectures decouple network
control and data forwarding functions, allowing for centralized
management and dynamic enforcement of security policies across
distributed network environments. Similarly, ZTA frameworks adopt a
least-privilege access model, whereby access controls are based on
continuous verification of user identity, device posture, and contextual
factors, thereby minimizing the attack surface and reducing the risk of
unauthorized access.

Network security remains a paramount concern for organizations

seeking to safeguard their digital assets, preserve the integrity of their
operations, and maintain stakeholder trust. By adopting a proactive and
5

holistic approach to network security, encompassing robust access

controls, advanced detection mechanisms, and emerging technologies,
organizations can fortify their defenses against evolving cyber threats
and mitigate the risks posed by malicious actors. Moreover, the
continued evolution of network security technologies, coupled with
ongoing collaboration and knowledge sharing within the Cybersecurity
community, will play a pivotal role in shaping the future of network
security and ensuring the resilience of digital ecosystems in the face of
emerging threats.

3. Dataset Description
Overview: The IoT Network Intrusion Dataset is a collection of network
traffic data captured from Internet of Things (IoT) devices in a simulated
environment. This dataset is intended for research and analysis of
network security, particularly in IoT ecosystem. This dataset can be used
for free for academic projects.

Content:
● The dataset contains network traffic logs from various IoT devices
like smart home devices, Wi-Fi camera, smartphones, laptops,
tablets, etc. connected to a smart home router.
● Each record in the dataset contains a network transaction,
including source and destination IP addresses, source and
destination port numbers, protocol, timestamp, no. of packets sent,
etc.
● Attributes include sent packet length, maximum packet length,
minimum packet length, etc.
Data Format:
6

● File Format: CSV (Comma-Separated Values)

● Each row represents a single network transaction, with columns
representing different attributes.

Dataset Size: IoT Network Intrusion Dataset.csv contains 625783 rows

and 86 columns.
Data Type: The dataset can be categorized into two types of data,
namely,
● Normal packets: These are normal packets which are received by
the router.
● Anomaly packets: These are malicious packets which are
received by the router. In this dataset, there are primarily four
types of attack packets,
o DoS
o Mirai
o MITM
o Scan

Binary Category Sub-Category

Normal Normal Normal
Anomaly Dos Syn Flooding
Mirai Brute Force, HTTP Flooding, UDP Flooding, ACK
MITM Flooding
Scan ARP Spoofing
Host Port, OS
Binary, Category and Sub-Category of IoT Network Intrusion Dataset.csv
The number of instances for each type are as follows:

Binary Label Distribution

Normal 40073
Anomaly 585710
7

Category Label Distribution

Type Instances
Normal 400073
DoS 59391
Mirai 415677
MITM 35377
Scan 75265

Sub-Category Label Distribution

Type Instances
Normal 400073
DoS 59391
Mirai Ack Flooding 55124
Mirai Brute Force 121181
Mirai HTTP Flooding 55818
Mirai UDP Flooding 183554
MITM 35377
Scan Host Port 22192
Scan Port OS 53073

Link to Dataset:
https://sites.google.com/view/iot-network-intrusion-dataset/home

4. Literacy Survey
In recent years, the increasing complexity and diversity of cyber threats
have made intrusion detection a critical component of cybersecurity.
 8

Traditional intrusion detection systems (IDS) often rely on supervised

learning algorithms, which require labelled training data to distinguish
between normal and malicious activities. However, the dynamic nature of
cyber-attacks and the emergence of zero-day exploits pose challenges
for supervised approaches. Unsupervised IDS methods, which do not
require labelled data for training, offer a promising alternative. This
literature survey explores the state-of-the-art in unsupervised intrusion
detection, including techniques, challenges, and future directions.

4.1 Fundamental Concepts

Unsupervised intrusion detection relies on techniques such as clustering,
anomaly detection, and density estimation. Clustering algorithms group
similar data points based on feature similarity, enabling the identification
of patterns in network traffic or system behaviour. Anomaly detection
methods, on the other hand, focus on detecting deviations from normal
behaviour without prior knowledge of attack signatures. Density
estimation techniques model the underlying distribution of data points to
identify outliers or anomalies.

4.2 Review of Unsupervised IDS Techniques

Unsupervised IDS techniques encompass a range of methodologies and
algorithms designed to detect anomalies, identify patterns, and cluster
similar data points without relying on labelled training data. These
techniques leverage machine learning algorithms, statistical methods,
and data mining approaches to analyze network traffic, system logs, or
other data sources for signs of intrusion.
Anomaly Detection Techniques:
Anomaly detection is a central aspect of unsupervised IDS, focusing on
identifying deviations from normal behaviour that may indicate the
 9

presence of security breaches. Various anomaly detection techniques

are employed in unsupervised IDS, including:
● Statistical Approaches: Statistical methods analyse the distribution
of features in the data and identify instances that significantly
deviate from the expected norms. These methods often include
measures such as z-score analysis, which quantifies the deviation
of data points from the mean.
● Machine Learning Algorithms: Machine learning-based anomaly
detection techniques learn patterns of normal behavior from
unlabeled data and identify instances that deviate from these
patterns as anomalies. Algorithms such as Isolation Forest,
One-Class Support Vector Machines (SVM), and autoencoders are
commonly used for anomaly detection in unsupervised IDS.
● Ensemble Methods: Ensemble methods combine multiple anomaly
detection models to improve detection accuracy and robustness.
Techniques such as bagging, boosting, and model averaging are
applied to combine the outputs of individual models and reduce the
impact of false positives.
Clustering Algorithms:
Clustering algorithms group similar data points together based on
feature similarity, enabling the identification of patterns or clusters in the
data. Clustering is a fundamental technique in unsupervised IDS,
allowing for the grouping of network traffic or system events into clusters
that may represent normal or anomalous behaviour. Common clustering
algorithms used in unsupervised IDS include:
● K-means: K-means is a partitioning algorithm that divides the data
into K clusters by minimizing the sum of squared distances
between data points and cluster centroids. It is simple, scalable,
and widely used for clustering network traffic or system logs in
unsupervised IDS.
 10

● Hierarchical Clustering: Hierarchical clustering builds a tree-like

hierarchy of clusters by recursively merging or splitting clusters
based on their proximity. It does not require the pre-specification of
the number of clusters and can capture hierarchical relationships
in the data, making it suitable for analyzing complex network
structures or system dependencies.
● Density-Based Clustering: Density-based clustering algorithms,
such as DBSCAN (Density-Based Spatial Clustering of
Applications with Noise), identify clusters based on density
connectivity, grouping data points that are closely packed together.
These algorithms are robust to noise and can identify clusters of
arbitrary shapes, making them suitable for detecting anomalies in
complex network environments.
Density Estimation Techniques:
Density estimation methods model the underlying probability distribution
of data points in the feature space, enabling the identification of regions
of low density or outliers. Density estimation is commonly used in
unsupervised IDS to model the normal behaviour of network traffic or
system events and detect anomalies based on deviations from this
model. Common density estimation techniques used in unsupervised
IDS include:
● Gaussian Mixture Models (GMMs): GMMs represent the probability
distribution of data points as a mixture of Gaussian distributions.
They can capture complex patterns in the data and are well-suited
for modelling multimodal distributions, making them suitable for
detecting anomalies in diverse network environments.
● Kernel Density Estimation (KDE): KDE estimates the probability
density function of data points by placing a kernel function at each
data point and summing the contributions to obtain the overall
density estimate. It provides a non-parametric approach to density
estimation and can adapt to the local structure of the data, making
11

it suitable for detecting anomalies in data with complex

distributions.
● By leveraging these techniques, unsupervised IDS can effectively
detect both known and unknown threats in real-time network
environments, providing organizations with enhanced security
against cyber-attacks and unauthorized access.

4.3 Evaluation Metrics and Datasets

Evaluation of unsupervised IDS methods requires appropriate metrics
and datasets. Commonly used metrics include detection rate, false
positive rate, precision, and recall. Datasets such as KDD Cup 1999,
NSL-KDD, and UNSW-NB15 provide standardized benchmarks for
assessing the performance of intrusion detection algorithms. However, it
is essential to ensure that evaluation metrics and datasets accurately
reflect real-world scenarios and capture the complexity of modern cyber
threats.

4.4 Challenges and Limitations

Unsupervised IDS methods face several challenges and limitations that
impact their effectiveness in real-world scenarios. These include:
● Difficulty in distinguishing between normal and abnormal
behaviour, leading to high false positive rates.
● Limited interpretability of IDS outputs, making it challenging to
understand the underlying reasons for detected anomalies.
● Scalability issues, particularly in high-volume network
environments where real-time processing of data is required.
● Lack of domain expertise among users, hindering the effective
deployment and maintenance of unsupervised IDS solutions.
12

Addressing these challenges requires ongoing research and

development efforts aimed at enhancing the robustness, scalability, and
interpretability of unsupervised IDS methods.

4.5 Future Directions and Research Challenges

Future research directions in unsupervised IDS include:
● Development of hybrid models combining supervised and
unsupervised techniques to improve detection accuracy and
reduce false positives.
● Integration of contextual information, such as network topology
and user behaviour, to enhance the contextual understanding of
detected anomalies.
● Exploration of deep learning approaches for anomaly detection,
leveraging the expressive power of neural networks to capture
complex patterns in system behaviour.
Addressing scalability challenges using distributed computing techniques
and efficient algorithms for real-time processing of large-scale data
streams.
Research efforts in these areas aim to advance the state-of-the-art in
unsupervised intrusion detection and better equip organizations to
defend against evolving cyber threats.

5. Machine Learning Models

5.1 Isolation Forest: -

Isolation Forest represents an ensemble method, akin to Random
Forest, primarily utilized for outlier detection. Its foundational concept
revolves around the notion that anomalies within data are sparse, hence
they should be readily distinguishable and isolatable compared to normal
 13

data points. The Isolation Forest algorithm amalgamates the predictions

of multiple decision trees by computing the average when assigning the
final anomaly score to a given data point. Unlike conventional anomaly
detection algorithms, which typically establish a definition of "normalcy"
and subsequently identify anything deviating from it as anomalous,
Isolation Forest undertakes to isolate anomalous data points from the
outset.
The Isolation Forest procedure comprises two sequential steps. Firstly,
to instantiate isolated trees (referred to as iTrees), a training dataset is
employed. Subsequently, each sample within the dataset is traversed
through the iTrees generated in the preceding step, culminating in the
assignment of an appropriate anomaly score to the sample.

The Mathematical formula for isolation forest is

S(x,n)=2-(E(h(x))/c(n)
h(x): path length of observation x
c(n): average path length of failed search of binary search tree and
number of external nodes.

5.2 Mutual_info_classif:-
Mutual information classification, often abbreviated as
mutual_info_classif, stands out as a feature selection technique utilized
to discern crucial features within a given feature set. The process of
selecting key features holds paramount importance as it contributes to
performance enhancement through noise reduction and the alleviation of
overfitting concerns. Additionally, it aids in improving interpretability by
concentrating on key factors pivotal to understanding underlying data
patterns.
 14

Mathematically, the mutual information (MI) between random variables X

and Y is articulated as follows:
I(X; Y) = ∑ x∈X∑y∈Y P(x,y)log[p(x)⋅p(y)/p(x,y)]

Here:
p(x,y)is the joint probability distribution function of X and Y
p(x) and p(y) are the marginal probability distribution functions of X and
Y.

The mutual_info_classif algorithm computes the mutual information

between each feature and the target class grounded on the observed
frequencies of feature-class pairs within the dataset. Through estimation
of probability distributions and subsequent calculation of mutual
information using the aforementioned formula, this approach furnishes
insights into the relevance of each feature vis-à-vis the classification
task.
5.3 K Means Clustering: -
K-Means Clustering represents an Unsupervised Machine Learning
algorithm that assign datapoint into distinct clusters. Initially, K-means
randomly places the centroids within the feature space. Subsequently,
each data point is assigned to the cluster whose centroid is closest in
terms of Euclidean distance. After assign datapoint to a particular cluster
the centroid of that cluster is updated. The K-means algorithm employs
the Euclidean Distance metric to determine the similarity or dissimilarity
between data points and cluster centroids. The mathematical formulation
for Euclidean distance between two points (x1, y1) and (x2, y2) is

Distance=√ [(x2 – x1)2 + (y2 – y1)2]

 15

In scenarios where the number of clusters is not predefined, the Elbow

method is commonly employed to ascertain the optimal number of
clusters. This method involves plotting the within-cluster sum of squares
(WCSS) against the number of clusters and identifying the point of
inflection or "elbow," which signifies an optimal trade-off between the
number of clusters and the compactness of the clusters.

5.4 XG Boost: -
XG Boost, short for "Extreme Gradient Boosting," represents a highly
optimized distributed gradient boosting library meticulously crafted for
the efficient and scalable training of machine learning models. This
method, an ensemble learning technique, amalgamates the predictions
of multiple weak models to generate a robust and accurate prediction.
XGBoost has garnered widespread popularity and acclaim in the
machine learning community owing to its prowess in handling large
datasets and its capacity to achieve state-of-the-art performance across
various tasks like classification and regression. Notably, XGBoost
distinguishes itself with its adeptness in managing missing values,
obviating the need for extensive preprocessing of real-world data.
Furthermore, XGBoost incorporates built-in support for parallel
processing, facilitating the training of models on extensive datasets
within reasonable time frames.
Mathematical Formula of XG Boost is
i) Training Dataset {(xi,yi)}i=1n where xi represent the feature and yi
represent the target variable.
ii) Objective function L(y/,y) to minimize, typically a differentiable loss
function such as squared error or log loss.
 16

XGBoost optimizes the objective function through techniques like

gradient descent or Newton's method, endeavouring to identify the
optimal parameters for the weak learners. This meticulous optimization
process ultimately yields a potent and precise predictive model.

5.5 Cat Boost: -

Cat Boost, short for Categorical Boosting, is an open-source boosting
library developed by Yandex. It serves as a powerful tool for regression
and classification tasks, particularly when dealing with datasets featuring
a large number of independent features. Unlike traditional gradient
boosting methods, Cat Boost excels in handling both categorical and
numerical features without necessitating explicit feature encoding
techniques like One-Hot Encoding or Label Encoding. One of Cat
Boost’s standout features is its utilization of the Symmetric Weighted
Quantile Sketch (SWQS) algorithm. This innovative approach effectively
handles missing values within the dataset, mitigating the risk of
overfitting and enhancing overall model performance. By automatically
managing missing data, Cat Boost streamlines the modelling process
and simplifies the preprocessing pipeline. In essence, Cat Boost
operates similarly to other gradient boosting algorithms by minimizing a
differentiable loss function L(y/,y) with respect to the predictions y/ .
However, its distinctive capability to handle categorical features
seamlessly, along with its advanced handling of missing values through
SWQS, sets Cat Boost apart as a versatile and efficient tool for a wide
range of machine learning tasks.

6. Proposed Model
Our proposed model combines unsupervised clustering and boosting
algorithms to enhance intrusion detection in network systems. The
foundation of our methodology relies on the IOT intrusion dataset, a
 17

comprehensive collection comprising 625,783 rows and 86 columns,

meticulously crafted to encapsulate diverse network behaviours and
potential intrusions. To ensure data integrity, missing values are filled
with column means, preserving essential information for subsequent
analysis.
Outliers, characterized by their significant deviation from the dataset's
norm, can emerge due to various factors, such as measurement errors,
data corruption, or rare occurrences. In the realm of intrusion detection
within network systems, outliers may signal abnormal network
behaviours or anomalous activities diverging from typical patterns.
Detecting and eliminating outliers is paramount for enhancing model
performance and safeguarding the accuracy and dependability of
intrusion detection systems.
These outliers can detrimentally impact model performance in multiple
ways. Firstly, they have the potential to distort statistical analyses and
machine learning algorithms, resulting in biased estimations and
inaccurate predictions. By influencing the mean and standard deviation
of the data, outliers can disrupt the distribution and skew the outcomes
of clustering or classification algorithms. Secondly, outliers can augment
the variance of the model, rendering it less resilient and more prone to
overfitting. Overemphasis on extreme data points may cause the model
to overlook underlying patterns and struggle to generalize effectively to
unseen data.
In the context of intrusion detection, outliers may signify irregular
network activities or malicious behaviours that deviate from normal traffic
patterns. Their removal facilitates the model's capacity to distinguish
between ordinary network behaviours and potential intrusions, thereby
enhancing its detection precision and reliability. Furthermore, outliers
may inject noise and extraneous information into the model, impeding its
efficacy in identifying authentic threats.
 18

To tackle these challenges, we employ the isolation forest algorithm,

renowned for its efficacy in identifying outliers in high-dimensional
datasets. This algorithm operates by recursively partitioning the dataset
into subsets and isolating outliers in fewer partitions, simplifying their
identification. By eliminating outliers from the dataset, we bolster the
model's performance and efficiency, ensuring more precise and
dependable intrusion detection in network systems.
Feature selection serves as a critical component of our model's efficacy,
playing a pivotal role in enhancing its performance and mitigating the
risks of overfitting. In the realm of machine learning and data analysis,
feature selection refers to the process of identifying and selecting the
most relevant features from a dataset, thereby reducing dimensionality
and improving the model's predictive power. In the context of intrusion
detection in network systems, where datasets often comprise numerous
attributes and variables, feature selection becomes imperative to
streamline the analysis and focus on the most discriminative features.

By leveraging mutual class information, our model identifies the top 10

features that are most pertinent to the output column, which denotes
different intrusion types. This strategic selection is guided by the
principle of maximizing the discriminative power of the selected features
while minimizing the risk of overfitting. Overfitting occurs when a model
learns to capture noise or irrelevant patterns in the data, leading to poor
generalization performance on unseen data. By focusing on the most
informative features, our model avoids the pitfalls of overfitting and
ensures robust performance in detecting network intrusions across
diverse scenarios.
The importance of feature selection in our model cannot be overstated.
Firstly, feature selection helps in reducing the computational complexity
and resource requirements of the model, as it focuses on a subset of
relevant features rather than the entire dataset. This not only enhances
 19

computational efficiency but also facilitates faster model training and

inference. Secondly, feature selection improves the interpretability and
comprehensibility of the model by identifying the most salient features
that contribute to the classification of network intrusions. This enables
security analysts and domain experts to gain insights into the underlying
factors driving intrusion detection and aids in decision-making
processes.
Moreover, feature selection plays a crucial role in addressing the curse
of dimensionality, particularly in high-dimensional datasets commonly
encountered in network security applications. By eliminating irrelevant or
redundant features, feature selection reduces the risk of model
overfitting and improves the model's ability to generalize to unseen data.
Additionally, feature selection enhances the model's sensitivity to subtle
patterns and anomalies in the data, thereby improving its overall
performance in detecting network intrusions with high accuracy and
reliability.
At the heart of our model lies the sophisticated integration of k-means
clustering, a widely acclaimed technique renowned for its versatility and
efficacy in partitioning datasets into cohesive clusters based on their
similarities. This methodology operates iteratively, assigning data points
to the nearest cluster centroid while refining cluster centers to minimize
the within-cluster variance. Such iterative refinement leads to
well-defined clusters that encapsulate data points exhibiting similar
characteristics, thereby facilitating the identification of underlying
patterns and structures within the dataset.
In the domain of intrusion detection within network systems, the
application of k-means clustering proves to be particularly invaluable.
Leveraging the rich and diverse IoT dataset at our disposal, our model
leverages k-means clustering to segment network packets into two
primary clusters: potential attacks and non-attack packets. By grouping
network packets based on shared attributes such as traffic patterns,
 20

packet size, and communication protocols, k-means clustering enables

the identification of clusters comprising data points exhibiting anomalous
behaviours, thus enabling the early detection of potential security
threats.

An inherent advantage of k-means clustering lies in its ability to operate

in an unsupervised manner, meaning it does not necessitate labelled
data for training. This aspect is of paramount importance in the context
of intrusion detection, where access to labelled datasets containing
instances of known attacks may be scarce or unavailable. By harnessing
the power of unsupervised clustering, our model can autonomously
identify and classify network traffic patterns without the need for explicit
labels, thereby enhancing its adaptability and scalability across diverse
network environments.

Moreover, the integration of k-means clustering serves as the

foundational pillar of our intrusion detection system, providing a robust
framework for subsequent analysis and decision-making. By segregating
network packets into distinct clusters representing potential attacks and
normal behaviors, our model is adept at swiftly identifying anomalous
network activities and potential security threats. This proactive approach
enables network administrators to respond promptly to emerging threats,
thereby mitigating risks and safeguarding network integrity.

Following the identification of the cluster containing potential attacks by

the k-means algorithm, the data is then forwarded to the boosting
algorithm for further analysis. Boosting algorithms such as Cat Boost or
Boost meticulously scrutinize the characteristics of each intrusion type
within the identified cluster, thereby enhancing the model's accuracy and
reliability in detecting and classifying network intrusions. This synergistic
 21

blend of k-means clustering and boosting algorithms empowers our

model to achieve heightened performance in identifying and addressing
network security threats, thereby bolstering the resilience of network
systems against evolving cyber threats.
Once potential attacks are identified by our K-Means Clustering model,
we seamlessly transition to the utilization of boosting algorithms. These
algorithms, including renowned classifiers such as CatBoost and
XGBoost, play a pivotal role in dissecting the characteristics of each
intrusion type identified within the dataset. This multi-stage approach is
instrumental in enhancing the efficacy of our system, as it enables us to
delve deeper into the intricacies of network intrusions.

Boosting algorithms are adept at refining the detection process by

leveraging the strengths of multiple weak learners to create a robust and
accurate classification model. By employing various classifiers, our
system can effectively analyse the subtle nuances and patterns
associated with different types of network intrusions. This
comprehensive analysis not only enhances our ability to detect threats
but also provides valuable insights into the underlying mechanisms
driving these intrusions.

Furthermore, the multi-stage approach adopted by our model empowers

us to not only identify potential attacks but also gain a deeper
understanding of their characteristics and behaviour. By dissecting the
features unique to each intrusion type, we can discern patterns, trends,
and anomalies that may otherwise go unnoticed. This nuanced
understanding allows us to develop more effective strategies for
mitigating risks and fortifying network security defences.
22

To ensure the reliability and robustness of our model, we subject it to

rigorous experimental evaluation. This evaluation encompasses a range
of key metrics, including detection rates, false positives, and
classification accuracies. Through meticulous testing across diverse
scenarios and datasets, we validate the effectiveness of our approach in
accurately identifying and classifying network intrusions.

The integration of boosting algorithms into our intrusion detection system

represents a significant advancement in network security. By leveraging
these algorithms, we not only enhance our ability to detect threats but
also gain valuable insights into the underlying dynamics of network
intrusions. This holistic approach enables us to develop more effective
strategies for mitigating risks and safeguarding network integrity in an
ever-evolving threat landscape.

In conclusion, our proposed model represents a groundbreaking

advancement in the field of intrusion detection, offering a holistic and
highly effective approach to fortify network security infrastructure. By
seamlessly integrating cutting-edge techniques such as unsupervised
clustering and boosting algorithms within a unified framework, our model
stands at the forefront of combating the ever-evolving landscape of
cyber threats.

At its core, our model harnesses the power of unsupervised clustering to

autonomously identify and analyse intricate patterns within network data.
This innovative approach enables our system to swiftly detect potential
 23

intrusions without relying on predefined labels, thereby enhancing its

adaptability and scalability across diverse network environments. By
uncovering anomalies and irregularities in network traffic, our model
serves as a proactive defence mechanism, enabling network
administrators to pre-emptively mitigate emerging threats before they
escalate.

Moreover, the incorporation of boosting algorithms further enhances the

efficacy of our model by providing a refined analysis of identified
intrusions. Through the utilization of sophisticated classifiers like CAT
Boost and XGBoost, our system meticulously dissects the characteristics
of network intrusions, enabling precise classification and categorization.
This in-depth analysis not only aids in accurately identifying malicious
activities but also provides invaluable insights into the underlying
mechanisms driving cyber-attacks, empowering organizations to develop
more robust security strategies.

Furthermore, by leveraging a unified framework that seamlessly

integrates unsupervised clustering and boosting algorithms, our model
offers a comprehensive solution to fortify network security infrastructure.
This integrated approach ensures the resilience and integrity of network
systems, even in the face of sophisticated and rapidly evolving cyber
threats. By proactively identifying and mitigating potential intrusions, our
model helps organizations stay one step ahead of cyber adversaries,
safeguarding critical assets and preserving the confidentiality and
integrity of sensitive data.
24

In essence, our proposed model represents a paradigm shift in intrusion

detection, offering a proactive and multifaceted approach to network
security. By leveraging advanced techniques within a unified framework,
our model not only fortifies network security infrastructure but also
empowers organizations to adapt and respond effectively to the dynamic
threat landscape. As cyber threats continue to evolve, our model stands
as a beacon of innovation and resilience, ensuring the continued
protection of network systems against emerging security challenges.

7. Performance Measurement Tools

25

Accuracy:
In machine learning, the accuracy score is calculated using a confusion
matrix. A confusion matrix is a table that provides a summary of a
classification model's performance by comparing the predicted and
actual output values. The matrix has rows that represent the predicted
output and columns that represent the actual output. The values in the
diagonal of the matrix represent the correctly classified instances, while
the values outside the diagonal represent the incorrectly classified
instances. By analysing the confusion matrix, one can determine the
model's accuracy, precision, recall, and other performance metrics. The
accuracy score is the percentage of correctly predicted output labels out
of the total number of output labels. It is one of the most commonly used
metrics to evaluate a classification model's performance.

Confusion Matrix Sample: -

The confusion matrix includes:

● True Positive (TP): the number of true positive predictions.
● False Positive (FP): the number of false positive predictions.
● True Negative (TN): the number of true negative predictions.
● False Negative (FN): the number of false negative predictions.
 26

Using these values, we can find the accuracy using the following
formula:
Accuracy = (TP + TN) / (TP + TN + FP + FN)

Precision: -
Precision is a fundamental performance metric that evaluates the ratio of
correctly predicted positive instances to the total number of positive
predictions made by a classification model. It is a crucial metric in the
context of the confusion matrix. The precision score can be computed
using the following formula:
Precision = True Positives / (True Positives + False Positives)
Here, True Positives refer to the number of correctly predicted positive
instances, and False Positives refer to the number of negative instances
that were incorrectly classified as positive by the model. By utilizing this
formula, one can easily calculate the precision score of a classification
model.

Recall: -
Recall is a metric used in performance evaluation that measures the
proportion of true positives to the total number of actual positive cases in
the dataset, within the confusion matrix. It is also known as sensitivity or
the true positive rate.
Recall can be calculated using the following equation:
Recall = True Positives / (True Positives + False Negatives)
To clarify, true positives are the cases where the model correctly
predicted the positive class, while false negatives are the cases where
the model incorrectly predicted the negative class.

F1 Score: -
The F1 score is a performance metric that combines precision and recall
to provide an overall measure of a classification model's performance. It
is the harmonic mean of precision and recall, with values ranging from 0
to 1, where a higher score indicates better performance.
 27

Precision and recall are defined as:

Precision = True Positives / (True Positives + False Positives)
Recall = True Positives / (True Positives + False Negatives) The F1
score can be calculated using the following formula:
F1 score = 2 * (precision * recall) / (precision + recall)
It is important to note that while the F1 score is a useful metric for
evaluating overall classification model performance, it may not always be
the most appropriate measure depending on the specific requirements of
the problem.

8. Result

8.1 Overview: This machine learning project utilizes a two-layer

machine learning approach to detect malicious packets in network traffic.
a. Layer 1: The first layer performs binary classification. It analyzes
incoming packets and categorizes them as either malicious or benign
(not malicious). b. Layer 2 (if malicious): If a packet is flagged as
malicious in layer 1, it's then directed to layer 2 for further analysis. This
layer identifies the specific type of attack the malicious packet is
associated with.
28

To gauge the effectiveness of this approach, a comprehensive

evaluation utilizing diverse metrics, including accuracy, precision, and F1
score, was
undertaken. These
metrics offer a holistic
view of the system's
performance, guiding
iterative
improvements and
optimizations to stay ahead of evolving cyber threats.

8.2. Layer 1 (Binary Classification): In the first layer of our machine

learning model, we aimed to classify the data into two categories:
malicious or benign. To achieve this, we employed a distance-based
unsupervised learning algorithm known as k-means clustering. This
algorithm partitioned the data into two distinct clusters, effectively
separating the malicious and benign samples. To evaluate the quality of
the clustering, we calculated the inertia metric. Inertia measures the
within-cluster variance, indicating how closely data points are grouped
within their assigned clusters. Mathematically, inertia is defined as:
 29

Where:

● N: Number of data samples

● Xi: Individual data point
● Ck: Centroid of the cluster (average value of all points within the
cluster)
● Σ : Addition of the values

In our case, the calculated inertia value was 1.024, suggesting a high
degree of separation between the clusters and tight grouping within each
cluster.

Following the clustering step, we assessed the model's performance

using various metrics.

8.2 LAYER 2 (MULTICLASS CLASSIFICATION):

In the second layer of our machine learning model, our primary objective
is to pinpoint the specific type of vulnerability lurking within a malicious
packet. This could be a Denial-of-Service (DoS) attack aiming to
overwhelm a system with traffic, a malicious attempt to infect a device
with the Mirai botnet, a Man-in-the-Middle (MITM) exploit designed to
eavesdrop on communication, or even a routine network scan. To
achieve this fine-grained classification, we've incorporated the power of
ensemble machine learning algorithms.

CatBoost: This is a gradient boosting algorithm specifically designed for

handling categorical features, which are often prevalent in network
security data. CatBoost utilizes a novel Ordered Boosting approach that
treats categorical features more effectively compared to traditional
methods. Here's a breakdown of the achieved performance:
30

o Accuracy: 0.989670
o Class 0:
▪ Precision : 0.999964

▪ Recall : 0.999637

▪ F1-Score : 0.999801
o Class 1:
▪ Precision : 0.935238

▪ Recall : 0.931933
▪ F1-Score: 0.933583
o Class 2:
31

▪ Precision : 0.991107
▪ Recall : 0.995378
▪ F1-Score : 0.993238
o Class 3:
▪ Precision : 0.999233

▪ Recall : 0.990452

▪ F1-Score : 0.994823
o Class 4:
▪ Precision : 0.994956

▪ Recall : 0.977599

▪ F1-Score : 0.986201

XGBoost: eXtreme Gradient Boosting (XGBoost) is another robust

gradient boosting algorithm known for its scalability, speed, and
regularization capabilities. It excels at handling complex data structures
and often achieves superior performance in various machine learning
tasks Here's a breakdown of the achieved performance:
32

o Accuracy: 0.989281
o Class 0:
▪ Precision : 1.000000

▪ Recall : 0.999982

▪ F1-Score : 0.999991
o Class 1:
▪ Precision : 0.938513

▪ Recall : 0.919863

▪ F1-Score : 0.929095
o Class 2:
▪ Precision : 0.990383

▪ Recall : 0.995907

▪ F1-Score : 0.993137
o Class 3:
▪ Precision : 0.997574

▪ Recall : 0.992524
▪ F1-Score : 0.995043
o Class 4:
▪ Precision : 0.994624

▪ Recall : 0.975845

▪ F1-Score : 0.9851453434