Bayesian Optimization with Machine Learning

Algorithms Towards Anomaly Detection

MohammadNoor Injadat∗ , Fadi Salo∗ , Ali Bou Nassif† , Aleksander Essex∗ , Abdallah Shami∗
∗ Department of Electrical and Computer Engineering, The University of Western Ontario, London, ON, Canada
Email: {minjadat, fsalo, aessex, ashami2}@uwo.ca
† Department of Electrical and Computer Engineering, University of Sharjah, Sharjah, UAE

Email: [email protected]

Abstract—Network attacks have been very prevalent as their [5]. On the other hand, anomaly-based detection systems rely
rate is growing tremendously. Both organization and individuals on the hypothesis that abnormal behavior differs from normal
are now concerned about their confidentiality, integrity and behavior. Therefore, any deviation from what is considered as
availability of their critical information which are often impacted
by network attacks. To that end, several previous machine normal is classified as anomalous or intrusive. Such systems
learning-based intrusion detection methods have been developed typically build models based on normal patterns and hence
to secure network infrastructure from such attacks. In this are capable of detecting unknown behaviors or intrusions
paper, an effective anomaly detection framework is proposed [6]. Although previous work on IDSs has shown promising
utilizing Bayesian Optimization technique to tune the parameters improvement, intrusion detection problem remains a prime
of Support Vector Machine with Gaussian Kernel (SVM-RBF),
Random Forest (RF), and k-Nearest Neighbor (k-NN) algorithms. concern, especially given the high volume of network traffic
The performance of the considered algorithms is evaluated data generated, the continuously changing environments, the
using the ISCX 2012 dataset. Experimental results show the plethora of features collected as part of training datasets (high
effectiveness of the proposed framework in term of accuracy dimensional datasets), and the need for real-time intrusion
rate, precision, low-false alarm rate, and recall. detection [7]. For instance, high dimensional datasets can have
Index Terms—Bayesian Optimization, network anomaly detec-
tion, Machine Learning (ML), ISCX 2012. irrelevant, redundant, or highly correlated features. This can
have a detrimental impact on the performance of IDSs as it
can slow the model training process. Additionally, choosing
I. I NTRODUCTION
the most suitable subset of features and optimizing the corre-
Computer networks and the Internet have become an essen- sponding parameters of the detection model can help improve
tial component of any organization in this high-tech world. its performance significantly [8].
Organizations heavily depend on their networks to conduct In this paper, we propose an effective intrusion detection
their daily work. Moreover, individuals are also dependent framework based on optimized machine learning classifiers in-
on the Internet as a means to communicate, conduct busi- cluding Support Vector Machine with Gaussian kernel (SVM-
ness, and store their personal information [1]. The topic of RBF), Random Forest (RF), and k-Nearest Neighbors (k-NN)
Cyber-security has garnered significant attention as it greatly using Bayesian Optimization (BO). These techniques have
impacts many entities including individuals, organizations, been selected based on the nature of the selected dataset,
and governmental agencies. Organizations have become more i.e. SVM-RBF is selected because the data is not linearly
concerned with their network security and are allocating more separable. Additional details about the utilized techniques are
resources to protect it against potential attacks or anomalous presented in section III. This is done to provide a robust
activities. Traditional network protection mechanisms have and accurate methodology to detect anomalies. The con-
been proposed such as adopting firewalls, authenticating users, sidered methods are titled BO-SVM, BO-RF, and BO-kNN
and integrating antivirus and malware programs as a first respectively. The performance is evaluated and compared by
line of defense [2]. Nonetheless, these mechanisms have not conducting different experiments with the ISCX 2012 dataset
been as efficient in providing complete protection for the that was collected from University of New Brunswick [9]. As
organizations’ networks, especially with contemporary attacks mentioned in Wu and Banzhaf [5], a robust IDS should have
[3]. a high detection rate/recall and a low false alarm rate (FAR).
Typical intrusion detection systems (IDSs) can be cate- Despite the fact that most of intrusion detection methods
gorized into two main types, namely signature-based detec- have high detection rate (DR), they suffer from higher FAR.
tion systems (misused detection) and anomaly-based detection Thus, this paper utilizes optimized machine learning models
systems [4]. Signature-based detection systems compare the to minimize the objective function that will maximize the
observed data with pre-defined attack patterns to detect intru- effectiveness of the considered methods. Totally, the feasibility
sion. Such systems are effective for attacks with well-known and efficiency of these optimized methods is compared using
signatures and patterns. However, these systems miss new various evaluation metrics such as accuracy (acc), precision,
attacks due to the ever-changing nature of intrusion attacks recall, and FAR. Furthermore, the performance of the three

978-1-5386-4727-1/18/$31.00 ©2018 IEEE

optimized methods in parameter setting are compared with III. THEORETIC ASPECTS OF THE TECHNIQUES
the standard approaches. The main contributions of this paper A. A. Support Vector Machines (SVM)
include the following:
SVM algorithm is a supervised machine learning classifi-
• Investigate the performance of the optimized machine cation technique that identifies the class positive and negative
learning algorithms using Bayesian Optimization to de- sample by determining the maximum separation hyperplane
tect anomalies. between the two classes [18]. Depending on the nature of
• Enhances the performance of the classification models the dataset, different kernels can be used as part of the
through the identification of the optimal parameters to- SVM technique since the kernel determines the shape of the
wards objective-function minimization. separating hyperplane. For example, a linear kernel can be
• UNB ISCX 2012, a benchmark intrusion dataset is used used in cases where the data is linearly separable by providing
for experimentation and validation purposes through the a linear equation to represent the hyperplane. However, other
visualization of the optimization process of the objective kernels are needed in cases where the data is not linearly
function of the considered machine learning models to separable. One such kernel is the Gaussian Kernel . This kernel
select the best approach that identifies anomalous network maps the data points from their original input space into a
traffic. To the best of our knowledge, no previous related high-dimensional feature space. The output of the SVM with
work has adopted Bayesian Optimization on the utilized Gaussian kernel (also known as SVM-RBF) is [19]:
dataset towards anomaly detection.
f (x) = wT Φ(x) + b (1)
The remainder of this paper is organized as follows. Sec-
where Φ(x) represents the used kernel. The goal is to deter-
tion II presents the related work. Section III gives a brief
mine the weight vector wT and intercept b that minimizes the
overview of SVM, RF, and k-NN algorithms along with
following objective function:
the utilized optimization method. Section IV discusses the
research methodology and the experimental results. Finally,
Section V concludes the paper and provides future research 1
min w2 +
directions. w,b 2
Xm
C [yi × cost1 (f (xi )) + (1 − yi ) × cost0 (f (xi ))] (2)
i=1
II. R ELATED W ORK
where C is a regularization parameter that penalizes incor-
rectly classified instances, costi is the squared error over the
The intrusion detection problem has been addressed as a training dataset.
classification problem by researchers. Different data mining-
based methodologies have been posited to tackle this problem B. k-Nearest Neighbors (k-NN)
including, SVM [10], Decision Trees [11], k-NN [12], and k-NN is a simple classification algorithm that determines
Naive Bayes [13] classifiers as shown in the short review the class of an instance based on the majority class of its k
presented in Tsai et al. [1]. Later, noteworthy research have nearest neighboring points. This is done by first evaluating
been implemented and acquired promising results through the distance from the data point to all other points within
proposing novel approaches based data mining techniques the training dataset. Different distance measures can be used
Wu and Banzhaf [5]. Recently, many research adopted op- such as the Euclidean distance or Mahalanoblis distance. After
timization techniques to improve the performance of their determining the distance, the k nearest points are identified and
approach. For instance, a hybrid approach proposed by Chung a majority voting-based decision is made on the class of the
and Wahid [14] including feature selection and classification considered data point [20].
with simplified swarm optimization (SSO). The performance
C. Random Forests (RF)
of SSO was further improved by using weighted local search
(WLS) to obtain better solutions from the neighborhood RF classifier is an ensemble learning classifier that combines
[14]. Their experimental results yielded accuracy of 93.3% several decision tree classifiers to predict the class [21]. Each
in detecting intrusions. Similarly, Kuang et al. [15] proposed tree is independently and randomly sampled with their results
a hybrid method incorporating genetic algorithm (GA) and combined using majority rule. The RF classifier sends any new
multi-layered SVM with kernel principal component anal- incoming data point to each of its trees and chooses the class
ysis (KPCA) to enhance the performance of the proposed that is classified by the most trees. RF algorithm works as
methodology. Another technique introduced by Zhang et al. follows [22]:
[16] combining misuse and anomaly detection using RF. A 1) Choose T number of trees to grow.
novel algorithm applied catfish effect named, Catfish-BPSO, 2) Choose m number of variables used to split each
had been used to select features and enhance the model node.m  M , where M is the number of input
performance [17]. Authors used leave-one-out cross-validation variables.
(LOOCV) with k-NN for fitness evaluation. 3) Grow trees; While growing each tree, do the following:
 •Construct a sample of size N from N training cases attributes were scaled between the range [0,1] by using Min-
with replacement and grow a tree from this new Max method to eliminate the bias of features with greater
sample. values, the mathematical computation is as follows:
• When growing a tree at each node, select m vari-
x − min(x)
ables at random from M and use them to find the x0 = (4)
best split. max(x) − min(x)
• Grow tree to maximum size without pruning. As most of the classifiers do not accept categorical features
4) To classify point X, collect votes from every tree in the [24], data mapping technique was used to transform the non-
forest and then use majority voting to decide on the class numeric values of the features into numeric ones, named
label. categorical in MATLAB.

D. Bayesian Optimization (BO) C. Prediction Performance Measures

Bayesian optimization algorithm [23] tries to minimize a To evaluate and compare prediction models quantitatively,
scalar objective function f (x) for x. Depending on whether the following measurements were utilized:
the function is deterministic or stochastic, the output will TP + TN
be different for the same input x. The minimization process Accuracy = (5)
TP + TN + FP + FN
is comprised of three main components: a Gaussian process
model for the objective function f (x), a Bayesian update TP
P recision = (6)
process that modifies the Gaussian model after each new TP + FP
evaluation of the objective function, and an acquisition func- TP
tion a(x). This acquisition function is maximized in order to Recall = (7)
TP + FN
identify the next evaluation point. The role of this function is
where T P is the true positive rate, T N is the true negative
to measure the expected improvement in the objective function
rate, F P is the false positive rate, and F N is the false negative
while discarding values that would increase it [23]. Hence, the
rate [25].
expected improvement (EI) is calculated as:
  D. Results Discussion
EI(x, Q) = EQ max(0, µQ (xbest ) − f (x)) (3) The aim of the work is to discover the optimized models’
where xbest is the location of the lowest posterior mean and parameters of the utilized classifiers to classify the network
µQ (xbest ) is the lowest value of the posterior mean. intrusion data with the selected parameters. The experimental
scheme has been done for each technique to reduce the cost
IV. EXPERIMENTAL SETUP AND RESULT function by tuning all possible parameters to obtain the highest
DISCUSSION classification accuracy and the minimum FAR. To that end,
A. Dataset Description BO technique is used to determine the optimal parameters
for the considered machine learning models. For instance, the
In this paper, the Information Security Centre of Excellence
optimal values of C and γ (for SVM), the depth of trees and
(ISCX) 2012 dataset was used to perform the experiments
the adopted ensemble method (for RF), and the value of k and
and evaluate the performance of the proposed approach to
the distance measure method (for k-NN) are determined.
detect anomalies. The entire dataset comprises nearly 1.5
For example, if we have a set of machine learning model
million network traffic packets, with 20 features and covered
parameters P ∗ = P1 , P2 , . . . , Pn where Pi is a parameter
seven days of network activity (i.e. normal and intrusion).
of the parameters subset that needs tuning, then BO tries to
Additional information about the dataset are available in [9].
minimize the following cost function:
A random subset has been extracted from the original dataset.
The training data contains 30,814 normal traces and 15,375 P ∗ = min J(P ) (8)
attack traces while the testing data contains 13,154 normal
traces and 6,580 additional attack traces. where J(P ) is the associated cost function.
To visualize the behavior of the BO technique combined
B. Experimental setup and Data Preprocessing with the machine learning technique on the training dataset,
The proposed techniques were implemented using MAT- Figures 1 and 2 depict how BO tunes the parameters towards
LAB 2018a. Experiments were carried out in an Intel® the global minimum value of the SVM cost function with
Core™ i7 processor @ 3.40 GHz system with 16GB RAM respect to C and γ as parameters subset. According to the
running Windows 10 operating system. The selected dataset figures, a unique global minimum is obtained for C = 433.32
was transformed from their original format into a new dataset and γ = 1.0586. This in turn leads to improving the model’s
consisting of 14 features. We eliminated the payload features training accuracy as shown in Table 1 from 99.58% without
which include the actual packet as most of their contents optimization to 99.95% after optimization. Additionally, the
were empty, while start time, and end time features have been testing accuracy increases from 99.59% to 99.84%. On the
replaced by duration feature. In the data normalization stage, other side, the FAR had promising results with a reduction
of 0.01 and 0.007 in the training and testing datasets respec-
Objective function model
tively. Table 2 also shows more details about the optimization

an
rm
Observed points

ea
processing time. Model mean

sp
Next point

n
ea
Model minimum feasible

lid
uc
se

i
sk
w
ko
Objective function model

in
m
103

is
ob
Observed points

an
al
Model mean

ah
Next point

d
ar
Model minimum feasible

cc
102

ja
g
in
m
m
ha

n
101

ea
id
cl
eu

ne
si
co
100

n
io
t
la
rre
co

v
he
yc
10-1

eb
ch

k
100 101 102 103 104

oc
bl
ty
ci
10-2

Fig. 3. Optimized k-NN Contour

10-3
10-3 10-2 10-1 100 101 102 103
C

Fig. 1. Optimized SVM Contour

Fig. 4. Optimized k-NN Objective Function Model

Figures 5, 6, and 7 visualize the change in the objective

function value vs the number of function evaluations for BO-
SVM, BO-RF, and BO-kNN respectively. It can be observed
Fig. 2. Optimized SVM objective function model that the objective function reaches its global minimum within
30 iterations at most. This reiterates the efficiency of the BO
Similarly, Figures 3 and 4 and Table 2 show how the BO technique in optimizing the considered algorithms.
technique is minimizing the cost function J(P ) for k-NN By applying BO-RF, a unique global minimum is achieved
algorithm with respect to the number of neighbors k and with 1004 tree splits (Tree Depth) and AdaBoost as a tree
the distance measuring method. A unique global minimum is method. The BO improves the training accuracy from 99.97%
achieved for the values of k = 1 and Mahalanobis distance as to 99.98% while the testing accuracy improves from 99.88%
the distance measuring method. According to Table 1, BO was to 99.92%. The FAR remains steady in the training dataset
able to improve the BO performed 30 iterations to evaluate the and is reduced by 0.001 in the testing dataset. Furthermore,
cost function in the aim to converge toward the optimal J(P ) Table 2 indicate that the BO find that AdaBoost is the best
of each classifier. ensemble method to build the tree.
 TABLE I
P ERFORMANCE RESULTS OF THE THREE CLASSIFIERS

Training Testing
Classifier Acc(%) Precision Recall FAR Acc(%) Precision Recall FAR
SVM-RBF 99.58 0.994 0.999 0.011 99.59 0.995 0.999 0.010
K-NN (k=5) 99.59 0.9965 0.998 0.008 99.36 0.994 0.996 0.012
RF 99.96 0.999 1.00 0.001 99.88 0.998 0.999 0.002
BO-SVM 99.95 0.999 1.00 0.001 99.84 0.998 0.999 0.003
BO-k-NN 99.98 0.999 1.00 0.001 99.93 0.999 0.999 0.001
BO-RF 99.98 0.999 1.00 0.001 99.92 0.999 0.999 0.001

Min objective vs. Number of function evaluations

0.06
Min observed objective
Min objective vs. Number of function evaluations
0.35 Estimated min objective

Min observed objective

Estimated min objective 0.05

0.3

0.04
0.25

Min objective
0.03
0.2
Min objective

0.15 0.02

0.1
0.01

0.05

0
0 5 10 15 20 25 30
Function evaluations
0
0 5 10 15 20 25 30
Function evaluations

Fig. 7. BO-RF Objective Function vs Number of Function Evaluations

Fig. 5. BO-SVM Objective Function vs Number of Function Evaluations

It is also worth mentioning that Naïve Bayes classifier was

utilized at the initial stage of the experiment. However, due to
the fact that the dataset’s features are not fully independent,
Min objective vs. Number of function evaluations the classifier shows a low accuracy of 87.23% and 87.65% on
0.05
Min observed objective the training and testing datasets respectively. Hence, the Naïve
Estimated min objective
0.045 Bayes classifier was excluded from the experiment.
0.04
Based on the previous publications, our results outperform
the results of previous experiments conducted using ISCX
0.035
2012 such as the results shown in [26] with their model
0.03 acheiving about 95% as overall accuracy using their proposed
Min objective

technique. Additionally, [27] reported the highest accuracy

0.025
of 99.8% and 99.0% for the training and testing phases
0.02 respectively.
0.015
V. C ONCLUSIONS
0.01
In this paper, we utilized a Bayesian optimization method to
0.005 enhance the performance of anomaly detection methodology
based on three conventional classifiers; Support Vector Ma-
0
0 5 10 15 20 25 30
Function evaluations
chine with Gaussian kernel (SVM-RBF), Random Forest (RF),
and k-Nearest Neighbor (k-NN). The BO optimization method
has been applied to set the parameters of these classifiers by
Fig. 6. BO-kNN Objective Function vs Number of Function Evaluations finding the global minimum of the corresponding objective
function. In order to have an efficient machine learning-based
 TABLE II
O PTIMIZATION PARAMETRS FOR EACH CLASSIFIER

BO-SVM BO-k-NN BO- RF

Best Parameters BoxConstraint (C) 433.32 NumNeighbors 1 Method AdaBoost
KernelScale (γ) 1.0586 Distance Mahalanobis MaxNumSplits 1004
Total function evaluations 30 30 30
Total elapsed time in seconds 6175.78 2272.50 771.24

anomaly detection system with high accuracy rate and a [12] W. Li, P. Yi, Y. Wu, L. Pan, and J. Li, “A new intrusion detection system
low false positive rate, BO was able to improve the utilized based on knn classification algorithm in wireless sensor network,”
Journal of Electrical and Computer Engineering, vol. 2014, 2014.
classifiers. The experimental results show not only is the [13] S. Aljawarneh, M. Aldwairi, and M. B. Yassein, “Anomaly-based in-
proposed optimization method more accurate in detecting trusion detection system through feature selection analysis and building
intrusions, but also it can find the global minimum of the hybrid efficient model,” Journal of Computational Science, 2017.
[14] Y. Y. Chung and N. Wahid, “A hybrid network intrusion detection system
objective function which leads to better classification results. using simplified swarm optimization (sso),” Applied Soft Computing,
Overall, k-NN with Bayesian optimization has achieved the vol. 12, no. 9, pp. 3014–3022, 2012.
optimum performance on ISCX 2012 dataset in terms of [15] F. Kuang, W. Xu, and S. Zhang, “A novel hybrid kpca and svm with
ga model for intrusion detection,” Applied Soft Computing, vol. 18, pp.
accuracy, precision, recall, and false alarm rate. In order to 178–184, 2014.
further improve the performance of the proposed approach, we [16] J. Zhang, M. Zulkernine, and A. Haque, “Random-forests-based network
plan to involve feature selection and parameter setting applied intrusion detection systems,” IEEE Transactions on Systems, Man, and
Cybernetics, Part C (Applications and Reviews), vol. 38, no. 5, pp. 649–
simultaneously in the optimization method. Moreover, the 659, 2008.
results of the proposed approach will be further improved by [17] A. J. Malik and F. A. Khan, “A hybrid technique using multi-objective
combining both supervised and unsupervised machine learning particle swarm optimization and random forests for probe attacks
detection in a network,” in Systems, Man, and Cybernetics (SMC), 2013
techniques to detect novel attacks with additional datasets such IEEE International Conference on. IEEE, 2013, pp. 2473–2478.
as the new release of the ISCX dataset. [18] I. S. Thaseen and C. A. Kumar, “Intrusion detection model using fusion
of chi-square feature selection and multi class svm,” Journal of King
Saud University-Computer and Information Sciences, vol. 29, no. 4, pp.
R EFERENCES 462–472, 2017.
[19] H. Bostani and M. Sheikhan, “Modification of supervised opf-based
[1] C.-F. Tsai, Y.-F. Hsu, C.-Y. Lin, and W.-Y. Lin, “Intrusion detection by intrusion detection systems using unsupervised learning and social
machine learning: A review,” Expert Systems with Applications, vol. 36, network concept,” Pattern Recognition, vol. 62, pp. 56–72, 2017.
no. 10, pp. 11 994–12 000, 2009. [20] W. Meng, W. Li, and L.-F. Kwok, “Design of intelligent knn-based alarm
[2] M. B. Salem, S. Hershkop, and S. J. Stolfo, “A survey of insider attack filter using knowledge-based alert verification in intrusion detection,”
detection research,” in Insider Attack and Cyber Security. Springer, Security and Communication Networks, vol. 8, no. 18, pp. 3883–3895,
2008, pp. 69–90. 2015.
[3] W. Bul’ajoul, A. James, and M. Pannu, “Improving network intrusion [21] M. Injadat, F. Salo, and A. B. Nassif, “Data mining techniques in
detection system performance through quality of service configuration social media: A survey,” Neurocomputing, vol. 214, pp. 654 – 670,
and parallel technology,” Journal of Computer and System Sciences, 2016. [Online]. Available: http://www.sciencedirect.com/science/article/
vol. 81, no. 6, pp. 981–999, 2015. pii/S092523121630683X
[4] S. M. H. Bamakan, B. Amiri, M. Mirzabagheri, and Y. Shi, “A new [22] A. J. Malik, W. Shahzad, and F. A. Khan, “Binary pso and random
intrusion detection approach using pso based multiple criteria linear forests algorithm for probe attacks detection in a network,” in 2011
programming,” Procedia Computer Science, vol. 55, pp. 231–237, 2015. IEEE Congress of Evolutionary Computation (CEC), June 2011, pp.
[5] S. X. Wu and W. Banzhaf, “The use of computational intelligence in 662–668.
intrusion detection systems: A review,” Applied soft computing, vol. 10, [23] E. Brochu, V. M. Cora, and N. De Freitas, “A tutorial on bayesian
no. 1, pp. 1–35, 2010. optimization of expensive cost functions, with application to active
[6] H.-J. Liao, C.-H. R. Lin, Y.-C. Lin, and K.-Y. Tung, “Intrusion detection user modeling and hierarchical reinforcement learning,” arXiv preprint
system: A comprehensive review,” Journal of Network and Computer arXiv:1012.2599, 2010.
Applications, vol. 36, no. 1, pp. 16–24, 2013. [24] M. Salem and U. Buehler, “Mining techniques in network security to
[7] S. Suthaharan, “Big data classification: Problems and challenges in net- enhance intrusion detection systems,” arXiv preprint arXiv:1212.2414,
work intrusion prediction with machine learning,” ACM SIGMETRICS 2012.
Performance Evaluation Review, vol. 41, no. 4, pp. 70–73, 2014. [25] M. H. Tang, C. Ching, S. Poon, S. S. Chan, W. Ng, M. Lam, C. Wong,
[8] J. Zhang and M. Zulkernine, “Anomaly based network intrusion de- R. Pao, A. Lau, and T. W. Mak, “Evaluation of three rapid oral fluid test
tection with unsupervised outlier detection,” in Communications, 2006. devices on the screening of multiple drugs of abuse including ketamine,”
ICC’06. IEEE International Conference on, vol. 5. IEEE, 2006, pp. Forensic science international, 2018.
2388–2393. [26] H. Huang, R. S. Khalid, W. Liu, and H. Yu, “Work-in-progress: a fast
online sequential learning accelerator for iot network intrusion detec-
[9] A. Shiravi, H. Shiravi, M. Tavallaee, and A. A. Ghorbani, “Toward
tion,” in Hardware/Software Codesign and System Synthesis (CODES+
developing a systematic approach to generate benchmark datasets for
ISSS), 2017 International Conference on. IEEE, 2017, pp. 1–2.
intrusion detection,” Computers Security, vol. 31, no. 3, pp. 357 –
[27] W. Yassin, N. I. Udzir, Z. Muda, M. N. Sulaiman et al., “Anomaly-
374, 2012. [Online]. Available: http://www.sciencedirect.com/science/
based intrusion detection through k-means clustering and naives bayes
article/pii/S0167404811001672
classification,” in Proc. 4th Int. Conf. Comput. Informatics, ICOCI,
[10] F. Kuang, W. Xu, and S. Zhang, “A novel hybrid kpca and svm with
no. 49, 2013, pp. 298–303.
ga model for intrusion detection,” Applied Soft Computing, vol. 18, pp.
178–184, 2014.
[11] A. S. Eesa, Z. Orman, and A. M. A. Brifcani, “A novel feature-selection
approach based on the cuttlefish optimization algorithm for intrusion
detection systems,” Expert Systems with Applications, vol. 42, no. 5,
pp. 2670–2679, 2015.