IET Communications

Review Article

Anomaly detection for cellular networks using ISSN 1751-8628

Received on 7th August 2019
Revised 4th September 2019
big data analytics Accepted on 24th September 2019
E-First on 11th November 2019
doi: 10.1049/iet-com.2019.0765
www.ietdl.org

Bing Li1, Shengjie Zhao1,2 , Rongqing Zhang2,3, Qingjiang Shi2, Kai Yang1
1Key Laboratory of Embedded System and Service Computing, Ministry of Education, Tongji University, Shanghai 201804, People's Republic of

China
2School of Software Engineering, Tongji University, Shanghai 201804, People's Republic of China
3Shandong Provincial Key Laboratory of Wireless Communication Technologies, Shandong University, Shandong, People's Republic of China

E-mail: [email protected]

Abstract: Broadband connectivity and mobile technology have been widely applied in the world. With these advanced
technologies, the proliferation of smart devices and their applications by accessing mobile internet have come up with a giant
leap forward, leading to the ever-increasing scale and complexity of cellular networks. This presents imminent challenges to
anomaly detection in cellular networks. In this study, the authors discuss challenges and current literature of anomaly detection
for cellular networks to embrace the ‘big data’ era. First, they review the state-of-the-art techniques in the area of anomaly
detection in cellular networks. Then, the challenges are pinpointed for anomaly detection due to the cellular network big data.
Finally, they introduce a big data analytic-based anomaly detection method for cellular networks.

1 Introduction achieved based on network measurements and parameters. In

cellular networks (e.g. LTE/LTE-A), there are lots of network
Over the past 20 years, due to the rapid development of parameters and measurements, which are consecutively exchanged,
information and communication technologies, the world has collected, and reported at/from the nodes and user equipment (UE)
witnessed profound social, cultural, and economic changes. In in the core networks and the radio access network, such as call
particular, after the 2.5G (i.e. global system for mobile detail record (CDR), location information, UE movement
communication (GSM), enhanced data GSM evolution), third behaviour, reference signal received power, radio link failure
generation (i.e. wideband code division multiple access, code report, and so on. We can detect network anomalies through the
division multiple access 2000, and time division-synchronous code analysis of the network parameters and measurements, and thus,
division multiple access), and fourth generation (4G) (i.e. long- the network behaviour can be monitored to avoid possible threats,
term evolution (LTE)/LTE-advanced (LTE-A)), mobile failure, and faults. The general system architecture of anomaly
communication systems are about to enter fifth generation (5G) in detection is shown in Fig. 1. Overall, anomaly detection in cellular
the near future. Meanwhile, broadband connectivity and mobile networks has three major advantages. First, it facilitates the
technology have been widely applied in developed countries and network operator to perform effective management of the cellular
gradually deployed in emerging markets all over developing world. networks. Second, it enables the optimisation and enhancement of
With the explosion in the number of smart devices (e.g. user QoE by exploring relevant historical information. Third,
smartphones, smart vehicles, smart sensors etc.) and the network-wide insights can be obtained to facilitate efficient
proliferation of mobile internet they access, the scale and network planning and deployment. Owing to the above benefits, it
complexity of cellular networks continue to expand. is very crucial to timely deal with the potential network anomalies.
As the cellular network evolves towards 5G, lots of wireless However, the 4G/emerging 5G and internet-of-things (IoT) are
technologies, such as massive multiple-input multiple-output, bringing more complicated operations of the network, more and
dense and heterogeneous networks, have the opportunity to provide more mobile nodes, and devices [7]. Therefore, the network
good user quality of experience (QoE). In cellular networks, user measurements and parameters (called data hereafter) are produced
QoE is impacted by various aspects such as traffic load, base in a very large range (volume), from myriad sources in/out of the
station (BS) configurations, and wireless coverage etc. For network (variety), with fast network input and/or output (velocity),
example, excessive increase of data communication in some and with possibly unwarranted data quality or trust (veracity).
particular hotspot areas will degrade the user QoE, and may finally These 4 V features are usually used to describe the so-called big
result in service interruption in some situation. As a result, data. It is easily envisioned that big data of cellular networks brings
abnormal traffic behaviour, which will reduce the robustness of the both new challenges and opportunities for network monitoring and
network must be detected and solved immediately [1–3]. On the management. Particularly, big data analytics that integrates
other hand, mobile devices face a lot of security threats with the advanced technologies of both hardware and software to collect,
increasing number of mobile applications [4, 5]. In particular, for analyse, and manage large-scale unstructured/structured data
various malicious purposes, including sending spam short timely, is demanded to perform effective anomaly detection in
messaging services (SMSs), stealing personal data, and initiating cellular networks.
denial of service attacks against core network components, In this study, we summarise the state-of-the-art techniques in
malware is often installed in smartphones. Authentication and the area of anomaly detection in cellular networks and the
licensing-based technologies leveraged by network operators may confronting challenges arising from big data in current cellular
not be able to provide overall protection against malware threats networks. Then, a new anomaly detection method using big data
[6]. This requires the detection of malware-induced anomalies in analytics for cellular networks is presented. The remaining part of
cellular networks as well. this paper is organised as follows. Section 2 presents the state-of-
Apparently, automatic anomaly detection is critical for effective the-art anomaly detection techniques. Section 3 discusses the
operation and cellular network management. This is generally challenges brought by the emergence of big data to anomaly

IET Commun., 2019, Vol. 13 Iss. 20, pp. 3351-3359 3351

© The Institution of Engineering and Technology 2019
 17518636, 2019, 20, Downloaded from https://ietresearch.onlinelibrary.wiley.com/doi/10.1049/iet-com.2019.0765, Wiley Online Library on [05/02/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
CELLID data (location information of the user's cellular network)
obtained from different sources, CADM detects the abnormality of
the user's physical location that changes rapidly over a while.
Furthermore, CADM can detect the location of the site where an
anomaly has occurred. Its notable advantage is a fast life-cycle to
reach target results, better ways to extract relevant rules without
training data, and the ability to exploit cost-effective big data
analytic methods. So far, there is rare research on the rule-based
anomaly detection method of cellular network except [8].
However, there are still many rule-based exception detection
methods in other fields that are worth referring to, such as [9–13].
Compared with the traditional smart home anomaly detection,
which focuses on existing user activities and checks whether it is
abnormal, the rule-based anomaly detection method proposed in
[9] takes environmental factors into account to perform early
anomaly detection. Considering the dynamic real-time data in the
intelligent home environment, dynamic creation of a ground
Markov logic network (MLN) method is proposed by improving
the machine learning (ML) method of MLN. It can find risks in the
environment according to the causal association rules and
recommend appropriate actions, which can help avoid abnormal
occurrences.
Aiming at improving computer network security, Mavroeidis et
al.[10] propose a network anomaly detection method based on
genetic algorithm, which is based on Management Information
Base (MIB). The proposed method makes IF–THEN rule easier to
understand by designing a new chromosome encoding scheme, and
Fig. 1 Architecture of anomaly detection in cellular networks more effective to judge the comprehensibility of classification rules
by designing a new sufficient function. Experiments show that this
detection. Section 4 further presents and examines a big data method improves the intelligibility and reduces the complexity of
analytic-based anomaly detection method for current cellular the genetic algorithm-based anomaly detection method.
networks, followed by the conclusion in Section 5. Duffield et al. [11] use the rule-based ML method for traffic
anomaly detection. Compared with the traditional method of
detecting anomalies at network entry points using high-speed
2 State-of-the-art: anomaly detection techniques monitors, the proposed method directly detects traffic statistics
This section elaborates on the state-of-the-art in the anomaly collected by internet service provider, which reduces the
detection area. We divide popular anomaly detection techniques deployment cost of a high-speed monitor. The proposed ML
into three classes, and then present in turn the core idea of each method utilises Snort rules to analyse the relationship between
class of anomaly detection techniques. packets and stream records, so as to be able to detect the
relationship between abnormal flows and packet-level features and
2.1 Learning-based Anomaly Detection (AD) methods implement flow-level alerts.
Xu et al. [12] propose a system performance anomaly detection
Learning in this field is based on computer learning (or training) of method based on fuzzy rules. The form of fuzzy rule is: if x → A
historical samples to obtain a decision or prediction model, which then y → B. A and B are language values defined by fuzzy sets on
can be extended to the identification or prediction of unknown the domains X and Y. ‘X → A’ is the premise, ‘y → B’ is the
samples. The approach or process of learning a model is called a conclusion. The proposed method uses genetic algorithm to design
learning method (algorithm). The application of the learning an anomaly detector, and then uses negative selection algorithm as
method in anomaly detection is of great significance for solving the a filter to eliminate an invalid detector to reduce the search space,
problems faced by anomaly detection, improving detection so as to obtain a lower false alarm probability (FP).
accuracy, reducing false alarm rate, false alarm rate, and detection Hassanzadeh and Nayak [13] also applied fuzzy rules to
overhead, and providing powerful security guarantee for the anomaly detection, in which an online social network anomaly
network. detection method combining fuzzy clustering, graph theory, and
fuzzy rules is proposed. Firstly, graph theory is used to model the
2.1.1 Rule-based methods: Owing to the effect in supervised behaviour of users in social networks, which is a method of partial
learning approaches, rule-based techniques have been used for analysis. Then, Fuzzy C-means (FCM) algorithm [14], a kind of
anomaly detection in cellular networks. The basic idea of these fuzzy clustering technique, is used to cluster the modelling results,
techniques is to learn knowledge from historic data, which is in which the expected maximisation algorithm and the cluster
generally characterised by a set of ‘IF condition THEN conclusion’ number defined by prior are adopted to overcome the shortcomings
rules. The ‘IF’ part is called the premise of the rule. The ‘THEN’ of FCM. Finally, the fuzzy inference engine is used to classify the
part is called the conclusion of the rule. For a given anomaly, if the results of fuzzy unsupervised clustering by using the generated
anomaly satisfies the conditions in the premises, we say that the rules and membership functions.
premises of the rule are satisfied and that the anomaly is classified The above rule-based anomaly detection methods are both
as then anomaly specified in the conclusion. One advantage of the expected to be well applied in cellular network anomaly detection.
rule-based anomaly detection approaches is that they are robust,
flexible, and can be even dynamically adjusted based on the needs 2.1.2 Neural network (NN): NN is a biologically-inspired
of operations. Moreover, their detection rate is generally high programming paradigm (shown in Fig. 2), which enables a
compared to other anomaly detection methodologies such as computer to learn an unknown non-linear map from observations.
statistical or clustering-based methods. As one of the celebrated ML approaches, NN can be used for either
Sfar et al. [8] present a rule-based AD approach CDR-based classification or regression and has been widely applied in image
anomaly detection method (CADM) based on CDR that analyses and speech recognition. The strength of NN for classification has
the users' calling activities and detects the unusual user activities in also been leveraged for the anomaly detection task. Within a
the real cellular network. Through the analysis of CITYID data typical AD application, the training data for NN consists of
(location information of the city that the user belongs to) and network measurement as NN input and anomaly labelled as NN

3352 IET Commun., 2019, Vol. 13 Iss. 20, pp. 3351-3359

© The Institution of Engineering and Technology 2019
 17518636, 2019, 20, Downloaded from https://ietresearch.onlinelibrary.wiley.com/doi/10.1049/iet-com.2019.0765, Wiley Online Library on [05/02/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
interruption and abnormal activity caused by demand surge. Based
on this, appropriate measures can be taken to optimise network
performance.
A simple but effective supervised method is leveraged in [1] to
detect and classify the traffic anomaly leveraging random NN
(RNN), which can be described as a combination of the classical
NN model and queuing networks. In this approach, RNN is used to
learn the relationship in batch-mode among m supervised traffic
features (i.e. some domain name system requests released within a
time-bin) and the type of relevant monitored traffic (i.e. anomaly
operation or normal type). The RNN model mainly comes up with
a certain non-linear transfer-block f ( ⋅ ): ℝn → K, such that
yt = f (Xt) = f (x1(t), x2(t), …xm(t)), where Xt = {x1(t), x2(t), …xm(t)}
denotes the m traffic features computed for a time-bin of length t,
K = {0, 1, 2, …, i, …, K − 1} defines the set of K predefined
Fig. 2 Architecture of typical NN classes (e.g. 0 for anomaly-free traffic and i for anomaly of type i).
Casas et al. [1] show that the proposed RNN-based approach can
accurately detect different kinds of network traffic anomalies in an
automatic way, meanwhile simplifying the tasks of network
operations management and enhancing the visibility. This RNN
approach is also used for abnormal detection of the cellular
networks in [17].
Based on the RNN (not the same NN as the one mentioned
above), Mamun et al. [18] also adopt an unsupervised method
called long short-term memory (LSTM) to conduct anomaly
detection. Different from the traditional RNN, LSTM adds a cell to
determine whether the information is useful or not. A cell has three
gates, called input/forget/output gate, respectively. A piece of
information can be judged useful or not when it enters the LSTM
network. That verified information will be left, while that
information that does not conform will be forgotten through the
forget gate. Under such repeated operation, LSTM can effectively
solve the problem of long sequence dependence. Based on the
LSTM algorithm, Mamun et al. [18] model the time series of KPI
values of cells in the LTE network and was able to detect long and
short interval anomalies in the cellular network at the same time,
achieving faster cell level anomaly detection.

2.1.3 Ensemble method: The ensemble method can work

effectively when no single traditional AD method can provide the
desired detection performance. As a supervised learning algorithm,
the ensemble method can be trained and then used to make
Fig. 3 Sketch of ensemble method
decisions. It builds on a set of models, in a certain way, those
individual decisions are generally combined by weighted or
output. Based on the training data, the NN is trained via some
unweighted voting. Empirically, the ensemble method tends to
sophisticated learning algorithms. Once the training of NN is
yield better results when there are significant differences between
completed, the obtained NN can be easily used to detect whether
models. Owing to this notable advantage, the ensemble method is
anomaly exists for each input network measurement. Beyond using
also an effective method to solve anomaly detection problems in
standard artificial NNs for AD, some works have used new types
cellular networks [2, 20–23].
of NN for AD [1, 15–18], which will be described in detail below.
The ensemble-method framework proposed in [2, 20, 21] builds
Sukkhawatchani and Usaha [15] applied the self-organising
on KPIs, which are measures of cell performance relative to cell
map (SOM), an unsupervised NN method, to the abnormal state
state. Specifically, Ciocarlie et al. [2] apply individual multivariate
detection of the cellular mobile network. As a competitive NN
(adapted to several KPIs) and univariate (adapted to each KPI)
model, SOM can conclude statistical rules from the input message,
methods on the trained data to generate a pool of different models
and encode them as weights in an unsupervised way. Therefore,
or predictors. Multiple univariate [Empirical Cumulative
this method can detect global anomalies and identify which key
Distribution Function (ECDF), Support Vector Machine (SVM),
performance indicator (KPI) anomalies are specific to the core
autoregressive, integrated moving average (ARIMA) etc.] and
network. Compared with unsupervised NN methods in [15] and
multivariate methods (SVM, Vector Auto-regressive (VAR) etc.)
other works, such as [19], some semi-supervised [16], and
are used to model KPI behaviour. Using the pool of predictors, the
supervised [1, 17, 18] NN methods with lower ambiguity of
framework establishes a composite detection approach by taking a
training samples, controllable prediction results and better model
weighted vote of different predictions, where the weights are
quality have been pervasive in cellular network anomaly detection.
learned by modelling the training data, eventually leading to the
Hussain et al. [16] based on the semi-supervised NN method
computation among the level of KPI degradation (i.e. deviation of
analyse user activity data extracted from the CDR generated by the
the KPI value compared with its normal states). Fig. 3 provides a
cellular network to detect abnormal behaviour of users. Semi-
sketch of the proposed ensemble methods, which are realised via
supervised learning is a kind of learning method combining
the modified weighted majority algorithm (WMA). By analysing
supervised learning and unsupervised learning, which uses a large
the context information, the modified WMA can update the
amount of unmarked data and marked data at the same time to
weights and create new models, and finally return a level of KPI
carry out pattern recognition. When semi-supervised learning is
degradation with a value from 0 to 1.
used, it will require as few people as possible to do the work, and
Ciocarlie et al. [20] further improve the feasibility of the
at the same time, it can bring high accuracy. Therefore, semi-
ensemble method proposed in [2], which studies the application of
supervised learning is getting more and more attention. This
this method to be applied in the actual operating environment. Data
method can detect abnormal cell dormancy caused by battery
from the actual cellular networks are used to infer KPI indicators of
IET Commun., 2019, Vol. 13 Iss. 20, pp. 3351-3359 3353
© The Institution of Engineering and Technology 2019
 17518636, 2019, 20, Downloaded from https://ietresearch.onlinelibrary.wiley.com/doi/10.1049/iet-com.2019.0765, Wiley Online Library on [05/02/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
original variable and the information contained does not repeat.
The basic idea of the PCA-based AD method is described as
follows. Let X be an m × T time-series measurement matrix where
the tth column corresponds to an n-dimensional vector of
measurements at time t. Apply PCA to yield a set of (say k)
principal components of X , stored in matrix P, with dominant
variances. Matrix P forms the basis of the normal measurement
space. Given a measurement vector x(t), the residual (I − PPT)x(t)
represents the possible abnormal measurement. If the residual is
evident, the occurrence of anomaly is claimed.
Based on the PCA, the authors of [29, 30] present a novel AD
scheme for the case where the network topology information is
incomplete or some false diagnoses are caused by the measurement
probe misalignment. The proposed scheme aims to fuse the
measurement data from multiple domains and transform the data to
detect correlated anomaly events. It utilises the source-side
Fig. 4 Architecture of typical PCA favourable position in the measurement trace to diagnose whether
the location of a correlative bottleneck anomaly incident is partial
network performance. The experimental result shows that the or external. Case studies show that the new framework can
proposed ensemble method has lower latency and computational differentiate between uncorrelated and correlated anomalies
overhead in a real operation environment, so it can be applied to automatically without complete network topology information.
real operation network anomaly detection. This method was further Meanwhile, the accuracy of detection is high and the probability of
extended into a set of visual manipulation tools in [21]. error is low.
Different from the direct integration methods in [2, 20, 21], the Kim et al. [31] applied two higher-order PCA methods, higher-
super learner method integrating a series of basic learning-based order singular value decomposition [32] and higher-order
methods is used for anomaly detection in [22, 23]. Based on cross orthogonal iteration (HOOI) [33] to large-scale anomaly detection
validation, a super learner trains a series of general algorithms and in cellular network. These methods can address scalability issues of
weighs the results. Among them, three types of weights are the large-scale network by displacing the M 2 × M 2 transformation
considered in [23]. MVuniform: assign the same weight to each matrix in anomaly detection with two M × M matrices.
general algorithm; MVaccuracy: weight according to the accuracy Furthermore, the HOOI algorithm can be implemented in a low
of the algorithm to realise the training set classification; MVexp: complexity way to improve the efficiency of the algorithm by
weight according to the exponential accuracy of the algorithm to utilising the sparsity of the link matrix and reducing unnecessary
realise the training set classification. According to the experimental computation. Experimental results indicate that the anomaly
results, the super learner can obtain better results than any single detection method based on higher-order PCA proposed in [31] can
model. enhance the scalability of large-scale network anomaly detection
and realise low complexity application.
2.2 Statistical AD methods Livani and Abadi [34] applied the distributed PCA (DPCA)
[35] method to sensor network anomaly detection. In this work, the
Statistical analysis method refers to a research method that can network is divided into groups containing a central node and
correctly explain and predict things by analysing and studying the several member nodes. Each member node uses fixed-width
quantitative relations of research objects, such as the scale, speed, clustering to cluster its sensor data and sends the clustering results
scope, degree etc., and recognising and revealing the mutual to the central node. Each central node uses DPCA for distributed
relations, changing rules, and developing trends among things. By anomaly detection on its group's data. This method can reduce
means of a mathematical method, the statistical analysis method communication overhead and energy consumption while ensuring
establishes the mathematical model, conducts mathematical the accuracy of anomaly detection. Similar to DPCA, Liu et al.
statistics and analyses various data and data obtained through the [36] also proposed a distributed PCA algorithm for network
investigation, and forms quantitative conclusions. anomaly detection, which can reduce space requirements and
communication costs.
2.2.1 Statistical signal processing: As an application field of Based on the pioneering work in [24], Callegari et al. [37]
statistics and signal processing, statistical signal processing treats proposed a novel anomaly detection technology based on PCA.
signals or measurements as a random process to handle its Specifically, the input data is constructed as timing signals for
statistical properties, such as mean value and covariance. Broadly anomaly detection, based on which the anomaly flow can be
speaking, statistical signal processing techniques can be used as detected instead of only anomaly aggregation. Furthermore, PCA is
preprocessing of network measurements whose results will be conducted on the Kullback–Leibler (K–L) divergence of data
finally used as features for anomaly detection. Hence, they have instead of just the entropy of data. Since K–L divergence can
been widely used in network anomaly detection. Typical statistical capture more information on data than entropy, the performance of
signal processing methods used in anomaly detection include anomaly detection proposed in this work is better and more stable.
distribution change detection [24], hypothesis test [25], sequence
pattern mining [26], ARIMA models, empirical cumulative 2.2.3 Clustering-based methods: The goal of clustering is to
distribution functions [27], counter-based and sketch-based place objects with similar properties in the same cluster. More
techniques [28]. These methods are usually used together with specifically, it can partition a data set into multiple subsets, so that
other AD methods mentioned elsewhere. For example, works [24, the defined distances between the data in the same subset is
27] will be introduced in other sections, so the statistical signal relatively closer. As an important statistical data analysis
processing method is not analysed in detail based on different technique, clustering has been pervasive in lots of fields, such as
references in this section. data mining, pattern recognition, ML etc. It can be treated as a
separate tool to obtain the data distribution, analyse each cluster
2.2.2 Principal component analysis (PCA): In statistics, PCA is feature of data, and play the role as a preprocessing step for further
an useful statistical tool (shown in Fig. 4). With an orthogonal analysis (such as classification and qualitative inductive
transformation, it can convert a set of observations of possibly algorithms).
correlated variables into corresponding values of linearly Clustering has been used for AD in cellular networks, e.g. [38–
uncorrelated variables, which are called principal components. 42], among which, a simple clustering-based AD method is to
Each principal component reflects most of the information of the simply regard the cluster with only a few data points as an

3354 IET Commun., 2019, Vol. 13 Iss. 20, pp. 3351-3359

© The Institution of Engineering and Technology 2019
 17518636, 2019, 20, Downloaded from https://ietresearch.onlinelibrary.wiley.com/doi/10.1049/iet-com.2019.0765, Wiley Online Library on [05/02/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
anomaly. For instance, based on two clustering techniques— weighted moving average [3]) will be leveraged to spot abrupt
hierarchical clustering and K-mean clustering, Parwez et al. [38] changes, which indicate the presence of potentially non-normal and
analyse anomalous behaviour of LTE mobile networks with CDR, harmful affairs (i.e. the detection of an anomaly). The pioneering
which is a set of mobile network data. In the proposed cluster- work [24] has shown that using entropy in the analysis of traffic
based AD methods, the number of clusters is determined by the distributions is capable of highly sensitive and accurate detection
Elbow method. After performing clustering on the CDR data set, of a large scale of anomalies. Besides, entropy analysis can be used
the cluster with the the fewest number of objects is considered to for feature extraction in graph-based detection of anomalies related
be an anomaly. Simulation results indicate that K-means clustering to billing in the cellular network [6]. These entropy-related
has a lower complexity than the hierarchical clustering whereas the features, as well as other features, can be subsequently leveraged
latter has better detection performance. for mobile devices identification with anomaly and the supervised
Similar to Parwez et al. [38], Kashif et al. [39] also apply the classification of network events, yielding superior detection
K-means clustering method to carry out abnormal detection of performance in a variety of scenarios.
network CDR data. This method further identifies the area where However, entropy-based analysis necessarily misses relevant
the exception occurred after detection, which helps to take information among the distribution of traffic data, inhibiting the
appropriate actions to improve the quality of service. Then, the effects of anomalies in many cases. Recently, Fiadino et al. [27]
original CDR data are cleaned based on the detection results to have challenged the feasibility of the entropy-based methods in
obtain anomaly-free data to conduct NN training. The mean square detection and diagnosis of traffic anomalies in the cellular network.
error results show that the NN model with stronger learning ability In particular, through the detection and diagnosis of macro-scale
could be trained based on anomaly-free data. Finally, the work traffic anomalies caused by smartphone devices and specific over
further predicts the traffic of users through an ARIMA model to the top services and in the actual cellular network, Fiadino et al.
further improve the service quality. [27] found that entropy-based anomaly detection tends to errors
In the traditional method based on the hard decision, the state of and ignores the typical characteristics of traffic anomalies. They
the cellular network is directly identified as normal or abnormal. concluded that the capabilities of change detection can be
As an improvement, Qin et al. [40] propose a soft decision improved with full statistical data, such as empirical probability
algorithm based on clustering, which can distinguish different distributions.
levels of anomalies and thus make the detection results more Zhang et al. [43] applied a cross entropy model to detect
accurate. Specifically, three network quality parameters, QV, Drop abnormal behaviours of offline users in the wireless network.
Rate (DR), and Wireless Access (WA), which, respectively, reflect Theoretically, cross entropy is a measure of similarity between two
network accessibility, maintainability, and integrity are selected as probability distributions q and p, and the calculation formula is
synthetical QoE (SQoE) key quality indicators (KQIs). A mixed expressed as H(p, q) = − ∑x p(x)log q(x). In this work,
clustering algorithm, combined with SOM and k-medoids, is membership function in fuzzy mathematics theory is used to
adopted to make soft decisions on KQIs. After that, k-medoids is optimise the cross entropy model, which improves the accuracy of
adopted to cluster the abnormal reason parameter represented KPIs abnormal detection of the cross entropy model and reduces its error
(rKPIs) corresponding to each network exception. With this detection rate, so that it can be applied to various scenarios such as
framework, the anomaly can be detected online in the cellular office, campus, and hotel. The effectiveness and feasibility of this
network, forming SQoE-driven anomaly detection and cause method are verified by the experimental results of the optimised
location system. cross entropy anomaly detection of Hadoop pre-processed data.
Different from the role of clustering in [38–40], clustering in Another entropy-based model, maximum entropy model, is also
[41, 42] serves as a preprocessing step for further anomaly used to detect network traffic anomalies of a large number of end-
analysis. The proposed method in [41] performs clustering on users. Coluccia et al. [44] proposed a distributed network anomaly
similar BSs, i.e. spatial aggregation. With the clustering results, the detection method. This method detects the changes of each traffic
periodicity of parameters is highlighted. Compared with the variable on different time scales, generates a discrete distribution
traditional detection in settled space such as BS and in equal time grid composed of variable and time, and then obtains the statistical
bins such as hourly aggregation, the proposed spatial aggregation distribution histogram of past behaviours on each time series. To
method is proven to be more efficient. Similar to the previous detect anomalies in each time series, using the generalised
method, the method proposed in [42] also uses clustering as a likelihood ratio test method, the consistency of the observed values
pretreatment step for SMS anomaly detection in cellular network of each sample mapped to several maximum entropy model
machine-to-machine (M2M) communication. The preprocessing parameters is measured. This method can be implement to perform
steps cluster M2M devices according to their communication types, large-scale end-user distributed traffic anomaly detection.
and then detect distributed denial of service attacks and system
failures through contact-based and volume-based anomaly
detection methods and automatically determined the causes of
3 Research challenges arising from big data
anomalies. The effectiveness of this algorithm has been verified by The explosively increasing data deluging from the wireless
the actual SMS traffic of M2M equipment. network consisting of tens of millions of mobile phones and other
devices gives rise to significant challenges on data acquisition,
2.3 Information-theoretic methods representation, storage, management, analysis, and visualisation.
There exist some literature discussing the obstacles in developing
Information-theoretic measures have been used for feature big data applications, including the data representation to energy
extraction leading to information-theoretic approaches for anomaly management [45]. Besides those summarised in the literature,
detection. Typical information-theoretic measures include entropy, developing big data applications for anomaly detection in wireless
relative entropy, conditional entropy, information cost, and networks faces some unique challenges, as listed in the sequel.
information gain. These measures can be used to describe the
characteristics of a given traffic data. • Data encryption. Wireless operators often collect measurement
Among the information-theoretic measures, entropy is a report, CDR, and a variety of traffic counters, termed as key
fundamental concept in information theory. The entropy of a indicators of performance to monitor the performance of a wireless
n
random variable can be defined as F(H) = ∑i = 1 p(hi)log(1/ p(hi)), network and detect any anomalies that may have an impact on the
in which H takes value hi with probability p(hi), i = 1, 2, …, n. It is user experience. These data sets are usually acquired by deep
readily known that the entropy value is smaller when the packet inspection (DPI), which examines the packet header as well
probability distribution is skewer while it is larger when the as the data part of the packet. For instance, an internet protocol (IP)
distribution is more even (the maximum entropy value is log n), i.e. packet is associated with source port, source address, protocol
the entropy can measure the dispersion of a distribution. With the types, destination port, as well as the destination address. Also,
entropy time-series of network measurement or signal (e.g. KPI DPI can classify the traffic IP types by looking into the packet
data), any adaptive-threshold simple detector (e.g. exponential headers. However, internet traffic data is increasingly encrypted,

IET Commun., 2019, Vol. 13 Iss. 20, pp. 3351-3359 3355

© The Institution of Engineering and Technology 2019
 17518636, 2019, 20, Downloaded from https://ietresearch.onlinelibrary.wiley.com/doi/10.1049/iet-com.2019.0765, Wiley Online Library on [05/02/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
making the DPI inapplicable in classifying the traffic types and detection, we compare it with the fingerprint of KPI related to
collecting KPIs of the wireless network. Therefore ML and data abnormal KQI in the established fingerprint database, then a series
mining techniques are needed to monitor the performance of a of potential root causes were determined. In what follows, the
wireless network in the absence of KPIs. DNA procedure is detailed.
• Data velocity. In addition to the data from mobile phones, Cisco
estimates there will be about 50 billion connected IoT devices by 4.1 Anomaly detection
2020. Such a massive number of IoT devices will inevitably
generate a plethora of data on an unprecedented scale. Moreover, We here present the anomaly detection module which includes
for mission-critical applications enabled by wireless three steps. The first step is data preprocessing for data cleaning,
communications, such as self-driving cars, augmented reality the second step is the key step which performs anomaly detection
robotic surgery, any network anomaly that may affect the service to generate a list of anomalies, and the final step is post-processing
quality should be detected and analysed timely. Also, actions need using a bunch of filters to improve the detection results. The details
to be taken to guarantee the quality of service including the delay are explained in the following.
and throughput. Such real-time requirement calls for the real-time
big data platform that can acquire, process, and detect the 4.1.1 Data preprocessing: The various KQI/KPI data sets
anomalies on the fly with extremely short time delay. required for DNA work are collected by a large number of cells
• Data heterogeneity. The data sources in the wireless network are over a while, such as one month. KQI/KPI values are collected
heterogeneous. Also, the data sets are often unstructured or semi- from the cells at pre-defined intervals, such as 5 min or 10 min etc.
structured, which cannot be managed by the traditional relational Since the original data usually has missing data and extreme data
database management system (RDBMS). As a remedy, NoSQL (which may be anomalies), data cleaning is necessary for the
database technologies, such as Mongodb and Cassandra, have been anomaly detection module. Considering that it is difficult to
developed. Compared to the traditional RDBMS, the NoSQL recover the missing values, the selector is used to pick out valid
databases offer scalability and replicability and thus are more KQI/KPI records. Also, the extreme values are excluded from our
efficient in storing and processing heterogeneous and rapid- training set but they will be included in the procedure of anomaly
changing data sets. Consequently, NoSQL databases are more detection.
suitable for real-time data analysis for big data.
• Data distributiveness. The wireless network is a distributed 4.1.2 Anomaly detector: Our anomaly detection algorithm runs
infrastructure. For example, the 4G cellular network typically the following three phases.
consists of mobile management entity, evolved node B, packet data Phase 1: splitting data set. The significance of data set splitting
network gateway, serving gateway etc. A variety of probes and is to divide the existing KQI/KPI data set into a training data set
sensors are placed across the entire network. The monitoring data Dtraining and testing data set Dtesting, in which two strategies are
is generated distributively. Today's service quality management considered: one shot strategy and slide-window strategy. The
(SQM) architecture aggregates all the data to a single point and former one simply slices the data set into training and test sets,
processes them together. Such an approach, however, may become while the sliding-window strategy means that the window slides as
difficult to realise or infeasible in the future due to the new data arrives and the oldest data is deleted from the window.
exponentially growing size and scale of the network. Hence, how Phase 2: training model. Following the previous step, we will
to carry out anomaly detection in a distributed way to minimise the train a model using the training set Dtraining. For efficiency, we
information exchange across different network elements is a big adopt the univariate anomaly detection (UAD) model in the
challenge. statistical model. The characteristic of the UAD model is that the
• Energy management. The storage, management, and processing parameter Top-Bottom-Percentile (TBP) representing the upper
of the plethora of network data inevitably incur significant energy and lower percentiles is taken as the parameter, also the threshold
consumption. The energy management problem will become is computed according to
increasingly challenging with the increase of the data volume as
well as various analytical demands. Therefore, it is critical to Dmax = quantile(Dtraining, TBP)
establish a system-level power management mechanism for big
data processing in wireless networks. Particularly, for latency KQI and throughput KQI, the top
percentile and the bottom percentile are, respectively, used as
4 Anomaly detection method using big data thresholds for anomaly detection.
analytics Phase 3: detecting anomaly. Next, the testing data set is
checked for anomaly detection. For those data recorded over the
As a potential solution for AD with big data, we here present a same period of time in the date set, we use the threshold value Dmax
simple yet effective AD method using big data analytics [46]. trained in phase 2 to find anomaly points and treat them as outliers.
Compared with the existing AD methods, we detect abnormal This process will repeat at any time.
values of key quality indicator (KQI) so as to monitor performance
regression of the communication network, which often indicates 4.1.3 Postprocessing: Since there often exist outliers that do not
poor user experience. Furthermore, we take a further step to represent network performance degradation, the anomaly detector
identify the corresponding root cause, i.e. perform root cause generates a lot of false anomalies. To lower the false anomaly, we
analysis (RCA), for anomaly occurrence which is a less explored design a bunch of filters that are detailed in the sequel.
area. Our goal is to use the big data analysis method to develop the Threshold filter: When the network has a light traffic load, KQI
general framework for the root causes analysis of KQI degradation. values, e.g. throughput, are always good. These values may be
The general framework developed here is termed as a deep detected as anomalies especially when the training data set Dtraining
network analyser (DNA), which is implemented in big data
is insufficient. To avoid such false anomaly, the threshold filter
platform (Spark) and tested utilising actual production data
adopted in the global scope sets a threshold learned from historic
including KQIs and KPIs. Particularly, the proposed framework is
data to eliminate the insignificant anomalies.
composed of two modules, named fingerprint learning (FL) module
Count filter: Those aforementioned KQIs/KPIs are averaged
and the anomaly detection and diagnosis module. For the previous
over a given time period, e.g. one hour. When we have only a few
module FL, a rare association rule (RAR) mining method that was
sessions in some hours, the corresponding KQI values are poor and
rarely used in previous research was employed to study the
maybe detected as anomalies. To remove this type of false
relationship between KPIs and KQIs. The method learns rules from
anomaly, a kind of counting filter is designed to filter the hours
historical data, which will be used in the next to build a fingerprint
which do not have enough sessions.
database for further root cause analysis. Then the statistical
Daily statistics filter: KQI values would degrade over
learning method is used for the anomaly detection of the input KQI
consecutive hours when some network problem occurs. For
data set. For the KQI anomaly detected in the above anomaly

3356 IET Commun., 2019, Vol. 13 Iss. 20, pp. 3351-3359

© The Institution of Engineering and Technology 2019
 17518636, 2019, 20, Downloaded from https://ietresearch.onlinelibrary.wiley.com/doi/10.1049/iet-com.2019.0765, Wiley Online Library on [05/02/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
Table 1 Precision and recall of AD results method to discover association rules from KPIs to the KQI
KQI Precision Recall anomaly.
KQI1 0.9 0.81 RARM is an improved FP-growth algorithm designed based on
our requirements. In RARM, holding and upgrade are both used to
KQI2 0.7 1.0
appraise the statistical meaning of one association rule with an
adjustable threshold T upgrade, and find out the relationships between
anomaly KQIs (such as KQI outliers) and KPIs. Hence, as a pre-
Table 2 Example of association rule possing step, RARM eliminates the KQI items which are not
Rule 1 Rule 2 exceptional KQI items. In addition, similar to the FP-growth
antecedent 3K ≤ KPI1 ≤ 68K 0.1M ≤ KPI2 ≤ 0.15M approach, RARM constructs an FP-growth tree by scanning the
consequent KQI ≤ 96K KQI ≤ 96K entire database twice. When the construction of the FP-growth tree
support 5113 4497
is finished, the bottom-up algorithm will be leveraged to discover
those high-frequency items, the number of which is beyond the
confidence 0.65 0.57
threshold we define. It must be noted that the frequent item sets
lift 3.26 2.86 including one KPI and one KQI can be extracted from the FP-
growth tree. With regard to each item set that we extract from the
tree, we can get the KPI item set by removing the KQIs. Then with
instance, network traffic fluctuation may result in occasional the KPI items, the FP-tree can be revisited to get the holding of the
outliers. We designed a daily statistics filter, which defines a KPI item set. supp(itemset)/supp(KPIitemset) represents the trust
threshold p, then calculates the number of outliers every day, and of the rule while dividing the trust by the consequent non-
uses the threshold to pick out the dates when the number of outliers conditional probability represents the upgrade. Rules related to
is greater than p. Finally, the outliers that are not in these dates are KQI outliers can be mined by RARM.
removed to avoid false anomaly.
Magnitude filter: In actual engineering applications, it is 4.2.2 Fingerprint matching: The association rules extracted from
necessary to further eliminate the non-significant anomalies. In the historical data by RARM are stored as fingerprints in the
order to meet this demand, we design the magnitude filter. We knowledge database. Each fingerprint is identified by domain
define mag(a) = (a − μ)/σ as the range, where KQI/KPI is experts for the root cause. Once an anomaly of KQI is detected, we
expressed as a, meanwhile the corresponding standard deviation use the cosine similarity criterion to calculate every fingerprint
and mean are expressed as σ and μ, respectively. Note that mag(x) stored in the database to match the corresponding abnormal
represents the level of outliers of x. With this magnitude, the filter fingerprint (KPIs). Finally, the K-nearest neighbour algorithm is
defines a threshold T mag to distinguish between other types of employed to determine the main source of detected KQI anomalies.
anomalies and significant anomalies.
4.3 Case study
4.2 Root cause analysis
To validate the proposed method, a KQI/KPI data set with two
For efficient operation of cellular networks, it is considerable to KQIs (KQI1, KQI2) and 11 KPIs is used for performance
effectively supervise the network and find out the source which assessment. Collected from major network operators, the data set
causes network performance degradation when an anomaly is includes millions of session records in a few weeks. DNA is
detected. Therefore, beyond anomaly detection, we also study a conducted using R language and it can be tested on a workstation
general method to find the correlation between KPI anomalies and equipped with 64 Gb memory, an 8-core CPU, and 1 Tb hard disk.
KQI, which will help to identify the root causes of the degradation Based on the ground truth provided by the domain experts, we
of KQI performance in a cellular network. The proposed approach measured the recall and accuracy of AD results using our method.
consists of two parts, one is a fingerprint generation, the other is From Table 1 we can see that our AD method achieves great
fingerprint matching, which is detailed in the sequel. performance among accuracy and recall on all KQIs, and this
validates the effectiveness and superiority of the AD approach.
4.2.1 Fingerprint generation: Data sets are collected from the Table 2 shows a rule mined from the data set. This rule includes
cellular network, which is composed of KQI data (throughput, the consequent (right-hand side), the antecedent (left-hand side),
latency etc.) and a set of corresponding KPIs data. These KQIs/ and some associated statistics of this rule, including support,
KPIs take either discrete or continuous values. Hence, we quantise confidence and lift. With such a rule, poor performance can be
these values using some discretisation algorithms. The root reason explained, e.g. KQI1 ≤ 96K occurs in the system.
of KQI outliers is analysed by using quantisation value and The KQI anomaly can be defined as hit-anomaly if it is
association rule mining. The corresponding statistical measures associated with at least one fingerprint. The term hit ratio is further
describing the quality and strength of the association rule are defined to describe the ratio of the hit-anomaly presented earlier.
expressed as follows: Fig. 5 illustrates the performance of the magnitude filters with
various magnitude thresholds. Domain experts also use domain
[vp1, vp2] ⟹ vq(k), knowledge to analyse the root causes and provide corresponding
labels. The red line represents the hit ratio of the rare association
Holding = 0.6, Trust = 0.8, Upgrade = 2.5 rules and the blue one shows the hit ratio of the engineer labelled
results. It is observed clearly from the plot that the hit ratio can be
vp(k) and vq(k) represent the quantised KPI and KQI over the kth improved significantly by the magnitude filter. Moreover, by
period of time. The association rule states the correlation between comparing the list of root causes provided by our method for each
vp1, vp2, and vq(k). From the formula, we can see that if vp1, vp2 KQI anomaly with the labels generated by the domain expert, we
appear in a grouping, the KQI vq(k) will appear with an 80% find that our method can achieve more than 90% of precision in
probability, called a transaction. The holding value shows that the root cause analysis.
grouping of vp1, vp2 and vq(k) occupies 60% of all transactions. The
upgrade measures the rate between the probability and the trust that 5 Conclusions
vq(k) occurs. When the lift is large, that means when vp1, vp2 are This study summarised the state-of-the-art anomaly detection
present, vq(k) is more likely to occur. methods in cellular networks and the corresponding challenges
Since abnormal KQI is rare, the traditional association rule arising from big data. A new anomaly detection method based on
learning method, e.g. the FP-growth algorithm [47] proposed for the analysis of big data in cellular networks was presented.
frequent item mining does not apply to the problem under Experimental results indicate that DNA achieves excellent
consideration. Therefore, we present a RAR mining (RARM)

IET Commun., 2019, Vol. 13 Iss. 20, pp. 3351-3359 3357

© The Institution of Engineering and Technology 2019
 17518636, 2019, 20, Downloaded from https://ietresearch.onlinelibrary.wiley.com/doi/10.1049/iet-com.2019.0765, Wiley Online Library on [05/02/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
Telecommunications and Information Technology, Krabi, Thailand, May
2008, pp. 361–364
[16] Hussain, B., Du, Q., Ren, P.: ‘Semi-supervised learning based big data-driven
anomaly detection in mobile wireless networks’, China Commun., 2018, 15,
(4), pp. 41–57
[17] Abdelrahman, O.H., Gelenbe, E.: ‘A data plane approach for detecting control
plane anomalies in mobile networks’, Int. Internet Things Summit, 2015, 360,
(1), pp. 210–221
[18] Mamun, A., Abdullah, S.M., Beyaz, M.: ‘LSTM recurrent neural network
(RNN) for anomaly detection in cellular mobile networks’. Int. Conf. on
Machine Learning for Networking, Paris, 2018, pp. 222–237
[19] Xie, M., Hu, J., Han, S., et al.: ‘Scalable hypergrid k-NN-based online
anomaly detection in wireless sensor networks’, IEEE Trans. Parallel Distrib.
Syst., 2013, 24, (8), pp. 1661–1670
[20] Ciocarlie, G., Lindqvist, U., Nitz, K., et al.: ‘DCAD: dynamic cell anomaly
detection for operational cellular networks’. Proc. 2014 IEEE Network
Operations and Management Symp. (NOMS), Krakow, Poland, May 2014,
pp. 1–2
[21] Ciocarlie, G., Lindqvist, U., Nitz, K., et al.: ‘On the feasibility of deploying
cell anomaly detection in operational cellular networks’. Proc. 2014 IEEE
Network Operations and Management Symp. (NOMS), Krakow, Poland, May
2014, pp. 1–6
[22] Casas, P., Vanerio, J.: ‘Super learning for anomaly detection in cellular
networks’. Proc. 2017 IEEE 13th Int. Conf. on Wireless and Mobile
Fig. 5 Hit ratio of the RCA algorithm after applying magnitude filters with Computing, Networking and Communications (WiMob), Rome, Italy, 2017,
different thresholds pp. 1–8
[23] Vanerio, J., Casas, P.: ‘Ensemble-learning approaches for network security
and anomaly detection’. Proc. The Workshop on Big Data Analytics and
performance in not only anomaly detection but also root cause Machine Learning for Data Communication Networks, Los Angeles, CA,
analysis for performance monitoring of a large cellular network. USA, 2017, pp. 1–6
[24] Lakhina, A., Crovella, M., Diot, C.: ‘Mining anomalies using traffic feature
distributions’. Proc. SIGCOMM Computer Communication Review,
6 Acknowledgments Philadelphia, PA, August 2005, vol. 35, no. 4, pp. 0146–4833
[25] Szilagyi, P., Novaczki, S.: ‘An automatic detection and diagnosis framework
This work was supported in part by the National Key Research and for mobile communication systems’, IEEE Trans. Netw. Serv. Manage., 2012,
Development Project under grant no. 2017YFE0119300, the 9, (2), pp. 184–197
National Natural Science Foundation of China under grant nos. [26] Sun, W., Qin, X., Shuang, T., et al.: ‘A QoE anomaly detection and diagnosis
61936014 and 61901302, and the Open Research Fund from the framework for cellular network operators’. Proc. 2015 IEEE Conf. on
Computer Communications Workshops (INFOCOM WKSHPS), Hong Kong,
Shandong Provincial Key Laboratory of Wireless Communication China, April 2015, pp. 450–455
Technologies (no. SDKLWCT-2019-02). [27] Fiadino, P., D'Alconzo, A., Schiavone, M., et al.: ‘Challenging entropy-based
anomaly detection and diagnosis in cellular networks’. Proc. 2015 ACM
Conf. on Special Interest Group on Data Communication, New York, USA,
7 References 2015, pp. 87–88
[28] Huang, Q., Lee, P.P.C.: ‘LD-sketch: a distributed sketching design for
[1] Casas, P., D'Alconzo, A., Fiadino, P., et al.: ‘Detecting and diagnosing
accurate and scalable anomaly detection in network data streams’. Proc. IEEE
anomalies in cellular networks using random neural networks’. Proc. 2016
Conf. on Computer Communications (INFOCOM 2014), Toronto, Canada,
Int. Wireless Communications and Mobile Computing Conf. (IWCMC),
April 2014, pp. 1420–1428
Cyprus, Paphos, September 2016, pp. 351–356
[29] Zhang, Y., Calyam, P., Debroy, S., et al.: ‘PCA-based network-wide
[2] Ciocarlie, G.F., Lindqvist, U., Novaczki, S., et al.: ‘Detecting anomalies in
correlated anomaly event detection and diagnosis’. Proc. 2015 11th Int. Conf.
cellular networks using an ensemble method’. Proc. 2013 9th Int. Conf. on
on the Design of Reliable Communication Networks (DRCN), Kansas City,
Network and Service Management (CNSM), Zürich, Switzerland, October
USA, March 2015, pp. 149–156
2013, pp. 171–174
[30] Zhang, Y., Debroy, S., Calyam, P.: ‘Network-wide anomaly event detection
[3] Fiadino, P., D'Alconzo, A., Schiavone, M., et al.: ‘RCATool – a framework
and diagnosis with perfSONAR’, IEEE Trans. Netw. Serv. Manage., 2016, 13,
for detecting and diagnosing anomalies in cellular networks’. Proc. 2015 27th
(3), pp. 666–680
Int. Teletraffic Congress, Ghent, September 2015, pp. 194–202
[31] Kim, H., Lee, S., Ma, X., et al.: ‘Higher-order PCA for anomaly detection in
[4] Zhang, R., Song, L., Han, Z., et al.: ‘Physical layer security for two-way
large-scale networks’. Proc. 2009 3rd IEEE Int. Workshop on Computational
untrusted relaying with friendly jammers’, IEEE Trans. Veh. Technol., 2012,
Advances in Multi-Sensor Adaptive Processing (CAMSAP), Aruba,
61, (8), pp. 3693–3704
Netherlands, December 2010, pp. 85–88
[5] Zhang, R., Cheng, X., Yang, L.: ‘Cooperation via spectrum sharing for
[32] Lathauwer, L.D., Moor, B.D., Vandewalle, J.: ‘A multilinear singular value
physical layer security in device-to-device communications underlaying
decomposition’, SIAM J. Matrix Anal. Appl., 2000, 21, (4), pp. 1253–1278
cellular networks’, IEEE Trans. Wirel. Commun., 2016, 15, (8), pp. 5651–
[33] Lathauwer, L.D., Moor, B.D., Vandewalle, J.: ‘On the best Rank-1 and Rank-
5663
(R1,R2,. . .,RN) approximation of higher-order tensors’, SIAM J. Matrix Anal.
[6] Papadopoulos, S., Drosou, A., Tzovaras, D.: ‘A novel graph-based descriptor
Appl., 2000, 21, (4), pp. 1324–1342
for the detection of billing-related anomalies in cellular mobile networks’,
[34] Livani, M.A., Abadi, M.: ‘Distributed PCA-based anomaly detection in
IEEE Trans. Mob. Comput., 2016, 15, (11), pp. 2655–2668
wireless sensor networks’. Proc. 2010 Int. Conf. for Internet Technology and
[7] Zhang, D., Zhao, S., Yang, L.T., et al.: ‘NextMe: localization using cellular
Secured Transactions, London, United Kingdom, November 2010, pp. 1–8
traces in internet of things’, IEEE Trans. Ind. Inf., 2015, 11, (2), pp. 302–312
[35] Bai, Z.J., Chan, R.H., Luk, F.T.: ‘Principal component analysis for distributed
[8] Karatepe, I.A., Zeydan, E.: ‘Anomaly detection in cellular network data using
data sets with updating’, Lect. Notes Comput. Sci., 2005, 3756, pp. 471–483
big data analytics’. 2014 20th Proc. European Wireless Conf., Barcelona,
[36] Liu, Y., Zhang, L., Guan, Y.: ‘Sketch-based streaming PCA algorithm for
Spain, May 2014, pp. 1–5
networkwide traffic anomaly detection’. Proc. 2010 IEEE 30th Int. Conf. on
[9] Sfar, H., Bouzeghoub, A., Raddaoui, B.: ‘Early anomaly detection in smart
Distributed Computing Systems, Genoa, Italy, June 2010, pp. 807–816
home: a causal association rule-based approach’, Artif. Intell. Med., 2018, 91,
[37] Callegari, C., Gazzarrini, L., Giordano, S., et al.: ‘A novel PCA-based
pp. 57–71
network anomaly detection’. Proc. 2011 IEEE Int. Conf. on Communications
[10] Mavroeidis, V., Vishi, K., Jøsang, A.: ‘Framework for data-driven physical
(ICC), Kyoto, Japan, June 2011, pp. 1–5
security and insider threat detection’. Proc. 2018 IEEE/ACM Int. Conf. on
[38] Parwez, M.S., Rawat, D.B., Garuba, M.: ‘Big data analytics for user activity
Advances in Social Networks Analysis and Mining (ASONAM), Barcelona,
analysis and user anomaly detection in mobile wireless network’, IEEE Trans.
Spain, 2018, pp. 1108–1115
Ind. Inf., 2017, 13, (4), pp. 2058–2065
[11] Duffield, N., Haffner, P., Krishnamurthy, E., et al.: ‘Rule-based anomaly
[39] Kashif, S., Hazrat, A., Zhongshan, Z.: ‘Call detail records driven anomaly
detection on IP flows’. Proc. INFOCOM, Rio de Janeiro, Brazil, April 2009,
detection and traffic prediction in mobile cellular networks’, IEEE Access,
pp. 424–432
2018, 6, pp. 1–1
[12] Xu, J., You, J., Liu, F.: ‘A fuzzy rules based approach for performance
[40] Qin, X., Tang, S., Chen, X., et al.: ‘SQoE KQIs anomaly detection in cellular
anomaly detection’. Proc. 2005 IEEE Networking, Sensing and Control,
networks: fast online detection framework with hourglass clustering’, China
Tucson, AZ, USA, March 2005, pp. 44–48
Commun., 2018, 15, (10), pp. 25–37
[13] Hassanzadeh, R., Nayak, R.: ‘A rule-based hybrid method for anomaly
[41] Yang, D., Miao, D., Qin, X., et al.: ‘A novel anomaly detection with temporal
detection in online-social-Network graphs’. Proc. IEEE Int. Conf. on Tools
and spatial aggregation in mobile networks’. Proc. 2016 8th Int. Conf. on
with Artificial Intelligence, Washington, USA, November 2013, pp. 351–357
Wireless Communications & Signal Processing (WCSP), Yangzhou, Jiangsu,
[14] Gath, I., Geva, A.B.: ‘Unsupervised optimal fuzzy clustering’, IEEE Trans.
China, October 2016, pp. 1–5
Pattern Anal. Mach. Intell., 1989, 11, (7), pp. 773–780
[42] Murynets, I., Jover, R.P.: ‘Anomaly detection in cellular machine-to-machine
[15] Sukkhawatchani, P., Usaha, W.: ‘Performance evaluation of anomaly
communications’. Proc. 2013 IEEE Int. Conf. on Communications (ICC),
detection in cellular core networks using self-organizing map’. Proc. 2008 5th
Budapest, Hungary, June 2013, pp. 2138–2143
Int. Conf. on Electrical Engineering/Electronics, Computer,

3358 IET Commun., 2019, Vol. 13 Iss. 20, pp. 3351-3359

© The Institution of Engineering and Technology 2019
 17518636, 2019, 20, Downloaded from https://ietresearch.onlinelibrary.wiley.com/doi/10.1049/iet-com.2019.0765, Wiley Online Library on [05/02/2024]. See the Terms and Conditions (https://onlinelibrary.wiley.com/terms-and-conditions) on Wiley Online Library for rules of use; OA articles are governed by the applicable Creative Commons License
[43] Zhang, C., Hu, Y., Zhu, X., et al.: ‘Anomaly detection for user behavior in [45] Labrinidis, A., Jagadish, H.V.: ‘Challenges and opportunities with big data’,
wireless network based on cross entropy’. 2015 IEEE 12th Int. Conf. on VLDB Endowment, 2012, 5, (12), pp. 2032–2033
Ubiquitous Intelligence and Computing and 2015 IEEE 12th Int. Conf. on [46] Yang, K., Liu, R., Sun, Y., et al.: ‘Deep network analyzer (DNA): a big data
Autonomic and Trusted Computing and 2015 IEEE 15th Int. Conf. on analytics platform for cellular networks’, IEEE Internet Things J., 2017, 4,
Scalable Computing and Communications and Its Associated Workshops (6), pp. 2019–2027
(UIC-ATC-ScalCom), Beijing, China, August 2016, pp. 1258–1263 [47] Han, J., Pei, J., Yin, Y.: ‘Mining frequent patterns without candidate
[44] Coluccia, A., D'Alconzo, A., Ricciato, F.: ‘Distribution-based anomaly generation’. SIGMOD Record, Dallas, Texas, USA, May 2000, vol. 29, no. 2,
detection via generalized likelihood ratio test: a general maximum entropy pp. 1–12
approach’, Comput. Netw., 2013, 57, (17), pp. 3446–3462

IET Commun., 2019, Vol. 13 Iss. 20, pp. 3351-3359 3359

© The Institution of Engineering and Technology 2019