0% found this document useful (0 votes)

103 views

Class Imbalance Paper

This article compares different oversampling techniques for handling class imbalance problems in predictive models, using a customer churn prediction case study. It surveys six sampling techniques: Mega-Trend Diffusion Function (MTDF), Synthetic Minority Oversampling Technique (SMOTE), Adaptive Synthetic Sampling approach (ADASYN), Couples Top-N Reverse k-Nearest Neighbor (TRkNN), Majority Weighted Minority Oversampling Technique (MWMOTE), and Immune centroids oversampling technique (ICOTE). Empirical results on publicly available datasets show that MTDF and rules generated using genetic algorithms performed best compared to other oversampling methods and rule generation algorithms at improving predictive performance on imbalanced data

Uploaded by

falconau

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

103 views

Class Imbalance Paper

Uploaded by

falconau

Available Formats

Download as PDF, TXT or read online on Scribd

You are on page 1/ 18

This article has been accepted for publication in a future issue of this journal, but has not been

fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2016.2619719, IEEE Access

JOURNAL OF IEEEACCESS 1

Comparing oversampling techniques to handle the

class imbalance problem: A customer churn
prediction case study
Adnan Amin, Sajid Anwar, Awais Adnan, Muhammad Nawaz, Newton Howard, Junaid Qadir, Ahmad Hawalah,
Amir Hussain

Abstract—Customer retention is a major issue for vari- [1]). Churn-prone industries such as the telecommunication
ous service-based organizations particularly telecom industry, industry typically maintain customer relationship management
wherein predictive models for observing the behavior of cus- (CRM) databases that are rich in unseen knowledge and
tomers is one of the great instruments in customer retention
process and inferring the future behavior of the customers. How- certain patterns that may be exploited for acquiring customer
ever, the performances of predictive models are greatly affected information on time for intelligent decision-making practice
when the real-world dataset is highly imbalanced. A dataset is of an industry [2].
called imbalanced if the samples size from one class is very much However, knowledge discovery in such rich CRM databases,
smaller or larger than the other classes. The most commonly which typically contains thousands or millions of customers’
used technique is over/under sampling for handling the class-
imbalance problem (CIP) in various domains. In this study, we information, is a challenging and difficult task. Therefore,
survey six well-known sampling techniques and compare the many industries have to inescapably depend on prediction
performances of these key techniques i.e., Mega-trend Diffusion models for customer churn if they want to remain in the
Function (MTDF), Synthetic Minority Oversampling Technique competitive market [1]. As a consequence, several competi-
(SMOTE), Adaptive Synthetic Sampling approach (ADASYN), tive industries have implemented a wide range of statistical
Couples Top-N Reverse k-Nearest Neighbor (TRkNN), Majority
Weighted Minority Oversampling Technique (MWMOTE) and and intelligent machine learning (ML) techniques to develop
Immune centroids oversampling technique (ICOTE). Moreover, predictive models that deal with customer churn [2].
this study also reveals the evaluation of four rules-generation al- Unfortunately, the performance of ML techniques is consid-
gorithms (the Learning from Example Module, version 2 (LEM2), erably affected by the CIP. The problem of imbalanced dataset
Covering, Exhaustive and Genetic algorithms) using publicly appears when the proportion of majority or negative class has
available datasets. The empirical results demonstrate that the
overall predictive performance of MTDF and rules-generation a higher ratio than positive or minority class [3], [4]. The
based on Genetic algorithms performed best as compared to the skewed distribution (imbalanced) of data in the dataset poses
rest of the evaluated oversampling methods and rule-generation challenges for machine learning and data mining algorithms
algorithms. [3], [5]. This is an area of research focusing on skewed class
Index Terms—SMOTE, ADASYN, Mega Trend Diffusion distribution where minority class is targeted for classification
Function, Class Imbalance, Rough Set, Customer Churn, mRMR. [6]. Consider a dataset where the imbalance ratio is 1:99 (i.e.,
ICOTE, MWMOTE, TRkNN. where 99% of the instances belong to the majority class, and
1% belongs to the minority class). A classifier may achieve the
I. I NTRODUCTION accuracy up to 99% just by ignoring that 1% of minority class
instances—however, adopting such an approach will result in

I N many subscription-based service industries such as

telecommunications companies, are constantly striving to
recognize customers that are looking to switch providers (i.e.,
the failure to correctly classify any instances of the class of
interest (often the minority class).
It is well known that churn is a rare object in service-
Customer churn). Reducing such churn is extremely important based industries, and that misclassification is more costly
in such competitive markets since acquiring new customers in for rare objects or events in the case of imbalance datasets
such markets is very difficult (attracting non-subscribers can [7]. Traditional approaches therefore can provide misleading
cost up to six times more than what it costs to retain the current results on the class-imbalanced dataset—a situation that is
customers by taking active steps to discourage churn behavior highly significant and one that occurs in many domains [3],
[6]. For instance, there are a significant number of real-
“a. Adnan Amin, Sajid Anwar, Awais Adnan and Muhammad Nawaz are
affiliated with the Institute of Management Sciences, Peshawar. Pakistan (e- world applications that are suffering from the class imbalance
mail: {adnan.amin, sajid.anwar, awais.adnan, m.nawaz}@imsciences.edu.pk). problem (e.g., medical and fault diagnosis, anomaly detection,
b. Newton Howard is affiliated with Computational Intelligence Lab, Oxford face recognition, telecommunication, the web & email classi-
University (e-mail: [email protected]).
c. Junaid Qadir is with Information Technology University (ITU)-Punjab, fication, ecology, biology and financial services [3], [4], [6]).
Lahore, Pakistan. (e-mail: [email protected]). Sampling is a commonly used technique for handling the
d. Ahmad Hawalah is with Taibah University, Saudi Arabia. (e-mail: CIP in various domains. Broadly speaking, sampling can be
[email protected]).
e. Amir Hussain is with Department of Computing Science and Maths, categorized into oversampling and undersampling. A large
University of Stirling, Scotland, UK. (e-mail: [email protected])” number of studies focused on handling the class imbalance

2169-3536 (c) 2016 IEEE. Translations and content mining are permitted for academic research only. Personal use is also permitted, but republication/redistribution requires IEEE permission. See
http://www.ieee.org/publications_standards/publications/rights/index.html for more information.
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI 10.1109/ACCESS.2016.2619719, IEEE Access

JOURNAL OF IEEEACCESS 2

problem have been reported in literature [2], [4]. These studies undersampling methods for handling the CIP; however, the
can be grouped into the following approaches based on their proposed study differs from the previous studies in that in
dealing with class imbalance issue: (i) the internal-level: addition to evaluating six oversampling techniques (SMOTE,
construct or update the existing methods to emphasize the sig- ADASYN, MTDF, ICOTE, MWMOTE and TRkNN), we also
nificance of the positive class and (ii) the external level: adding compare the performance of four rules-generation algorithms
data in preprocessing stage where the distribution of class (Exh, Gen, Cov and LEM2). The proposed study is also
is resampled in order to reduce the influence of imbalanced focused on considering the following research questions (RQ):
distribution of class in the classification process. The internal • RQ1: What is the list of attributes that is highly symp-
level approach is further divided into two groups [5]. Firstly, tomatic in the targeted data set for prediction of customer
the cost-sensitive approach, which falls between internal and churn?
external level approaches, is based on reducing incorrect • RQ2: Which of the oversampling technique (e.g. SMOTE,
classification costs for positive or minority class leading to MTDF, ADASYN, ICOTE, MWMOTE and TRkNN) is
reduction of the overall cost for both internal and external level more suitable for creating synthetically samples that not
approaches. Secondly, the ensemble/boosting approach, which only handle the CIP in a dataset of the telecommunication
adopts the use of multiple classifiers to follow the similar idea sector but also improves the classification performance.
adopted by the internal approach. In this study, we have used • RQ3: Which of the rule-generation algorithm (Exh, Gen,
six well-known advanced oversampling techniques—namely, Cov & LEM2) is more suitable using RST based classi-
Mega-trend Diffusion Function (MTDF), Synthetic Minor- fication for customer churn prediction in the imbalanced
ity Oversampling Technique (SMOTE), Adaptive Synthetic dataset.
Sampling approach (ADASYN), Majority Weighted Minor- Remaining paper is organized as follows: the next section
ity Oversampling Technique (MWMOTE), Immune centroids presents the existing work of class imbalance and approaches
oversampling technique (ICOTE) and Couples Top-N Reverse to handle the CIP. The background study and evaluation
k-Nearest Neighbor (TRkNN). measures are explored in section III. The experiments are
SMOTE [6] is commonly used as a benchmark for over- detailed in section IV. The section V explains the results
sampling algorithm [7], [8]. ADASYN is also an important followed by section VI that concludes the paper.
oversampling technique which improves the learning about the
samples distribution in an efficient way [9]. MTDF was first
II. H ANDLING THE C LASS -I MBALANCE P ROBLEM
proposed by Li et al. [10] and reported improved performance
of classifying imbalanced medical datasets [11]. CUBE is also A. Class imbalance problem (CIP)
another advanced oversampling technique [12] but we have In this section, firstly it is explained that the CIP in
not considered this approach since as noted by Japkowicz the context of classification followed by techniques used in
[13], CUBE oversampling technique does not increase the handling the CIP in a dataset and its relationship to potential
predictive performance of the classifier. The above-mentioned domains. This section also contains a brief literature review
oversampling approaches are used to handle the imbalanced on class imbalance/skewed distribution of samples in datasets.
dataset and improve the performance of predictive models Prior to the overview of handling CIP, first, there is a need to
for customer churn, particularly in the telecommunication address the notion of classification. The aim of classification
sector. Rough Set Theory (RST) is applied to the four dif- is to train the classifier on some dataset, making it capable
ferent rule-generation algorithms—(i) Learning from Example to correctly classify the unknown classes of unseen objects
Module, version 2 (LEM2), (ii) Genetic (Gen), (iii) Cov- [4], [17]. If the samples in the dataset are not balanced, there
ering (Cov) and (iv) Exhaustive (Exh) algorithms—in this is a great chance that the classification task will result in
study to observe the behavior of customer churn and all misleading results.
experiments are applied on the publicly available dataset. CIP exists in many real-world classifications including So-
It is specified here that this paper is an extended version cial Network Services [18], [19], [20], [21], [22]], Banks &
of our previous work [14], and makes the following con- Financial Services [16], [23], [24], [25], [26], Credit Card Ac-
tributions: (i) more datasets are employed to obtained more count Services [27], [28], Online Gaming Services [29], [30],
generalized results for the selected oversampling techniques Human Resource Management [31], [32], [33], Discussion &
and the rules-generation algorithms, (ii) another well-known Answer forums [34], Fault Prediction & Diagnosis [35], [11],
oversampling technique—namely, ADASYN—is also used User’s profile personalization [36], Wireless Networks [37],
(iii) detailed analysis and discussion on the performance of [38], 5G future network [39] and Insurance & Subscription
targeted oversampling techniques (namely, ADASYN, MTDF, Services [40], [41], [42]. Considering the scenario of class
SMOTE, MWMOTE, ICOTE and TRkNN) followed by the imbalance in any application domain, almost all the objects
rules-generation algorithms—namely, Gen, Cov, LEM2 and belong to specific class (majority class) and far less number
Exh, and (iv) detailed performance evaluation—in terms of of objects are assigned to other class (minority class) [26].
the balance accuracy, the imbalance ratio, the area under The classification problems observation shows that training
the curve (AUC) and the McNemar’s statistical test—is per- the classifier using conventional classification techniques re-
formed to validate the results and avoid any biases. Many sults on higher performance, but it tends to classify all
comparative studies [2], [7], [11], [15], [16] have already the samples data into the majority class; usually, which is
been carried out on the comparison of oversampling and often not the desired goal of the classification study [26]. In

JOURNAL OF IEEEACCESS 3

contrast, research studies [43], [44] show that latest machine them with the samples population of the majority class
learning techniques result in low performance due to dealing [26]. Classification techniques usually produce a better
with large imbalanced datasets. Also, the class imbalance performance when the samples of both classes are nearly
may cause classification approaches to pass from difficulties equally distributed in the dataset.
in learning, which eventually results in poor classification Figure 2(a) depicts the examples of random ignorance of the
performance [15]. Therefore, learning from imbalanced class majority class sample while figure 2(b) replicate the minority
data has received a tremendous amount of attention from class samples.
the machine learning and data mining research community
[45]. The following figures illustrate difficulties in imbalanced
datasets such as figure 1 (a) describes the overlapping problem
& small disjoints in figure 1(b).

Fig. 1: When overlapping occurs then hard to induce the

discriminative rules [see figure 1 (a)] while the figure 1 (b)
reflects the existence of small disjoints that increases the Fig. 2: (a) Reflects the randomly removes the instances from
complexity due to unbalanced instances. majority class. Figure 2(b) illustrate oversampling method
by replicating the instances of minority class to balance the
1) Approaches for handling the CIP: Handling the CIP is dataset.
one of the most important approaches for dealing with rarity
in sampling. The basic idea of handling CIP is to reduce the
negative class or increase the positive class samples by altering A. Methods for dealing with CIP
the samples in the dataset distribution. Following are the three The methods for dealing with the CIP, as discussed in the
major approaches used for handling CIP: introduction section, can be classified into two groups depend-
• The basic sampling method for under/over-sampling: ing on how these methods are used, for either undersampling
The basic sampling method for under/over-sampling is or oversampling, are the following:
usually referred to a manual approach which is the • Internal (algorithm level) approaches: In this approach,
simplest form to deal with the undersampling or over- the concept is to resample the data through existing
sampling in datasets [2]. In this method, some of the classification algorithms in order to rebalance the class
samples from the negative class are either eliminated or distribution [4], [5]. Where the oversampling approach
the samples of minority class are duplicated to balance increases the minority class by bias data, while eliminat-
the data distribution in the dataset. However, this method ing examples from the majority class is done through un-
for undersampling has a drawback [46], i.e., it discards dersampling approach. The algorithm level methods are
potentially important samples (lack of data) in majority more dependent on problem & require special knowledge
class and thus can receive low classifier’s performance. of targeted classifier(s) along with domain knowledge
On the other hand, oversampling replicates data in the [4], [47]. This method is further divided into two groups
dataset. This does not degrade the classifier’s performance [5]. Firstly, cost-sensitive learning, which is based on
but can usually take more time to train a classifier. reducing the misclassification costs for minority class,
• The advanced sampling method for under/over- and on reducing the total cost errors of both classes.
sampling: The advanced sampling methods may involve Secondly, ensemble or boosting method is based on
some data mining or statistical approach to cut the multiple sets of classifiers (ensemble technique) to handle
samples or combine the undersampling and oversampling the CIP [4].
techniques [2]. There are many intelligence techniques • External (data level) approach: In this approach, a
which are used for handling CIP such as SMOTE[6], preprocessing step is involved in order to reduce the
ADASYN [9], MTDF[10], etc. effect of imbalanced class distribution in the classification
• Random undersampling/oversampling technique: In process. There is no need to have special knowledge of
this technique, randomly removed samples from the classifiers and domain. Therefore, data level approach is
majority class are combined with the minority class usually more versatile. The objective of this approach
samples. While in random oversampling technique, the is to rebalance the skewed distribution of samples by
samples are replicated for random times and combined resampling the data space [48]. Resampling method is

JOURNAL OF IEEEACCESS 4

used to rebalance the distribution of the class data [4], oversampling with undersampling of minority and majority
[5], [47], and therefore avoids the modification of the classes respectively. They performed different experiments
learning algorithm. such as undersampling the data in negative class followed by
The data level approaches are usually considered to preprocess oversampling the data in positive class and finally combining
the data before training the classifier and then include into the oversampled with under-sampled data. The conclusive
learning algorithms. [4], [49]. On the other hand, cost-sensitive results did not show significant improvement. Tang et al. [54],
and internal level methods are depended on the problem and used support vector machine (SVM) and granular computing
require special knowledge of targeted classifier along with the for handling CIP through underdamping technique. They have
domain. Similarly, the cost-sensitive technique has a major removed the noisy data (e.g., redundant data or irrelevant
problem of defining misclassification costs, which are not usu- data) from the dataset while keeping only those samples that
ally known/available in the data space [50]. The comparison have maximum relevant information. They investigated that
between oversampling and undersampling has already been undersampling can significantly increase the classification per-
performed [2] with the conclusion that oversampling performs formance, but it was also observed that random undersampling
best as compared to undersampling for handling CIP. It is also might not provide highly accurate classification. On the other
reported that undersampling technique has a major problem of hand, G. Wu & Chang [55] performed an experiment showing
losing classifier’s performance when some potential samples results weak performance of SVM on the dataset that suffered
are discarded from the majority class [46]. Due to these from CIP. Foster Probost [56] empirically observed that during
motives, our objective is to deeply review the state-of-the- classification process with imbalanced data the number of
art data level methods to address binary CIP. The data level instances of positive class are usually very less. Therefore,
method have following advantages: (i) it is independent of the trained classifiers can accurately recognize the objects of
the obligation to train the classifier; (ii) it is usually used negative class instead of positive class. The reason is that the
in preprocessing data stage of other methods (i.e. ensemble- positive class cannot contribute more as compared to negative
based approach, cost-sensitive level); and (iii) it can be easily class. Due to this reason, the misclassification of instances that
incorporated into other methods (i.e. internal methods) [51]. belong to minority class cannot be reduced in CIP. Batista
et al. [48] introduced a comparative investigation of various
sampling schema (i.e., Edited Nearest Neighbor rule or ENN
B. Techniques for handling CIP and SMOTE) to balance the training-set. They removed the
Kubat & Matwin [52] have addressed the CIP by applying redundant or irrelevant data from the training process, which
undersampling technique for the negative class and kept the improved the mean number of induced rules and increased the
original instances of the positive class. They applied geometric performance of SMOTE+ENN. In connection to this work,
mean (related to ROC Curve) for performance evaluation of another sampling technique was proposed by H. Guo [57] for
classifiers. Burez & Poel [2] reported that random undersam- handling CIP. He modified the existing procedure of DataBoost
pling can improve the prediction accuracy as compared to the which performed much better than SMOTEBoost [49].
boosting techniques but did not help them in their experiments. Haibo He. et al. [9] introduced ADASYN oversampling
However, by randomly oversampling or resampling the minor- algorithm which was an extension of the SMOTE algorithm.
ity class may result in over-fitting. Chawla et al. [6] introduced They reported that ADASYN algorithm can self-decide the
a novel SMOTE considered as widespread technique for over- number of artificial data samples that are required to be pro-
sampling. SMOTE produces new positive observations based duced for minority class. They also investigated that ADASYN
on weighted mean/average of the k-nearest neighbor giving not only provided a balanced data distribution but also forced
positive observations. It reduces the samples inconsistency and the learning algorithm to focus on complex samples in the
creates a correlation between objects of the positive class. The dataset. On the other hand, SMOTE algorithm [6], gener-
SMOTE oversampling technique is experimentally evaluated ated alike numbers of artificial data for minority class while
on a variety of datasets with various levels of imbalance and DataBoost-IM [57] algorithm has generated various weightage
different sizes of data. SMOTE with C4.5 and Ripper algo- for changed minority class samples to compensate for the
rithms, outperformed as compared to Ripper’s Loss Ratio and distribution of skewed data. However, in their study, they have
Naı̈ve Bayes [6], [45]. Wouter Verbeke et al. [15] illustrated shown that ADASYN has produced more efficient results than
an oversampling technique by simply copying the minority SMOTE and DataBoost-IM.
class data and adding it to the training set. They reported that
just oversampling the minority class by same data (i.e. copied III. BACKGROUND : OVERSAMPLING TECHNIQUES AND
EVALUATION METRICS
samples) did not show significant improvement in performance
of the classifier. Therefore, they have suggested using more This section presents a study of six well-known oversam-
appropriate oversampling methods (i.e. SMOTE). On the other pling techniques (i.e., MTDF, SMOTE, ADASYN, MWMOTE
hand, Jo & Japkowicz [3] showed that the decision tree C4.5 and ICOTE), feature selection algorithm (i.e., mRMR) and
and Backpropagation Neural Network (BNN) algorithms both Rough Set Theory (RST).
degrade the performance of classifiers on small and complex
dataset due to class imbalance. They have proposed to use the A. Mega-Trend-Diffusion Function (MTDF)
cluster-based oversampling for a small and complex dataset Der-Chiang Li et al. [10] introduced the MTDF, procedure
with class imbalance dataset. Ling & Li [53] have combined to facilitate the estimation of domain samples systematically. It

JOURNAL OF IEEEACCESS 5

generates artificial data to balance the dataset. In their work, C. mRMR

MTDF is applied for oversampling in order to address the Hanchuan Peng et al. [58], introduced Minimal Redundancy
CIP for customer churn prediction in the targeted domain. Maximal Relevance (mRMR) technique for attributes selection
Due to insufficient data, small samples size or imbalance as per the procedure of the maximal statistical dependency
distribution of class samples provides imprecise information criterion. mRMR algorithm not only minimizes the features
[10]. Therefore, MTDF apply a mutual diffusion function to space by increasing the mutual dissimilarity with the class but
diffuse the data. Let hset , is diffusion coefficient set, i.e. also extracts more appropriate features subset. The set S in
hset = Ŝx2 /n (1) term of relevance of attributes for class c is expressed by the
n
average value of MI between each attribute fi and c as shown
2 X in equation (9):
Ŝ = (Xi − X)2 /n − 1 (2)
x i=1 1 X
D(S, c) = I(f i; c) (9)
Where equation (2) is sample set variance and n is representing |s|
fi ∈S
the size of sample. If the set of the sample is variant, then, i.e.,
The redundancy of all attributes in the set=S is the mean
uset = min+max
2 , The lower and upper boundaries of diffused
value of mutual information between the attributes (features)
sample are;
fi and fj .
1 X
q
a = uset − −2 × Ŝx2 /NL × ln(ϕ(a)) (3) R(S) = 2 I(fi : fj ) (10)
q |S| f i∈S
b = uset + −2 × Ŝx2 /NU × ln(ϕ(b)) (4) The mRMR criterion is a group of two measures given in
NL NU equations (9) and (10), which can be defined as follows (i.e.,
Let SkewL = and SkewU =
(NL +NU ) are the (NL +NU )
p equation (11)):
left and right skewness magnitudes of −2 × h × ln(ϕ(a))  
in order to characterize the asymmetric diffusion. In propose
 
S  1 X 1 X 
study the diffusion function is used and accordingly revised mRM R =  I(f i, c) − 2 I(f i, f j)
max |S| |s| f i,f j∈S
as: f i∈S
r (11)
2 The equation above may then be expressed as an optimiza-
a = uset − SkewL × −2 × Ŝ /NL × ln(ϕ(a) (5)
x tion problem:
$P
n
Pn %
r max i=1 ci xi j=1 aij xi xj
2 mM RM = n Pn − Pn
b = uset − SkewU × −2 × Ŝ /NU × ln(ϕ(b) (6) x ∈ {0, 1} i−1 xi ( i=1 xi )2
x
(12)
Besides these it is important to note that if the variance of
any features for a target class is zero, then the value of NU
D. MWMOTE
and NL will be zero. Under such conditions equations (5) and
(6) are not used. However, they propose that either equation Sukarna Barua et al. [59] have presented a new method for
(7) or (8) can be used. efficiently handling the CIP, called MWMOTE. MWMOTE
first identifies the difficult to learn important minority class
min
a= (7) samples and assigns them weightage based on Euclidean
5 distance from the nearest larger class samples. For in-depth
b = max × 5 (8) study, read the original work of Sukarna Barua et al. [59].

B. SMOTE E. TRkN
To overcome the issue of over-fitting and extend the de- TRkNN originally proposed by Tsal & Yu [60] in 2016 to
cision area of the positive class samples, a novel technique overcome the CIP and improve the accuracy rate in predicting
SMOTE “Synthetic Minority Oversampling TEchnique” was the samples of majority and minority classes. TRkNN algo-
introduced by Chawla et al. [45], This technique produces rithm also solves the issue of noisy and borderline sample in
artificial samples by using the feature space rather than data CIP. The advantage of TRkNN is to avoid the production of
space. It is used for oversampling of positive class by creating unnecessary minority examples. For in-depth study, read the
the artificial data instead of using replacement or randomized original work of Tsal et al. [60].
sampling techniques. It was the first technique which intro-
duced new samples in the learning dataset to enhance the data
space and counter the scarcity in the distribution of samples F. ICOTE
[45]. The oversampling technique is a standard procedure in Xushegn et al. [61] introduced another oversampling tech-
the classification of imbalance data (e.g., minority class) [7]. It nique “ICOTE” in 2015 to improve the performance of clas-
has received incredible effort from the researcher of machine sification in CIP. ICOTE is based on an immune network and
learning domain in a recent decade. The pseudo code of the it produces a set of immune centroids to broaden the decision
SMOTE algorithm and detail can be found in [45]. space of the positive class. In this algorithm, the immune

JOURNAL OF IEEEACCESS 6

network is used to produce artificial samples on clusters It is applied to minimize the computational cost in
with the high data densities and these immune centroids are complex and large IS [64], [67].
considered synthetic examples in order to resolve the CIP. For • Covering Algorithm (Cov): It is the modified implemen-
in-depth study, read the original work of Xushegn et al. [61]. tation of the Learning from Example Module, version
1 (LEM1) algorithm and deployed in the Rough Set
G. Rough Set Theory (RST) Exploration System (RSES) as a rule-generation method.
RST [62] was initially proposed by Pawlak in 1982, which It was presented by Jerzy Grzymala [68].
is used as a mathematical tool in order to address ambiguity. • RSES LEM2 Algorithm (LEM2): It is a divide-and-
RST philosophy is centered on the assumption that there is conquer based method which coupled with the RST
information (knowledge, data) associated with all instances in approximation and it depends on the local covering de-
the universe of discourse. RST has a precise idea of rough termination of every instance from the decision attribute
set approximation (i.e., LB=lower bound and U B=upper [68], [69].
bound), and the boundary region (BR). The BR separates
the LB from U B (i.e. boundary-line). For example, those I. Evaluation Measures
samples that may not be classified with certainty are members It may not be possible to construct classifier that could
of either the LB or U B. It is difficult to characterize the perfectly classify all the objects of the validation set [42], [2].
borderline samples due to unavailability of clear knowledge To evaluate the classification performance, we calculate the
about these elements. Therefore, any rough concept is replaced count of T P (e.g., True Positive), F N (e.g., False Negative),
by either LB or U B approximation of vague concept [58–60]. F P (e.g., False Positive) and T N (e.g., True Negative). The
Mathematically, the concepts of LB, U B and BR have been F P value is part of N or negative but incorrectly classified as
defined as; let X ⊆ U and B is an equivalence relation (i.e. the P or positive. The F N result actually belongs to P . it can be
partition of the universe set U to create new subset of interest formulate as P = F N + T P but when incorrectly classified
from U which has the same value of outcome attribute) in the instances will then belongs to N . Mathematically it can
information system or IS = (U, B) of non-empty finite set be expressed as: N = F P + T N . The following evaluations
U and B, where U is the universe of objects
S and B is a set measures are used for performance validation of the proposed
which contains features. Then LB = Y ∈ U/B : Y ⊆ X approach.
is a LBS approximation Tand an exact member of X while • Regular Accuracy (RA): It is a measure that calculates
U B = Y ∈ U/B : Y X 6= φ is U B approximation that the classifier’s overall accuracy. It is formulated as:
can be an element of X. BR = LB − U B in the boundary TN + TP
region. The detail study can be found at [62], [63], [64], [65]. RA = (13)
N +P
• Sensitivity (Recall): It is the proportion of those cases
H. Rules Generation
which are correctly classified as true positive, and calcu-
Decision rules are often denoted as “IF C then D” where D lated as:
represents the decision feature and C is the set of conditional TP
attributes in the decision table [63]. Given two unary predicate Recall = (14)
P
formulae are α(χ) and β(χ), where χ executes over a finite • Precision: It is fraction of the predicted positive instances
set U . Łukasiewicz defined this in 1913 as: i.e. cardkα(χ)k
card(U ) , that characterized as correctly churned. Formally, it can
assign to α(χ) where kα(χ)k = {χU : χ}satisfies α while be expressed as;
the fractional value is assigned to implication α(x) => β(x) TP
is then card(|α(χ)β(χ)|) P recision = (15)
card(|α(χ)|) with assumption that ||α(x)|| 6= φ. TP + FP
The decision rules can easily be built by overlaying the • F-Measure: It is based on the harmonic mean between
reduct sets over the IS. Mathematically, it can be represented both the precision and recall. The high F-measure value
∧
as: (ai1 = v1 ) ..............∧ (aik = vk ) => d = vd ,where represents that both precision and recall are reasonably
1 ≤ ij ≺ .......... ≺ ik ≤ m, vi ∈ Vai ; for simplicity it can high. It can also be considered as the weighted-average
be represented in IF-ELSE statement as “IF C then D” where of recall and precision.
C is set of conditions and D is decision part. To retrieve Recall × P recision
the decision rules, the following well-known rule-generation F.M easure = 2 × (16)
Recall + P recision
algorithms are used [65]:
• Coverage: The ratio of classified objects that are recog-
• Exhaustive Algorithm (Exh): It takes subsets of at-
nized by a classifier from the class to the total number
tributes incrementally and then returns reduced set and
of instances in the class. Where C is a classifier, A is a
minimal decision rules. The generated decision rules
decision table, Match A (C) is a subset of objects in A
are those rules, which have minimal descriptors in the
that is classified by classifier C.
conditional attributes. It requires more focus due to the
|M atch A(C)|
extensive computations needed in the case of large and Coverage A(C) = (17)
complex Boolean reasoning method [66]. ||A||
• Genetic Algorithm (Gen): This method depends on • Mutual Information (MI): It measures the information
order-based genetic algorithm combined with a heuristic. regarding how much the attribute’s value pays to creating

JOURNAL OF IEEEACCESS 7

the right decision in classification. It is computing the MI

between the predicted churn label (i.e., ) and actual churn
(i.e., y). The MI between predicted and actual churn label
is as following:
P
1 P1 p(ŷ,y)
I Ŷ , Y = ŷ=0 y=0 p (ŷ) (y) log p(ŷ)p(y) (18)

• Imbalanced Ratio (IR): The imbalance ratio can be

expressed as the fraction of a number of samples in the
negative class to the number of samples in the positive
class [70]. It can be formulated as;
N umber of negative class instances
IR = (19) Fig. 3: Original class sample distribution, where the vertically
N umber of positive class instances striped bars represent the samples of the churn class while
• Area Under Curve (AUC): It is the fraction of total horizontally striped bars represent the samples of non-churn.
area that lies under the ROC graph. The ROC is used
to compute an overall measure of quality while AUC TABLE I: Summary Description Of Imbalanced Datasets
provides a single value for performance evaluation of Source #Ins #Att Class {Pos, {%Pos, IR
classifier. It can also be used as evaluation measure for Neg} %Neg}
the imbalanced datasets. The AUC measure is computed
Dataset 1 3333 21 {Churn, 14.49%, 5.94
as [5], [51]:
[71] Non-Churn} 85.51%
T Prate + T Nrate
AU C = (20) Dataset 2 5782 232 {-1, 1} 87.84%, 7.24
2 [72] 12.14%
• Balanced Accuracy (CBA): The use of Balanced Accu- Dataset 3 38162 251 {3G, 2G} 92.72%, 12.75
racy is not widespread in the class imbalance literature, [73] 7.27%
likely because of the aforementioned shortcoming, This Dataset 4 7043 20 {Yes, No} 73.46%, 2.77
study would use balanced accuracy for 2x2 confusion [74] 26.54%
table. i,j ∈ G, where G denotes the set of all possible
class labels.
Pk cij the majority or negative class), the percentage of instances in
i max(ci ,cj )
CBA = (21) each class (%Pos, %Neg) and imbalance ratio (IR) is shown.
k Table I, also shows the obtained IRs for different datasets:
from highly imbalanced ratio value to low imbalanced dataset.
IV. E VALUATION S ETUP
The IR 2.77 is the lowest and 12.75 is the highest value, which
In this section, an experimental environment is established shows that dataset 4 is low imbalanced dataset while dataset
for the propose approach. mRMR algorithm is used for se- 3 is a highly imbalanced dataset.
lecting more suitable attributes subset. six well-known over-
sampling techniques—SMOTE, ADASYN, MTDF, ICOTE, B. Features/Attributes selection & Dataset Preparation
MWMOTE and TRkNN—have been used to address the CIP
using publicly available datasets. Finally, a more appropriate For attributes selection, we used mRMR method [58]. The
approach for customer churn prediction is proposed. Classi- following attributes were selected for preparation of final
fication tasks, and trade-off among the four rules-generation datasets as shown in Table II, also reports to RQ1.
algorithms, are performed using RST. To focus on the CIP using available datasets, SMOTE,
MTDF, ADASYN, MWMOTE, ICOTE and TRkNN meth-
ods are used to generate artificial samples in positive class.
A. Dataset SMOTE algorithm is used using the Weka toolkit [75] and
In the proposed study, we have used publicly available MTDF is used manually while ADASYN technique [9],
dataset related to telecom-sector. These datasets can be ob- TRkNN [60], ICOTE [61]and MWMOTE [59] are applied
tained from URLs (i.e.,given in table I). Figures 3 present using MATLAB Code. Table III reflects the ranges of each
the original distribution of class samples & sample sizes in feature based on obtained values for “a” and “b” after applying
datasets. The vertical bar represent the instances of non-churn MTDF. The artificial samples are produced for “churn” based
class and horizontal bars used for the instances of churn class on ranges of “a” and “b” while the churn and non-churn
in four datasets. It is clear from the figure 3 that CIP exists in class size was equal to each other. It was investigated that
the available dataset. MTDF method produces values between 0 and 5 range for
Table I illustrates the description of the imbalanced datasets both features “VMail Plan” and “Intl Plan”, but it is observed
used in this empirical study. For each dataset, dataset source, that the required values for these features were in the range
the number of instances (#Ins), the number of attributes between 0 and 1. Therefore, normalization process was applied
(#Att), class name of each class (pos and neg where “pos” by using a standardized function to obtained the required
represent the positive or minority class while “neg” represent values [76]. . .

JOURNAL OF IEEEACCESS 8

TABLE II: Feature Subset Extracted Through mRMR

Features Dataset 1 Dataset 2 Dataset 3 Dataset 4
Sequence Features List Score Features List Score Features List Score Features List Score
F1 Intl Plan 0.041 Var ar flag 0.053 Var126 0.016 Contract 0.094
F2 Day Charges 0.040 Avg call intran 0.035 Var72 0.009 PaymentMethod 0.018
F3 Day Mins 0.039 Tot usage days 0.034 Var7 0.008 Tenure 0.009
F4 CustServ calls 0.013 Avg usage days 0.034 Var189 0.007 DeviceProtection 0.005
F5 Intl Charges 0.010 Avg call 0.033 Var65 0.006 Dependents 0.002
F6 Intl Mins 0.010 Highend prog flag 0.033 Var113 0.005 Gender -0.001
F7 Eve Charges 0.007 Avg call local 0.032 Var133 0.005 PaperlessBilling -0.002
F8 Eve Mins 0.007 Avg call ob 0.031 Var16 0.003 SeniorCitizen -0.005
F9 Account Length 0.006 Std vas arc 0.030 Var153 0.002 Partner -0.021
F10 VMail Plan 0.003 Avg mins 0.029 Var73 0.002 PhoneService -0.048
F11 Class label attribute Class label attribute Class label attribute Class label attribute

TABLE III: The a and b values of MTDF Function for each attribute
Features Dataset 1 Dataset 2 Dataset 3 Dataset 4
Sequence MTDF Ranges MTDF Ranges MTDF Ranges MTDF Ranges
a b a b a b a b
F1 28.4636 44.6562 0.0000 1.0000 -30 56.88 0.873547 6.655153
F2 0.0000 1.0000 380.069 393.496 3.45 15.58 2.412609 8.02709
F3 173.517 210.653 91.9169 127.634 10.43 20.17 0.361109 4.892115
F4 0.0000 1.0000 15.2519 29.8818 308.83 426.43 0.881878 6.282582
F5 111.32 138.139 2409.06 2443.80 22.93 36.07 35.89648 63.77696
F6 207.681 239.084 0.0000 1.0000 3552064 3263292 0.348405 5.293856
F7 2.72140 9.83579 2403.23 2437.72 5444049 5451211 0.350999 5.211826
F8 2.0746 5.80143 1292.73 1318.407 37.58 576.24 0.388244 4.034051
F9 9.2768 16.4491 98.0268 103.876 6155159 6168114 0.348485 5.291311
F10 15.665 24.8207 7646.24 7687.29 105.22 136.59 0.410333 3.335514

TABLE IV: Attributes for the Decision table using Dataset 1 TABLE VI: Attributes for the Decision table using Dataset 3
Sets Desciption Sets Desciption
Number of Objects {5700 distinct objects} Number of Objects {38162 distinct objects}
Conditional Attributes {Intl Plan, Day charges, Day mins, Conditional Attributes {Var126, Var72, Var7, Var189, Var65,
Custserv calls, Intl charges, Eve mins, Var113, Var133, Var16, Var153, Var73}
Account length, Vmail plan} Decision Attribute {Churn}
Decision Attribute {Churn}
.
TABLE VII: Attributes for the Decision table using Dataset 4
Sets Desciption
TABLE V: Attributes for the Decision table using Dataset 2
Number of Objects {7043 distinct objects}
Sets Desciption Conditional Attributes {Contacts, PaymentMethod, Dependents,
Number of Objects {38162 distinct objects} DeviceProtection, Tenure, Gender,
Conditional Attributes {Var ar flag, Avg call intran, Tot usage days, PaperlessBilling, SeniorCitizen, Partner,
Avg usage days, Avg call, Highend program PhoneService}
flag, Avg call local, Avg call ob, Std vas,arc, Decision Attribute {Churn}
Avg mins}
Decision Attribute {Churn}

JOURNAL OF IEEEACCESS 9

Once the range of individual attributes (i.e., a and b) is of the predictive models on known and unknown objects [77].
defined by using MTDF, the next step is to generate the To perform the validation process, the following procedure is
artificial samples using the above mentioned oversampling applied: (i) some of the data is excluded from the training
technique. Then prepare the decision tables to apply the base- set that is used for the learning process of the classifier, (ii)
classifier (i.e., rough set theory). Tables IV, V, VI and VII when the training process is finished, and the classifier is to be
represent the prepared decision tables for dataset 1, 2, 3 and considered as trained, the excluded samples can be included
4 respectively. to validate the results of the trained classifier on unseen
These tables contain objects, attributes with condition that data samples. This procedure is known as the cross-validation
needs to be fulfilled and the decision attributes. For each exper- process. The K-fold cross-validation is applied in the proposed
iment (i.e. original population extended by MTDF, SMOTE, study to avoid bias during the validation of methods.
ADASYN, MWMOTE, ICOTE and TRkNN oversampling
techniques) the same structure of decision tables with four
V. R ESULTS AND D ISCUSSION
different datasets were used.
The cut and discretization process is an important approach In this section, the four rules-generation algorithms using
to reduce the dataset horizontally in order to handle the RST on dataset 1, 2, 3, and 4 that were expanded using
large data efficiently. It is a common approach used in the MTDF, SMOTE, ADASYN, ICOTE, MWMOTE and TRkNN
RST where the attributes that contain continuous values are to handle the CIP were considered. Tables IX, X, XI and
split into a finite number of intervals [47]. The cut and XII reflect the performance of classifiers through evaluation
discretization processes were carefully accomplished on the measures. The following tables IX, X, XI and XII clearly
prepared decision table. These cuts on the decision table were shows the performance of various oversampling techniques
made at every iteration to minimize the number of cuts to the (i.e., MTDF, SMOTE, ADASYN, MWMOTE, ICOTE and
data. This was done in light of the recommendation in [65]. TRkNN) on publically available telecommunication’s dataset
For example, the cuts of attribute “Day Min” in dataset 1 were by applying classification process based on rough set theory
grouped after discretization process as listed in Table VIII. The and four rules generation algorithms (i.e., Gen, Exh, Cov
first field denotes the groups that are represented by numeric and LEM2). It can be observed (i.e., as given tables IX-
numbers for simplicity purpose and listed in ascending order. XII) that the Cov and LEM2 rules generation algorithms have
The second column represents the intervals that are obtained achieved maximum performances as compared to Gen and Exh
after the discretization process. The third column is the count algorithms but these two (i.e. Cov & LEM2) have not been
of the attribute’s values that fall into certain groups while covered all the instances available in the dataset. Therefore, all
the last column is the percentage of variable’s value in each the underlined values corresponding to targeted oversampling
interval. It is clear from table VIII that the range of Day Min techniques are ignored from further analysis in this study while
has been changed from the continuous nature into 14 different the bold values reflect the best performed techniques in our
intervals or groups after cut and discretization process. empirical environment.
To report RQ2 and RQ3, the results reflect in Tables IX, X,
TABLE VIII: Cuts distribution of attribute Day Min XI, and XII were thoroughly analyzed to cover each relevant
Group# Intervals Count Percentage detail. It is observed that both algorithms (e.g., Cov & LEM2)
shown higher accuracy, but do not deliver full coverage (e.g.,
1 {263.55, 281.05} 115 3.45%
the underlined values does not seem to follow from full
2 {151.05, 163.45} 295 8.85% coverage). The coverage of a learning algorithm is the number
3 {237.85, 251.85} 158 4.74% of samples or instances that can be learned by that algorithm
4 {281, **} 103 3.09% from samples of a given size for given accuracy [78].
5 {163.45, 178.15} 339 10.17% On the other hand, both algorithms (i.e. Exh and Gen) have
6 {217.65, 178.15} 337 10.11% covered all instances in given datasets (i.e. Dataset 1, 2, 3 &
7 {178.15, 189.25} 235 7.05% 4). It can be observed in Figure 4 that both the Cov and LEM2
8 {251.85, 263.55} 95 2.85% algorithms achieved higher accuracy as compared to Exh and
9 {108.8, 151.05} 687 20.61% Gen algorithms but these results seem not to follow the full
10 {195.45, 208.85} 302 9.06% objects coverage. So these results (i.e. not fully covering the
11 {189.25, 195.45} 168 5.04% dataset objects) were ignored and not shown in Figures 5 (a),
12 {*, 78.65} 109 3.27% (b), (c) and (d) for datasets 1, 2, 3 and 4 respectively. Figure
13 {78.65, 108.8} 202 6.06% 5 describes the performance of oversampling algorithms (e.g.,
14 {208.85, 217.65} 188 5.64% MTDF, SMOTE, ADASYN, ICOTE, MWMOTE and TRkNN)
**=Maximum number, *=Minimum number on the datasets that were prepared and balanced by using the
same algorithms. It is observed from the results that Gen and
Exh algorithms have performed better as compared to the rest
C. Training and Validation Sets of the other two. Although, Gen and Exh algorithms have
In data mining, training and validation is an extremely achieved alike performance, however, the Gen algorithm has
important step. Once the training process is finished, then the acquired more accuracies 0.929, 0.862, 0.982 and 0.813 with
validation step is to be performed to confirm the performance 100% objects coverage on dataset 1, 2, 3, and 4 respectively.

JOURNAL OF IEEEACCESS 10

TABLE IX: Performance evaluation of classifier for dataset 1

Techniques & Algorithm Accuracy Recall Precision F-Measure Coverage MI
Exh M 0.935 0.960 0.911 0.933 1.0 0.66
Gen M 0.936 0.964 0.913 0.934 1.0 0.67
MTDF
Cov M 0.914 0.919 0.913 0.912 0.528 0.58
LEM2 M 0.964 0.979 0.944 0.967 0.799 0.78
Exh M 0.924 0.930 0.917 0.923 1.0 0.61
Gen M 0.925 0.933 0.916 0.924 1.0 0.62
SMOTE
Cov M 0.953 0.969 0.973 0.864 0.226 0.41
LEM2 M 0.941 0.944 0.944 0.938 0.752 0.68
Exh M 0.925 0.931 0.920 0.925 1.0 0.62
Gen M 0.924 0.933 0.917 0.924 1.0 0.62
ADASYN
Cov M 0.944 0.919 0.913 0.912 0.216 0.08
LEM2 M 0.950 0.951 0.953 0.947 0.75 0.71
Exh M 0.952 0.942 0.960 0.953 1.0 0.725
Gen M 0.969 0.961 0.976 0.969 1.0 0.803
MWMOTE
Cov M 0.992 0.962 0.962 0.996 0.41 0.40
LEM2 M 0.985 0.96 0.888 0.992 0.42 0.35
Exh M 0.926 0.951 0.906 0.924 1.0 0.63
Gen M 0.927 0.958 0.904 0.926 1.0 0.64
ICOTE
Cov M 0.927 0.917 0.905 0.938 0.629 0.60
LEM2 M 0.959 0.970 0.940 0.962 0.80 0.75
Exh M 0.88 0.866 0.892 0.882 1.0 0.47
Gen M 0.89 0.879 0.898 0.891 1.0 0.50
TRkNN
Cov M 0.861 0.799 0.822 0.892 0.347 0.38
LEM2 M 0.874 0.843 0.857 0.893 0.638 0.44

TABLE X: Performance evaluation of classifier for dataset 2

Techniques & Algorithm Accuracy Recall Precision F-Measure Coverage MI
Exh M 0.940 0.955 0.927 0.940 1.0 0.68
Gen M 0.946 0.930 0.964 0.945 1.0 0.70
MTDF
Cov M 0.789 0.859 0.795 0.732 0.38 0.237
LEM2 M 0.834 0.861 0.872 0.780 0.579 0.317
Exh M 0.847 0.851 0.880 0.824 1.0 0.376
Gen M 0.849 0.841 0.812 0.867 1.0 0.379
SMOTE
Cov M 0.999 1.0 0.666 0.999 0.199 0.0099
LEM2 M 0.898 0.885 0.873 0.912 0.67 0.5096
Exh M 0.85 0.85 0.88 0.82 1.0 0.38
Gen M 0.854 0.867 0.85 0.84 1.0 0.38
ADASYN
Cov M 0.98 0.66 0.97 0.99 0.19 0.11
LEM2 M 0.90 0.89 0.87 0.91 0.67 0.52
Exh M 0.743 0.756 0.741 0.739 1.0 0.18
Gen M 0.746 0.756 0.741 0.743 1.0 0.18
MWMOTE
Cov M 0.689 0.66 0.589 0.736 0.25 0.10
LEM2 M 0.780 0.821 0.814 0.722 0.34 0.22
Exh M 0.860 0.839 0.876 0.863 1.0 0.418
Gen M 0.862 0.847 0.873 0.864 1.0 0.422
ICOTE
Cov M 0.981 0.979 0.894 0.988 0.547 0.48
LEM2 M 0.973 1.00 0.939 0.975 0.843 0.027
Exh M 0.802 0.80 0.803 0.802 1.0 0.28
Gen M 0.812 0.794 0.823 0.815 1.0 0.30
TRkNN
Cov M 0.770 0.736 0.68 0.786 0.362 0.22
LEM2 M 0.831 0.813 0.813 0.843 0.658 0.34

JOURNAL OF IEEEACCESS 11

TABLE XI: Performance evaluation of classifier for dataset 3

Techniques & Algorithm Accuracy Recall Precision F-Measure Coverage MI
Exh M 0.970 0.97 0.98 0.97 1.0 0.85
Gen M 0.980 0.96 0.99 0.981 1.0 0.88
MTDF
Cov M 0.990 0.99 0.98 0.980 0.70 0.85
LEM2 M 0.980 0.96 0.99 0.981 0.97 0.87
Exh M 0.920 0.91 0.92 0.92 1.0 0.61
Gen M 0.921 0.91 0.93 0.92 1.0 0.60
SMOTE
Cov M 0.920 0.91 0.93 0.921 0.46 0.59
LEM2 M 0.950 0.96 0.93 0.96 0.76 0.73
Exh M 0.928 0.92 0.93 0.92 1.0 0.62
Gen M 0.930 0.917 0.94 0.93 1.0 0.63
ADASYN
Cov M 0.916 0.909 0.931 0.91 0.45 0.58
LEM2 M 0.960 0.97 0.94 0.97 0.76 0.77
Exh M 0.745 0.756 0.741 0.742 1.0 0.182
Gen M 0.746 0.756 0.741 0.743 1.0 0.18
MWMOTE
Cov M 0.692 0.66 0.589 0.736 0.387 0.096
LEM2 M 0.779 0.821 0.814 0.722 0.682 0.217
Exh M 0.907 0.926 0.892 0.905 1.0 0.555
Gen M 0.915 0.94 0.895 0.912 1.0 0.584
ICOTE
Cov M 0.895 0.914 0.803 0.918 0.70 0.480
LEM2 M 0.936 0.970 0.886 0.943 0.816 0.663
Exh M 0.692 0.712 0.684 0.685 1.0 0.109
Gen M 0.706 0.722 0.699 0.701 1.0 0.126
TRkNN
Cov M 0.671 0.665 0.658 0.692 0.422 0.092
LEM2 M 0.710 0.681 0.625 0.748 0.555 0.119

TABLE XII: Performance evaluation of classifier for dataset 4

Techniques & Algorithm Accuracy Recall Precision F-Measure Coverage MI
Exh M 0.812 0.73 0.871 0.825 1.0 0.314
Gen M 0.848 0.794 0.881 0.858 1.0 0.389
MTDF
Cov M 0.999 0.5 0.5 0.999 0.213 6.904
LEM2 M 0.857 0.830 0.830 0.871 0.649 0.402
Exh M 0.808 0.729 0.866 0.822 1.0 0.308
Gen M 0.811 0.808 0.813 0.811 1.0 0.301
SMOTE
Cov M 0.825 0.796 0.904 0.797 0.329 0.335
LEM2 M 0.854 0.772 0.860 0.881 0.681 0.379
Exh M 0.772 0.697 0.873 0.769 1.0 0.251
Gen M 0.806 0.731 0.860 0.821 1.0 0.302
ADASYN
Cov M 0.823 0.813 0.806 0.808 0.315 0.296
LEM2 M 0.849 0.814 0.809 0.810 0.650 0.301
Exh M 0.693 0.664 0.704 0.701 1.0 0.110
Gen M 0.673 0.642 0.684 0.682 1.0 0.088
MWMOTE
Cov M 0.647 0.623 0.733 0.615 0.4 0.066
LEM2 M 0.703 0.703 0.679 0.715 0.486 0.122
Exh M 0.802 0.794 0.806 0.803 1.0 0.282
Gen M 0.793 0.788 0.795 0.794 1.0 0.264
ICOTE
Cov M 0.805 0.832 0.736 0.823 0.384 0.288
LEM2 M 0.836 0.841 0.868 0.812 0.672 0.348
Exh M 0.693 0.724 0.681 0.683 1.0 0.110
Gen M 0.691 0.734 0.675 0.677 1.0 0.108
TRkNN
Cov M 0.674 0.664 0.614 0.705 0.373 0.088
LEM2 M 0.699 0.736 0.703 0.675 0.492 0.116

JOURNAL OF IEEEACCESS 12

The following acronyms have been used in Figures 4- 6:

Gen M: Genertic Algorithm & MTDF (dataset oversampled with MTDF).

Exh S: Exhaustive Algorithm & SMOTE (dataset oversampled with SMOTE).
Gen S: Genertic Algorithm & SMOTE (dataset oversampled with SMOTE).
Exh A: Exhaustive Algorithm & ADASYN (dataset oversampled with ADASYN).
Gen A: Genetic Algorithm & ADASYN (dataset oversampled with ADASYN).
Exh M: Exhaustive Algorithm & MTDF (dataset oversampled with MTDF)
Gen I: Genetic Algorithm & ICOTE (dataset oversampled with ICOTE)
Exh I: Exhaustive Algorithm & ICOTE (dataset oversampled with ICOTE)
Gen W: Genetic Algorithms & MWMOTE (dataset oversampled with MW-
MOTE)
Exh W: Exhaustive Algorithm & MWMOTE (dataset oversampled with MW- (a)
MOTE)
Gen T: Genetic Algorithms & TRkNN (dataset oversampled with TRkNN)
Exh T: Exhaustive Algorithm & TRkNN (dataset oversampled with TRkNN)

(b)

(a)

(d)
(c)
Fig. 5: The positive predictive value (precision) and sensitivity
(recall) followed by F-measure (the weighted harmonic mean
of precision and recall) evaluate the classifiers’ performance on
dataset (1-4) where (a), (b), (c) and (d) reflects the preciseness
and robustness the targeted algorithms. F-measure reaches its
worst value at 0 & best value at 1.

(d)

Fig. 4: Coverage of objects and Accuracy of techniques on The F-Measure and MI can give the best result. The MI
all datasets (1-4), where (a), (b), (c) and (d) represents the value of Gen M is larger than other targeted algorithms. It
accuracy and coverage of selected algorithms on dataset 1,2, describes the correct ranking through MI measure between
3 and 4 respectively. List of algorithms is given on x-axis the projected and positive churn in all datasets (e.g., datasets
while y-axis reflects the number of samples (instances) 1, 2, 3 & 4).

JOURNAL OF IEEEACCESS 13

TABLE XIII: Comparison between regular accuracy & class

balance accuracy
Dataset 1 Dataset 2 Dataset 3 Dataset 4
Classifiers
RA+CBA RA+CBA RA+ CBA RA+CBA

Gen I 0.929+0.978 0.862+0.923 0.915+0.959 0.802+0.885

Exh I 0.927+0.974 0.860+0.919 0.907+0.952 0.793+0.882
Gen S 0.925+0.965 0.849+0.921 0.923+0.965 0.811+0.890
Exh S 0.924+0.963 0.849+0.920 0.923+0.964 0.809+0.901
Gen A 0.927+0.964 0.852+0.919 0.930+0.969 0.807+0.864
Exh A 0.927+0.963 0.847+0.920 0.929+0.964 0.773+0.847
Gen W 0.969+0.979 0.746+0.865 0.746+0.865 0.673+0.807 Fig. 6: Comparison of selected algorithms based on AUC
Exh W 0.879+0.930 0.745+0.854 0.744+0.862 0.693+0.819 values.
Gen M 0.935+0.980 0.946+0.971 0.993+0.994 0.848+0.896
Exh M 0.936+0.980 0.941+0.967 0.975+0.991 0.811+0.864
Gen T 0.890+0.938 0.812+0.885 0.706+0.847 0.691+0.852 B. Simulation Results
Exh T 0.881+0.931 0.802+0.888 0.692+0.842 0.693+0.847 Tables IX, X, XI, XII and XIII summarize the results of
SMOTE, ADASYN, MTDF, MWMOTE, ICOTE and TRkNN
algorithms on four publically available telecommunication
related datasets. Each result is the average of the 10 inde-
A. Comparison between Binary Class Imbalance Accuracy
pendent executions of classifiers and selected the best result
and Class Balance Accuracy (CBA)
of each targeted algorithms for this study. In tables IX—XII
precision, recall also included as evaluation measures of
The balanced accuracy, which accounts for the recall over different techniques but these evaluations are also used as
each class, has the weakest ability to discriminate between input for other evaluation measures (i.e., F-measure). The
confusion matrix inputs. It is the ability of CBA measure bold-face values in the recall column of each technique in
to account for rows and column sum simultaneously that all datasets (e.g., Tables IX—XII) have to misclassify many
allows it to discriminate between all other measures (i.e. negative class samples as positive. Hence, in spite of the
Regular Accuracy, F-Measure, Mutual Information, Precision, higher recall, these targeted techniques reduce the F-measure
and Recall). CBA’s ability to detect changes in the false and AUC because the erroneous generation of the synthetic
negative and false positive counts separates it from overall positive class samples falls inside the negative class region.
regular accuracy. Therefore, table XIII shows the overall The probability of wrongly classified minority class samples
regular accuracy followed by the AUC results (+ for standard thus providing improve precision and decrease recall. Unlike
deviation). The comparison of the CBA and regular accuracy recall and precision, the overall simulation results are based
for the binary class imbalance is shown in Table XIII, where on F-measure, AUC and CBA. The AUC values are shown in
each table represent the results of each dataset (i.e. datasets 1, figure 6 which illustrate a comparison of selected algorithms.
2, 3 & 4) respectively. The bold values in each table represent It is observed that the performances of SMOTE, ADASYN
the best-performing method. it can be observed in the tables and ICOTE algorithms are closed to each other and these
XIII that both Gen M (genetic algorithm applied on oversam- algorithms have achieved a performance higher than TRkNN
pled dataset using MTDF) and Exh M (exhaustive algorithm algorithm and lower than the performance of MTDF. It is
applied on oversampled dataset using MTDF) performed better also investigated that the performance of SMOTE, ADASYN
as compared to the rest of the techniques. It is also investigated and ICOTE have much better results than MWMOTE where
that out of these two techniques (i.e. Gen M and Exh M) the the dataset have less categorical values attributes but low
Gen M have achieved overall much better results. performance obtained where the data have too many categor-
The AUC can be used as evaluation measure to evaluate ical values in attributes as compare to MWMOTE. It is also
the performance of classifiers. It is observed that the best investigated that MWMOTE can outperform on such dataset
samples distribution for training set tends to be very balanced which has much more categorical values attributes in a dataset.
class samples distribution [48]. The CIP significantly decreases on the other hand, SMOTE have not performed better (e.g.,
the performance of imbalanced datasets. It might be projected obtained AUC values 0.92, 0.46, 0.69 and 0.57 on dataset 1, 2,
that the AUC decreases for the very imbalanced dataset. The 3 and 4 respectively) due to over-generalization and variance
results obtained for AUC from experiments are shown in figure because SMOTE assign the same number of synthetic samples
6. As stated earlier, these results were calculated through for every genuine minority samples without considering to
10-fold cross validation and AUCs were obtained from the the samples in neighbored while over generalization is major
over-sampled dataset by applying target algorithms. However, drawback of the SMOTE algorithm [79]. Similarly, ICOTE
here only two algorithms (i.e. the Genetic and Exhaustive) also uses the generalization procedure and create a larger
have been shown because these two algorithms provided full cluster during the learning process of a classification [61].
coverage during classification of the validation set instances To get this over generalization issue, ADASYN algorithm,
(e.g. shown in figure 4). which is based on adaptive sampling method [8], [9][8],

JOURNAL OF IEEEACCESS 14

[9], was applied. ADASYN uses a weighted distribution for TABLE XIV: p-Value (P), M values obtained from McNemar’s
positive class samples and reducing the bias in generated test
synthetic samples [79]. ADASYN algorithm also has not been Algorithms Dataset 1 Dataset 2 Dataset 3 Dataset 4 Avg. Values
sufficiently achieved higher performance (e.g., average per-
P 0.012 0.078 0.003 0.0008 0.0234
formance based on obtained AUC values 0.92, 0.85, 0.93 and Gen M
M 7.04878 3.84845 9.30769 8.27944 7.01359
0.80 on dataset 1, 2, 3 and 4 respectively), due to interpolation
P 0.542 0.199 0.41 0.656 0.5092
between positive class samples and their random same class Gen S
M 0.58139 1.86452 0.92452 0.05581 0.8565
neighbors for producing the artificial samples. According to
P 0.661 0.326 0.322 0.0007 0.32742
Rashma et al. [79] sometime both algorithms (i.e., ADASYN Gen A
M 0.0088 0.0127 -0.0117 8.27751 2.07183
and SMOTE) become inappropriate and cannot provide the
P 0.0001 0.9000 0.9980 0.851 0.79975
required amount of useful synthetic positive class samples. Gen W
M 7.70916 0.03937 0.0397 0.29388 0.27044
Finally, it is summarized from all the simulation results given
P 0.018 0.235 0.508 0.9800 0.43525
in tables IX—XIII based on various evaluation measures (de- Gen I
M 7.2867 1.52039 0.73529 0.01207 2.38861
tailed given in section III) that MTDF have better performed
P 0.526 0.815 0.851 0.473 0.66625
as a whole as compared to the rest of applied oversampling Gen T
M 0.59346 0.17234 0.08707 0.59838 0.36281
techniques (e.g., SMOTE, ADASYN, MWMOTE, ICOTE and
TRkNN) on the targeted domain datasets (i.e., Tables I and
II). It is also observed about MTDF that it can solve the
issue of CIP of small dataset very efficiently. However, the statistic distribution χ2 test for goodness-of-fit that compares
other oversampling techniques needed a considerable number the distribution of the counts expected under the H0 to the
of samples during the learning process to approximate the observed values. it is sufficed that to reject the H0 in favor
real function. Consequently, the existing number of samples of of the hypothesis that the multiple algorithms have different
used datasets have created difficulty when using small datasets performance when trained on the targeted data set. McNemar’s
for average performed (i.e., SMOTE, ADASYN and ICOTE) Test value can be calculated using the following formula given
and worst performed (MWMOTE and TRkNN) algorithms in in equation (22) [83], [84]:
this study. In order to completely cover the information gap in 2
(|n01 − n10 |)
datasets (e.g., Tables I and II) and avoid the over estimation M= > χ21,α (22)
n01 − n10
of samples, MTDF more appropriately substituted the required
samples with the help of both data trend estimation and mega The probability for the quantity of M is larger than χ21,0.95 =
diffusion [80]. On the other hand, MTDF is based on normal 3.841459 is less than 0.05 for 95% confidence test with 1
distribution which is a compulsory condition in the statistical degree of freedom [84]. If the null hypothesis that the debate
data-analysis [11]. Therefore, MTDF is best techniques for had no effect were true then Gen M = Gen S = Gen A as chi
more systematically estimating the existing samples or data in statistic χ2 where the degree of freedom is 1. In case the H0
the dataset. is correct, then the probability that this quantity is greater than
the χ21,0.95 = 3.841459 is less than 0.05 for 95% confidence
C. Statistical Test test [84]. McNemar’s Test value can be calculated using a
In order to compare different classifiers and algorithms, this formula (i.e. discussed in section 3.2 and equation (22)). We
study supports our comparative assumptions through statistical can reject the H0 , as these classifiers have the same error
evidence. For this work, a non-parametric test was used to rate which may also consider in favor of the null hypothesis
provide statistical comparisons of some classifiers according that these algorithms have different performance when trained
to recommendation suggested by [4], [70], [81]. The reasons on the same datasets (e.g. dataset 1, 2, 3 and 4). Table XIV
why this study uses a nonparametric statistical test as follows reflects the performance of classifiers.
[81], [82]: (i) these tests can handle both normally and non- “M” values is greater than chi statistic χ2 = 3.841459
normally distributed data while parametric test, usually apply reflecting to reject the null hypothesis (shown in underlined)
on normally distributed data only, (ii) non-parametric test with 95% confidence and 1 degree of freedom while “P”
are guaranteed the reliability of the parametric test, and (iii) values is lower than 0.05 (shown in underlined) indicating that
parametric tests are more closed to reject the null-hypothesis the performance difference between classifiers is statistically
than the non-parametric tests unless their assumptions are significant. The overall best average performance between the
violated. Therefore, Janez Demsar [81] recommended using classifiers is shown in bold.
the non-parametric tests than using parametric test. These tests
may not be satisfied causing the statistical analysis to lose its
credibility. Furthermore, McNemar’s test [83] was applied to D. Threats to Validity
evaluate the classifiers’ performance by comparing the results • Open source tools and public dataset: To investigate the
of best-performing algorithms (e.g. Gen M, Gen S, Gen A, performance of the proposed solution, this study has used
Gen I, Gen W and Gen T). four publicly available datasets from different sources
Under the H0 (e.g., null hypothesis), different algorithms related to the telecom sector. Also open source tools for
should have the same error rate, which means that classi- evaluation and classification process were used, therefore,
fier A = Classifier B. McNemar’s test is based on the chi the results may not be generalizable to closed-source

JOURNAL OF IEEEACCESS 15

tools or proprietary data, which may lead to variance in R EFERENCES

performance.
• Distribution of artificial samples: this study has also [1] W. Verbeke, D. Martens, C. Mues, and B. Baesens, “Building compre-
hensible customer churn prediction models with advanced rule induction
applied the randomization of artificial samples to avoid techniques,” Expert Systems with Applications, vol. 38, no. 3, pp. 2354–
the biases in the distribution of samples population over 2364, 2011.
the decision classes through the adoption of the following [2] J. Burez and D. Van den Poel, “Handling class imbalance in customer
steps: (i) producing a unique number, used as sequence churn prediction,” Expert Systems with Applications, vol. 36, no. 3, pp.
4626–4636, 2009.
for each class label using a rand-between statistical [3] T. Jo and N. Japkowicz, “Class imbalances versus small disjuncts,” ACM
function, then (ii) sorting function is used to arrange the Sigkdd Explorations Newsletter, vol. 6, no. 1, pp. 40–49, 2004.
generated values, (iii) finally, the normalization proce- [4] M. Galar, A. Fernandez, E. Barrenechea, H. Bustince, and F. Herrera,
“A review on ensembles for the class imbalance problem: bagging-,
dure using standardized statistical function particularly boosting-, and hybrid-based approaches,” IEEE Transactions on Systems,
on “VMail Message and Intl Plan”. Applying different Man, and Cybernetics, Part C (Applications and Reviews), vol. 42, no. 4,
function or method for normalization may not lead to pp. 463–484, 2012.
[5] S. Maheshwari, J. Agrawal, and S. Sharma, “New approach for classi-
consistent results. fication of highly imbalanced datasets using evolutionary algorithms,”
• Evaluation methods: our work has used K-fold cross International Journal of Scientific & Engineering Research, vol. 2, no. 7,
validation to avoid biases in the proposed work. Apply- pp. 1–5, 2011.
ing other validation methods (e.g., hold-out validation [6] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, “Smote:
synthetic minority over-sampling technique,” Journal of artificial intel-
method) might not provide the same results. Different ligence research, vol. 16, pp. 321–357, 2002.
splits may also yield different performances. [7] J. Wang, M. Xu, H. Wang, and J. Zhang, “Classification of imbalanced
data by using the smote algorithm and locally linear embedding,” in
2006 8th international Conference on Signal Processing, vol. 3, 2006
VI. C ONCLUSION AND F UTURE WORK 8th international Conference on Signal Processing. IEEE, 2006.
[8] Ş. Ertekin, “Adaptive oversampling for imbalanced data classification,”
This study has addressed the class imbalance problem in Information Sciences and Systems 2013, Information Sciences and
Systems 2013. Springer, 2013, pp. 261–269.
(CIP) and has focused on comparing the performance of [9] H. He, Y. Bai, E. A. Garcia, and S. Li, “Adasyn: Adaptive synthetic
six oversampling solutions for dealing with CIP—Mega-trend sampling approach for imbalanced learning.” 2008 IEEE International
Diffusion Function (MTDF), Synthetic Minority Oversampling Joint Conference on Neural Networks (IEEE World Congress on Com-
putational Intelligence), 2008, pp. 1322–1328.
Technique (SMOTE), Adaptive Synthetic Sampling approach [10] D.-C. Li, C.-S. Wu, T.-I. Tsai, and Y.-S. Lina, “Using mega-trend-
(ADASYN), Majority Weighted Minority Oversampling Tech- diffusion and artificial samples in small data set learning for early
nique (MWMOTE), immune centroids oversampling tech- flexible manufacturing system scheduling knowledge,” Computers &
Operations Research, vol. 34, no. 4, pp. 966–982, 2007.
nique (ICOTE) and Couples Top-N Reverse k-Nearest Neigh-
[11] C. C. Teck, L. Xiang, Z. Junhong, L. Xiaoli, C. Hong, and D. Woon,
bor (TRkNN)—on four publicly available datasets for the “Hybrid rebalancing approach to handle imbalanced dataset for fault
telecommunications sector. We examined the performance of diagnosis in manufacturing systems,” in 2012 7th IEEE Conference
these oversampling techniques with four (4) different rules- on Industrial Electronics and Applications (ICIEA). 2012 7th IEEE
Conference on Industrial Electronics and Applications (ICIEA), 2012,
generation algorithms based on the Rough Set Theory (RST) pp. 1224–1229.
classification approach—i.e., Learning from Example Module, [12] J.-C. Deville and Y. Tillé, “Efficient balanced sampling: the cube
version 2 (LEM2), Covering, Exhaustive and Genetic algo- method,” Biometrika, vol. 91, no. 4, pp. 893–912, 2004.
[13] N. Japkowicz, “The class imbalance problem: Significance and strate-
rithms. The experiments performed show that the Genetic gies,” in Proc. of the Intl. Conf. on Artificial Intelligence, Proc. of the
algorithm using MTDF for oversampling yielded the best Intl. Conf. on Artificial Intelligence. Citeseer, 2000.
performance in dealing with the CIP. Future studies might [14] A. Amin, F. Rahim, I. Ali, C. Khan, and S. Anwar, “A comparison
of two oversampling techniques (smote vs mtdf) for handling class
delineate more specific issues such as an extension to address imbalance problem: A case study of customer churn prediction,” in
the multi-class learning problems instead of considering only New Contributions in Information Systems and Technologies, New
two class problem, analyzing time complexity of targeted Contributions in Information Systems and Technologies. Springer,
2015, pp. 215–225.
oversampling techniques (i.e. applied in proposed study). This
[15] W. Verbeke, K. Dejaeger, D. Martens, J. Hur, and B. Baesens, “New
will further help in investigating the performances of the over- insights into churn prediction in the telecommunication sector: A profit
sampling techniques not only on two-class but also multi-class driven data mining approach,” European Journal of Operational Re-
problems. Moreover, another natural extension of this study is search, vol. 218, no. 1, pp. 211–229, 2012.
[16] D. Van den Poel and B. Lariviere, “Customer attrition analysis for
to analyze the Receiver operating characteristic (ROC) curves financial services using proportional hazard models,” European journal
obtained from the two-class & multi-class classifiers. This of operational research, vol. 157, no. 1, pp. 196–217, 2004.
might provide us with a deeper understanding of the behavior [17] R. O. Duda, P. E. Hart, and D. G. Stork, Pattern classification. John
Wiley & Sons, 2012.
of balancing and data preparation of large imbalanced datasets. [18] M. Karnstedt, M. Rowe, J. Chan, H. Alani, and C. Hayes, “The effect
of user features on churn in social networks,” in Proceedings of the 3rd
International Web Science Conference. ACM, 2011, p. 23.
ACKNOWLEDGMENTS [19] X. Long, W. Yin, L. An, H. Ni, L. Huang, Q. Luo, and Y. Chen, “Churn
analysis of online social network users using data mining techniques,”
The authors are grateful to the anonymous reviewers for in Proceedings of the International MultiConference of Engineers and
their insightful comments and suggestions, which helped im- Computer Scientists, vol. 1. Proceedings of the International Multi-
prove the quality of this paper. Professor A. Hussain is sup- Conference of Engineers and Computer Scientists, 2012.
[20] W. Verbeke, D. Martens, and B. Baesens, “Social network analysis for
ported by the UK Engineering and Physical Sciences Research customer churn prediction,” Applied Soft Computing, vol. 14, pp. 431–
Council (EPSRC) grant no. EP/M026981/1. 446, 2014.

JOURNAL OF IEEEACCESS 16

[21] P. D. Kusuma, D. Radosavljevik, F. W. Takes, and P. van der Putten, [43] N. Chawla, N. Japkowicz, and A. Kolcz, “Special issue on learning from
“Combining customer attribute and social network mining for prepaid imbalanced datasets,” in ACM SIGKDD Explorations, ACM SIGKDD
mobile churn prediction.” Proc. the 23rd Annual Belgian Dutch Explorations. ACM SIGKDD, 2004.
Conference on Machine Learning (BENELEARN), 2013, pp. 50–58. [44] S. Visa and A. Ralescu, “Issues in mining imbalanced data sets-a review
[22] W. Jingjing, J. Chunxiao, and Q. e. a. Tony, “The value strength aided in- paper,” vol. 2005, pp. 67–73, 2005.
formation diffusion in socially-aware mobile networks,” Special section [45] N. V. Chawla, “Data mining for imbalanced datasets: An overview,” in
on recent advances in socially aware mobile networking IEEEAccess, Data mining and knowledge discovery handbook. Springer, 2005, pp.
vol. 4, pp. 3907–3919, Aug 2016. 853–867.
[23] U. D. Prasad and S. Madhavi, “Prediction of churn behavior of bank [46] G. M. Weiss, “Mining with rarity: a unifying framework,” ACM SIGKDD
customers using data mining tools,” Business Intelligence Journal, vol. 5, Explorations Newsletter, vol. 6, no. 1, pp. 7–19, 2004.
no. 1, pp. 96–101, 2012. [47] F. He, X. Wang, and B. Liu, “Attack detection by rough set theory in
[24] K. Chitra and B. Subashini, “Customer retention in banking sector using recommendation system,” in Granular Computing (GrC), 2010 IEEE
predictive data mining technique.” ICIT 2011 The 5th International International Conference on. IEEE, 2010, pp. 692–695.
Conference on Information Technology, 2011. [48] G. E. Batista, R. C. Prati, and M. C. Monard, “A study of the behavior
[25] J. Bloemer, K. De Ruyter, and P. Peeters, “Investigating drivers of bank of several methods for balancing machine learning training data,” ACM
loyalty: the complex relationship between image, service quality and Sigkdd Explorations Newsletter, vol. 6, no. 1, pp. 20–29, 2004.
satisfaction,” International Journal of bank marketing, vol. 16, no. 7, [49] N. V. Chawla, A. Lazarevic, L. O. Hall, and K. W. Bowyer, “Smoteboost:
pp. 276–286, 1998. Improving prediction of the minority class in boosting,” in European
[26] M. A. H. Farquad, V. Ravi, and S. B. Raju, “Churn prediction using Conference on Principles of Data Mining and Knowledge Discovery.
comprehensible support vector machine: An analytical crm application,” Springer, 2003, pp. 107–119.
Applied Soft Computing, vol. 19, pp. 31–40, 2014. [50] B. Liu, Y. Ma, and C. K. Wong, “Improving an association rule based
[27] C.-S. Lin, G.-H. Tzeng, and Y.-C. Chin, “Combined rough set theory and classifier,” in European Conference on Principles of Data Mining and
flow network graph to predict customer churn in credit card accounts,” Knowledge Discovery. Springer, 2000, pp. 504–509.
Expert Systems with Applications, vol. 38, no. 1, pp. 8–15, 2011. [51] K. Satyasree and J. Murthy, “An exhaustive literature review on class
[28] K. Lee, N. Chung, and K. Shin, “An artificial intelligence-based data imbalance problem,” Int. J. Emerg. Trends Technol. Comput. Sci, vol. 2,
mining approach to extracting strategies for reducing the churning rate in pp. 109–118, 2013.
credit card industry,” Journal of Intelligent Information Systems, vol. 8, [52] M. Kubat, S. Matwin et al., “Addressing the curse of imbalanced training
no. 2, pp. 15–35, 2002. sets: one-sided selection,” in ICML, vol. 97. Nashville, USA, 1997,
[29] M. Suznjevic, I. Stupar, and M. Matijasevic, “Mmorpg player behavior pp. 179–186.
model based on player action categories,” Proceedings of the 10th [53] C. X. Ling and C. Li, “Data mining for direct marketing: Problems and
Annual Workshop on Network and Systems Support for Games. IEEE solutions.” in KDD, vol. 98, 1998, pp. 73–79.
Press, 2011, p. 6. [54] Y. Tang, S. Krasser, D. Alperovitch, and P. Judge, “Spam sender
[30] J. Kawale, A. Pal, and J. Srivastava, “Churn prediction in mmorpgs: detection with classification modeling on highly imbalanced mail server
A social influence based approach,” in International Conference on behavior data.” in Artificial Intelligence and Pattern Recognition, 2008,
Computational Science and Engineering, 2009. CSE’09, vol. 4. IEEE, pp. 174–180.
2009, pp. 423–428. [55] E. Y. G. Wu and Chang, “Kernel boundary alignment considering
unbalanced data distribution,” IEEE Trans. Knowl. Data Eng, vol. 17,
[31] M. L. Kane-Sellers, “Predictive models of employee voluntary turnover
no. 6, pp. 786–796, 2006.
in a north american professional sales force using data-mining analysis,”
2007. [56] P. Foster, “Machine learning from imbalanced data sets 101.” Proceed-
ings of the AAAI 2000 workshop on imbalanced data sets, 2000, pp.
[32] V. V. Saradhi and G. K. Palshikar, “Employee churn prediction,” Expert
1–3.
Systems with Applications, vol. 38, no. 3, pp. 1999–2006, 2011.
[57] H. Guo and H. L. Viktor, “Learning from imbalanced data sets
[33] M. Saron and Z. A. Othman, “Academic talent model based on human with boosting and data generation: the databoost-im approach,” ACM
resource data mart,” International Journal of Research in Computer SIGKDD Explorations Newsletter, vol. 6, no. 1, pp. 30–39, 2004.
Science, vol. 2, no. 5, p. 29, 2012.
[58] H. Peng, F. Long, and C. Ding, “Feature selection based on mu-
[34] G. Dror, D. Pelleg, O. Rokhlenko, and I. Szpektor, “Churn prediction tual information criteria of max-dependency, max-relevance, and min-
in new users of yahoo! answers.” Proceedings of the 21st International redundancy,” IEEE Transactions on pattern analysis and machine intel-
Conference on World Wide Web, ACM, 2012, pp. 829–834. ligence, vol. 27, no. 8, pp. 1226–1238, 2005.
[35] M. Jaudet, N. Iqbal, and A. Hussain, “Neural networks for fault- [59] S. Barua, M. M. Islam, X. Yao, and K. Murase, “Mwmote–majority
prediction in a telecommunications network,” Multitopic Conference, weighted minority oversampling technique for imbalanced data set
2004. Proceedings of INMIC 2004. 8th International. IEEE, 2004, pp. learning,” IEEE Transactions on Knowledge and Data Engineering,
315–320. vol. 26, no. 2, pp. 405–425, 2014.
[36] A. Hawalah and M. Fasli, “Dynamic user profiles for web personalisa- [60] M.-F. Tsai and S.-S. Yu, “Distance metric based oversampling method
tion,” Expert Systems with Applications, vol. 42, no. 5, pp. 2547–2569, for bioinformatics and performance evaluation,” Journal of medical
2015. systems, vol. 40, no. 7, pp. 1–9, 2016.
[37] N. Ahad, J. Qadir, and N. Ahsan, “Neural networks in wireless net- [61] X. Ai, J. Wu, V. S. Sheng, P. Zhao, Y. Yao, and Z. Cui, “Immune cen-
works: Techniques, applications and guidelines,” Journal of Network troids over-sampling method for multi-class classification,” in Pacific-
and Computer Applications, vol. 68, pp. 1–27, 2016. Asia Conference on Knowledge Discovery and Data Mining. Springer,
[38] Z. L. Cheng Fang, Jun Liu, “Fine-grained http web traffic analysis based 2015, pp. 251–263.
on large-scale mobile datasets,” IEEEAccess, vol. 4, pp. 4364–43–73, [62] Z. Pawlak, “Rough sets,” International Journal of Computer & Infor-
Aug 2016. mation Sciences, vol. 11, no. 5, pp. 341–356, 1982.
[39] T. S. Rappaport, S. Sun, R. Mayzus, H. Zhao, Y. Azar, K. Wang, G. N. [63] Z. Pawlak and A. Skowron, “Rough sets and conflict analysis,” in E-
Wong, J. K. Schulz, M. Samimi, and F. Gutierrez, “Millimeter wave Service Intelligence. Springer, 2007, pp. 35–74.
mobile communications for 5g cellular: It will work!” IEEE access, [64] J. G. Bazan, H. S. Nguyen, S. H. Nguyen, P. Synak, and J. Wróblewski,
vol. 1, pp. 335–349, 2013. “Rough set algorithms in classification problem,” in Rough set methods
[40] R. A. Soeini and K. V. Rodpysh, “Applying data mining to insurance and applications. Springer, 2000, pp. 49–88.
customer churn management,” International Proceedings of Computer [65] J. G. Bazan and M. Szczuka, “The rough set exploration system,” in
Science and Information Technology, vol. 30, pp. 82–92, 2012. Transactions on Rough Sets III. Springer, 2005, pp. 37–56.
[41] K. Coussement and D. Van den Poel, “Churn prediction in subscription [66] H. Nguyen and S. Nguyen, “Analysis of stulong data by rough set ex-
services: An application of support vector machines while comparing ploration system (rses),” in Proceedings of the ECML/PKDD Workshop,
two parameter-selection techniques,” Expert systems with applications, 2003, pp. 71–82.
vol. 34, no. 1, pp. 313–327, 2008. [67] J. Wróblewski, “Genetic algorithms in decomposition and classification
[42] J. Burez and D. Van den Poel, “Crm at a pay-tv company: Using problems,” in Rough Sets in Knowledge Discovery 2. Springer, 1998,
analytical models to reduce customer attrition by targeted marketing pp. 471–487.
for subscription services,” Expert Systems with Applications, vol. 32, [68] J. W. Grzymala-Busse, “A new version of the rule induction system
no. 2, pp. 277–288, 2007. lers,” Fundamenta Informaticae, vol. 31, no. 1, pp. 27–39, 1997.

JOURNAL OF IEEEACCESS 17

[69] J. W. Grzymala Busse, “Lers-a system for learning from examples based Sajid Anwar obtained his BSc (Comp. Sc) and MSc
on rough sets,” in Intelligent decision support. Intelligent decision (Comp. Sc) degrees from University of Peshawar
support: Springer, 1992, pp. 3–18. in 1997 and 1999 respectively. He obtained his
[70] V. López, A. Fernández, S. Garcı́a, V. Palade, and F. Herrera, “An insight MS (Comp.Sc) and PhD (in Software Architecture)
into classification with imbalanced data: Empirical results and current from the University of NUCES-FAST, Pakistan, in
trends on using data intrinsic characteristics,” Information Sciences, vol. 2007 and 2011 respectively. He is currently Assistant
250, pp. 113–141, 2013. Professor of Computing Science, and coordinator of
[71] “Data Source,” http://www.sgi.com/tech/mlc/db/, Aug. 2015, [Online; the BS-Software Engineering at the Institute of Man-
accessed 1-Aug-2015]. agement Sciences Peshawar, Pakistan. His research
[72] “Data Source,” http://lamda.nju.edu.cn/yuy/dm07/assign2.htm, Oct. interests are concerned with Software Architecture,
2015, [Online; accessed 15-Oct-2015]. Software Requirement Engineering, Searched Based
[73] “KDD’06 Challenge Dataset,” http://www3.ntu.edu.sg/sce/pakdd2006/, Software Engineering and Mining Software Repository.
Oct. 2015, [Online; accessed 15-Oct-2015].
[74] “IBM Telecom Dataset,” https://www.ibm.com/analytics/watson-
analytics/community/predictive-insights-in-the-telco-customer-churn-
data-set/, Jan. 2016, [Online; accessed 1-Jan-2016].
[75] G. Holmes, A. Donkin, and I. H. Witten, “Weka: A machine learning
workbench,” in Intelligent Information Systems, 1994. Proceedings of
the 1994 Second Australian and New Zealand Conference on. IEEE,
1994, pp. 357–361.
[76] Microsoft, “Standardize Function,” http://office.microsoft.com/en-
001/excel-help/standardize-function-HP010342919.aspx , 2014, [Online;
accessed 9-Nov-2014].
[77] R. Bellazzi and B. Zupan, “Predictive data mining in clinical medicine: Awais Adnan is Assistant Professor and Cooerdina-
current issues and guidelines,” International journal of medical infor- tor of Master Program, Department of Comptuer Sci-
matics, vol. 77, no. 2, pp. 81–97, 2008. ence in Institute of Management Sciences Peshawar.
[78] H. S. Almuallim, “Concept coverage and its application to two learning He has done his Ph.D. from IMSciences|Peshawar
tasks,” 1992. and MS from NUST Islamabad. He is manager
[79] M. R. K. Dhurjad and M. S. Banait, “A survey on oversampling of ORIC in IMSciences|Peshawar to promote and
techniques for imbalanced learning,” International journal of Appl. or facilitate the research students in commercialization
Innovation. Engineering. Management, vol. 3, no. 1, pp. 279–284, 2014. of their research. His major areas of interest are
[80] N. H. Ruparel, N. M. Shahane, and D. P. Bhamare, “Learning from Multimedia and Machine Learning.
small data set to build classification model: A survey,” IJCA Proceedings
on International Conference on Recent Trends in Engineering and
Technology 2013, vol. ICRTET, no. 4, pp. 23–26, May 2013, full text
available.
[81] J. Demsar, “Statistical comparisons of classifiers over multiple data sets,”
Journal of Machine learning research, vol. 7, no. Jan, pp. 1–30, 2006.
[82] Minitab, “Choosing Between a Nonparametric Test and a Paramet-
ric Test,” http://blog.minitab.com/blog/adventures-in-statistics/choosing-
between-a-nonparametric-test-and-a-parametric-test, 2015, [Online; ac-
cessed 19-Dec-2015].
[83] B. S. Everitt, The analysis of contingency tables. CRC Press, 1992.
[84] T. G. Dietterich, “Approximate statistical tests for comparing supervised Muhammad Nawaz received his MSc (Computer
classification learning algorithms,” Neural computation, vol. 10, no. 7, Science) and MS in Information Technology from
pp. 1895–1923, 1998. University of Peshawar-Pakistan. He worked as a
lecturer at the University of Peshawar; followed
by working as a Computer Programmer at Khyber
Teaching Hospital, Peshawar; and then appointed as
Assistant Professor in Multimedia at the Institute of
Management Sciences, Peshawar - a position he still
holds. Currently he is the Head of PhD and MS-
Computer Sciences at the IMSciences |Peshawar.

Newton Howard is the Chairman of the Brain Sci-

Adnan Amin received the MSc degree in Computer ences Foundation and currently serves as Associate
Science from University of Peshawar, MS degree Professor of Computational Neuroscience and Func-
(with Distinction) in Computer Science with major tional Neurosurgery at the University of Oxford,
Databases from Institute of Management Sciences where he manages the newly-formed Computational
Peshawar, Pakistan in 2008 and 2015 respectively. Neuroscience Lab. He is also the Director of the
He is currently a PhD Scholar and Lecturer at Synthetic Intelligence Laboratory at MIT, where
Department of Computing Sciences at Institute of he had served as the Founder and Director of the
Management Sciences Peshawar. His research inter- MIT Mind Machine Project from 2008 to 2012. Dr
ests include Data mining, Databases, Big Data and Howard is an active member of several research
Machine Learning. laboratories worldwide, including the Descartes In-
stitute, the Brain Physics Group and INSERM, in Paris.

JOURNAL OF IEEEACCESS 18

Junaid Qadir completed PhD from University of

New South Wales, Australia in 2008 and his Bach-
elors in Electrical Engineering from UET, Lahore,
Pakistan in 2000. He is joining Information Technol-
ogy University?Punjab, Lahore in December 2015
as an Associate Professor. Previously, he has served
as an Assistant Professor at the School of Electri-
cal Engineering and Computer Sciences (SEECS),
National University of Sciences and Technology
(NUST), from 2008 to 2015. He is an award-winning
teacher who has been awarded the highest national
teaching award in Pakistan?the higher education commission?s (HEC) best
university teacher award?for the year 2012-2013. He has been nominated for
the HEC best university teacher award award twice from NUST (2011, and
2012-2013). He has 7+ years of teaching experience in which he has taught a
wide portfolio of courses in systems and networking; signal processing; and
wireless communications and networking. His primary research interests are in
the areas of computer systems and networking and using ICT for development
(ICT4D). He has served in the program committee of a number of international
conferences and reviews regularly for various high-quality journals. He is
an Associate Editor for IEEE Access, Springer/BioMed Central?s Big Data
Analytics journal, and the IEEE Communications Magazine. He is a member
of ACM, and a senior member of IEEE.

Ahmad Hawala is the vice Dean and Head of IS

department Taibah University College of the Com-
puter Science and Engineering Saudi Arabia Madina.
His research interests include Data mining, Artificial
Intelligence and Information sciences.

Amir Hussain obtained his BEng (with the high-

est 1st Class Honours) and PhD (in novel neural
network architectures and algorithms) from the Uni-
versity of Strathclyde in Glasgow, Scotland, UK, in
1992 and 1997 respectively. He is currently Profes-
sor of Computing Science, and founding Director of
the Cognitive Signal Image and Control Processing
Research (COSIPRA) Laboratory at the University
of Stirling in Scotland, UK. He has conducted and
led collaborative research with industry; partnered in
major European research programs, and supervised
more than 20 PhDs. He has published over 270 papers, including over a
dozen books and 80 journal papers. He is founding Editor-in-Chief of the
journals: Cognitive Computation (Springer Neuroscience, USA), and Big Data
Analytics (BioMed Central), and Chief-Editor of the Springer Book Series on
Socio-Affective Computing, and SpringerBriefs on Cognitive Computation.
He is an Associate Editor of the IEEE Transactions on Neural Networks and
Learning Systems, a member of several Technical Committees of the IEEE
Computational Intelligence Society (CIS), founding publications co-Chair of
the IINNS Big Data Section and its annual INNS Conference on Big Data,
and Chapter Chair of the IEEE UK and RI Industry Applications Society.

Warrior 1400 Illustrated Parts Catalog Revision 11
100% (1)
Warrior 1400 Illustrated Parts Catalog Revision 11
215 pages
Grit Scale
100% (2)
Grit Scale
2 pages
TEMJournalAugust2024_1927_1936
No ratings yet
TEMJournalAugust2024_1927_1936
10 pages
Comparative_Study_of_Customer_Churn_Prediction_Based_on_Data_Ensemble_Approach
No ratings yet
Comparative_Study_of_Customer_Churn_Prediction_Based_on_Data_Ensemble_Approach
10 pages
A Customer Churn Prediction Using Pearson Correlation Function and K Nearest Neighbor Algorithm For Telecommunication Industry
No ratings yet
A Customer Churn Prediction Using Pearson Correlation Function and K Nearest Neighbor Algorithm For Telecommunication Industry
14 pages
Churn Prediction
100% (3)
Churn Prediction
41 pages
Churn Prediction Using Machine Learning Models
No ratings yet
Churn Prediction Using Machine Learning Models
6 pages
Literature Review
No ratings yet
Literature Review
4 pages
Ispanyakonferansi
No ratings yet
Ispanyakonferansi
5 pages
Research paper_Tushar Agrawal
No ratings yet
Research paper_Tushar Agrawal
3 pages
Customer_Churn_Prediction_employing_Ensemble_Learning
No ratings yet
Customer_Churn_Prediction_employing_Ensemble_Learning
5 pages
Customer Churn Prediction Using Machine Learning Algorithms
No ratings yet
Customer Churn Prediction Using Machine Learning Algorithms
5 pages
Customer Churn Prediction Using Machine Learning Approaches
No ratings yet
Customer Churn Prediction Using Machine Learning Approaches
7 pages
An Effective Classifier for Predicting Churn in Telecommunication
No ratings yet
An Effective Classifier for Predicting Churn in Telecommunication
9 pages
Customer_Churn_Prediction_Using_Machine_Learning_Algorithms
No ratings yet
Customer_Churn_Prediction_Using_Machine_Learning_Algorithms
6 pages
Customer Churn Prediction
No ratings yet
Customer Churn Prediction
6 pages
Ref 1
No ratings yet
Ref 1
10 pages
Décortication article 1
No ratings yet
Décortication article 1
4 pages
Customer Churn Prediction in Telecommunication
No ratings yet
Customer Churn Prediction in Telecommunication
13 pages
Customer Churn Analysis and Prediction
No ratings yet
Customer Churn Analysis and Prediction
4 pages
Churn PredictionITNACC
No ratings yet
Churn PredictionITNACC
7 pages
Computing Efficient Features Using Rough Set Theory Combined With Ensemble Classification Techniques To Improve The Customer Churn Prediction in Telecommunication Sector
No ratings yet
Computing Efficient Features Using Rough Set Theory Combined With Ensemble Classification Techniques To Improve The Customer Churn Prediction in Telecommunication Sector
22 pages
Customer Churn Prediction System: A Machine Learning Approach
No ratings yet
Customer Churn Prediction System: A Machine Learning Approach
24 pages
Churn Analysis Report
No ratings yet
Churn Analysis Report
28 pages
A Novel Approach For Churn Prediction Using Minimal Factor Selection Approach
No ratings yet
A Novel Approach For Churn Prediction Using Minimal Factor Selection Approach
5 pages
ssrn-4976040
No ratings yet
ssrn-4976040
14 pages
1-s2.0-S2590123024014208-main
No ratings yet
1-s2.0-S2590123024014208-main
12 pages
PFEreport
No ratings yet
PFEreport
43 pages
56 Customer Churn Analysis and Prediction Using Data Mining Models in Banking Industry
No ratings yet
56 Customer Churn Analysis and Prediction Using Data Mining Models in Banking Industry
6 pages
Customer Churn Prediction in Telecom Sector Using Machine Learning Techniques
No ratings yet
Customer Churn Prediction in Telecom Sector Using Machine Learning Techniques
16 pages
Ref 4
No ratings yet
Ref 4
16 pages
Machine Learning Based Customer Churn Prediction in Banking: November 2020
No ratings yet
Machine Learning Based Customer Churn Prediction in Banking: November 2020
7 pages
ADD6
No ratings yet
ADD6
9 pages
CHURNFORGE Research Paper Kajal
No ratings yet
CHURNFORGE Research Paper Kajal
6 pages
major project
No ratings yet
major project
27 pages
stress final report
No ratings yet
stress final report
57 pages
Seminar Synopsisreport
No ratings yet
Seminar Synopsisreport
6 pages
Customer Churn Prediction For Telecom Services: Utku Yabas Hakki Candan Cankaya Turker Ince
No ratings yet
Customer Churn Prediction For Telecom Services: Utku Yabas Hakki Candan Cankaya Turker Ince
2 pages
BIL report 2
No ratings yet
BIL report 2
11 pages
Bank Customer Churn Prediction
No ratings yet
Bank Customer Churn Prediction
5 pages
Customer Churn Prediction
No ratings yet
Customer Churn Prediction
5 pages
IJSC_Vol_10_Iss_2_Paper_5_2054_2060
No ratings yet
IJSC_Vol_10_Iss_2_Paper_5_2054_2060
7 pages
Paper Published
No ratings yet
Paper Published
5 pages
Anticipating Customer Churn in Telecommunication Using Machine Learning Algorithms For Customer Retention
No ratings yet
Anticipating Customer Churn in Telecommunication Using Machine Learning Algorithms For Customer Retention
7 pages
A Comparison of Machine Learning Algorithms for Customer Churn Prediction
No ratings yet
A Comparison of Machine Learning Algorithms for Customer Churn Prediction
6 pages
Ali Tamaddoni Jahromi, Mehrad Moeini, Issar Akbari, Aram Akbarzadeh
No ratings yet
Ali Tamaddoni Jahromi, Mehrad Moeini, Issar Akbari, Aram Akbarzadeh
11 pages
Vicky
No ratings yet
Vicky
11 pages
A Survey and Implementation of Machine Learning Algorithms For Customer Churn Prediction
No ratings yet
A Survey and Implementation of Machine Learning Algorithms For Customer Churn Prediction
7 pages
2017 Paper 10
No ratings yet
2017 Paper 10
5 pages
A Proposed Churn Prediction Model: Essam Shaaban, Yehia Helmy, Ayman Khedr, Mona Nasr
No ratings yet
A Proposed Churn Prediction Model: Essam Shaaban, Yehia Helmy, Ayman Khedr, Mona Nasr
5 pages
AReviewon Oversampling Techniquesfor Solvingthe Data Imbalance Problemin Classification
No ratings yet
AReviewon Oversampling Techniquesfor Solvingthe Data Imbalance Problemin Classification
11 pages
Model 1 - Customer Churn Prediction in Telecom Using ML
No ratings yet
Model 1 - Customer Churn Prediction in Telecom Using ML
24 pages
Customer Churn Prediction in Banking Industry Using Power-Bi
No ratings yet
Customer Churn Prediction in Banking Industry Using Power-Bi
9 pages
2 Customer churning analysis using machine learning algorithms
No ratings yet
2 Customer churning analysis using machine learning algorithms
10 pages
CUSTOMER CHURN PREDICTION AND CATEGORIZATION A MACHINE LEARNING APPROACH TO ANALYSE CUSTOMER BEHAVIOR AND DECISION MAKING IN THE TELECOMMUNICATIONS INDUSTRY
No ratings yet
CUSTOMER CHURN PREDICTION AND CATEGORIZATION A MACHINE LEARNING APPROACH TO ANALYSE CUSTOMER BEHAVIOR AND DECISION MAKING IN THE TELECOMMUNICATIONS INDUSTRY
30 pages
Full Text 01
No ratings yet
Full Text 01
26 pages
Rahman 2020
No ratings yet
Rahman 2020
6 pages
review1-1
No ratings yet
review1-1
16 pages
A Review On Churn Prediction and Customer Segmentation Using Machine Learning
No ratings yet
A Review On Churn Prediction and Customer Segmentation Using Machine Learning
5 pages
Comparison of Learning Techniques For Prediction of Customer Churn in Telecommunication
No ratings yet
Comparison of Learning Techniques For Prediction of Customer Churn in Telecommunication
36 pages
Multi Agent System: Fundamentals and Applications
From Everand
Multi Agent System: Fundamentals and Applications
Fouad Sabry
No ratings yet
Data Mining 101: Core Concepts and Algorithms
From Everand
Data Mining 101: Core Concepts and Algorithms
Swarnalata Verma
No ratings yet
Computer Anxiety Scale Paper
No ratings yet
Computer Anxiety Scale Paper
6 pages
R Programming
No ratings yet
R Programming
37 pages
Course Plan Data Mining
No ratings yet
Course Plan Data Mining
1 page
SPSS Training Course: Hazhar Talaat Abubaker Blbas
No ratings yet
SPSS Training Course: Hazhar Talaat Abubaker Blbas
79 pages
Sorter
No ratings yet
Sorter
1 page
The Six Minute Book Summary of The Goal by Eliyahu M
100% (1)
The Six Minute Book Summary of The Goal by Eliyahu M
5 pages
Improvement of Aerodynamic Characteristics
No ratings yet
Improvement of Aerodynamic Characteristics
9 pages
Passenger Elevator Brochure
No ratings yet
Passenger Elevator Brochure
20 pages
What Is The Difference Between End To End To Implementation
No ratings yet
What Is The Difference Between End To End To Implementation
2 pages
05-1-Collection of Problems PDF
No ratings yet
05-1-Collection of Problems PDF
15 pages
Sunbreak 13.0.0.0 (TU3) Gunlance Meta Sets
No ratings yet
Sunbreak 13.0.0.0 (TU3) Gunlance Meta Sets
14 pages
Uauc 2015
No ratings yet
Uauc 2015
230 pages
Knowledge: As New Member EAC Community of Learners
No ratings yet
Knowledge: As New Member EAC Community of Learners
2 pages
Homework
No ratings yet
Homework
4 pages
One View by AKTU SDC Team1
No ratings yet
One View by AKTU SDC Team1
2 pages
Main Relief Valve
No ratings yet
Main Relief Valve
2 pages
BLGD PRM Form 1 Instructions in Filling Out The Provincial Results Matrices (Working File)
No ratings yet
BLGD PRM Form 1 Instructions in Filling Out The Provincial Results Matrices (Working File)
3 pages
TM100 Manual
No ratings yet
TM100 Manual
2 pages
Brain Fingerprinting Technology
No ratings yet
Brain Fingerprinting Technology
18 pages
class-xii-result-analysis-project
No ratings yet
class-xii-result-analysis-project
16 pages
Conversion of Shapes and Textures
No ratings yet
Conversion of Shapes and Textures
3 pages
Mba Mco101 Unit 5
No ratings yet
Mba Mco101 Unit 5
15 pages
Composite Materials
100% (1)
Composite Materials
399 pages
Duplex Stainless Steels - A Simple Guide: How The Austenite/Ferrite Balance Is Achieved
No ratings yet
Duplex Stainless Steels - A Simple Guide: How The Austenite/Ferrite Balance Is Achieved
4 pages
Two Post Surface Mounted Lift: I, O & M M
No ratings yet
Two Post Surface Mounted Lift: I, O & M M
15 pages
Paul Virilio - The Third Interval - Artículo
No ratings yet
Paul Virilio - The Third Interval - Artículo
7 pages
Mandiberg TheSocialMediaReader CC by Sa NC
100% (1)
Mandiberg TheSocialMediaReader CC by Sa NC
300 pages
Basic Safety Induction
No ratings yet
Basic Safety Induction
37 pages
Test Bank for Applied Statistics in Business and Economics, 4th Edition: David Doanepdf download
100% (7)
Test Bank for Applied Statistics in Business and Economics, 4th Edition: David Doanepdf download
45 pages
CH 6 Well Control
No ratings yet
CH 6 Well Control
115 pages
Pressure Vessels SHO Programme
No ratings yet
Pressure Vessels SHO Programme
42 pages
Pharmacy Informatics
100% (1)
Pharmacy Informatics
51 pages
CIM, Database System and Product Life Cycle
No ratings yet
CIM, Database System and Product Life Cycle
23 pages
WDR Jhoni XLV
No ratings yet
WDR Jhoni XLV
3 pages
Grounding, Backfeed Breakers & PV Disconnect 2002
No ratings yet
Grounding, Backfeed Breakers & PV Disconnect 2002
3 pages

Class Imbalance Paper

Uploaded by

Class Imbalance Paper

Uploaded by

This article has been accepted for publication in a future issue of this journal, but has not been

Comparing oversampling techniques to handle the

I N many subscription-based service industries such as

Fig. 1: When overlapping occurs then hard to induce the

generates artificial data to balance the dataset. In their work, C. mRMR

the right decision in classification. It is computing the MI

• Imbalanced Ratio (IR): The imbalance ratio can be

TABLE II: Feature Subset Extracted Through mRMR

TABLE IX: Performance evaluation of classifier for dataset 1

TABLE X: Performance evaluation of classifier for dataset 2

TABLE XI: Performance evaluation of classifier for dataset 3

TABLE XII: Performance evaluation of classifier for dataset 4

The following acronyms have been used in Figures 4- 6:

Gen M: Genertic Algorithm & MTDF (dataset oversampled with MTDF).

TABLE XIII: Comparison between regular accuracy & class

Gen I 0.929+0.978 0.862+0.923 0.915+0.959 0.802+0.885

tools or proprietary data, which may lead to variance in R EFERENCES

Newton Howard is the Chairman of the Brain Sci-

Junaid Qadir completed PhD from University of

Ahmad Hawala is the vice Dean and Head of IS

Amir Hussain obtained his BEng (with the high-

You might also like