2024 3rd International Conference on Advancement in

Electrical and Electronic Engineering (ICAEEE 2024),

25-27 April 2024, Gazipur, Bangladesh

PID 89: Analysis and Performance

Evaluation of Credit Card Fraud Detection
by Multi-model ML
PRESENTED BY
Md Aminul Islam, A. T. M. Asif Imran, Md
Habibur Rahman ,
Md Amran Hossen Pabel
Corresponding Email: [email protected] or
[email protected]
Table of Contents
Abstract 03

Research Keywords 04

Problem Statement 05

Research Questions 06

Literature Review 07-08

Methodology 09-12

Performance Matrices 13-14

Conclusion 15

References 16

2
Abstract

• The project Credit Card Fraud aims to address the significant impact of credit card fraud on
financial institutions and individuals by leveraging machine learning techniques for more accurate
and efficient fraud detection.

• The study aims to overcome challenges such as unbalanced data, dynamic fraud tendencies, and
false alarms, ultimately improving detection precision and reducing financial losses.

• Through comparative evaluation, the project seeks to demonstrate the superiority of the proposed
multi-model approach, particularly highlighting the effectiveness of data balancing techniques in
reducing false negatives and enhancing overall detection performance.

3
Research Keywords
This refers to the field of study that enables computers to learn and improve from
experience without being explicitly programmed. In this project, various machine
Machine Learning learning techniques such as Extreme Learning, Decision Trees, Random Forests,
Support Vector Machines, Logistic Regression, and XG Boost are employed to detect
credit card fraud.
This keyword pertains to the payment mechanism used in financial transactions,
where individuals can make purchases or borrow funds up to a predetermined credit
Credit Card
limit. Credit card fraud occurs when unauthorized transactions are made using stolen
or compromised credit card information.
Fraud involves dishonesty for personal or financial gain. In the context of credit
Fraud cards, fraud refers to unauthorized or illegal activities such as unauthorized
transactions, identity theft, or account takeover.
Detection refers to the process of identifying and recognizing instances of fraud
Detection within credit card transactions. In this project, machine learning techniques are
utilized for detecting patterns and anomalies indicative of fraudulent activities.
Public data encompasses information that is openly available and accessible to the
general public. In the context of this project, the availability of public data may
Public Data
influence the analysis and performance evaluation of credit card fraud detection
methods.

4
Problem Statement Scope of the Study

The prevalence of credit card fraud poses This study focuses on analyzing and addressing
significant financial losses, mental health challenges associated with credit card fraud
concerns, and reputational risks for financial detection using machine learning techniques. It
institutions. Despite the availability of public data, aims to evaluate the effectiveness of various ML
unbalanced datasets, dynamic fraud tendencies, models, including LR, SVM, GaussianNB,
and false alarms remain obstacles in effectively Quadratic Discriminant Analysis, and RF
detecting and preventing fraud. Therefore, there is Classifier, on a European card benchmark dataset.
a pressing need to develop and implement The scope extends to comparing the performance
advanced machine learning techniques to enhance of these models in terms of accuracy, precision, f1-
credit card fraud detection and mitigate its adverse score, and AUC curves. Additionally, the study
effects. investigates the impact of data balancing
techniques on reducing false negatives and
improving overall detection efficiency.

5
Research Questions

• What are the key challenges in credit card fraud detection, and how can machine learning
techniques address them?
• How do different machine learning models, including Logistic Regression, Support Vector
Machine, GaussianNB, Quadratic Discriminant Analysis, and Random Forest Classifier, perform
in detecting credit card fraud?
• What is the impact of incorporating data balancing techniques on improving the accuracy and
efficiency of credit card fraud detection models?
• How does the proposed model compare to contemporary machine learning approaches in terms
of performance and effectiveness?
• What are the implications of this study for real-world applications of credit card fraud detection
systems?

6
Literature Review
● Emilija S et al. (2023)
○ Proposed a novel data augmentation method called K-CGAN to enhance classification performance
in credit card fraud detection.
● Seyedeh K et al., (2023)
○ Employed weight-tuning and Bayesian optimization to preprocess imbalanced data and utilized
CatBoost and XGBoost models.

● Haicho D et al., (2023)

○ Introduced an autoencoder with probabilistic LightGBM and AED-LGB algorithms for fraud
detection.

● Faawaz K et al., (2022)

○ Compared ML and deep learning models, achieving efficient results with a deep learning approach
and proposing a real-world fraud detection method with 96.34% accuracy in CNN.

7
Literature Review
● Saeid B et al., (2023)
○ Explored various data mining solutions using LightGBM and LiteMORT algorithms. They obtained
high accuracies and suggested deep neural networks for future accuracy enhancements.
● Dhiman S et al., (2020)
○ Proposed a methodology employing a community detection algorithm for detecting potential
instances of bank fraud. Their agile methodology-based model architecture comprised training,
storage, and detection modules.

● J Femila et al., (2022)

○ Introduced the Long Short-Term Memory-Recurrent Neural Network (LSTM-RNN) for fraud
detection, achieving 100% accuracy and comparing their model with Naïve Bayes, SVM, and ANN.

● Oona V et al., (2021)

○ Demonstrated a deep learning approach using binary cross-entropy loss function and the Adam
optimizer. They achieved high classification accuracy of 99.76% on test data with 3000 samples.

8
Methodology

The process begins with obtaining the

Datasets original dataset.

Preprocessin Focused on three preprocessing

g including Missing Values, Normalization,
Flowcha Data Encoding

t Comprehensi A thorough examination of the

preprocessed data to understand
ve Analysis patterns, correlations, and potential
outliers.
The performance of the model is
evaluated using various metrics such as
Evaluation accuracy, precision, recall, F1-score,
etc.
9
Working Flowchart

This diagram illustrates the sequential flow of steps

involved in a machine learning project, from data
acquisition to model evaluation, highlighting the
importance of preprocessing, model training, and
evaluation in the process.

10
Accuracy of Various models
Name of the methods Accuracy
(%)
Quadratic Discriminant Analysis 94
GaussianNB 91

DT Classifier 90

RF Classifier 90
Extra Trees Classifier 90
Bagging Classifier 90

XGBoost Classifier 90

KNeighbors Classifier 90

Linear Discriminant Analysis 88

Nearest Centroid 88

Linear SVM 85

11
Accuracy of Various models
Name of the methods Accuracy
(%)
SVM 84
Ada Boost Classifier 83
Extra Tree Classifier 83
Bernoulli NB 81
Perception 84
SVM 84
Logistic Regression 80
Calibrated Classifier CV 77
LGB Classifier 76
SGD Classifier 75
Ridge Classifier 72
Ridge Classifier CV 72
Passive Aggressive Classifier 68
Dummy Classifier 50

12
Different Performance Metrics score

13
Comparison of LR Performance Metric

14
Ref. Data set Model Precision Accuracy
(%) (%)
[3] Kaggle Random Forest 100 99
dataset Nearest Neighbor 100 99
Logistic Regression 100 99
[4] Credit Card Logistic Regression 96 97
Dataset LGBM 75 99
XGB 78 99
CatBoost 64 99
Credit Card AED-LGB – 99
[5] Transaction
Dataset
[6] Credit Card CNN _ 96.34
Dataset BL _ 99.72
RF _ 99.92
SVM _
[7] Credit Card LightGBM 85.04 99.09
Fraud LiteMORT 88.65 99.26
Detection Simple Average 90.71 99.35
dataset from Weighted Average 92.79 95.2
Kaggle
website
[8] Kaggle SVM 68 86.56
Dataset ANN 92 91.84
LSTM-RNN 100 100
[9] Kaggle LSTM 98 96
website
Limitations, and Future Work
● Real Time Data
● Robust Data
● More Complex but Simple Solution

16
Conclusions

The study evaluated 23 distinct machine learning classifiers for the detection of credit card
fraud, with Quadratic Discriminant Analysis achieving the highest accuracy rate of 94%.
Through comparative analysis, it was found that XGBoost outperformed other methods in
terms of accuracy, Area Under the Curve (AUC), Precision, and F1 score.

Efficient identification of credit card fraud is essential to mitigate losses for both cardholders
and financial institutions. The study presents an intelligent framework utilizing authentic
datasets and cutting-edge machine-learning techniques for fraud detection. Empirical
findings underscore the superiority of XGBoost and emphasize the importance of competent
parameter optimization strategies for enhancing predictive capabilities. Future research can
leverage these findings to develop advanced tools for protecting credit card users, utilizing
the latest datasets and experiences gained from this study.

17
References
[1] Van Belle, R., Baesens, B., & De Weerdt, J. (2023). Catchm: A novel network-based credit card fraud detection method
using node representation learning. Decision Support Systems, 164, 113866. https://doi.org/10.1016/j.dss.2022.113866
[2] Forensic credit card fraud detection using Deep Neural Network. (2023). Journal of Southwest Jiaotong University, 58(1).
https://doi.org/10.35741/issn.0258-2724.58.1.33
[3] Strelcenia, E., & Prakoonwit, S. (2023). Improving classification performance in credit card fraud detection by using new
data augmentation. AI, 4(1), 172–198. https://doi.org/10.3390/ai4010008
[4] Hashemi, S. K., Mirtaheri, S. L., & Greco, S. (2023). Fraud detection in banking data by Machine Learning Techniques.
IEEE Access, 11, 3034–3043. https://doi.org/10.1109/access.2022.3232287
[5] Du, H., Lv, L., Guo, A., & Wang, H. (2023). Autoencoder and LightGBM for credit card fraud detection problems.
Symmetry, 15(4), 870. https://doi.org/10.3390/sym15040870
[6] Alarfaj, F. K., Malik, I., Khan, H. U., Almusallam, N., Ramzan, M., & Ahmed, M. (2022). Credit card fraud detection using
state-of-the-art machine learning and Deep Learning Algorithms. IEEE Access, 10, 39700–39715.
https://doi.org/10.1109/access.2022.3166891
[7] Bakhtiari, S., Nasiri, Z., & Vahidi, J. (2023). Credit card fraud detection using ensemble data mining methods. Multimedia
Tools and Applications, 82(19), 29057–29075. https://doi.org/10.1007/s11042-023-14698-2

18
FOR ANY INQUIRIES, CONTACT US

Md Aminul Islam
School of Engineering,
Computing and Mathematics,
Oxford Brookes University, UK
Email: [email protected]

Md Habibur Rahman
Department of Mathematics,
University of Chittagong,
Chittagong, Bangladesh
Email: [email protected]

19