A Major Project Report

On
CREDIT CARD APPROVAL PREDICTION
In partial fulfilment of the requirements for the award of the degree of

Bachelor of Technology
In
Artificial Intelligence and Machine Learning
Submitted By
Himanshu Kumar (Roll No. 2101921640017)
Priyanshu Kumar (Roll No. 2101921640036)
Anubhav Singh (Roll No. 2101921640003)
Shreyansh Jaiswal (Roll No. 2101921640053)
Under the Supervision of
Ms. Shabanam Kumari

G.L. Bajaj Institute of Technology & Management,

Greater Noida
Affiliated to

Dr. A. P. J. Abdul Kalam Technical University, Lucknow

(2024-25)
 Declaration

We hereby declare that the project work presented in this report, entitled
"CREDIT CARD APPROVAL PREDICTION", submitted in partial
fulfillment of the requirements for the award of the degree of Bachelor of
Technology in Artificial Intelligence & Machine Learning, affiliated to Dr.
A.P.J. Abdul Kalam Technical University, Lucknow, is based on our original
work carried out in the Department of Computer Science & Engineering
(AIML) at G.L. Bajaj Institute of Technology & Management, Greater Noida.

We further certify that the work contained in this report is original and has not
been submitted by us for the award of any other degree or diploma.

Date:
Place: Greater Noida

Signature: Signature:
Name: Himanshu Kumar Name: Priyanshu Kumar
Roll No: 2101921640017 Roll No: 2101921640036

Signature: Signature:
Name: Anubhav Singh Name: Shreyansh Jaiswal
Roll No: 2101921640004 Roll No: 2101921640053

ii
 Certificate

This is to certify that the project report, entitled "CREDIT CARD APPROVAL
PREDICTION", completed by Himanshu Kumar (2101921640017), Priyanshu
Kumar (2101921640036), Anubhav Singh (2101921640004), and Shreyansh
Jaiswal (2101921640053) of the Artificial Intelligence & Machine Learning
branch, is an original work carried out by them in the Department of Computer
Science & Engineering (AIML) at G.L. Bajaj Institute of Technology &
Management, Greater Noida, under my guidance.

To the best of my knowledge and belief, the content embodied in this project work
has not been submitted earlier for the award of any degree or diploma.

Date:

Ms. Shabanam Kumari Dr. Naresh Kumar

(Assistant Professor) (Head Of Department)

Department of CSE (AIML) Department of CSE (AIML)

iii
 Acknowledgement

The merciful guidance bestowed to us by the Almighty has enabled us to make

significant progress in this project. We humbly pray for His continued blessings to
guide us to its successful completion.

We extend our sincere thanks to our project guide, Ms. Shabanam Kumari, for
her invaluable guidance and support throughout this project. Her extensive
knowledge and insight were instrumental in overcoming the challenges we faced
during the course of this work.

We are profoundly grateful to our Head of Department, Dr. Naresh Kumar, for
his unwavering support and encouragement throughout this project.

We also take this opportunity to thank everyone who has directly or indirectly
contributed to the completion of this project. Their help and support have been a
source of great strength.

We want to thanks our friends who have always encouraged us during this project.

Lastly, we thank all the faculty members of the CSE (AIML) department for their
valuable suggestions and guidance during the course of this project.

iv
 Abstract

The aim of this project is to develop a machine learning-based system to predict

the likelihood of credit card application approval or rejection before submission.
By utilizing various applicant features such as income, employment history,
marital status, and credit-related data, this system seeks to provide an automated
prediction that could help individuals assess their chances of approval, thereby
saving time and effort.

Currently, the project involves extensive data preprocessing, including handling

missing values, encoding categorical variables, and feature engineering to derive
essential information such as account age and risk indicators. Various machine
learning algorithms, such as Logistic Regression and Random Forest, are being
employed to model the data and provide accurate predictions. While the model is
still being refined, the goal is to achieve a robust system capable of assisting
applicants in determining the likelihood of their credit card applications being
approved.

Upon completion, the system will provide users with the ability to predict whether
their application will be approved or rejected, based on their data inputs, assisting
both applicants and financial institutions in decision-making processes. Further
work includes model optimization, addressing potential biases, and integrating
real-time data to improve prediction accuracy.

v
 TABLE OF CONTENT

Declaration ............................................................................................................ (ii)

Certificate ............................................................................................................ (iii)
Acknowledgement ............................................................................................... (iv)
Abstract ................................................................................................................. (v)
Table of Content ................................................................................................... (vi)
List of Figures ..................................................................................................... (vii)

Chapter 1. Introduction ............................................................................... 1

1.1. Overview ..................................................................................... 1
1.2. Problem Definition ..................................................................... 1
1.3. Objectives ................................................................................... 2

Chapter 2. Literature Review ..................................................................... 4

2.1. Evolution of Credit Card Approval Systems .............................. 4
2.2. Machine Learning Approaches ................................................... 4
2.3. Comparative Analysis of Classification Algorithms ................... 5
2.4. Performance Evaluation and Practical Implementation ............. 5
2.5. Research Gap and Current Work ................................................ 5

Chapter 3. Methodology/Approach ............................................................ 6

3.1. Data Collection and Overview .................................................... 6
3.2. Data Preprocessing and Cleaning ............................................... 6
3.3. Model Selection and Training ..................................................... 7
3.4. Model Evaluation and Optimization ........................................... 10

Chapter 4. Result & Experiment ................................................................ 12

4.1. Data Collection & Preprocessing ................................................ 12
4.2. Feature Engineering & Transformation ...................................... 13
4.3. Model Preparation ...................................................................... 14
4.4. Model Optimisation .................................................................... 15

Chapter 5. Conclusion & Future Scope ..................................................... 16

References

vi
 LIST OF FIGURES

Figure No. Description Page No.

Fig. 1.1. Credit Card Application Process 1
Fig. 3.1. Data Preprocessing Steps 7
Fig. 3.2. Logistic Regression Visualization 8
Fig. 3.3. Random Forest Visualization 8
Fig. 3.4. Support Vector Machine Visualization 9
Fig. 3.5. Adboost Classifier Visualization 9
Fig. 3.6. Gradient Boosting Classifier Visualization 10
Fig. 4.1. M.L. Pipeline for Credit Card Approval Prediction 12

vii
 Chapter 1
INTRODUCTION

1.1. Overview
Credit card approval prediction is an essential area of research in the finance
sector, particularly in streamlining the credit application process, as illustrated in
Fig. 1.1. Financial institutions are increasingly relying on data-driven models to
determine creditworthiness, and this project aims to build such a model using
machine learning. The focus is on predicting whether a credit card application will
be approved or rejected based on a set of applicant features such as income,
employment status, and credit history.

Fig. 1.1. Credit Card Application Process

1.2. Problem Definition

Credit card approval is a critical decision made by financial institutions that
impacts both consumers and organizations. Traditionally, credit card applications
1
are processed manually, requiring the applicant to submit detailed financial and
personal information, which is then assessed by the bank. However, with
advancements in machine learning (ML), this process can be streamlined and
automated, reducing human error and increasing efficiency. The central problem
addressed in this project is to predict the approval or rejection of credit card
applications based on historical data.

Financial institutions, especially banks, face significant challenges in handling a

large volume of credit card applications. These applications contain sensitive
personal data and can include hundreds of features. Evaluating each application
manually is time-consuming and error-prone. Moreover, the criteria used to assess
whether a person should receive a credit card can vary between institutions. The
goal of this project is to create a machine learning model that can accurately
predict the likelihood of credit card approval based on data provided by the
applicant. This predictive model aims to support users by allowing them to
determine if their credit card application is likely to be approved or rejected, even
before submitting it.

The focus of the project is on leveraging available applicant data—such as income,

marital status, credit history, and employment—along with machine learning
techniques, to create an automated system that can predict the outcome of a credit
card application. The predictive system will provide applicants with a prediction
of whether their credit card application is likely to be approved or rejected.

1.3. Objectives
The main objectives of the project are:
◼ Develop a Predictive Model: The primary objective of this project is to
develop a machine learning model that accurately predicts the approval or
rejection of credit card applications based on historical data.
◼ Preprocess the Dataset: The raw dataset needs to be cleaned and
preprocessed. This involves dealing with missing values, encoding
categorical variables, and scaling numerical features.
2
◼ Model Selection and Evaluation: We aim to test and compare various
machine learning algorithms to identify the one that offers the best
predictive accuracy. Possible models include Logistic Regression, Random
Forest, and Support Vector Machines (SVM).
◼ Optimize the Model: Once the best model is selected, the next step is to
tune the hyperparameters for optimal performance. This can be done using
grid search or random search methods.
◼ Deploy the Model: The final model will be integrated into a web
application or API to allow users to input their data and receive predictions
on whether their credit card application will be approved.

3
 Chapter 2
LITERATURE REVIEW

The automation of credit card approval processes through machine learning has
gained significant attention in recent years. This review examines various
approaches and methodologies implemented in this domain.

2.1. Evolution of Credit Card Approval Systems

Traditional credit card approval systems relied heavily on manual processing and
fixed rules. However, with the advancement of machine learning techniques,
automated systems have become increasingly sophisticated. Peiris [1]
demonstrated that machine learning algorithms can significantly improve the
accuracy and efficiency of credit card approval processes. This shift towards
automation is particularly relevant given the increasing volume of non-cash
transactions, as highlighted by Marginingsih and Sari [2] in their analysis of
economic growth patterns.

2.2. Machine Learning Approaches

Recent research has explored various machine learning techniques for credit
approval prediction. Khan and Ghosh [3] introduced an innovative approach
combining random wheel methodology with explainable AI, making the
prediction process more transparent and interpretable. Their work achieved
notable accuracy while maintaining interpretability of results. In a different
approach, Zarnaz et al. [4] implemented Nonnegative Tensor Factorization,
demonstrating its effectiveness in handling multi-dimensional credit card approval
data.

4
2.3. Comparative Analysis of Classification Algorithms
Several studies have focused on comparing different classification algorithms for
credit card approval prediction. Dalsania et al. [5] conducted a comprehensive
comparison of various classification algorithms, providing insights into their
relative performance and applicability. Building on this, Kibria and Sevkli [6]
extended the comparison to include deep learning techniques, offering valuable
insights into the advantages and limitations of different approaches.

2.4. Performance Evaluation and Practical Implementation

The practical implementation aspects of credit approval systems have been
extensively studied. Duan [7] evaluated the performance of supervised data
mining algorithms in real-world credit card approval scenarios, highlighting the
importance of model selection based on specific business requirements. Hemkiran
et al. [8] focused on the design aspects of automatic credit card approval systems,
emphasizing the need for robust and scalable solutions.

2.5. Research Gap and Current Work

While existing research has made significant contributions, there remains a need
for a comprehensive approach that combines robust preprocessing, effective
feature engineering, and optimal model selection. The current work aims to
address this gap by developing a complete pipeline for credit card approval
prediction, incorporating insights from previous studies while introducing novel
improvements in data preprocessing and model optimization.

5
 Chapter 3
METHODOLOGY / APPROACH

3.1. Data Collection and Overview

The project utilizes publicly available datasets for credit card application approval
prediction. The data is typically made up of various demographic and financial
attributes of applicants. Features often included in the dataset are:
• Personal Information: Gender, marital status, family size, etc.
• Financial Information: Annual income, amount of outstanding debt,
number of credit inquiries, etc.
• Credit History: History of payments, whether there are any outstanding
loans, default records, etc.
• Employment Details: Job type, years of employment, and income status.

3.2. Data Preprocessing and Cleaning

Preprocessing the data is one of the most critical steps in machine learning
projects. The dataset is often incomplete and may contain errors that need to be
addressed before training the model. The preprocessing steps, as illustrated in Fig.
3.1. undertaken in this project include:
i. Handling Missing Data: Missing values in the dataset are imputed or
removed. For numerical features, imputation might involve filling missing
values with the mean, median, or mode. For categorical features, the most
frequent category or a placeholder (such as "Unknown") might be used.
ii. Feature Encoding: The dataset often includes categorical variables, such
as "Gender" or "Marital Status", which need to be converted into a
numerical format so that the machine learning models can process them.
Methods such as one-hot encoding or label encoding are used to transform
categorical features into numerical values.

6
 iii. Feature Scaling: Some machine learning models, especially those that
rely on distance-based algorithms (e.g., SVM, KNN), benefit from feature
scaling. Standardization or normalization is applied to scale features so
that they have the same range.
iv. Outlier Detection: Outliers are extreme values that may distort the
training process. These are detected and handled appropriately by either
removing or transforming them to avoid skewing the model’s predictions.
v. Feature Engineering: Creating new features based on existing data can
improve model performance. For instance, derived features like "credit
utilization ratio" (credit limit vs. outstanding balance) or "account age"
(calculated from account opening date) may provide additional insights
into an applicant's financial behavior.

Fig. 3.1. Data Preprocessing Steps

3.3. Model Selection and Training

Multiple machine learning algorithms are evaluated for predicting credit card
approval. Common algorithms used in classification tasks include:

7
• Logistic Regression: A simple, yet powerful algorithm for binary
classification. It estimates the probabilities of approval or rejection based
on the input features, as illustrated in Fig. 3.2.

Fig. 3.2. Logistic Regression Visualization

• Random Forest: A robust, ensemble learning method that uses multiple

decision trees to make a final prediction. It’s known for its ability to
handle complex data with high accuracy, as illustrated in Fig. 3.3.

Fig. 3.3. Random Forest Visualization

• Support Vector Machines (SVM): SVM creates an optimal hyperplane

that separates the different classes (approval or rejection) with the
maximum margin, as illustrated in Fig. 3.4.
8
 Fig. 3.4. Support Vector Machine Visualization

• Adaboost Classifier: Adaboost (Adaptive Boosting) is an ensemble

method that combines weak classifiers (e.g., decision trees) to create a
strong classifier. It works by iteratively fitting a classifier to the data, with
each subsequent model focusing more on the samples that were
misclassified by the previous ones, as illustrated in Fig. 3.5.

Fig. 3.5. Adaboost Classifier Visualization

9
 • Gradient Boosting Classifier: Gradient Boosting is another ensemble
technique where models are trained sequentially. Each new model corrects
the errors of the previous ones by focusing on the residuals (errors) of the
previous model's predictions. This method is powerful and often yields
high-performance results for classification tasks, as illustrated in Fig. 3.6.

Fig. 3.6. Gradient Boosting Classifier Visualization

Each of these models is trained on a training dataset, and their performance is

evaluated using metrics such as accuracy, precision, recall, F1-score, and ROC-
AUC. Cross-validation is used to ensure the models generalize well to unseen data.

3.4. Model Evaluation and Optimization

Once the models are trained, their effectiveness is evaluated using several metrics:
• Accuracy: The proportion of correctly predicted outcomes out of all
predictions made.
• Precision and Recall: These are particularly important in credit card
prediction, as they help measure how many of the positive predictions are
correct (precision) and how many of the actual positive instances are
identified (recall).
• F1-Score: A balance between precision and recall, which is particularly
useful when the data is imbalanced (e.g., more rejections than approvals).
• ROC-AUC Curve: This curve plots the true positive rate against the false
positive rate, allowing for the assessment of model performance in terms
of class separability.

10
Hyperparameter tuning is then performed using techniques like grid search or
random search to optimize the model’s performance.

11
 Chapter 4
RESULT & EXPERIMENT

The implementation of this credit card approval prediction system follows a

structured machine learning pipeline, as illustrated in Fig. 4.1. The pipeline
encompasses several key stages: data retrieval, data preparation (including
processing, feature engineering, and scaling), modeling, evaluation, and
deployment.

Fig. 4.1. Machine Learning Pipeline for Credit Card Approval Prediction

4.1. Data Collection and Preprocessing

4.1.1. Data Import & Target Variable Creation
• The datasets application_record.csv and credit_record.csv have
been successfully loaded into DataFrames.
• The data includes demographic, financial, and credit-related
information about applicants, essential for the model training.
• A new target variable, is_risky, was created to classify applicants
as high-risk or low-risk based on their payment status history.
• Applicants who defaulted on payments at any point were classified
as high-risk (1), while others were classified as low-risk (0).

12
 4.1.2. Data Merging
• The account_age_(months) feature was derived by calculating the
minimum balance month for each applicant in the
credit_record.csv dataset and merging it into the main
application_record.csv.

The dataset has been successfully acquired and preprocessed. Missing values have
been handled, categorical variables have been encoded, and feature scaling has
been applied.

4.2. Feature Engineering and Transformation

4.2.1. Feature Transformation
• Categorical features such as gender, marital_status, and
accommodation were transformed using one-hot encoding.
• Numerical features like annual_income and employment_duration
were scaled using Min-Max scaling to standardize the range of
values.

4.2.2. Handling Outliers & Skewness

• Outliers were handled using the OutlierRemover class. Features
such as family_size, child_count, annual_income, and
employment_duration were identified and outliers beyond the 1.5
IQR range were removed.
• Skewed features like annual_income and days_lived were
transformed using a cubic root transformation to make the data
more normally distributed.

Several new features, including "account age" and "credit utilization ratio," have
been successfully created to enhance the predictive power of the models.

13
4.3. Model Preparation
4.3.1. Feature Selection
• Unnecessary features, such as ID and has_mobile, were dropped
using the DropFeatures class, ensuring that only relevant features
were retained for model training.

4.3.2. Pipeline Creation

• A comprehensive preprocessing pipeline was developed using
scikit-learn’s Pipeline and TransformerMixin. This pipeline
integrates all the transformation steps, including outlier removal,
one-hot encoding, scaling, skewness handling, and SMOTE for
data balancing.
• This pipeline allows for a seamless and automated data
preprocessing workflow, ensuring consistency and reproducibility.

4.3.3. Initial Model Training

Thirteen different classifiers were implemented and evaluated:
• Basic Models: SGD, Logistic Regression, Decision Tree
• Ensemble Methods: Random Forest, Gradient Boosting,
AdaBoost, Extra Trees, Bagging
• Advanced Models: SVM, Neural Network (MLP), Linear
Discriminant Analysis
• Probabilistic Models: Gaussian Naive Bayes
• Instance-based Learning: K-Nearest Neighbors

4.3.4. Top Performing Models

• Gradient Boosting Classifier: Highest overall recall score,
Excellent ROC curve characteristics, Strong handling of non-linear
relationships.
• Support Vector Machine: Consistent performance, Effective
handling of high-dimensional data.

14
 • AdaBoost Classifier: Strong ensemble learning capabilities, good
balance of bias and variance.

Each model was evaluated using cross-validation with comprehensive metrics

including classification reports, ROC curves, and confusion matrices. The results
were saved using joblib for reproducibility and future use.

4.4. Model Optimization

4.4.1. Hyperparameter Tuning
The top three models underwent extensive hyperparameter optimization
using RandomizedSearchCV:
• Gradient Boosting: Optimized for n_estimators, learning_rate,
max_depth, and other tree-specific parameters
• SVM: Tuned C, gamma, kernel, and degree parameters
• AdaBoost: Optimized n_estimators, learning_rate, and base
estimator parameters

4.4.2. Final Model Selection

• After hyperparameter tuning and final evaluation, the Gradient
Boosting Classifier emerged as the best performing model. The
final model demonstrated robust performance on both training and
test sets, with balanced precision and recall scores suitable for
credit risk assessment in current market conditions.

The iterative process of model selection and hyperparameter tuning, particularly

focusing on Gradient Boosting, SVM, and AdaBoost classifiers, led to significant
improvements in model performance. Most notably, the final implementation
achieved a remarkable 94% accuracy in credit card approval prediction, validating
the effectiveness of our chosen methodology and implementation strategy.

15
 Chapter 5
CONCLUSION & FUTURE SCOPE

The implementation of our credit card approval prediction system has

demonstrated remarkable success through a systematic and comprehensive
approach. The project established a robust data preprocessing pipeline that
effectively handled various challenges including missing values, outliers, and
categorical variables. Through careful feature engineering and transformation,
including the creation of key features like account_age_(months) and the
implementation of SMOTE for class imbalance, we built a strong foundation for
model training. The most significant achievement was the final model's
performance, with the Gradient Boosting Classifier achieving 94% accuracy in
predicting credit card approvals, demonstrating the effectiveness of our
methodological approach and implementation strategy.

The extensive evaluation of thirteen different classifiers, followed by focused

optimization of the top three performers - Gradient Boosting, SVM, and AdaBoost
- showcases the thoroughness of our model selection process. The final selection
of the Gradient Boosting Classifier, optimized through comprehensive
hyperparameter tuning, proved to be particularly effective for credit risk
assessment in current market conditions, providing both accuracy and reliability
in predictions.

The project presents several promising avenues for future development and
enhancement. The primary focus would be on developing an interactive web
application that enables users to input their personal and financial information and
receive instant predictions about their credit card approval likelihood. This user-
friendly interface would incorporate real-time data validation and provide clear,

16
interpretable results along with basic recommendations based on the prediction
outcome.

Further enhancements to the system could include the integration of additional

relevant data sources and the implementation of more sophisticated feature
engineering techniques. The model could be regularly retrained with new data to
maintain its accuracy and adapt to changing financial patterns. Additionally, the
system could be expanded to include API endpoints for third-party integration and
mobile application development, making it more accessible to a broader user base.
These developments, combined with robust security measures and privacy
protocols, would transform the current model into a comprehensive credit card
approval prediction platform, beneficial for both financial institutions and
applicants.

17
References
[1] M. P. C. Peiris, “Credit Card Approval Prediction by Using Machine
Learning Techniques,” University of Colombo, 2022. Accessed: Mar. 07,
2023. [Online]. Available: https://dl.ucsc.cmb.ac.lk/jspui/handle/12345
6789/4593
[2] R. Marginingsih and I. Sari, “Nilai Transaksi Non Tunai Terhadap
Pertumbuhan Ekonomi Di Indonesia Periode 2014-2018,” Inovator, vol. 8,
no. 2, p. 24, Sep. 2019, doi: 10.32832/INOVATOR.V8I1.2504.
[3] A. Khan and S. K. Ghosh, “Machine assistance for credit approval?
Random wheel can recommend and explain,” Expert Syst Appl, vol. 215, p.
119231, Apr. 2023, doi: 10.1016/J.ESWA.2022.119231.
[4] Z. Zarnaz, D. Biswas, and K. M. A. Hasan, “Credit Card Approval
Prediction by Nonnegative Tensor Factorization,” International Conference
on Robotics, Electrical and Signal Processing Techniques, pp. 319–323,
2021, doi: 10.1109/ICREST51555.2021.9331172.
[5] N. Dalsania, D. Punatar, and D. Kothari, “Credit Card Approval Prediction
using Classification Algorithms,” vol. 10, 2022, doi:
10.22214/ijraset.2022.47369.
[6] Md. G. Kibria and M. Sevkli, “Application of Deep Learning for Credit
Card Approval: A Comparison with Two Machine Learning Techniques,”
Int J Mach Learn Comput, vol. 11, no. 4, pp. 286–290, Aug. 2021, doi:
10.18178/IJMLC.2021.11.4.1049.
[7] L. Duan, “Performance evaluation and practical use of supervised data
mining algorithms for credit card approval,” Proceedings - 2020
International Conference on Computing and Data Science, CDS 2020, pp.
251–254, Aug. 2020, doi: 10.1109/CDS49703.2020.00057.
[8] S. Hemkiran, G. Sudha Sadasivam, A. Prasanna Rahavendra, and A. K.
Anjhanna, “Design of Automatic Credit Card Approval System Using
Machine Learning,” Lecture Notes in Electrical Engineering, vol. 806, pp.
1–9, 2022, doi: 10.1007/978-981-16- 6448-9_1.

18