International Journal of Information Technology & Decision Making

Vol. 11, No. 4 (2012) 857
874

°c World Scienti¯c Publishing Company
DOI: 10.1142/S021962201250023X

A NOVEL FIVE-CATEGORY LOAN-RISK EVALUATION

MODEL USING MULTICLASS LS-SVM BY PSO

JIE CAO*,§, HONGKE LU†,¶, WEIWEI WANG* and JIAN WANG‡

Int. J. Info. Tech. Dec. Mak. 2012.11:857-874. Downloaded from www.worldscientific.com
by UNIVERSITY OF NEW ENGLAND LIBRARIES on 01/23/15. For personal use only.

*School of Economics and Management

Nanjing University of Information Science & Technology
Nanjing 210044, China
†
School of Economics & Management
Southeast University, Nanjing 210096, China
‡Jiangsu Jinnong Information Co., Ltd.
Nanjing 210019, China
§
[email protected]
¶
[email protected]

Five-category loan classi¯cation (FCLC) is an international ¯nancial regulation approach.

Recently, the application and implementation of FCLC in the Chinese micro¯nance bank has
mostly relied on subjective judgment, and it is di±cult to control and lower loan risk. In view of
this, this paper is dedicated to researching and solving this problem by constructing the FCLC
model based on improved particle-swarm optimization (PSO) and the multiclass, least-square,
support-vector machine (LS-SVM). First, LS-SVM is the extension of SVM, which is proposed
to achieve multiclass classi¯cation. Then, improved PSO is employed to determine the para-
meters of multiclass LS-SVM for improving classi¯cation accuracy. Finally, some experiments
are carried out based on rural credit cooperative data to demonstrate the performance of our
proposed model. The results show that the proposed model makes a distinct improvement in the
accuracy rate compared with one-vs.-one (1-v-1) LS-SVM, one-vs.-rest (1-v-r) LS-SVM, 1-v-1
SVM, and 1-v-r SVM. In addition, it is an e®ective tool in solving the problem of loan-risk
rating.

Keywords: Particle-swarm optimization; least-squares support-vector machine; credit risk; ¯ve-

category classi¯cation.

1. Introduction
There is no doubt that credit-risk assessment has become an increasingly important
area for ¯nancial institutions. All banking institutions and their regulators attempt
to search for a precise internal credit system to measure the credit quality of their
borrowers. E®ective credit-risk assessment has become a crucial factor for gaining
competitive advantages in credit markets, which can help ¯nancial institutions to
grant credit to credit-worthy customers and reject non-credit-worthy customers to
reduce loss.

857
 858 J. Cao et al.

To further strengthen credit-risk management and re°ect the quality of loans

comprehensively and dynamically, state-owned commercial banks, joint-stock com-
mercial banks, and urban commercial banks have followed international ¯nancial
standards to implement ¯ve-category loan-quality classi¯cation management to
control loan risk. The main idea behind ¯ve-category loan classi¯cation is to estimate
the probability of applicant default in terms of the characteristics recorded on the
application form or by a credit bureau, using a quantitative model that is built based
on information from past applicants. All of the applicants are classi¯ed into ¯ve
classes: Passed, Special Mention, Substandard, Doubtful, and Loss. However, re-
cently the application and implementation of FCLC in the Chinese micro¯nance
Int. J. Info. Tech. Dec. Mak. 2012.11:857-874. Downloaded from www.worldscientific.com
by UNIVERSITY OF NEW ENGLAND LIBRARIES on 01/23/15. For personal use only.

banking sector has been simple, relying mostly on subjective judgment. In addition,
it is di±cult to grasp the changes of loans in the future and to control loan risk, so the
application has been limited.1
Numerous methods have been put forward to constitute a satisfactory multiclass
model to evaluate loan risk. In particular, statistical methods, arti¯cial intelligence,
machine-learning techniques (e.g., arti¯cial neural networks (ANNs) and support-
vector machines (SVMs)), as well as other hybrid approaches have been used to solve
the FCLC problem. Xue and Ke2 proposed a two-stage credit-evaluation model by
combining rough sets and Elman neural networks to study ¯ve-category loan-risk
classi¯cation in China's commercial banks. An integrated model of rough set and BP
neural network for FCLC has been proposed by Ke and Feng.1 In light of the present
status of China's commercial banks, Peng et al.3 have proposed a multiclass logistic
regression model for FCLC. A SVM ensemble method based on fuzzy integral4 is
presented to deal with the problem of ¯ve-category loan-risk classi¯cation. This
method aggregates the outputs of separate-component SVM with the importance of
each component, which is using fuzzy logic.
SVM, one of the new techniques for pattern classi¯cation, has been successfully
and widely used in pattern recognition, regression estimation, probabilistic density
estimation, time-series prediction, software defect predictors and credit-risk
evaluation.5
10 For binary-class classi¯cations, SVM constructs an optimal separating
hyperplane between the positive and negative classes with the maximal margin. It can
be formulated as a quadratic programming problem involving inequality constraints.
In most of these applications, SVM's generalization performance either matches or is
signi¯cantly better than that of other competing methods.11 LS-SVM was recently
proposed,12,13 which involves equality constraints only. Hence, the solution is obtained
by solving a system of linear equations. Extensive empirical study12 has shown that
LS-SVM is comparable to SVM in terms of generalization performance.
Although classical SVM and LS-SVM models have good performance in classi¯-
cation, they are sensitive to sample and parameter settings. However, classical SVM
is a binary classi¯cation method. Many research issues we need to study are multi-
class classi¯cation problems, such as ¯ve-category loan classi¯cation, image recog-
nition, and fault diagnosis. Thus, how to apply SVM to multiclass classi¯cation
e®ectively is one of the most important issues.
 Novel Five-Category Loan-Risk Evaluation Model 859

In recent years, multiclass SVM algorithms have been applied to deal with mul-
ticlass classi¯cation. One approach is to consider the issue as a collection of binary
classi¯cation problems. Methods such as 1-v-r and 1-v-1 have been used widely in the
SVM literature to solve multiclass classi¯cation problems. Another way to solve
multiclass problems is to construct a decision function by considering all classes in
one case.14,15
Several multiclass SVM models16 are constructed to study the credit-scoring
problem, and comparison has been made with these algorithms. For consumer be-
havior of credit card customers, the credit card behavior-evaluation model based on
genetic algorithms and multiclass SVM is constructed.17 Kim and Ahn18 proposed a
Int. J. Info. Tech. Dec. Mak. 2012.11:857-874. Downloaded from www.worldscientific.com
by UNIVERSITY OF NEW ENGLAND LIBRARIES on 01/23/15. For personal use only.

new credit-rating model using multiclass SVMs with an ordinal pairwise partitioning
strategy.
However, the literature about multiclass SVM used in commercial bank credit-
risk assessment still contains relatively few studies in the ¯eld of credit-risk
evaluation. Additionally, it has not been very long since FCLC approaches were
used in China's commercial banks, and loan credit data is insu±cient. Thus, there
are few evaluation models about ¯ve-category credit-risk classi¯cation in com-
mercial banks. It is very signi¯cant for improving the credit-risk management to
construct e®ective and practical loan-risk rating tools to predict the real condition
of loans.
In this paper, the traditional binary classi¯cation model is extended to multiclass.
The model is suitable to deal with the FCLC problem in China's commercial banks.
LS-SVM is employed to establish sub-classi¯ers by using 1-v-1 technology. The
classi¯cation results and the importance to ¯nal a decision of each sub-classi¯er are
both taken into account by this method. To achieve high classi¯cation performance,
the parameters of 1-v-1 LS-SVM are optimized by the improved PSO algorithm.
Some empirical studies have been done, and the results have demonstrated that the
model has a better performance.
The rest of this paper is organized as follows. In Sec. 2, some methodologies are
¯rst introduced, including LS-SVM, multiclass SVM and improved PSO, and then
the framework of the PSO-improved 1-v-1 LS-SVM model and some strategies are
illustrated. To verify the e®ectiveness of our proposed methods, empirical analysis of
the models is given in Sec. 3. In Sec. 4, some conclusions and a discussion are
presented.

2. Methodology
2.1. Least-squares support-vector machines
LS-SVM is a variant of SVM, which leads to solving the linear Karush
Kuhn

Tucker (KKT) systems.20 A nice aspect of nonlinear SVM is that one solves non-
linear regression problems by means of convex quadratic programs. LS-SVM di®ers
from SVM, where the quadratic programming is transformed to linear programming,
 860 J. Cao et al.

which o®ers a simpler formulation and fewer computational requirements. In addi-

tion, the threshold b is calculated as a whole in the procedure.
Given a training set of instance-label pairs fxi ; yi g, i ¼ 1; . . . ; n, where xi 2 Rl ,
yi 2 fþ1;
1g. The LS-SVM model in feature space is as follows:

yðxÞ ¼ w T
ðxÞ þ b; ð1Þ

where the nonlinear mapping
ðÞ will map the input data to the high-dimensional
feature space.
The optimization of LS-SVM applies the approximate function:
Int. J. Info. Tech. Dec. Mak. 2012.11:857-874. Downloaded from www.worldscientific.com

X
by UNIVERSITY OF NEW ENGLAND LIBRARIES on 01/23/15. For personal use only.

n
min J ðw; Þ ¼ 1=2w T w þ ðC =2Þ  2i
w;b;
i¼1 ; ð2Þ
subject to yi ¼
ðxi Þ  w þ b þ i ; i ¼ 1; . . . ; n
where i  0 is the non-negative slack variable and C > 0 is a penalty parameter on
the training error. This optimization model can be solved using the Lagrangian
method, which is almost equivalent to the method for solving the optimization
problem in the separable case:
X
n X
n
L ¼ jjwjj2 þ ðC =2Þ  2i
i ð
ðxi Þ  w þ b þ i
yi Þ; ð3Þ
i¼1 i¼1

where i are Lagrange multipliers.

According to the condition of KTT:
@L=@w ¼ 0; @L=@b ¼ 0; @L=@ ¼ 0; @L=@ ¼ 0: ð4Þ
Then, the following result can be obtained:
X
n X
n
w¼ i
ðxi Þ; i ¼ 0; i ¼ C  i ;
ðxi Þw þ b þ i
yi ¼ 0: ð5Þ
i¼1 i¼1

According to Eq. (5), the following linear equation is obtained by eliminating the
parameters w and :


0 eT


b 0 ð6Þ
e NN T þ C
1 I ðnþ1Þðnþ1Þ  ¼ y ;

where e is the vector ðn  1Þ whose element is 1. I is an N -dimensional identity matrix,

 ¼ ð1 ; 2 ; . . . ; n ÞT , y ¼ ðy1 ; y2 ; . . . ; yn ÞT , N ¼ ð
ðx1 ÞT ;
ðx2 ÞT ; . . . ;
ðxn ÞT ÞT .
According to Mercer, the radial basis function (RBF) is chosen as the kernel
function, which is de¯ned as K ðxi ; xj Þ ¼
ðxi Þ 
ðxj Þ.
The linear decision function is obtained as follows:
X
n
f ðxÞ ¼ i K ðxi ; xj Þ þ b: ð7Þ
i¼1
 Novel Five-Category Loan-Risk Evaluation Model 861

The kernel functions used most frequently are the polynomial, sigmoid, and the
RBF. The RBF is applied most frequently because it can classify multi-dimensional
data, unlike a linear kernel function. Additionally, the RBF has fewer parameters
than a polynomial kernel. Extensive empirical study has shown that the RBF kernel
function has better generalization performance.15 Thus, the RBF kernel has been
chosen, and its expression is described as follows:
K ðxi ; xj Þ ¼ expð
jjxi
xj jj=22 Þ; ð8Þ
where  is the parameter that shows the width of the RBF kernel.
Int. J. Info. Tech. Dec. Mak. 2012.11:857-874. Downloaded from www.worldscientific.com
by UNIVERSITY OF NEW ENGLAND LIBRARIES on 01/23/15. For personal use only.

2.2. Multi-class SVM

In this section, we discuss the multiclass (N -class) problem solved by considering the
problem as a collection of binary classi¯cation problems. The two multiclass SVMs,
1-v-r SVM and 1-v-1 SVM, are brie°y explained.
(1) 1-v-r SVM
The earliest used implementation of the SVM multiclass classi¯cation is probably the
1-v-r method (1-v-r SVM). It constructs k SVM models, where k is the number of
classes. The ith SVM is trained with all of the examples in the ith class with positive
labels, and all other examples with negative labels. Thus, given l training data
ðx1 ; y1 Þ; . . . ; ðxl ; yl Þ, where xi 2 Rn ; i ¼ 1; . . . ; l and yi 2 f1; . . . ; kg is the class thereof,
the ith SVM solves the following problem:

1 i T i Xt
min ðw Þ w þ C  ij ðw i ÞT
w ;b ;i
i i 2 j¼1
ðw i ÞT
ðxj Þ þ bi  1
 ij ; if yi ¼ i ; ð9Þ
ðw i ÞT
ðxj Þ þ bi 
1 þ  ij ; if yi 6¼ i
 ij  0; j ¼ 1; . . . ; l

where the training data xi are mapped to a higher-dimensional space by the function

, and C is the penalty parameter.
(2) 1-v-1 SVM
Another major method is called the one-against-one method.21 This method con-
structs kðk
1Þ=2 classi¯ers, where each one is trained on data from two classes. For
training data from the ith and the jth classes, the following binary classi¯cation
problem is solved:
1 ij T ij X ij
min ðw Þ w þ C t
w ij ;bij ; 2
t
ðw ij ÞT
ðxt Þ þ bij  1
 ijt ; if yt ¼ i; ð10Þ
ðw ij ÞT
ðx tÞ þ bij 
1 þ  ijt ; if yt ¼ j;
 ijt  0:
 862 J. Cao et al.

There are di®erent methods for doing the future testing after all kðk
1Þ=2
classi¯ers are constructed. After some tests, we decided to use the following voting
strategy.22
If signððw ij ÞT
ðxt Þ þ bij Þ determines x in the ith class, the vote for the ith class is
added by one; otherwise, the jth is increased by one. Then we predict x is in the class
with the largest vote. The voting approach described above is also called the \Max
Wins" strategy. In case two classes have identical votes, though it may not be a good
strategy, the one with the smaller index is simply selected.
Practically, we solve the dual of Eq. (10), which has the same number of variables
as the number of data in two classes. Hence, if on average each class has l=k data
Int. J. Info. Tech. Dec. Mak. 2012.11:857-874. Downloaded from www.worldscientific.com
by UNIVERSITY OF NEW ENGLAND LIBRARIES on 01/23/15. For personal use only.

points, we have to solve kðk
1Þ=2 quadratic programming problems, where each of
them has about 2l=k variables.

2.3. Improved particle-swarm optimization

The particle-swarm optimization (PSO) is a computation intelligence technique that
is motivated by the organisms' behavior, such as schooling of ¯sh and °ocking
of birds. The global optimizing model proposed by Kennedy and Eberhart23 is
described as follows:
vkþ1 ¼ vk þ c1 r1 ðpbestk
xk Þ þ c2 r2 ðgbestk
xk Þ;
ð11Þ
xkþ1 ¼ xk þ vkþ1 ;
where vk is the velocity of particle i, representing the distance to be travelled by this
particle from its current position; k is the number of iterations; xk represents the
particle position; w is the inertial weight; c1 and c2 are the positive constant para-
meters; r1 and r2 are the random functions in the range [0,1]; pbestk (local best
solution) is the best position of the kth particle and gbestk (global best solution) is the
best position among all particles in the swarm.
In general, the parameters w; c1 ; c2 ; r1 ; r2 are the important factors that in°uence
the convergence of the PSO. However, parameters r1 and r2 cannot guarantee er-
godicity of the optimization entirely in phase space because they are absolutely
random in the traditional PSO. By studying the basic model above, many
researchers have shown that if the acceleration constants c1 and c2 , as well as
parameters such as maximum speed, are too large, the particle swarm may miss the
optimal solution, resulting in the fact that the algorithm does not converge. In the
case of convergence, all particles move to the direction of the algorithm's best so-
lution and tend to be in the same state, causing the latter part of the convergence
rate to slow signi¯cantly. Moreover, while the algorithm converges to one certain
accuracy, it cannot continue to be optimized any more. Thus, the last accuracy of the
algorithm is not very high.
To solve the problem, the convergence factor  and inertia weight ! can be
applied to improve the basic PSO model. Xia et al.24 proposed a new, improved form
by synthesizing the existing model of PSO. To search for the optimal solution, each
 Novel Five-Category Loan-Risk Evaluation Model 863

particle changes its velocity according to the cognition and social parts as follows:
vkþ1 ¼   ½!ðtÞvk þ c1 r1 ðpbestk
xk Þ þ c2 r2 ðgbestk
xk Þ; ð12Þ
xkþ1 ¼ xk þ vkþ1 !ðtÞ; ð13Þ
2 t
¼ pffiffiffiffiffiffiffiffiffiffiffiffiffiffiffi ; !ðtÞ ¼ 0:9
 0:5;
j2
c
c2
4cj Tmax

where c1 indicates the cognition learning factor, c2 indicates the social learning
factor, r1 and r2 are random numbers uniformly distributed in U ð0; 1Þ, and Tmax is
the maximum iteration. With the increase of t, parameter ! will decrease from 0.9 to
Int. J. Info. Tech. Dec. Mak. 2012.11:857-874. Downloaded from www.worldscientific.com
by UNIVERSITY OF NEW ENGLAND LIBRARIES on 01/23/15. For personal use only.

0.4 linearly. Meanwhile, in order to balance the e®ect of randomness, according to

relevant research,25 we set parameters c1 ¼ c2 ¼ 2.

2.4. Establish multiclass classi¯cation LS-SVM model

There are many classi¯cation algorithms based on the binary SVM for multiclass
issues. Research studies have found that the 1-v-1 SVM classi¯cation algorithm has
good performance.26 The approach used in the research as LS-SVM can overcome the
low speed when training samples are large. Thus, LS-SVM is utilized to establish a
multiclass classi¯cation model.
Suppose that there are training samples S ¼ fðx1 ; y1 Þ; ðx2 ; y2 Þ; . . . ; ðxl ; yl Þg, where
l is the number of training samples, xi 2 Rn is with n dimension, and yi 2
f1; 2; . . . ; kg is the classi¯cation rating of the ith category sample.
Given a training set of instance-label pairs fxi ; yi g; i ¼ 1; . . . ; n, where
xi 2 Rl ; yi 2 fþ1;
1g. The LS-SVM model in feature space is as follows:
yðxÞ ¼ w T
ðxÞ þ b: ð14Þ
Two di®erent classes of samples are adopted by 1-v-1 SVM to establish the one
classi¯er fij ði 6¼ jÞ. This method is used to construct C k2 ¼ kðk
1Þ=2 classi¯ers,
where each one is trained on data from two classes. For training data from the ith
and jth classes, we solve the following binary classi¯cation problem:
" #
1 i;j 2 X i;j
min jjw jj þ C =2 ð t Þ ;
2
w i;j ;bi:j ;i;j 2
t¼1

s:t: ½ðw i;j ÞT
ðxt Þ þ bi;j 
1 þ  i;j

t  0; if yt ¼ i; ð15Þ
½ðw i;j ÞT
ðx tÞ þ bi;j 
1 þ  i;j
t  0; if yt ¼ j;
 i;j
t  0;

where i  0 is the relaxation parameter and C > 0 is the penalty function.

For kðk
1Þ=2 sub-classi¯ers, there are kðk
1Þ=2 decision functions. The deci-
sion function between the ith and jth categories is as follows:
f ðxÞ ¼ sgnðw i;j  ’ðxÞ þ bij Þ: ð16Þ
 864 J. Cao et al.

kðk
1Þ=2 sub-classi¯ers are used to judge unknown sample x. The following
strategy is employed.
If classi¯ers judge that x belongs to the ith class, then the vote for the ith class is
increased by one. Otherwise, the jth is increased by one. Then we predict that x is in
the class with the largest vote.

2.5. Parameter selection

The fact that the penalty function C and the kernel function  will a®ect the per-
formance of LS-SVM merely means di®erent impacts on di®erent data sets. The
Int. J. Info. Tech. Dec. Mak. 2012.11:857-874. Downloaded from www.worldscientific.com

optimal parameter will greatly improve the accuracy of LS-SVM,27,28 and therefore
by UNIVERSITY OF NEW ENGLAND LIBRARIES on 01/23/15. For personal use only.

the parameter selection is a key issue for the successful application of the algorithm.
In this paper, the improved PSO algorithm is used for parameter selection. To
implement our proposed approach, we use the RBF kernel function (de¯ned
by Eq. (17)) for the LS-SVM classi¯er because it can be used to analyze higher-
dimensional data. The formulation is given below.
K ðxi ; xj Þ ¼ expð
jjxi
xj jj=22 Þ: ð17Þ

Additionally, the ¯tness function plays an important role in evolutionary algo-

rithms. In this paper, mean absolute percentage error (MAPE) is employed as the
¯tness function, which can be de¯ned as
X
n
f ¼ 1=n jðyi
y^i Þ=2yi j; ð18Þ
1

where n is the number of the test sample, yi is the actual value, y^i is the prediction
value, and f is the value of the ¯tness. Using the improved PSO algorithm, the
parameter-selection process of 1-v-1 LS-SVM is shown in Fig. 1.
The detailed steps are described as follows:

Step 1: Initialize the parameters of a particle-swarm-optimization algorithm. In

a D-dimension space, randomly generate D-dimensional position x i1 t
;
x i2 ; . . . ; x in for the particle i at iteration t, constitute the initial population
t t

XðtÞ, randomly generate the velocity of each particle v i1 t

; v i2
t
; . . . ; v in
t
, and
build up the velocity matrix V ðtÞ.
Step 2: Measure the ¯tness of each particle in the population; the formulation can
P
then be de¯ned as: f ¼ 1=n n1 jðyi
y^i Þ=2yi j.
Step 3: For each particle, compare the current ¯tness f ðxi Þ with the best position
¯tness f ðpbestk Þ of the group, if f ðxi Þ < f ðgbestk Þ, then global optimal so-
lution gbestk ¼ xi .
Step 4: Compute the velocity of each particle with Eqs. (12) and (13), and generate
a new population Xðt þ 1Þ. The speed-adjustment rules are as follows:

Vmax ; vi > Vmax
vi ¼ : ð19Þ

Vmax ; vi <
Vmax
 Novel Five-Category Loan-Risk Evaluation Model 865

Particle-Swarm Optimization

Given a population of particles with

random positions and velocities

Evaluate fitness of particles

Update particles best and global best

Int. J. Info. Tech. Dec. Mak. 2012.11:857-874. Downloaded from www.worldscientific.com
by UNIVERSITY OF NEW ENGLAND LIBRARIES on 01/23/15. For personal use only.

Update particle velocity and position

No Is maximum iteration
reached?

Yes

Optimal 1-v-1 LS-SVM parameters obtained

Optimal PSO-improved 1-v-1 LS-SVM prediction model

Fig. 1. The process of optimizing the 1-v-1 LS-SVM parameters with PSO.

Step 5: The iteration is terminated if the number of iterations reaches the pre-
determined maximum and returns the current best individual as a result;
return to Step 2 otherwise.
Step 6: Through the above process, the optimal parameters  and C of 1-v-1 LS-
SVM will be obtained.

2.6. FCLC model

In this paper, the FCLC model is employed by incorporating the advantages of PSO
and 1-v-1 LS-SVM. The proposed hybrid model is composed of six parts, as shown in
Fig. 2.
The algorithm of our loan-default-discrimination model is as follows:

Step 1: Sample data selection and index system construction. The loan data set
attributes have to be acquired ¯rst, and the index system is then con-
structed based on current credit and loan rating research.
Step 2: Data preprocessing and normalization. Continuous and symbolic variables
have to be preprocessed. Symbolic variables will be preprocessed by discrete
operation. Normalization processing of data before the training is crucial
 866 J. Cao et al.

Sample set selection

Attribute value preprocessing

Data preprocessing
Normalization

Training set Test set

Int. J. Info. Tech. Dec. Mak. 2012.11:857-874. Downloaded from www.worldscientific.com
by UNIVERSITY OF NEW ENGLAND LIBRARIES on 01/23/15. For personal use only.

Parameter selection
PSO 1-v-1 multiclass classification
LS-SVM model

Prediction Output

Fig. 2. The proposed model based on PSO and 1-v-1 multiclass classi¯cation LS-SVM.

with the goal of speeding up the convergence of the model and reducing the
impact of imbalance of the data capacity to the network classi¯er.
Step 3: Construction of a multiclass PSO improved 1-v-1 LS-SVM model. The
processed data are imputed into the multiclass classi¯cation model. The
optimal parameters, including  and C , will be searched and determined by
the improved PSO.
Step 4: Classi¯cation output. Our proposed FCLC model that has been trained is
employed to determine the category for the test sample.
Step 5: Algorithm comparison. For the same samples, 1-v-1 LS-SVM, 1-v-r LS-
SVM, 1-v-1 SVM, and 1-v-r SVM models are tested, respectively. Their
results are compared with the output of the multiclass LS-SVM model.

3. Experimental Analyzes
3.1. Index system and sample data
In order to verify the availability of the proposed approaches, one loan-default data
set consisting of 412 samples is acquired from one of China's rural credit coopera-
tives. In the data set there are 16 samples with ¯nancial data seriously absent. Thus,
396 samples are used for analysis. Loan levels include Passed, Special Mention,
Substandard, Doubtful, and Loss. Therein, numbers 1 through 5 are selected to
represent them, respectively. The composition of the sample is shown in Table 1.
For each experiment, 264 customers were selected randomly as a training sample
set, while the remaining 132 samples were used as a test sample set. In the training
sample set, there were 86 \Passed" customers, 78 \Special Mention" customers, 26
\Substandard" customers, 56 \Doubtful" customers, and 18 \Loss" customers. The
test sample set, which was composed of 40 \Passed" customers, 37 \Special Mention"
 Novel Five-Category Loan-Risk Evaluation Model 867

Table 1. Samples composition.

Passed Special mention Substandard Doubtful Loss Total amount

Sample amount 126 115 39 88 28 396

Table 2. Complete attribute table of personal loans in the rural credit cooperative.

Element Index
Int. J. Info. Tech. Dec. Mak. 2012.11:857-874. Downloaded from www.worldscientific.com
by UNIVERSITY OF NEW ENGLAND LIBRARIES on 01/23/15. For personal use only.

Loan-identi¯cation attributes Loan-contract ID, loan-account ID, account ID, currency, customer
ID
Customer-status attributes Birth date, gender, marital status, nationality, language, country of
residence, legal status, annual income range, household income
range, highest education, highest degree, position, occupation,
title, years of work, political status, religion, dependent popu-
lation, the amount for the month, year of marriage, number of
children, personal interest, investment interest, risk type,
whether to accept marketing
Customer-loan-account attributes Loan-attribute ID, release amount, loan purpose, release date, due
date, interest settlement date, interest settlement cycle

customers, 13 \Substandard" customers, 32 \Doubtful" customers, and 10 \Loss"

customers, was applied to testify to the prediction accuracy of the proposed model.
The experiments were carried out three times.
In the original loan database, the attributes were composed of three parts, in-
cluding account-identi¯cation attributes, loan-identi¯cation attributes, and cus-
tomer loan-account attributes. Table 2 shows the features/variables used in data sets.
The customer loan ¯ve-category index system was established based on demog-
raphy characteristics and loan account indices.29,30 We set up an index system made
up of 16 ¯nancial indices, such as age, gender, marital status, legal status, annual
income range, and other ¯nancial information, which synthetically re°ects the ¯-
nancial condition of listed companies. Table 3 shows the personal loan-evaluation
indices used in this paper.

3.2. Data preprocess

As shown in Table 3, birth date, loan amount, and repayment deadline are contin-
uous variables, and loan purpose is a symbolic variable. These four variables should
be preprocessed. Additionally, the range of index value is very large. For instance,
loan amount can be 3000 to 500,000. Thus, normalization processing of data before
the model training is crucial with the goal of speeding up the convergence of
the model and reducing the impact of imbalance of the data capacity to the
network classi¯er. We use linear di®erential analysis, which is de¯ned as
xij
mini
x ij0 ¼ max i
mini
2 ½0; 1, where maxi and mini denote the maximum and minimal
 868 J. Cao et al.

Table 3. Personal loan evaluation indices of rural credit cooperatives.

Index Type Number De¯nition in database

Loan account Loan-contract ID char N1 Identify the loan contract

uniquely
Loan-account ID char N2 Identify the loan account
uniquely
Age char C1 Data of birth
Gender char C2 1-Male; 2-Female
Martial status char C3 10-Single; 20-Married;
21-First marriage;
Int. J. Info. Tech. Dec. Mak. 2012.11:857-874. Downloaded from www.worldscientific.com

22-Married; 23-Remar-
by UNIVERSITY OF NEW ENGLAND LIBRARIES on 01/23/15. For personal use only.

ried; 30-Widowed;
40-Divorce; 90-Marital
status unspeci¯ed
Demographic attributes Legal status char C4 0-Farmers; 1-Non-farmers;
of loan account 2-Farmers
UNPROFOR
Annual income range numeric C5 1-02000; 2-20005000;
3-500010,000;
4-10,00030,000;
5-30,00060,000;
6-60,000100,000;
7-100,000300,000;
8-300,0001000,000;
9->1000,000
Household annual numeric C6 1-02000; 2-20005000;
income range 3-500010,000;
4-10,00030,000;
5-30,00060,000;
6-60,000100,000;
7-100,000300,000;
8-300,0001000,000;
9->1000,000
Highest degree char C7 10-Graduate; 20-Bachelor;
30-College; 40-Second-
ary vocational schools;
50-Technical school;
60-Senior high school;
70-Junior high school;
80-Primary school;
90-Illiteracy or semi-
literate; 99-Unknown
Position char C8 1-Senior leader; 2-Interme-
diate leadership; 3-
General sta®; 4-Other;
9-Unknown
Occupation char C9 0-Other; 1-Cadre; 2-Wor-
ker; 3-Individual worker;
4-Farmer; 5-Soldier
Title char C10 0-No; 1-Senior; 2-Interme-
diate; 3-Junior;
9-Unknown
 Novel Five-Category Loan-Risk Evaluation Model 869

Table 3. (Continued )

Index Type Number De¯nition in database

Department char C11 1-State organs; 2-Science,

education; 3-Commer-
cial trade; 4-Telecom-
munication; 5-Finance
and insurance; 6-Social
services; 7-Water, elec-
tric power supply; 8-In-
dustrial and
Transportation; 9-Real
Int. J. Info. Tech. Dec. Mak. 2012.11:857-874. Downloaded from www.worldscientific.com
by UNIVERSITY OF NEW ENGLAND LIBRARIES on 01/23/15. For personal use only.

estate construction; 10-

others
Dependent numeric C12 0; 1; 2; 3; 4
population
Loan amount numeric C13 3000 to 500,000
Due date date C14 1 year to 6 years
Loan-account attributes Interest char C15 1- Day-to-day interest rate;
Settlement cycles 2-Month-to-day interest
rate; 3-Quarter-to-day
interest rate; 4-Semi-
annual-to-day interest
rate; 5-Each year-to-day
interest rate; 6-Month-
to-month interest rate;
7-Quarter interest rate;
8-Semi-annual interest
rate; 9-Year-end interest
rate; 10-Interest no bulk
junction; 11-The ¯rst
interest
Loan purpose char C16 1-Tuition; 2-Feed stock;
3-Liquidity fund;
4-Move back to settle-
ment; 5-Buy house;
6-Buy car.
Loan current status Loan current status char Y 1-Passed; 2-Special Men-
tion; 3-Substandard;
4-Doubtful; 5-Loss

values of all sample data in attribute Ci , respectively; xij denotes the ith attribute in
the jth sample; and x ij0 denotes the data after being normalized.

3.3. Experiments
Our implementation platform is carried out on the MATLAB7.1, a mathematical
development environment, by extending the Libsvm version 2.82, which was origi-
nally designed by Chang and Lin.31 The empirical evaluation is performed on AMD
Turion 64 X2 Duo Core CPU running at 2.00 GHz and 2 GB RAM. Experiment
con¯guration is shown in Table 4.
 870 J. Cao et al.

Table 4. Experiment con¯guration.

Hardware AMD Turion 64 X2 CPU 2.00 GHz, 2.00 GB Memory

Operation system Windows XP Professional
Development platform MATLAB7.1
Toolkit LibSVM2.82

According to the nonlinear characteristic of customer loan data, the RBF kernel
function formulation is K ðx; xk Þ ¼ expð
jjx
xk jj2 =22 Þ. Improved PSO is used to
select the parameters of the proposed model. In our proposed model, the number of
Int. J. Info. Tech. Dec. Mak. 2012.11:857-874. Downloaded from www.worldscientific.com

the population size is 20, and the maximum number of iterations is 100. We set
by UNIVERSITY OF NEW ENGLAND LIBRARIES on 01/23/15. For personal use only.

c1 ¼ c2 ¼ 2,  ¼ 0:729, ! ¼ 0:9.
In our experiment, we compare the performance of our classi¯er with other four
popular methods: (I) 1-v-1 LS-SVM, (II) 1-v-r LS-SVM, (III) 1-v-1 SVM, and (IV)
1-v-r SVM. We set  ¼ 2:1 and C ¼ 7:3.

3.4. Comparison of experiment results

(1) Classi¯cation comparison of the ¯rst experiment
In this paper, the experiments are carried out three times. The average value of three
experiments is used to measure the performance of our proposed model. Speci¯cally,
the calculation process of the ¯rst experiment is shown in Table 5. Among the 132
samples, 109 samples are classi¯ed correctly, and the average accuracy rate is
82.58%. Therein, the classi¯cation accuracy rate for \Passed," \Special Mention,"
\Substandard," \Doubtful," and \Loss" was 82.50%, 81.08%, 69.24%, 87.50%, and
90.00%, respectively.
(2) Comparison of classi¯cation results in three experiments
The calculation process of the other two experiments can be obtained in the same
way. The last classi¯cation results are demonstrated in Table 6 and Fig. 3. In these
three experiments, the classi¯cation accuracy of 1-v-1 LS-SVM is 81.82%, 82.58%,
and 81.06%, respectively. That of 1-v-r LS-SVM is 81.06%, 81.06%, and 80.30%.
That of 1-v-1 SVM is 80.30%, 81.82%, and 81.06%. Finally that of 1-v-r SVM is

Table 5. Classi¯cation results of the ¯rst experiment.

Forecast value

Original Special Sample Misjudgment

value Passed mention Substandard Doubtful Loss amount ratio (%)

Passed 33(82.50%) 4(10.00%) 2(5.00%) 1(2.50%) 0(0.00%) 40 17.50

Special 4(10.81%) 30(81.08%) 2(5.41%) 1(2.70%) 0(0.00%) 37 18.92
Mention
Substandard 0(0.00%) 1(7.69%) 9(69.24%) 2(15.38%) 1(7.69%) 13 30.76
Doubtful 0(0.00%) 2(6.25%) 2(6.25%) 28(87.50%) 0(0.00%) 32 12.50
Loss 0(0.00%) 0(0.00%) 0(0.00%) 1(10.00%) 9(90.00%) 10 10.00
 Novel Five-Category Loan-Risk Evaluation Model 871

Table 6. Comparison of classi¯cation results in three experiments.

1st 2nd 3rd Average

Model Experiment (%) Experiment (%) Experiment (%) accuracy rate (%)

The proposed model 82.58 83.33 81.82 82.58

1-v-1 LS-SVM 81.82 82.58 81.06 81.82
1-v-r LS-SVM 81.06 81.06 80.30 80.81
1-v-1 SVM 80.30 81.82 81.06 81.06
1-v-r SVM 79.55 80.30 78.79 79.55
Int. J. Info. Tech. Dec. Mak. 2012.11:857-874. Downloaded from www.worldscientific.com
by UNIVERSITY OF NEW ENGLAND LIBRARIES on 01/23/15. For personal use only.

Fig. 3. Comparison of the classi¯cation result with di®erent models in three experiments.

79.55%, 80.30%, and 78.79%, respectively. Comparison results could be depicted

more clearly in Fig. 3.
As shown in Table 6, the average classi¯cation accuracy of the proposed rating
model is 82.58%. For 1-v-1 LS-SVM it is 81.82%, for 1-v-r LS-SVM it is 80.81%, for
1-v-1 SVM it is 81.06%, and for 1-v-r SVM it is 79.55%. As shown in Table 6, the
classi¯cation accuracy of 1-v-1 LS-SVM is higher than that of 1-v-r LS-SVM. Sim-
ilarly, the classi¯cation accuracy of 1-v-1 SVM is higher than that of 1-v-r SVM.
Thus, the 1-v-1 method performs much better in classi¯cation. In other words, the
proposed model can obtain a better result in improving classi¯cation accuracy.

4. Conclusion
In this paper, a novel model based on 1-v-1 LS-SVM and improved PSO is used to
research the loan-risk ¯ve-category classi¯cation. LS-SVM is a powerful machine-
learning technique in solving the complex problem of nonlinear function estimation.
The improved PSO is updated from the classical PSO by increasing convergence
index  and inertia weight !, which are used to optimize parameters of LS-SVM. In
 872 J. Cao et al.

empirical studies, the data set is derived from rural credit cooperatives. Five models
are constructed in case analysis, including the proposed model, 1-v-1 LS-SVM, 1-v-r
LS-SVM, 1-v-1 SVM, and 1-v-r SVM. To verify the e®ectiveness of the proposed
model, three experiments are carried out to demonstrate that the proposed model has
higher accuracy than the other four models, so the proposed model has better per-
formance than the others. To sum up, the proposed model provides a new approach
to solving the FCLC problem for Chinese commercial banks.

Acknowledgment
Int. J. Info. Tech. Dec. Mak. 2012.11:857-874. Downloaded from www.worldscientific.com
by UNIVERSITY OF NEW ENGLAND LIBRARIES on 01/23/15. For personal use only.

The authors would like to thank the anonymous referees for their valuable comments
and suggestions. Their comments helped to improve the quality of the paper im-
mensely. This work is partially supported by NSFC(60804047); the Science and
Technology Project of Jiangsu Province, China (BE2010201); the Ministry of Edu-
cation, Humanities, and Social Sciences Research Project (11YJCZH005); the
Jiangsu Provincial Department of Education, Philosophy, and Social Science Project
(2010SJB790025); and the Priority Academic Program Development of Jiangsu
Higher Education Institutions.

References
1. K. L. Ke and Z. X. Feng, Five-category classi¯cation of loan risk based on integration
of rough sets and neural network, Control Theory and Application 25(4) (2008)
759
763.
2. F. Xue and K. L. Ke, Five-category evaluation of commercial bank's loan based on
integration of rough sets and neural network, Systems Engineering


Theory & Practice
1 (2008) 40
45.
3. J. G. Peng, H. B. Tu, J. He and Y. H. Zhou, The application of ordered logistic regression
model in the default probability measure, The Theory and Practice of Finance and
Economics 30(160) (2009) 1
7.
4. C. Wu, Y. J. Guo and H. Xia, The model of credit risk assessment in commercial banks on
fuzzy integral support vector machines ensemble, Operations Research and Management
Science 18(2) (2009) 115
119.
5. J. P. Li, L. W. Wei, G. Li and W. X. Xu, An evolution-strategy based multiple kernels
multi-criteria programming approach: The case of credit decision making, Decision
Support System 51(2) (2011) 292
298.
6. L. W. Wei, Z. Y. Chen and J. P. Li, Evolution strategies based adaptive Lp LS-SVM,
Information Sciences 181(14) (2011) 3000
3016.
7. J. L. Liu, J. P. Li, W. X. Xu and Y. Shi, A weighted Lq adaptive least squares support
vector machine classi¯ers-robust and sparse approximation, Expert Systems with Appli-
cations 38(3) (2011) 2253
2259.
8. L. Yu, X. Yao, S. Y. Wang and K. K. Lai, Credit risk evaluation using a weighted least
squares SVM classi¯er with design of experiment for parameter selection, Expert Systems
with Applications 38 (2011) 15392
15399.
9. G. Wang and J. Ma, A hybrid ensemble approach for enterprise credit risk assessment
based on Support Vector Machine, Expert Systems with Applications 39 (2012)
5325
5331.
 Novel Five-Category Loan-Risk Evaluation Model 873

10. Y. Peng, G. Kou, G. Wang, W. Wu and Y. Shi, Ensemble of software defect predictors:
An AHP-based evaluation method, International Journal of Information Technology &
Decision Making 10(1) (2011) 187
206.
11. C. J. C. Burges, A tutorial on support vector machines for pattern recognition, Data
Mining and Knowledge Discovery 2(2) (1998) 121
167.
12. T. V. Gestel, J. A. K. Suykens, B. Baesens, S. Viaene, J. Vanthienen, G. Dedene, B. De
Moor and J. Vandewalle, Benchmarking least squares support vector machine classi¯ers,
Machine Learning 54(1) (2004) 5
32.
13. J. A. K. Suykens, T. V. Gestel, J. D. Brabanter, B. D. Moor and J. Vandewalle, Least
Squares Support Vector Machines (World Scienti¯c Pub. Co., Singapore, 2002).
14. J. Weston and C. Watkins, Support vector machines for multi-class pattern recognition,
Int. J. Info. Tech. Dec. Mak. 2012.11:857-874. Downloaded from www.worldscientific.com

ESANN'99 (1999).
by UNIVERSITY OF NEW ENGLAND LIBRARIES on 01/23/15. For personal use only.

15. Y. Peng, G. Kou, G. Wang and Y. Shi, FAMCDM: A fusion approach of MCDM methods
to rank multiclass classi¯cation algorithms, Omega 39(6) (2011) 677
689.
16. G. Tsoumakas and I. Katakis, Multi-label classi¯cation: An overview, International
Journal of Data Warehousing and Mining 3(3) (2007) 1
13.
17. C. Cambell, Kernel methods: A survey of current techniques, Neurocomputing 48(124)
(2002) 63
84.
18. K. J. Kim and H. C. Ahn, A corporate credit rating model using multi-class support
vector machines with an ordinal pairwise partitioning approach, Computers and
Operations Research 39 (2012) 1800
1811.
19. Y. Peng, G. Kou, Y. Shi and Z. Chen, A descriptive framework for the ¯eld of data mining
and knowledge discovery, International Journal of Information Technology & Decision
Making 7(4) (2008) 639
682.
20. L. Zhang and B. Zhang, Geometrical representation of McCulloch-Pitts neural model and
its applications, IEEE Transactions on Neural Networks 10(4) (1999) 925
929.
21. S. Knerr, L. Personnaz and G. Dreyfus, Single-layer learning revisited: A stepwise pro-
cedure for building and training a neural network, Neurocomputing: Algorithms, Archi-
tectures and Applications, ed. J. Fogelman (Springer-Verlag, New York, 1990).
22. J. Friedman, Another approach to polychotomous classi¯cation, Dept. Statist., Stanford
Univ., Stanford, CA (1996), http://www-stat.stanford.edu/reports/friedman/poly.ps.z.
23. J. Kennedy and R. C. Eberhart, Particle swarm optimization, Proceedings of IEEE
International Conference on Neural Networks (IEEE Press, Piscataway, 1995), pp.
1942
1948.
24. K. W. Xia, Y. Dong and H. L. Du, Oil layer recognition model of LS-SVM based on
improved PSO algorithm, Control and Decision 22(12) (2007) 1385
1389.
25. Y. Shi and R. C. Eberhart, Parameter selection in particle swarm optimization, Evolu-
tionary Programming VII, Proceedings of the Seventh Annual Conference on Evolu-
tionary Programming, New York (1998).
26. C. W. Hsu and C. J. Lin, A comparison of methods for multi-class support vector
machine, IEEE Transactions on Neural Networks 13 (2002) 415
425.
27. M. Pardo and G. Sberveglieri, Classi¯cation of electronic nose data with support vector
machines, Sensors and Actuators B Chemical 107(2) (2005) 730
737.
28. S. W. Lin, K. C. Ying, S. C. Chen et al., Particle swarm optimization for parameter
determination and feature selection of support vector machines, Expert Systems with
Applications 35(4) (2008) 1817
1824.
29. M. H. Jiang and X. C. Yuan, Construction and application of GA-SVM model for per-
sonal credit scoring, Journal of Hefei University of Technology 31(2) (2008) 267
283.
 874 J. Cao et al.

30. H. K. Lu and J. Cao, personal credit scoring model based on integration of rough set and
GA-neural network, Journal of Nanjing University of Science and Technology (Natural
Science) 33 (2009) 1
5.
31. C. C. Chang and C. J. Lin, LIBSVM: A library for support vector machines [EB/OL]
(2008), http://www.csie.ntu.edu.tw/cjlin/libsvm/index.html.
Int. J. Info. Tech. Dec. Mak. 2012.11:857-874. Downloaded from www.worldscientific.com
by UNIVERSITY OF NEW ENGLAND LIBRARIES on 01/23/15. For personal use only.