There are many factors influencing personal credit. We introduce Lasso technique to personal credit evaluation, and establish Lasso-logistic, Lasso-SVM and Group lasso-logistic models respectively. Variable selection and parameter estimation are also conducted simultaneously. Based on the personal credit data set from a certain lending platform, it can be concluded through experiments that compared with the full-variable Logistic model and the stepwise Logistic model, the variable selection ability of Group lasso-logistic model was the strongest, followed by Lasso-logistic and Lasso-SVM respectively. All three models based on Lasso variable selection have better filtering capability than stepwise selection. In the meantime, the Group lasso-logistic model can eliminate or retain relevant virtual variables as a group to facilitate model interpretation. In terms of prediction accuracy, Lasso-SVM had the highest prediction accuracy for default users in the training set, while in the test set, Group lasso-logistic had the best classification accuracy for default users. Whether in the training set or in the test set, the Lasso-logistic model has the best classification accuracy for non-default users. The model based on Lasso variable selection can also better screen out the key factors influencing personal credit risk.
In the 21st century, with the rapid development of China’s economy, the concept of Chinese people’s consumption has undergone tremendous changes, and the credit industry has developed rapidly. Among them, the development of credit card business is increasing day by day, and the credit risk that comes with it is not to be underestimated. Credit scoring model has been the core of credit risk management. In fact, the credit scoring model is a statistical model that analyzes a large number of customers’ historical data, extracts key factors affecting credit risk, and then constructs a suitable model to evaluate the credit risk of new applicants or existing customers. Therefore, the construction of the personal credit scoring model can respond to credit risk in a timely and effective manner, which will play an important role in both banks and regulatory authorities.
In this era of information explosion, however, the emergence of big data has also led to some credit information, and the existing scoring models often cannot effectively screen out dangerous customers. At the same time, the increasing of the high customer information can lead to the complexity of the credit scoring, model bias and instability, thus variable selection becomes the key issues and difficulties in personal credit evaluation model. It is of great significance to apply the variable selection method to the development of the credit scoring model. In the credit scoring model, the subset selection such as stepwise regression is a discrete and unstable process, and the variable selection will be changed by small changes in the data set. Selection and parameter estimation also need to be carried out in two steps. Subsequent parameter estimation does not take into account the bias caused by variable selection, and accordingly it underestimates the actual variance. The calculation of subset selection is also quite complicated. In view of these defects, we adopt the Lasso method which can simultaneously perform variable selection and parameter estimation. After quantifying many explanatory variables, it is necessary to establish dummy variables as explanatory variables of the model. When using stepwise regression to select variables, only one dummy variable can be selected, which is the reason why the results are difficult to explain. However, the above problems can be well solved by Group lasso when it performs variable selection on group variables, making the dummy variables belonging to the same group be completely retained or fully eliminated in the model.
In this paper, the Logistic and SVM models of Lasso were mainly used to select and classify the influencing factors of personal credit evaluation. Then, the prediction accuracy of several models for default users is compared.
Typical credit evaluation models are: linear discriminant analysis, logistic regression, K-nearest neighbor, classification tree, neural network, genetic algorithm, support vector machine [
On the contrary, in China, the construction of the credit score system just started. Shi and Jin [
There are two main methods for selecting variables: subset selection method and coefficient compression method. Subset selection method is that in linear model, all variables form a set, and each subset of the set corresponds to a model. According to certain criteria, an optimal subset fitted regression model is selected from all subsets or partial subsets.
The main research on subset selection are AIC (Akaike Information Criterion) [
With the continuing research, the variable selection method based on penalty function has been widely concerned by statistical researchers. The basic idea of this method is to add a new penalized term to the least squares or maximum likelihood function and we then minimize or maximize the augmented objective function. Thus, by compressing the regression coefficients of the insignificant variables to zero, the variables are eliminated, and the significant variables are compressed very little or it can be retained in the regression model without compression. Hence it performs the variables selection and parameters estimation simultaneously, greatly improving the speed of calculation. Regarding the penalty function, the earliest penalty function is the ridge regression method proposed by Hoerl and Kennard [
In the third section, we present the algorithm models of Lasso-logistic, Lasso-SVM and Group lasso-logistic, and propose the method to select the parameter lambda in the model. In the fourth section, with the help of the credit data of the credit platform, SPSS software is used to preprocess the data. Section five and six use R language to compare and analyze the variable selection ability and prediction accuracy of the model through numerical experiments, so as to draw relevant conclusions.
Logistic regression is a probabilistic nonlinear model, which is a multivariable analysis method used to study the relationship between binary observation results and some influencing factors. Its basic idea is to study whether a result occurs under certain factors. For example, this paper uses some variable indicators to judge a person’s credit status. Logistic regression can be expressed as:
P = 1 1 + e − s ,
s = β 0 + ∑ i = 1 n β i x i .
where x i ( i = 1 , 2 , ⋯ , n ) is the explanatory variable in the credit risk assessment (or the characteristic indicator of the individual), β i ( i = 1 , 2 , ⋯ , n ) regression coefficient. Logistic regression value P ∈ ( 0 , 1 ) is the discriminant result of credit risk.
The graph of the function in Logistic regression model has an s type distribution, as shown in
As you can see from
lim s → + ∞ P = lim s → + ∞ 1 1 + e − s = 1 ,
lim s → − ∞ P = lim s → − ∞ 1 1 + e − s = 0.
For someone i ( i = 1 , 2 , ⋯ , n ) , if P i is close to 1 (or P i ≈ 1 ), then it is judged as a “poor” credit person (or risk of default); if P i is close to 0 (or P i ≈ 0 ),
then the person is judged to be “good”. That is, the value of P i farther away from 1 indicates that the person is less likely to fall into default set. On the contrary, it means that the risk of default is greater.
Suppose there are data variables ( x i , y i ) , i = 1 , 2 , ⋯ , n ,
where x i = ( x i 1 , x i 2 , ⋯ , x i m ) which is the observed value of the explanatory variable and y i ∈ { 0 , 1 } is the observed value of the interpreted variable. In the general regression model, the observed values of the explanatory variable and the interpreted variable are often considered to be independent. In addition,
assume that x i j is standardized. Namely, 1 n ∑ i x i j = 0 , 1 n ∑ i x i j 2 = 1 . Let
P i = P ( y i = 1 | x i ) be the conditional probability of y i = 1 given x i . The conditional probability under the same conditions is P ( y i = 0 | x i ) = 1 − P i . Then, given a test sample ( x i , y i ) , its probability is:
P ( y i ) = P i y i ( 1 − P i ) 1 − y i ,
where P i = e β 0 + β 1 x i 1 + ⋯ + β m x i m 1 + e β 0 + β 1 x i 1 + ⋯ + β m x i m . Assume that each sample is independent of each
other. Their joint distribution (i.e., likelihood function) can be expressed as:
L ( β 0 , β 1 , ⋯ , β m ) = ∏ i = 1 n P i y i ( 1 − P i ) 1 − y i .
The Maximum Likehood method is a good choice to estimate the parameter β . Because it can maximize the possibility that the observed value of each sample is equal to its true value. In other words, it can maximize the log likelihood function in the logistic model:
ln ( L ( β 0 , β 1 , ⋯ , β m ) ) = ln ( ∏ i = 1 n P i y i ( 1 − P i ) 1 − y i ) = ∑ i = 1 n y i ( X i β − ln ( 1 + e X i β ) ) .
For convenience, we set X i = ( 1 , x i ) and β = ( β 0 , β 1 , ⋯ , β m ) T . Estimating the model’s parameter β by maximum likelihood estimation is equivalent to solve the following problem:
β ^ = arg max l ( β ) ,
It is easy to know that l ( β ) is concave and continuously differentiable, and therefore its local maximizer is the global maximizer. Calculate partial derivatives and make it to be zero, which leads to the likelihood equations:
∂ ln ( L ( β 0 , β 1 , ⋯ , β m ) ) ∂ β 0 = ∑ i = 1 n ( y i − e β 0 + β 1 x i 1 + ⋯ + β m x i m 1 + e β 0 + β 1 x i 1 + ⋯ + β m x i m ) = 0 ,
∂ ln ( L ( β 0 , β 1 , ⋯ , β m ) ) ∂ β j = ∑ i = 1 n ( y i − e β 0 + β 1 x i 1 + ⋯ + β m x i m 1 + e β 0 + β 1 x i 1 + ⋯ + β m x i m ) x i j = 0.
But it is difficult to get an explicit solution. It needs to be solved by some iterative methods such as Newton-Raphson, EM and gradient descent algorithms. The estimated β j obtained by the likelihood equation is called the maximum likelihood estimate, and the corresponding conditional probability P i is estimated by P i ^ .
Logistic has a wide range of applications in credit scoring. The traditional Logistic method is very simple, but it is sensitive to multi-collinearity interference between individual credit variables. Therefore, some redundant variables are selected, resulting in poor prediction results. That is why we improve this method.
Tishirani proposed the Lasso method which is motivated by non-negative Garrote [
Let β ^ = ( β ^ 1 , ⋯ , β ^ m ) T , the estimator ( α ^ , β ^ ) of the lasso method is:
( α ^ , β ^ ) = arg min α , β { ∑ i n ( y i − α − ∑ j = 1 m β j x i j ) 2 } , s . t . ∑ j = 1 m | β j | ≤ t ,
where t ≥ 0 is the regularization parameter. For all t, one has an estimator α ^ = y ¯ of α . Without loss of generality, we assume that y ¯ = 0 , Above problem can be rearranged into the following form:
β ^ = arg min β { ∑ i n ( y i − ∑ j = 1 m β j x i j ) 2 } , s . t . ∑ j | β j | ≤ t .
It can also be expressed in the form of the following penalty function:
β ^ = arg min β { ∑ i n [ ( y i − ∑ j = 1 m β j x i j ) 2 + λ ∑ j = 1 m | β j | ] } .
The first part of the formula represents the goodness of the model fit, and the second part represents the penalty of the parameter. The harmonic coefficient λ ∈ [ 0 , + ∞ ] is smaller. The smaller role of the penalty term plays the more variables is retained; the larger lambda is, the more roles of the penalty term plays, and the fewer variables are retained.
The Lasso method is mainly applied to linear models. The essence is to add a penalty function to the sum of squared residuals. When estimating parameters, the coefficients are compressed, and some coefficients are even compressed to 0 to achieve model variable selection. But for credit default prediction, the dependent variable is a binary value. In this case, the linear regression model cannot be used. Instead, Lasso-logistic [
The conditional probability of the logistic linear regression model can be expressed as:
log { P ( y i = 1 | x i ) 1 − P ( y i = 1 | x i ) } = η β ( x i ) ,
where η β ( x i ) = X i β .
The coefficient estimate β λ ^ in the Lasso-logistic regression model is given by the minimum value of the convex function of the following form:
S λ ( β ) = − l ( β ) + λ ∑ j = 1 m | β j | ,
where
l ( β ) = ∑ i = 1 n { y i η β ( x i ) − log { 1 + exp [ η β ( x i ) ] } }
The estimator β ^ in Lasso-logistic regression model can be given as:
β ^ = arg min β − ∑ i = 1 n { y i η β ( x i ) − log { 1 + exp [ η β ( x i ) ] } } + λ ∑ j = 1 m | β j | .
The standard SVM model does not have feature selection capabilities. The specific approach of adding regularization to the SVM model is to use the regularization term with sparsity to replace the L 2 norm in the standard SVM. The L 1 norm is convex functions, with Lipschitz continuum, having properties better than other norms. L1-SVM and its similar extensions have evolved into one of the most important tools for data analysis. The general form of Lasso-SVM is given below:
min ∑ j = 1 m | β i | + C ∑ i = 1 n ξ i
s . t . y i ( X i T β ) ≥ 1 − ξ i , i = 1 , 2 , ⋯ , n
ξ i ≥ 0.
Lasso-SVM can also be written in the following form:
min ∑ i = 1 n [ 1 − y i f ( X i ) ] + + λ ∑ j = 1 m | β i | .
where [ 1 − y i f ( X i ) ] + is a Hinge loss function and λ is a regularization parameter.
Group lasso was introduced by Yuan and Lin (2006), allowing pre-defined covariates to be grouped together and selected from the model. All variables in a particular group can be included or not included. It is very useful in many settings. Group lasso algorithm for logistic regression was first proposed by Kim et al., and then Meier et al. [
Suppose there is an independent and identical distribution of observation ( x i , y i ) , i = 1 , 2 , ⋯ , n . x i = ( x i 1 , x i 2 , ⋯ , x i m ) which is an m-dimensional vector that can be divided into G groups, and the dependent variable is a binary variable y i ∈ { 0 , 1 } . The independent variable can be a continuous variable or a classified variable. Assume that the degree of freedom of the group g argument is d f g , X i = ( 1 , x i , 1 , x i , 2 , ⋯ , x i , G ) , ( g = 1 , 2 , ⋯ , G ) . x i , g denotes the x i , g group of variables of the observation x i , g . Similarly, β can be expressed as ( β 0 ; β 1 ; β 2 ; ⋯ ; β G ) , β g denotes the coefficients corresponding to group G g variables, where the labeling method is used to distinguish the β j fraction in the case of no grouping. The probability of “default” of the dependent variable P β ( x i ) = P β ( y = 1 | x i ) can be expressed by the following model:
log { P β ( x i ) 1 − P β ( x i ) } = η β ( x i ) = β 0 + ∑ g = 1 G x i , g T β g = X i β ,
where β 0 denotes intercept, β g is the coefficient vector corresponding to group g and β is the whole coefficient vector.
The parameter β ^ ( λ ) can be estimated by minimizing the convex function:
S λ ( β ) = − l ( β ) + λ ∑ g = 1 G s ( d f g ) ‖ β g ‖ 2 ,
where l ( β ) is a logarithmic likelihood function:
l ( β ) = ∑ i = 1 n { y i η β ( x i ) − log { 1 + exp [ η β ( x i ) ] } } ,
s ( d f g ) = ( d f g ) 1 / 2 and s ( ⋅ ) is used to rescale the parameter β g vector.
In the variable selection model, the key lies in the selection of the harmonic parameter lambda. That is to say, the optimal lambda determines the prediction accuracy and robustness of the model. The common methods for the optimal lambda are AIC, BIC, Cross-validation, Generalized cross-validation. Here, we use K-fold cross-validation to determine the optimal lambda.
The main idea of K-fold cross validation is that the data are randomly divided into K (usually 5 or 10) identical parts. Each k = 1 , ⋯ , K , uses the data of the K part as the test sample, and uses the remaining K-1 parts of the data as the training sample to fit the model. Loop K times until all k are traversed. We denote the estimator by β ^ − k . The harmonic parameter β ^ ( λ ) corresponds to a classification model and the corresponding estimator β ^ ( λ ) . The generalization error of each model corresponding to lambda is given by the mean square prediction error. That means Cross-Validation Error (CVE) is estimated:
C V ( λ ) = 1 K ∑ k = 1 K 1 n k ∑ i ∈ C k ( y i − X i T β ^ − k ) 2 ,
where C k is the k-th partial cross-check sample, n k = | C k | = n K . Minimize the
above formula to find the most appropriate harmonic parameters, and the corresponding model can be considered to be the model with the best performance based on cross-check error.
The original data is mainly from a domestic lending institution. There are a total of 8000 records in this data set, including 25 fields. Among them, 23 fields describe the personal characteristics of the lender, including the basic personal identity information: domicile, gender, local work, education level and marital status. Also include personal economic ability: whether there is a CPF salary level. Data set also includes personal debt and debt repayment record: frequency of personal housing loan, personal commercial housing loan pen number and frequency of other loan credit card account number, number, frequency of delinquent loans, loans overdue month loan highest monthly overdue amount, maximum length, loan account number of the contract amount, loan balance has been used lines, the average individual loan maximum contract value, the average individual loans minimum contract amount, the last six months on average use. Finally, the data set also gives the total number of times of individual approval query and loan number. The result, where “0” is the performance customer and “1” is the default customer.
There are missing and abnormal data in the original data, and the missing value filling and outlier detection are needed before analysis. The method of dealing with missing values in this paper is the average filling, and using the scatter plot to detect outliers. In the original data, such as contract amount, loan balance and used amount are continuous variables. In order to overcome the influence of the dimension, the F-score needs to be standardized and analyzed. At the same time, the ratio of the number of compliance users and default users in the sample data is about 9:2, and it is an asymmetric distribution problem, which affects the prediction accuracy of the model for default customers with relatively small data capacity. Therefore, the under-sampling method is adopted for compliance users. That means some representative data are selected from the data with more samples. In order to reduce the majority of the sample, the data balance is achieved. The final data set is divided into a training set and a test set, wherein the training set has 3002 data, including 1500 compliance data and 1502 default data, and the test set has 519 data including 258 compliance data and 261 default data.
The classified and encoded variables in the data are shown in
The full-variable logistic and stepwise logistic regression models were implemented by using SPSS 22.0. The Lasso-logistic model was implemented by using the glmnet package in R language, the Lasso-SVM model was implemented by using the gcdnet package, and the Group lasso-logistic model was implemented by using grpreg. The code package uses the generalized coordinate descent method [
Through the K-fold cross-validation, the Lasso-logistic, Lasso-SVM, and Group lasso-logistic models are changed with the value of lambda, and the model error is changed. At the top of
| ID | Variable Name | The Values |
|---|---|---|
| y | default | y = 0 Not default; y = 1 Default |
| x 1 | registered area | x 1 , 1 = Northeast; x 1 , 2 = North China Plain; x 1 , 3 = Central China; x 1 ， 4 = Eastern China |
| x 2 | gender | x 2 ， 1 = Male; x 2.2 = Female |
| x 3 | Whether work is local | x 3 , 1 = Local; x 3.2 = Not local |
| x 4 | edu level | x 4 , 1 = Junior high school/Senior high school/others; x 4 , 2 = Junior college/Junior college and below; x 4 ， 3 = Undergraduate; x 4 , 4 = Master/Doctor |
| x 5 | marital status | x 5 , 1 = Maid; x 5 , 2 = Married; x 5 , 3 = Others (divorced/widowed) |
| x 6 | Whether accumulation fund | x 6 , 1 = Not; x 6 , 2 = Yes |
| x 7 | pay grades | x 7 , 1 = 0 - 3500; x 7 ， 2 = 3501 - 8000; x 7 , 3 = 8000 above |
| x 8 | The number of individual housing loans | x 8 ∈ N |
| x 9 | The number of individual commercial housing loans | x 9 ∈ N |
| x 10 | Other loans | x 10 ∈ N |
| x 11 | Debit card account number | x 11 ∈ N |
| x 12 | The number of overdue loans | x 12 ∈ N |
| x 13 | Months of overdue loans | x 13 ∈ N |
| x 14 | The maximum amount of overdue loans per month | x 14 ∈ [ 0 , + ∞ ] |
| x 15 | Maximum length of loan (year) | x 15 ∈ N |
| x 16 | Total number of approval inquiries | x 16 ∈ N |
| x 17 | Loan number | x 17 ∈ N |
| x 18 | Loan account number | x 18 ∈ N |
| x 19 | contract amount | x 19 ∈ [ 0 , + ∞ ] |
| x 20 | loan balance | x 20 ∈ [ 0 , + ∞ ] |
| x 21 | Have used limit | x 21 ∈ [ 0 , + ∞ ] |
| x 22 | Average maximum contract amount for a single lender | x 22 ∈ [ 0 , + ∞ ] |
| x 23 | Average minimum contract amount for a single lender | x 23 ∈ [ 0 , + ∞ ] |
| x 24 | Average usage in the last 6 months | x 24 ∈ [ 0 , + ∞ ] |
In the logistic regression model, the dependent variable is log-occurrence ratio logit. When the log-occurrence ratio increases, the value of P also increases
accordingly, which means the probability for judging credit as 1 (i.e., default) increases. When the coefficient β i is negative, it means that the variable x i has a reverse restrictive effect on the default. When the coefficient β i is positive, the corresponding variable x i has a positive effect on the default, and the greater the value of β i , the greater the promoting effect of the corresponding x i on the customer’s credit judgment as default. In full-variable logistic model, the variable x6 (Whether accumulation fund), x11 (Debit card account number), x14 (The maximum amount of overdue loans per month), x22 (Average maximum contract amount for a single lender), x13 (Average minimum contract amount for a single lender) before the coefficients were not significant, which means the model contains too many variables, and the model is too complicated. The above non-significant variables were eliminated by stepwise regression. At the same time, x18 (Loan account number) and x21 (Have used limit) were also eliminated. Finally, 13 variables were removed for both forward and backward modes.
For Lasso-logistic model, there are 16 variables whose coefficient is compressed to 0. In other words, 18 important variables are selected to enter the
model. The Lasso-SVM model eliminates 15 variables, leaving 19 variables. However, it can be seen that when using stepwise regression, Lasso-logistic and Lasso-SVM models for variable selection, the variables are excluded as classification variables, and some dummy variables in the same group are partially retained and partially eliminated, such as x4 (edu level), which makes the result difficult to explain, showing in
Using Group lasso, after variable selection, 18 variables were removed and 16 variables were retained. In addition, Group lasso-logistic can retain or eliminate related dummy variables of the same group as a whole, making the dummy variables have explanatory significance. We obtained from the coefficient table of the Group lasso model that in the regional variable (x1), the north China area is the high default area, and the central China area has the lowest default risk. There was a significant gender (x2) difference in credit risk that the default probability of male was generally higher than that of female. In the salary scale (x7), people with low incomes were more at risk of default than those with medium and high incomes. In historical credit records, customers with overdue loans are at greater risk of default, and the number of overdue loans (x12) and months of overdue loans (x13) are more likely to default. More total number of approval inquiries (x16) affect an individual’s credit history. Variables with a
| Variate | Full variables | Forward | Backwards | Lasso-logistic | Lasso-SVM | Group-lasso |
|---|---|---|---|---|---|---|
| X1_1 | 1.715 | 0 | 1.721 | 0 | −0.555 | 0.025 |
| X1_2 | 1.674 | 0 | 1.695 | 0.010 | −0.567 | 0.001 |
| X1_3 | −4.562 | −4.247 | −4.561 | −4.354 | 0.667 | −6.520 |
| X1_4 | −1.534 | −1.681 | 0 | −1.440 | 0.016 | −1.637 |
| X2 | −0.868 | −0.854 | −0.858 | −0.630 | 0.209 | −0.824 |
| X3 | −0.497 | −0.504 | −0.502 | −0.352 | 0.122 | −0.471 |
| X4_1 | 1.408 | 0.341 | 0.501 | 0.105 | −0.003 | 0.207 |
| X4_2 | 0.780 | 0.301 | 0.218 | 0 | 0 | −0.007 |
| X4_3 | 0.575 | 0 | 0 | −0.111 | 0.015 | −0.195 |
| X4_4 | 0.761 | 0 | 0 | 0 | 0 | −0.682 |
| X5_1 | −0.025 | −0.002 | 0 | 0 | 0 | 0.062 |
| X5_2 | −0.179 | 0 | −0.201 | −0.037 | 0 | −0.072 |
| X5_3 | −0.034 | 0 | 0 | 0 | 0 | 0.073 |
| X6 | 0.072 | 0 | 0 | 0 | 0 | 0.065 |
| X7_1 | 0.306 | 0.393 | 0.311 | 0.355 | −0.085 | 0.570 |
| X7_2 | −0.893 | 0.205 | −1.111 | −0.582 | 0.226 | −0.554 |
| X7_3 | 0.934 | 0 | 0 | 0 | 0 | 0.251 |
| X8 | 0.123 | 0.101 | 0.098 | 0 | 0 | 0 |
| X9 | 0.058 | 0 | 0 | 0 | 0 | 0.039 |
| X10 | −0.265 | −0.213 | −0.212 | −0.104 | 0.053 | −0.194 |
| X11 | −0.078 | 0 | 0 | −0.003 | 0.013 | −0.016 |
| X12 | −0.100 | −0.111 | −0.109 | −0.084 | 0.033 | −0.108 |
| X13 | −0.190 | −0.206 | −0.197 | −0.064 | 0.050 | −0.170 |
| X14 | −0.033 | 0 | 0 | 0 | 0 | −0.021 |
| X15 | 0.130 | 0.121 | 0.119 | 0 | −0.016 | 0.101 |
| X16 | 0.455 | 0.459 | 0.457 | 0.306 | −0.142 | 0.431 |
| X17 | 0.222 | −0.218 | −0.209 | −0.173 | 0.084 | −0.179 |
| X18 | 0.164 | 0 | 0 | 0 | 0 | 0 |
| X19 | −0.405 | 0 | 0 | −0.021 | 0 | −0.062 |
| X20 | 0.276 | 0 | 0 | 0 | 0 | 0 |
| X21 | 0.136 | 0 | 0 | 0 | 0 | 0 |
| X22 | 0.065 | 0 | 0 | 0 | 0 | 0 |
| X23 | 0.041 | 0 | 0 | 0 | 0 | 0.035 |
| X24 | −0.325 | −0.249 | −0.263 | −0.178 | 0.067 | −0.212 |
| Intercept term | 1.026 | 1.275 | 1.836 | 2.575 | −0.382 | 2.980 |
coefficient 0 indicate that they have been removed from the model and have little effect on credit rating.
Showing in
In the actual credit risk assessment, the misclassification of default users into non-defaulting users is more of a potential loss to banks or society. Therefore, the model is more important for to correctly classify the default users than to take non-defaulting users into consideration. It is easy to see in
In the personal credit evaluation, the Logistic model is most widely used, and the newly proposed SVM method in statistical learning also has certain application in credit evaluation. By comparing the simulation experiment analysis, the whole variable, Forward selection, Backward selection, Lasso-logistic, Lasso-SVM, and Group lasso-logistic models and empirically analyzing the personal credit data of a domestic lending platform, it can be concluded:
First, the experiment found that when all the variables were included in the full-variable Logistic mode, the coefficients before many variables could not pass
| Model | Training set | Test set | ||||
|---|---|---|---|---|---|---|
| Good | Bad | Total | Good | Bad | Total | |
| Full variables | 71.3 | 75.6 | 73.4 | 67.0 | 71.7 | 69.3 |
| Forward selection | 70.7 | 75.6 | 73.1 | 65.5 | 72.1 | 68.9 |
| Backward selection | 72.3 | 73.8 | 73.1 | 69.3 | 67.1 | 68.2 |
| Lasso-logistic | 74.5 | 80.0 | 77.2 | 74.7 | 79.5 | 77.1 |
| Lasso-SVM | 73.7 | 80.2 | 76.9 | 73.6 | 80.2 | 76.8 |
| Group lasso | 74.4 | 79.4 | 76.9 | 75.1 | 78.0 | 76.5 |
the significant level test. Thus, to some extent, the complexity of the model was increased. The interpretability of the model was reduced. The choice and Lasso overcome the multicollinearity of the full-variable model, and the coefficients of the insignificant variables in the model are compressed. Compared with the stepwise regression, the Group lasso-logistic culling variable is the strongest, followed by Lasso-logistic, Lasso-SVM model. The algorithm model based on Lasso variable selection can better select important variables, and Group lasso-logistic will retain the whole group or the entire group when the same group of dummy variables is selected, which will enhance the variables in the model to some extent.
Second, in the training set, the Lasso-SVM model has the highest prediction accuracy rate for default users; in the test set, Group lasso-logistic ranks the first in the classification accuracy of default users. Whether in the training set or in the test set, the best classification accuracy of non-defaulting users is the Lasso-logistic model. Moreover, in the training set, the overall prediction accuracy of the Lasso-logistic model is also the best. In the test set, the Group lasso-logistic model has the best overall prediction accuracy. Regardless of the prediction of defaulting users, the prediction of non-defaulting users and the overall forecasting accuracy, Lasso is better than stepwise selection. It shows that the credit scoring model based on Lasso variable selection has good extrapolation.
Therefore, based on the Logistic and SVM models established by the Lasso variable selection method, the explanatory variables can be selected more scientifically and have use value in personal credit risk assessment, which can well reduce personal credit risk.
To sum up, it is not difficult to find that in the actual rating, we often encounter some relationships between variables, thus forming a grouping structure. The traditional variable selection method cannot process the dummy variables of related groups as a whole, resulting in partial retention and partial elimination of the variables of the whole Group. In this way, the results are difficult to be explained, and Group lasso can well solve the above problems. Therefore, the Logistic and SVM models established based on the Lasso variable selection method can more scientifically select explanatory variables, which have application value in personal credit risk assessment and can well reduce personal credit risk.
In future work, we will consider individual credit ratings for unbalanced datasets. When we use Group lasso for intra-group variable selection, the coefficient of some individual variables within the Group may not be significant. In this case, the two-layer variable selection is introduced to solve such problems.
The author declares no conflicts of interest regarding the publication of this paper.
Li, Q.Q. (2019) Logistic and SVM Credit Score Models Based on Lasso Variable Selection. Journal of Applied Mathematics and Physics, 7, 1131-1148. https://doi.org/10.4236/jamp.2019.75076