可解释的信用风险评估模型:基于注意力机制的规则提取方法

doi:10.11896/jsjkx.250300059

摘要/Abstract

摘要： 信用风险评估旨在预判客户是否会违约,被视为一项复杂的非线性二分类难题。尽管传统的统计模型在信用评估领域具有一定的应用价值,但其局限性也日益显现。鉴于此,机器学习技术,特别是支持向量机、深度神经网络和集成学习等先进方法,在信用风险评估领域得到了广泛应用,旨在提升模型的准确性和预测精度。然而,尽管这些机器学习模型性能卓越,但其内在的复杂性和不透明性导致模型预测结果难以向用户阐释,在实施过程中面临诸多挑战。为解决这一问题,提出了一种可解释的信用风险评估模型,该模型融合了注意力机制与树集成规则提取技术,能够自动识别训练数据中的复杂非线性关系,实现模型自身的可解释。首先从训练好的树集成模型中提炼出众多可解释的规则,并将这些规则转换为新的特征变量,然后将这些新的特征变量作为注意力神经网络的输入,以精确计算每条规则的注意力权重。在此基础上,模型根据注意力权重、目标函数及约束条件,综合考虑规则子集的预测精度、稳定性和可解释性,可在线性时间内高效地求得最优规则子集。在3个公开数据集上进行了实验,结果表明,所提方法在保持模型较高预测精度的前提下,实现了模型可解释性的显著提升。

关键词: 机器学习可解释性, 信用风险评估, 注意力机制, 规则生成算法, 树集成模型

Abstract: Due to the limitations of traditional statistical models for credit risk assessment,machine learning techniques have significantly enhanced model accuracy and predictive capabilities.However,the complexity and opacity pose significant challenges in terms of interpretation.To address this issue,this paper introduces an interpretable machine learning model for credit risk assessment that integrates the attention mechanism with tree ensemble rule extraction approach.This model automatically identifies complex nonlinear relationships within the data,extracts a large number of interpretable rules from the trained tree ensemble model,encodes these rules into new feature variables,and inputs them into an attention neural network to obtain attention weights for each rule.Subsequently,based on the attention weights,objective function,and constraints,the model balances the predictive performance,stability,and interpretability of the rule subset.The optimal rule subset can be derived in O(n) time.Experimental results,based on three public datasets,demonstrate that the proposed approach not only maintains high predictive accuracy but also substantially enhances the model's interpretability.

Key words: Interpretable machine learning,Credit risk assessment,Attention mechanism,Rule generation algorithm,Tree ensemble models

中图分类号:

TP391

王宝财, 吴国伟. 可解释的信用风险评估模型:基于注意力机制的规则提取方法[J]. 计算机科学, 2025, 52(10): 50-59. https://doi.org/10.11896/jsjkx.250300059

WANG Baocai, WU Guowei. Interpretable Credit Risk Assessment Model:Rule Extraction Approach Based on AttentionMechanism[J]. Computer Science, 2025, 52(10): 50-59. https://doi.org/10.11896/jsjkx.250300059

参考文献

[1]BAESENS B.Using neural network rule extraction and decision tables for credit-risk evaluation[J].Management Science,2003,49(3):312-329.
[2]SERRANO-CINCA C,GUTIERREZ-NIETO B.Partial LeastSquare Discriminant Analysis for bankruptcy prediction[J].Decision Support Systems,2013,54(3):1245-1255.
[3]MIYAMOTO M,MIYAMOTO M.Credit risk assessment for a small bank by using a multinomial logistic regression model[J].International Journal of Finance and Accounting,2014,3(5):327-334.
[4]COSTA E SILVA E,LOPES C,CORREIA A,et al.A logistic regression model for consumer default risk[J].Journal of Applied Statistics,2020,47:1-17.
[5]BEQUÉ A,COUSSEMENT K,GAYLER R,et al.Approachesfor credit scorecard calibration:an empirical analysis[J].Know-ledge-Based Systems,2017,134:213-227.
[6]LI T,WANG H,WU J,et al.Sparse Bayesian learning for credit risk evaluation[J].Journal of Computer Applications,2013,33(11):4.
[7]BHATTACHARYA A,WILSON S P,SOYER R.A Bayesianapproach to modeling mortgage default and prepayment[J].European Journal of Operational Research,2019,274(3):1112-1124.
[8]MELNYK K V,BORYSOVA N V.Improving the quality ofcredit activity by using scoring model[J].Radio Electronics Computer Science Control,2019(2):60-70.
[9]DAMRONGSAKMETHEE T,NEAGOE V E.Principal Component Analysis and ReliefF Cascaded with Decision Tree for Credit Scoring[M]//Artificial Intelligence Methods in Intelligent Algorithms.Cham:Springer,2019.
[10]CHERN C C,LEI W U,HUANG K L,et al.A decision treeclassifier for credit assessment problems in big data environments[J].Information Systems and e-Business Management,2021,19(1):363-386.
[11]GOH R,LEE L S.Credit Scoring:A Review on Support Vector Machines and Metaheuristic Approaches[J].Advances in Operations Research,2019,2019:1-30.
[12]LEE I G,YOON S W,WON D.A Mixed Integer Linear Programming Support Vector Machine for Cost-Effective Group Feature Selection:Branch-Cut-and-Price Approach[J].European Journal of Operational Research,2022,299(3):1055-1068.
[13]SHEN F,YANG Z,ZHAO X,et al.Reject inference in credit scoring using a three-way decision and safe semi-supervised support vector machine[J].Information Sciences,2022,606:614-627.
[14]WANG A Q,HAN Z C,WANG Y L.Risk assessment of logistics finance enterprises based on BP neural network and fuzzy mathematical model[J].Journal of Intelligent & Fuzzy Systems,2020,39:5915-5925.
[15]FRAISSE H,LAPORTE M.Return on investment on artificial intelligence:The case of bank capital requirement[J].Journal of Banking & Finance,2022,138:106401.
[16]KELLNER R,NAGL M,ROSCH D.Opening the black box－Quantile neural networks for loss given default prediction[J].Journal of Banking & Finance,2022,134:106334.
[17]BHATORE S,MOHAN L,REDDY Y R.Machine learningtechniques for credit risk evaluation:a systematic literature review[J].Journal of Banking and Financial Technology,2020,4(1):111-138.
[18]DASTILE X,CELIK T,POTSANE M.Statistical and machine learning models in credit scoring:A systematic literature survey[J].Applied Soft Computing,2020,91:106263.
[19]LENKA S R,BISOY S K,PRIYADARSHINI R,et al.Empirical Analysis of Ensemble Learning for Imbalanced Credit Scoring Datasets:A Systematic Review[J].Wireless Communications and Mobile Computing,2022,2022:6584352.
[20]HOFMAN J M,SHARMA A,WATTS D J.Prediction and explanation in social systems[J].Science,2017,355(6324):486-488.
[21]CHEN D X,YE J H,YE W C.Interpretable selective learning in credit risk[J].Research in International Business and Finance,2023,65:101940.
[22]DAVIS R,LO A W,MISHRA S,et al.Explainable Machine Learning Models of Consumer Credit Risk[J].Journal of Financial Data Science,2023,5(4).
[23]DUVNJAK M,MERĆEP A,KOSTANJČAR Z.Intrinsically Interpretable Models for Credit Risk Assessment[C]//2024 47th MIPRO ICT and Electronics Convention.IEEE,2024:31-36.
[24]Equal Credit Opportunity Act[S].United States Code,title 15,chapter 41,subchapter IV,1974.
[25]HOOFNAGLE C J,VAN DER SLOOT B,ZUIDERVEENBORGESIUS F.The European Union general data protection regulation:what it is and what it means[J].Information & Communications Technology Law,2019,28(1):65-98.
[26]MASHAYEKHI M,GRAS R.Rule extraction from decisiontrees ensembles:new algorithms based on heuristic search and sparse group lasso methods[J].International Journal of Information Technology & Decision Making,2017,16(6):1707-1727.
[27]HADDOUCHI M,BERRADO A.A survey and taxonomy ofmethods interpreting random forest models [J].arXiv:2407.12759,2024.
[28]MARTENS D,BAESENS B,GESTEL T V,et al.Comprehensible credit scoring models using rule extraction from support vector machines[J].European Journal of Operational Research,2007,183(3):1466-1476.
[29]HADDOUCHI M,BERRADO A.Forest-ORE:Mining an optimal rule ensemble to interpret random forest models[J].Engineering Applications of Artificial Intelligence,2025,143:109997.
[30]BIRBIL S I,EDALI M,YUCEOGLU B.Rule Covering for Interpretation and Boosting[J].Information Fusion,2020,63:196-207.
[31]MANZALI Y,ELFAR M.Optimizing the number of branches in a decision forest using association rule metrics[J].Knowledge and Information Systems,2024,66(6):3261-3281.
[32]BORUAH A N,BISWAS S K,BANDYOPADHYAY S.Transparent rule generator random forest(TRG-RF):an interpretable random forest[J].Evolving Systems,2023,14(1):69-83.
[33]BOLOGNA G.A rule extraction technique applied to ensembles of neural networks,random forests,and gradient-boosted trees[J].Algorithms,2021,14(12):339.
[34]EDALI M.Performance analysis of set partitioning formulations on the rule extraction from random forests[J].Pamukkale University Journal of Engineering Sciences,2021,27(4):513-519.
[35]CHEN M,HUO J,DUAN Y.An interpretable model for sepsis prediction using multi-objective rule extraction[J].Journal of Intelligent Information Systems,2024,62(5):1403-1429.
[36]SHAMS Z,DIMANOV B,KOLA S,et al.REM:An Integrative Rule Extraction Methodology for Explainable Data Analysis in Healthcare[R].medRxiv,2021.
[37]WANG S,WANG Y,WANG D,et al.An improved random forest-based rule extraction method for breast cancer diagnosis[J].Applied Soft Computing Journal,2020,86:105941.
[38]MASHAYEKHI M,GRAS R.Rule extraction from random forest:the RF+HC methods[M]//Advances in Artificial Intelligence.Cham:Springer,2015:223-237.
[39]DENG H.Interpreting tree ensembles with intrees[J].International Journal of Data Science and Analytics,2019,7(4):277-287.
[40]DONG L,YE X,YANG G.Two-stage rule extraction methodbased on tree ensemble model for interpretable loan evaluation[J].Information Sciences,2021,573:46-64.
[41]FRIEDMAN J H,POPESCU B E.Predictive learning via ruleensembles[J].The Annals of Applied Statistics,2008,2(3):916-954.
[42]DUMITRESCU E,SULLIVAN H,HURLIN C,et al.Machine Learning or Econometrics for Credit Scoring:Let's Get the Best of Both Worlds[J].Working Papers,2021.
[43]KATO H,HANADA H,TAKEUCHI I.Safe rulefit:Learning optimal sparse rule model by meta safe screening[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2023,45(2):2330-2343.
[44]LEI X N,LIN L F,XIAO B Q,et al.Re-exploration of small and micro enterprises' default characteristics based on machine learning models with SHAP[J].China Journal of Management Science,2024,32(5):1-12.
[45]LIU X Y,QU Y W,ZHOU Q Y.Self-attention credit evaluation model[J].Computer Engineering and Applications,2019,55(13):36-41.
[46]ZHAO X F,WU D L,WU W W,et al.BM-Linear credit loanevaluation model based on multi-head attention mechanism[J].Journal of Systems & Management,2023,32(1):118.
[47]ZHANG M Q,ZHOU H,CAO J G.Directed sentiment textclassification based on attention mechanism and dual BERT[J].CAAI Transactions on Intelligent Systems,2022,17(6):1220-1227.
[48]FAWCETT T.An introduction to ROC analysis[J].PatternRecognition Letters,2006,27:861-874.
[49]VERBRAKEN T,BRAVO C,WEBER R,et al.Developmentand application of consumer credit scoring models using profit-based classification measures[J].European Journal of Operational Research,2014,238:505-513.
[50]QIAN X,CAI H H,INNAB N,et al.A novel deep learning approach to enhance creditworthiness evaluation and ethical lending practices in the economy [J].Annals of Operations Research,2025,346:1597-1619.
[51]YANG F,ABEDIN M Z,HAJEK P.An explainable federated learning and blockchain-based secure credit modeling method [J].European Journal of Operational Research,2024,317(2):449-467.
[52]XIA Y,JIANG S,MENG L,et al.XGBoost-B-GHM:An Ensemble Model with Feature Selection and GHM Loss Function Optimization for Credit Scoring [J].Systems,2024,12(7):254.
[53]TRINH L T.A comparative analysis of consumer credit riskmodels in Peer-to-Peer Lending [J].Journal of Economics,Finance and Administrative Science,2024,29(58):346-365

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed