Computer Science ›› 2025, Vol. 52 ›› Issue (10): 50-59.doi: 10.11896/jsjkx.250300059

• Digital Intelligence Enabling FinTech Frontiers • Previous Articles     Next Articles

Interpretable Credit Risk Assessment Model:Rule Extraction Approach Based on AttentionMechanism

WANG Baocai, WU Guowei   

  1. School of Software Technology,Dalian University of Technology,Dalian,Liaoning 116000,China
  • Received:2025-03-11 Revised:2025-05-19 Online:2025-10-15 Published:2025-10-14
  • About author:WANG Baocai,born in 1988,postgra-duate.His main research interests include machine learning interpretability and intelligent credit risk control systems.
    WU Guowei,born in 1973,Ph.D,professor,Ph.D supervisor,new century out-standing talents of Ministry of Education,Executive member of CCF System Software Special Committee.His main research interests include advanced computing and intelligent systems.

Abstract: Due to the limitations of traditional statistical models for credit risk assessment,machine learning techniques have significantly enhanced model accuracy and predictive capabilities.However,the complexity and opacity pose significant challenges in terms of interpretation.To address this issue,this paper introduces an interpretable machine learning model for credit risk assessment that integrates the attention mechanism with tree ensemble rule extraction approach.This model automatically identifies complex nonlinear relationships within the data,extracts a large number of interpretable rules from the trained tree ensemble model,encodes these rules into new feature variables,and inputs them into an attention neural network to obtain attention weights for each rule.Subsequently,based on the attention weights,objective function,and constraints,the model balances the predictive performance,stability,and interpretability of the rule subset.The optimal rule subset can be derived in O(n) time.Experimental results,based on three public datasets,demonstrate that the proposed approach not only maintains high predictive accuracy but also substantially enhances the model's interpretability.

Key words: Interpretable machine learning,Credit risk assessment,Attention mechanism,Rule generation algorithm,Tree ensemble models

CLC Number: 

  • TP391
[1]BAESENS B.Using neural network rule extraction and decision tables for credit-risk evaluation[J].Management Science,2003,49(3):312-329.
[2]SERRANO-CINCA C,GUTIERREZ-NIETO B.Partial LeastSquare Discriminant Analysis for bankruptcy prediction[J].Decision Support Systems,2013,54(3):1245-1255.
[3]MIYAMOTO M,MIYAMOTO M.Credit risk assessment for a small bank by using a multinomial logistic regression model[J].International Journal of Finance and Accounting,2014,3(5):327-334.
[4]COSTA E SILVA E,LOPES C,CORREIA A,et al.A logistic regression model for consumer default risk[J].Journal of Applied Statistics,2020,47:1-17.
[5]BEQUÉ A,COUSSEMENT K,GAYLER R,et al.Approachesfor credit scorecard calibration:an empirical analysis[J].Know-ledge-Based Systems,2017,134:213-227.
[6]LI T,WANG H,WU J,et al.Sparse Bayesian learning for credit risk evaluation[J].Journal of Computer Applications,2013,33(11):4.
[7]BHATTACHARYA A,WILSON S P,SOYER R.A Bayesianapproach to modeling mortgage default and prepayment[J].European Journal of Operational Research,2019,274(3):1112-1124.
[8]MELNYK K V,BORYSOVA N V.Improving the quality ofcredit activity by using scoring model[J].Radio Electronics Computer Science Control,2019(2):60-70.
[9]DAMRONGSAKMETHEE T,NEAGOE V E.Principal Component Analysis and ReliefF Cascaded with Decision Tree for Credit Scoring[M]//Artificial Intelligence Methods in Intelligent Algorithms.Cham:Springer,2019.
[10]CHERN C C,LEI W U,HUANG K L,et al.A decision treeclassifier for credit assessment problems in big data environments[J].Information Systems and e-Business Management,2021,19(1):363-386.
[11]GOH R,LEE L S.Credit Scoring:A Review on Support Vector Machines and Metaheuristic Approaches[J].Advances in Operations Research,2019,2019:1-30.
[12]LEE I G,YOON S W,WON D.A Mixed Integer Linear Programming Support Vector Machine for Cost-Effective Group Feature Selection:Branch-Cut-and-Price Approach[J].European Journal of Operational Research,2022,299(3):1055-1068.
[13]SHEN F,YANG Z,ZHAO X,et al.Reject inference in credit scoring using a three-way decision and safe semi-supervised support vector machine[J].Information Sciences,2022,606:614-627.
[14]WANG A Q,HAN Z C,WANG Y L.Risk assessment of logistics finance enterprises based on BP neural network and fuzzy mathematical model[J].Journal of Intelligent & Fuzzy Systems,2020,39:5915-5925.
[15]FRAISSE H,LAPORTE M.Return on investment on artificial intelligence:The case of bank capital requirement[J].Journal of Banking & Finance,2022,138:106401.
[16]KELLNER R,NAGL M,ROSCH D.Opening the black box-Quantile neural networks for loss given default prediction[J].Journal of Banking & Finance,2022,134:106334.
[17]BHATORE S,MOHAN L,REDDY Y R.Machine learningtechniques for credit risk evaluation:a systematic literature review[J].Journal of Banking and Financial Technology,2020,4(1):111-138.
[18]DASTILE X,CELIK T,POTSANE M.Statistical and machine learning models in credit scoring:A systematic literature survey[J].Applied Soft Computing,2020,91:106263.
[19]LENKA S R,BISOY S K,PRIYADARSHINI R,et al.Empirical Analysis of Ensemble Learning for Imbalanced Credit Scoring Datasets:A Systematic Review[J].Wireless Communications and Mobile Computing,2022,2022:6584352.
[20]HOFMAN J M,SHARMA A,WATTS D J.Prediction and explanation in social systems[J].Science,2017,355(6324):486-488.
[21]CHEN D X,YE J H,YE W C.Interpretable selective learning in credit risk[J].Research in International Business and Finance,2023,65:101940.
[22]DAVIS R,LO A W,MISHRA S,et al.Explainable Machine Learning Models of Consumer Credit Risk[J].Journal of Financial Data Science,2023,5(4).
[23]DUVNJAK M,MERĆEP A,KOSTANJČAR Z.Intrinsically Interpretable Models for Credit Risk Assessment[C]//2024 47th MIPRO ICT and Electronics Convention.IEEE,2024:31-36.
[24]Equal Credit Opportunity Act[S].United States Code,title 15,chapter 41,subchapter IV,1974.
[25]HOOFNAGLE C J,VAN DER SLOOT B,ZUIDERVEENBORGESIUS F.The European Union general data protection regulation:what it is and what it means[J].Information & Communications Technology Law,2019,28(1):65-98.
[26]MASHAYEKHI M,GRAS R.Rule extraction from decisiontrees ensembles:new algorithms based on heuristic search and sparse group lasso methods[J].International Journal of Information Technology & Decision Making,2017,16(6):1707-1727.
[27]HADDOUCHI M,BERRADO A.A survey and taxonomy ofmethods interpreting random forest models [J].arXiv:2407.12759,2024.
[28]MARTENS D,BAESENS B,GESTEL T V,et al.Comprehensible credit scoring models using rule extraction from support vector machines[J].European Journal of Operational Research,2007,183(3):1466-1476.
[29]HADDOUCHI M,BERRADO A.Forest-ORE:Mining an optimal rule ensemble to interpret random forest models[J].Engineering Applications of Artificial Intelligence,2025,143:109997.
[30]BIRBIL S I,EDALI M,YUCEOGLU B.Rule Covering for Interpretation and Boosting[J].Information Fusion,2020,63:196-207.
[31]MANZALI Y,ELFAR M.Optimizing the number of branches in a decision forest using association rule metrics[J].Knowledge and Information Systems,2024,66(6):3261-3281.
[32]BORUAH A N,BISWAS S K,BANDYOPADHYAY S.Transparent rule generator random forest(TRG-RF):an interpretable random forest[J].Evolving Systems,2023,14(1):69-83.
[33]BOLOGNA G.A rule extraction technique applied to ensembles of neural networks,random forests,and gradient-boosted trees[J].Algorithms,2021,14(12):339.
[34]EDALI M.Performance analysis of set partitioning formulations on the rule extraction from random forests[J].Pamukkale University Journal of Engineering Sciences,2021,27(4):513-519.
[35]CHEN M,HUO J,DUAN Y.An interpretable model for sepsis prediction using multi-objective rule extraction[J].Journal of Intelligent Information Systems,2024,62(5):1403-1429.
[36]SHAMS Z,DIMANOV B,KOLA S,et al.REM:An Integrative Rule Extraction Methodology for Explainable Data Analysis in Healthcare[R].medRxiv,2021.
[37]WANG S,WANG Y,WANG D,et al.An improved random forest-based rule extraction method for breast cancer diagnosis[J].Applied Soft Computing Journal,2020,86:105941.
[38]MASHAYEKHI M,GRAS R.Rule extraction from random forest:the RF+HC methods[M]//Advances in Artificial Intelligence.Cham:Springer,2015:223-237.
[39]DENG H.Interpreting tree ensembles with intrees[J].International Journal of Data Science and Analytics,2019,7(4):277-287.
[40]DONG L,YE X,YANG G.Two-stage rule extraction methodbased on tree ensemble model for interpretable loan evaluation[J].Information Sciences,2021,573:46-64.
[41]FRIEDMAN J H,POPESCU B E.Predictive learning via ruleensembles[J].The Annals of Applied Statistics,2008,2(3):916-954.
[42]DUMITRESCU E,SULLIVAN H,HURLIN C,et al.Machine Learning or Econometrics for Credit Scoring:Let's Get the Best of Both Worlds[J].Working Papers,2021.
[43]KATO H,HANADA H,TAKEUCHI I.Safe rulefit:Learning optimal sparse rule model by meta safe screening[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2023,45(2):2330-2343.
[44]LEI X N,LIN L F,XIAO B Q,et al.Re-exploration of small and micro enterprises' default characteristics based on machine learning models with SHAP[J].China Journal of Management Science,2024,32(5):1-12.
[45]LIU X Y,QU Y W,ZHOU Q Y.Self-attention credit evaluation model[J].Computer Engineering and Applications,2019,55(13):36-41.
[46]ZHAO X F,WU D L,WU W W,et al.BM-Linear credit loanevaluation model based on multi-head attention mechanism[J].Journal of Systems & Management,2023,32(1):118.
[47]ZHANG M Q,ZHOU H,CAO J G.Directed sentiment textclassification based on attention mechanism and dual BERT[J].CAAI Transactions on Intelligent Systems,2022,17(6):1220-1227.
[48]FAWCETT T.An introduction to ROC analysis[J].PatternRecognition Letters,2006,27:861-874.
[49]VERBRAKEN T,BRAVO C,WEBER R,et al.Developmentand application of consumer credit scoring models using profit-based classification measures[J].European Journal of Operational Research,2014,238:505-513.
[50]QIAN X,CAI H H,INNAB N,et al.A novel deep learning approach to enhance creditworthiness evaluation and ethical lending practices in the economy [J].Annals of Operations Research,2025,346:1597-1619.
[51]YANG F,ABEDIN M Z,HAJEK P.An explainable federated learning and blockchain-based secure credit modeling method [J].European Journal of Operational Research,2024,317(2):449-467.
[52]XIA Y,JIANG S,MENG L,et al.XGBoost-B-GHM:An Ensemble Model with Feature Selection and GHM Loss Function Optimization for Credit Scoring [J].Systems,2024,12(7):254.
[53]TRINH L T.A comparative analysis of consumer credit riskmodels in Peer-to-Peer Lending [J].Journal of Economics,Finance and Administrative Science,2024,29(58):346-365
[1] ZHENG Hanyuan, GE Rongjun, HE Shengji, LI Nan. Direct PET to CT Attenuation Correction Algorithm Based on Imaging Slice Continuity [J]. Computer Science, 2025, 52(10): 115-122.
[2] XU Hengyu, CHEN Kun, XU Lin, SUN Mingzhai, LU Zhou. SAM-Retina:Arteriovenous Segmentation in Dual-modal Retinal Image Based on SAM [J]. Computer Science, 2025, 52(10): 123-133.
[3] WEN Jing, ZHANG Songsong, LI Xufeng. Target Tracking Method Based on Cross Scale Fusion of Features and Trajectory Prompts [J]. Computer Science, 2025, 52(10): 144-150.
[4] SHENG Xiaomeng, ZHAO Junli, WANG Guodong, WANG Yang. Immediate Generation Algorithm of High-fidelity Head Avatars Based on NeRF [J]. Computer Science, 2025, 52(10): 159-167.
[5] ZHENG Dichen, HE Jikai, LIU Yi, GAO Fan, ZHANG Dengyin. Low Light Image Adaptive Enhancement Algorithm Based on Retinex Theory [J]. Computer Science, 2025, 52(10): 168-175.
[6] RUAN Ning, LI Chun, MA Haoyue, JIA Yi, LI Tao. Review of Quantum-inspired Metaheuristic Algorithms and Its Applications [J]. Computer Science, 2025, 52(10): 190-200.
[7] XIONG Zhuozhi, GU Zhouhong, FENG Hongwei, XIAO Yanghua. Subject Knowledge Evaluation Method for Language Models Based on Multiple ChoiceQuestions [J]. Computer Science, 2025, 52(10): 201-207.
[8] WANG Jian, WANG Jingling, ZHANG Ge, WANG Zhangquan, GUO Shiyuan, YU Guiming. Multimodal Information Extraction Fusion Method Based on Dempster-Shafer Theory [J]. Computer Science, 2025, 52(10): 208-216.
[9] CHEN Yuyan, JIA Jiyuan, CHANG Jingwen, ZUO Kaiwen, XIAO Yanghua. SPEAKSMART:Evaluating Empathetic Persuasive Responses by Large Language Models [J]. Computer Science, 2025, 52(10): 217-230.
[10] LI Sihui, CAI Guoyong, JIANG Hang, WEN Yimin. Novel Discrete Diffusion Text Generation Model with Convex Loss Function [J]. Computer Science, 2025, 52(10): 231-238.
[11] ZHANG Jiawei, WANG Zhongqing, CHEN Jiali. Multi-grained Sentiment Analysis of Comments Based on Text Generation [J]. Computer Science, 2025, 52(10): 239-246.
[12] CHEN Jiahao, DUAN Liguo, CHANG Xuanwei, LI Aiping, CUI Juanjuan, HAO Yuanbin. Text Sentiment Classification Method Based on Large-batch Adversarial Strategy and EnhancedFeature Extraction [J]. Computer Science, 2025, 52(10): 247-257.
[13] WANG Ye, WANG Zhongqing. Text Simplification for Aspect-based Sentiment Analysis Based on Large Language Model [J]. Computer Science, 2025, 52(10): 258-265.
[14] ZHAO Jinshuang, HUANG Degen. Summary Faithfulness Evaluation Based on Data Augmentation and Two-stage Training [J]. Computer Science, 2025, 52(10): 266-274.
[15] SUN Liangxu, LI Linlin, LIU Guoli. Sub-problem Effectiveness Guided Multi-objective Evolution Algorithm [J]. Computer Science, 2025, 52(10): 296-307.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!