计算机科学 ›› 2024, Vol. 51 ›› Issue (2): 245-251.doi: 10.11896/jsjkx.230300028

• 人工智能 • 上一篇    下一篇


周晟昊, 袁伟伟, 关东海   

  1. 南京航空航天大学计算机科学与技术学院 南京211100
  • 收稿日期:2023-03-03 修回日期:2023-06-25 出版日期:2024-02-15 发布日期:2024-02-22
  • 通讯作者: 袁伟伟(yuanweiwei@nuaa.edu.cn)
  • 作者简介:(cenhelm@nuaa.edu.cn)
  • 基金资助:

Local Interpretable Model-agnostic Explanations Based on Active Learning and Rational Quadratic Kernel

ZHOU Shenghao, YUAN Weiwei, GUAN Donghai   

  1. College of Computer Science and Technology,Nanjing University of Aeronautics and Astronautics,Nanjing 211100,China
  • Received:2023-03-03 Revised:2023-06-25 Online:2024-02-15 Published:2024-02-22
  • About author:ZHOU Shenghao,born in 1999,master.His main research interests include data mining and machine learninginterpre-tability.YUAN Weiwei,born in 1981,Ph.D,professor.Her main research interests include data mining and intelligence computing.
  • Supported by:
    National Defense Basic Scientific Research program of China(JCKY2020204C009).

摘要: 深度学习模型的广泛使用,在更大程度上使人们意识到模型的决策是亟需解决的问题,复杂难以解释的黑盒模型阻碍了算法在实际场景中部署。LIME作为最流行的局部解释方法,生成的扰动数据却具有不稳定性,导致最终的解释产生偏差。针对上述问题,提出了一种基于主动学习和二次有理核的模型无关局部解释方法ActiveLIME,使得局部解释模型更加忠于原始分类器。ActiveLIME生成扰动数据后,通过主动学习的查询策略对扰动数据进行采样,筛选不确定性高的扰动集训练,使用迭代过程中准确度最高的局部模型对感兴趣实例生成解释。并且,针对容易陷入局部过拟合的高维稀疏样本,在模型损失函数中引入了二次有理核来减少过拟合。实验结果表明,所提出的ActiveLIME方法引比传统局部解释方法具有更高的局部保真度和解释质量。

关键词: 局部解释, 扰动采样, 主动学习查询策略, 二次有理核

Abstract: With the widespread use of deep learning models,people are more aware that the decision-making of model is a problem that needs to be solved urgently.Complex and difficult-to-interpret black-box models hinder the deployment of algorithms in actual scenarios.LIME is the most popular method of local interpretation,but the resulting perturbed data is unstable,leading to bias in the final explanation.To solve the above problems,local interpretable model-agnostic explanations based on active learning and rational quadratic kernel,ActiveLIME,is proposed,which makes the local interpretable model more faithful to the original classifier.After ActiveLIME generates the perturbed data,it samples the perturbation through the query strategy of active lear-ning,selects the perturbation with high uncertainty for training,and uses the local model with the highest accuracy in the iteration to generate explanations for the instances of interest.And for high-dimensional sparse samples that are prone to local overfitting,a rational quadratic kernel is introduced into model’s loss function to reduce overfitting.Experiments indicate that the proposed ActiveLIME has better local fidelity and quality of explanations than traditional local explanation algorithms.

Key words: Local explanation, Perturbation sampling, Query strategy of active learning, Rational quadratic kernel


  • TP391
[1]BAI X,WANG X,LIU X,et al.Explainable deep learning for efficient and robust pattern recognition:A survey of recent deve-lopments[J].Pattern Recognition,2021,120:108102.
[2]BREIMAN L.Classification and regression trees[M].Rout-ledge,2017.
[3]LINARDATOS P,PAPASTEFANOPOULOS V,KOTSIAN-TIS S.Explainable ai:A review of machine learning interpre-tability methods[J].Entropy,2020,23(1):18.
[4]DU M,LIU N,HU X.Techniques for interpretable machine learning[J].Communications of the ACM,2019,63(1):68-77.
[5]WANG F,KAUSHAL R,KHULLAR D.Should health care demand interpretable artificial intelligence or accept “black box” medicine?[J].Annals of Internal Medicine,2020,172(1):59-60.
[6]SHANKARANARAYANA S M,RUNJE D.ALIME:Autoencoder based approach for local interpretability[C]//Intelligent Data Engineering and Automated Learning-IDEAL 2019:20th International Conference,Manchester,UK,November 14-16,2019,Proceedings,Part I 20.Springer International Publishing,2019:454-463.
[7]RIBEIRO M T,SINGH S,GUESTRIN C.“Why should I trustyou?” Explaining the predictions of any classifier[C]//Procee-dings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.2016:1135-1144.
[8]LAKKARAJU H,BACH S H,LESKOVEC J.Interpretable decision sets:A joint framework for description and prediction[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.2016:1675-1684.
[9]SETTLES B.Active learning literature survey[R].ComputerSciences Technical Report 1648,University of Wisconsin-Madison,2009.
[10]MUSLEA I,MINTON S,KNOBLOCK C A.Active learningwith multiple views[J].Journal of Artificial Intelligence Research,2006,27:203-233.
[11]SEUNG H S,OPPER M,SOMPOLINSKY H.Query by committee[C]//Proceedings of the fifth Annual Workshop on Computational Learning Theory.1992:287-294.
[12]ZAFAR M R,KHAN N.Deterministic local interpretable mo-del-agnostic explanations for stable explainability[J].Machine Learning and Knowledge Extraction,2021,3(3):525-541.
[13]RANJBAR N,SAFABAKHSH R.Using decision tree as local interpretable model in autoencoder-based lime[C]//2022 27th International Computer Conference.Computer Society of Iran(CSICC),IEEE,2022:1-7.
[14]LAUGEL T,RENARD X,LESOT M J,et al.Defining locality for surrogates in post-hoc interpretablity[J].arXiv:1806.07498,2018.
[15]RIBEIRO M T,SINGH S,GUESTRIN C.Anchors:High-precision model-agnostic explanations[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2018.
[16]RIBEIRO M T,SINGH S,GUESTRIN C.Nothing else mat-ters:Model-agnostic explanations by identifying prediction invariance[J].arXiv:1611.05817,2016.
[17]ZHAO X,HUANG W,HUANG X,et al.Baylime:Bayesian local interpretable model-agnostic explanations[C]//Uncertainty in Artificial Intelligence(PMLR).2021:887-896.
[18]ADLER P,FALK C,FRIEDLER S A,et al.Auditing black-box models for indirect influence[J].Knowledge and Information Systems,2018,54:95-122.
[19]BRAMHALL S,HORN H,TIEU M,et al.Qlime-a quadratic local interpretable model-agnostic explanation approach[J].SMU Data Science Review,2020,3(1):4.
[20]HOFMANN T,SCHÖLKOPF B,SMOLA A J.Kernel methods in machine learning[J].The Annals of Statistics,2008,36(3):1171.
[21]JANOCHA K,CZARNECKI W M.On loss functions for deep neural networks in classification[J].arXiv:1702.05659,2017.
[22]GUO G,WANG H,BELL D,et al.KNN model-based approach in classification[C]//OTM Confederated International Confe-rences on The Move to Meaningful Internet Systems 2003:CoopIS,DOA,and ODBASE.2003:986-996.
[23]NIELSEN F,NIELSEN F.Hierarchical clustering[J/OL].Introduction to HPC with MPI for Data Science,2016:195-211.https://link.springer.com/chapter/10.1007/978-3-319-21903-5_8.
[24]VILONE G,LONGO L.Notions of explainability and evaluation approaches for explainable artificial intelligence[J].Information Fusion,2021,76:89-106.
[25]GRANDINI M,BAGLI E,VISANI G.Metrics for multi-class classification:an overview[J].arXiv:2008.05756,2020.
[26]JIA Y,BAILEY J,RAMAMOHANARAO K,et al.Improving the quality of explanations with local embedding perturbations[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.2019:875-884.
[27]KEERTHI S S,LIN C J.Asymptotic behaviors of support vector machines with Gaussian kernel[J].Neural Computation,2003,15(7):1667-1689.
Full text



No Suggested Reading articles found!