计算机科学

计算机科学 ›› 2023, Vol. 50 ›› Issue (6A): 220600212-7.doi: 10.11896/jsjkx.220600212

• 人工智能 • 上一篇    下一篇

人工智能可解释性:发展与应用

王冬丽1, 杨珊1, 欧阳万里2, 李抱朴3, 周彦1   

  1. 1 湘潭大学自动化与电子信息学院 湖南 湘潭 411105;
    2 悉尼大学电气与信息工程学院 悉尼 2006;
    3 百度美国研究院 森尼韦尔 94086
  • 出版日期:2023-06-10 发布日期:2023-06-12
  • 通讯作者: 周彦(yanzhou@xtu.edu.cn)
  • 作者简介:(wangdl@xtu.edu.cn)
  • 基金资助:
    国家重点研发计划项目(2020YFA0713503);国家自然科学基金项目(61773330);国家航空科学基金项目(20200020114004);湖南省科技创新计划项目(2020GK2036)

Explainability of Artificial Intelligence:Development and Application

WANG Dongli1, YANG Shan1, OUYANG Wanli2, LI Baopu3, ZHOU Yan1   

  1. 1 School of Automation and Electronics Information,Xiangtan University,Xiangtan,Hunan 411105,China;
    2 School of Electrical and Information Engineering,The University of Sydney,Sydney 2006,Australia;
    3 Baidu Research(USA),Sunnyvale,CA 94086,USA
  • Online:2023-06-10 Published:2023-06-12
  • About author:WANG Dongli,Ph.D,associate professor.Her main research interests include pattern recognition and distributed learning,intelligent decision-making and information processing. ZHOU Yan,Ph.D,professor.His main research interests include machine vision and cluster robotics,signal processing and information fusion.
  • Supported by:
    National Key Research and Development Program of China(2020YFA0713503),National Natural Science Foundation of China(61773330),Aeronautical Science Foundation of China(20200020114004) and Science and Technology Innovation Program of Hunan Province,China(2020GK2036).

PDF (PC)

1137

摘要： 近年来人工智能在诸多领域和学科中的广泛应用展现出了其卓越的性能,这种性能的提升通常需要牺牲模型的透明度来获取。然而,人工智能模型的复杂性和黑盒性质已成为其应用于高风险领域最主要的瓶颈,这严重阻碍了人工智能在特定领域的进一步应用。因此,亟需提高模型的可解释性,以证明其可靠性。为此,从机器学习模型可解释性、深度学习模型可解释性、混合模型可解释性3个方面对人工智能可解释性研究的典型模型和方法进行了介绍,进一步讲述了可解释人工智能在教学分析、司法判案、医疗诊断3个领域的应用情况,并对现有可解释方法存在的不足进行总结与分析,提出人工智能可解释性未来的发展趋势,希望进一步推动可解释性研究的发展与应用。

关键词: 人工智能, 机器学习, 深度学习, 可解释性

Abstract: In recent years,the extensive application of artificial intelligence in many fields and disciplines has shown its excellent performance.The improvement of this performance usually needs to sacrifice the transparency of the model.However,the complexity and black box nature of artificial intelligence models have become the main bottleneck in its application in high-risk fields,which seriously hinders the further application of artificial intelligence in specific fields.Therefore,it is urgent to improve the interpretability of the model to prove its reliability.Therefore,this paper introduces the typical models and methods of AI interpretability research from three aspects:machine learning model interpretability,deep learning model interpretability,and hybrid model interpretability,further describes the application of interpretable AI in teaching analysis,judicial judgment,and medical diagnosis,and summarizes and analyzes the shortcomings of existing interpretable methods,puts forward the development trend of the future research direction of AI interpretability,and hope to further promote the development and application of interpretability research.

Key words: Artificial intelligence, Machine learning, Deep learning, Interpretability

中图分类号: 

  • TP391
[1]CHAO L M,YIN X L.AI Governance and System:Current Si-tuation and Trend[J].Computer Science,2021,48(9):1-8.
[2]HUA Y Y,ZHANG D C,GE S M.Research Progress in the Interpretability of Deep Learning Models[J].Journal of Cyber Security,2020,5(3):1-12.
[3]KONG X W,TANG X Z,WANG Z M.A Survey of Explainable Artificial Intelligence Decision[J].Systems Engineering-Theory &Practice,2021,41(2):524-53.
[4]ZENG C Y,YANK,WANG Z F,et al.Survey of Interpretability Research on Deep Learning Models[J].Computer Engineering and Applications,2021,57(8):1-9.
[5]ALAIN G,BENGIO Y.Understanding intermediate layers using linear classifier probes[J].arXiv:1610.01644,2016.
[6]WANG C,SHI Y,FAN X,et al.Attribute Reduction Based on K-nearest Neighborhood Rough Sets[J].International Journal of Approximate Reasoning,2019,106:18-31.
[7]ZHENG S,DING C.A Group Lasso Based Sparse KNN Classifier[J].Pattern Recognition Letters,2020,131:227-233.
[8]ZHOU Z J,CAO Y,HU C H,et al.The Interpretability of Rule-based Modeling Approach and Its Development[J].Acta Automatica Sinica,2021,47(6):1201-1216.
[9]RIBEIRO M T,SINGH S,GUESTRIN C." Why should i trust you?" Explaining the Predictions of Any Classifier[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.2016:1135-1144.
[10]GUO W B,XU J.Using Lemna to Explain the Application ofDeep Learning in Network Security(Part I)[J].China Education Network,2019(z1):40-43.
[11]SETZU M,GUIDOTTI R,MONREALE A,et al.Glocalx-from Local to Global Explanations of Black Box AI Models[J].Artificial Intelligence,2021,294:103457.
[12]HINTON G,VINYALS O,DEAN J.Distilling the Knowledgein A Neural Network[J].Computer Science,2015,14(7):38-39.
[13]ZHAO L,PENG X,CHEN Y,et al.Knowledge as Priors:Cross-modal Knowledge Generalization for Datasets without Superior Knowledge[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:6528-6537.
[14]SPRINGENBERG J T,DOSOVITSKIY A,BROX T,et al.Striving for simplicity:The all convolutional net[C]//Proceedings of 3rd ICLR(Workshop Track).2015.
[15]SUNDARARAJAN M,TALY A,YAN Q.Axiomatic Attribution for Deep Networks[C]//International Conference on Machine Learning.PMLR,2017:3319-3328.
[16]SMILKOV D,THORAT N,KIM B,et al.Smoothgrad:removing noise by adding noise[C]//ICML Workshop on Visualization for Deep Learning.2017.
[17]ZHOU B,KHOSLA A,LAPEDRIZA A,et al.Learning Deep Features for Discriminative Localization[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:2921-2929.
[18]SELVARAJU R R, COGSWELL M, DAS A,et al.Grad-CAM:Visual Explanations from Deep Networks via Gradient-based Localization[J].International Journal of Computer Vision,2020,128(2):336-359.
[19]CHATTOPADHAY A,SARKAR A,HOWLADER P,et al.Grad-cam++:Generalized gradient-based visual explanations for deep convolutional networks[C]//IEEE Winter Conference on Applications of Computer Vision.2018:839-847.
[20]BINDER A,MONTAVON G,LAPUSCHKIN S,et al.Layer-wise relevance propagation for neural networks with local renormalization layers[C]//International Conference on Artificial Neural Networks.2016:63-71.
[21]SHRIKUMAR A,GREENSIDE P,KUNDAJE A.Learning important features through propagating activation differences[C]//International Conference on Machine Learning.PMLR,2017:3145-3153.
[22]DU M,LIU N,SONG Q,et al.Towards explanation of dnn-based prediction with guided feature inversion[C]//Proceedings of the 24th ACM SIGKDD International Conference on Know-ledge Discovery & Data Mining.2018:1358-1367.
[23]ELSHAWI R,SHERIFY,AL-MALLAH M,et al.ILIME:Local and global interpretable model-agnostic explainer of black-box decision[C]//European Conference on Advances in Databases and Information Systems.Cham:Springer,2019:53-68.
[24]LIN Z,FENG M,SANTOS C N D,et al.A structured self-attentive sentence embedding[C]//Proceedings of the 5th International Conference on Learning Representations.Toulon,France,2017:1-15.
[25]GALASSI A,LIPPI M,TORRONI P.Attention,please! a critical review of neural attention models in natural language processing[J].arXiv:1902.02181,2019.
[26]DEVLIN J,CHANG M W,LEE K,et al.Bert:Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2019:4171-4186.
[27]HU J,SHEN L,ALBANIE S,et al.Squeeze-and-excitation networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2019,42(8):2011-2023.
[28]DOSOVITSKIY A,BEYER L,KOLESNIKOV A,et al.Animage is worth 16x16 words:Transformers for image recognition at scale[C]//Proceedings of the 9th International Confe-rence on Learning Representations.2021.
[29]LUNDBERG S M,LEE S I.A unified approach to interpreting model predictions[J].Advances in Neural Information Proces-sing Systems,2017,30.
[30]LUNDBERG S M,ERION G,CHEN H,et al.From local explanations to global understanding with explainable AI for trees[J].Nature Machine Intelligence,2020,2(1):56-67.
[31]HU Z T,MA X Z,LIU Z Z,et al.Harnessing deep neural networks with logic rules[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics(Vo-lume 1:Long Papers).2016:2410-2420.
[32]WANG T,LIN Q.Hybrid predictive models:when an interpretable model collaborates with a black-box model[J].Journal of Machine Learning Research,2021,22(137):1-38.
[33]YEGANEJOU M,DICK S,MILLER J.Interpretable deep convo-lutional fuzzy classifier[J].IEEE Transactions on Fuzzy Systems,2019,28(7):1407-1419.
[34]WANG S X.AI Empowerment Education[J]China education network,2021(1):15.
[35]HASIB K M,RAHMAN F,HASNAT R,et al.A machinelearning and explainable AI approach for predicting secondary school student performance[C]//2022 IEEE 12thAnnual Computing and Communication Workshop and Conference(CCWC).2022:399-405.
[36]LI J,ZHANG G,YU L,et al.Research and design on cognitive computing framework for predicting judicial decisions[J].Journal of Signal Processing Systems,2019,91(10):1159-1167.
[37]BAO Q,ZAN H,GONG P,et al.Charge prediction with legal attention[C]//CCF International Conference on Natural Language Processing and Chinese Computing.Cham:Springer,2019:447-458.
[38]SOARES E,ANGELOV P,BIASO S,et al.SARS-CoV-2 CT-scan dataset:A large dataset of real patients CT scans for SARS-CoV-2 identification[J].MedRxiv,2020.
[39]COUTEAUX V,NEMPONT O,PIZAINE G,et al.Towards interpretability of segmentation networks by analyzing Deep-Dreams[M]//Interpretability of machine intelligencein medical image computing and multimodal learning for clinical decision support.Cham:Springer,2019:56-63.
[40]BIEN J,TIBSHIRANI R.Prototype selection for interpretableclassification[J].The Annals of Applied Statistics,2011,5(4):2403-2424.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发