Computer Science

Computer Science ›› 2023, Vol. 50 ›› Issue (6A): 220600212-7.doi: 10.11896/jsjkx.220600212

• Artificial Intelligence • Previous Articles     Next Articles

Explainability of Artificial Intelligence:Development and Application

WANG Dongli1, YANG Shan1, OUYANG Wanli2, LI Baopu3, ZHOU Yan1   

  1. 1 School of Automation and Electronics Information,Xiangtan University,Xiangtan,Hunan 411105,China;
    2 School of Electrical and Information Engineering,The University of Sydney,Sydney 2006,Australia;
    3 Baidu Research(USA),Sunnyvale,CA 94086,USA
  • Online:2023-06-10 Published:2023-06-12
  • About author:WANG Dongli,Ph.D,associate professor.Her main research interests include pattern recognition and distributed learning,intelligent decision-making and information processing. ZHOU Yan,Ph.D,professor.His main research interests include machine vision and cluster robotics,signal processing and information fusion.
  • Supported by:
    National Key Research and Development Program of China(2020YFA0713503),National Natural Science Foundation of China(61773330),Aeronautical Science Foundation of China(20200020114004) and Science and Technology Innovation Program of Hunan Province,China(2020GK2036).

PDF (PC)

522

Abstract: In recent years,the extensive application of artificial intelligence in many fields and disciplines has shown its excellent performance.The improvement of this performance usually needs to sacrifice the transparency of the model.However,the complexity and black box nature of artificial intelligence models have become the main bottleneck in its application in high-risk fields,which seriously hinders the further application of artificial intelligence in specific fields.Therefore,it is urgent to improve the interpretability of the model to prove its reliability.Therefore,this paper introduces the typical models and methods of AI interpretability research from three aspects:machine learning model interpretability,deep learning model interpretability,and hybrid model interpretability,further describes the application of interpretable AI in teaching analysis,judicial judgment,and medical diagnosis,and summarizes and analyzes the shortcomings of existing interpretable methods,puts forward the development trend of the future research direction of AI interpretability,and hope to further promote the development and application of interpretability research.

Key words: Artificial intelligence, Machine learning, Deep learning, Interpretability

CLC Number: 

  • TP391
[1]CHAO L M,YIN X L.AI Governance and System:Current Si-tuation and Trend[J].Computer Science,2021,48(9):1-8.
[2]HUA Y Y,ZHANG D C,GE S M.Research Progress in the Interpretability of Deep Learning Models[J].Journal of Cyber Security,2020,5(3):1-12.
[3]KONG X W,TANG X Z,WANG Z M.A Survey of Explainable Artificial Intelligence Decision[J].Systems Engineering-Theory &Practice,2021,41(2):524-53.
[4]ZENG C Y,YANK,WANG Z F,et al.Survey of Interpretability Research on Deep Learning Models[J].Computer Engineering and Applications,2021,57(8):1-9.
[5]ALAIN G,BENGIO Y.Understanding intermediate layers using linear classifier probes[J].arXiv:1610.01644,2016.
[6]WANG C,SHI Y,FAN X,et al.Attribute Reduction Based on K-nearest Neighborhood Rough Sets[J].International Journal of Approximate Reasoning,2019,106:18-31.
[7]ZHENG S,DING C.A Group Lasso Based Sparse KNN Classifier[J].Pattern Recognition Letters,2020,131:227-233.
[8]ZHOU Z J,CAO Y,HU C H,et al.The Interpretability of Rule-based Modeling Approach and Its Development[J].Acta Automatica Sinica,2021,47(6):1201-1216.
[9]RIBEIRO M T,SINGH S,GUESTRIN C." Why should i trust you?" Explaining the Predictions of Any Classifier[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.2016:1135-1144.
[10]GUO W B,XU J.Using Lemna to Explain the Application ofDeep Learning in Network Security(Part I)[J].China Education Network,2019(z1):40-43.
[11]SETZU M,GUIDOTTI R,MONREALE A,et al.Glocalx-from Local to Global Explanations of Black Box AI Models[J].Artificial Intelligence,2021,294:103457.
[12]HINTON G,VINYALS O,DEAN J.Distilling the Knowledgein A Neural Network[J].Computer Science,2015,14(7):38-39.
[13]ZHAO L,PENG X,CHEN Y,et al.Knowledge as Priors:Cross-modal Knowledge Generalization for Datasets without Superior Knowledge[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:6528-6537.
[14]SPRINGENBERG J T,DOSOVITSKIY A,BROX T,et al.Striving for simplicity:The all convolutional net[C]//Proceedings of 3rd ICLR(Workshop Track).2015.
[15]SUNDARARAJAN M,TALY A,YAN Q.Axiomatic Attribution for Deep Networks[C]//International Conference on Machine Learning.PMLR,2017:3319-3328.
[16]SMILKOV D,THORAT N,KIM B,et al.Smoothgrad:removing noise by adding noise[C]//ICML Workshop on Visualization for Deep Learning.2017.
[17]ZHOU B,KHOSLA A,LAPEDRIZA A,et al.Learning Deep Features for Discriminative Localization[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:2921-2929.
[18]SELVARAJU R R, COGSWELL M, DAS A,et al.Grad-CAM:Visual Explanations from Deep Networks via Gradient-based Localization[J].International Journal of Computer Vision,2020,128(2):336-359.
[19]CHATTOPADHAY A,SARKAR A,HOWLADER P,et al.Grad-cam++:Generalized gradient-based visual explanations for deep convolutional networks[C]//IEEE Winter Conference on Applications of Computer Vision.2018:839-847.
[20]BINDER A,MONTAVON G,LAPUSCHKIN S,et al.Layer-wise relevance propagation for neural networks with local renormalization layers[C]//International Conference on Artificial Neural Networks.2016:63-71.
[21]SHRIKUMAR A,GREENSIDE P,KUNDAJE A.Learning important features through propagating activation differences[C]//International Conference on Machine Learning.PMLR,2017:3145-3153.
[22]DU M,LIU N,SONG Q,et al.Towards explanation of dnn-based prediction with guided feature inversion[C]//Proceedings of the 24th ACM SIGKDD International Conference on Know-ledge Discovery & Data Mining.2018:1358-1367.
[23]ELSHAWI R,SHERIFY,AL-MALLAH M,et al.ILIME:Local and global interpretable model-agnostic explainer of black-box decision[C]//European Conference on Advances in Databases and Information Systems.Cham:Springer,2019:53-68.
[24]LIN Z,FENG M,SANTOS C N D,et al.A structured self-attentive sentence embedding[C]//Proceedings of the 5th International Conference on Learning Representations.Toulon,France,2017:1-15.
[25]GALASSI A,LIPPI M,TORRONI P.Attention,please! a critical review of neural attention models in natural language processing[J].arXiv:1902.02181,2019.
[26]DEVLIN J,CHANG M W,LEE K,et al.Bert:Pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2019:4171-4186.
[27]HU J,SHEN L,ALBANIE S,et al.Squeeze-and-excitation networks[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2019,42(8):2011-2023.
[28]DOSOVITSKIY A,BEYER L,KOLESNIKOV A,et al.Animage is worth 16x16 words:Transformers for image recognition at scale[C]//Proceedings of the 9th International Confe-rence on Learning Representations.2021.
[29]LUNDBERG S M,LEE S I.A unified approach to interpreting model predictions[J].Advances in Neural Information Proces-sing Systems,2017,30.
[30]LUNDBERG S M,ERION G,CHEN H,et al.From local explanations to global understanding with explainable AI for trees[J].Nature Machine Intelligence,2020,2(1):56-67.
[31]HU Z T,MA X Z,LIU Z Z,et al.Harnessing deep neural networks with logic rules[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics(Vo-lume 1:Long Papers).2016:2410-2420.
[32]WANG T,LIN Q.Hybrid predictive models:when an interpretable model collaborates with a black-box model[J].Journal of Machine Learning Research,2021,22(137):1-38.
[33]YEGANEJOU M,DICK S,MILLER J.Interpretable deep convo-lutional fuzzy classifier[J].IEEE Transactions on Fuzzy Systems,2019,28(7):1407-1419.
[34]WANG S X.AI Empowerment Education[J]China education network,2021(1):15.
[35]HASIB K M,RAHMAN F,HASNAT R,et al.A machinelearning and explainable AI approach for predicting secondary school student performance[C]//2022 IEEE 12thAnnual Computing and Communication Workshop and Conference(CCWC).2022:399-405.
[36]LI J,ZHANG G,YU L,et al.Research and design on cognitive computing framework for predicting judicial decisions[J].Journal of Signal Processing Systems,2019,91(10):1159-1167.
[37]BAO Q,ZAN H,GONG P,et al.Charge prediction with legal attention[C]//CCF International Conference on Natural Language Processing and Chinese Computing.Cham:Springer,2019:447-458.
[38]SOARES E,ANGELOV P,BIASO S,et al.SARS-CoV-2 CT-scan dataset:A large dataset of real patients CT scans for SARS-CoV-2 identification[J].MedRxiv,2020.
[39]COUTEAUX V,NEMPONT O,PIZAINE G,et al.Towards interpretability of segmentation networks by analyzing Deep-Dreams[M]//Interpretability of machine intelligencein medical image computing and multimodal learning for clinical decision support.Cham:Springer,2019:56-63.
[40]BIEN J,TIBSHIRANI R.Prototype selection for interpretableclassification[J].The Annals of Applied Statistics,2011,5(4):2403-2424.
[1] LIU Xiang, ZHU Jing, ZHONG Guoqiang, GU Yongjian, CUI Liyuan. Quantum Prototype Clustering [J]. Computer Science, 2023, 50(8): 27-36.
[2] ZHANG Yian, YANG Ying, REN Gang, WANG Gang. Study on Multimodal Online Reviews Helpfulness Prediction Based on Attention Mechanism [J]. Computer Science, 2023, 50(8): 37-44.
[3] SONG Xinyang, YAN Zhiyuan, SUN Muyi, DAI Linlin, LI Qi, SUN Zhenan. Review of Talking Face Generation [J]. Computer Science, 2023, 50(8): 68-78.
[4] WANG Xu, WU Yanxia, ZHANG Xue, HONG Ruize, LI Guangsheng. Survey of Rotating Object Detection Research in Computer Vision [J]. Computer Science, 2023, 50(8): 79-92.
[5] ZHOU Ziyi, XIONG Hailing. Image Captioning Optimization Strategy Based on Deep Learning [J]. Computer Science, 2023, 50(8): 99-110.
[6] ZHANG Xiao, DONG Hongbin. Lightweight Multi-view Stereo Integrating Coarse Cost Volume and Bilateral Grid [J]. Computer Science, 2023, 50(8): 125-132.
[7] WANG Yu, WANG Zuchao, PAN Rui. Survey of DGA Domain Name Detection Based on Character Feature [J]. Computer Science, 2023, 50(8): 251-259.
[8] LI Yang, LI Zhenhua, XIN Xianlong. Attack Economics Based Fraud Detection for MVNO [J]. Computer Science, 2023, 50(8): 260-270.
[9] ZHOU Fengfan, LING Hefei, ZHANG Jinyuan, XIA Ziwei, SHI Yuxuan, LI Ping. Facial Physical Adversarial Example Performance Prediction Algorithm Based on Multi-modal Feature Fusion [J]. Computer Science, 2023, 50(8): 280-285.
[10] ZHU Boyu, CHEN Xiao, SHA Letian, XIAO Fu. Two-layer IoT Device Classification Recognition Model Based on Traffic and Text Fingerprints [J]. Computer Science, 2023, 50(8): 304-313.
[11] LU Xingyuan, CHEN Jingwei, FENG Yong, WU Wenyuan. Privacy-preserving Data Classification Protocol Based on Homomorphic Encryption [J]. Computer Science, 2023, 50(8): 321-332.
[12] WANG Mingxia, XIONG Yun. Disease Diagnosis Prediction Algorithm Based on Contrastive Learning [J]. Computer Science, 2023, 50(7): 46-52.
[13] SHEN Zhehui, WANG Kailai, KONG Xiangjie. Exploring Station Spatio-Temporal Mobility Pattern:A Short and Long-term Traffic Prediction Framework [J]. Computer Science, 2023, 50(7): 98-106.
[14] HUO Weile, JING Tao, REN Shuang. Review of 3D Object Detection for Autonomous Driving [J]. Computer Science, 2023, 50(7): 107-118.
[15] ZHOU Bo, JIANG Peifeng, DUAN Chang, LUO Yuetong. Study on Single Background Object Detection Oriented Improved-RetinaNet Model and Its Application [J]. Computer Science, 2023, 50(7): 137-142.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.