Computer Science ›› 2022, Vol. 49 ›› Issue (5): 64-70.doi: 10.11896/jsjkx.210400176

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Interpretability Optimization Method Based on Mutual Transfer of Local Attention Map

CHENG Ke-yang, WANG Ning, CUI Hong-gang, ZHAN Yong-zhao   

  1. School of Computer Science and Communication Engineering,Jiangsu University,Zhenjiang,Jiangsu 212013,China
  • Received:2021-04-18 Revised:2021-09-07 Online:2022-05-15 Published:2022-05-06
  • About author:CHENG Ke-yang,born in 1982,Ph.D,professor,is a member of China Computer Federation.His main research interests include computer vision and machine learning.
  • Supported by:
    National Natural Science Foundation of China(61972183,61672268) and Director Foundation Project of National Engineering Laboratory for Public Safety Risk Perception and Control by Big Data.

Abstract: At present,deep learning models have been widely deployed in various industrial fields.However,the complexity and inexplicability of deep learning model have become the main bottleneck of its application in high-risk fields.The most important method is the visual interpretation,in which the attention map is the main representation of the visual interpretation method.The decision area in the sample image can be marked to visually display the decision basis of the model.In the existing visual interpretation methods based on attention map,the single model attention map has the problem of insufficient confidence of visualization interpretability due to the annotation error easily appearing in the annotated region.To solve the above problems,this paper proposes an interpretable optimization method based on the mutual transfer of local attention map,aiming at improving the annotation accuracy of the model attention map and displaying the precise decision area,so as to strengthen the visual interpretable ability for the model decision basis.Specifically,the structure of the intermigration network is constructed by using the lightweight model,the feature maps are extracted and superimposed between the layers of the single model,and the global attention map is divided locally.Pearson correlation coefficient is used to measure the similarity of the corresponding local attention map between the models,and then the local attention map is regularized and transferred combined with the cross-entropy function..Experimental results show that the proposed algorithm significantly improves the accuracy of the model attention map label accuracy.The proposed algorithm achieves an average drop rate of 28.2% and an average increase rate of 29.5%,respectively,and achieves an increase of 3.3% in the average decline rate compared with the most advanced algorithm.The above experiments show that the proposed algorithm can successfully find out the most responsive region in the sample image,rather than being limited to the vi-sual visualization region.Compared with the existing similar methods,the proposed method can more accurately reveal the decision basis of the original CNN model.

Key words: Attention diagram, Interpretability, Mutual transfer, Regional division, Similarity

CLC Number: 

  • TP391.4
[1]YUAN X X,WU Q.Object detection in remote sensing images based on saliency feature and angle information[J].Computer Science,2021,48(4):174-179.
[2]YADAV A,VISHWAKARMA D K.Sentiment analysis usingdeep learning architectures:a review[J].Artificial Intelligence Review,2020,53(6):4335-4385.
[3]ZEILER M D,FERGUS R.Visualizing and understanding convolutional networks[C]//Proceedings of European Conference on Computer Vision.Berlin:Springer,2014:818-833.
[4]GAUR M,FALDUK,SHETH A.Semantics of the black-box:can knowledge graphs help make deep learning systems more interpretable and explainable?[J].IEEE Internet Computing,2021,25(1):51-59.
[5]DENIL M,BAZZANI L,LAROCHELLE H,et al.Learningwhere to attend with deep architectures for image tracking[J].Neural Computation,2012,24(8):2151-2184.
[6]SELVARAJU R R,COGSWELL M,DAS A,et al.Grad-cam:Visual explanations from deep networks via gradient-based localization[C]//Proceedings of the IEEE International Conference on Computer Vision.2017:618-626.
[7]ZHOU B,KHOSLA A,LAPEDRIZA A,et al.Learning deep features for discriminative localization[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:2921-2929.
[8]ZENG T,ACUNA D E.Modeling citation worthiness by using attention-based bidirectional long short-term memory networks and interpretable models[J].Scientometrics,2020,5(1):1-30.
[9]HINTON G,VINYALS O,DEAN J.Distilling the knowledge in a neural network[J].Computer Science,2015,14(7):38-39.
[10]YIM J,JOO D,BAE J,et al.A gift from knowledge distillation:fast optimization,network minimization and transfer learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:4133-4141.
[11]HSU E Y,LIU C L,TSENG V S.Multivariate time series early classification with interpretability using deep learning and attention mechanism[M].Advances in Knowledge Discovery and Data Mining.2019.
[12]LIU Q,MUKHOPADHYAY S.Unsupervised learning usingpretrained CNN and associative memory bank[C]//2018 International Joint Conference on Neural Networks (IJCNN).IEEE,2018:1-8.
[13]PETSIUK V,DAS A,SAENKO K.Rise:randomized input sampling for explanation of black-box models[EB/OL].(2018-09-25)[2018-09-25].https://arxiv.org/abs/1806.07421.
[14]SELVARAJU R R,COGSWELL M,DAS A,et al.Grad-CAM:visual explanations from deep networks via gradient-based loca-lization[J].International Journal of Computer Vision,2020,128(2):336-359.
[15]CHATTOPADHYAY A,SARKAR A,HOWLADER P,et al.Grad-cam++:Generalized gradient-based visual explanations for deep convolutional networks[C]//2018 IEEE Winter Conference on Applications of Computer Vision (WACV).IEEE,2018:839-847.
[16]WANG H,WANG Z,DU M,et al.Score-CAM:Score-weighted visual explanations for convolutional neural networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops.2020:24-25.
[17]ZHANG Y,XIANG T,HOSPEDALES T M,et al.Deep mutual learning[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:4320-4328.
[18]LIU Q,MUKHOPADHYAY S.Unsupervised learning usingpretrained CNN and associative memory bank[C]//2018 Interational Joint Conference on Neural Networks (IJCNN).IEEE,2018:1-8.
[19]CHEN W C,CHANG C C,LEE C R.Knowledge distillationwith feature maps for image classification[C]//Asian Conference on Computer Vision.Cham:Springer,2018:200-215.
[1] WU Zi-yi, LI Shao-mei, JIANG Meng-han, ZHANG Jian-peng. Ontology Alignment Method Based on Self-attention [J]. Computer Science, 2022, 49(9): 215-220.
[2] WANG Ming, WU Wen-fang, WANG Da-ling, FENG Shi, ZHANG Yi-fei. Generative Link Tree:A Counterfactual Explanation Generation Approach with High Data Fidelity [J]. Computer Science, 2022, 49(9): 33-40.
[3] CHAI Hui-min, ZHANG Yong, FANG Min. Aerial Target Grouping Method Based on Feature Similarity Clustering [J]. Computer Science, 2022, 49(9): 70-75.
[4] ZHENG Wen-ping, LIU Mei-lin, YANG Gui. Community Detection Algorithm Based on Node Stability and Neighbor Similarity [J]. Computer Science, 2022, 49(9): 83-91.
[5] LI Bin, WAN Yuan. Unsupervised Multi-view Feature Selection Based on Similarity Matrix Learning and Matrix Alignment [J]. Computer Science, 2022, 49(8): 86-96.
[6] ZENG Zhi-xian, CAO Jian-jun, WENG Nian-feng, JIANG Guo-quan, XU Bin. Fine-grained Semantic Association Video-Text Cross-modal Entity Resolution Based on Attention Mechanism [J]. Computer Science, 2022, 49(7): 106-112.
[7] CAI Xiao-juan, TAN Wen-an. Improved Collaborative Filtering Algorithm Combining Similarity and Trust [J]. Computer Science, 2022, 49(6A): 238-241.
[8] WANG Yi, LI Zheng-hao, CHEN Xing. Recommendation of Android Application Services via User Scenarios [J]. Computer Science, 2022, 49(6A): 267-271.
[9] ZHAO Lu, YUAN Li-ming, HAO Kun. Review of Multi-instance Learning Algorithms [J]. Computer Science, 2022, 49(6A): 93-99.
[10] HUANG Shao-bin, SUN Xue-wei, LI Rong-sheng. Relation Classification Method Based on Cross-sentence Contextual Information for Neural Network [J]. Computer Science, 2022, 49(6A): 119-124.
[11] CHEN Zhuang, ZOU Hai-tao, ZHENG Shang, YU Hua-long, GAO Shang. Diversity Recommendation Algorithm Based on User Coverage and Rating Differences [J]. Computer Science, 2022, 49(5): 159-164.
[12] YANG Xu-hua, WANG Lei, YE Lei, ZHANG Duan, ZHOU Yan-bo, LONG Hai-xia. Complex Network Community Detection Algorithm Based on Node Similarity and Network Embedding [J]. Computer Science, 2022, 49(3): 121-128.
[13] YU Dong, XIE Wan-ying, GU Shu-hao, FENG Yang. Similarity-based Curriculum Learning for Multilingual Neural Machine Translation [J]. Computer Science, 2022, 49(1): 24-30.
[14] LUO Yue-tong, WANG Tao, YANG Meng-nan, ZHANG Yan-kong. Historical Driving Track Set Based Visual Vehicle Behavior Analytic Method [J]. Computer Science, 2021, 48(9): 86-94.
[15] ZHENG Jian-wei, HUANG Juan-juan, QIN Meng-jie, XU Hong-hui, LIU Zhi. Hyperspectral Image Denoising Based on Non-local Similarity and Weighted-truncated NuclearNorm [J]. Computer Science, 2021, 48(9): 160-167.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!