计算机科学

计算机科学 ›› 2025, Vol. 52 ›› Issue (6A): 240500140-7.doi: 10.11896/jsjkx.240500140

• 图像处理&多媒体技术 • 上一篇    下一篇

基于空间权重和层间相关性的可解释浅层类激活映射算法研究

程艳1, 何慧娟2, 陈彦滢2, 姚楠楠2, 林国波2   

  1. 1 江西师范大学软件学院 南昌 330022
    2 江西师范大学数字产业学院 江西 上饶 334006
  • 出版日期:2025-06-16 发布日期:2025-06-12
  • 通讯作者: 程艳(chyan88888@jxnu.edu.cn)
  • 基金资助:
    国家自然科学基金(62167006);江西省科技创新基地计划——智能信息处理与情感计算江西省重点实验室(2024SSY03131);江西省主要学科学术和技术带头人培养计划领军人才项目(20213BCJL22047);江西省03专项及5G项目(20212ABC03A22)

Study on interpretable Shallow Class Activation Mapping Algorithm Based on Spatial Weights andInter Layer Correlation

CHENG Yan1, HE Huijuan2, CHEN Yanying2, YAO Nannan2, LIN Guobo2   

  1. 1 school of software,Jiangxi Normal University,Nanchang 330022,China
    2 School of Digital industry,Jiangxi Normal University,Shangrao,Jiangxi 334006,China
  • Online:2025-06-16 Published:2025-06-12
  • About author:CHEN Yan,born in 1976,Ph.D,professor,Ph.D supervisor.Hermain research interests include artificial intelligence and deep learning,intelligent information,etc.
  • Supported by:
    National Natural Science Foundation of China(62167006),Jiangxi Province Science and Technology Innovation Base Plan－Key Laboratory of Intelligent Information Processing and Emotional Computing(2024SSY03131),Leading Talents Program for Academic and Technical Leaders in Major Disciplines in Jiangxi Province(20213BCJL22047) and Jiangxi Province 03 Special Project and 5G Project(20212ABC03A22).

PDF (PC)

40

摘要： 卷积神经网络在计算机视觉领域具有重要作用,然而其黑盒特性使人们理解其决策理由变得困难,严重阻碍了其在某些安全领域的应用。传统的类激活映射(Class Activation Mapping,CAM)算法通常受限于深层神经元的可解释性,对浅层神经元的解释能力较弱且存在较多噪声。为了应对这一挑战,提出一种可解释浅层的类激活映射算法,并生成细粒度的解释。该算法基于相关性传播理论,考虑相邻层之间的相关性,得到层间相关性权重,并将应用了空间权重的特征图作为掩码,与层间相关性权重相乘,从而实现浅层解释。实验结果表明,所提算法与解释浅层最优的LayerCAM相比,卷积神经网络每层生成的类激活图的删除插入测试综合评分在ILSVRC2012 val数据集上最高提高了2.73,最低提高了0.24,在CUB-200-2011数据集上最高提高了1.31,最低提高了0.38。

关键词: 类激活映射算法, 卷积神经网络, 浅层神经元, 空间权重, 层间相关性

Abstract: Convolutional neural networks play an important role in the field of computer vision,but their black box nature makes it difficult for people to understand the reasons for their decisions,seriously hindering their application in certain security areas.Traditional class activation mapping(CAM) algorithms are often limited by the interpretability of deep neurons,resulting in weaker interpretability of shallow neurons and the presence of significant noise.To address this challenge,we propose an interpretable shallow class activation mapping algorithm that can generate fine-grained explanations.This algorithm is based on the theory of correlation propagation,considering the correlation between adjacent layers,obtaining inter layer correlation weights,and using the feature map with spatial weights as a mask,multiplying it with inter layer correlation weights to achieve shallow interpretation.Experimental results show that compared with LayerCAM,which explains the shallow layer best,the proposed algorithm improves the comprehensive score of deletion and insertion tests for the class activation maps generated by each layer of the con-volutional neural network by a maximum of 2.73 and a minimum of 0.24 on the ILSVRC2012 val dataset,and a maximum of 1.31 and a minimum of 0.38 on the CUB-200-2011 dataset.

Key words: Class activation mapping algorithm, Convolutional neural network, Shallow neurons, Spatial weight, Interlayer correlation

中图分类号: 

  • TP183
[1]CHENG M M,JIANG P T,HAN L H,et al.Deeply Explain CNN via Hierarchical Decomposition[J].arXiv:2201.09205,2022.
[2]SUN H,SHI Y L,WANG R.Research on Class ActivationMapping Algorithm from Coarse to Fine Based on Comparative Hierarchical Correlation Propagation [J].Journal of Electronics and Information Science,2023,45(4):1454-1463.
[3]ZEILER M D,FERGUS R.Visualizing and understanding convo-lutional networks[C]//13th European Conference on ComputerVision.Zurich,Switzerland,2014:818-833.
[4]PETSIUK V,DAS A,SAENKO K.Rise:Randomized inputsampling for explanation of black-box models[C]//British Machine Vision Conference(BMVC).2018.
[5]AGARWAL C,SCHONFELD D,NGUYEN A.Removing input features via a generative model to explain their attributions to classifier’s decisions[J].arXiv:1910.04256,2019.
[6]CHANG C H,CREAGER E,GOLDENBERG A,et al.Explaining image classifiers by counterfactual generation[C]//Proceedings of the 7th International Conference on Learning Representations.New Orleans,USA,2019.
[7]SI N W,ZHANG W L,QU D,et al.A Review of Convolutional Neural Network Representation Visualization Research [J].Journal of Automation,2022,48(8):1890-1920.
[8]BAEHRENS D,SCHROETER T,HARMELING S,Kawanabe M,Hansen K,Müller K R.How to explain individual classification decisions.[J] Journal of Machine Learning Research,2010,11(61):1803-1831.
[9]SIMONYAN K,VEDALDI A,ZISSERMAN A.Deep insideconvolutional networks:Visualising image classification models and saliency maps[J].arXiv:1312.6034,2013.
[10]CHENG L,FANG P,LIANG Y,et al.TSGB:Target-Selective Gradient Backprop for Probing CNN Visual Saliency[J].IEEE transactions on image processing:a publication of the IEEE Signal Processing Society,2022,31:2529-2540.
[11]GU J,YANG Y,TRESP V.Understanding individual decisions of cnns via contrastive backpropagation[C]//Proceedings of the 14th Asian Conference on Computer Vision.Perth,Australia,2018:119-134.
[12]BACH S.Layer-Wise Relevance Propagation for Deep NeuralNetwork Architectures[C]//ICISA.Singapore:Springer,2016:913-922.
[13]ZHOU B L,KHOSLA A,LAPEDRIZA A,et al.Learning deep features for discriminative localization[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016.
[14]SELVARAJU R R,COGSWELL M,DAS A,et al.Grad-cam:Visual explanations from deep networks via gradient-based localization[C]//Proceedings of the IEEE International Confe-rence on Computer Vision.2017.
[15]CHATTOPADHAY A,SARKAR A,HOWLADERP,et al.Grad-cam++:Generalized gradient-based visual explanations for deep convolutional networks[C]//2018 IEEE Winter Conference on Applications of Computer Vision(WACV).IEEE,2018:839-847.
[16]RAMASWAMY H G.Ablation-cam:Visual explanations fordeep convolutional network via gradient-free localization[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.2020:983-991.
[17]SMILKOVD,THORAT N,KIM B,et al.SmoothGrad:removing noise by adding noise[J].arXiv:1706.03825,2017.
[18]SATTARZADEH S,SUDHAKAR M,PLATANIOTISK N,et al.Integrated grad-cam:Sensitivity-aware visual explanation of deep convolutional networks via integrated gradient-based scoring[C]//2021 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP 2021).IEEE,2021:1775-1779.
[19]LUCAS M,LERMA M,FURST J,et al.RSI-Grad-CAM:Visual explanations from deep networks via Riemann-Stieltjes integratedgradient-based localization[C]//International Symposium on Visual Computing.Cham:Springer International Publishing,2022:262-274.
[20]FU R,HU Q,DONG X,et al.Axiom-based Grad-CAM:Towards Accurate Visualization and Explanation of CNNs(BMVC2020 Oral)[J].arXiv:2008.02312,2020.
[21]WANG H F,WANG Z F,DU M N,et al.Score-cam:Score-weighted visual explanations for convolutional neural networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,Workshop on Fair,Data Efficient and Trusted Computer Vision.2020.
[22]ZHANG Q,RAO L,YANG Y.Group-cam:Group score-weighted visual explanations for deep convolutional networks[J].ar-Xiv:2103.13859,2021.
[23]FENG Z,JI H,DAKOVIC M,et al.Cluster-CAM:Cluster-Weighted Visual Interpretation of CNNs’ Decision in Image Classification[J].arXiv:2302.01642,2023.
[24]JIANG P T,ZHANG C B,HOU Q,et al.LayerCAM:Exploring Hierarchical Class Activation Maps for Localization[J].IEEE Transactions on Image Processing,2021,30:5875-5888.
[25]LEE J R,KIM S,PARK I,EO T,et al.Relevance-CAM:Your Model Already Knows Where to Look[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR 2021).Nashville,TN,USA,2021:14939-14948.
[26]GU J,YANG Y,TRESP V.Understanding individual decisions of cnns via contrastive backpropagation[C]//Proceedings of the 14th Asian Conference on Computer Vision.Perth,Australia.2018:119-134.
[27]RUSSAKOVSKY O,DENG J,SU H,et al.Imagenet large scale visual recognition challenge[J].International Journal of Computer Vision,2015,115(3):211-252.
[28]ADEBAYO J,GILMER J,MUELLY M,et al.Sanity checks for saliency maps[C]//Advances in Neural Information Processing Systems.2018:9505-9515.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发