Computer Science ›› 2025, Vol. 52 ›› Issue (6A): 240500140-7.doi: 10.11896/jsjkx.240500140

• Image Processing & Multimedia Technology • Previous Articles     Next Articles

Study on interpretable Shallow Class Activation Mapping Algorithm Based on Spatial Weights andInter Layer Correlation

CHENG Yan1, HE Huijuan2, CHEN Yanying2, YAO Nannan2, LIN Guobo2   

  1. 1 school of software,Jiangxi Normal University,Nanchang 330022,China
    2 School of Digital industry,Jiangxi Normal University,Shangrao,Jiangxi 334006,China
  • Online:2025-06-16 Published:2025-06-12
  • About author:CHEN Yan,born in 1976,Ph.D,professor,Ph.D supervisor.Hermain research interests include artificial intelligence and deep learning,intelligent information,etc.
  • Supported by:
    National Natural Science Foundation of China(62167006),Jiangxi Province Science and Technology Innovation Base Plan-Key Laboratory of Intelligent Information Processing and Emotional Computing(2024SSY03131),Leading Talents Program for Academic and Technical Leaders in Major Disciplines in Jiangxi Province(20213BCJL22047) and Jiangxi Province 03 Special Project and 5G Project(20212ABC03A22).

Abstract: Convolutional neural networks play an important role in the field of computer vision,but their black box nature makes it difficult for people to understand the reasons for their decisions,seriously hindering their application in certain security areas.Traditional class activation mapping(CAM) algorithms are often limited by the interpretability of deep neurons,resulting in weaker interpretability of shallow neurons and the presence of significant noise.To address this challenge,we propose an interpretable shallow class activation mapping algorithm that can generate fine-grained explanations.This algorithm is based on the theory of correlation propagation,considering the correlation between adjacent layers,obtaining inter layer correlation weights,and using the feature map with spatial weights as a mask,multiplying it with inter layer correlation weights to achieve shallow interpretation.Experimental results show that compared with LayerCAM,which explains the shallow layer best,the proposed algorithm improves the comprehensive score of deletion and insertion tests for the class activation maps generated by each layer of the con-volutional neural network by a maximum of 2.73 and a minimum of 0.24 on the ILSVRC2012 val dataset,and a maximum of 1.31 and a minimum of 0.38 on the CUB-200-2011 dataset.

Key words: Class activation mapping algorithm, Convolutional neural network, Shallow neurons, Spatial weight, Interlayer correlation

CLC Number: 

  • TP183
[1]CHENG M M,JIANG P T,HAN L H,et al.Deeply Explain CNN via Hierarchical Decomposition[J].arXiv:2201.09205,2022.
[2]SUN H,SHI Y L,WANG R.Research on Class ActivationMapping Algorithm from Coarse to Fine Based on Comparative Hierarchical Correlation Propagation [J].Journal of Electronics and Information Science,2023,45(4):1454-1463.
[3]ZEILER M D,FERGUS R.Visualizing and understanding convo-lutional networks[C]//13th European Conference on ComputerVision.Zurich,Switzerland,2014:818-833.
[4]PETSIUK V,DAS A,SAENKO K.Rise:Randomized inputsampling for explanation of black-box models[C]//British Machine Vision Conference(BMVC).2018.
[5]AGARWAL C,SCHONFELD D,NGUYEN A.Removing input features via a generative model to explain their attributions to classifier’s decisions[J].arXiv:1910.04256,2019.
[6]CHANG C H,CREAGER E,GOLDENBERG A,et al.Explaining image classifiers by counterfactual generation[C]//Proceedings of the 7th International Conference on Learning Representations.New Orleans,USA,2019.
[7]SI N W,ZHANG W L,QU D,et al.A Review of Convolutional Neural Network Representation Visualization Research [J].Journal of Automation,2022,48(8):1890-1920.
[8]BAEHRENS D,SCHROETER T,HARMELING S,Kawanabe M,Hansen K,Müller K R.How to explain individual classification decisions.[J] Journal of Machine Learning Research,2010,11(61):1803-1831.
[9]SIMONYAN K,VEDALDI A,ZISSERMAN A.Deep insideconvolutional networks:Visualising image classification models and saliency maps[J].arXiv:1312.6034,2013.
[10]CHENG L,FANG P,LIANG Y,et al.TSGB:Target-Selective Gradient Backprop for Probing CNN Visual Saliency[J].IEEE transactions on image processing:a publication of the IEEE Signal Processing Society,2022,31:2529-2540.
[11]GU J,YANG Y,TRESP V.Understanding individual decisions of cnns via contrastive backpropagation[C]//Proceedings of the 14th Asian Conference on Computer Vision.Perth,Australia,2018:119-134.
[12]BACH S.Layer-Wise Relevance Propagation for Deep NeuralNetwork Architectures[C]//ICISA.Singapore:Springer,2016:913-922.
[13]ZHOU B L,KHOSLA A,LAPEDRIZA A,et al.Learning deep features for discriminative localization[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016.
[14]SELVARAJU R R,COGSWELL M,DAS A,et al.Grad-cam:Visual explanations from deep networks via gradient-based localization[C]//Proceedings of the IEEE International Confe-rence on Computer Vision.2017.
[15]CHATTOPADHAY A,SARKAR A,HOWLADERP,et al.Grad-cam++:Generalized gradient-based visual explanations for deep convolutional networks[C]//2018 IEEE Winter Conference on Applications of Computer Vision(WACV).IEEE,2018:839-847.
[16]RAMASWAMY H G.Ablation-cam:Visual explanations fordeep convolutional network via gradient-free localization[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.2020:983-991.
[17]SMILKOVD,THORAT N,KIM B,et al.SmoothGrad:removing noise by adding noise[J].arXiv:1706.03825,2017.
[18]SATTARZADEH S,SUDHAKAR M,PLATANIOTISK N,et al.Integrated grad-cam:Sensitivity-aware visual explanation of deep convolutional networks via integrated gradient-based scoring[C]//2021 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP 2021).IEEE,2021:1775-1779.
[19]LUCAS M,LERMA M,FURST J,et al.RSI-Grad-CAM:Visual explanations from deep networks via Riemann-Stieltjes integratedgradient-based localization[C]//International Symposium on Visual Computing.Cham:Springer International Publishing,2022:262-274.
[20]FU R,HU Q,DONG X,et al.Axiom-based Grad-CAM:Towards Accurate Visualization and Explanation of CNNs(BMVC2020 Oral)[J].arXiv:2008.02312,2020.
[21]WANG H F,WANG Z F,DU M N,et al.Score-cam:Score-weighted visual explanations for convolutional neural networks[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,Workshop on Fair,Data Efficient and Trusted Computer Vision.2020.
[22]ZHANG Q,RAO L,YANG Y.Group-cam:Group score-weighted visual explanations for deep convolutional networks[J].ar-Xiv:2103.13859,2021.
[23]FENG Z,JI H,DAKOVIC M,et al.Cluster-CAM:Cluster-Weighted Visual Interpretation of CNNs’ Decision in Image Classification[J].arXiv:2302.01642,2023.
[24]JIANG P T,ZHANG C B,HOU Q,et al.LayerCAM:Exploring Hierarchical Class Activation Maps for Localization[J].IEEE Transactions on Image Processing,2021,30:5875-5888.
[25]LEE J R,KIM S,PARK I,EO T,et al.Relevance-CAM:Your Model Already Knows Where to Look[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR 2021).Nashville,TN,USA,2021:14939-14948.
[26]GU J,YANG Y,TRESP V.Understanding individual decisions of cnns via contrastive backpropagation[C]//Proceedings of the 14th Asian Conference on Computer Vision.Perth,Australia.2018:119-134.
[27]RUSSAKOVSKY O,DENG J,SU H,et al.Imagenet large scale visual recognition challenge[J].International Journal of Computer Vision,2015,115(3):211-252.
[28]ADEBAYO J,GILMER J,MUELLY M,et al.Sanity checks for saliency maps[C]//Advances in Neural Information Processing Systems.2018:9505-9515.
[1] SHI Xincheng, WANG Baohui, YU Litao, DU Hui. Study on Segmentation Algorithm of Lower Limb Bone Anatomical Structure Based on 3D CTImages [J]. Computer Science, 2025, 52(6A): 240500119-9.
[2] LONG Xiao, HUANG Wei, HU Kai. Bi-MI ViT:Bi-directional Multi-level Interaction Vision Transformer for Lung CT ImageClassification [J]. Computer Science, 2025, 52(6A): 240700183-6.
[3] WANG Jiamin, WU Wenhong, NIU Hengmao, SHI Bao, WU Nier, HAO Xu, ZHANG Chao, FU Rongsheng. Review of Concrete Defect Detection Methods Based on Deep Learning [J]. Computer Science, 2025, 52(6A): 240900137-12.
[4] WANG Baohui, GAO Zhan, XU Lin, TAN Yingjie. Research and Implementation of Mine Gas Concentration Prediction Algorithm Based on Deep Learning [J]. Computer Science, 2025, 52(6A): 240400188-7.
[5] GUO Yecai, HU Xiaowei, MAO Xiangnan. Multi-scale Feature Fusion Residual Denoising Network Based on Cascade [J]. Computer Science, 2025, 52(6): 239-246.
[6] WANG Chenyuan, ZHANG Yanmei, YUAN Guan. Class Integration Test Order Generation Approach Fused with Deep Reinforcement Learning andGraph Convolutional Neural Network [J]. Computer Science, 2025, 52(6): 58-65.
[7] WEI Xiaohui, GUAN Zeyu, WANG Chenyang, YUE Hengshan, WU Qi. Hardware-Software Co-design Fault-tolerant Strategies for Systolic Array Accelerators [J]. Computer Science, 2025, 52(5): 91-100.
[8] PANG Mingyi, WEI Xianglin, ZHANG Yunxiang, WANG Bin, ZHUANG Jianjun. Efficient Adaptive CNN Accelerator for Resource-limited Chips [J]. Computer Science, 2025, 52(4): 94-100.
[9] XIONG Qibing, MIAO Qiguang, YANG Tian, YUAN Benzheng, FEI Yangyang. Malicious Code Detection Method Based on Hybrid Quantum Convolutional Neural Network [J]. Computer Science, 2025, 52(3): 385-390.
[10] LIU Hui, JI Ke, CHEN Zhenxiang, SUN Runyuan, MA Kun, WU Jun. Malicious Attack Detection in Recommendation Systems Combining Graph Convolutional Neural Networks and Ensemble Methods [J]. Computer Science, 2024, 51(6A): 230700003-9.
[11] HUANG Rui, XU Ji. Text Classification Based on Invariant Graph Convolutional Neural Networks [J]. Computer Science, 2024, 51(6A): 230900018-5.
[12] SUN Yang, DING Jianwei, ZHANG Qi, WEI Huiwen, TIAN Bowen. Study on Super-resolution Image Reconstruction Using Residual Feature Aggregation NetworkBased on Attention Mechanism [J]. Computer Science, 2024, 51(6A): 230600039-6.
[13] YUAN Zhen, LIU Jinfeng. Denoising Autoencoders Based on Lossy Compress Coding [J]. Computer Science, 2024, 51(6A): 230400172-7.
[14] DAI Yongdong, JIN Yang, DAI Yufan, FU Jing, WANG Maofei, LIU Xi. Study on Intelligent Defect Recognition Algorithm of Aerial Insulator Image [J]. Computer Science, 2024, 51(6A): 230700172-5.
[15] LYU Yiming, WANG Jiyang. Iron Ore Image Classification Method Based on Improved Efficientnetv2 [J]. Computer Science, 2024, 51(6A): 230600212-6.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!