计算机科学 ›› 2021, Vol. 48 ›› Issue (6): 227-233.doi: 10.11896/jsjkx.200800016
李杉1,2, 许新征1,2,3
LI Shan1,2, XU Xin-zheng1,2,3
摘要： 近年来,对卷积神经网络的轻量化工作更多的是根据滤波器的范数值来进行裁剪,范数值越小,裁剪之后对网络的影响就越小。这种思路充分利用了滤波器的数值特性,但也忽略了滤波器的结构特性。基于上述观点,文中尝试将凝聚层次聚类算法AHCF(Agglomerative Hierarchical Clustering Method for Filter)应用到VGG16上,并利用此算法的结果对滤波器进行冗余性分析和数值分析;然后,提出了双角度并行剪枝方法,从滤波器的数值角度和结构角度对滤波器同时进行裁剪。所提方法裁剪了VGG16网络的冗余滤波器,降低了网络的参数数量,在提高网络分类精度的同时,基本保留了原有网络的学习速率。在CIFAR10数据集上,所提方法的分类精度相比原始网络提高了0.71%;在MNIST上,所提方法基本和原始网络保持相近的分类精度。
中图分类号:
| [1]DENTON E,ZAREMBA W,BRUNA J,et al.Exploiting Linear Structure Within Convolutional Networks for Efficient Evaluation[C]//Neural Information Processing Systems.MIT Press,2014:1269-1277. [2]HE K,ZHANG X,REN S,et al.Deep Residual Learning forImage Recognition[C]//IEEE Conference on Computer Vision &Pattern Recognition.IEEE Computer Society,2016:770-778. [3]HE Y,KANG G,DONG X,et al.Soft Filter Pruning for Accele-rating Deep Convolutional Neural Networks[C]//International Joint Conference on Artificial Intelligence.AAAI Press,2018:2234-2240. [4]HAN S,POOL J,TRAN J,et al.Learning both weightsand con-nections for efficient neural network[C]//Neural Information Processing Systems Conference.MIT Press:2015,1135-1143. [5]CARREIRA-PERPINAN M A,IDELBAYEV Y.learning-compression algorithms for neural net pruning[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2018:8532-8541. [6]LI H,KADAV A,DURDANOVIC I,et al.Pruning filters for efficient ConvNets[J].arXiv:1608.08710,2016. [7]YU R,LI A,DAVIS L S,et al.NISP:Pruning networks using neuron importance score propagation[C]//Computer Vision and Pattern Recognition.IEEE,2018:9194-9203. [8]SZEGEDY C,LIU W,JIA Y,et al.Going deeper with convolutions[C]//Computer Vision and Pattern Recognition.IEEE,2015:1-9. [9]HAN S,MAO H,DALLY W J.Deep Compression:Compres-sing Deep Neural Networks with Pruning,Trained Quantization and Huffman Coding[J].Fiber,2015,56(4):3-7. [10]YE J B,LU X,LIN Z,et al.Rethinking the smaller-norm-less-informative assumption in channel pruning of convolution layers[J].arXiv:1802.00124,2018. [11]LECUN Y,DENKER J S,SOLLA S A,et al.Optimal BrainDamage[C]//Advances in Neural Information Processing Systems.Massachusetts Institute of Technology Press,1989:598-605. [12]HASSIBI B,STORK D G.Second order derivatives for network pruning:Optimal Brain Surgeon[C]//Neural Information Processing Systems.Massachusetts Institute of Technology Press.1992:164-171. [13]HINTON G,VINYALS O,DEAN J.Distilling the Knowledgein a Neural Network[J].Computer Science,2015,14(7):38-39. [14]YANG T,CHEN Y,SZE V,et al.Designing Energy-EfficientConvolutional Neural Networks Using Energy-Aware Pruning[C]//Computer Vision and Pattern Recognition.IEEE,2017:6071-6079. [15]ZHUO H,QIAN X,FU Y,et al.SCSP:Spectral Clustering Filter Pruning with Soft Self-adaption Manners[J].arXiv:1806.05320,2018. [16]WANG D,ZHOU L,ZHANG X N,et al.Exploring Linear Relationship in Feature Map Subspace for ConvNets Compression[J].arXiv:1803.05729,2018. [17]BYUNGMIN A,KIM T.Deeper Weight Pruning without Accuracy Loss in Deep Neural Networks[C]//2020 Design,Automation & Test in Europe Conference.EDA Consortium,2020:73-78. [18]HANSON S J,PRATT L Y.Comparing biases for minimal network construction with back-propagation[C]//Advances in Neural Information Processing Systems.MIT Press,1989:177-185. [19]TUNG F,MORI G.Deep Neural Network Compression by In-Parallel Pruning-Quantization[J].IEEE Transactions on Pattern Analysis and Machine Intelligence ,2020:42(3):568-579. [20]ZHANG T,YE S,ZHANG K,et al.A systematic dnn weight pruning framework using alternating direction method of multipliers[C]//Computer Vision and Pattern Recognition.Lecture Notes in Computer Science,2018:191-207. [21]GUO Y,YAO A,CHEN Y,et al.Dynamic network surgery for efficient DNNs[C]//Neural Information Processing Systems.Curran Associates,Inc.,2016:1387-1395. [22]DONG X,CHEN S,PAN S.Learning to prune deep neural networks via layer-wise optimal brain surgeon[C]//Advances in Neural Information Processing Systems.Curran Associates Inc.,2017:4857-4867. [23]LIU Z,LI J,SHEN Z,et al.Learning Efficient Convolutional Networks through Network Slimming[C]//International Conference on Computer Vision.IEEE,2017:2755-2763. [24]LIN M B,JI R R,WANG Y,et al.HRank:Filter Pruning using High-Rank Feature Map[J].arXiv:2002.10179. [25]LUO J,WU J,LIN W,et al.ThiNet:A Filter Level Pruning Method for Deep Neural Network Compression[C]//International Conference on Computer Vision.IEEE,2017:5068-5076. [26]HE Y,ZHANG X,SUN J,et al.Channel Pruning for Accelerating Very Deep Neural Networks[C]// International Confe-rence on Computer Vision.IEEE Computer Society,2017:1398-1406. [27]MOLCHANOV P,TYREE S,KARRAS T,et al.Pruning Convo-lutional Neural Networks for Resource Efficient Inference[J].arXiv:1611.06440. [28]DUBEY A,CHATTERJEE M,AHUJA N,et al.Coreset-Based Neural Network Compression[C]//European Conference on Computer Vision.Springer International Publishing,2018:469-486. [29]SUAU X,ZAPPELLA L,PALAKKODE V,et al.Principal filter analysis for guided network compression[J].arXiv:1807.10585,2018. [30]WANG D,ZHOU L,ZHANG X,et al.Exploring linear relationship in feature map subspace for convnets compression[J].ar-Xiv:1803.05729,2018. [31]NG A Y,JORDAN M I,WEISS Y,et al.On Spectral Clustering:Analysis and an algorithm[C]//Neural Information Processing Systems.MIT Press,2001:849-856. [32]KUNCHEVA L I,BEZDEK J C.Nearest prototype classifica-tion:clustering,genetic algorithms,or random search?[J].IEEE Transactions on Systems Man Cybernetics-Systems,1998,28(1):160-164. [33]ZHU Z L,LIU Y J.A Novel Algorithm by Incorporating Chaos Optimization and Improved Fuzzy C-Means for Image Segmentation[J].Acta Electronica Sinica,2020,48(5):975-984. [34]ZHANG C X,PEI L J,CHEN Z,et al.RGBD Scene Flow Estimation Based on FRFCM Clustering and Depth Optimization[J].Acta Electronica Sinica,2020,48(7):1380-1386. [35]ZHANG Z Y,LIN J,MIAO R S.Hesitant fuzzy language agglomerative hierarchical clustering algorithm and its application [J].Statistics and Decision Making,2019,35(21):71-74. [36]LU X K.Research on parallel processing of multi-source information fusion based on clustering algorithm [J].Journal of China Academy of Electronic Sciences,2019,14(3):251-255,264. [37]BANDYOPADHYAY S,COYLE E J.An energy efficient hie-rarchical clustering algorithm for wireless sensor networks[C]//Twenty-Second Annual Joint Conference of the IEEE Computer and Communications(INFOCOM 2003).IEEE,2003:1713-1723. [38]ZHAO Y,KARYPIS G.Evaluation of hierarchical clustering algorithms for document Datasets[C]//Conference on Information and Knowledge Management.Association for Computing Machinery.2002:515-524. [39]LI G,YUAN X K,XU A D,et al.Application of artificial bee colony algorithm in big data based on MapReduce [J].Computer and Digital Engineering,2020,48(1):124-129,146. [40]MURTAGH F,LEGENDRE P.Ward’s Hierarchical Agglome-rative Clustering Method:Which Algorithms Implement Ward’s Criterion?[J].Journal of Classification,2014,31(3):274-295. [41]KRIZHEVSKY A,HINTON G.Learning multiple layers of features from tiny images[J].Handbook of Systemic Autoimmune Diseases,2009,1(4):1-60. [42]LECUN Y,BOTTOU L,HAFFNER P.Gradient-based learning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324. [43]SIMONYAN K,ZISSERMAN A.Very Deep Convolutional Networks for Large Scale Image Recognition[J].arXiv:1409.1556,2014. |
| [1] | 柴慧敏, 张勇, 方敏. 基于特征相似度聚类的空中目标分群方法 Aerial Target Grouping Method Based on Feature Similarity Clustering 计算机科学, 2022, 49(9): 70-75. https://doi.org/10.11896/jsjkx.210800203 |
| [2] | 周乐员, 张剑华, 袁甜甜, 陈胜勇. 多层注意力机制融合的序列到序列中国连续手语识别和翻译 Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion 计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026 |
| [3] | 李宗民, 张玉鹏, 刘玉杰, 李华. 基于可变形图卷积的点云表征学习 Deformable Graph Convolutional Networks Based Point Cloud Representation Learning 计算机科学, 2022, 49(8): 273-278. https://doi.org/10.11896/jsjkx.210900023 |
| [4] | 陈泳全, 姜瑛. 基于卷积神经网络的APP用户行为分析方法 Analysis Method of APP User Behavior Based on Convolutional Neural Network 计算机科学, 2022, 49(8): 78-85. https://doi.org/10.11896/jsjkx.210700121 |
| [5] | 朱承璋, 黄嘉儿, 肖亚龙, 王晗, 邹北骥. 基于注意力机制的医学影像深度哈希检索算法 Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism 计算机科学, 2022, 49(8): 113-119. https://doi.org/10.11896/jsjkx.210700153 |
| [6] | 檀莹莹, 王俊丽, 张超波. 基于图卷积神经网络的文本分类方法研究综述 Review of Text Classification Methods Based on Graph Convolutional Network 计算机科学, 2022, 49(8): 205-216. https://doi.org/10.11896/jsjkx.210800064 |
| [7] | 张颖涛, 张杰, 张睿, 张文强. 全局信息引导的真实图像风格迁移 Photorealistic Style Transfer Guided by Global Information 计算机科学, 2022, 49(7): 100-105. https://doi.org/10.11896/jsjkx.210600036 |
| [8] | 戴朝霞, 李锦欣, 张向东, 徐旭, 梅林, 张亮. 基于DNGAN的磁共振图像超分辨率重建算法 Super-resolution Reconstruction of MRI Based on DNGAN 计算机科学, 2022, 49(7): 113-119. https://doi.org/10.11896/jsjkx.210600105 |
| [9] | 刘月红, 牛少华, 神显豪. 基于卷积神经网络的虚拟现实视频帧内预测编码 Virtual Reality Video Intraframe Prediction Coding Based on Convolutional Neural Network 计算机科学, 2022, 49(7): 127-131. https://doi.org/10.11896/jsjkx.211100179 |
| [10] | 徐鸣珂, 张帆. Head Fusion:一种提高语音情绪识别的准确性和鲁棒性的方法 Head Fusion:A Method to Improve Accuracy and Robustness of Speech Emotion Recognition 计算机科学, 2022, 49(7): 132-141. https://doi.org/10.11896/jsjkx.210100085 |
| [11] | 金方焱, 王秀利. 融合RACNN和BiLSTM的金融领域事件隐式因果关系抽取 Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM 计算机科学, 2022, 49(7): 179-186. https://doi.org/10.11896/jsjkx.210500190 |
| [12] | 张嘉淏, 刘峰, 齐佳音. 一种基于Bottleneck Transformer的轻量级微表情识别架构 Lightweight Micro-expression Recognition Architecture Based on Bottleneck Transformer 计算机科学, 2022, 49(6A): 370-377. https://doi.org/10.11896/jsjkx.210500023 |
| [13] | 王建明, 陈响育, 杨自忠, 史晨阳, 张宇航, 钱正坤. 不同数据增强方法对模型识别精度的影响 Influence of Different Data Augmentation Methods on Model Recognition Accuracy 计算机科学, 2022, 49(6A): 418-423. https://doi.org/10.11896/jsjkx.210700210 |
| [14] | 孙洁琪, 李亚峰, 张文博, 刘鹏辉. 基于离散小波变换的双域特征融合深度卷积神经网络 Dual-field Feature Fusion Deep Convolutional Neural Network Based on Discrete Wavelet Transformation 计算机科学, 2022, 49(6A): 434-440. https://doi.org/10.11896/jsjkx.210900199 |
| [15] | 孙福权, 崔志清, 邹彭, 张琨. 基于多尺度特征的脑肿瘤分割算法 Brain Tumor Segmentation Algorithm Based on Multi-scale Features 计算机科学, 2022, 49(6A): 12-16. https://doi.org/10.11896/jsjkx.210700217 |
|
||
首页