基于改进深度残差网络的低功耗表情识别

doi:10.11896／j.issn.1002-137X.2018.09.051

摘要/Abstract

摘要： 为了提高表情识别率并降低表情识别的功耗,提出一种基于改进深度残差网络的表情识别方法。残差学习在解决深度卷积神经网络退化问题、使网络层次大幅加深的同时,进一步增加了网络的功耗。为此,引入具有生物真实性的激活函数来代替已有的整流线性单元(Rectified Linear Units,ReLU)函数,并将其作为卷积层激活函数对深度残差网络进行改进。该方法不仅提高了残差网络的精度,而且训练出的网络权重可直接作为与该深度残差网络具有相同结构的深度脉冲神经网络的权重。将该深度脉冲神经网络部署在类脑硬件上时,其能够以较高的识别率和较低的能耗进行表情识别。

关键词: Leaky Integrate and Fire(LIF)神经元, 表情识别, 残差网络, 卷积神经网络

Abstract: To improve recognition rate and reduce power consumption of facial expression recognition systems,this paper proposed a facial expression recognition method using an improved deep residual networks(ResNets).Residual learning solves the degradation problem of the deep Convolutional Neural Networks(CNNs) to a certain degree and increases the network layers infinitely,but it makes deep CNNs face a more serious power consumption problem.To solve this problem,this paper introduced a new biologically-plausible activation function to improve ResNets and get a facial expression recognition method with both higher performance and lower power consumption.The Rectified Linear Units(ReLU) in the convolutional layers of ResNets are replaced with the new activation function Noisy Softplus.The obtained weights by using the improved ResNets can be directly applied to a deep Spiking Neural Networks(SNNs) architecture derived from the ResNets.The experimental results suggest that the proposed facial expression recognition method is able to achieve higher recognition rate and lower power consumption on a neuromorphic hardware.

Key words: Convolutional neural networks, Facial expression recognition, Leaky Integrate and Fire(LIF) neurons, Residual networks

中图分类号:

TP183

杜进, 陈云华, 张灵, 麦应潮. 基于改进深度残差网络的低功耗表情识别[J]. 计算机科学, 2018, 45(9): 303-307. https://doi.org/10.11896／j.issn.1002-137X.2018.09.051

DU Jin, CHEN Yun-hua, ZHANG Ling, MAI Ying-chao. Energy-efficient Facial Expression Recognition Based on Improved Deep Residual Networks[J]. Computer Science, 2018, 45(9): 303-307. https://doi.org/10.11896／j.issn.1002-137X.2018.09.051

参考文献

[1]SHAN C,GONG S,MCOWAN P W.Facial expression recognition based on Local Binary Patterns:A comprehen-sive study[J].Image & Vision Computing,2009,27(6):803-816.
[2]HU Y,ZENG Z,YIN L,et al.Multi-view facial expression re-cognition[C]∥Proceedings of IEEE International Conference on Automatic Face & Gesture Recognition.Amsterdam:IEEE Press,2008:1-6.
[3]TARIQ U,LIN K H,LI Z,et al.Emotion recognition from an ensemble of features[C]∥Proceedings of IEEE International Conference on Automatic Face & Gesture Recognition and Workshops.Santa Barbara:IEEE Press,2012:872-877.
[4]HINTON G E,OSINDERO S,TEH Y W. A Fast Learning Algorithm for Deep Belief Nets[J].Neural Computation,2014,18(7):1527-1554.
[5]LIU M,LI S,SHAN S,et al.Deeply Learning Deformable Facial Action Parts Model for Dynamic Expression Analysis[C]∥Proceedings of Asian Conference on Computer Vision.Singapore:Springer,2014:143-157.
[6]KAHOU S E,PAL C,BOUTHILLIER X,et al.Combining modality specific deep neural networks for emotion recognition in video[C]∥Proceedings of ACM on International Conference on Multimodal Interaction.Sydney:ACM Press,2013:543-550.
[7]KIM B K,LEE H,ROH J,et al.Hierarchical Committee of Deep CNNs with Exponentially-Weighted Decision Fusion for Static Facial Expression Recognition[C]∥Proceedings of ACM on International Conference on Multimodal Interaction.New York:ACM Press,2015:427-434.
[8]SUN S,CHEN W,WANG L,et al.On the Depth of Deep Neural Networks:A Theoretical View[C]∥Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence.Phoenix:AAAI Press,2015:2066-2072.
[9]SZEGEDY C,LIU W,JIA Y,et al.Going deeper with convolutions[C]∥Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Boston:IEEE Press,2014:1-9.
[10]HE K,ZHANG X,REN S,et al.Deep Residual Learning for Ima-ge Recognition[C]∥Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas:IEEE Press,2015:770-778.
[11]DENG J,DONG W,SOCHER R,et al.ImageNet:A large scale hierarchical image database[C]∥Proceedings of IEEE Con-ference on Computer Vision and Pattern Recognition.Miami:IEEE Press,2009:248-255.
[12]IZHIKEVICH E M.Simple model of spiking neurons[J].IEEE Transactions on Neural Networks,2004,14(6):1569-1572.
[13]PAUGAMMOISY H,BOHTE S.Computing with Spiking Neuron Networks[M].The Netherlands:Springer,2012:335-376.
[14]LIU Q,FURBER S.Noisy Softplus:A Biology Inspired Activation Function[C]∥International Conference on Neural Information Processing.Kyoto:Springer,2016:405-412.
[15]MNIST database[OL].http://yann.lecun.com/exdb/mnist.
[16]DAYAN P,ABBOTT L F.Theoretical Neuroscience[M].Cambridge:Mit Press,2001:180-191.
[17]IZHIKEVICH E M.Which model to use for cortical spiking
neurons[J].IEEE Transactions on Neural Networks,2004,15(5):1063-1070.
[18]BRETTE R,GERSTNER W.Adaptive exponential integrate-and-fire model as an effective description of neuronal activity[J].Journal of Neurophysiology,2005,94(5):36-37.
[19]LIU Y H,WANG X J.Spike-frequency adaptation of a genera-lized leaky integrate-and-fire model neuron[J].Journal of Computational Neuroscience,2001,10(1):25-45.
[20]GLOROT X,BORDES A,BENGIO Y.Deep Sparse Rectifier
Neural Networks[C]∥Proceedings of the InternationalCon-ference on Artificial Intelligence and Statistics.Fort Lauderdale:MIT press 2011:315-323.
[21]HE K M,ZHANG X,REN S,et al.Identity Mappings in Deep Residual Networks[C]∥Proceedings of European Conference on Computer Vision.Amsterdam:Springer,2016:630-645.
[22]LUCEY P,COHN J F,KANADE T,et al.The extended cohn-kanade dataset (ck+):A complete dataset for action unit and emotion-specified expression[C]∥Proceedings of IEEE Compu-ter Society Conference on Computer Vision and Pattern Recognition Workshops.San Francisco:IEEE Press,2010:94-101.
[23]KDEF database[OL].http://www.emotionlab.se/resources/kdef.
[24]GENKI-4K database[OL].http://mplab.ucsd.edu.
[25]MOLLAHOSSEINI A,Chan D,MAHOOR M H.Going deeper in facial expression recognition using deep neural networks[C]∥Proceedings of Winter Conference on Applications of Computer Vision.New York:IEEE Press,2016:1-10.
[26]LIEW C F,YAIRI T.A comparison study of feature spaces and classification methods for facial expression recognition[C]∥Proceedings of International Conference on Robotics and Biomimetics.Shenzhen:IEEE Press,2013:1294-1299.
[27]SANTRA B,MAUKHERJEE D P.Local Dominant Binary Patterns for Recognition of Multiview Facial Expressions[C]∥Proceedings of the Asian Conference on Computer Vision.Guwahati:ACM Press,2016:1-25.
[28]RUIZ A,ELSHAW M,ALTAHHAN A,et al.Stacked deep convolutional auto-encoders for emotion recognition from facial expressions[C]∥Proceedings of International Joint Conference on Neural Networks.Anchorage:IEEE Press,2017:1586-1593.
[29]CUI D,HUANG G B,LIU T.Smile detection using Pair-wise Distance Vector and Extreme Learning Machine[C]∥Procee-dings of International Joint Conference on Neural Networks.Vancouver:IEEE Press,2016:2298-2305.
[30]ZHANG L,TJONDRONEGORO D,CHANGRAN V,et al.Towards robust automatic affective classification of images using facial expressions for practical applications [J].Multimedia Tools & Applications,2016,75(8):1-27.
[31]CRUZ-ALBRECHT J M,YUNG M W,SRINIVASA N.Energy-Efficient Neuron,Synapse and STDP Integrated Circuits[J].IEEE Transactions on Biomedical Circuits & Systems,2012,6(3):246-256.
[32]MEROLLA P,ARTHUR J,AKOPYAN F,et al.A digital neurosynaptic core using embedded crossbar memory with 45pJ per spike in 45nm[C]∥IEEE Custom Integrated Circuits Con-ference.San Jose:IEEE Press,2011:1-4.
[33]TRIMBERGER S M.Field-Programmable Gate Array Techno-logy[M].New York:Springer,1994:68-87.
[34]CAO Y,CHEN Y,KHOSLA D.Spiking Deep Convolutional Neural Networks for Energy-Efficient Object Recognition[J].International Journal of Computer Vision,2015,113(1):54-66.
[35]FARABET C,MARTINI B,AKSELROD P,et al. Hardware accelerated convolutional neural networks for synthetic vision systems[C]∥Proceedings of IEEE International Symposium on Circuits and Systems.Paris:IEEE Press,2010:257-260.

相关文章 15

[1]	周乐员, 张剑华, 袁甜甜, 陈胜勇. 多层注意力机制融合的序列到序列中国连续手语识别和翻译 Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion 计算机科学, 2022, 49(9): 155-161. https://doi.org/10.11896/jsjkx.210800026
[2]	李宗民, 张玉鹏, 刘玉杰, 李华. 基于可变形图卷积的点云表征学习 Deformable Graph Convolutional Networks Based Point Cloud Representation Learning 计算机科学, 2022, 49(8): 273-278. https://doi.org/10.11896/jsjkx.210900023
[3]	王馨彤, 王璇, 孙知信. 基于多尺度记忆残差网络的网络流量异常检测模型 Network Traffic Anomaly Detection Method Based on Multi-scale Memory Residual Network 计算机科学, 2022, 49(8): 314-322. https://doi.org/10.11896/jsjkx.220200011
[4]	陈泳全, 姜瑛. 基于卷积神经网络的APP用户行为分析方法 Analysis Method of APP User Behavior Based on Convolutional Neural Network 计算机科学, 2022, 49(8): 78-85. https://doi.org/10.11896/jsjkx.210700121
[5]	朱承璋, 黄嘉儿, 肖亚龙, 王晗, 邹北骥. 基于注意力机制的医学影像深度哈希检索算法 Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism 计算机科学, 2022, 49(8): 113-119. https://doi.org/10.11896/jsjkx.210700153
[6]	檀莹莹, 王俊丽, 张超波. 基于图卷积神经网络的文本分类方法研究综述 Review of Text Classification Methods Based on Graph Convolutional Network 计算机科学, 2022, 49(8): 205-216. https://doi.org/10.11896/jsjkx.210800064
[7]	张颖涛, 张杰, 张睿, 张文强. 全局信息引导的真实图像风格迁移 Photorealistic Style Transfer Guided by Global Information 计算机科学, 2022, 49(7): 100-105. https://doi.org/10.11896/jsjkx.210600036
[8]	戴朝霞, 李锦欣, 张向东, 徐旭, 梅林, 张亮. 基于DNGAN的磁共振图像超分辨率重建算法 Super-resolution Reconstruction of MRI Based on DNGAN 计算机科学, 2022, 49(7): 113-119. https://doi.org/10.11896/jsjkx.210600105
[9]	刘月红, 牛少华, 神显豪. 基于卷积神经网络的虚拟现实视频帧内预测编码 Virtual Reality Video Intraframe Prediction Coding Based on Convolutional Neural Network 计算机科学, 2022, 49(7): 127-131. https://doi.org/10.11896/jsjkx.211100179
[10]	徐鸣珂, 张帆. Head Fusion:一种提高语音情绪识别的准确性和鲁棒性的方法 Head Fusion:A Method to Improve Accuracy and Robustness of Speech Emotion Recognition 计算机科学, 2022, 49(7): 132-141. https://doi.org/10.11896/jsjkx.210100085
[11]	金方焱, 王秀利. 融合RACNN和BiLSTM的金融领域事件隐式因果关系抽取 Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM 计算机科学, 2022, 49(7): 179-186. https://doi.org/10.11896/jsjkx.210500190
[12]	杨玥, 冯涛, 梁虹, 杨扬. 融合交叉注意力机制的图像任意风格迁移 Image Arbitrary Style Transfer via Criss-cross Attention 计算机科学, 2022, 49(6A): 345-352. https://doi.org/10.11896/jsjkx.210700236
[13]	杨健楠, 张帆. 一种结合双注意力机制和层次网络结构的细碎农作物分类方法 Classification Method for Small Crops Combining Dual Attention Mechanisms and Hierarchical Network Structure 计算机科学, 2022, 49(6A): 353-357. https://doi.org/10.11896/jsjkx.210200169
[14]	孙福权, 崔志清, 邹彭, 张琨. 基于多尺度特征的脑肿瘤分割算法 Brain Tumor Segmentation Algorithm Based on Multi-scale Features 计算机科学, 2022, 49(6A): 12-16. https://doi.org/10.11896/jsjkx.210700217
[15]	吴子斌, 闫巧. 基于动量的映射式梯度下降算法 Projected Gradient Descent Algorithm with Momentum 计算机科学, 2022, 49(6A): 178-183. https://doi.org/10.11896/jsjkx.210500039

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed