Computer Science ›› 2018, Vol. 45 ›› Issue (9): 303-307.doi: 10.11896/j.issn.1002-137X.2018.09.051

• Graphics, Image & Pattern Recognition • Previous Articles     Next Articles

Energy-efficient Facial Expression Recognition Based on Improved Deep Residual Networks

DU Jin, CHEN Yun-hua, ZHANG Ling, MAI Ying-chao   

  1. School of Computers,Guangdong University of Technology,Guangzhou 510000,China
  • Received:2017-07-26 Online:2018-09-20 Published:2018-10-10

Abstract: To improve recognition rate and reduce power consumption of facial expression recognition systems,this paper proposed a facial expression recognition method using an improved deep residual networks(ResNets).Residual learning solves the degradation problem of the deep Convolutional Neural Networks(CNNs) to a certain degree and increases the network layers infinitely,but it makes deep CNNs face a more serious power consumption problem.To solve this problem,this paper introduced a new biologically-plausible activation function to improve ResNets and get a facial expression recognition method with both higher performance and lower power consumption.The Rectified Linear Units(ReLU) in the convolutional layers of ResNets are replaced with the new activation function Noisy Softplus.The obtained weights by using the improved ResNets can be directly applied to a deep Spiking Neural Networks(SNNs) architecture derived from the ResNets.The experimental results suggest that the proposed facial expression recognition method is able to achieve higher recognition rate and lower power consumption on a neuromorphic hardware.

Key words: Facial expression recognition, Residual networks, Leaky Integrate and Fire(LIF) neurons, Convolutional neural networks

CLC Number: 

  • TP183
[1] SHAN C,GONG S,MCOWAN P W.Facial expression recognition based on Local Binary Patterns:A comprehen-sive study[J].Image & Vision Computing,2009,27(6):803-816.
[2] HU Y,ZENG Z,YIN L,et al.Multi-view facial expression re-cognition[C]∥Proceedings of IEEE International Conference on Automatic Face & Gesture Recognition.Amsterdam:IEEE Press,2008:1-6.
[3] TARIQ U,LIN K H,LI Z,et al.Emotion recognition from an ensemble of features[C]∥Proceedings of IEEE International Conference on Automatic Face & Gesture Recognition and Workshops.Santa Barbara:IEEE Press,2012:872-877.
[4] HINTON G E,OSINDERO S,TEH Y W. A Fast Learning Algorithm for Deep Belief Nets[J].Neural Computation,2014,18(7):1527-1554.
[5] LIU M,LI S,SHAN S,et al.Deeply Learning Deformable Facial Action Parts Model for Dynamic Expression Analysis[C]∥Proceedings of Asian Conference on Computer Vision.Singapore:Springer,2014:143-157.
[6] KAHOU S E,PAL C,BOUTHILLIER X,et al.Combining modality specific deep neural networks for emotion recognition in video[C]∥Proceedings of ACM on International Conference on Multimodal Interaction.Sydney:ACM Press,2013:543-550.
[7] KIM B K,LEE H,ROH J,et al.Hierarchical Committee of Deep CNNs with Exponentially-Weighted Decision Fusion for Static Facial Expression Recognition[C]∥Proceedings of ACM on International Conference on Multimodal Interaction.New York:ACM Press,2015:427-434.
[8] SUN S,CHEN W,WANG L,et al.On the Depth of Deep Neural Networks:A Theoretical View[C]∥Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence.Phoenix:AAAI Press,2015:2066-2072.
[9] SZEGEDY C,LIU W,JIA Y,et al.Going deeper with convolutions[C]∥Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Boston:IEEE Press,2014:1-9.
[10] HE K,ZHANG X,REN S,et al.Deep Residual Learning for Ima-ge Recognition[C]∥Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.Las Vegas:IEEE Press,2015:770-778.
[11] DENG J,DONG W,SOCHER R,et al.ImageNet:A large scale hierarchical image database[C]∥Proceedings of IEEE Con-ference on Computer Vision and Pattern Recognition.Miami:IEEE Press,2009:248-255.
[12] IZHIKEVICH E M.Simple model of spiking neurons[J].IEEE Transactions on Neural Networks,2004,14(6):1569-1572.
[13] PAUGAMMOISY H,BOHTE S.Computing with Spiking Neuron Networks[M].The Netherlands:Springer,2012:335-376.
[14] LIU Q,FURBER S.Noisy Softplus:A Biology Inspired Activation Function[C]∥International Conference on Neural Information Processing.Kyoto:Springer,2016:405-412.
[15] MNIST database[OL].
[16] DAYAN P,ABBOTT L F.Theoretical Neuroscience[M].Cambridge:Mit Press,2001:180-191.
[17] IZHIKEVICH E M.Which model to use for cortical spikingneurons[J].IEEE Transactions on Neural Networks,2004,15(5):1063-1070.
[18] BRETTE R,GERSTNER W.Adaptive exponential integrate-and-fire model as an effective description of neuronal activity[J].Journal of Neurophysiology,2005,94(5):36-37.
[19] LIU Y H,WANG X J.Spike-frequency adaptation of a genera-lized leaky integrate-and-fire model neuron[J].Journal of Computational Neuroscience,2001,10(1):25-45.
[20] GLOROT X,BORDES A,BENGIO Y.Deep Sparse RectifierNeural Networks[C]∥Proceedings of the InternationalCon-ference on Artificial Intelligence and Statistics.Fort Lauderdale:MIT press 2011:315-323.
[21] HE K M,ZHANG X,REN S,et al.Identity Mappings in Deep Residual Networks[C]∥Proceedings of European Conference on Computer Vision.Amsterdam:Springer,2016:630-645.
[22] LUCEY P,COHN J F,KANADE T,et al.The extended cohn-kanade dataset (ck+):A complete dataset for action unit and emotion-specified expression[C]∥Proceedings of IEEE Compu-ter Society Conference on Computer Vision and Pattern Recognition Workshops.San Francisco:IEEE Press,2010:94-101.
[23] KDEF database[OL].
[24] GENKI-4K database[OL].
[25] MOLLAHOSSEINI A,Chan D,MAHOOR M H.Going deeper in facial expression recognition using deep neural networks[C]∥Proceedings of Winter Conference on Applications of Computer Vision.New York:IEEE Press,2016:1-10.
[26] LIEW C F,YAIRI T.A comparison study of feature spaces and classification methods for facial expression recognition[C]∥Proceedings of International Conference on Robotics and Biomimetics.Shenzhen:IEEE Press,2013:1294-1299.
[27] SANTRA B,MAUKHERJEE D P.Local Dominant Binary Patterns for Recognition of Multiview Facial Expressions[C]∥Proceedings of the Asian Conference on Computer Vision.Guwahati:ACM Press,2016:1-25.
[28] RUIZ A,ELSHAW M,ALTAHHAN A,et al.Stacked deep convolutional auto-encoders for emotion recognition from facial expressions[C]∥Proceedings of International Joint Conference on Neural Networks.Anchorage:IEEE Press,2017:1586-1593.
[29] CUI D,HUANG G B,LIU T.Smile detection using Pair-wise Distance Vector and Extreme Learning Machine[C]∥Procee-dings of International Joint Conference on Neural Networks.Vancouver:IEEE Press,2016:2298-2305.
[30] ZHANG L,TJONDRONEGORO D,CHANGRAN V,et al.Towards robust automatic affective classification of images using facial expressions for practical applications [J].Multimedia Tools & Applications,2016,75(8):1-27.
[31] CRUZ-ALBRECHT J M,YUNG M W,SRINIVASA N.Energy-Efficient Neuron,Synapse and STDP Integrated Circuits[J].IEEE Transactions on Biomedical Circuits & Systems,2012,6(3):246-256.
[32] MEROLLA P,ARTHUR J,AKOPYAN F,et al.A digital neurosynaptic core using embedded crossbar memory with 45pJ per spike in 45nm[C]∥IEEE Custom Integrated Circuits Con-ference.San Jose:IEEE Press,2011:1-4.
[33] TRIMBERGER S M.Field-Programmable Gate Array Techno-logy[M].New York:Springer,1994:68-87.
[34] CAO Y,CHEN Y,KHOSLA D.Spiking Deep Convolutional Neural Networks for Energy-Efficient Object Recognition[J].International Journal of Computer Vision,2015,113(1):54-66.
[35] FARABET C,MARTINI B,AKSELROD P,et al. Hardware accelerated convolutional neural networks for synthetic vision systems[C]∥Proceedings of IEEE International Symposium on Circuits and Systems.Paris:IEEE Press,2010:257-260.
[1] SUN Yan-li, YE Jiong-yao. Convolutional Neural Networks Compression Based on Pruning and Quantization [J]. Computer Science, 2020, 47(8): 261-266.
[2] MA Hai-Jiang. Recommendation Algorithm Based on Convolutional Neural Network and Constrained Probability Matrix Factorization [J]. Computer Science, 2020, 47(6A): 540-545.
[3] PENG Xian, PENG Yu-xu, TANG Qiang, SONG Yan-qi. Crowd Counting Based on Single-column Multi-scale Convolutional Neural Network [J]. Computer Science, 2020, 47(4): 150-156.
[4] LIU Yu-hong,LIU Shu-ying,FU Fu-xiang. Optimization of Compressed Sensing Reconstruction Algorithms Based on Convolutional Neural Network [J]. Computer Science, 2020, 47(3): 143-148.
[5] HUANG Hong-wei,LIU Yu-jiao,SHEN Zhuo-kai,ZHANG Shao-wei,CHEN Zhi-min,GAO Yang. End-to-end Track Association Based on Deep Learning Network Model [J]. Computer Science, 2020, 47(3): 200-205.
[6] WANG Li-hua,DU Ming-hui,LIANG Ya-ling. Classification Net Based on Angular Feature [J]. Computer Science, 2020, 47(2): 83-87.
[7] FU Xue-yang,SUN Qi,HUANG Yue,DING Xing-hao. Single Image De-raining Method Based on Deep Adjacently Connected Networks [J]. Computer Science, 2020, 47(2): 106-111.
[8] SHAO Yang-xue, MENG Wei, KONG Deng-zhen, HAN Lin-xuan, LIU Yang. Cross-modal Retrieval Method for Special Vehicles Based on Deep Learning [J]. Computer Science, 2020, 47(12): 205-209.
[9] HAN Rui, GU Chun-li, LI Zhe, WU Kang, GAO Feng, SHEN Wen-hai. Global Typhoon Message Collection Method Based on CNN-typhoon Model [J]. Computer Science, 2020, 47(11A): 11-17.
[10] ZHANG Mei-yu, LIU Yue-hui, QIN Xu-jia, WU Liang-wu. Neural Style Transfer Method Based on Laplace Operator to Suppress Artifacts [J]. Computer Science, 2020, 47(11A): 209-214.
[11] HUA Ming, LI Dong-dong, WANG Zhe, GAO Da-qi. End-to-End Speaker Recognition Based on Frame-level Features [J]. Computer Science, 2020, 47(10): 169-173.
[12] SHI Xiao-hong, HUANG Qin-kai, MIAO Jia-xin, SU Zhuo. Edge-preserving Filtering Method Based on Convolutional Neural Networks [J]. Computer Science, 2019, 46(9): 277-283.
[13] JIANG Bin,GAN Yong,ZHANG Huan-long,ZHANG Qiu-wen. Survey on Non-frontal Facial Expression Recognition Methods [J]. Computer Science, 2019, 46(3): 53-62.
[14] JING Jie, CHEN Tan, DU Wen-li, LIU Zhi-kang, YIN Hao. Spatio-temporal Features Extraction of Traffic Based on Deep Neural Network [J]. Computer Science, 2019, 46(11A): 1-4.
[15] ZHANG Rui, ZHAN Yong-song, YANG Ming-hao. Handwritten Drawing Order Recovery Method Based on Endpoint Sequential Prediction [J]. Computer Science, 2019, 46(11A): 264-267.
Full text



[1] . [J]. Computer Science, 2018, 1(1): 1 .
[2] LEI Li-hui and WANG Jing. Parallelization of LTL Model Checking Based on Possibility Measure[J]. Computer Science, 2018, 45(4): 71 -75 .
[3] SUN Qi, JIN Yan, HE Kun and XU Ling-xuan. Hybrid Evolutionary Algorithm for Solving Mixed Capacitated General Routing Problem[J]. Computer Science, 2018, 45(4): 76 -82 .
[4] ZHANG Jia-nan and XIAO Ming-yu. Approximation Algorithm for Weighted Mixed Domination Problem[J]. Computer Science, 2018, 45(4): 83 -88 .
[5] WU Jian-hui, HUANG Zhong-xiang, LI Wu, WU Jian-hui, PENG Xin and ZHANG Sheng. Robustness Optimization of Sequence Decision in Urban Road Construction[J]. Computer Science, 2018, 45(4): 89 -93 .
[6] SHI Wen-jun, WU Ji-gang and LUO Yu-chun. Fast and Efficient Scheduling Algorithms for Mobile Cloud Offloading[J]. Computer Science, 2018, 45(4): 94 -99 .
[7] ZHOU Yan-ping and YE Qiao-lin. L1-norm Distance Based Least Squares Twin Support Vector Machine[J]. Computer Science, 2018, 45(4): 100 -105 .
[8] LIU Bo-yi, TANG Xiang-yan and CHENG Jie-ren. Recognition Method for Corn Borer Based on Templates Matching in Muliple Growth Periods[J]. Computer Science, 2018, 45(4): 106 -111 .
[9] GENG Hai-jun, SHI Xin-gang, WANG Zhi-liang, YIN Xia and YIN Shao-ping. Energy-efficient Intra-domain Routing Algorithm Based on Directed Acyclic Graph[J]. Computer Science, 2018, 45(4): 112 -116 .
[10] CUI Qiong, LI Jian-hua, WANG Hong and NAN Ming-li. Resilience Analysis Model of Networked Command Information System Based on Node Repairability[J]. Computer Science, 2018, 45(4): 117 -121 .