Computer Science ›› 2020, Vol. 47 ›› Issue (8): 227-232.doi: 10.11896/jsjkx.190700009

Previous Articles     Next Articles

Three-dimensional Convolutional Neural Network Evolution Method for Facial Micro-expression Auto-recognition

LIANG Zheng-you, HE Jing-lin, SUN Yu   

  1. School of Computer and Electronics Information, Guangxi University, Nanning 530004, China
  • Online:2020-08-15 Published:2020-08-10
  • About author:LIANG Zheng-you, born in 1968, Ph.D, professor, is a member of China Computer Federation.His main research interests include computer vision, wireless sensor networks, parallel distributed computing and artificial intelligence.
  • Supported by:
    This work was supported by the National Natural Science Foundation of China(61763002).

Abstract: Due to the short duration of micro-expressions and the small amplitude of motion, the automatic recognition of micro-expressions is still a challenging problem.Aiming at the problems, this paper proposes a Three-Dimensional Convolutional Neural Network Evolution (C3DEvol) method for micro-expression recognition.In the C3DEvol, three-dimensional Convolutional Neural Network (C3D) which can extract dynamic information effectively is used to extract micro-expression features in time domain and space domain.At the same time, the genetic algorithm with the capabilities of global search and optimization is used to optimize the network structure of C3D in order to obtain the optimal network structure and avoid local optimization.Experiments are performed on a workstation with two NVIDIA Titan X GPUs using the CASME2 dataset.Experiments show that the accuracy of C3DEvol micro-expression automatic recognition reaches 63.71%, which is better than the existing micro-expression automatic recognition method.

Key words: Feature extraction, Genetic algorithm, Micro-expression recognition, Network structure optimization, Three-dimensional convolutional neural network

CLC Number: 

  • TP391
[1]CORNEANU C, OLIU M, COHN J F, et al.Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition:History, Trends, and Affect-related Applications[J].IEEE Transactions on Pattern Analysis & Machine Intelligence, 2016, 38(8):1548-1568.
[2]RUSSELL T A, CHU E, PHILLIPS M L.A pilot study to in-vestigate the effectiveness of emotion recognition remediation in schizophrenia using the micro-expression training tool[J].British Journal of Clinical Psychology, 2006, 45(4):579-583.
[3]PFISTER T, LI X, ZHAO G, et al.Recognising spontaneous facial micro-expressions[C]∥2011 IEEE International Conference on Computer Vision (ICCV).IEEE, 2011:1449-1456.
[4]WANG Y, SEE J, PHAN R C W, et al.Lbp with sixintersection points:Reducing redundant information in lbp-top for micro-expression recognition[C]∥Asian Conference on Computer Vision.Cham:Springer, 2014:525-537.
[5]WANG Y, SEE J, PHAN C W, et al.Efficient Spatio-Temporal Local Binary Patterns for Spontaneous Facial Micro-Expression Recognition[J].Plos One, 2015, 10(5):1-20.
[6]HUANG X, ZHAO G, HONG X, et al.Spontaneous facial mi-cro-expression analysis using spatiotemporal completed local quantized patterns[J].Neurocomputing, 2016, 175:564-578.
[7]LIU Y J, ZHANG J K, YAN W J, et al.A main directional mean optical flow feature for spontaneous micro-expression recognition[J].IEEE Transactions on Affective Computing, 2016, 7(4):299-310.
[8]HUANG W, FAN L, HARANDI M, et al.Toward Efficient Action Recognition:Principal Backpropagation for Training Two-Stream Networks[J].IEEE Transactions on Image Processing, 2019, 28(4):1773-1782.
[9]YOUNG T, HAZARIKA D, PORIA S, et al.Recent trends indeep learning based natural language processing[J].IEEE Computational Intelligence Magazine, 2018, 13(3):55-75.
[10]XIONG W, WU L, ALLEVA F, et al.The Microsoft 2017 conversational speech recognition system[C]∥2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).IEEE, 2018:5934-5938.
[11]TRAN D, BOURDEV L, FERGUS R, et al.Learning spatiotemporal features with 3d convolutional networks[C]∥Proceedings of the IEEE International Conference on Computer Vision.2015:4489-4497.
[12]KIM D H, BADDAR W J, JANG J, et al.Multi-objective based spatio-temporal feature representation learning robust to expression intensity variations for facial expression recognition[J].IEEE Transactions on Affective Computing, 2017, 10(2):223-236.
[13]PENG M, WANG C, CHEN T, et al.Dual temporal scale convolutional neural network for micro-expression recognition[J].Frontiers in Psychology, 2017, 8:1-12.
[14]PETKE J, HARALDSSON S O, HARMAN M, et al.Genetic Improvement of Software:A Comprehensive Survey[J].IEEE Transactions on Evolutionary Computation, 2018, 22(3):415-432.
[15]KIM Y H, YOON Y, GEEM Z W.A comparison study of harmony search and genetic algorithm for the max-cut problem[J].Swarm and Evolutionary Computation, 2019, 44:130-135.
[16]METEVIER B, SAINI A K, SPECTOR L.Lexicase SelectionBeyond Genetic Programming[M]∥Genetic Programming Theo-ry and Practice XVI.Cham:Springer, 2019:123-136.
[17]NGUYEN S, ZHANG M, JOHNSTON M, et al.Genetic Programming for Job Shop Scheduling[M]∥Evolutionary and Swarm Intelligence Algorithms.Cham:Springer, 2019:143-167.
[18]SHANMUGAPRIYA K, MALAR R M S M.An EffectiveTechnique to Track Objects with the Aid of Rough Set Theory and Evolutionary Programming[J].Journal of Intelligent Systems, 2019, 28(1):1-13.
[19]CHEN Z, XIA J, BAI J, et al.Feature extraction algorithm based on evolutionary deep learning[J].Computer Science, 2015, 42(11):288-292.
[20]IJJINA E P, CHALAVADI K M.Human action recognitionusing genetic algorithms and convolutional neural networks[J].Pattern Recognition, 2016, 59:199-212.
[21]OULLETTE R, BROWNE M, HIRASAWA K.Genetic algo-rithm optimization of a convolutional neural network for autonomous crack detection[C]∥Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat.No.04TH8753).IEEE, 2004:516-521.
[22]RIKHTEGAR A, POOYAN M, MANZURI-SHALMANI M T.Genetic algorithm-optimised structure of convolutional neural network for face recognition applications[J].IET Computer Vision, 2016, 10(6):559-566.
[23]JI S, XU W, YANG M, et al.3D convolutional neural networks for human action recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(1):221-231.
[24]YAN W J, WU Q, LIU Y J, et al.CASME database:a dataset of spontaneous micro-expressions collected from neutralized faces[C]∥2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).IEEE, 2013:1-7.
[25]YAN W J, LI X, WANG S J, et al.CASME II:An Improved Spontaneous Micro-Expression Database and the Baseline Evaluation[J].Plos One, 2014, 9(1):1-8.
[26]LI X, PFISTER T, HUANG X, et al.A spontaneous micro-expression database:Inducement, collection and baseline[C]∥2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).IEEE, 2013:1-6.
[27]VIODA P.Rapid object detection using a boosted cascade ofsimple features[C]∥Proc.IEEE CVPR 2001.2001:905-910.
[28]ZHOU Z, ZHAO G, PIETIKINEN M.Towards a practical lipreading system[C]∥2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE, 2011:137-144.
[29]LIONG S T, SEE J, WONG K S, et al.Less is more:Micro-expression recognition from video using apex frame[J].Signal Processing:Image Communication, 2018, 62:82-92.
[30]XU F, ZHANG J, WANG J Z.Micro-expression identificationand categorization using a facial dynamics map[J].IEEE Tran-sactions on Affective Computing, 2017, 8(2):254-267.
[31]HE J, HU J F, LU X, et al.Multi-task mid-level feature learning for micro-expression recognition[J].Pattern Recognition, 2017, 66:44-52.
[32]PATEL D, HONG X, ZHAO G.Selective deep features for micro-expression recognition[C]∥2016 23rd International Confe-rence on Pattern Recognition (ICPR).IEEE, 2016:2258-2263.
[1] ZHANG Yuan, KANG Le, GONG Zhao-hui, ZHANG Zhi-hong. Related Transaction Behavior Detection in Futures Market Based on Bi-LSTM [J]. Computer Science, 2022, 49(7): 31-39.
[2] ZENG Zhi-xian, CAO Jian-jun, WENG Nian-feng, JIANG Guo-quan, XU Bin. Fine-grained Semantic Association Video-Text Cross-modal Entity Resolution Based on Attention Mechanism [J]. Computer Science, 2022, 49(7): 106-112.
[3] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[4] ZHANG Jia-hao, LIU Feng, QI Jia-yin. Lightweight Micro-expression Recognition Architecture Based on Bottleneck Transformer [J]. Computer Science, 2022, 49(6A): 370-377.
[5] LIU Wei-ye, LU Hui-min, LI Yu-peng, MA Ning. Survey on Finger Vein Recognition Research [J]. Computer Science, 2022, 49(6A): 1-11.
[6] YANG Hao-xiong, GAO Jing, SHAO En-lu. Vehicle Routing Problem with Time Window of Takeaway Food ConsideringOne-order-multi-product Order Delivery [J]. Computer Science, 2022, 49(6A): 191-198.
[7] GAO Yuan-hao, LUO Xiao-qing, ZHANG Zhan-cheng. Infrared and Visible Image Fusion Based on Feature Separation [J]. Computer Science, 2022, 49(5): 58-63.
[8] ZUO Jie-ge, LIU Xiao-ming, CAI Bing. Outdoor Image Weather Recognition Based on Image Blocks and Feature Fusion [J]. Computer Science, 2022, 49(3): 197-203.
[9] LI Xing-ran, ZHANG Li-yan, YAO Shu-jing. Micro-expression Recognition Method Combining Feature Fusion and Attention Mechanism [J]. Computer Science, 2022, 49(2): 4-11.
[10] SHEN Biao, SHEN Li-wei, LI Yi. Dynamic Task Scheduling Method for Space Crowdsourcing [J]. Computer Science, 2022, 49(2): 231-240.
[11] REN Shou-peng, LI Jin, WANG Jing-ru, YUE Kun. Ensemble Regression Decision Trees-based lncRNA-disease Association Prediction [J]. Computer Science, 2022, 49(2): 265-271.
[12] ZHANG Shi-peng, LI Yong-zhong. Intrusion Detection Method Based on Denoising Autoencoder and Three-way Decisions [J]. Computer Science, 2021, 48(9): 345-351.
[13] FENG Xia, HU Zhi-yi, LIU Cai-hua. Survey of Research Progress on Cross-modal Retrieval [J]. Computer Science, 2021, 48(8): 13-23.
[14] ZHANG Li-qian, LI Meng-hang, GAO Shan-shan, ZHANG Cai-ming. Summary of Computer-assisted Tongue Diagnosis Solutions for Key Problems [J]. Computer Science, 2021, 48(7): 256-269.
[15] WU Shan-jie, WANG Xin. Prediction of Tectonic Coal Thickness Based on AGA-DBSCAN Optimized RBF Neural Networks [J]. Computer Science, 2021, 48(7): 308-315.
Full text



No Suggested Reading articles found!