一种用于微表情自动识别的三维卷积神经网络进化方法

doi:10.11896/jsjkx.190700009

Abstract

Abstract: Due to the short duration of micro-expressions and the small amplitude of motion, the automatic recognition of micro-expressions is still a challenging problem.Aiming at the problems, this paper proposes a Three-Dimensional Convolutional Neural Network Evolution (C3DEvol) method for micro-expression recognition.In the C3DEvol, three-dimensional Convolutional Neural Network (C3D) which can extract dynamic information effectively is used to extract micro-expression features in time domain and space domain.At the same time, the genetic algorithm with the capabilities of global search and optimization is used to optimize the network structure of C3D in order to obtain the optimal network structure and avoid local optimization.Experiments are performed on a workstation with two NVIDIA Titan X GPUs using the CASME2 dataset.Experiments show that the accuracy of C3DEvol micro-expression automatic recognition reaches 63.71%, which is better than the existing micro-expression automatic recognition method.

Key words: Micro-expression recognition, Genetic algorithm, Three-dimensional convolutional neural network, Feature extraction, Network structure optimization

CLC Number:

TP391

LIANG Zheng-you, HE Jing-lin, SUN Yu. Three-dimensional Convolutional Neural Network Evolution Method for Facial Micro-expression Auto-recognition[J].Computer Science, 2020, 47(8): 227-232.

References 32

[1]	CORNEANU C, OLIU M, COHN J F, et al.Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition:History, Trends, and Affect-related Applications[J].IEEE Transactions on Pattern Analysis & Machine Intelligence, 2016, 38(8):1548-1568.
[2]	RUSSELL T A, CHU E, PHILLIPS M L.A pilot study to in-vestigate the effectiveness of emotion recognition remediation in schizophrenia using the micro-expression training tool[J].British Journal of Clinical Psychology, 2006, 45(4):579-583.
[3]	PFISTER T, LI X, ZHAO G, et al.Recognising spontaneous facial micro-expressions[C]∥2011 IEEE International Conference on Computer Vision (ICCV).IEEE, 2011:1449-1456.
[4]	WANG Y, SEE J, PHAN R C W, et al.Lbp with sixintersection points:Reducing redundant information in lbp-top for micro-expression recognition[C]∥Asian Conference on Computer Vision.Cham:Springer, 2014:525-537.
[5]	WANG Y, SEE J, PHAN C W, et al.Efficient Spatio-Temporal Local Binary Patterns for Spontaneous Facial Micro-Expression Recognition[J].Plos One, 2015, 10(5):1-20.
[6]	HUANG X, ZHAO G, HONG X, et al.Spontaneous facial mi-cro-expression analysis using spatiotemporal completed local quantized patterns[J].Neurocomputing, 2016, 175:564-578.
[7]	LIU Y J, ZHANG J K, YAN W J, et al.A main directional mean optical flow feature for spontaneous micro-expression recognition[J].IEEE Transactions on Affective Computing, 2016, 7(4):299-310.
[8]	HUANG W, FAN L, HARANDI M, et al.Toward Efficient Action Recognition:Principal Backpropagation for Training Two-Stream Networks[J].IEEE Transactions on Image Processing, 2019, 28(4):1773-1782.
[9]	YOUNG T, HAZARIKA D, PORIA S, et al.Recent trends indeep learning based natural language processing[J].IEEE Computational Intelligence Magazine, 2018, 13(3):55-75.
[10]	XIONG W, WU L, ALLEVA F, et al.The Microsoft 2017 conversational speech recognition system[C]∥2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).IEEE, 2018:5934-5938.
[11]	TRAN D, BOURDEV L, FERGUS R, et al.Learning spatiotemporal features with 3d convolutional networks[C]∥Proceedings of the IEEE International Conference on Computer Vision.2015:4489-4497.
[12]	KIM D H, BADDAR W J, JANG J, et al.Multi-objective based spatio-temporal feature representation learning robust to expression intensity variations for facial expression recognition[J].IEEE Transactions on Affective Computing, 2017, 10(2):223-236.
[13]	PENG M, WANG C, CHEN T, et al.Dual temporal scale convolutional neural network for micro-expression recognition[J].Frontiers in Psychology, 2017, 8:1-12.
[14]	PETKE J, HARALDSSON S O, HARMAN M, et al.Genetic Improvement of Software:A Comprehensive Survey[J].IEEE Transactions on Evolutionary Computation, 2018, 22(3):415-432.
[15]	KIM Y H, YOON Y, GEEM Z W.A comparison study of harmony search and genetic algorithm for the max-cut problem[J].Swarm and Evolutionary Computation, 2019, 44:130-135.
[16]	METEVIER B, SAINI A K, SPECTOR L.Lexicase SelectionBeyond Genetic Programming[M]∥Genetic Programming Theo-ry and Practice XVI.Cham:Springer, 2019:123-136.
[17]	NGUYEN S, ZHANG M, JOHNSTON M, et al.Genetic Programming for Job Shop Scheduling[M]∥Evolutionary and Swarm Intelligence Algorithms.Cham:Springer, 2019:143-167.
[18]	SHANMUGAPRIYA K, MALAR R M S M.An EffectiveTechnique to Track Objects with the Aid of Rough Set Theory and Evolutionary Programming[J].Journal of Intelligent Systems, 2019, 28(1):1-13.
[19]	CHEN Z, XIA J, BAI J, et al.Feature extraction algorithm based on evolutionary deep learning[J].Computer Science, 2015, 42(11):288-292.
[20]	IJJINA E P, CHALAVADI K M.Human action recognitionusing genetic algorithms and convolutional neural networks[J].Pattern Recognition, 2016, 59:199-212.
[21]	OULLETTE R, BROWNE M, HIRASAWA K.Genetic algo-rithm optimization of a convolutional neural network for autonomous crack detection[C]∥Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat.No.04TH8753).IEEE, 2004:516-521.
[22]	RIKHTEGAR A, POOYAN M, MANZURI-SHALMANI M T.Genetic algorithm-optimised structure of convolutional neural network for face recognition applications[J].IET Computer Vision, 2016, 10(6):559-566.
[23]	JI S, XU W, YANG M, et al.3D convolutional neural networks for human action recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(1):221-231.
[24]	YAN W J, WU Q, LIU Y J, et al.CASME database:a dataset of spontaneous micro-expressions collected from neutralized faces[C]∥2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).IEEE, 2013:1-7.
[25]	YAN W J, LI X, WANG S J, et al.CASME II:An Improved Spontaneous Micro-Expression Database and the Baseline Evaluation[J].Plos One, 2014, 9(1):1-8.
[26]	LI X, PFISTER T, HUANG X, et al.A spontaneous micro-expression database:Inducement, collection and baseline[C]∥2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).IEEE, 2013:1-6.
[27]	VIODA P.Rapid object detection using a boosted cascade ofsimple features[C]∥Proc.IEEE CVPR 2001.2001:905-910.
[28]	ZHOU Z, ZHAO G, PIETIKINEN M.Towards a practical lipreading system[C]∥2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE, 2011:137-144.
[29]	LIONG S T, SEE J, WONG K S, et al.Less is more:Micro-expression recognition from video using apex frame[J].Signal Processing:Image Communication, 2018, 62:82-92.
[30]	XU F, ZHANG J, WANG J Z.Micro-expression identificationand categorization using a facial dynamics map[J].IEEE Tran-sactions on Affective Computing, 2017, 8(2):254-267.
[31]	HE J, HU J F, LU X, et al.Multi-task mid-level feature learning for micro-expression recognition[J].Pattern Recognition, 2017, 66:44-52.
[32]	PATEL D, HONG X, ZHAO G.Selective deep features for micro-expression recognition[C]∥2016 23rd International Confe-rence on Pattern Recognition (ICPR).IEEE, 2016:2258-2263.

Related Articles 15

[1]	GAO Ji-xu, WANG Jun. Multi-edge Collaborative Computing Unloading Scheme Based on Genetic Algorithm [J]. Computer Science, 2021, 48(1): 72-80.
[2]	LIU Yang, JIN Zhong. Fine-grained Image Recognition Method Combining with Non-local and Multi-region Attention Mechanism [J]. Computer Science, 2021, 48(1): 197-203.
[3]	JI Shun-hui, ZHANG Peng-cheng. Test Case Generation Approach for Data Flow Based on Dominance Relations [J]. Computer Science, 2020, 47(9): 40-46.
[4]	BAO Yu-xuan, LU Tian-liang, DU Yan-hui. Overview of Deepfake Video Detection Technology [J]. Computer Science, 2020, 47(9): 283-292.
[5]	WANG Liang, ZHOU Xin-zhi, YNA Hua. Real-time SIFT Algorithm Based on GPU [J]. Computer Science, 2020, 47(8): 105-111.
[6]	DONG Ming-gang, HUANG Yu-yang, JING Chao. K-Nearest Neighbor Classification Training Set Optimization Method Based on Genetic Instance and Feature Selection [J]. Computer Science, 2020, 47(8): 178-184.
[7]	YANG De-cheng, LI Feng-qi, WANG Yi, WANG Sheng-fa, YIN Hui-shu. Intelligent 3D Printing Path Planning Algorithm [J]. Computer Science, 2020, 47(8): 267-271.
[8]	YANG Wei-chao, GUO Yuan-bo, LI Tao, ZHU Ben-quan. Method Based on Traffic Fingerprint for IoT Device Identification and IoT Security Model [J]. Computer Science, 2020, 47(7): 299-306.
[9]	FENG Bing-chao and WU Jing-li. Partheno-genetic Algorithm for Solving Static Rebalance Problem of Bicycle Sharing System [J]. Computer Science, 2020, 47(6A): 114-118.
[10]	YAO Min. Multi-population Genetic Algorithm for Multi-skill Resource-constrained ProJect Scheduling Problem [J]. Computer Science, 2020, 47(6A): 124-129.
[11]	LAN Zhang-li, SHEN De-xing, CAO Juan and ZHANG Yu-xin. Content-independent Method for Basis Image Extraction and Image Reconstruction [J]. Computer Science, 2020, 47(6A): 226-229.
[12]	ZHOU Li-peng, MENG Li-min, ZHOU Lei, JIANG Wei and DONG Jian-ping. Fall Detection Algorithm Based on BP Neural Network [J]. Computer Science, 2020, 47(6A): 242-246.
[13]	YUAN De-yu, ZHANG Yi-fan, GAO Jian and SUN Hai-chun. Abnormal User Detection Method in Sina Weibo Based on User Feature Extraction [J]. Computer Science, 2020, 47(6A): 364-368.
[14]	BAO Zhen-shan, GUO Jun-nan, XIE Yuan and ZHANG Wen-bo. Model for Stock Price Trend Prediction Based on LSTM and GA [J]. Computer Science, 2020, 47(6A): 467-473.
[15]	MA Chuang, LV Xiao-fei and LIANG yan-ming. Agricultural Product Quality Classification Based on GA-SVM [J]. Computer Science, 2020, 47(6A): 517-520.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 10

[1]	LEI Li-hui and WANG Jing. Parallelization of LTL Model Checking Based on Possibility Measure[J]. Computer Science, 2018, 45(4): 71 -75 .
[2]	SUN Qi, JIN Yan, HE Kun and XU Ling-xuan. Hybrid Evolutionary Algorithm for Solving Mixed Capacitated General Routing Problem[J]. Computer Science, 2018, 45(4): 76 -82 .
[3]	ZHANG Jia-nan and XIAO Ming-yu. Approximation Algorithm for Weighted Mixed Domination Problem[J]. Computer Science, 2018, 45(4): 83 -88 .
[4]	WU Jian-hui, HUANG Zhong-xiang, LI Wu, WU Jian-hui, PENG Xin and ZHANG Sheng. Robustness Optimization of Sequence Decision in Urban Road Construction[J]. Computer Science, 2018, 45(4): 89 -93 .
[5]	SHI Wen-jun, WU Ji-gang and LUO Yu-chun. Fast and Efficient Scheduling Algorithms for Mobile Cloud Offloading[J]. Computer Science, 2018, 45(4): 94 -99 .
[6]	ZHOU Yan-ping and YE Qiao-lin. L1-norm Distance Based Least Squares Twin Support Vector Machine[J]. Computer Science, 2018, 45(4): 100 -105 .
[7]	LIU Bo-yi, TANG Xiang-yan and CHENG Jie-ren. Recognition Method for Corn Borer Based on Templates Matching in Muliple Growth Periods[J]. Computer Science, 2018, 45(4): 106 -111 .
[8]	GENG Hai-jun, SHI Xin-gang, WANG Zhi-liang, YIN Xia and YIN Shao-ping. Energy-efficient Intra-domain Routing Algorithm Based on Directed Acyclic Graph[J]. Computer Science, 2018, 45(4): 112 -116 .
[9]	CUI Qiong, LI Jian-hua, WANG Hong and NAN Ming-li. Resilience Analysis Model of Networked Command Information System Based on Node Repairability[J]. Computer Science, 2018, 45(4): 117 -121 .
[10]	WANG Zhen-chao, HOU Huan-huan and LIAN Rui. Path Optimization Scheme for Restraining Degree of Disorder in CMT[J]. Computer Science, 2018, 45(4): 122 -125 .

Three-dimensional Convolutional Neural Network Evolution Method for Facial Micro-expression Auto-recognition

PDF (PC)

Abstract

Cite this article

share this article

References 32

Related Articles 15

Metrics

Comments

Recommended 10