一种用于微表情自动识别的三维卷积神经网络进化方法

doi:10.11896/jsjkx.190700009

摘要/Abstract

摘要： 由于微表情持续时间短、动作幅度小, 因此微表情自动识别一直是一个具有挑战性的问题。针对上述问题, 提出一种用于微表情识别的三维卷积神经网络进化(Three-Dimensional Convolutional Neural Network Evolution, C3DEvol)方法。该方法使用能有效提取动态信息的三维卷积神经网络(Three-Dimensional Convolutional Neural Network, C3D)来提取微表情在时域和空域上的特征;同时使用具有全局搜索和优化能力的遗传算法对C3D的网络结构进行优化, 以获取最优的C3D网络结构和避免局部优化。利用CASME2数据集在带有两块NVIDIA Titan X GPU的工作站上开展了实验, 结果表明C3DEvol微表情自动识别的准确率达到63.71%, 优于现有的微表情自动识别方法。

关键词: 三维卷积神经网络, 特征提取, 网络结构优化, 微表情识别, 遗传算法

Abstract: Due to the short duration of micro-expressions and the small amplitude of motion, the automatic recognition of micro-expressions is still a challenging problem.Aiming at the problems, this paper proposes a Three-Dimensional Convolutional Neural Network Evolution (C3DEvol) method for micro-expression recognition.In the C3DEvol, three-dimensional Convolutional Neural Network (C3D) which can extract dynamic information effectively is used to extract micro-expression features in time domain and space domain.At the same time, the genetic algorithm with the capabilities of global search and optimization is used to optimize the network structure of C3D in order to obtain the optimal network structure and avoid local optimization.Experiments are performed on a workstation with two NVIDIA Titan X GPUs using the CASME2 dataset.Experiments show that the accuracy of C3DEvol micro-expression automatic recognition reaches 63.71%, which is better than the existing micro-expression automatic recognition method.

Key words: Feature extraction, Genetic algorithm, Micro-expression recognition, Network structure optimization, Three-dimensional convolutional neural network

中图分类号:

TP391

梁正友, 何景琳, 孙宇. 一种用于微表情自动识别的三维卷积神经网络进化方法[J]. 计算机科学, 2020, 47(8): 227-232. https://doi.org/10.11896/jsjkx.190700009

LIANG Zheng-you, HE Jing-lin, SUN Yu. Three-dimensional Convolutional Neural Network Evolution Method for Facial Micro-expression Auto-recognition[J]. Computer Science, 2020, 47(8): 227-232. https://doi.org/10.11896/jsjkx.190700009

参考文献

[1]CORNEANU C, OLIU M, COHN J F, et al.Survey on RGB, 3D, Thermal, and Multimodal Approaches for Facial Expression Recognition:History, Trends, and Affect-related Applications[J].IEEE Transactions on Pattern Analysis & Machine Intelligence, 2016, 38(8):1548-1568.
[2]RUSSELL T A, CHU E, PHILLIPS M L.A pilot study to in-vestigate the effectiveness of emotion recognition remediation in schizophrenia using the micro-expression training tool[J].British Journal of Clinical Psychology, 2006, 45(4):579-583.
[3]PFISTER T, LI X, ZHAO G, et al.Recognising spontaneous facial micro-expressions[C]∥2011 IEEE International Conference on Computer Vision (ICCV).IEEE, 2011:1449-1456.
[4]WANG Y, SEE J, PHAN R C W, et al.Lbp with sixintersection points:Reducing redundant information in lbp-top for micro-expression recognition[C]∥Asian Conference on Computer Vision.Cham:Springer, 2014:525-537.
[5]WANG Y, SEE J, PHAN C W, et al.Efficient Spatio-Temporal Local Binary Patterns for Spontaneous Facial Micro-Expression Recognition[J].Plos One, 2015, 10(5):1-20.
[6]HUANG X, ZHAO G, HONG X, et al.Spontaneous facial mi-cro-expression analysis using spatiotemporal completed local quantized patterns[J].Neurocomputing, 2016, 175:564-578.
[7]LIU Y J, ZHANG J K, YAN W J, et al.A main directional mean optical flow feature for spontaneous micro-expression recognition[J].IEEE Transactions on Affective Computing, 2016, 7(4):299-310.
[8]HUANG W, FAN L, HARANDI M, et al.Toward Efficient Action Recognition:Principal Backpropagation for Training Two-Stream Networks[J].IEEE Transactions on Image Processing, 2019, 28(4):1773-1782.
[9]YOUNG T, HAZARIKA D, PORIA S, et al.Recent trends indeep learning based natural language processing[J].IEEE Computational Intelligence Magazine, 2018, 13(3):55-75.
[10]XIONG W, WU L, ALLEVA F, et al.The Microsoft 2017 conversational speech recognition system[C]∥2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).IEEE, 2018:5934-5938.
[11]TRAN D, BOURDEV L, FERGUS R, et al.Learning spatiotemporal features with 3d convolutional networks[C]∥Proceedings of the IEEE International Conference on Computer Vision.2015:4489-4497.
[12]KIM D H, BADDAR W J, JANG J, et al.Multi-objective based spatio-temporal feature representation learning robust to expression intensity variations for facial expression recognition[J].IEEE Transactions on Affective Computing, 2017, 10(2):223-236.
[13]PENG M, WANG C, CHEN T, et al.Dual temporal scale convolutional neural network for micro-expression recognition[J].Frontiers in Psychology, 2017, 8:1-12.
[14]PETKE J, HARALDSSON S O, HARMAN M, et al.Genetic Improvement of Software:A Comprehensive Survey[J].IEEE Transactions on Evolutionary Computation, 2018, 22(3):415-432.
[15]KIM Y H, YOON Y, GEEM Z W.A comparison study of harmony search and genetic algorithm for the max-cut problem[J].Swarm and Evolutionary Computation, 2019, 44:130-135.
[16]METEVIER B, SAINI A K, SPECTOR L.Lexicase SelectionBeyond Genetic Programming[M]∥Genetic Programming Theo-ry and Practice XVI.Cham:Springer, 2019:123-136.
[17]NGUYEN S, ZHANG M, JOHNSTON M, et al.Genetic Programming for Job Shop Scheduling[M]∥Evolutionary and Swarm Intelligence Algorithms.Cham:Springer, 2019:143-167.
[18]SHANMUGAPRIYA K, MALAR R M S M.An EffectiveTechnique to Track Objects with the Aid of Rough Set Theory and Evolutionary Programming[J].Journal of Intelligent Systems, 2019, 28(1):1-13.
[19]CHEN Z, XIA J, BAI J, et al.Feature extraction algorithm based on evolutionary deep learning[J].Computer Science, 2015, 42(11):288-292.
[20]IJJINA E P, CHALAVADI K M.Human action recognitionusing genetic algorithms and convolutional neural networks[J].Pattern Recognition, 2016, 59:199-212.
[21]OULLETTE R, BROWNE M, HIRASAWA K.Genetic algo-rithm optimization of a convolutional neural network for autonomous crack detection[C]∥Proceedings of the 2004 Congress on Evolutionary Computation (IEEE Cat.No.04TH8753).IEEE, 2004:516-521.
[22]RIKHTEGAR A, POOYAN M, MANZURI-SHALMANI M T.Genetic algorithm-optimised structure of convolutional neural network for face recognition applications[J].IET Computer Vision, 2016, 10(6):559-566.
[23]JI S, XU W, YANG M, et al.3D convolutional neural networks for human action recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 35(1):221-231.
[24]YAN W J, WU Q, LIU Y J, et al.CASME database:a dataset of spontaneous micro-expressions collected from neutralized faces[C]∥2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).IEEE, 2013:1-7.
[25]YAN W J, LI X, WANG S J, et al.CASME II:An Improved Spontaneous Micro-Expression Database and the Baseline Evaluation[J].Plos One, 2014, 9(1):1-8.
[26]LI X, PFISTER T, HUANG X, et al.A spontaneous micro-expression database:Inducement, collection and baseline[C]∥2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG).IEEE, 2013:1-6.
[27]VIODA P.Rapid object detection using a boosted cascade ofsimple features[C]∥Proc.IEEE CVPR 2001.2001:905-910.
[28]ZHOU Z, ZHAO G, PIETIKINEN M.Towards a practical lipreading system[C]∥2011 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).IEEE, 2011:137-144.
[29]LIONG S T, SEE J, WONG K S, et al.Less is more:Micro-expression recognition from video using apex frame[J].Signal Processing:Image Communication, 2018, 62:82-92.
[30]XU F, ZHANG J, WANG J Z.Micro-expression identificationand categorization using a facial dynamics map[J].IEEE Tran-sactions on Affective Computing, 2017, 8(2):254-267.
[31]HE J, HU J F, LU X, et al.Multi-task mid-level feature learning for micro-expression recognition[J].Pattern Recognition, 2017, 66:44-52.
[32]PATEL D, HONG X, ZHAO G.Selective deep features for micro-expression recognition[C]∥2016 23rd International Confe-rence on Pattern Recognition (ICPR).IEEE, 2016:2258-2263.

相关文章 15

[1]	张源, 康乐, 宫朝辉, 张志鸿. 基于Bi-LSTM的期货市场关联交易行为检测方法 Related Transaction Behavior Detection in Futures Market Based on Bi-LSTM 计算机科学, 2022, 49(7): 31-39. https://doi.org/10.11896/jsjkx.210400304
[2]	曾志贤, 曹建军, 翁年凤, 蒋国权, 徐滨. 基于注意力机制的细粒度语义关联视频-文本跨模态实体分辨 Fine-grained Semantic Association Video-Text Cross-modal Entity Resolution Based on Attention Mechanism 计算机科学, 2022, 49(7): 106-112. https://doi.org/10.11896/jsjkx.210500224
[3]	程成, 降爱莲. 基于多路径特征提取的实时语义分割方法 Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction 计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[4]	刘伟业, 鲁慧民, 李玉鹏, 马宁. 指静脉识别技术研究综述 Survey on Finger Vein Recognition Research 计算机科学, 2022, 49(6A): 1-11. https://doi.org/10.11896/jsjkx.210400056
[5]	杨浩雄, 高晶, 邵恩露. 考虑一单多品的外卖订单配送时间的带时间窗的车辆路径问题 Vehicle Routing Problem with Time Window of Takeaway Food ConsideringOne-order-multi-product Order Delivery 计算机科学, 2022, 49(6A): 191-198. https://doi.org/10.11896/jsjkx.210400005
[6]	张嘉淏, 刘峰, 齐佳音. 一种基于Bottleneck Transformer的轻量级微表情识别架构 Lightweight Micro-expression Recognition Architecture Based on Bottleneck Transformer 计算机科学, 2022, 49(6A): 370-377. https://doi.org/10.11896/jsjkx.210500023
[7]	高元浩, 罗晓清, 张战成. 基于特征分离的红外与可见光图像融合算法 Infrared and Visible Image Fusion Based on Feature Separation 计算机科学, 2022, 49(5): 58-63. https://doi.org/10.11896/jsjkx.210200148
[8]	左杰格, 柳晓鸣, 蔡兵. 基于图像分块与特征融合的户外图像天气识别 Outdoor Image Weather Recognition Based on Image Blocks and Feature Fusion 计算机科学, 2022, 49(3): 197-203. https://doi.org/10.11896/jsjkx.201200263
[9]	李星燃, 张立言, 姚树婧. 结合特征融合和注意力机制的微表情识别方法 Micro-expression Recognition Method Combining Feature Fusion and Attention Mechanism 计算机科学, 2022, 49(2): 4-11. https://doi.org/10.11896/jsjkx.210900028
[10]	沈彪, 沈立炜, 李弋. 空间众包任务的路径动态调度方法 Dynamic Task Scheduling Method for Space Crowdsourcing 计算机科学, 2022, 49(2): 231-240. https://doi.org/10.11896/jsjkx.210400249
[11]	任首朋, 李劲, 王静茹, 岳昆. 基于集成回归决策树的lncRNA-疾病关联预测方法 Ensemble Regression Decision Trees-based lncRNA-disease Association Prediction 计算机科学, 2022, 49(2): 265-271. https://doi.org/10.11896/jsjkx.201100132
[12]	张师鹏, 李永忠. 基于降噪自编码器和三支决策的入侵检测方法 Intrusion Detection Method Based on Denoising Autoencoder and Three-way Decisions 计算机科学, 2021, 48(9): 345-351. https://doi.org/10.11896/jsjkx.200500059
[13]	冯霞, 胡志毅, 刘才华. 跨模态检索研究进展综述 Survey of Research Progress on Cross-modal Retrieval 计算机科学, 2021, 48(8): 13-23. https://doi.org/10.11896/jsjkx.200800165
[14]	张丽倩, 李孟航, 高珊珊, 张彩明. 面向计算机辅助舌诊关键问题的解决方案综述 Summary of Computer-assisted Tongue Diagnosis Solutions for Key Problems 计算机科学, 2021, 48(7): 256-269. https://doi.org/10.11896/jsjkx.200800223
[15]	吴善杰, 王新. 基于AGA-DBSCAN优化的RBF神经网络构造煤厚度预测方法 Prediction of Tectonic Coal Thickness Based on AGA-DBSCAN Optimized RBF Neural Networks 计算机科学, 2021, 48(7): 308-315. https://doi.org/10.11896/jsjkx.200800110

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed