计算机科学

计算机科学 ›› 2024, Vol. 51 ›› Issue (11A): 231000053-7.doi: 10.11896/jsjkx.231000053

• 图像处理&多媒体技术 • 上一篇    下一篇

基于ME-ResNet人脸微表情识别方法江

江盛, 朱建鸿   

  1. 江南大学轻工过程先进控制教育部重点实验室 江苏 无锡 214122
  • 出版日期:2024-11-16 发布日期:2024-11-13
  • 通讯作者: 朱建鸿(466597129@qq.com)
  • 作者简介:(466597129@qq.com)
  • 基金资助:
    国家自然科学基金(61973139)

Face Micro-expression Recognition Method Based on ME-ResNet

JIANG Sheng, ZHU Jianhong   

  1. Key Laboratory of Advanced Process Control for Light Industry,Ministry of Education,Jiangnan University,Wuxi,Jiangsu 214122,China
  • Online:2024-11-16 Published:2024-11-13
  • About author:JIANG Sheng,born in 1998,postgra-duate.His main research interests include computer vision and micro-expression recognition.
    ZHU Jianhong,born in 1964,Ph.D,professor,Ph.D supervisor.His main research interests include visual Internet of Things and deep learning.
  • Supported by:
    National Natural Science Foundation of China(61973139).

PDF (PC)

156

摘要： 人脸微表情具有持续时间短、动作幅度小的特点。数据集样本量较少等因素也给微表情识别带来了巨大挑战。针对上述问题,提出一种基于ME-ResNet残差网络的微表情识别方法。首先,在预处理阶段,等间隔提取微表情视频片段起始帧至顶点帧之间的关键帧序列,利用改进Farneback光流法提取微表情关键帧序列的面部光流运动特征;接着,构建基于3D卷积的ResNet50网络,并将空间通道注意力CBAM机制加入网络Bottleneck模块,以增强网络对面部关键运动特征的聚焦学习能力,并构建ME-ResNet网络模型,将所提取的面部光流运动特征送入网络进行训练;最后,使用数据增强增加网络训练样本量,将ME-ResNet网络模型用于微表情识别任务,并在CASMEII,SMIC和SAMM数据集上进行实验验证,所提算法识别率达到了84.42%,72.56%,70.41%,与其他算法相比具有较高的识别能力。

关键词: 微表情识别, Farneback光流法, 卷积神经网络, 运动特征, 数据增强

Abstract: Face micro-expressions have the characteristics of short duration and small amplitude of movement.Factors such as the small sample size of dataset also bring great challenges to micro-expression recognition.To solve these problems,this paper proposes a micro-expression recognition method based on ME-ResNet residual network.First,in the pre-processing stage,extract the key frame sequence between the start frame and the vertex frame in the micro-expression video clip at equal intervals and then,use the improved Farneback optical flow method to extract the motion features of the micro-expression key frame sequence.Se-cond,construct a ResNet50 network based on 3D convolution and add the spatial channel attention CBAM mechanism to the network Bottleneck module,so as to enhance the ability to focus on key facial motor features.Next,construct the ME-ResNet network model and sent the extracted facial optical flow motion features to the network for training.Finally,use the data enhancement to increase the sample size of network training and apply the ME-ResNet network model to micro-expression recognition tasks.Also,experimental results on CASME II,SMIC and SAMM datasets show that the recognition rate of the proposed algorithm reaches 84.42%,72.56% and 70.41% respectively.It has higher recognition ability compared with other algorithms.

Key words: Micro-expression recognition, Farneback optical flow method, Convolutional neural networks, Motion features, Data augmentation

中图分类号: 

  • TP391
[1]PENG Y W.Research on the application of micro-expressionanalysis technology in investigation and interrogation[J].Chinese Journal of Criminal Law,2015(2):95-103.
[2]ZHU C L,CHEN X Y,ZHANG J X,et al.Comparison of Ecological Micro-Expression Recognition in Patients with Depression and Healthy Individuals[J].Frontiers in Behavioral Neuroscience,2017,11(6):199-224.
[3]PFISTER T,LI X,ZHAO G,et al.Recognising Spontaneous Facial Micro-Expressions[C]//2011 International Conference on Computer Vision.IEEE,2011:1449-1456.
[4]HUANG X H,ZHAO G Y,HONG X P,et al.Spontaneous facial micro-expression analysis using spatiotemporal completed local quantized patterns[J].Neurocomputing,2016,175:564-578.
[5]SHREVE M,GODAVARTHY S,MANOHAR V,et al.To-wards Macro-and Micro-Expression Spotting in Video Using Strain Patterns[C]//2009 Workshop on Applications of Computer Vision(WACV).IEEE,2009:1-6.
[6]LIU Y J,ZHANG J K,YAN W J,et al.A main directional mean optical flow feature for spontaneous micro-expression recognition[J].IEEE Transactions on Affective Computing,2016,7(4):299-310.
[7]WANG S J,LI B J,LIU Y J,et al.Micro-Expression Recognition with Small Sample Size by Transferring Long-Term Convolutional Neural Network[J].Neurocomputing,2018,312:251-262.
[8]PENG M,WANG C,BI T,et al.A Novel Apex-Time Network for Cross-Dataset Micro-Expression Recognition[C]//2019 8th International Conference on Affective Computing and Intelligent Interaction(ACII).2024.
[9]GAN Y S,LIONG S T,YAU W C,et al.OFF-ApexNet on micro-expression recognition system[J].Signal Processing:Image Communication,2019,74:129-139.
[10]GAN Y S,LIONG S T,YAU W C,et al.OFF-ApexNet on micro-expression recognition system[J].Signal Processing:Image Communication,2019,74:129-139.
[11]LIANG Z Y,LIU D Z,SUN Y.Micro-expression recognitionmethod combining transfer learning and separable three-dimensional convolution[J].Computer Engineering,2022,48(1):228-235.
[12]SUN S K,FAN J,SUN Z Q,et al.A review of image data enhancement based on deep learning[J/OL].Computer Science:1-23[2023-10-06].http://kns.cnki.net/kcms/detail/50.1075.TP.20230926.0857.012.html.
[13]FARNEBACK,BIGUN J,GUSTAVSSON T,Two-Frame Motion Estimation Based on Polynomial Expansion[J].Image Analysis Volume,2003:363-370.
[14]LI B ,LIMA D.Facial expression recognition via ResNet-50[J].International Journal of Cognitive Computing in Engineering,2021.DOI:10.1016/j.ijcce.2021.02.002.
[15]LIU S H ,REN Y S ,LI L T ,et al.Micro-expression recognition based on SqueezeNet and C3D[J].Multimedia Systems,2022,28(6):2227-2236.
[16] WANG C,CHANG J.CSI Cross-domain Gesture RecognitionMethod Based on 3D Convolutional Neutal Network[J].Computer Science,2021,48(8):322-327.
[17]WANG C,CHANG J.CSI Cross-domain Gesture Recognition Method Based on 3D Convolutional Neural Network[J].Computer Science,2021,48(8):322-327.
[18]WOO S,PARK J,LEE J Y,et al.CBAM:Convolutional BlockAttention Module[C]//ECCV 2018.Cham:Springer,2018.
[19]LI X R,ZHANG L Y,YAO S J.Micro-expression Recognition Method Combining Feature Fusion and AttentionMechanim[J].Computer Science,2022,49(2):4-11.
[20]LI X R,ZHANG L Y,YAO S J.Micro-expression Recognition Method Combining Feature Fusion and Attention Mechanism[J].Computer Science,2022,49(2):4-11.
[21]ZHU X,CHENG D,ZHANG Z,et al.An Empirical Study ofSpatial Attention Mechanisms in Deep Networks[C]//2019 IEEE/CVF International Conference on Computer Vision(ICCV).IEEE,2020.
[22]DAVISON A K,LANSLEY C,COSTEN N,et al.SAMM:A spontaneous micro-facial movement dataset[J].IEEE Transactions on Affective Computing,2018,9(1):116-129.
[23]YAN W J,LI X B,WANG S J,et al.CASME II:An improved spontaneous micro-expression database and the baseline evaluation[J].PLoS One,2014,9(1):e86041.
[24]DAVISON A K,LANSLEY C,COSTEN N,et al.SAMM:Aspontaneous micro-facial movement dataset[J].IEEE Transactions on Affective Computing,2018,9(1):116-129.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发