计算机科学

计算机科学 ›› 2026, Vol. 53 ›› Issue (8): 174-181.doi: 10.11896/jsjkx.250500012

• 计算机图形学 & 多媒体 • 上一篇    下一篇

基于光流分区特征融合的多层级轻量化微表情识别

胡长玉, 范馨予, 张智, 丁子胥, 张正悦, 彭菊红   

  1. 湖北大学人工智能学院 武汉 430062
    湖北大学智能感知系统与安全教育部重点实验室 武汉 430062
  • 收稿日期:2025-05-06 修回日期:2025-09-22 出版日期:2026-08-15 发布日期:2026-08-17
  • 通讯作者: 彭菊红(juhongpeng@hubu.edu.cn)
  • 作者简介:(1181973936@qq.com)
  • 基金资助:
    湖北省自然科学基金(JCZRQN202500839)

Hierarchical Lightweight Micro-expression Recognition Based on Optical Flow Partitioned FeatureFusion

HU Changyu, FAN Xinyu, ZHANG Zhi, DING Zixu, ZHANG Zhengyue, PENG Juhong   

  1. School of Artificial Intelligence, Hubei University, Wuhan 430062, China
    Key Laboratory of Intelligent Perception Systems and Security, Ministry of Education, Hubei University, Wuhan 430062, China
  • Received:2025-05-06 Revised:2025-09-22 Published:2026-08-15 Online:2026-08-17
  • About author:HU Changyu,born in 2002,postgra-duate,is a member of CCF(No.X4218G).Her main research interests include computer vision and micro-expression recognition.
    PENG Juhong,born in 1978,associate professor.Her main research interests include information processing and AI algorithms.
  • Supported by:
    Natural Science Foundation of Hubei Province,China(JCZRQN202500839).

PDF (PC)

108

摘要： 针对微表情因难以捕捉、数据集有限及训练耗时而占用大量算力等问题,设计一种简洁高效的基于光流分区特征融合的多层级轻量化模型(Hierarchical Lightweight Model with Feature Fusion Based on Optical Flow Partitioning,HLFM-OFP),用于微表情识别。首先对数据集进行预处理,通过TV-L1方法提取光流信息,分区域剪裁特征图,并送入轻量化网络进行多层级特征融合;从内在特征中提取额外的特征图,实现对底层信息更加全面的表示。随后通过时空特征融合模块(STModel)协同捕捉输入的动静态特征,然后进行分类。在复合数据集上进行验证,模型参数量仅为2.8×106,训练时间减少88.45%。准确率、UF1、UAR分别为94.23%,91.23%,91.66%。实验表明,所提模型有效减少了模型参数量,同时提高了识别精度,且在SMIC和SAMM这两个具有更强的类别不平衡与复杂场景的数据集上表现优异,具有较强的鲁棒性。

关键词: 微表情识别, 光流分区特征融合, 多层级轻量化设计, 动静态特征融合, H-Swish激活函数

Abstract: Micro-expression recognition remains a challenging task due to the difficulty in capturing subtle facial movements,li-mited dataset sizes,and high computational costs.To address these issues,this paper proposes a lightweight and efficient model,named HLFM-OFP(Hierarchical Lightweight Model with Feature Fusion Based on Optical Flow Partitioning).The model employs TV-L1 optical flow for motion extraction and utilizes a hierarchical lightweight architecture to enrich the representation of low-level features.STModel(Spatiotemporal Feature Fusion Module)is further introduced to jointly capture spatial and temporal dynamics.Experimental results on composite datasets demonstrate that HLFM-OFP contains only 2.8 million parameters and reduces training time by 88.45%,while achieving 94.23% accuracy,91.23% UF1,and 91.66% UAR.Experiments show that the proposed model reduces parameters while improving accuracy,and demonstrates strong robustness on the more challenging SMIC and SAMM datasets.

Key words: Micro-expression recognition, Optical flow partitioned feature fusion, Multi-level lightweight design, Dynamic and static feature fusion, H-Swish loss function

中图分类号: 

  • TP391.41
[1] OJALA T,PIETIKÄINEN M,MÄENPÄÄ T.MultiresolutionGray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns[J].IEEE Transactions on Pattern Ana-lysis and Machine Intelligence,2002,24(7):971-987.
[2] PFISTER T,LI X,ZHAO G,et al.Recognising spontaneous facial micro-expressions[C]//International Conference on Computer Vision.IEEE,2011:1449-1456.
[3] HUANG X,ZHAO G,HONG X,et al.Spontaneous facial mi-cro-expression analysis using Spatiotemporal Completed Local Quantized Patterns[J].Neurocomputing,2016,175:564-578.
[4] LIONG S,SEE J,PHAN C R,et al.Spontaneous subtle expression detection and recognition based on facial strain[J].Signal Processing:Image Communication,2016,47:170-182.
[5] LIONG S T,SEE J,WONG K S,et al.Less is more:Micro-expression recognition from video using apex frame[J].Signal Processing:Image Communication,2018(62):82-92.
[6] WANG S J,LI B J,LIU Y J,et al.Micro-expression recognitionwith small sample size by transferring longterm convolutional neural network[J].Neurocomputing,2018,312:251-262.
[7] LIONG S T,GAN Y S,SEE J,et al.Shallow triple stream three-dimensional CNN(STSTNet) for micro-expression recognition[C]//2019 14th IEEE International Conference on Automatic Face & Gesture Recognition.IEEE,2019:1-5.
[8] HONG J,LEE C,JUNG H.Late fusion-based video transformerfor facial micro-expression recognition[J].Applied Sciences,2022,12(3):1169.
[9] WANG Z,ZHANG K,LUO W,et al.HTNet for micro-expression recognition[J].Neurocomputing,2024,602:128196.
[10] LI X,PFISTER T,HUANG X,et al.A spontaneous micro-expression database:Inducement,collection and baseline[C]//2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition.IEEE,2013:1-6.
[11] DAVISON A K,LANSLEY C,COSTEN N,et al.SAMM:A spontaneous micro-facial movement dataset[J].IEEE Transactions on Affective Computing,2016,9(1):116-129.
[12] YAN W J,LI X,WANG S J,et al.CASME II:an improved spontaneous micro-expression database and the baseline evaluation[J].PLoS One,2014,9(1):1-8.
[13] KU H,DONG W.Face recognition based on MTCNN and con-volutional neural network[J].Frontiers in Signal Processing,2020,4(1):37-42.
[14] LUCAS B D,KANADE T.An iterative image registration technique with an application to stereo vision[C]//7th International Joint Conference on Artificial Intelligence(IJCAI’81).1981:674-679.
[15] HORN B K P,SCHUNCK B G.Determining optical flow[J].Artificial Intelligence,1981,17:185-203.
[16] FARNEBÄCK G.Two-frame motion estimation based on polynomial expansion[C]//Image Analysis:13th Scandinavian Conference.Berlin:Springer,2003:363-370.
[17] CAO C P,ZHANG D.Micro-expression Recognition Based onMulti-Region Features and Feature Fusion.Mini-micro Systems[J].Journal of Chinese Computer Systems,2025,46(8):1986-1992.
[18] HAN K,WANG Y,TIAN Q,et al.Ghostnet:More featuresfrom cheap operations[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:1580-1589.
[19] SENGUPTA A,YE Y,WANG R,et al.Going deeper in spiking neural networks:Vgg and residual architectures[J].Frontiers in Neuroscience,2019:13:95.
[20] GREFF K,SRIVASTAVA R K,KOUTNÍK J,et al.LSTM:A Search Space Odyssey[J].IEEE Transactions on Neural Networks and Learning Systems,2017,28(10):2222-2232.
[21] DESHMUKH V S,ZARANDI S N.An optimized deep learning based depthwise separable MobileNetV3 approach for automatic finger vein recognition system[J].Multimedia Tools and Applications,2024,83(24):64285-64313.
[22] ZHENG H,HUANG S,LI J,et al.Dual-ATME:Dual-Branch Attention Network for Micro-Expression Recognition[J].Entropy,2023,25(3):460.
[23] ZHANG H,ZHANG H,A review of micro-expression recognition based on deep learning[C]//2022 International Joint Conference on Neural Networks.IEEE,2022:1-8.
[24] NIE X,MADHUMITA A T,DUAN M Y,et al.GEME:dual-stream multi-task GEnder-based Micro-Expression recognition[J].Neurocomputing,2021,427:13-28.
[25] ZHANG B,WU Y F.An Algorithm for Micro-Expression Re-cognition Based on Two-Branch Lightweight Network[J].Laser &Optoelectronics Progress,2024,61(14):334-343.
[26] ZHOU L,MAO Q R,HUANG X H,et al.Feature refinement:An expression-specific feature learning and fusion method for micro-expression recognition[J].Pattern Recognition,2022,122:108275.
[27] PAN H,XIE L,WANG Z L.C3DBed:Facial micro-expression recognition with three-dimensional convolutional neural network embedding in transformer model[J].Engineering Applications of Artificial Intelligence,2023,123(PA):106258.
[28] YU Z Y,CHEN X J,QU C.SDGSA:a lightweight shallow dual-group symmetric attention network for micro-expression recognition[J].Complex & Intelligent Systems,2024,10:8143-8162.
[29] SONG J F,LEI S Z,WU W Z.Microexpression RecognitionMethod Based on ADP-DSTN Feature Fusion and Convolutional Block Attention Module[J].Electronics,2024,13(20):4012.
[30] DING X M,LI Y Y,WU Y L,et al.Enhancement-suppression driven lightweight fine-grained micro-expression recognition[J].Journal of Visual Communication and Image Representation,2025,107:104383.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市两江新区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发