Computer Science ›› 2026, Vol. 53 ›› Issue (8): 174-181.doi: 10.11896/jsjkx.250500012

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Hierarchical Lightweight Micro-expression Recognition Based on Optical Flow Partitioned FeatureFusion

HU Changyu, FAN Xinyu, ZHANG Zhi, DING Zixu, ZHANG Zhengyue, PENG Juhong   

  1. School of Artificial Intelligence, Hubei University, Wuhan 430062, China
    Key Laboratory of Intelligent Perception Systems and Security, Ministry of Education, Hubei University, Wuhan 430062, China
  • Received:2025-05-06 Revised:2025-09-22 Online:2026-08-15 Published:2026-08-17
  • About author:HU Changyu,born in 2002,postgra-duate,is a member of CCF(No.X4218G).Her main research interests include computer vision and micro-expression recognition.
    PENG Juhong,born in 1978,associate professor.Her main research interests include information processing and AI algorithms.
  • Supported by:
    Natural Science Foundation of Hubei Province,China(JCZRQN202500839).

Abstract: Micro-expression recognition remains a challenging task due to the difficulty in capturing subtle facial movements,li-mited dataset sizes,and high computational costs.To address these issues,this paper proposes a lightweight and efficient model,named HLFM-OFP(Hierarchical Lightweight Model with Feature Fusion Based on Optical Flow Partitioning).The model employs TV-L1 optical flow for motion extraction and utilizes a hierarchical lightweight architecture to enrich the representation of low-level features.STModel(Spatiotemporal Feature Fusion Module)is further introduced to jointly capture spatial and temporal dynamics.Experimental results on composite datasets demonstrate that HLFM-OFP contains only 2.8 million parameters and reduces training time by 88.45%,while achieving 94.23% accuracy,91.23% UF1,and 91.66% UAR.Experiments show that the proposed model reduces parameters while improving accuracy,and demonstrates strong robustness on the more challenging SMIC and SAMM datasets.

Key words: Micro-expression recognition, Optical flow partitioned feature fusion, Multi-level lightweight design, Dynamic and static feature fusion, H-Swish loss function

CLC Number: 

  • TP391.41
[1] OJALA T,PIETIKÄINEN M,MÄENPÄÄ T.MultiresolutionGray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns[J].IEEE Transactions on Pattern Ana-lysis and Machine Intelligence,2002,24(7):971-987.
[2] PFISTER T,LI X,ZHAO G,et al.Recognising spontaneous facial micro-expressions[C]//International Conference on Computer Vision.IEEE,2011:1449-1456.
[3] HUANG X,ZHAO G,HONG X,et al.Spontaneous facial mi-cro-expression analysis using Spatiotemporal Completed Local Quantized Patterns[J].Neurocomputing,2016,175:564-578.
[4] LIONG S,SEE J,PHAN C R,et al.Spontaneous subtle expression detection and recognition based on facial strain[J].Signal Processing:Image Communication,2016,47:170-182.
[5] LIONG S T,SEE J,WONG K S,et al.Less is more:Micro-expression recognition from video using apex frame[J].Signal Processing:Image Communication,2018(62):82-92.
[6] WANG S J,LI B J,LIU Y J,et al.Micro-expression recognitionwith small sample size by transferring longterm convolutional neural network[J].Neurocomputing,2018,312:251-262.
[7] LIONG S T,GAN Y S,SEE J,et al.Shallow triple stream three-dimensional CNN(STSTNet) for micro-expression recognition[C]//2019 14th IEEE International Conference on Automatic Face & Gesture Recognition.IEEE,2019:1-5.
[8] HONG J,LEE C,JUNG H.Late fusion-based video transformerfor facial micro-expression recognition[J].Applied Sciences,2022,12(3):1169.
[9] WANG Z,ZHANG K,LUO W,et al.HTNet for micro-expression recognition[J].Neurocomputing,2024,602:128196.
[10] LI X,PFISTER T,HUANG X,et al.A spontaneous micro-expression database:Inducement,collection and baseline[C]//2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition.IEEE,2013:1-6.
[11] DAVISON A K,LANSLEY C,COSTEN N,et al.SAMM:A spontaneous micro-facial movement dataset[J].IEEE Transactions on Affective Computing,2016,9(1):116-129.
[12] YAN W J,LI X,WANG S J,et al.CASME II:an improved spontaneous micro-expression database and the baseline evaluation[J].PLoS One,2014,9(1):1-8.
[13] KU H,DONG W.Face recognition based on MTCNN and con-volutional neural network[J].Frontiers in Signal Processing,2020,4(1):37-42.
[14] LUCAS B D,KANADE T.An iterative image registration technique with an application to stereo vision[C]//7th International Joint Conference on Artificial Intelligence(IJCAI’81).1981:674-679.
[15] HORN B K P,SCHUNCK B G.Determining optical flow[J].Artificial Intelligence,1981,17:185-203.
[16] FARNEBÄCK G.Two-frame motion estimation based on polynomial expansion[C]//Image Analysis:13th Scandinavian Conference.Berlin:Springer,2003:363-370.
[17] CAO C P,ZHANG D.Micro-expression Recognition Based onMulti-Region Features and Feature Fusion.Mini-micro Systems[J].Journal of Chinese Computer Systems,2025,46(8):1986-1992.
[18] HAN K,WANG Y,TIAN Q,et al.Ghostnet:More featuresfrom cheap operations[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:1580-1589.
[19] SENGUPTA A,YE Y,WANG R,et al.Going deeper in spiking neural networks:Vgg and residual architectures[J].Frontiers in Neuroscience,2019:13:95.
[20] GREFF K,SRIVASTAVA R K,KOUTNÍK J,et al.LSTM:A Search Space Odyssey[J].IEEE Transactions on Neural Networks and Learning Systems,2017,28(10):2222-2232.
[21] DESHMUKH V S,ZARANDI S N.An optimized deep learning based depthwise separable MobileNetV3 approach for automatic finger vein recognition system[J].Multimedia Tools and Applications,2024,83(24):64285-64313.
[22] ZHENG H,HUANG S,LI J,et al.Dual-ATME:Dual-Branch Attention Network for Micro-Expression Recognition[J].Entropy,2023,25(3):460.
[23] ZHANG H,ZHANG H,A review of micro-expression recognition based on deep learning[C]//2022 International Joint Conference on Neural Networks.IEEE,2022:1-8.
[24] NIE X,MADHUMITA A T,DUAN M Y,et al.GEME:dual-stream multi-task GEnder-based Micro-Expression recognition[J].Neurocomputing,2021,427:13-28.
[25] ZHANG B,WU Y F.An Algorithm for Micro-Expression Re-cognition Based on Two-Branch Lightweight Network[J].Laser &Optoelectronics Progress,2024,61(14):334-343.
[26] ZHOU L,MAO Q R,HUANG X H,et al.Feature refinement:An expression-specific feature learning and fusion method for micro-expression recognition[J].Pattern Recognition,2022,122:108275.
[27] PAN H,XIE L,WANG Z L.C3DBed:Facial micro-expression recognition with three-dimensional convolutional neural network embedding in transformer model[J].Engineering Applications of Artificial Intelligence,2023,123(PA):106258.
[28] YU Z Y,CHEN X J,QU C.SDGSA:a lightweight shallow dual-group symmetric attention network for micro-expression recognition[J].Complex & Intelligent Systems,2024,10:8143-8162.
[29] SONG J F,LEI S Z,WU W Z.Microexpression RecognitionMethod Based on ADP-DSTN Feature Fusion and Convolutional Block Attention Module[J].Electronics,2024,13(20):4012.
[30] DING X M,LI Y Y,WU Y L,et al.Enhancement-suppression driven lightweight fine-grained micro-expression recognition[J].Journal of Visual Communication and Image Representation,2025,107:104383.
[1] JIANG Sheng, ZHU Jianhong. Face Micro-expression Recognition Method Based on ME-ResNet [J]. Computer Science, 2024, 51(11A): 231000053-7.
[2] ZHANG Jia-hao, LIU Feng, QI Jia-yin. Lightweight Micro-expression Recognition Architecture Based on Bottleneck Transformer [J]. Computer Science, 2022, 49(6A): 370-377.
[3] LI Xing-ran, ZHANG Li-yan, YAO Shu-jing. Micro-expression Recognition Method Combining Feature Fusion and Attention Mechanism [J]. Computer Science, 2022, 49(2): 4-11.
[4] LIANG Zheng-you, HE Jing-lin, SUN Yu. Three-dimensional Convolutional Neural Network Evolution Method for Facial Micro-expression Auto-recognition [J]. Computer Science, 2020, 47(8): 227-232.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!