人形机器人面部情感表达技术研究综述

doi:10.11896/jsjkx.250600166

摘要/Abstract

摘要： 人形机器人面部情感表达技术是实现自然人机交互的核心支撑,近年来在医疗陪护、教育服务与社交娱乐等领域展现出广泛的应用前景。回顾了该领域的发展历程,按照技术能力演进将其划分为基础情感表达、情感交互增强与情感共鸣个性化3个阶段,系统探讨了各阶段在仿生机械结构设计与多模态情感交互系统构建方面的关键突破。重点分析了面部动作编码系统、柔性仿生皮肤材料、深度学习与多模态大模型等在情感建模中的应用,进一步探讨了多模态时空对齐机制在表情、语音与动作协同优化中的作用。最后,展望了未来的研究方向,包括高精度驱动与传感融合、多模态大模型集成、个性化与文化适应性增强,以及恐怖谷效应的缓解策略,为推动人形机器人面部情感表达技术的智能化和自然化发展提供了理论支持。

关键词: 人形机器人, 面部情感表达, 仿生机械结构, 情感建模, 多模态时空对齐, 恐怖谷效应

Abstract: Facial emotional expression in humanoid robots represents a cornerstone technology for enabling naturalistic human-robot interaction,exhibiting broad application prospects in sectors such as medical support,educational services,and social entertainment.This paper reviews the developmental trajectory of the field,delineating its evolution across three stages based on technological capability:foundational emotional expression,enhanced affective interaction,and personalized emotional resonance.It systematically examines the key breakthroughs within each stage concerning the design of biomimetic mechanical structures and the construction of multimodal affective interaction systems.The review places particular emphasis on the application of the facial action coding system,flexible biomimetic skin materials,deep learning,and multimodal large models(MLMs) in affective mode-ling.Furthermore,it investigates the critical role of multimodal spatio-temporal alignment mechanisms in the coordinated optimization of facial expressions,vocalizations,and physical actions.Finally,the paper outlines future research trajectories,including the fusion of high-precision actuation and sensing,the integration of multimodal large models,the enhancement of personalization and cultural adaptability,and strategies for mitigating the uncanny valley effect.This work provides a theoretical foundation to advance the intelligent and naturalistic development of facial emotional expression technology in humanoid robots.

Key words: Humanoid robot, Facial emotional expression, Biomimetic mechanical structure, Affective modeling, Multimodal spatio-temporal alignment, Uncanny valley effect

中图分类号:

TP242

柯显信, 李玄, 宋骏骐. 人形机器人面部情感表达技术研究综述[J]. 计算机科学, 2026, 53(7): 1-8. https://doi.org/10.11896/jsjkx.250600166

KE Xianxin, LI Xuan, SONG Junqi. Research on Facial Emotion Expression Technologies for Humanoid Robots[J]. Computer Science, 2026, 53(7): 1-8. https://doi.org/10.11896/jsjkx.250600166

参考文献

[1]ESFANDBOD A,ROKHI Z,MEGHDARI A F,et al.Utilizing an emotional robot capable of lip-syncing in robot-assisted speech therapy sessions for children with language disorders[J].International Journal of Social Robotics,2023,15(2):165-183.
[2]IVANOV S,WEBSTER C,SEYITOĞLU F.Humans and/or ro-bots? Tourists' preferences towards the humans-robots mix in the service delivery system[J].Service Business,2023,17(1):195-231.
[3]PODPEČAN V.Can you dance? A study of child-robot interaction and emotional response using the NAO robot[J].Multimodal Technologies and Interaction,2023,7(9):85.
[4]TAKANISHI A,HIRANO S,SATO K.Development of an anthropomorphic head-eye system for a humanoid robot-realization of human-like head-eye motion using eyelids adjusting to brightness[C]//Proceedings 1998 IEEE International Conference on Robotics and Automation.IEEE,1998:1308-1314.
[5]BREAZEAL C L.Sociable machines:expressive social exchange between humans and robots[M].Cambridge,MA:Massachusetts Institute of Technology,2000.
[6]WANG W.Top 10 Innovative Humanoid Robot Products of2024 Revealed [N].China Electronics News,2025-01-17(6).
[7]TAKANISHI A,TAKANOBU H,KATO I,et al.Development of the anthropomorphic head-eye robot WE-3RII with an autonomous facial expression mechanism[C]//Proceedings 1999 IEEE International Conference on Robotics and Automation.IEEE,1999:3255-3260.
[8]ALLMAN T.Nexi robot,the(A Great Idea)[M].Norwood House Press,2009.
[9]MIWA H,ITOH K,MATSUMOTO M,et al.Effective emo-tional expressions with emotion expression humanoid robot WE-4RII[C]//2004 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS).IEEE,2004:2203-2208.
[10]FORTUNATI L,SORRENTINO A,FIORINI L,et al.The rise of the roboid[J].International Journal of Social Robotics,2021,13(6):1457-1471.
[11]SHARMA N,PANDEY J K,MONDAL S.A review of mobile robots:applications and future prospect[J].International Journal of Precision Engineering and Manufacturing,2023,24(9):1695-1706.
[12]YOUSIF M.Humanoid robot enhancing social and communication skills of autistic children:review[J].Artificial Intelligence &Robotics Development Journal,2021,1(2):80-92.
[13]ABDOLLAHI H,MAHOOR M H,ZANDIE R,et al.Artificial emotional intelligence in socially assistive robots for older adults:a pilot study[J].IEEE Transactions on Affective Computing,2023,14(3):2020-2032.
[14]KISHI T,OTANI T,ENDO N,et al.Development of expressive robotic head for bipedal humanoid robot[C]//2012 IEEE/RSJ International Conference on Intelligent Robots and Systems.IEEE,2012:4584-4589.
[15]HE M,BI J,WANG Y J,et al.A Review of Emotional Expression Technology in Humanoid Robots [J].Journal of Chongqing University of Technology(Natural Science),2022,36(11):183-192.
[16]HASHIMOTO T,HITRAMATSU S,TSUJI T,et al.Development of the face robot SAYA for rich facial expressions[C]//2006 SICE-ICASE International Joint Conference.2006:5423-5428.
[17]WU W G,SONG C,MENG Q M.Research and Experiment of Voice and Lip-shaped System of Android Head Robot “H&frobot-Ⅲ” [J].Journal of Machine Design,2008(1):15-19.
[18]CHEN W L,HUANG J H,YANG Z P.Research and Development of AI Autonomous Guided Tour Robot Based on 3D Vision [J].Computer Knowledge and Technology,2024,20(5):95-98.
[19]OH J H,HANSON D,KIM W,et al.Design of android type humanoid robot albert HUBO[C]//2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.IEEE,2006:1428-1433.
[20]ISHIHARA H,YOSHIKAWA Y,ASADA M.Realistic childrobot “affetto” for understanding the caregiver-child attachment relationship that guides the child development[C]//2011 IEEE International Conference on Development and Learning(ICDL).2011:1-5.
[21]MAZZEI D,ZARAKI A,LAZZERI N,et al.Recognition andexpression of emotions by a symbiotic android head[C]//2014 IEEE-RAS International Conference on Humanoid Robots.IEEE,2014:134-139.
[22]GHORBANDAEI POUR A,TAHERI A,ALEMI M,et al.Human-robot facial expression reciprocal interaction platform:Case studies on children with autism[J].International Journal of Social Robotics,2018,10(2):179-198.
[23]SUN J L.Manufacturing the Future:Integration and Innovation of Technology and Livelihood [J].Contemporary Television,2020(2):45-47,98.
[24]EKMAN P,FRIESEN W V.Facial action coding system[M]//Environmental Psychology & Nonverbal Behavior.1978.
[25]LI Y,ZHU L,ZHANG Z,et al.Humanoid robot heads for human-robot interaction:a review[J].Science China Technological Sciences,2024,67(2):357-379.
[26]HUSSAIN N,ERZIN E,SEZGIN T M,et al.Training sociallyengaging robots:modeling backchannel behaviors with batch reinforcement learning[J].IEEE Transactions on Affective Computing,2022,13(4):1840-1853.
[27]AUFLEM M,KOHTALA C,JUNG M F,et al.Facing the FACS-Using AI to Evaluate and Control Facial Action Units in Humanoid Robot Face Development[J].Frontiers in Robotics and AI,2022,9:887645.
[28]ZHANG D,PENG J,JIAO Y,et al.ExFace:expressive facialcontrol for humanoid robots with diffusion transformers and bootstrap training[J].arXiv:2504.14477,2025.
[29]SNYDER D E,SAPPER E D.See me,feel me,touch me,heal me:a contextual overview of conductive polymer composites as synthetic human skin[J].Journal of Composites Science,2022,6(5):141.
[30]HABIB A,DAS S K,BOGDAN I C,et al.Learning human-like facial expressions for android phillip K.Dick[C]//2014 IEEE International Conference on Automation Science and Engineering(CASE).IEEE,2014:1159-1165.
[31]MORI M,MACDORMAN K,KAGEKI N.The uncanny valley [from the field][J].IEEE Robotics & Automation Magazine,2012,19(2):98-100.
[32]TANG H,SEBE N.Facial expression translation using land-mark guided GANs[J].IEEE Transactions on Affective Computing,2022,13(4):1986-1997.
[33]ISHIHARA H,HAYASHI R,LAVIEILLE F,et al.Automatic generation of dynamic arousal expression based on decaying wave synthesis for robot faces[J].Journal of Robotics and Mechatronics,2024,36(6):1481-1494.
[34]KIMIZUKA K,AZHARI S,TOKUNO S,et al.Haptic in-sensor computing device made of carbon nanotube-polydimethylsiloxane nanocomposites[J].arXiv:2406.03958,2024.
[35]XU Y,SUN Z,BAI Z,et al.Bionic e-skin with precise multi-directional droplet sliding sensing for enhanced robotic perception[J].Nature Communications,2024,15(1):6022.
[36]CHEN G W,ZHANG P Z,WANG T,et al.Overview of Multimodal Sentiment Analysis [J].Journal of Communication University of China(Natural Science Edition),2022,29(2):70-78.
[37]PANDEY A K,GELIN R.A mass-produced sociable humanoid robot:Pepper:the first machine of its kind[J].IEEE Robotics & Automation Magazine,2018,25(3):40-48.
[38]MIWA H,ITOH K,ITO D,et al.Design and control of 9-DOFsemotion expression humanoid arm[C]//IEEE International Conference on Robotics and Automation(ICRA'04).2004.IEEE,2004:128-133.
[39]MEHRABIAN A.Pleasure-arousal-dominance:A general frame-work for describing and measuring individual differences in temperament[J].Current Psychology,1996,14(4):261-292.
[40]HUANG H C,LIU N,HU M,et al.A Game-Based Cognitive Emotional Interaction Model for Robots [J].Journal of Elec-tronics and Information Technology,2019,41(10):2471-2478.
[41]LIU X,NI R,YANG B,et al.Unlocking human-like facial expressions in humanoid robots:a novel approach for action unit driven facial expression disentangled synthesis[J].IEEE Transactions on Robotics,2024,40:3850-3865.
[42]MISHRA C,VERDONSCHOT R,HAGOORT P,et al.Real-time emotion generation in human-robot dialogue using large language models[J].Frontiers in Robotics and AI,2023,10:1271610.
[43]MOSTAFAVI M,PORTER M D.How emoji and word embedding helps to unveil emotional transitions during online messaging[C]//2021 IEEE International Systems Conference(syscon).2021:1-8.
[44]RAWAL N,KOERT D,TURAN C,et al.ExGenNet:learning to generate robotic facial expression using facial expression re-cognition[J].Frontiers in Robotics and AI,2022,8:730317.
[45]ISHIHARA H,HAYASHI R,LAVIEILLE F,et al.Automatic generation of dynamic arousal expression based on decaying wave synthesis for robot faces[J].Journal of Robotics and Mechatronics,2024,36(6):1481-1494.
[46]MANDEEL A R,AL-RADHI M S,CSAPÓ T G.Speaker adaptation experiments with limited data for end-to-end text-to-speech synthesis using Tacotron2[J].Infocommunications Journal,2022,14(3):55-62.
[47]ZHOU K,SISMAN B,RANA R,et al.Speech synthesis withmixed emotions[J].IEEE Transactions on Affective Computing,2023,14(4):3120-3134.
[48]LIU S,CAO Y,MENG H.Emotional voice conversion with cycle-consistent adversarial network[J].arXiv:2004.03781,2020.
[49]FU C,DENG Q,SHEN J,et al.A preliminary study on realizing human-robot mental comforting dialogue via sharing experience emotionally[J].Sensors,2022,22(3):991.
[50]LEE J.Generating robotic speech prosody for human robot interaction:a preliminary study[J].Applied Sciences,2021,11(8):3468.
[51]SHUSTER K,XU J,KOMEILI M,et al.Blender Bot 3:a de-ployed conversational agent that continually learns to responsibly engage[J].arXiv:2208.03188,2022.
[52]CHEBOTAR Y,VUONG Q,HAUSMAN K,et al.Q-transformer:scalable offline reinforcement learning via autoregressive Q-functions[C]//Conference on Robot Learning.PMLR,2023:3909-3928.
[53]GARCIA S,GOMEZ-DONOSO F,CAZORLA M.Enhancinghuman-robot interaction:development of multimodal robotic assistant for user emotion recognition[J].Applied Sciences,2024,14(24):11914.
[54]HUANG W C,HSUEH Y L.An emotional dialogue systemusing conditional generative adversarial networks with a sequence-to-sequence transformer encoder[J].ACM Transactions on Asian and Low-Resource Language Information Processing,2024,23(12):164:1-164:23.
[55]ALY A,TAPUS A.Multimodal adapted robot behavior synthesis within a narrative human-robot interaction[C]//2015 IEEE/RSJ International Conference on Intelligent Robots and Systems(IROS).2015:2986-2993.
[56]YAN Z,SONG Y,ZHOU R,et al.Facial expression realization of humanoid robot head and strain-based anthropomorphic eva-luation of robot facial expressions[J].Biomimetics,2024,9(3):122.
[57]WANG C,HASLER S,TANNEBERG D,et al.LaMI:large language models for multi-modal human-robot interaction[C]//Extended Abstracts of the CHI Conference on Human Factors in Computing Systems.2024:1-10.
[58]INDURKHYA B.Ethical aspects of faking emotions in chatbots and social robots[C]//2023 32nd IEEE International Conference on Robot and Human Interactive Communication(RO-MAN).2023:1719-1724.
[59]RUDSCHIES C,SCHNEIDER I.The long and winding road to bans for artificial intelligence:from public pressure and regulatory initiatives to the EU AI act[J].Digital Society,2025,4(2):57.
[60]DORAFSHANIAN M,AITSAM M,MEJRI M,et al.Beyonddata collection:safeguarding user privacy in social robotics[C]//2024 IEEE International Conference on Industrial Technology(ICIT).2024:1-6.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed