计算机科学 ›› 2024, Vol. 51 ›› Issue (10): 105-111.doi: 10.11896/jsjkx.240300059
张笑云1, 赵晖2
ZHANG Xiaoyun1, ZHAO Hui2
摘要： 智能教育快速发展,运用人工智能提升教育质量和效率已成为趋势。学生作为教育的核心,其情绪状态对教育成效具有至关重要的影响。为了深入研究学生情绪,收集了课堂场景中的学生学习视频,包括听课和小组讨论两种情境,并据此建立了一个多任务学生情绪数据库。面部作为内在情绪状态的直接外在体现,显示出AU与情绪之间的紧密关联。在此基础上,提出了一个基于多任务学习的学生情绪识别模型Multi-SER。该模型通过结合AU识别和学生情绪识别两项任务,挖掘各个AU与学生情绪之间的关联关系,进而提升模型在学生情绪识别方面的性能。在多任务实验中,Multi-SER模型的情绪识别准确率达到了80.87%,相比单情绪识别任务模型SE-C3DNet+,效果提升了3.11%。实验结果表明,通过多任务学习挖掘AU和情绪之间的关联关系,模型在分类各种情绪方面的性能得到了提升。
中图分类号:
| [1]中共中央国务院印发《中国教育现代化2035》[N].人民日报,2019-02-24(001). [2]教育部关于印发《教育信息化2.0行动计划》的通知[J].中华人民共和国教育部公报,2018,(4):118-125. [3]WEI Y T,LEI F,HU M J et al.A review of student expression recognition[J].China Educational Informatization,2020(21):48-55. [4]YANG H,PENG C,SUN R,et al.Student Expression Recognition in Smart Education Environment based on Convolutional Neural Network[C]//2022 9th International Conference on Dependable Systems and Their Applications(DSA).2022. [5]PANDIMURUGAN V,SINGH A,TIWARI A.Facial Emotion Recognition for Students Using Machine Learning[C]//2023 International Conference on Computer Communication and Informatics(ICCCI).IEEE,2023:1-4. [6]SUMALAKSHMI C H,VASUKI P.Fused deep learning based Facial Expression Recognition of students in online learning mode[J].Concurrency and Computation:Practice and Expe-rience,2022,34(21):e7137. [7]DAI X,WEI P,ZENG Y,et al.Students' Facial Expression Re-cognition Based on Multi-head Attention Mechanism[J].Journal of Physics:Conference Series.IOP Publishing,2022(2493):012004. [8]JIAO S,YAN Y X.Research on Students' Emotional StateBased on Expression Recognition Technology－Taking the Ju-nior Middle School Classroom as an Example[J].Modern Information Technology,2022(18):6. [9]HAN L,LI Y,ZHOU Z J et al.Teaching Effect Analysis Based on the Facial Expression Recognition in Classroom[J].Modern Distance Education Research,2017(4):8. [10]GAO Q.Spontaneous learning facial expression recognitionbased on RealSense research [D].Wuhan:Central China normal university,2020. [11]CHEN Z J.Research on learning emotion recognition methodand application based on facial expression in online learning environment [D].Wuhan:Central China normal university,2020. [12]XU Z G.Emotion Recognition of learning picture in Intelligent Learning Environment and its application [D].Jinan:Shandong Normal University,2019. [13]PLUTCHIK R.A general psychoevolutionary theory of emotion[M]//Theories of Emotion.Academic Press,1980:3-33. [14]EKMAN P,FRIESEN W V.Facial action coding system[J/OL].https://www.paulekman.com/facial-action-coding-system/. [15]COSTA W,TALAVERA E,OLIVEIRA R,et al.A survey on datasets for emotion recognition from vision:Limitations and in-the-wild applicability[J].Applied Sciences,2023,13(9):5697. [16]ZHAO H,WANG Z L,LIU Y F.A Survey of Automatic Facial Action Units Recognition[J].Journal of Computer-Aided Design &Computer Graphics,2010,22(5):894-906. [17]LIU C,ZHANG X,LIU X,et al.Facial Expression Recognition Based on Multi-Modal Features for Videos in the Wild[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2023:5871-5878. [18]HOSSAIN S,UMER S,ROUT R K,et al.Fine-grained imageanalysis for facial expression recognition using deep convolu-tional neural networks with bilinear pooling[J].Applied Soft Computing,2023,134:109997. [19]ZHANG Y,YANG Q.A survey on multi-task learning[J].IEEE Transactions on Know ledge and Data Engineering,2021,34(12):5586-5609. [20]ZHANG S,YIN C,YIN Z.Multimodal sentiment recognitionwith multi-task learning[J].IEEE Transactions on Emerging Topics in Computational Intelligence,2022,7(1):200-209. [21]ZHANG Y,RONG L,LI X,et al.Multi-modal sentiment andemotion joint analysis with a deep attentive multi-task learning model[C]//European Conference on Information Retrieval.Cham:Springer International Publishing,2022:518-532. [22]LI Y,KAZEMEINI A,MEHTA Y,et al.Multitask learning for emotion and personality traits detection[J].Neurocomputing,2022,493:340-350. [23]BIAN C,ZHANG Y,YANG F,et al.Spontaneous facial expression database for academic emotion inference in online learning[J].IET Computer Vision,2019,13(3):329-337. [24]GUNES H,PICCARDI M.A bimodal face and body gesture database for automatic analysis of human nonverbal affective behavior[C]//18th International Conference on Pattern Recognition(ICPR'06).IEEE,2006,1:1148-1153. [25]WALLBOTT H G.Bodily expression of emotion[J].European Journal of Social Psychology,1998,28(6):879-896. [26]EL KALIOUBY R A.Mind-reading machines:automated infe-rence of complex mental states[R].University of Cambridge,Computer Laboratory,2005. [27]KOSTI R,ALVAREZ J M,RECASENS A,et al.Emotion re-cognition in context[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:1667-1675. [28]RASTGOO R,KIANI K,ESCALERA S,et al.Multi-modal zero-shot dynamic hand gesture recognition[J].Expert Systems with Applications,2024,247:123349. [29]HARA K,KATAOKA H,SATOH Y.Can Spatiotemporal 3D CNNs Retrace the History of 2D CNNs and ImageNet?[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.IEEE,2018. [30]WANG H,LI B,WU S,et al.Rethinking the Learning Paradigm for Dynamic Facial Expression Recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2023:17958-17968. [31]ROMEO L,CAVALLO A,PEPA L,et al.Multiple instancelearning for emotion recognition using physiological signals[J].IEEE Transactions on Affective Computing,2019,13(1):389-407. [32]JIANG X,ZONG Y,ZHENG W,et al.DFEW:A Large-ScaleDatabase for Recognizing Dynamic Facial Expressions in the Wild[C]//Proceedings of the 28th ACM International Confe-rence on Multimedia.2020:2881-2889. [33]MA F,SUN B,LI S.Spatio-temporal transformer for dynamic facial expression recognition in the wild[J].arXiv:2205.04749,2022. |
|
||
首页