计算机科学 ›› 2022, Vol. 49 ›› Issue (11A): 211000049-6.doi: 10.11896/jsjkx.211000049
魏艳涛1,2, 罗洁琳1,2, 胡美佳1,2, 李文昊1, 姚璜1
WEI Yan-tao1,2, LUO Jie-lin1,2, HU Mei-jia1,2, LI Wen-hao1, YAO Huang1
摘要： 随着疫情防控的常态化开展,在线学习已成为日常教学活动的主要形式之一。然而,随着在线学习活动的大规模开展,“情感缺失”问题日益凸显,已成为阻碍在线学习成效的主要原因之一。针对上述问题,探讨利用视频数据的非侵入式在线学习情感状态识别方法。首先采集了22名学生在线学习的面部视频和心率数据,构建了双模态在线学习情感数据库。其次,采用帧注意网络从学习视频中提取表情特征,识别在线学习情感状态,识别精度达到了87.8%。最后,探讨了视频心率识别方法在在线学习情感分析中的应用,研究结果表明,困惑状态下的心率水平具有显著性。本文从学习视频数据挖掘出发,重点探讨了基于表情和视频心率的学习情感识别,为提高在线学习情感状态感知提供了新思路。
中图分类号:
| [1]HUANG C Q,TU Y X,YU J H,et al.Research and implemen-tation of Data-driven Online Learning Burnout Early Warning Model[J].E-education Research,2021,42(2):47-54. [2]GUO C L.Solutions to the Lack of Teaching Emotion between Teachers and Students in Online Teaching under the Epidemic Situation[J].The Chinese Journal of ICT in Education,2021(7):65-69. [3]HUANG C.Research on Learner Emotion Modeling and Application for E-learning[D].Wuhan:Central China Normal University,2014. [4]WEI R J,DING Y Z,ZHANG L,et al.Design and Implementation of Emotion Recognition Module in Online Learning System[J].Modern Educational Technology,2014(3):115-122. [5]CAO X M,ZHANG Y H,PAN M,et al.Research on Recognition Method of Learning Participation from the Perspective of Artificial Intelligence-Analysis of Deep Learning Experiment based on Multimodal Data Fusion[J].Journal of Distance Education,2019(1):32-44. [6]TAO X M,CHEN X Y.Research on emotion recognition based on eye movement features in online learning environment[J].Computer Technology and Development,2021,31(3):186-190. [7]YU Z T,LI X B,ZHAO G Y.Emotion Recognition and Education[J].artificial intelligence,2019(3):29-36. [8]PISE A,VADAPALL H,SANDERS I.Facial emotion recognition using temporal relational network:an application to E-learning[J].Multimedia Tools and Applications,2020(2):1-12. [9]DINDAR M,JARVELA S,AHOLA S,et al.Leaders and fol-lowers identified by emotional mimicry during collaborative learning:A facial expression recognition study on emotional valence[J].IEEE Transactions on Affective Computing,2020,13(3):1390-1400. [10]PEKRUN R,VOGL E,MUIS K R,et al.Measuring emotions during epistemic activities:the Epistemically-Related Emotion Scales[J].Cognition & emotion,2017,31(6):1268-1276. [11]ASHWIN T S,RAM M R G.Automatic detection of students’ affective states in classroom environment using hybrid convolutional neural networks[J].Education and information technologies,2020,25(2):1387-1415. [12]XU Z G.Emotion Recognition of Learning Picture in Intelligent Learning Environment and Its Application[D].Jinan:Shandong Normal University,2019. [13]MA Y T.Design and Implementation of Emotion Analysis Mo-del for Online Learning based on Multimodality [D].Nanjing:Nanjing Normal University,2019. [14]JIA L Y,ZHANG Z H,ZHAO X Y,et al.Classroom Student State Analysis Based on Artificial Intelligence Video Processing [J].Modern Educational Technology,2019,29(12):82-88. [15]CHEN X,CHENG J,SONG R,et al.Video-Based Heart Rate Measurement:Recent Advances and Future Prospects[J].IEEE Transactions on Instrumentation and Measurement,2018,68(10):3600-3615. [16]LUCEY P,COHN J F,KANADE T,et al.The Extended Cohn Kanade Dataset(CK+):A complete dataset for action unit and emotion-specified expression[C]//IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.2010:94-101. [17]BARROS P,CHURAMAN N,LAKOMKIN E,et al.The omg-emotionbehavior dataset[C]//International Joint Conference on Neural Networks (IJCNN).2018:1-17. [18]LI J,CHEN Y,XIAO S,et al.Estimation of affective level in the wild with multiple memory networks[C]//IEEE Conference on Computer Vision and Pattern Recognition Workshops.2017:1-8. [19]WIEM M B H,LACHIRI Z.Emotion classification in arousalvalence model using MAHNOB-HCI database[J].International Journal of Advanced Computer Science and Applications,2017,8(3):318-323. [20]XY Z G,ZHANG G W,MENG X Z,et al.Learners’ Emotion Recognition and Application Based on Deep Learning[J].e-EDUCATION RESEARCH,2019,40(2):87-94. [21]HE X K,GAO Q,LI Y Y,et al.Research on SpontaneousLearning Expression Recognition Method Based on Deep Learning Model [J].Computer Applications and Software,2019,36(3):180-186. [22]BAKER R S,D’MELLO S K,RODRIGO M,et al.Better to be frustrated than bored:The incidence and prevalence of affect during interactions with three computer-based learning environments [J].International Journal of Human-Computer Studies,2010,68(4):223-241. [23]GUO Y,ZHANG L,HU Y,et al.Ms-celeb-1m:A dataset and benchmark for large-scale face recognition[C]//European Conference on Computer Vision.Cham:Springer,2016:87-102. [24]BARSOUM E,ZHANG C,FERRER C C,et al.Training deep networks for facial expression recognition with crowd-sourced label distribution [C]//Proceedings of the 18th ACM International Conference on Multimodal Interaction.2016:279-283. [25]HE K,ZHANG X,REN S,et al.Deep residual learning for image recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:770-778. [26]MENG D,PENG X,WANG K,et al.Frame attention networks for facial expression recognition in videos[C]//2019 IEEE International Conference on Image Processing(ICIP).IEEE,2019:3866-3870. [27]CHEN X,CHENG J,SONG R,et al.Video-based heart ratemeasurement:Recent advances and future prospects[J].IEEE Transactions on Instrumentation and Measurement,2018,68(10):3600-3615. [28]NIU X S,HANG H,SHAN S G.Review of Physiological Index Measurement Methods Based on R-ppg[J].Journal of Image andGraphics,2020,25(11):2321-2336. [29]WANG W,DENG B A C,STUIJK S,et al.Algorithmic principles of remote PPG[J].IEEE Transactions on Biomedical Engineering,2016,64(7):1479-1491. |
| [1] | 曾梓琳, 胡志刚, 尚鹏, 王新征, 付东辽. 基于面部视频的非接触式心率检测方法研究 Non-contact Heart Rate Detection Based on Facial Video 计算机科学, 2022, 49(11A): 211000182-6. https://doi.org/10.11896/jsjkx.211000182 |
| [2] | 柴冰, 李冬冬, 王喆, 高大启. 融合频率和通道卷积注意的脑电(EEG)情感识别 EEG Emotion Recognition Based on Frequency and Channel Convolutional Attention 计算机科学, 2021, 48(12): 312-318. https://doi.org/10.11896/jsjkx.201000141 |
| [3] | 刘凌云, 钱辉, 邢红杰, 董春茹, 张峰. 一种基于Q-学习算法的增量分类模型 Incremental Classification Model Based on Q-learning Algorithm 计算机科学, 2020, 47(8): 171-177. https://doi.org/10.11896/jsjkx.190600150 |
| [4] | 孔芳, 李奇之, 李帅. 在线影响力最大化研究综述 Survey on Online Influence Maximization 计算机科学, 2020, 47(5): 7-13. https://doi.org/10.11896/jsjkx.200200071 |
| [5] | 郑纯军, 王春立, 贾宁. 语音任务下声学特征提取综述 Survey of Acoustic Feature Extraction in Speech Tasks 计算机科学, 2020, 47(5): 110-119. https://doi.org/10.11896/jsjkx.190400122 |
| [6] | 何孝文, 胡一飞, 王海平, 陈默. 在线学习非负矩阵分解 Online Learning Nonnegative Matrix Factorization 计算机科学, 2019, 46(6A): 473-477. |
| [7] | 贾宁, 李瑛达. 基于智能可穿戴设备的个性化健康监管平台的构建 Construction of Personalized Health Monitoring Platform Based on Intelligent Wearable Device 计算机科学, 2019, 46(6A): 566-570. |
| [8] | 李德权,董翘,周跃进. 分布式在线条件梯度优化算法 Distributed Online Conditional Gradient Optimization Algorithm 计算机科学, 2019, 46(3): 332-337. https://doi.org/10.11896/j.issn.1002-137X.2019.03.049 |
| [9] | 杨海民, 潘志松, 白玮. 时间序列预测方法综述 Review of Time Series Prediction Methods 计算机科学, 2019, 46(1): 21-28. https://doi.org/10.11896／j.issn.1002-137X.2019.01.004 |
| [10] | 秦一休, 文益民, 何倩. 概念漂移数据流分类中的多源在线迁移学习算法 Multi-source Online Transfer Learning Algorithm for Classification of Data Streams with Concept Drift 计算机科学, 2019, 46(1): 64-72. https://doi.org/10.11896／j.issn.1002-137X.2019.01.010 |
| [11] | 张春霞,牛振东,施重阳,商建云. 基于细粒度学习情感本体的学习效果评估方法 ——以算法设计与分析课程为例 Learning Effect Evaluation Method Based on Fine-granularity Learning Emotion Ontology ——Taking Algorithm Design and Analysis Course as Example 计算机科学, 2018, 45(6A): 58-62. |
| [12] | 陈晋音, 方航, 林翔, 郑海斌, 杨东勇, 周晓. 基于在线学习行为分析的个性化学习推荐 Personal Learning Recommendation Based on Online Learning Behavior Analysis 计算机科学, 2018, 45(11A): 422-426. |
| [13] | 曹剑剑,冯军,汤文明,余瑛. 非接触式心率检测方法的颜色空间选择 Color Space Selection of Non-contact Heart Rate Measurement 计算机科学, 2017, 44(Z11): 260-262. https://doi.org/10.11896/j.issn.1002-137X.2017.11A.054 |
| [14] | 赵强利,蒋艳凰. 类别严重不均衡应用的在线数据流学习算法 Online Data Stream Mining for Seriously Unbalanced Applications 计算机科学, 2017, 44(6): 255-259. https://doi.org/10.11896/j.issn.1002-137X.2017.06.044 |
| [15] | 王长宝,李青雯,于化龙. 面向类别不平衡数据的主动在线加权极限学习机算法 Active,Online and Weighted Extreme Learning Machine Algorithm for Class Imbalance Data 计算机科学, 2017, 44(12): 221-226. https://doi.org/10.11896/j.issn.1002-137X.2017.12.040 |
|
||
首页