计算机科学

计算机科学 ›› 2024, Vol. 51 ›› Issue (11A): 231200133-9.doi: 10.11896/jsjkx.231200133

• 交叉&应用 • 上一篇    下一篇

在线课堂学习者互动状态识别方法

饶怡1, 袁博川1, 袁玉波1,2   

  1. 1 华东理工大学信息与科学工程学院 上海 200237
    2 上海大数据与互联网受众工程技术研究中心 上海 200072
  • 出版日期:2024-11-16 发布日期:2024-11-13
  • 通讯作者: 袁玉波(ybyuan@ecust.edu.cn)
  • 作者简介:(ryecust@163.com)
  • 基金资助:
    上海市工程技术中心项目(18DZ2252300)

Recognition Method of Online Classroom Interaction Based on Learner State

RAO Yi1, YUAN Bochuan1, YUAN Yubo1,2   

  1. 1 School of Information Science and Engineering,East China University of Science and Technology,Shanghai 200237,China
    2 Shanghai Engineering Research Center of Big Data & Internet Audience,Shanghai 200072,China
  • Online:2024-11-16 Published:2024-11-13
  • About author:RAO Yi,born in 1999,postgraduate.Her main research interests include big data analysis and data mining.
    YUAN Yubo,born in 1976,Ph.D,associate pprofessor.His main research inte-rests include artificial intelligence,data science,big data analysis,data quality assessment and data mining.
  • Supported by:
    Shanghai Engineering Research Technology Center Project(18DZ2252300).

PDF (PC)

166

摘要： 随着人工智能在教育领域的广泛应用,在线课堂已成为当今极为便捷高效的新教育模式。然而,如何有效管理学习者的课堂学习状态成为一项重要的教育管理难题。鉴于此,提出一种在线课堂学习者互动状态识别方法。首先,将在线课堂数据源分为视频数据和音频数据,基于视频数据构建了包括学习者上肢姿态特征、表情特征以及面部特征等多维度的互动状态特征,基于音频数据构建了学习者的课堂应答状态特征。其次,通过特征选择算法筛选出的关键特征,构建二分类模型,采用贝叶斯优化实现对学生课堂互动状态的精确识别。最后,设计了一个在线课堂总体情况评估模型,为教师提供全面的课堂评估结果,优化教学策略。在自建的在线课堂视频数据集上,该单名学习者课中互动状态识别算法的准确率能够达到93%以上。

关键词: 人工智能, 教育管理, 面部特征, 姿态特征, 表情识别, 课堂评估

Abstract: With the widespread application of artificial intelligence in the field of education,online classrooms have become a highly convenient and efficient mode of modern education.However,effectively managing the learning status of students in the classroom has become an important challenge in education management.In light of this,method for recognizing learner interaction states in online classrooms is proposed.First,the online classroom data source is divided into video data and audio data.Based on video data,a multidimensional set of interaction state features is constructed,including learner upper body posture features,facial expression features,and facial features.Based on audio data,classroom response state features of the learners are built.Next,using a feature selection algorithm to select key features,a binary classification model is constructed to achieve precise recognition of students′ classroom interaction states using Bayesian optimization.Finally,an overall classroom assessment model is designed to provide comprehensive classroom assessment results for teachers and optimize teaching strategies.The accuracy of the learner interaction state recognition algorithm in this single-student classroom exceeds 93%,as validated on a self-constructed online classroom video data set.

Key words: Artificial intelligence, Education management, Facial feature, Posture feature, Facial expression recognition, Assessment of class

中图分类号: 

  • TP399
[1]ZHANG W.In the Blink of an Eye:70 Years of Monumental Changes in Chinese Educational Informatization[J].China Education Network,2019(10):7-10.
[2]SENSMEIERJ,ANDERSON C.ANI CONNECTION:Techno-logy Informatics Guiding Education Reform Moves Into Phase III:Implementation[J].Cin Computers Informatics Nursing,2009,27(4):265-266.
[3] WEI Y,QIN D,HU J,et al.Student Classroom Behavior Recognition Based on Deep Learning[J].Modern Educational Technology,2019,29(7):5-7.
[4]MASSOZ Q,LANGOHR T,FRANCOIS C,et al.The ULgmultimodality drowsiness database(called DROZY) and examples of use[C]// Winter Conference on Applications of Compu-ter Vision.Lake Placid:IEEE,2016:1-7.
[5]SVENSSON U.Blink behaviour based drowsiness detection:method development and validation[J].Journal of Swedish National Road and Transport Research Institute,2024,362A(1):37-39.
[6]ZALETELJ J,KOIR A.Predicting students' attention in theclassroom from Kinect facial and body features[J].Eurasip Journal on Image & Video Processing,2017,2017(1):80-82.
[7]SHINODA K,YOSHII M,YAMAGUCHI H,et al.DaytimeSleepiness Level Prediction Using Respiratory Information[C]//Twenty-Eighth International Joint Conference on Artificial Intelligence.Macao:IJCAI,2019:5967-5974.
[8]CAI Y W.Optimization Research on Wisdom Tree and Tencent Classroom under Computer Artificial Intelligence Algorithm[C]//International Conference on Data Analytics,Computing and Artificial Intelligence.Zakopane:ICDACAI,2022:334-337.
[9]ZHAO Y,YAN H,WANG Z.The Advisable Technology ofKey-Point Detection and Expression Recognition for an Intelligent Class System[J].Journal of Physics:Conference Series,2019,1187:052011.
[10]CHEN G,JI J,HUANG C.Student Classroom behavior Recognition based on OpenPose and Deep Learning[C]//7th International Conference on Intelligent Computing and Signal Processing.Xi'an:IEEE,2022:576-579.
[11]CHEN Y.Multidimensional Classroom Interaction AnalysisBased on Speech Recognition[D].Wuahn:Huazhong Normal University,2021.
[12]PENG S,NAGAO K.Recognition of Students' Mental States in Discussion Based on Multimodal Data and its Application to Educational Support[J].IEEE Access,2021,PP(99):1-1.
[13]EMILY D,TRENT K,DAVE D. Rethinking classroom observation. Journal of Educational Leadership,2014,71(8):24-29.
[14]OCHOA X,CHILUIZA K,MÉNDEZ,et al.Expertise estimation based on simple multimodal features[C]//ACM on International Conference on Multimodal Interaction.ACM,2013.
[15]MALDONADO-MAHAUAD J,PÉREZ-SANAGUSTÍN M,MORENO-MARCOS P M,et al.Predicting learners' success in a self-paced MOOC through sequence patterns of self-regulated learning[C]//Lifelong Technology-Enhanced Learning:13th European Conference on Technology Enhanced Learning(EC-TEL 2018).Leeds,UK,Springer International Publishing,2018:355-369.
[16]OVIATT S,GRAFSGAARD J,CHEN L,et al.Multimodallearning analytics:assessing learners' mental state during the process of learning[M]//The Handbook of Multimodal-Multisensor Interfaces:Signal Processing,Architectures,and Detection of Emotion and Cognition-Volume 2.Association for Computing Machinery and Morgan & Claypool,2018:331-374.
[17]GIANNAKOS M N,SHARMA K,PAPPAS I O,et al.Multimodal data as a means to understand the learning experience[J].International Journal of Information Management,2019,48:108-119.
[18]BADAVE H,KUBER M.Evaluation of Person Recognition Accuracy based on OpenPose parameters[C]//5th International Conference on Intelligent Computing and Control Systems.Madurai:ICICCS,2021:635-640.
[19]ARRIAGA O,VALDENEGRO-TORO M,PLÖGER P.Real-time Convolutional Neural Networks for Emotion and Gender Classification[J].arXiv:1710.07557,2017.
[20]XU N,LI S,WU X,et al.An APT Malware Classification Method Based on Adaboost Feature Selection and LightGBM[C]//Sixth International Conference on Data Science in Cyberspace.Shenzhen:IEEE,2021:635-639.
[21]ZHOU Y,SONG X,ZHOU M.Supply Chain Fraud PredictionBased on XGBoost Method[C]//2nd International Conference on Big Data,Artificial Intelligence and Internet of Things Engineering.Nanchang:IEEE,2021:539-542.
[22]ZHANG X,WU G X.Text Classification Method of DongbaClassics Based on CatBoost Algorithm[C]//The 8th International Symposium on Test Automation & Instrumentation.Online:ISTAI,2020:133-139.
[23]HOWARD A G,ZHU M,CHEN B et al.MobileNets:Efficient Convolutional Neural Networks for Mobile Vision Applications[J].arXiv:1704.04861V1,2017.
[24]HE K,ZHANG X,REN S,et al.Deep Residual Learning forImage Recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2016:770-778.
[25]CHEN B,LIN R,ZOU H.A Short Term Load Periodic Prediction Model Based on GBDT[C]//18th International Conference on Communication Technology.Chongqing:IEEE,2018:1402-1406.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发