Computer Science

Computer Science ›› 2024, Vol. 51 ›› Issue (11A): 231200133-9.doi: 10.11896/jsjkx.231200133

• Interdiscipline & Application • Previous Articles     Next Articles

Recognition Method of Online Classroom Interaction Based on Learner State

RAO Yi1, YUAN Bochuan1, YUAN Yubo1,2   

  1. 1 School of Information Science and Engineering,East China University of Science and Technology,Shanghai 200237,China
    2 Shanghai Engineering Research Center of Big Data & Internet Audience,Shanghai 200072,China
  • Online:2024-11-16 Published:2024-11-13
  • About author:RAO Yi,born in 1999,postgraduate.Her main research interests include big data analysis and data mining.
    YUAN Yubo,born in 1976,Ph.D,associate pprofessor.His main research inte-rests include artificial intelligence,data science,big data analysis,data quality assessment and data mining.
  • Supported by:
    Shanghai Engineering Research Technology Center Project(18DZ2252300).

PDF (PC)

165

Abstract: With the widespread application of artificial intelligence in the field of education,online classrooms have become a highly convenient and efficient mode of modern education.However,effectively managing the learning status of students in the classroom has become an important challenge in education management.In light of this,method for recognizing learner interaction states in online classrooms is proposed.First,the online classroom data source is divided into video data and audio data.Based on video data,a multidimensional set of interaction state features is constructed,including learner upper body posture features,facial expression features,and facial features.Based on audio data,classroom response state features of the learners are built.Next,using a feature selection algorithm to select key features,a binary classification model is constructed to achieve precise recognition of students′ classroom interaction states using Bayesian optimization.Finally,an overall classroom assessment model is designed to provide comprehensive classroom assessment results for teachers and optimize teaching strategies.The accuracy of the learner interaction state recognition algorithm in this single-student classroom exceeds 93%,as validated on a self-constructed online classroom video data set.

Key words: Artificial intelligence, Education management, Facial feature, Posture feature, Facial expression recognition, Assessment of class

CLC Number: 

  • TP399
[1]ZHANG W.In the Blink of an Eye:70 Years of Monumental Changes in Chinese Educational Informatization[J].China Education Network,2019(10):7-10.
[2]SENSMEIERJ,ANDERSON C.ANI CONNECTION:Techno-logy Informatics Guiding Education Reform Moves Into Phase III:Implementation[J].Cin Computers Informatics Nursing,2009,27(4):265-266.
[3] WEI Y,QIN D,HU J,et al.Student Classroom Behavior Recognition Based on Deep Learning[J].Modern Educational Technology,2019,29(7):5-7.
[4]MASSOZ Q,LANGOHR T,FRANCOIS C,et al.The ULgmultimodality drowsiness database(called DROZY) and examples of use[C]// Winter Conference on Applications of Compu-ter Vision.Lake Placid:IEEE,2016:1-7.
[5]SVENSSON U.Blink behaviour based drowsiness detection:method development and validation[J].Journal of Swedish National Road and Transport Research Institute,2024,362A(1):37-39.
[6]ZALETELJ J,KOIR A.Predicting students' attention in theclassroom from Kinect facial and body features[J].Eurasip Journal on Image & Video Processing,2017,2017(1):80-82.
[7]SHINODA K,YOSHII M,YAMAGUCHI H,et al.DaytimeSleepiness Level Prediction Using Respiratory Information[C]//Twenty-Eighth International Joint Conference on Artificial Intelligence.Macao:IJCAI,2019:5967-5974.
[8]CAI Y W.Optimization Research on Wisdom Tree and Tencent Classroom under Computer Artificial Intelligence Algorithm[C]//International Conference on Data Analytics,Computing and Artificial Intelligence.Zakopane:ICDACAI,2022:334-337.
[9]ZHAO Y,YAN H,WANG Z.The Advisable Technology ofKey-Point Detection and Expression Recognition for an Intelligent Class System[J].Journal of Physics:Conference Series,2019,1187:052011.
[10]CHEN G,JI J,HUANG C.Student Classroom behavior Recognition based on OpenPose and Deep Learning[C]//7th International Conference on Intelligent Computing and Signal Processing.Xi'an:IEEE,2022:576-579.
[11]CHEN Y.Multidimensional Classroom Interaction AnalysisBased on Speech Recognition[D].Wuahn:Huazhong Normal University,2021.
[12]PENG S,NAGAO K.Recognition of Students' Mental States in Discussion Based on Multimodal Data and its Application to Educational Support[J].IEEE Access,2021,PP(99):1-1.
[13]EMILY D,TRENT K,DAVE D. Rethinking classroom observation. Journal of Educational Leadership,2014,71(8):24-29.
[14]OCHOA X,CHILUIZA K,MÉNDEZ,et al.Expertise estimation based on simple multimodal features[C]//ACM on International Conference on Multimodal Interaction.ACM,2013.
[15]MALDONADO-MAHAUAD J,PÉREZ-SANAGUSTÍN M,MORENO-MARCOS P M,et al.Predicting learners' success in a self-paced MOOC through sequence patterns of self-regulated learning[C]//Lifelong Technology-Enhanced Learning:13th European Conference on Technology Enhanced Learning(EC-TEL 2018).Leeds,UK,Springer International Publishing,2018:355-369.
[16]OVIATT S,GRAFSGAARD J,CHEN L,et al.Multimodallearning analytics:assessing learners' mental state during the process of learning[M]//The Handbook of Multimodal-Multisensor Interfaces:Signal Processing,Architectures,and Detection of Emotion and Cognition-Volume 2.Association for Computing Machinery and Morgan & Claypool,2018:331-374.
[17]GIANNAKOS M N,SHARMA K,PAPPAS I O,et al.Multimodal data as a means to understand the learning experience[J].International Journal of Information Management,2019,48:108-119.
[18]BADAVE H,KUBER M.Evaluation of Person Recognition Accuracy based on OpenPose parameters[C]//5th International Conference on Intelligent Computing and Control Systems.Madurai:ICICCS,2021:635-640.
[19]ARRIAGA O,VALDENEGRO-TORO M,PLÖGER P.Real-time Convolutional Neural Networks for Emotion and Gender Classification[J].arXiv:1710.07557,2017.
[20]XU N,LI S,WU X,et al.An APT Malware Classification Method Based on Adaboost Feature Selection and LightGBM[C]//Sixth International Conference on Data Science in Cyberspace.Shenzhen:IEEE,2021:635-639.
[21]ZHOU Y,SONG X,ZHOU M.Supply Chain Fraud PredictionBased on XGBoost Method[C]//2nd International Conference on Big Data,Artificial Intelligence and Internet of Things Engineering.Nanchang:IEEE,2021:539-542.
[22]ZHANG X,WU G X.Text Classification Method of DongbaClassics Based on CatBoost Algorithm[C]//The 8th International Symposium on Test Automation & Instrumentation.Online:ISTAI,2020:133-139.
[23]HOWARD A G,ZHU M,CHEN B et al.MobileNets:Efficient Convolutional Neural Networks for Mobile Vision Applications[J].arXiv:1704.04861V1,2017.
[24]HE K,ZHANG X,REN S,et al.Deep Residual Learning forImage Recognition[C]//2016 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).IEEE,2016:770-778.
[25]CHEN B,LIN R,ZOU H.A Short Term Load Periodic Prediction Model Based on GBDT[C]//18th International Conference on Communication Technology.Chongqing:IEEE,2018:1402-1406.
[1] ZHANG Huazhong, PAN Yuekai, TU Xiaoguang, LIU Jianhua, XU Luopeng, ZHOU Chao. Facial Expression Recognition Integrating 3D Facial Dynamic Information and Optical Flow Information [J]. Computer Science, 2024, 51(6A): 230700210-7.
[2] JIANG Rui, YANG Kaihui, WANG Xiaoming, LI Dapeng, XU Youyun. Attentional Interaction-based Deep Learning Model for Chinese Question Answering [J]. Computer Science, 2024, 51(6): 325-330.
[3] GUO Shangzhi, LIAO Xiaofeng, XIAN Kaiyi. Logical Regression Click Prediction Algorithm Based on Combination Structure [J]. Computer Science, 2024, 51(2): 73-78.
[4] WANG Shuaiwei, LEI Jie, FENG Zunlei, LIANG Ronghua. Review of Visual Representation Learning [J]. Computer Science, 2024, 51(11): 112-132.
[5] YAO Tianlei, CHEN Xiliang, YU Peiyi. Review of Generative Reinforcement Learning Based on Sequence Modeling [J]. Computer Science, 2024, 51(11): 213-228.
[6] XU Jun, ZHOU Peijin, ZHANG Haijing, ZHANG Hao, XU Yuzhong. Analysis of User Evaluation Indicator for AIGC Digital Illustration Design Principles [J]. Computer Science, 2024, 51(11): 47-53.
[7] DING Weilong, LIU Jinlong, ZHU Wei, LIAO Wanyin. Review of Quality Control Algorithms for Pathological Slides Based on Deep Learning [J]. Computer Science, 2024, 51(10): 276-286.
[8] ZHANG Ce, CHU Dianhui, ZHANG Qiao, LIU Peng, WEI Meng, LIU Xiaoying. Metaverse Teaching:A Higher Form of Digital Teaching Transformation in Higher Education [J]. Computer Science, 2024, 51(10): 1-9.
[9] DUAN Xinran, WANG Mei, HAN Tianli, ZHOU Hongyu, GUO Junqi, JI Weixing, HUANG Hua. Perception and Analysis of Teaching Process Based on Video Understanding [J]. Computer Science, 2024, 51(10): 56-66.
[10] ZHANG Wenqiong, LI Yun. Fairness Metrics of Machine Learning:Review of Status,Challenges and Future Directions [J]. Computer Science, 2024, 51(1): 266-272.
[11] WANG Zibo, ZHANG Yaofang, CHEN Yilu, LIU Hongri, WANG Bailing, WANG Chonghua. Hierarchical Task Network Planning Based Attack Path Discovery [J]. Computer Science, 2023, 50(9): 35-43.
[12] ZHOU Fengfan, LING Hefei, ZHANG Jinyuan, XIA Ziwei, SHI Yuxuan, LI Ping. Facial Physical Adversarial Example Performance Prediction Algorithm Based on Multi-modal Feature Fusion [J]. Computer Science, 2023, 50(8): 280-285.
[13] HUANG Pei, LIU Minghao, MA Feifei, ZHANG Jian. Automated Reasoning Techniques for Solving Combinatorial Mathematical Problems:A Survey [J]. Computer Science, 2023, 50(7): 167-175.
[14] WANG Dongli, YANG Shan, OUYANG Wanli, LI Baopu, ZHOU Yan. Explainability of Artificial Intelligence:Development and Application [J]. Computer Science, 2023, 50(6A): 220600212-7.
[15] WANG Zihan, TONG Xiangrong. Research Progress of Multi-agent Path Finding Based on Conflict-based Search Algorithms [J]. Computer Science, 2023, 50(6): 358-368.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.