计算机科学

计算机科学 ›› 2024, Vol. 51 ›› Issue (10): 10-16.doi: 10.11896/jsjkx.240400112

• 智能教育技术及应用 • 上一篇    下一篇

智能教育中可计算感知技术:系统性综述

刘峰1,2, 刘雅旋3, 柴新宇2, 季昊涵2, 郑之行2   

  1. 1 华东师范大学上海国际首席技术官学院 上海 200062
    2 华东师范大学计算机科学与技术学院 上海 200062
    3 华东师范大学教育学部 上海 200062
  • 收稿日期:2024-04-15 修回日期:2024-07-07 出版日期:2024-10-15 发布日期:2024-10-11
  • 通讯作者: 刘峰(lsttoy@163.com)
  • 基金资助:
    上海市科学技术委员会资助项目(22511105901);北京大学行为与心理健康北京市重点实验室资助项目;华东师范大学计算机科学与技术学院“人工智能赋能心理／教育学科交叉人才培养专项基金(2024JCRC10)”

Computational Perception Technologies in Intelligent Education:Systematic Review

LIU Feng1,2, LIU Yaxuan3, CHAI Xinyu2, JI Haohan2, ZHENG Zhixing2   

  1. 1 Shanghai International School of Chief Technology Officer,East China Normal University,Shanghai 200062,China
    2 School of Computer Science and Technology,East China Normal University,Shanghai 200062,China
    3 Faculty of Education,East China Normal University,Shanghai 200062,China
  • Received:2024-04-15 Revised:2024-07-07 Online:2024-10-15 Published:2024-10-11
  • About author:LIU Feng,born in 1988,Ph.D,is a se-nior member of CCF(No.93542S).His main research interests include deep learning technology,computation affection and blockchain.
  • Supported by:
    Shanghai Municipal Science and Technology Commission(22511105901),Beijing Key Laboratory of Behavior and Mental Health at Peking University and Special Fund for Talent Cultivation of Artificial Intelligence Enabled Psychology/Education Interdisciplinary Cross-disciplinary Talents(2024JCRCG10),School of Computer Science and Technology,East China Normal University.

PDF (PC)

333

摘要： 随着教育技术的进步和发展,以深度学习、机器学习、虚拟现实、物联网为代表的可计算感知技术在以人为本的教育中的应用愈发广泛,同时也是赋能教育发展变革的关键力量之一。然而,目前对于可计算感知技术在智能教育中的应用还知之甚少。以面部表情、语音、文本、眼动、触觉、生理信号等可感知的物理及虚拟空间中可感知数据为基础,对当前公开发表的期刊文献和会议论文进行筛选和统计,系统分析了智能教育中可计算感知技术的进展与应用,并提出了其对教育实践的潜在影响和挑战。最后,基于结论对智能教育中可计算感知技术的未来发展方向展开了前瞻讨论。

关键词: 可计算感知, 智能教育, 计算机视觉, 自然语言处理, 物联网, 虚拟现实

Abstract: With the advancement and development of educational technology,computational perception technologies such as deep learning,machine learning,virtual reality,and the Internet of Things are increasingly being applied in human-centric,empathetic education,playing a key role in empowering educational transformation.However,there is still limited knowledge regarding the application of these technologies in intelligent education.This study systematically analyzes the progress and applications of computational perception technologies in intelligent education,drawing on perceptible data from both physical and virtual spaces,such as facial expressions,speech,text,eye movements,touch,and physiological signals.This study conducts a statistical analysis and screening of currently published journal articles and conference papers.The study explores the advancements and applications of these technologies,discussing their potential impacts and challenges in educational practice.Finally,a forward-looking discussion on the future development directions of computational perception technologies in intelligent education is presented based on the conclusions of this study.

Key words: Computational perception, Intelligent education, Computer vision, Natural language processing, Internet of Things, Virtual reality

中图分类号: 

  • G642
[1]SHETH A.Computing for human experience:Semantics-em-powered sensors,services,and social computing on the ubiquitous web[J].IEEE Internet Computing,2009,14(1):88-91.
[2]YU Z W,LI Q Y,WANG Q R,et al.Research progress andprospect of human-centred perceptual computing[J].Foresight Technology,2023,2(1):118-131.
[3]SHETH A,ANANTHARAM P,HENSON C.Semantic,cognitive,and perceptual computing:Paradigms that shape human experience[J].Computer,2016,49(3):64-72.
[4]LIB X.Letting students really feel the objective world:Classroom perceptual learning and its representational inference[J].Educational Research and Experimentation,2023(6):40-49.
[5]GREGORY R L.Knowledge in perception and illusion.Philo-sophical Transactions of the Royal Society of London[J].Series B:Biological Sciences,1997,352(1358):1121-1127.
[6]INSEL T R.The NIMH research domain criteria(RDoC) project:precision medicine for psychiatry[J].American Journal of Psychiatry,2014,171(4):395-397.
[7]ONNELA J P,RAUCH S L.Harnessing smartphone-based di-gital phenotyping to enhance behavioral and mental health[J].Neuropsychopharmacology,2016,41(7):1691-1696.
[8]TOROUS J,BUCCI S,BELL I H,et al.The growing field of digital psychiatry:current evidence and the future of apps,social media,chatbots,and virtual reality[J].World Psychiatry,2021,20(3):318-335.
[9]YUAN Z G.Digital Transformation in Education:What to Turn and How?[J].Journal of East China Normal University(Educational Sciences),2023,41(3):1-11.
[10]CHEN L,CHEN P,LIN Z.Artificial intelligence in education:A review[J].IEEE Access.2020,8:75264-75278.
[11]ZHU Z T,PENG H C,LEI Y H.Intelligent education:Thepractical path of intelligent education[J].Open Education Research,2018,24(4):13-24,42.
[12]YUAN Z G.Artificial Intelligence Facilitates the Realization of Personalized Education [N].China Information Weekly,2021-12-13(012).
[13]YUAN Z G.Educational Governance under the Perspective ofDigital Transformation[J].Journal of the Chinese Society of Education,2022(8):1-6,18.
[14]WANG Y F,LIU B Q.An overview of intelligent education application research[J].Modern Education Technology,2018,28(1):5-11.
[15]ZHOU A,YANG X,WEN W,et al.Computational affection:A catalyst for human-centered education[J].Science,2021,374:28-30.
[16]LIU F,WANG H,SHEN S,et al.OPO-FCM:A Computational Affection Based OCC-PAD-OCEAN Federation Cognitive Mo-deling Approach[J].IEEE Transactions on Computational Social Systems,2022,10(4):1813-1825.
[17]SHEN S,LIU F,ZHOU A.Mingling or Misalignment? Temporal Shift for Speech Emotion Recognition with Pre-Trained Representations[C]//2023 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP).Greece:IEEE,2023:1-5.
[18]WANG H,LI B,WU S,et al.Rethinking the Learning Paradigmfor Dynamic Facial Expression Recognition[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).Vancouver,BC,Canada:IEEE,2023:17958-17968.
[19]HAMMAL Z,COHN J F.Automatic,objective,and efficientmeasurement of pain using automated face analysis[M]//Social and Interpersonal Dynamics in Pain:We DoŃt Suffer Alone.2018:121-146.
[20]SAGANOWSKI S,KOMOSZYŃSKA J,BEHNKE M,et al.Emognition dataset:emotion recognition with self-reports,facial expressions,and physiology using wearables[J].Scientific Data,2022,9(1):158.
[21]RUDNICKA A R,OWEN C G.An introduction to systematic reviews and meta-analyses in health care[J].Ophthalmic and Physiological Optics,2012,32(3):174-183.
[22]GIRSHICK R.Fast r-cnn [C]//2015 IEEE International Conference on Computer Vision(ICCV).Santiago:Chile,2015:1440-1448.
[23]TERVEN J,CÓRDOVA-ESPARZA D M,ROMERO-GONZ-ÁLEZ J A.A Comprehensive Review of YOLO Architectures in Computer Vision:From YOLOv1 to YOLOv8 and YOLO-NAS [J].Machine Learning and Knowledge Extraction,2023,5(4):1680-1716.
[24]LIU S,WANG Y,ZHENG Y.Computational intelligence inscience education [C]//Proceedings of the 6th International Conference on Frontiers of Educational Technologies(ICFET'20).Tokyo Japan:Association for Computing Machinery,2020:34-38.
[25]WU T T,LEE H Y,WANG W S,et al.Leveraging computer vi-sion for adaptive learning in STEM education:effect of engagement and self-efficacy [J].International Journal of Educational Technology in Higher Education,2023,20(1):53.
[26]ZHANG Y,QIU C,ZHONG N,et al.AI Education Based onEvaluating Concentration of Students in Class:Using Machine Vision to Recognize Students' Classroom Behavior [C]//Proceedings of the 2021 5th International Conference on Video and Image Processing(ICVIP '21).New York,United States:Association for Computing Machinery,2021:126-133.
[27]GIPPERT C,HÖRTER P,JUNKER R,et al.Professional vision of teaching as a focus-specific or focus-integrated skill e Conceptual considerations and video-based assessment [J].Teaching and Teacher Education,2022(117):103797.
[28]ALKHALISY M A E,ABD S H.Student Abnormal BehaviorDetection Using Dlib Combined With YOLO Models [C]//2022 International Conference on Intelligent Technology,System and Service for Internet of Everything(ITSS-IoE).Hadhramaut,Yemen:2022:1-7.
[29]SULTANA S M,RAO M K,SANTHI S G.Detecting Cheating in Online Assessments Using Face Recognition and Head Pose Estimation[C]//Rising Threats in Expert Applications and Solutions.Springer,Singapore:2022:615-622.
[30]ZHANG X,ZHANG X,DOLAH J B.Intelligent ClassroomTeaching Assessment System Based on Deep Learning Model Face Recognition Technology [J].Scientific Programming,2022(10):1851409.
[31]THOLETI P,THIRUMALESH B,ANU B,et al.Facial and Behavioural Analysis for Classroom Management using Computer Vision[EB/OL].[2024-02-18].https://doi.org/10.1109/icaaic56838.2023.10140607.
[32]NAIDU S L,BEBINA H,BHATIA P,et al.Classroom engagement evaluation using 360-degree view of the camera with deep learning techniques [EB/OL].[2024-02-18].https://doi.org/10.1117/12.2664147.
[33]DAVIDSON J L.Multi-Label Classification Computer VisionModels to Measure Online Classroom Engagement[D].Terre Haute:Indiana State University,2023:91-92.
[34]AGARWAL A,SEN B,MUKHOPADHYAY R,et al.Towardsmoocs for lipreading:using synthetic talking heads to train humans in lipreading at scale[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.Waikoloa,HI,U.S.A,2023:2217-2226.
[35]DAI Q.Application of a short video caption generation algo-rithm in international chinese education and teaching[J].Int.J.Web-Based Learn.Teach.Technol.,2023,18:1-19.
[36]SRIRAM R,HARALALKA R R J.Image to audio captioning for the visually impaired[C]//2023 International Conference on Recent Advances in Science and Engineering Technology(ICRASET).B G Nagara,India,2023:1-5.
[37]BALLENGER S.Voices from industry access for deaf and hard of hearing individuals in informational and educational remote sessions[J].Assistive Technology Outcomes and Benefits,2022,16(Special Issue 2):45-55.
[38]YASHINA E,RUBANIK T,CHUKHRAY A.Speech-to-textsoftware design for the high education learning[C]//International Workshop of IT-professionals on Artificial Intelligence.Łódź,Poland,2022:44-56.
[39]FILIGHERA A,BONGARD L,STEUER T,et al.Towards avocalization feedback pipeline for language learners[C]//2022 International Conference on Advanced Learning Technologies(ICALT).Bucharest,Romania,2022:248-252.
[40]WANG L.A machine learning assessment system for spokenenglish based on linear predictive coding[J].Mobile Inf.Syst.,2022,2022:1-12.
[41]LO C K.What is the impact of ChatGPT on education? A rapid review of the literature[J].Education Sciences,2023,13(4):410.
[42]ELSHINY R M,HAMDY A.Automatic question generationusing natural language processing and transformers[C]//2023 International Conference on Computer and Applications(ICCA).Cairo,Egypt,2023:1-6.
[43]FAKHRURROJA H,UTARI T,VICTOR A,et al.Talk to me:artificial intelligence “virtual friend” for depression sufferers using term frequency-inverse document frequency(TF-IDF) and finite state machine method[C]//2022 International Conference on ICT for Smart Society(ICISS).Bandung,Indonesia,2022:1-6.
[44]DEHBOZORGI N,KUNUKU M T.Affective computing:a to-pic-based ser approach on collaborative discussions in academic setting[C]//2023 IEEE Frontiers in Education Conference(FIE).College Station,Texas,USA,2023:1-7.
[45]TAYAL M A,JOSHI R,DARVEKAR M,et al.Automatedexam paper checking using semantic analysis[C]//2023 OITS International Conference on Information Technology(OCIT).Raipur,India,2023:957-962.
[46]BAI Y,HUBERS F,CUCCHIARINI C,et al.Asr-based evaluation and feedback for individualized reading practice[C]//Proc.Interspeech 2020.Shanghai,China,2020:3870-3874.
[47]ALQAHTANI T,BADRELDIN H A,ALRASHED M,et al.The emergent role of artificial intelligence,natural learning processing,and large language models in higher education and research[J].Research in Social and Administrative Pharmacy,2023,19(8):1236-1242.
[48]PUTJORN T,PUTJORN P.Augmented imagination:exploring generative ai from the perspectives of young learners[C]//2023 15th International Conference on Information Technology and Electrical Engineering(ICITEE).Chiang Mai,Thailand,2023:353-358.
[49]NARIAI T,ITAI S,KOJIMA H.The effect of english textreadability on speech duration of second language learners[C]//2022 6th International Conference on Universal Village(UV).Boston,Massachusetts,U.S.A,IEEE,2022:1-6.
[50]CHEN C Y,HSU Y H,CHANG C.Accelerating hakka speech recognition research and development using the whisper model[C]//Proceedings of the 35th Conference on Computational Linguistics and Speech Processing(ROCLING 2023).Taipei City,Taiwan:The Association for Computational Linguistics and Chinese Language Processin(ACLCLP),2023:367-370.
[51]YU T,FRIESKE R,XU P,et al.Automatic speech recognition datasets in cantonese:a survey and new dataset[C]//Procee-dings of the 13th Conference on Language Resources and Evaluation(LREC 2022).Marseille:ELRA,2022:6487-6494.
[52]ANDERSON N,MCGOWAN A,GALWAY L,et al.Implementing generative ai and large language models in education[C]//2023 7th International Symposium on Innovative Approaches in Smart Technologies(ISAS).Istanbul,Turkey:IEEE,2023:1-6.
[53]SHAIK T,TAO X,LI Y,et al.A review of the trends and challenges in adopting natural language processing methods for education feedback analysis[J].IEEE Access,2022,10:56720-56739.
[54]YU J,LUO G,XIAO T,et al.Mooccube:a large-scale data repository for nlp applications in moocs[C]//Proceedings of the 2020 Conference on Empirical Methods.Online:Association for Computational Linguistics,2020:3135-3142.
[55]ASHTON K.That ‘internet of things' thing[J].RFID Journal,2009,22(7):97-114.
[56]GUBBI J,BUYYA R,MARUSIC S,et al.Internet of Things(IoT):A vision,architectural elements,and future directions.Future generation computer systems[J].Future Generation Computer Systems,2013,29(7):1645-1660.
[57]GURGU E,ANDRONIE M,ANDRONIE M,et al.Does theconvergence of the blockchain,the internet of things and artificial intelligence changing our lives,education and the known world of the internet?Some changes and perspectives for the international economy[C]//International Conference on Economic Sciences and Business Administration.2019:69-88.
[58]HUANG K,LIU R,CHENG Y,et al.IoT Based Smart Classroom Environmental Monitoring System Design and Implementation[C]//International Conference on Wireless Communications,Networking and Applications.Singapore:Springer Nature Singapore,2022:827-834.
[59]JASIM N A,ALRIKABI H T S,FARHAN M S.Internet of Things(IoT) application in the assessment of learning process[C]//IOP Conference Series:Materials Science and Enginee-ring.IOP Publishing,2021.
[60]AL-TAAI S H H,KANBER H A,AL-DULAIMI W A M.The importance of using the internet of things in education[J].Int.J.Emerg.Technol.Learn.,2023,18(1):19-39.
[61]LI M.Research on Service Mode and Technical Framework ofSmart Library from the Perspective of Metaverse[J].Information Studies:Theory & Application,2022,45(3):89-93,88.
[62]WANG J,JING L.A Virtual Reality Whiteboard System for Remote Collaboration Using Natural Handwriting[J].Electronics,2022,11(24):4152.
[63]SLATER M,SANCHEZ-VIVES M V.Enhancing our lives with immersive virtual reality[J].Frontiers in Robotics and AI,2016,3:74.
[64]DING Y,LI Y,CHENG L.Application of Internet of Things and virtual reality technology in college physical education[J].IEEE Access,2020,8:96065-96074.
[65]PAN Z,SUN Y,YAO Z W,et al.Application of virtual reality in English teaching[C]//2021 3rd World Symposium on Artificial Intelligence(WSAI).IEEE,2021:64-71.
[66]LUCAS J,GAJJAR D.Influence of virtual reality on studentlearning in undergraduate construction education[J].International Journal of Construction Education and Research,2022,18(4):374-387.
[67]CHEN P H C,GADEPALLI K,MACDONALD R,et al.An augmented reality microscope with real-time artificial intelligence integration for cancer diagnosis[J].Nature Medicine,2019,25(9):1453-1457.
[68]MAAS M J,HUGHES J M.Virtual,augmented and mixed rea-lity in K-12 education:A review of the literature[J].Technology,Pedagogy and Education,2020,29(2):231-249.
[69]XIE F.Application and research on immersive classroom teaching based on virtual reality technology under 5G network[J].Computer Engineering and Science,2019,41(S1):14-17.
[70]LUO Z H,XU M Y.Integration of Teaching and Evaluation in Junior High School English Courses Supported by Smart Learning Spaces[J].Digital Teaching in Primary and Secondary Schools,2022(4):74-78.
[71]YU Z W,YU Z Y,ZHOU X S.Socially aware computing:Concepts,issues and their research progress[J].Journal of Computing,2012,35(1):16-26.
[72]YANG X M,ZHANG H,GUO L M,et al.Development challenges and breakthrough path of educational artificial intelligence[J].Modern Distance Education Research,2018(3):30-38.
[73]ZHENG Y H,ZHOU D H,ZHANG Y H,et al.ChatGPT in theperspective of computational pedagogy:connotation,themes,reflections and challenges[J].Journal of East China Normal University(Education Science Edition),2023,41(7):91-102.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发