Computer Science

Computer Science ›› 2024, Vol. 51 ›› Issue (10): 10-16.doi: 10.11896/jsjkx.240400112

• Technology and Application of Intelligent Education • Previous Articles     Next Articles

Computational Perception Technologies in Intelligent Education:Systematic Review

LIU Feng1,2, LIU Yaxuan3, CHAI Xinyu2, JI Haohan2, ZHENG Zhixing2   

  1. 1 Shanghai International School of Chief Technology Officer,East China Normal University,Shanghai 200062,China
    2 School of Computer Science and Technology,East China Normal University,Shanghai 200062,China
    3 Faculty of Education,East China Normal University,Shanghai 200062,China
  • Received:2024-04-15 Revised:2024-07-07 Online:2024-10-15 Published:2024-10-11
  • About author:LIU Feng,born in 1988,Ph.D,is a se-nior member of CCF(No.93542S).His main research interests include deep learning technology,computation affection and blockchain.
  • Supported by:
    Shanghai Municipal Science and Technology Commission(22511105901),Beijing Key Laboratory of Behavior and Mental Health at Peking University and Special Fund for Talent Cultivation of Artificial Intelligence Enabled Psychology/Education Interdisciplinary Cross-disciplinary Talents(2024JCRCG10),School of Computer Science and Technology,East China Normal University.

PDF (PC)

327

Abstract: With the advancement and development of educational technology,computational perception technologies such as deep learning,machine learning,virtual reality,and the Internet of Things are increasingly being applied in human-centric,empathetic education,playing a key role in empowering educational transformation.However,there is still limited knowledge regarding the application of these technologies in intelligent education.This study systematically analyzes the progress and applications of computational perception technologies in intelligent education,drawing on perceptible data from both physical and virtual spaces,such as facial expressions,speech,text,eye movements,touch,and physiological signals.This study conducts a statistical analysis and screening of currently published journal articles and conference papers.The study explores the advancements and applications of these technologies,discussing their potential impacts and challenges in educational practice.Finally,a forward-looking discussion on the future development directions of computational perception technologies in intelligent education is presented based on the conclusions of this study.

Key words: Computational perception, Intelligent education, Computer vision, Natural language processing, Internet of Things, Virtual reality

CLC Number: 

  • G642
[1]SHETH A.Computing for human experience:Semantics-em-powered sensors,services,and social computing on the ubiquitous web[J].IEEE Internet Computing,2009,14(1):88-91.
[2]YU Z W,LI Q Y,WANG Q R,et al.Research progress andprospect of human-centred perceptual computing[J].Foresight Technology,2023,2(1):118-131.
[3]SHETH A,ANANTHARAM P,HENSON C.Semantic,cognitive,and perceptual computing:Paradigms that shape human experience[J].Computer,2016,49(3):64-72.
[4]LIB X.Letting students really feel the objective world:Classroom perceptual learning and its representational inference[J].Educational Research and Experimentation,2023(6):40-49.
[5]GREGORY R L.Knowledge in perception and illusion.Philo-sophical Transactions of the Royal Society of London[J].Series B:Biological Sciences,1997,352(1358):1121-1127.
[6]INSEL T R.The NIMH research domain criteria(RDoC) project:precision medicine for psychiatry[J].American Journal of Psychiatry,2014,171(4):395-397.
[7]ONNELA J P,RAUCH S L.Harnessing smartphone-based di-gital phenotyping to enhance behavioral and mental health[J].Neuropsychopharmacology,2016,41(7):1691-1696.
[8]TOROUS J,BUCCI S,BELL I H,et al.The growing field of digital psychiatry:current evidence and the future of apps,social media,chatbots,and virtual reality[J].World Psychiatry,2021,20(3):318-335.
[9]YUAN Z G.Digital Transformation in Education:What to Turn and How?[J].Journal of East China Normal University(Educational Sciences),2023,41(3):1-11.
[10]CHEN L,CHEN P,LIN Z.Artificial intelligence in education:A review[J].IEEE Access.2020,8:75264-75278.
[11]ZHU Z T,PENG H C,LEI Y H.Intelligent education:Thepractical path of intelligent education[J].Open Education Research,2018,24(4):13-24,42.
[12]YUAN Z G.Artificial Intelligence Facilitates the Realization of Personalized Education [N].China Information Weekly,2021-12-13(012).
[13]YUAN Z G.Educational Governance under the Perspective ofDigital Transformation[J].Journal of the Chinese Society of Education,2022(8):1-6,18.
[14]WANG Y F,LIU B Q.An overview of intelligent education application research[J].Modern Education Technology,2018,28(1):5-11.
[15]ZHOU A,YANG X,WEN W,et al.Computational affection:A catalyst for human-centered education[J].Science,2021,374:28-30.
[16]LIU F,WANG H,SHEN S,et al.OPO-FCM:A Computational Affection Based OCC-PAD-OCEAN Federation Cognitive Mo-deling Approach[J].IEEE Transactions on Computational Social Systems,2022,10(4):1813-1825.
[17]SHEN S,LIU F,ZHOU A.Mingling or Misalignment? Temporal Shift for Speech Emotion Recognition with Pre-Trained Representations[C]//2023 IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP).Greece:IEEE,2023:1-5.
[18]WANG H,LI B,WU S,et al.Rethinking the Learning Paradigmfor Dynamic Facial Expression Recognition[C]//2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition(CVPR).Vancouver,BC,Canada:IEEE,2023:17958-17968.
[19]HAMMAL Z,COHN J F.Automatic,objective,and efficientmeasurement of pain using automated face analysis[M]//Social and Interpersonal Dynamics in Pain:We DoŃt Suffer Alone.2018:121-146.
[20]SAGANOWSKI S,KOMOSZYŃSKA J,BEHNKE M,et al.Emognition dataset:emotion recognition with self-reports,facial expressions,and physiology using wearables[J].Scientific Data,2022,9(1):158.
[21]RUDNICKA A R,OWEN C G.An introduction to systematic reviews and meta-analyses in health care[J].Ophthalmic and Physiological Optics,2012,32(3):174-183.
[22]GIRSHICK R.Fast r-cnn [C]//2015 IEEE International Conference on Computer Vision(ICCV).Santiago:Chile,2015:1440-1448.
[23]TERVEN J,CÓRDOVA-ESPARZA D M,ROMERO-GONZ-ÁLEZ J A.A Comprehensive Review of YOLO Architectures in Computer Vision:From YOLOv1 to YOLOv8 and YOLO-NAS [J].Machine Learning and Knowledge Extraction,2023,5(4):1680-1716.
[24]LIU S,WANG Y,ZHENG Y.Computational intelligence inscience education [C]//Proceedings of the 6th International Conference on Frontiers of Educational Technologies(ICFET'20).Tokyo Japan:Association for Computing Machinery,2020:34-38.
[25]WU T T,LEE H Y,WANG W S,et al.Leveraging computer vi-sion for adaptive learning in STEM education:effect of engagement and self-efficacy [J].International Journal of Educational Technology in Higher Education,2023,20(1):53.
[26]ZHANG Y,QIU C,ZHONG N,et al.AI Education Based onEvaluating Concentration of Students in Class:Using Machine Vision to Recognize Students' Classroom Behavior [C]//Proceedings of the 2021 5th International Conference on Video and Image Processing(ICVIP '21).New York,United States:Association for Computing Machinery,2021:126-133.
[27]GIPPERT C,HÖRTER P,JUNKER R,et al.Professional vision of teaching as a focus-specific or focus-integrated skill e Conceptual considerations and video-based assessment [J].Teaching and Teacher Education,2022(117):103797.
[28]ALKHALISY M A E,ABD S H.Student Abnormal BehaviorDetection Using Dlib Combined With YOLO Models [C]//2022 International Conference on Intelligent Technology,System and Service for Internet of Everything(ITSS-IoE).Hadhramaut,Yemen:2022:1-7.
[29]SULTANA S M,RAO M K,SANTHI S G.Detecting Cheating in Online Assessments Using Face Recognition and Head Pose Estimation[C]//Rising Threats in Expert Applications and Solutions.Springer,Singapore:2022:615-622.
[30]ZHANG X,ZHANG X,DOLAH J B.Intelligent ClassroomTeaching Assessment System Based on Deep Learning Model Face Recognition Technology [J].Scientific Programming,2022(10):1851409.
[31]THOLETI P,THIRUMALESH B,ANU B,et al.Facial and Behavioural Analysis for Classroom Management using Computer Vision[EB/OL].[2024-02-18].https://doi.org/10.1109/icaaic56838.2023.10140607.
[32]NAIDU S L,BEBINA H,BHATIA P,et al.Classroom engagement evaluation using 360-degree view of the camera with deep learning techniques [EB/OL].[2024-02-18].https://doi.org/10.1117/12.2664147.
[33]DAVIDSON J L.Multi-Label Classification Computer VisionModels to Measure Online Classroom Engagement[D].Terre Haute:Indiana State University,2023:91-92.
[34]AGARWAL A,SEN B,MUKHOPADHYAY R,et al.Towardsmoocs for lipreading:using synthetic talking heads to train humans in lipreading at scale[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.Waikoloa,HI,U.S.A,2023:2217-2226.
[35]DAI Q.Application of a short video caption generation algo-rithm in international chinese education and teaching[J].Int.J.Web-Based Learn.Teach.Technol.,2023,18:1-19.
[36]SRIRAM R,HARALALKA R R J.Image to audio captioning for the visually impaired[C]//2023 International Conference on Recent Advances in Science and Engineering Technology(ICRASET).B G Nagara,India,2023:1-5.
[37]BALLENGER S.Voices from industry access for deaf and hard of hearing individuals in informational and educational remote sessions[J].Assistive Technology Outcomes and Benefits,2022,16(Special Issue 2):45-55.
[38]YASHINA E,RUBANIK T,CHUKHRAY A.Speech-to-textsoftware design for the high education learning[C]//International Workshop of IT-professionals on Artificial Intelligence.Łódź,Poland,2022:44-56.
[39]FILIGHERA A,BONGARD L,STEUER T,et al.Towards avocalization feedback pipeline for language learners[C]//2022 International Conference on Advanced Learning Technologies(ICALT).Bucharest,Romania,2022:248-252.
[40]WANG L.A machine learning assessment system for spokenenglish based on linear predictive coding[J].Mobile Inf.Syst.,2022,2022:1-12.
[41]LO C K.What is the impact of ChatGPT on education? A rapid review of the literature[J].Education Sciences,2023,13(4):410.
[42]ELSHINY R M,HAMDY A.Automatic question generationusing natural language processing and transformers[C]//2023 International Conference on Computer and Applications(ICCA).Cairo,Egypt,2023:1-6.
[43]FAKHRURROJA H,UTARI T,VICTOR A,et al.Talk to me:artificial intelligence “virtual friend” for depression sufferers using term frequency-inverse document frequency(TF-IDF) and finite state machine method[C]//2022 International Conference on ICT for Smart Society(ICISS).Bandung,Indonesia,2022:1-6.
[44]DEHBOZORGI N,KUNUKU M T.Affective computing:a to-pic-based ser approach on collaborative discussions in academic setting[C]//2023 IEEE Frontiers in Education Conference(FIE).College Station,Texas,USA,2023:1-7.
[45]TAYAL M A,JOSHI R,DARVEKAR M,et al.Automatedexam paper checking using semantic analysis[C]//2023 OITS International Conference on Information Technology(OCIT).Raipur,India,2023:957-962.
[46]BAI Y,HUBERS F,CUCCHIARINI C,et al.Asr-based evaluation and feedback for individualized reading practice[C]//Proc.Interspeech 2020.Shanghai,China,2020:3870-3874.
[47]ALQAHTANI T,BADRELDIN H A,ALRASHED M,et al.The emergent role of artificial intelligence,natural learning processing,and large language models in higher education and research[J].Research in Social and Administrative Pharmacy,2023,19(8):1236-1242.
[48]PUTJORN T,PUTJORN P.Augmented imagination:exploring generative ai from the perspectives of young learners[C]//2023 15th International Conference on Information Technology and Electrical Engineering(ICITEE).Chiang Mai,Thailand,2023:353-358.
[49]NARIAI T,ITAI S,KOJIMA H.The effect of english textreadability on speech duration of second language learners[C]//2022 6th International Conference on Universal Village(UV).Boston,Massachusetts,U.S.A,IEEE,2022:1-6.
[50]CHEN C Y,HSU Y H,CHANG C.Accelerating hakka speech recognition research and development using the whisper model[C]//Proceedings of the 35th Conference on Computational Linguistics and Speech Processing(ROCLING 2023).Taipei City,Taiwan:The Association for Computational Linguistics and Chinese Language Processin(ACLCLP),2023:367-370.
[51]YU T,FRIESKE R,XU P,et al.Automatic speech recognition datasets in cantonese:a survey and new dataset[C]//Procee-dings of the 13th Conference on Language Resources and Evaluation(LREC 2022).Marseille:ELRA,2022:6487-6494.
[52]ANDERSON N,MCGOWAN A,GALWAY L,et al.Implementing generative ai and large language models in education[C]//2023 7th International Symposium on Innovative Approaches in Smart Technologies(ISAS).Istanbul,Turkey:IEEE,2023:1-6.
[53]SHAIK T,TAO X,LI Y,et al.A review of the trends and challenges in adopting natural language processing methods for education feedback analysis[J].IEEE Access,2022,10:56720-56739.
[54]YU J,LUO G,XIAO T,et al.Mooccube:a large-scale data repository for nlp applications in moocs[C]//Proceedings of the 2020 Conference on Empirical Methods.Online:Association for Computational Linguistics,2020:3135-3142.
[55]ASHTON K.That ‘internet of things' thing[J].RFID Journal,2009,22(7):97-114.
[56]GUBBI J,BUYYA R,MARUSIC S,et al.Internet of Things(IoT):A vision,architectural elements,and future directions.Future generation computer systems[J].Future Generation Computer Systems,2013,29(7):1645-1660.
[57]GURGU E,ANDRONIE M,ANDRONIE M,et al.Does theconvergence of the blockchain,the internet of things and artificial intelligence changing our lives,education and the known world of the internet?Some changes and perspectives for the international economy[C]//International Conference on Economic Sciences and Business Administration.2019:69-88.
[58]HUANG K,LIU R,CHENG Y,et al.IoT Based Smart Classroom Environmental Monitoring System Design and Implementation[C]//International Conference on Wireless Communications,Networking and Applications.Singapore:Springer Nature Singapore,2022:827-834.
[59]JASIM N A,ALRIKABI H T S,FARHAN M S.Internet of Things(IoT) application in the assessment of learning process[C]//IOP Conference Series:Materials Science and Enginee-ring.IOP Publishing,2021.
[60]AL-TAAI S H H,KANBER H A,AL-DULAIMI W A M.The importance of using the internet of things in education[J].Int.J.Emerg.Technol.Learn.,2023,18(1):19-39.
[61]LI M.Research on Service Mode and Technical Framework ofSmart Library from the Perspective of Metaverse[J].Information Studies:Theory & Application,2022,45(3):89-93,88.
[62]WANG J,JING L.A Virtual Reality Whiteboard System for Remote Collaboration Using Natural Handwriting[J].Electronics,2022,11(24):4152.
[63]SLATER M,SANCHEZ-VIVES M V.Enhancing our lives with immersive virtual reality[J].Frontiers in Robotics and AI,2016,3:74.
[64]DING Y,LI Y,CHENG L.Application of Internet of Things and virtual reality technology in college physical education[J].IEEE Access,2020,8:96065-96074.
[65]PAN Z,SUN Y,YAO Z W,et al.Application of virtual reality in English teaching[C]//2021 3rd World Symposium on Artificial Intelligence(WSAI).IEEE,2021:64-71.
[66]LUCAS J,GAJJAR D.Influence of virtual reality on studentlearning in undergraduate construction education[J].International Journal of Construction Education and Research,2022,18(4):374-387.
[67]CHEN P H C,GADEPALLI K,MACDONALD R,et al.An augmented reality microscope with real-time artificial intelligence integration for cancer diagnosis[J].Nature Medicine,2019,25(9):1453-1457.
[68]MAAS M J,HUGHES J M.Virtual,augmented and mixed rea-lity in K-12 education:A review of the literature[J].Technology,Pedagogy and Education,2020,29(2):231-249.
[69]XIE F.Application and research on immersive classroom teaching based on virtual reality technology under 5G network[J].Computer Engineering and Science,2019,41(S1):14-17.
[70]LUO Z H,XU M Y.Integration of Teaching and Evaluation in Junior High School English Courses Supported by Smart Learning Spaces[J].Digital Teaching in Primary and Secondary Schools,2022(4):74-78.
[71]YU Z W,YU Z Y,ZHOU X S.Socially aware computing:Concepts,issues and their research progress[J].Journal of Computing,2012,35(1):16-26.
[72]YANG X M,ZHANG H,GUO L M,et al.Development challenges and breakthrough path of educational artificial intelligence[J].Modern Distance Education Research,2018(3):30-38.
[73]ZHENG Y H,ZHOU D H,ZHANG Y H,et al.ChatGPT in theperspective of computational pedagogy:connotation,themes,reflections and challenges[J].Journal of East China Normal University(Education Science Edition),2023,41(7):91-102.
[1] ZHU Fukun, TENG Zhen, SHAO Wenze, GE Qi, SUN Yubao. Semantic-guided Neural Network Critical Data Routing Path [J]. Computer Science, 2024, 51(9): 155-161.
[2] GUO Zhiqiang, GUAN Donghai, YUAN Weiwei. Word-Character Model with Low Lexical Information Loss for Chinese NER [J]. Computer Science, 2024, 51(8): 272-280.
[3] CHEN Liang, LI Zhihua. Abnormal Traffic Detection Method for Multi-stage Attacks of Internet of Things Botnets [J]. Computer Science, 2024, 51(8): 379-386.
[4] CAI Wenliang, HUANG Jun. Lane Detection Method Based on RepVGG [J]. Computer Science, 2024, 51(7): 236-243.
[5] HUANG Haixin, CAI Mingqi, WANG Yuyao. Review of Point Cloud Semantic Segmentation Based on Graph Convolutional Neural Networks [J]. Computer Science, 2024, 51(6A): 230400196-7.
[6] YANG Binxia, LUO Xudong, SUN Kaili. Recent Progress on Machine Translation Based on Pre-trained Language Models [J]. Computer Science, 2024, 51(6A): 230700112-8.
[7] WANG Yingjie, ZHANG Chengye, BAI Fengbo, WANG Zumin. Named Entity Recognition Approach of Judicial Documents Based on Transformer [J]. Computer Science, 2024, 51(6A): 230500164-9.
[8] LI Minzhe, YIN Jibin. TCM Named Entity Recognition Model Combining BERT Model and Lexical Enhancement [J]. Computer Science, 2024, 51(6A): 230900030-6.
[9] PENG Bo, LI Yaodong, GONG Xianfu, LI Hao. Method for Entity Relation Extraction Based on Heterogeneous Graph Neural Networks and TextSemantic Enhancement [J]. Computer Science, 2024, 51(6A): 230700071-5.
[10] YUAN Zhen, LIU Jinfeng. Denoising Autoencoders Based on Lossy Compress Coding [J]. Computer Science, 2024, 51(6A): 230400172-7.
[11] LU Dongsheng, LONG Hua. Method for Homologous Spectrum Monitoring Data Identification Based on Spectrum SIFT [J]. Computer Science, 2024, 51(6A): 230300177-7.
[12] ZANG Hongrui, YANG Tingting, LIU Hongbo, MA Kai. Study on Cryptographic Verification of Distributed Federated Learning for Internet of Things [J]. Computer Science, 2024, 51(6A): 230700217-5.
[13] LI Bin, WANG Haochang. Implementation and Application of Chinese Grammatical Error Diagnosis System Based on CRF [J]. Computer Science, 2024, 51(6A): 230900073-6.
[14] ZHANG Mingdao, ZHOU Xin, WU Xiaohong, QING Linbo, HE Xiaohai. Unified Fake News Detection Based on Semantic Expansion and HDGCN [J]. Computer Science, 2024, 51(4): 299-306.
[15] TU Xin, ZHANG Wei, LI Jidong, LI Meijiao , LONG Xiangbo. Study on Automatic Classification of English Tense Exercises for Intelligent Online Teaching [J]. Computer Science, 2024, 51(4): 353-358.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.