计算机科学

计算机科学 ›› 2024, Vol. 51 ›› Issue (10): 135-143.doi: 10.11896/jsjkx.240400089

• 智能教育技术及应用 • 上一篇    下一篇

基于多尺度特征图卷积网络的教学行为识别及分析

李佳楠1, 李锐宜1, 赵至夫2, 宋娟1, 韩嘉泷1, 朱桐3   

  1. 1 西安电子科技大学计算机科学与技术学院 西安 710071
    2 西安电子科技大学人工智能学院 西安 710071
    3 西安电子科技大学前沿交叉研究院 西安 710071
  • 收稿日期:2024-04-15 修回日期:2024-07-05 出版日期:2024-10-15 发布日期:2024-10-11
  • 通讯作者: 赵至夫(zfzhao@xidian.edu.cn)
  • 作者简介:(lijianan@xidian.edu.cn)
  • 基金资助:
    西安电子科技大学教育教学改革重点项目(A2304);中央高校基本科研业务费项目(ZYTS24092,QTZX24085);国家自然科学基金青年科学基金(62202356,62302373)

Recognition and Analysis of Teaching Behavior Based on Multi-scale GCN

LI Jia'nan1, LI Ruiyi1, ZHAO Zhifu2, SONG Juan1, HAN Jialong1, ZHU Tong3   

  1. 1 School of Computer and Technology,Xidian University,Xi'an 710071,China
    2 School of Artificial Intelligence,Xidian University,Xi'an 710071,China
    3 Academy of Advanced Interdisciplinary Research,Xidian University,Xi'an 710071,China
  • Received:2024-04-15 Revised:2024-07-05 Online:2024-10-15 Published:2024-10-11
  • About author:LI Jia'nan,born in 1991,Ph.D,is a member of CCF(No.K8171M).Her main research interests include video understanding and action recognition.
    ZHAO Zhifu,born in 1990,Ph.D,is a member of CCF(No.A0143M).His main research interests include deep learning,video understanding,and compressive sensing.
  • Supported by:
    Key Project of Education Teaching Reform of Xidian University(A2304),Fundamental Research Funds for the Central Universities(ZYTS24092,QTZX24085) and Young Scientists Fund of the National Natural Science Foundation of China(62202356,62302373).

PDF (PC)

279

摘要： 在教育领域,课堂教学评价是提高教学质量的关键环节之一。随着数字化教育的推广,寻求一种智能化的评价方法变得尤为重要。为此,提出了一种基于骨架行为识别和滞后序列分析的新型方法,旨在更准确地对教师的教学行为进行捕获和分析,在减少人力资源消耗的同时,降低教学评价的主观性。首先,提出多尺度特征图卷积网络,并将其用于教师课堂行为分析。该网络在空间维度上使用多尺度语义特征融合模块捕捉骨架点和肢体部位两个尺度的特征;在时间维度上使用多尺度时序特征提取模块,并分别从全局和局部两个角度提取骨架数据的时间特征。然后,构建了教师课堂行为分析数据集,并在该数据集上验证了所提方法的有效性。最后,利用所提的骨架行为识别模型和滞后序列分析法,搭建了一套教学行为识别与分析系统。在进行不同课堂教学行为识别时,所提方法在教室行为识别与分析方面具有显著的优势。

关键词: 教师行为分析, 骨架序列, 数字化教育, 图卷积网络, 行为识别

Abstract: In the field of education,classroom teaching evaluation stands as a pivotal element in enhancing teaching quality.With the widespread adoption of digital education,the quest for an intelligent evaluation method becomes increasingly crucial.Therefore,this paper proposes a novel method based on skeleton action recognition and lagged sequence analysis,aiming to more accurately capture and analyze teachers' teaching behaviors while reducing manpower consumption and diminishing the subjectivity of teaching evaluations.Firstly,a multi-scale feature graph convolutional network is proposed and applied to analyze teacher classroom behaviors.This network utilizes a multi-scale semantic feature fusion module to capture features at two scales,skeleton points,and body parts,in the spatial dimension.In the temporal dimension,a multi-scale temporal feature extraction module is employed to extract temporal features of skeleton data from both global and local perspectives.Subsequently,a dataset for analyzing teachers' classroom behaviors is constructed,and the effectiveness of the proposed method is validated on this dataset.Finally,leveraging the proposed skeleton action recognition model and lagged sequence analysis,a system for recognizing and analyzing teaching behaviors is developed.The proposed method demonstrates significant advantages in classroom behavior recognition and analysis when applied to various classroom teaching scenarios.

Key words: Teaching behavior analysis, Skeleton sequence, Digital education, Graph convolution, Action recognition

中图分类号: 

  • TP391
[1]LI B H,ZHANG X Y.Design and Implementation of Classroom Teaching Video Analysis Software Based on ITIAS[J].Software,2019,40(1):46-50.
[2]ZHANG N L.Research on the Design of a Practical CourseTeaching Process Management System Based on Achievement Analysis[J].Education and Teaching Forum,2021(41):4.
[3]CHEN J T.Research on Intelligent Image Recognition andAnalysis of Classroom Behavior [D].Hangzhou:Zhejiang University,2019.
[4]TAN B,YANG S.Research on Student Classroom Behavior Detection Algorithm Based on Faster R-CNN [J].Modern Computer,2018(22):45-47.
[5]ZHENG Y.A Teacher Teaching Behavior Evaluation MethodBased on Posture Recognition [J].Software Engineering,2021,24(4);6-9.
[6]HUSSEIN M E,TORKI M,GOWAYYED M A,et al.Humanaction recognition using a temporal hierarchy of covariance descriptors on 3D joint locations[C]//International Joint Confe-rence on Artificial Intelligence(IJCAI).2013:2466-2472.
[7]VEMULAPALLI R,ARRATE F,CHELLAPPA R.Human action recognition by representing 3D skeletons as points in a lie group[C]//2014 IEEE Conference on Computer Vision and Pattern Recognition(CVPR).2014:588-595.
[8]VEERIAH V,ZHUANG N,QI G J.Differential recurrent neural networks for action recognition[C]//IEEE International Conference on Computer Vision(ICCV).2015:4041-4049.
[9]ZHANG P,LAN C,XING J,et al.View adaptive recurrent neural networks for high performance human action recognition from skeleton data[C]//IEEE International Conference on Computer Vision(ICCV).2017:2136-2145.
[10]LIU J,WANG G,DUAN L Y,et al.Skeleton-based human action recognition with global context-aware attention LSTM networks[J].IEEE Transactions on Image Processing,2017,27(4):1586-1599.
[11]YAN S,XIONG Y,LIN D.Spatial temporal graph convolutional networks for skeleton-based action recognition[C]//Procee-dings of the AAAI Conference on Artificial Intelligence.2018.
[12]SHI L,ZHANG Y,CHENG J,et al.Two-stream adaptive graph convolutional networks for skeleton-based action recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:12026-12035.
[13]LI B,LI X,ZHANG Z,et al.Spatio-temporal graph routing for skeleton-based action recognition[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2019:8561-8568.
[14]LI S,LI W,COOK C,et al.Independently recurrent neural network(indrnn):Building a longer and deeper rnn[C]//Procee-dings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:5457-5466.
[15]MONTI F,BOSCAINI D,MASCI J,et al.Geometric deep lear-ning on graphs and manifolds using mixture model cnns[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2017:5115-5124.
[16]LI M,CHEN S,CHEN X,et al.Actional-structural graph con-volutional networks for skeleton-based action recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:3595-3603.
[17]ZHANG X,XU C,TIAN X,et al.Graph edge convolutionalneural networks for skeleton-based action recognition[J].IEEE Transactions on Neural Networks and Learning Systems,2019,31(8):3047-3060.
[18]XU K,YE F,ZHONG Q,et al.Topology-aware convolutional neural network for efficient skeleton-based action recognition[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2022:2866-2874.
[19]CHI H,HA M H,CHI S,et al.Infogcn:Representation learning for human skeleton-based action recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2022:20186-20196.
[20]YANG W,ZHANG J,CAI J,et al.HybridNet:Integrating GCNand CNN for skeleton-based action recognition[J].Applied Intelligence,2023,53(1):574-585.
[21]LIU Z,ZHANG H,CHEN Z,et al.Disentangling and unifying graph convolutions for skeleton-based action recognition[C]//Proceedings of the IEEE/CVFConference on Computer Vision and Pattern Recognition.2020:143-152.
[22]CHEN Y,ZHANG Z,YUAN C,et al.Channel-wise topology refinement graph convolution for skeleton-based action recognition[C]//Proceedings of the IEEE/CVF International Confe-rence on Computer Vision.2021:13359-13368.
[23]LIU Y,ZHANG H,XU D,et al.Graph transformer networkwith temporal kernel attention for skeleton-based action recognition[J].Knowledge-Based Systems,2022,240:108146.
[24]SHAHROUDY A,LIU J,NG T T,et al.Ntu rgb+ d:A large scale dataset for 3d human activity analysis[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:1010-1019.
[25]YAN S,XIONG Y,LIN D.Spatial temporal graph convolutional networks for skeleton-based action recognition[C]//Procee-dings of the AAAI Conference on Artificial Intelligence.2018.
[26]SHI L,ZHANG Y,CHENG J,et al.Two-stream adaptive graph convolutional networks for skeleton-based action recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:12026-12035.
[27]CHEN Z,LI S,YANG B,et al.Multi-scale spatial temporalgraph convolutional network for skeleton-based action recognition[C]//Proceedings of the AAAI Conference on Artiffcial Intelligence.2021:1113-1122.
[28]SONG Y F,ZHANG Z,SHAN C,et al.Constructing stronger and faster baselines for skeleton-based action recognition[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2022,45(2):1474-1488.
[29]LEE J,LEE M,LEE D,et al.Hierarchically decomposed graph convolutional networks for skeleton-based action recognition[C]//Proceedings of the IEEE/CVF International Conference on Computer Vision.2023:10444-10453.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发