计算机科学 ›› 2021, Vol. 48 ›› Issue (8): 334-339.doi: 10.11896/jsjkx.201000036

• 人机交互 • 上一篇    

基于骨骼轨迹聚合模型的课堂交互群体发现

高岩, 闫秋艳, 夏士雄, 张紫涵   

  1. 中国矿业大学计算机科学与技术学院 江苏 徐州221116
  • 收稿日期:2020-10-09 修回日期:2020-11-08 发布日期:2021-08-10
  • 通讯作者: 闫秋艳(yanqy@cumt.edu.cn)
  • 基金资助:
    国家自然科学基金(61977061,51934007)

Interactive Group Discovery Based on Skeleton Trajectory Aggregation Model in ClassEnvironment

GAO Yan, YAN Qiu-yan, XIA Shi-xiong, ZHANG Zi-han   

  1. School of Computer Science & Technology,China University of Mining and Technology,Xuzhou,Jiangsu 221116,China
  • Received:2020-10-09 Revised:2020-11-08 Published:2021-08-10
  • About author:GAO Yan,born in 1997,postgraduate,is a member of China Computer Federation.Her main research interests include education big data analysis and time series data mining.(TS19170003A31@cumt.edu.cn)YAN Qiu-yan,born in 1978,Ph.D,associate professor,is a member of China Computer Federation.Her main research interests include multi-modal image behavior recognition,education big data analysis and time series data mining.
  • Supported by:
    National Natural Science Foundation of China(61977061,51934007).

摘要: 传统的课堂行为识别方法侧重于交互行为本身的辨识,而非群体发现。课堂环境下实现交互群体的准确定位与发现,是进行个体行为识别的基础,但存在由遮挡造成的行为数据缺失问题。使用骨骼数据表示人体行为及运动轨迹,具有不受光线和背景干扰、数据表达简单等优点。针对骨骼数据的多人交互群体发现进行研究,提出了一种基于骨骼轨迹聚合模型的交互群体发现算法(Interactive Group Discovery Algorithm Based on Skeleton Trajectory Aggregation,IGSTA)。首先,将骨骼数据标准化到以人为中心的坐标系,减小尺寸变化和初始位置不同对识别精度的影响;其次,提出了一种多核表示的骨骼轨迹聚合模型,准确描述了学生交互行为群体的变化;最后,对聚合后的骨骼轨迹进行聚类,实现交互群体发现。采用Kinect获取模拟的课堂学生交互行为视频,通过实验验证了该方法的有效性,即在骨骼节点缺失的情况下,仍可准确发现课堂环境下的学生交互群体。

关键词: 交互群体, 骨骼数据, 轨迹聚类, 课堂交互

Abstract: Traditional class action recognition methods focus on the recognition of interactive behavior itself,not the group discovery.Accurate positioning and discovery of interactive groups in class environment is the basis for further individual behavior recognition,but there is a problem of missing behavior data caused by occlusion.Skeletal data represent human behavior and motion trajectory,and have the advantages of not being disturbed by light and background,and simple data expression.Aiming at multi-person interactive group discovery of skeleton data,an interactive group discovery algorithm based on skeleton trajectory aggregation (IGSTA) is proposed.Firstly,this paper standardizes the skeleton data into the human-centered coordinate system to reduce the impact on recognition accuracy of different body size and initial position of person.Secondly,a skeleton trajectory aggregation model based on multi-core representation is proposed to accurately describe the changes of students' interactive beha-vior groups.Finally,the aggregated skeleton trajectories are clustered to realize the discovery of interactive group.Kinect is used to obtain the simulated video of classroom student interaction behavior.Experiments proves the validity of the method,that is,in the case of missing skeleton nodes,the interaction group of students can be accurately found in class environment.

Key words: Interactive group, Skeleton data, Trajectory clustering, Class interaction

中图分类号: 

  • TP391
[1]KIM D,KIM D H,KWAK K C.Classification of K-Pop dance movements based on skeleton information obtained by a kinect sensor[J].Sensors,2017,17(6):1261.
[2]MILLE J,LEBORGNE A,TOUGNE L.Euclidean Distance-Based Skeletons:A Few Notes on Average Outward Flux and Ridgeness[J].Journal of Mathematical Imaging and Vision,2019,61(3):310-330.
[3]GHORBEL E,BOUTTEAU R,BOONAERT J,et al.Kinematic Spline Curves:A temporal invariant descriptor for fast action recognition[J].Image and Vision Computing,2018,77:60-71.
[4]BISWAS J,VELOSO M.Depth camera based indoor mobile robot localization and navigation[C]//Proceedings-IEEE International Conference on Robotics and Automation.2011:1697-1702.
[5]WU Q,CHEN Z,ZHOU Y X,et al.Human-computer interaction based on Kinect skeleton tracking technology[J].Journal of Changzhou University (Natural Science Edition),2014,26(3):6-12.
[6]LI J,TONG L J,YING X,et al.Simulation of 3D Human Wal-king Posture Based on Skeleton Data [J].Digital Technology and Application,2017(10):68-71.
[7]GHAZAL S,KHAN U S.Human posture classification using skeleton information[C]//2018 International Conference on Computing,Mathematics and Engineering Technologies (iCoMET).IEEE,2018:1-4.
[8]WEI Y T,QIN D Y,HU J M,et al.The Recognition ofStudents' Classroom Behaviors based on Deep Learning[J].Modern Educational Technology,2019,29(7):88-92.
[9]CAI H Z.Research and Application of Classroom Behavior Reco-gnition Based on Space-Time Interest Points[D].Nanjing:Nanjing Normal University,2014.
[10]CHEN J,XIE R M.Research on the Detection of Students'Classroom States Based on HOG Features[J].Journal of Chongqing University of Science and Technology (Natural Scien-ces Edition),2019,21(4):89-92.
[11]PAN M,WANG J,LUO Z.Modelling Study on Learning Affects for Classroom Teaching/Learning Auto-Evaluation[J].Science,2018,6(3):81-86.
[12]ZALETELJ J,KOŠIR A.Predicting students' attention in the classroom from Kinect facial and body features[J/OL].EURASIP Journal on Image and Video Processing,2017(1):80.https://doi.org/10.1186/s13640-017-0228-8.
[13]HUYNH-THE T,BANOS O,LE B V,et al.PAM-based flexible generative topic model for 3D interactive activity recognition[C]//2015 International Conference on Advanced Technologies for Communications (ATC).IEEE,2015.
[14]JI Y,YE G,CHENG H.Interactive body part contrast miningfor human interaction recognition[C]//IEEE International Conference on Multimedia and Expo Workshops (ICMEW).IEEE,2014.
[15]JI Y,CHENG H,ZHENG Y,et al.Learning contrastive feature distribution model for interaction recognition[J].Journal of Vi-sual Communication and Image Representation,2015,33:340-349.
[16]WU H,SHAO J,XU X,et al.Recognition and Detection ofTwo-Person Interactive Actions Using Automatically Selected Skeleton Features[J].IEEE Transactions on Human-Machine Systems,2017(99):1-7.
[17]BLOOM V,ARGYRIOU V,MAKRIS D.Hierarchical transfer learning for online recognition of compound actions[J].Compu-ter Vision and Image Understanding,2016,144(3):62-72.
[18]COPPOLA C,FARIA D R,NUNES U,et al.Social activity recognition based on probabilistic merging of skeleton features with proximity priors from RGB-D data[C]//2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).IEEE,2016.
[19]XU H,ZHOU Y,LIN W,et al.Unsupervised Trajectory Clustering via Adaptive Multi-kernel Based Shrinkage [C]//IEEE International Conference on Computer Vision.IEEE,2015.
[20]CAO Z,SIMON T,WEI S,et al.Realtime Multi-person 2DPose Estimation Using Part Affinity Fields[C]//2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).Honolulu,HI,USA,2017:1302-1310.
[21]WU D,SHAO L.Leveraging Hierarchical Parametric Networks for Skeletal Joints Based Action Segmentation and Recognition[C]//IEEE Conference on Computer Vision & Pattern Recognition.IEEE,2014.
[22]DAI S W,WU X J,GAO C F,et al.Fuzzy Clustering Based On Multiple Kernel[J].Computer Engineering and Applications,2016,52(2):65-66.
[1] 许华杰, 吴青华, 胡小明. 基于轨迹多特性的隐私保护算法[J]. 计算机科学, 2019, 46(1): 190-195.
[2] 范菁,阮体洪,吴佳敏,董天阳. 基于二次谱聚类和HMM-RF混合模型的车辆行为识别方法研究[J]. 计算机科学, 2016, 43(5): 288-293.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] 孙启,金燕,何琨,徐凌轩. 用于求解混合车辆路径问题的混合进化算法[J]. 计算机科学, 2018, 45(4): 76 -82 .
[2] 贾伟,华庆一,张敏军,陈锐,姬翔,王博. 基于改进粒子群优化的移动界面模式聚类算法[J]. 计算机科学, 2018, 45(4): 220 -226 .
[3] 邓霞, 常乐, 梁俊斌, 蒋婵. 移动机会网络组播路由的研究进展[J]. 计算机科学, 2018, 45(6): 19 -26 .
[4] 张昱, 高克宁, 于戈. 一种融合节点属性信息的社会网络链接预测方法[J]. 计算机科学, 2018, 45(6): 41 -45 .
[5] 胡雅鹏, 丁维龙, 王桂玲. 一种面向异构大数据计算框架的监控及调度服务[J]. 计算机科学, 2018, 45(6): 67 -71 .
[6] 张盼盼, 彭长根, 郝晨艳. 一种基于隐私偏好的隐私保护模型及其量化方法[J]. 计算机科学, 2018, 45(6): 130 -134 .
[7] 成静, 张涛, 王涛, 董占伟. 一种基于图复杂度的移动导航服务回归测试优先方法[J]. 计算机科学, 2018, 45(6): 141 -144 .
[8] 陈翔, 王秋萍. 基于代码修改的多目标有监督缺陷预测建模方法[J]. 计算机科学, 2018, 45(6): 161 -165 .
[9] 郭莹莹, 张丽平, 李松. 障碍环境中线段组最近邻查询方法研究[J]. 计算机科学, 2018, 45(6): 172 -175 .
[10] 沈夏炯, 张俊涛, 韩道军. 基于梯度提升回归树的短时交通流预测模型[J]. 计算机科学, 2018, 45(6): 222 -227 .