Computer Science ›› 2022, Vol. 49 ›› Issue (2): 156-161.doi: 10.11896/jsjkx.220100061

• Computer Vision: Theory and Application • Previous Articles     Next Articles

Graph Convolutional Skeleton-based Action Recognition Method for Intelligent Behavior Analysis

MIAO Qi-guang, XIN Wen-tian, LIU Ru-yi, XIE Kun, WANG Quan, YANG Zong-kai   

  1. School of Computer Science and Technology,Xidian University,Xi'an 710071,China
  • Received:2022-01-06 Revised:2022-01-09 Online:2022-02-15 Published:2022-02-23
  • About author:MIAO Qi-guang,born in 1972,Ph.D,professor,Ph.D supervisor,is a senior member of China Computer Federation and AC of CCF YOCSEF.His main research interests include CV and ML.
    LIU Ru-yi,born in 1989,Ph.,lecturer,is a member of China Computer Federation.Her main research interests include computer vision,big data analysis and object detection in remote sen-sing.
  • Supported by:
    National New Engineering Research and Practice Project(E-GCJYZL20200818),Computer Basic Education Teaching Research Project of the National Institute of Computer Basic Education Research Association(2021-AFCEC-459),China Adult Education Association's “14th Five-Year Plan” Adult Continuing Education Research Plan Key Project(2021-414ZA),Key Research/Key Projects of Shaanxi Higher Education Teaching Reform Research(21JG001,21BZ014),Guangxi Key Laboratory of Trusted Software(KX202061,KX202041),Xidian University Education and Teaching Reform Research Key Research Project(A21003),New Experimental Development and New Experimental Equipment Development Key Projects(SY21022I) and Academy of Integrated Circuit Innovation of Xidian University in Chongqing IUR Project(CQIRI-CXYHT-2021-06).

Abstract: Smart education is a new education model using modern information technology,and smart behavior analysis is the core component.In the complex classroom scenarios,traditional action recognition algorithms are seriously deficient in accuracy and timeliness.A graph convolutional method based on separation and attention mechanism (DSA-GCN) is proposed to solve the above problems.First,in order to solve the challenge that traditional algorithms are inherently inadequate in aggregating information in the channel domain,multidimensional channel mapping is performed by point-wise convolution,combining the ability of ST-GC to preserve the original spatio-temporal information with the separation ability of depth-separable convolution in spatial and channel feature learning to enhance model feature learning and abstract expressivity.Second,a multi-dimensional fused attention mechanism is used to enhance the model dynamic sensitivity in the spatial convolution domain using self-attention and channel attention mechanisms,and to enhance the key frame discrimination in the temporal convolution domain using temporal and channel attention fusion method.Experiment results show that DSA-GCN achieves better accuracy and effectiveness performance on NTU RGB+D and N-UCLA datasets,and prove the improvement of the ability to aggregate channel information.

Key words: Action recognition, Attention mechanism, Depth-wise separable convolution, Graph convolutional neural network, Skeleton-based action classification, Smart behavior analysis

CLC Number: 

  • TP391
[1]VEMULAPALLI R,ARRATE F,CHELLAPPA R.Human action recognition by representing 3d skeletons as points in a lie group[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2014:588-595.
[2]FERNANDO B,GAVVES E,ORAMAS J M,et al.Modelingvideo evolution for action recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:5378-5387.
[3]DU Y,WANG W,WANG L.Hierarchical recurrent neural network for skeleton based action recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2015:1110-1118.
[4]YAN S,XIONG Y,LIN D.Spatial temporal graph convolutional networks for skeleton-based action recognition[C]//Thirty-second AAAI Conference on Artificial Intelligence.2018.
[5]SHI L,ZHANG Y,CHENG J,et al.Two-stream adaptive graph convolutional networks for skeleton-based action recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:12026-12035.
[6]LIU Z,ZHANG H,CHEN Z,et al.Disentangling and unifying graph convolutions for skeleton-based action recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:143-152.
[7]YE F,PU S,ZHONG Q,et al.Dynamic GCN:Context-enriched topology learning for skeleton-based action recognition[C]//Proceedings of the 28th ACM International Conference on Multimedia.2020:55-63.
[8]DEGARDIN B,LOPES V,PROENÇA H.REGINA-Reasoning Graph Convolutional Networks in Human Action Recognition[J].arXiv:2105.06711,2021.
[9]QIN Z,LIU Y,JI P,et al.Leveraging Third-Order Features in Skeleton-Based Action Recognition[J].arXiv:2105.01563,2021.
[10]CHEN Z,LI S,YANG B,et al.Multi-Scale Spatial TemporalGraph Convolutional Network for Skeleton-Based Action Re-cognition[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2021:1113-1122.
[11]CHEN Y,ZHANG Z,YUAN C,et al.Channel-wise topologyrefinement graph convolution for skeleton-based action recognition[C]//Proceedings of the IEEE/CVF International Confe-rence on Computer Vision.2021:13359-13368.
[12]SI C,JING Y,WANG W,et al.Skeleton-based action recognition with spatial reasoning and temporal stack learning[C]//Proceedings of the European Conference on Computer Vision (ECCV).2018:103-118.
[13]TANG Y,TIAN Y,LU J,et al.Deep progressive reinforcement learning for skeleton-based action recognition[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2018:5323-5332.
[14]ZHANG X,XU C,TIAN X,et al.Graph edge convolutionalneural networks for skeleton-based action recognition[J].IEEE Transactions on Neural Networks and Learning Systems,2019,31(8):3047-3060.
[15]KIPF T N,WELLING M.Semi-supervised classification withgraph convolutional networks[J].arXiv:1609.02907,2016.
[16]BRUNA J,ZAREMBA W,SZLAM A,et al.Spectral networks and locally connected networks on graphs[J].arXiv:1312.6203,2013.
[17]HAMMOND D K,VANDERGHEYNST P,GRIBONVAL R.Wavelets on graphs via spectral graph theory[J].Applied and Computational Harmonic Analysis,2011,30(2):129-150.
[18]SHAHROUDY A,LIU J,NG T T,et al.Ntu rgb+ d:A large scale dataset for 3d human activity analysis[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.2016:1010-1019.
[19]WANG J,NIE X,XIA Y,et al.Cross-view action modeling,learning and recognition[C]//Proceedings of the IEEE Confe-rence on Computer Vision and Pattern Recognition.2014:2649-2656.
[1] ZHOU Fang-quan, CHENG Wei-qing. Sequence Recommendation Based on Global Enhanced Graph Neural Network [J]. Computer Science, 2022, 49(9): 55-63.
[2] DAI Yu, XU Lin-feng. Cross-image Text Reading Method Based on Text Line Matching [J]. Computer Science, 2022, 49(9): 139-145.
[3] ZHOU Le-yuan, ZHANG Jian-hua, YUAN Tian-tian, CHEN Sheng-yong. Sequence-to-Sequence Chinese Continuous Sign Language Recognition and Translation with Multi- layer Attention Mechanism Fusion [J]. Computer Science, 2022, 49(9): 155-161.
[4] XIONG Li-qin, CAO Lei, LAI Jun, CHEN Xi-liang. Overview of Multi-agent Deep Reinforcement Learning Based on Value Factorization [J]. Computer Science, 2022, 49(9): 172-182.
[5] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[6] WANG Ming, PENG Jian, HUANG Fei-hu. Multi-time Scale Spatial-Temporal Graph Neural Network for Traffic Flow Prediction [J]. Computer Science, 2022, 49(8): 40-48.
[7] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[8] ZHU Cheng-zhang, HUANG Jia-er, XIAO Ya-long, WANG Han, ZOU Bei-ji. Deep Hash Retrieval Algorithm for Medical Images Based on Attention Mechanism [J]. Computer Science, 2022, 49(8): 113-119.
[9] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[10] YAN Jia-dan, JIA Cai-yan. Text Classification Method Based on Information Fusion of Dual-graph Neural Network [J]. Computer Science, 2022, 49(8): 230-236.
[11] ZHANG Ying-tao, ZHANG Jie, ZHANG Rui, ZHANG Wen-qiang. Photorealistic Style Transfer Guided by Global Information [J]. Computer Science, 2022, 49(7): 100-105.
[12] ZENG Zhi-xian, CAO Jian-jun, WENG Nian-feng, JIANG Guo-quan, XU Bin. Fine-grained Semantic Association Video-Text Cross-modal Entity Resolution Based on Attention Mechanism [J]. Computer Science, 2022, 49(7): 106-112.
[13] XU Ming-ke, ZHANG Fan. Head Fusion:A Method to Improve Accuracy and Robustness of Speech Emotion Recognition [J]. Computer Science, 2022, 49(7): 132-141.
[14] MENG Yue-bo, MU Si-rong, LIU Guang-hui, XU Sheng-jun, HAN Jiu-qiang. Person Re-identification Method Based on GoogLeNet-GMP Based on Vector Attention Mechanism [J]. Computer Science, 2022, 49(7): 142-147.
[15] JIN Fang-yan, WANG Xiu-li. Implicit Causality Extraction of Financial Events Integrating RACNN and BiLSTM [J]. Computer Science, 2022, 49(7): 179-186.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!