Computer Science ›› 2022, Vol. 49 ›› Issue (2): 62-68.doi: 10.11896/jsjkx.210900059

• Computer Vision: Theory and Application • Previous Articles     Next Articles

Human Skeleton Action Recognition Algorithm Based on Dynamic Topological Graph

XIE Yu1, YANG Rui-ling1, LIU Gong-xu2, LI De-yu1, WANG Wen-jian1   

  1. 1 School of Computer and Information Technology,Shanxi University,Taiyuan 030006,China
    2 School of Electronic Engineering,Xidian University,Xi'an 710071,China
  • Received:2021-09-07 Revised:2021-09-22 Online:2022-02-15 Published:2022-02-23
  • About author:XIE Yu,born in 1993,Ph.D,lecturer,is a member of China Computer Federation.His main research interests include graph neural networks and so on.
    WANG Wen-jian,born in 1968,Ph.D,professor,Ph.D supervisor,is a distinguished member of China Computer Federation.Her main research interests include machine learning and neural networks.
  • Supported by:
    National Natural Science Foundation of China(62076154,62106131,62106134),Program of Central Funds Guiding the Local Science and Technology Development(YDZX20201400001224) and Key R & D program of Shanxi Province(International Cooperation)(201903D421050).

Abstract: Traditional human skeleton action recognition algorithms manually construct topological graphs to model the action sequence contained in multiple video frames and learn each video frame to reflect the data changes,which may lead to the high computational cost,low network generalization performance and catastrophic forgetting.To solve these problems,a human skeleton action recognition algorithm based on dynamic topological graph is proposed,in which the human skeleton topological graph is dynamically constructed based on continuous learning.Specifically,human skeleton sequence data with multi-relationship characte-ristics are recoded into relationship triplets,and feature embedding is learned in a decoupling manner via the long short-term me-mory network.When handling new skeleton relationship triplets,we dynamically construct the human skeleton topological graph by a partial update mechanism,and then send it to the skeleton action recognition algorithm based on spatio-temporal graph convolution network for action recognition.Experimental results demonstrate that the proposed algorithm achieves 40%,85% and 90% recognition accuracy on three benchmark datasets,namely Kinetics-Skeleton,NTU-RGB+D(X-Sub) and NTU-RGB+D(X-View),respectively,which improve the accuracy of human skeleton action recognition.

Key words: Catastrophic forgetting, Continual learning, Graph convolution network, Human action recognition, Human skeleton data

CLC Number: 

  • TP391
[1]CHEN Y,TIAN Y,HE M.Monocular Human Pose Estimation:A Survey of Deep Learning-Based Methods[J].Computer Vision and Image Understanding,2020,192:102897.
[2]SONG S,LAN C,XING J,et al.An End-to-End Spatio-Temporal Attention Model for Human Action Recognition from Skeleton Data[C]//AAAI Conference on Artificial Intelligence.2017:4263-4270.
[3]YAN S,XIONG Y,LIN D.Spatial Temporal Graph Convolu-tional Networks for Skeleton-Based Action Recognition[C]//AAAI Conference on Artificial Intelligence.2018:7444-7452.
[4]XIONG X,MIN W,ZHENG W S,et al.S3D-CNN:Skeleton-Based 3D Consecutive-Low-Pooling Neural Network for Fall Detection[J].Applied Intelligence,2020,50(10):3521-3534.
[5]SHI L,ZHANG Y,CHENG J,et al.Two-Stream AdaptiveGraph Convolutional Networks for Skeleton-Based Action Re-cognition[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.2019:12026-12035.
[6]ZHANG P,LAN C,ZENG W,et al.Semantics-Guided NeuralNetworks for Efficient Skeleton-Based Human Action Recognition[C]//IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:1112-1121.
[7]DING C Y,LIU K,LI G,et al.Spatio-Temporal Weighted Posture Motion Features for Human Skeleton Action Recognition Research[J].Chinese Journal of Computers,2020,43(1):29-40.
[8]TIAN Z Q,DENG C H,ZHANG J W.Human Behavior Recognition Algorithm Based on Skeletal Temporal Divergence Feature[J].Journal of Computer Applications,2021,41(5):1450-1457.
[9]SHI L,ZHANG Y,CHENG J,et al.Skeleton-based Action Re-cognition with Directed Graph Neural Networks[C]//IEEE Conference on Computer Vision and Pattern Recognition.2019:7912-7921.
[10]TANG Y,TIAN Y,LU J,et al.Deep Progressive Reinforcement Learning for Skeleton-Based Action Recognition[C]//IEEE Conference on Computer Vision and Pattern Recognition.2018:5323-5332.
[11]THAKKAR K,NARAYANAN P J.Part-Based Graph Convolutional Network for Action Recognition[J].arXiv:1809.04983,2018.
[12]LI M,CHEN S,CHEN X,et al.Symbiotic Graph Neural Networks for 3D Skeleton-based Human Action Recognition and Motion Prediction[J/OL].IEEE Transactions on Pattern Ana-lysis and Machine Intelligence.
[13]LI B,LI X,ZHANG Z,et al.Spatio-temporal Graph Routing for Skeleton-based Action Recognition[C]//AAAI Conference on Artificial Intelligence.2019:8561-8568.
[14]HADSELL R,RAO D,RUSU A A,et al.Embracing Change:Continual Learning in Deep Neural Networks[J].Trends in Cognitive Sciences,2020,24(12):1028-1040.
[15]CHEN P H,WEI W,HSIEH C,et al.Overcoming Catastrophic Forgetting by Generative Regularization[J].arXiv:1912.01238,2019.
[16]D'AUTUME C D M,RUDER S,KONG L,et al.EpisodicMemory in Lifelong Language Learning[J].Advances in Neural Information Processing Systems,2019,32:13143-13152.
[17]ROLNICK D, AHUJA A, SCHWARZ J,et al.Experience Replay for Continual Learning[J].Advances in Neural Information Processing Systems,2019,32:350-360.
[18]LIU L,PU H Y.Real-time LSTM-based Multi-dimensional Features Gesture Recognition[J].Computer Science,2021,48(8):328-333.
[19]KOU X,LIN Y,LIU S,et al.Disentangle-based ContinualGraph Representation Learning[C]//Conference on Empirical Methods in Natural Language Processing.2020:2961-2972.
[20]PENG W,HONG X,CHEN H,et al.Learning Graph Convolutional Network for Skeleton-based Human Action Recognition by Neural Searching[C]//AAAI Conference on Artificial Intelligence.2020:2669-2676.
[21]PLIZZARI C,CANNICI M,MATTEUCCI M.Skeleton-basedAction Recognition Via Spatial and Temporal Transformer Networks[J].Computer Vision and Image Understanding,2021,208:103219.
[1] LIU Dong-mei, XU Yang, WU Ze-bin, LIU Qian, SONG Bin, WEI Zhi-hui. Incremental Object Detection Method Based on Border Distance Measurement [J]. Computer Science, 2022, 49(8): 136-142.
[2] LI Jian-zhi, WANG Hong-ling, WANG Zhong-qing. Automatic Generation of Patent Summarization Based on Graph Convolution Network [J]. Computer Science, 2022, 49(6A): 172-177.
[3] CHENG Si-wei, GE Wei-yi, WANG Yu, XU Jian. BGCN:Trigger Detection Based on BERT and Graph Convolution Network [J]. Computer Science, 2021, 48(7): 292-298.
[4] WANG Wen-bo, LUO Heng-li. Complete Graph Face Clustering Based on Graph Convolution Network [J]. Computer Science, 2021, 48(11A): 275-277.
[5] HONG Yao-qiu. Visual Human Action Recognition Based on Deep Belief Network [J]. Computer Science, 2021, 48(11A): 400-403.
[6] CAI Qiang, DENG Yi-biao, LI Hai-sheng, YU Le, MING Shao-feng. Survey on Human Action Recognition Based on Deep Learning [J]. Computer Science, 2020, 47(4): 85-93.
[7] HUANG Hai-xin, WANG Rui-peng, LIU Xiao-yang. Review of Human Action Recognition Technology Based on 3D Convolution [J]. Computer Science, 2020, 47(11A): 139-144.
[8] JI Chong, WANG Sheng and LU Jian-feng. Human Action Recognition Based on Fisher Discrimination Dictionary Learning [J]. Computer Science, 2017, 44(7): 270-274.
[9] LEI Qing,CHEN Duan-sheng and LI Shao-zi. Advances on Human Action Recognition in Realistic Scenes [J]. Computer Science, 2014, 41(12): 1-7.
Full text



No Suggested Reading articles found!