Computer Science ›› 2024, Vol. 51 ›› Issue (4): 236-242.doi: 10.11896/jsjkx.221200120

• Computer Graphics & Multimedia • Previous Articles     Next Articles

Human Action Recognition Algorithm Based on Adaptive Shifted Graph Convolutional Neural
Network with 3D Skeleton Similarity

YAN Wenjie, YIN Yiying   

  1. School of Artificial Intelligence,Hebei University of Technology,Tianjin 300401,China
  • Received:2022-12-20 Revised:2023-03-30 Online:2024-04-15 Published:2024-04-10
  • Supported by:
    National Natural Science Foundation of China(61702157).

Abstract: Graph convolutional neural network(GCN) has achieved good results in the field of human action recognition based on 3D skeleton.However,in most of the existing GCN methods,the construction of the behavior diagram is based on the manual setting of the physical structure of the human body.In the training stage,each graph node can only establish the connection accor-ding to the manual setting,which cannot perceive new connections between bone nodes during action,resulting in the unreasonable and inflexible topology of the graph.The shifted graph convolutional neural network(Shift-GCN) makes the receptive field more flexible by changing its structure,and achieves satisfied results in the global shift angle.In order to tackle the above pro-blems of graph structure,an adaptive shift graph convolutional neural network(AS-GCN) is proposed to make up for the above shortcomings.AS-GCN draws on the idea of shifted graph convolutional neural network,and proposes to use the characteristics of each human action to guide the graph network to perform shift operation,so as to select the nodes that need to expand the receptive field as accurately as possible.On the general skeleton-based action recognition dataset NTU-RGBD,the AS-GCN is verified by extensive experiments under the premise of whether the skeleton has physical relationship constraints or not.Compared with the existing advanced algorithms,the accuracy of action recognition of AS-GCN is improved by 12% and 4.84% respectively in CV and CS angles on average with skeleton physical constraints.While under the condition of no skeleton physical constraint,the average improvement is 20% and 14.49% in CV and CS angles,respectively.

Key words: Skeleton-based action classification, Graph convolutional neural network, Action recognition, Adaptive shift

CLC Number: 

  • TP391
[1]KONG W,LIU Y,LI H,et al.A survey of action recognitionmethods based on graph convolutional network [J].Control and Decision,2021,36(7):1537-1546.
[2]LIANG X,LI W X,ZHANG H N.Review of research on human action recognition methods [J].Application Research of Computers,2022,39(3):651-660.
[3]ZHAO X H,YE S,LI X.Multi-algorithm Fusion Behavior Classification Method for Body Bone Information Reconstruction [J].Computer Science,2022,49(6):269-275.
[4]MIAO G Q,XIN W T,LIU R Y,et al.Graph ConvolutionalSkeleton-based Action Recognition Method for Intelligent Behavior Analysis [J].Computer Science,2022,49(2):156-161.
[5]ZHANG P,XUE J,LAN C,et al.EleAtt-RNN:Adding atten-tiveness to neurons in recurrent neural networks[J].IEEE Transactions on Image Processing,2019,29:1061-1073.
[6]LI M,SUN Q.3D Skeletal Human Action Recognition Using a CNN Fusion Model [J].Mathematical Problems in Engineering,2021(18):6650632.1-6650632.11.
[7]LI Y Z,YUAN J Z,LIU H Z.Human skeleton-based action re-cognition algorithm based on spatiotemporal attention graph convolutional network model [J].Journal of Computer Applications,2021,44(7):1915-1921.
[8]YAN S,XIONG Y,LIN D.Spatial temporal graph convolutional networks for skeleton-based action recognition[C]//Thirty-se-cond AAAI Conference on Artificial Intelligence.Palo Alto,Cali-fornia USA:AAAI Press,2018:7444-7452.
[9]CHENG K,ZHANG Y,HE X,et al.Skeleton-based action re-cognition with shift graph convolutional network[C]//Procee-dings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.San Francisco,CA,USA:IEEE,2020:183-192.
[10]SHI L,ZHANG Y,CHENG J,et al.Two-stream adaptive graph convolutional networks forskeleton-based action recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.San Francisco,CA,USA:IEEE,2019:12026-12035.
[11]SHI L,ZHANG Y,CHENG J,et al.Skeleton-based action re-cognition with multi-stream adaptive graph convolutional networks [J].IEEE Transactions on Image Processing,2020,29:9532-9545.
[12]SHAHROUDY A,LIU J,NG T T,et al.Ntu rgb+ d:A large scale dataset for 3d human activity analysis[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.San Francisco,CA,USA:IEEE,2016:1010-1019.
[13]LI B,LI X,ZHANG Z,et al.Spatio-temporal graph routing for skeleton-based action recognition[C]//Proceedings of the AAAI Conference on Artificial Intelligence.Palo Alto,California USA:AAAI Press,2019,33(01):8561-8568.
[14]ZHANG P,LAN C,ZENG W,et al.Semantics-guided neuralnetworks for efficient skeleton-based human action recognition[C]//proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.San Francisco,CA,USA:IEEE,2020:1112-1121.
[15]PENG W,HONG X,CHEN H,et al.Learning graph convolutional network for skeleton-based human action recognition by neural searching[C]//Proceedings of the AAAI Conference on Artificial Intelligence.Palo Alto,California USA:AAAI Press,2020,34(3):2669-2676.
[16]CHENG K,ZHANG Y,CAO C,et al.Decoupling gcn withdropgraph module for skeleton-based actionrecognition[C]//European Conference on Computer Vision.Cham:Springer,2020:536-553.
[1] ZHANG Liying, SUN Haihang, SUN Yufa , SHI Bingbo. Review of Node Classification Methods Based on Graph Convolutional Neural Networks [J]. Computer Science, 2024, 51(4): 95-105.
[2] HUANG Yujiao, CHEN Mingkai, ZHENG Yuan, FAN Xinggang, XIAO Jie, LONG Haixia. Text Classification Based on Weakened Graph Convolutional Networks [J]. Computer Science, 2023, 50(6A): 220700039-5.
[3] ZHANG Xue, ZHAO Hui. Sentiment Analysis Based on Multi-event Semantic Enhancement [J]. Computer Science, 2023, 50(5): 238-247.
[4] SHAO Yunfei, SONG You, WANG Baohui. Study on Degree of Node Based Personalized Propagation of Neural Predictions forSocial Networks [J]. Computer Science, 2023, 50(4): 16-21.
[5] CHI Tang, CHE Chao. Entity Alignment Method Combining Iterative Relationship Graph Matching and Attribute Semantic Embedding [J]. Computer Science, 2023, 50(11A): 230200041-6.
[6] LUO Huilan, YU Yawei, WANG Chanjuan. Multi-dimensional Feature Excitation Network for Video Action Recognition [J]. Computer Science, 2023, 50(11A): 230300115-8.
[7] ZHANG Kaixuan, CAI Guoyong, ZHU Kunri. Image Aesthetics-enhanced Visual Perception Recommendation System [J]. Computer Science, 2023, 50(11A): 221100083-8.
[8] LI Hua, ZHAO Lingdi, CHEN Yujie, YANG Yang, DU Xinzhao. Lightweight Graph Convolution Action Recognition Algorithm Based on Multi-streamFusion [J]. Computer Science, 2023, 50(11A): 220800147-6.
[9] MA Handa, FANG Yuqing. Dynamic Negative Sampling for Graph Convolution Network Based Collaborative Filtering Recommendation Model [J]. Computer Science, 2023, 50(11A): 230200149-7.
[10] WU Yushan, XU Zengmin, ZHANG Xuelian, WANG Tao. Self-supervised Action Recognition Based on Skeleton Data Augmentation and Double Nearest Neighbor Retrieval [J]. Computer Science, 2023, 50(11): 97-106.
[11] LI Zi-yi, ZHOU Xia-bing, WANG Zhong-qing, ZHANG Min. Stance Detection Based on User Connection [J]. Computer Science, 2022, 49(5): 221-226.
[12] GAO Yue, FU Xiang-ling, OUYANG Tian-xiong, CHEN Song-ling, YAN Chen-wei. EEG Emotion Recognition Based on Spatiotemporal Self-Adaptive Graph ConvolutionalNeural Network [J]. Computer Science, 2022, 49(4): 30-36.
[13] LI Hao, ZHANG Lan, YANG Bing, YANG Hai-xiao, KOU Yong-qi, WANG Fei, KANG Yan. Fine-grained Sentiment Classification of Chinese Microblogs Combining Dual Weight Mechanismand Graph Convolutional Neural Network [J]. Computer Science, 2022, 49(3): 246-254.
[14] XIE Yu, YANG Rui-ling, LIU Gong-xu, LI De-yu, WANG Wen-jian. Human Skeleton Action Recognition Algorithm Based on Dynamic Topological Graph [J]. Computer Science, 2022, 49(2): 62-68.
[15] MIAO Qi-guang, XIN Wen-tian, LIU Ru-yi, XIE Kun, WANG Quan, YANG Zong-kai. Graph Convolutional Skeleton-based Action Recognition Method for Intelligent Behavior Analysis [J]. Computer Science, 2022, 49(2): 156-161.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!