计算机科学 ›› 2024, Vol. 51 ›› Issue (4): 236-242.doi: 10.11896/jsjkx.221200120

• 计算机图形学&多媒体 • 上一篇    下一篇

基于3D骨架相似性的自适应移位图卷积神经网络人体行为识别算法

闫文杰, 尹艺颖   

  1. 河北工业大学人工智能与数据科学学院 天津300401
  • 收稿日期:2022-12-20 修回日期:2023-03-30 出版日期:2024-04-15 发布日期:2024-04-10
  • 通讯作者: 闫文杰(wenjieyanhit@163.com)
  • 基金资助:
    国家自然科学基金(61702157)

Human Action Recognition Algorithm Based on Adaptive Shifted Graph Convolutional Neural
Network with 3D Skeleton Similarity

YAN Wenjie, YIN Yiying   

  1. School of Artificial Intelligence,Hebei University of Technology,Tianjin 300401,China
  • Received:2022-12-20 Revised:2023-03-30 Online:2024-04-15 Published:2024-04-10
  • Supported by:
    National Natural Science Foundation of China(61702157).

摘要: 图卷积神经网络(Graph Convolutional Neural network,GCN)在基于3D骨架的人体行为识别领域取得了良好效果。然而,现有的大多数GCN方法对行为动作图的构建都是基于人体物理结构的手动设置,训练阶段各个图节点只能根据手动设置建立联系,无法感知动作行为过程中骨骼节点之间产生的新联系,导致图拓扑结构不合理和不灵活。移位图卷积网络通过改变图网络结构使得感受野更加灵活,并且在全局移位角度取得了良好效果。因此,提出了一种基于自适应移位图卷积神经网络(Adaptive Shift Graph Convolutional Neural network,AS-GCN)的人体行为识别算法来弥补前述GCN方法的不足。AS-GCN借鉴了移位图卷积网络的思想,提出用每个人体动作的本身特点来指导图神经网络进行移位操作,以尽可能准确地选定需要扩大感受野的节点。在基于骨架的通用动作识别数据集NTU-RGBD上,所提算法在骨骼有无物理关系约束的前提条件下均进行了实验验证。与现有的先进算法相比,AS-GCN算法的动作识别准确率在有骨骼物理约束的条件下的CV和CS角度上平均提高了12%和4.84%;在无骨骼物理约束的条件下的CV和CS角度上平均提高了20%和14.49%。

关键词: 骨架动作分类, 图卷积神经网络, 行为识别, 自适应移位

Abstract: Graph convolutional neural network(GCN) has achieved good results in the field of human action recognition based on 3D skeleton.However,in most of the existing GCN methods,the construction of the behavior diagram is based on the manual setting of the physical structure of the human body.In the training stage,each graph node can only establish the connection accor-ding to the manual setting,which cannot perceive new connections between bone nodes during action,resulting in the unreasonable and inflexible topology of the graph.The shifted graph convolutional neural network(Shift-GCN) makes the receptive field more flexible by changing its structure,and achieves satisfied results in the global shift angle.In order to tackle the above pro-blems of graph structure,an adaptive shift graph convolutional neural network(AS-GCN) is proposed to make up for the above shortcomings.AS-GCN draws on the idea of shifted graph convolutional neural network,and proposes to use the characteristics of each human action to guide the graph network to perform shift operation,so as to select the nodes that need to expand the receptive field as accurately as possible.On the general skeleton-based action recognition dataset NTU-RGBD,the AS-GCN is verified by extensive experiments under the premise of whether the skeleton has physical relationship constraints or not.Compared with the existing advanced algorithms,the accuracy of action recognition of AS-GCN is improved by 12% and 4.84% respectively in CV and CS angles on average with skeleton physical constraints.While under the condition of no skeleton physical constraint,the average improvement is 20% and 14.49% in CV and CS angles,respectively.

Key words: Skeleton-based action classification, Graph convolutional neural network, Action recognition, Adaptive shift

中图分类号: 

  • TP391
[1]KONG W,LIU Y,LI H,et al.A survey of action recognitionmethods based on graph convolutional network [J].Control and Decision,2021,36(7):1537-1546.
[2]LIANG X,LI W X,ZHANG H N.Review of research on human action recognition methods [J].Application Research of Computers,2022,39(3):651-660.
[3]ZHAO X H,YE S,LI X.Multi-algorithm Fusion Behavior Classification Method for Body Bone Information Reconstruction [J].Computer Science,2022,49(6):269-275.
[4]MIAO G Q,XIN W T,LIU R Y,et al.Graph ConvolutionalSkeleton-based Action Recognition Method for Intelligent Behavior Analysis [J].Computer Science,2022,49(2):156-161.
[5]ZHANG P,XUE J,LAN C,et al.EleAtt-RNN:Adding atten-tiveness to neurons in recurrent neural networks[J].IEEE Transactions on Image Processing,2019,29:1061-1073.
[6]LI M,SUN Q.3D Skeletal Human Action Recognition Using a CNN Fusion Model [J].Mathematical Problems in Engineering,2021(18):6650632.1-6650632.11.
[7]LI Y Z,YUAN J Z,LIU H Z.Human skeleton-based action re-cognition algorithm based on spatiotemporal attention graph convolutional network model [J].Journal of Computer Applications,2021,44(7):1915-1921.
[8]YAN S,XIONG Y,LIN D.Spatial temporal graph convolutional networks for skeleton-based action recognition[C]//Thirty-se-cond AAAI Conference on Artificial Intelligence.Palo Alto,Cali-fornia USA:AAAI Press,2018:7444-7452.
[9]CHENG K,ZHANG Y,HE X,et al.Skeleton-based action re-cognition with shift graph convolutional network[C]//Procee-dings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.San Francisco,CA,USA:IEEE,2020:183-192.
[10]SHI L,ZHANG Y,CHENG J,et al.Two-stream adaptive graph convolutional networks forskeleton-based action recognition[C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.San Francisco,CA,USA:IEEE,2019:12026-12035.
[11]SHI L,ZHANG Y,CHENG J,et al.Skeleton-based action re-cognition with multi-stream adaptive graph convolutional networks [J].IEEE Transactions on Image Processing,2020,29:9532-9545.
[12]SHAHROUDY A,LIU J,NG T T,et al.Ntu rgb+ d:A large scale dataset for 3d human activity analysis[C]//Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition.San Francisco,CA,USA:IEEE,2016:1010-1019.
[13]LI B,LI X,ZHANG Z,et al.Spatio-temporal graph routing for skeleton-based action recognition[C]//Proceedings of the AAAI Conference on Artificial Intelligence.Palo Alto,California USA:AAAI Press,2019,33(01):8561-8568.
[14]ZHANG P,LAN C,ZENG W,et al.Semantics-guided neuralnetworks for efficient skeleton-based human action recognition[C]//proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.San Francisco,CA,USA:IEEE,2020:1112-1121.
[15]PENG W,HONG X,CHEN H,et al.Learning graph convolutional network for skeleton-based human action recognition by neural searching[C]//Proceedings of the AAAI Conference on Artificial Intelligence.Palo Alto,California USA:AAAI Press,2020,34(3):2669-2676.
[16]CHENG K,ZHANG Y,CAO C,et al.Decoupling gcn withdropgraph module for skeleton-based actionrecognition[C]//European Conference on Computer Vision.Cham:Springer,2020:536-553.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!