Abstract: Considering the importance of different multi-source class knowledge and the independence of various body movements,this paper proposes a zero-shot skeleton-based action recognition method based on category knowledge fusion and latent space optimization.The method integrates multi-source category knowledge through a sparse attention mechanism with importance sampling,and introduces a total correlation loss to reduce redundant correlations among independent variables in the latent space.Specifically,it firstly employs pre-trained models to extract skeleton features and multi-source textual features,then learns text fusion representations through an importance sampling-based feature fusion approach.Subsequently,a total correlation-constrained generative cross-modal alignment method establishes semantic associations between skeleton and textual features.Fina-lly,latent space features generated from unseen class representations are utilized to train classifiers for recognizing unseen actions.Experiments on NTU RGB+D,NTU RGB+D 120,and PKU-MMD datasets demonstrate that the proposed method significantly outperforms all existing mainstream approaches.

Key words: Zero-shot learning, Skeleton action recognition, Cross-model alignment, Attention, Text feature fusion