基于路径分层建模的长距离知识图谱嵌入方法

doi:10.11896/jsjkx.250800116

摘要/Abstract

摘要： 知识图谱嵌入在自然语言处理和推荐系统等领域应用广泛。但传统方法处理多跳关系关联实体时,难以精准建模多跳关系路径累积语义,易导致语义衰减并忽略全局结构特征,难体现深层关联;且长距离路径中冗余实体既增加计算负担,又掩盖关键实体作用,干扰语义捕捉的精准性,致使多步推理任务表现不佳。针对现有方法,如KnowFormer的过度压缩问题和HittER 的浅层网络限制,提出一种同时处理长距离语义传递和高效邻居聚合的统一框架。与现有方法相比,所提方法创新性地将分层路径建模与多头注意力机制深度融合,实现了O(｜V｜²)的线性复杂度优化。针对长距离关系路径建模不足的问题,提出多头自注意力机制增强的分层关系路径建模(MHA-HRPM)方法。核心创新包括:1)分层关系路径建模,通过分层构建与逐层交互,使路径表示同时保留自身特征与上层增强特征,实现长距离路径语义的逐层传递与精准捕捉;2)引入多头自注意力机制,从多视角动态调整邻居实体的权重,自适应选择关键邻居,隐式捕获高阶交互特征,丰富实体表示的语义多样性。实验结果表明,该方法在长距离关系路径依赖建模上表现出色。

关键词: 知识图谱嵌入, 长距离关系, 分层关系路径, 注意力, 语义捕捉

Abstract: Knowledge graph embedding is widely applied in fields such as natural language processing and recommendation systems.However,when traditional methods process entities associated through multi-hop relations,they struggle to accurately mo-del the cumulative semantics of multi-hop relational paths,which easily leads to semantic decay and neglects global structural features,making it difficult to capture deep associations.Moreover,redundant entities in long-distance paths not only increase computational burden but also obscure the roles of key entities,interfering with the accuracy of semantic capture and resulting in poor performance in multi-step reasoning tasks.Addressing the over-compression issue of existing methods such as KnowFormer and the shallow network limitation of HittER,this paper proposes a unified framework that simultaneously solves long-distance semantic propagation and efficient neighbor aggregation.Compared with existing methods,this paper innovatively integrates hiera-rchical path modeling with multi-head attention mechanisms in depth,achievs linear complexity optimization from O(｜V｜²).Addressing the insufficient modeling of long-distance relational paths,this paper proposes a MHA-HRPM(Multi-Head Self-Attention enhanced Hierarchical Relational Path Modeling) method.The core innovations include:Firstly,hierarchical relational path modeling,which through hierarchical construction and layer-by-layer interaction,enables path representations to simultaneously preserve their own features and enhanced features from upper layers,achieving layer-wise propagation and precise capture of long-distance path semantics;Secondly,the introduction of a multi-head self-attention mechanism,which dynamically adjusts neighbor entity weights from multiple perspectives,adaptively selects key neighbors,implicitly captures high-order interaction features,and enriches the semantic diversity of entity representations.Experimental results demonstrate that the proposed method exhibits excellent performance in modeling long-distance relational path dependencies.

Key words: Knowledge graph embedding, Long distance relations, Hierarchical relation paths, Attention, Semantic capture

中图分类号:

TP391

谢新兴, 刘琰, 贾萌萌, 王凌. 基于路径分层建模的长距离知识图谱嵌入方法[J]. 计算机科学, 2026, 53(10): 301-311. https://doi.org/10.11896/jsjkx.250800116

XIE Xinxing, LIU Yan, JIA Mengmeng, WANG Ling. Long-distance Knowledge Graph Embedding Method Based on Hierarchical Path Modeling[J]. Computer Science, 2026, 53(10): 301-311. https://doi.org/10.11896/jsjkx.250800116

参考文献

[1] CHEN X J,JIA S B,XIANG Y.A review:Knowledge reasoning over knowledge graph [J].Expert Systems with Applications,2020,141:112948.
[2] BORDES A,WESTON J,USUNIER N.Open question answe-ring with weakly supervised embedding models [C]//Procee-dings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases.Berlin:Springer,2014:165-180.
[3] LIU Z Y,SUN M S,LIN Y K,et al.Knowledge representation learning:Review [J].Journal of Computer Research and Deve-lopment,2016,53(2):247-261.
[4] WU Z,JAIN D,WRIGHT K,et al.Representing long-rangecontext for graph neural networks with global attention [C]//Proceedings of Advances in Neural Information Processing Systems.Cambridge:MIT Press,2022:13182-13195.
[5] CHEN D,MANNING C D,KUSNER M J.Expanding thescope:Inductive knowledge graph reasoning with multi-starting progressive propagation [J].arXiv:2407.10430,2024.
[6] SUN Q.Critical Structure-aware Graph Neural Networks [C]//Proceedings of the ACM Turing Award Celebration Conference 2024(ACM-TURC’24).ACM,2024.
[7] BORDES A,USUNIER N,GARCIA A,et al.Translating embeddings for modeling multi-relational data [J].Advances in Neural Information Processing Systems,2013,26:2787-2795.
[8] SCHLICHTKRULL M,KIPF T N,BLOEM P,et al.Modeling relational data with graph convolutional networks [C]//Proceedings of the European Semantic Web Conference.Springer,2018:593-607.
[9] SHANG C,TANG Y,HUANG J,et al.End-to-end structure-aware convolutional networks for knowledge base completion [C]//Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence.AAAI,2019:3060-3067.
[10] LIU J,CUI Y,LI C,et al.KnowFormer:Revisiting transformers for knowledge graph reasoning [J].arXiv:2409.12865,2024.
[11] CHEN S X,LIU X D,GAO J F,et al.HittER:Hierarchical transformers for knowledge graph embeddings [C]//Procee-dings of the 2021 Conference on Empirical Methods in Natural Language Processing.Stroudsburg:ACL,2021:10395-10407.
[12] YANG H,CHEN H,PAN S,et al.Neighbor-augmented know-ledge graph attention network for recommendation [J].Neural Processing Letters,2023,55(4):4407-4429.
[13] CHEN H,LI H,HUANG G,et al.DT4KGR:Decision trans-former for fast and effective multi-hop reasoning over know-ledge graphs [J].Information Processing & Management,2024,61(2):103606.
[14] WANG Q,LIU J,LUO Y,et al.Dynamic semantic graph construction and reasoning for explainable multi-hop science question answering [C]//Proceedings of the AAAI Conference on Artificial Intelligence.Palo Alto:AAAI,2021:13943-13951.
[15] LI Z,JIN X,LI W,et al.Hierarchical graph attention network for temporal knowledge graph reasoning [J].Neurocomputing,2023,550:126484.
[16] LIN Y K,LIU Z Y,LUAN H B,et al.Modeling relation paths for representation learning of knowledge bases [C]//Procee-dings of the 2015 Conference on Empirical Methods in Natural Language Processing.Stroudsburg:ACL,2015:705-714.
[17] DEVLIN J,CHANG M W,LEE K,et al.BERT:Pre-training of deep bidirectional transformers for language understanding [C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics.Stroudsburg:ACL,2019:4171-4186.
[18] LIN X V,SOCHER R,XIONG C M.Multi-hop knowledgegraph reasoning with reward shaping [C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.ACL,2018:3243-3253.
[19] QIU L,XIAO Y,QU Y,et al.Dynamically fused graph network for multi-hop reasoning [C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.Stroudsburg:ACL,2020:6140-6150.
[20] SHEIKH N,KHADANGI E,BAGHSHAH M S,et al.Know-ledge graph embedding using graph convolutional networks with relation-aware attention [J].arXiv:2102.07200,2021.
[21] BONDARENKO Y,NAGEL M,BLANKEVOORT T.Quantizable transformers:Removing outliers by helping attention heads do nothing [C]//Proceedings of Advances in Neural Information Processing Systems.2023:75067-75096.
[22] LIN Z H,TIAN C X,HOU Y P,et al.Improving graph collaborative filtering with neighborhood-enriched contrastive learning [J].arXiv:2202.06200,2022.
[23] ZHANG Y,YAO Q,SHAO Y,et al.Collaborative knowledge-aware recommendation based on neighborhood negative sampling [J].Expert Systems with Applications,2023,216:119421.
[24] ZHANG D,YIN J,ZHU X,et al.Graph attention network with dynamic representation of relations for knowledge graph completion [J].Expert Systems with Applications,2023,213:119020.
[25] LIU X,MA Y,ZHOU L,et al.Semantic relation-aware graphattention network with noise augmented layer-wise contrastive learning for recommendation [J].Knowledge-Based Systems,2025,306:112643.
[26] YANG L,CHEN Z,CUI J,et al.Knowledge reasoning based on graph neural networks with multi-layer top-p message passing and sparse negative sampling [J].Knowledge-Based Systems,2025,310:112518.
[27] MAO A,MOHRI M,ZHONG Y.Cross-Entropy Loss Func-tions:Theoretical Analysis and Applications [C]//Proceedings of the 40th International Conference on Machine Learning.ICML,2023.
[28] SHAO P,ZHANG D,YANG G,et al.Knowledge reasoningwith multiple relational paths [J].Connection Science,2022,34(1):2865-2896.
[29] YANG B S,YIH W T,HE X D,et al.Embedding entities and relations for learning and inference in knowledge bases [C]//Proceedings of the 3rd International Conference on Learning Representations.San Diego:ICLR,2015.
[30] DETTMERS T,MINERVINI P,STENETORP P,et al.Convolutional 2D knowledge graph embeddings [C]//Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence.AAAI,2018:1811-1818.
[31] JIANG X T,WANG Q,WANG B.Adaptive convolution formulti-relational learning [C]//Proceedings of the 2019 Confe-rence of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Stroudsburg:ACL,2019:978-987.
[32] ZHANG Y,SHENG M,ZHOU R,et al.A neighborhood-aware graph self-attention mechanism-based pre-training model for knowledge graph reasoning [J].Information Sciences,2023,649:119614.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed