基于自适应注意力机制的知识图谱补全算法

doi:10.11896/jsjkx.210400129

摘要/Abstract

摘要： 现有的知识图谱补全模型通常将多源信息整合为实体和关系学习单一的静态特征表示,但无法表征不同上下文中出现的实体和关系的细差含义和动态属性,即实体和关系在涉及不同的三元组时可能有着不同的角色和含义,并因此表现出不同的属性。为此,提出了一种自适应注意力网络用于知识图谱补全,引入自适应注意力建模每个特征维度对特定任务的贡献程度,为目标实体和关系生成动态可变的嵌入表示。具体而言,所提模型通过定义邻居编码器和路径聚合器来处理实体邻域子图中的两种结构,自适应地调整邻居实体和关系路径的注意力得分,以捕获逻辑上与任务最相关的属性特征,为实体和关系赋予符合当前任务的细粒度语义。在链接预测任务中的实验结果表明,所提模型在FB15K-237数据集中的MeanRank指标比PathCon降低了6.9%,Hits@1比PathCon提高了2.3%;在稀疏数据集NELL-995和DDB14上,其Hits@1分别达到了87.9%和98%,证明了引入自适应注意力机制能够有效提取实体和关系的动态属性,为二者生成更全面的表示形式,从而提高知识图谱补全精度。

关键词: 邻域子图, 知识表示, 知识图谱补全, 自适应注意力

Abstract: Existing knowledge graph completion models learn a single static feature representation for entities and relationships by integrating multi-source information.But they can't represent the subtle meaning and dynamic attributes of entities and relationships that appear in different contexts.That is,entities and relationships will show different attributes,because they have different roles and meanings when they are involved in different triples.To solve above problems,an adaptive attention network for knowledge graph completion is proposed,which uses adaptive attention to model the contribution of each task-specified feature dimension,and generates dynamic and variable embedding representations for target entities and relationships.Specifically,the proposed model defines the neighbor encoder and the path aggregator to process two structures in the entity neighborhood subgraph,adaptively learn the attention weights to capture the most logically related features of the task,and to give the entities and relationships with fine-grained semantics in line with the current task.Experimental results in link prediction task show that,the MeanRank of the proposed model on FB15K-237 dataset is 6.9% lower than PathCon,and Hits@1 is 2.3% higher than PathCon.For the sparse datasets NELL-995 and DDB14,its Hits@1 reaches 87.9% and 98% respectively.Therefore,it proves that the introduction of adaptive attention mechanism can effectively extract the dynamic attributes of entities and relationships to generate a more comprehensive embedding representation,and improves the accuracy of knowledge graph completion.

Key words: Adaptive attention, Knowledge graph completion, Knowledge representation, Neighborhood subgraph

中图分类号:

TP182

王杰, 李晓楠, 李冠宇. 基于自适应注意力机制的知识图谱补全算法[J]. 计算机科学, 2022, 49(7): 204-211. https://doi.org/10.11896/jsjkx.210400129

WANG Jie, LI Xiao-nan, LI Guan-yu. Adaptive Attention-based Knowledge Graph Completion[J]. Computer Science, 2022, 49(7): 204-211. https://doi.org/10.11896/jsjkx.210400129

参考文献

[1]BOLLACKER K,EVANS C,PARITOSH P,et al.Freebase:ACollaboratively Created Graph Database for Structuring Human Knowledge[C]//Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data.2008:1247-1250.
[2]MILLER G A.WordNet:A Lexical Database for English[J].Communications of the ACM,1995,38(11):39-41.
[3]XIANG R,WU Z,HE W,et al.Cotype:Joint Extraction ofTyped Entities and Relations with Knowledge Bases[C]//Proceedings of the 26th International Conference on World Wide Web.Perth:ACM,2017:1015-1024.
[4]HUANG X,ZHANG J,LI D,et al.Knowledge Graph Embedding based Question Answering [C]//Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining.Melbourne:ACM,2019:105-113.
[5]WANG H,ZHAO M,XIE X,et al.Knowledge Graph Convolutional Networks for Recommender Systems[C]//Proceedings of the World Wide Web Conference.San Francisco:ACM,2019:3307-3313.
[6]DING J H,JIA W J.Summary of knowledge graph completion algorithms[J].Information and Communication Technology,2018,12(1):56-62.
[7]ARORA S.A survey on graph neural networks for knowledgegraph completion[J].arXiv:2007.12374,2020.
[8]OH B,SEO S,LEE K H.Knowledge graph completion by context-aware convolutional learning with multi-hop neighborhoods[C]//Proceedings of the 27th ACM International Conference on Information and Knowledge Management.2018:257-266.
[9]WANG C,SHA Y.Neighborhood Aggregation Embedding Modelfor Link Prediction in Knowledge Graphs[C]//International Conference on Web Engineering.Cham:Springer,2020:188-203.
[10]BORDES A,USUNIER N,et al.Translating Embeddings forModeling Multi-relational Data[C]//Proceedings of the NIPS.Cambridge:MIT Press,2013:2787-2795.
[11]WANG Z,ZHANG J, FENG J, et al.Knowledge Graph Embedding by Translating on Hyperplanes[C]//Proceedings of the AAAI.Menlo Park,CA:AAAI, 2014:1112-1119.
[12]LIN Y,LIU Z,SUN M,et al.Learning Entity and Relation Embeddings For Knowledge Graph Completion[C]//Proceedings of the AAAI.Menlo Park,CA:AAAI,2014:2181-2187.
[13]NGUYEN D Q,SIRTS K,QU L,et al.STransE:A Novel Embedding Model of Entities and Relationships in Knowledge Bases [C]//Proceedings of the North American Chapter of the Asso-ciation for Computational Linguistics.San Diego California:The Association for Computational Linguistics,2016:460-466.
[14]SUN Z,DENG Z,NIE J,et al.Rotate:Knowledge Graph Embedding by Relational Rotation in Complex Space[J].arXiv:1902.10197,2019.
[15]YANG B,YIH W,HE X,et al.Embedding Entities and Relations for Learning and Inference in Knowledge Bases[J].arXiv:1412.6575,2014.
[16]TROUILLON T,WELBL J,RIEDEL S,et al.Complex Embeddings for Simple Link Prediction[C]//Proceedings of the 33nd International Conference on Machine Learning.New York City:JMLR.org,2016:2071-2080.
[17]KAZEMI S,POOLE D.Simple Embedding for Link Prediction in Knowledge Graphs[C]//Advances in Neural Information Processing Systems.Canada,2018:4284-4295.
[18]ZHANG S,YI T,YAO L,et al.Quaternion Knowledge Graph Embeddings[C]//Advances in Neural Information Processing Systems.Vancouver,2019:2731-2741.
[19]DETTMERS T,MINERVINI P,STENETORP P,et al.Convolutional 2D Knowledge Graph Embeddings[C]//Proceedings of the AAAI.New Orleans:AAAIPress,2018:1811-1818.
[20]DAI Q N,TU D N,NGUYEN D Q,et al.A Novel Embedding Model for Knowledge Base Completion based on Convolutional Neural Network[C]//Proceedings of the NAACL.New Or-leans:Association for Computational Linguistics,2018:327-333.
[21]VU T,NGUYEN T D,NGUYEN D Q,et al.A Capsule Network-Based Embedding Model for Knowledge Graph Completion and Search Personalization[C]//Proceedings of the NAACL-HLT.Minneapolis:Association for ComputationalLinguistics,2019:2180-2189.
[22]SADEGHIAN A,ARMANDPOUR M,DING P,et al.DRUM:End-To-End Differentiable Rule Mining On Knowledge Graphs [C]//Advances in Neural Information Processing Systems.Vancouver,2019:15321-15331.
[23]LIN Y,LIU Z,LUAN H,et al.Modeling Relation Paths for Representation Learning of Knowledge Bases[C]//Proceedings of the EMNLP.Lisbon,The Association for Computational Linguistics,2015:705-714.
[24]NIU G,LI YANG,TANG C,et al.Relational Learning with Gated and Attentive Neighbor Aggregator for Few-Shot Know-ledge Graph Completion[J].arXiv:2104.13095,2021.
[25]WANG K,LIU Y,XU X,et al.Enhancing knowledge graph embedding by composite neighbors for link prediction[J].Computing,2020,102(12):2587-2606.
[26]LIU X,TAN H,CHEN Q,et al.RAGAT:Relation AwareGraph Attention Network for Knowledge Graph Completion[J].IEEE Access,2021,9:20840-20849.
[27]CAI L,YAN B,MAI G,et al.TransGCN:Coupling Transformation Assumptions with Graph Convolutional Networks for Link Prediction[C]//Proceedings of the 10th International Confe-rence on Knowledge Capture.Marina Del Rey:ACM,2019:131-138.
[28]SCHLICHTKRULL M,KIPF T N,BLOEM P,et al.Modeling Relational Data with Graph Convolutional Networks[C]//15th International Conference on Extended Semantic Web Confe-rence.ESWC,2018:593-607.
[29]XIONG W,YU M,CHANG S,et al.One-Shot Relational Lear-ning for Knowledge Graphs[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.Brussels:Association for Computational Linguistics,2018:1980-1990.
[30]WANG H,REN H,LESKOVEC J.Entity Context and Relational Paths for Knowledge Graph Completion[J].arXiv:2002.06757,2020.
[31]ZHANG C,YAO H,HUANG C,et al.Few-Shot KnowledgeGraph Completion [C]//Proceedings of the AAAI Conference on Artificial Intelligence.New York:AAAI Press,2020:3041-3048.
[32]ZHAO X J,JIA Y,LI A P,et al.Research on Link Prediction Model Based on Hierarchical Attention Mechanism[J].Journal of Communications,2021,42(3):36-44.
[33]XIE Z,ZHOU G,LIU J,et al.ReInceptionE:Relation-aware Inception Network with Joint Local-Global Structural Information for Knowledge Graph Embedding[C]//Proceedings of the 58th ACL.Online:Association for Computational Linguistics,2020:5929-5939.
[34]XIONG W,HOANG T,WANG W Y.DeepPath:A Reinforcement Learning Method for Knowledge Graph Reasoning[C]//Proceedings of the EMNLP.Copenhagen:Association for Computational Linguistics,2017:564-573.
[35]ZHANG Z,ZHUANG F,ZHU H,et al.Relational Graph Neural Network with Hierarchical Attention for Knowledge Graph Completion[J].Proceedings of the AAAI Conference on Artificial Intelligence,2020,34(5):9612-9619.
[36]SUN Z,VASHISHTH S,SANYAL S,et al.A Re-evaluation of Knowledge Graph Completion Methods[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.Online:Association for Computational Linguistics,2020:5516-5522.

相关文章 15

[1]	陈恒, 王维美, 李冠宇, 史一民. 四元数关系旋转的知识图谱补全模型 Knowledge Graph Completion Model Using Quaternion as Relational Rotation 计算机科学, 2021, 48(5): 225-231. https://doi.org/10.11896/jsjkx.200300093
[2]	杭婷婷, 冯钧, 陆佳民. 知识图谱构建技术:分类、调查和未来方向 Knowledge Graph Construction Techniques:Taxonomy,Survey and Future Directions 计算机科学, 2021, 48(2): 175-189. https://doi.org/10.11896/jsjkx.200700010
[3]	郑宏亮, 侯雪辉, 宋笑迎, 庞阔, 邹丽. 基于犹豫模糊可信度的知识推理 Approach for Knowledge Reasoning Based on Hesitate Fuzzy Credibility 计算机科学, 2019, 46(1): 131-137. https://doi.org/10.11896／j.issn.1002-137X.2019.01.020
[4]	陈晖,马亚平. 面向不确定性知识表示的描述逻辑SROIQ(D)不确定性扩展 Uncertain Extension of Description Logic SROIQ(D) for Uncertain Knowledge Representation 计算机科学, 2016, 43(Z11): 88-92. https://doi.org/10.11896/j.issn.1002-137X.2016.11A.019
[5]	孟华,原雅燕,储节磊,王红军. AGM信念收缩算子的拓扑式刻画 Topological Characterization of AGM Belief Contraction Operator 计算机科学, 2016, 43(9): 87-90. https://doi.org/10.11896/j.issn.1002-137X.2016.09.016
[6]	周如旗,陈忆群,冯嘉礼. 基于扩展模糊Petri网的知识推理方法研究 Knowledge Reasoning Method Based on Extended Fuzzy Petri Net 计算机科学, 2016, 43(3): 75-79. https://doi.org/10.11896/j.issn.1002-137X.2016.03.015
[7]	陈敏,赵书良,郭晓波,李晓超,柳萌萌. 一种新的关联规则Vis-Meta图知识表示方法 New Vis-Meta Graph Knowledge Representation for Association Rules 计算机科学, 2015, 42(3): 218-223. https://doi.org/10.11896/j.issn.1002-137X.2015.03.045
[8]	周如旗,冯嘉礼. 基于属性Petri网的属性粒推理研究 Attribute Granular Reasoning Based on Attribute Petri Net 计算机科学, 2014, 41(8): 101-105. https://doi.org/10.11896/j.issn.1002-137X.2014.08.022
[9]	刘斌,姚莉,郝智勇,龚勇. 用于计算辩论的论据本体研究 Research on Argument Ontology for Computational Argumentation 计算机科学, 2014, 41(2): 226-231.
[10]	郭晓波,赵书良,刘军丹,赵娇娇,王长宾. 基于概念图的关联规则知识表示 Knowledge Presentation of Association Rules Based on Conceptual Graphs 计算机科学, 2013, 40(8): 261-265.
[11]	郭亚飞,侯俊杰,石胜友. 面向典型宇航产品的工艺知识库研究 Research of Process Knowledgebase Oriented to Typical Aerospace Product 计算机科学, 2013, 40(5): 189-192.
[12]	常亮,陈立民. 基于动态描述逻辑DDL的动作理论 Action Theory Based on the Dynamic Description Logic DDL 计算机科学, 2011, 38(7): 203-208.
[13]	李占山,韩文成,郭霆. 约束分解技术回顾与展望 Retrospect and Prospect of Decomposition Technology 计算机科学, 2011, 38(10): 29-33.
[14]	杨劲松,凌培亮. 一种模糊Petri网的逆向知识推理方法设计实现 Design and Implement on a Backward Reasoning Algorithm Based on Fuzzy Petri Net 计算机科学, 2009, 36(12): 158-160.
[15]	刘宗田,黄美丽,周文,仲兆满,付剑锋,单建芳,智慧来. 面向事件的本体研究 Research on Event-oriented Ontology Model 计算机科学, 2009, 36(11): 189-192.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed