Computer Science ›› 2022, Vol. 49 ›› Issue (7): 204-211.doi: 10.11896/jsjkx.210400129

• Artificial Intelligence • Previous Articles     Next Articles

Adaptive Attention-based Knowledge Graph Completion

WANG Jie, LI Xiao-nan, LI Guan-yu   

  1. Information Science & Technology College,Dalian Maritime University,Dalian,Liaoning 116026,China
  • Received:2021-04-14 Revised:2021-10-23 Online:2022-07-15 Published:2022-07-12
  • About author:WANG Jie,born in 1997,postgraduate,is a member of China Computer Federation.Her main research interests include intelligent information processing and knowledge graph completion.
    LI Guan-yu,born in 1963,Ph.D,professor,is a member of China Computer Federation.His main research interests include intelligent information proces-sing and knowledge graph.
  • Supported by:
    National Natural Science Foundation of China(61976032,62002039).

Abstract: Existing knowledge graph completion models learn a single static feature representation for entities and relationships by integrating multi-source information.But they can't represent the subtle meaning and dynamic attributes of entities and relationships that appear in different contexts.That is,entities and relationships will show different attributes,because they have different roles and meanings when they are involved in different triples.To solve above problems,an adaptive attention network for knowledge graph completion is proposed,which uses adaptive attention to model the contribution of each task-specified feature dimension,and generates dynamic and variable embedding representations for target entities and relationships.Specifically,the proposed model defines the neighbor encoder and the path aggregator to process two structures in the entity neighborhood subgraph,adaptively learn the attention weights to capture the most logically related features of the task,and to give the entities and relationships with fine-grained semantics in line with the current task.Experimental results in link prediction task show that,the MeanRank of the proposed model on FB15K-237 dataset is 6.9% lower than PathCon,and Hits@1 is 2.3% higher than PathCon.For the sparse datasets NELL-995 and DDB14,its Hits@1 reaches 87.9% and 98% respectively.Therefore,it proves that the introduction of adaptive attention mechanism can effectively extract the dynamic attributes of entities and relationships to generate a more comprehensive embedding representation,and improves the accuracy of knowledge graph completion.

Key words: Adaptive attention, Knowledge graph completion, Knowledge representation, Neighborhood subgraph

CLC Number: 

  • TP182
[1]BOLLACKER K,EVANS C,PARITOSH P,et al.Freebase:ACollaboratively Created Graph Database for Structuring Human Knowledge[C]//Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data.2008:1247-1250.
[2]MILLER G A.WordNet:A Lexical Database for English[J].Communications of the ACM,1995,38(11):39-41.
[3]XIANG R,WU Z,HE W,et al.Cotype:Joint Extraction ofTyped Entities and Relations with Knowledge Bases[C]//Proceedings of the 26th International Conference on World Wide Web.Perth:ACM,2017:1015-1024.
[4]HUANG X,ZHANG J,LI D,et al.Knowledge Graph Embedding based Question Answering [C]//Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining.Melbourne:ACM,2019:105-113.
[5]WANG H,ZHAO M,XIE X,et al.Knowledge Graph Convolutional Networks for Recommender Systems[C]//Proceedings of the World Wide Web Conference.San Francisco:ACM,2019:3307-3313.
[6]DING J H,JIA W J.Summary of knowledge graph completion algorithms[J].Information and Communication Technology,2018,12(1):56-62.
[7]ARORA S.A survey on graph neural networks for knowledgegraph completion[J].arXiv:2007.12374,2020.
[8]OH B,SEO S,LEE K H.Knowledge graph completion by context-aware convolutional learning with multi-hop neighborhoods[C]//Proceedings of the 27th ACM International Conference on Information and Knowledge Management.2018:257-266.
[9]WANG C,SHA Y.Neighborhood Aggregation Embedding Modelfor Link Prediction in Knowledge Graphs[C]//International Conference on Web Engineering.Cham:Springer,2020:188-203.
[10]BORDES A,USUNIER N,et al.Translating Embeddings forModeling Multi-relational Data[C]//Proceedings of the NIPS.Cambridge:MIT Press,2013:2787-2795.
[11]WANG Z,ZHANG J, FENG J, et al.Knowledge Graph Embedding by Translating on Hyperplanes[C]//Proceedings of the AAAI.Menlo Park,CA:AAAI, 2014:1112-1119.
[12]LIN Y,LIU Z,SUN M,et al.Learning Entity and Relation Embeddings For Knowledge Graph Completion[C]//Proceedings of the AAAI.Menlo Park,CA:AAAI,2014:2181-2187.
[13]NGUYEN D Q,SIRTS K,QU L,et al.STransE:A Novel Embedding Model of Entities and Relationships in Knowledge Bases [C]//Proceedings of the North American Chapter of the Asso-ciation for Computational Linguistics.San Diego California:The Association for Computational Linguistics,2016:460-466.
[14]SUN Z,DENG Z,NIE J,et al.Rotate:Knowledge Graph Embedding by Relational Rotation in Complex Space[J].arXiv:1902.10197,2019.
[15]YANG B,YIH W,HE X,et al.Embedding Entities and Relations for Learning and Inference in Knowledge Bases[J].arXiv:1412.6575,2014.
[16]TROUILLON T,WELBL J,RIEDEL S,et al.Complex Embeddings for Simple Link Prediction[C]//Proceedings of the 33nd International Conference on Machine Learning.New York City:JMLR.org,2016:2071-2080.
[17]KAZEMI S,POOLE D.Simple Embedding for Link Prediction in Knowledge Graphs[C]//Advances in Neural Information Processing Systems.Canada,2018:4284-4295.
[18]ZHANG S,YI T,YAO L,et al.Quaternion Knowledge Graph Embeddings[C]//Advances in Neural Information Processing Systems.Vancouver,2019:2731-2741.
[19]DETTMERS T,MINERVINI P,STENETORP P,et al.Convolutional 2D Knowledge Graph Embeddings[C]//Proceedings of the AAAI.New Orleans:AAAIPress,2018:1811-1818.
[20]DAI Q N,TU D N,NGUYEN D Q,et al.A Novel Embedding Model for Knowledge Base Completion based on Convolutional Neural Network[C]//Proceedings of the NAACL.New Or-leans:Association for Computational Linguistics,2018:327-333.
[21]VU T,NGUYEN T D,NGUYEN D Q,et al.A Capsule Network-Based Embedding Model for Knowledge Graph Completion and Search Personalization[C]//Proceedings of the NAACL-HLT.Minneapolis:Association for ComputationalLinguistics,2019:2180-2189.
[22]SADEGHIAN A,ARMANDPOUR M,DING P,et al.DRUM:End-To-End Differentiable Rule Mining On Knowledge Graphs [C]//Advances in Neural Information Processing Systems.Vancouver,2019:15321-15331.
[23]LIN Y,LIU Z,LUAN H,et al.Modeling Relation Paths for Representation Learning of Knowledge Bases[C]//Proceedings of the EMNLP.Lisbon,The Association for Computational Linguistics,2015:705-714.
[24]NIU G,LI YANG,TANG C,et al.Relational Learning with Gated and Attentive Neighbor Aggregator for Few-Shot Know-ledge Graph Completion[J].arXiv:2104.13095,2021.
[25]WANG K,LIU Y,XU X,et al.Enhancing knowledge graph embedding by composite neighbors for link prediction[J].Computing,2020,102(12):2587-2606.
[26]LIU X,TAN H,CHEN Q,et al.RAGAT:Relation AwareGraph Attention Network for Knowledge Graph Completion[J].IEEE Access,2021,9:20840-20849.
[27]CAI L,YAN B,MAI G,et al.TransGCN:Coupling Transformation Assumptions with Graph Convolutional Networks for Link Prediction[C]//Proceedings of the 10th International Confe-rence on Knowledge Capture.Marina Del Rey:ACM,2019:131-138.
[28]SCHLICHTKRULL M,KIPF T N,BLOEM P,et al.Modeling Relational Data with Graph Convolutional Networks[C]//15th International Conference on Extended Semantic Web Confe-rence.ESWC,2018:593-607.
[29]XIONG W,YU M,CHANG S,et al.One-Shot Relational Lear-ning for Knowledge Graphs[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.Brussels:Association for Computational Linguistics,2018:1980-1990.
[30]WANG H,REN H,LESKOVEC J.Entity Context and Relational Paths for Knowledge Graph Completion[J].arXiv:2002.06757,2020.
[31]ZHANG C,YAO H,HUANG C,et al.Few-Shot KnowledgeGraph Completion [C]//Proceedings of the AAAI Conference on Artificial Intelligence.New York:AAAI Press,2020:3041-3048.
[32]ZHAO X J,JIA Y,LI A P,et al.Research on Link Prediction Model Based on Hierarchical Attention Mechanism[J].Journal of Communications,2021,42(3):36-44.
[33]XIE Z,ZHOU G,LIU J,et al.ReInceptionE:Relation-aware Inception Network with Joint Local-Global Structural Information for Knowledge Graph Embedding[C]//Proceedings of the 58th ACL.Online:Association for Computational Linguistics,2020:5929-5939.
[34]XIONG W,HOANG T,WANG W Y.DeepPath:A Reinforcement Learning Method for Knowledge Graph Reasoning[C]//Proceedings of the EMNLP.Copenhagen:Association for Computational Linguistics,2017:564-573.
[35]ZHANG Z,ZHUANG F,ZHU H,et al.Relational Graph Neural Network with Hierarchical Attention for Knowledge Graph Completion[J].Proceedings of the AAAI Conference on Artificial Intelligence,2020,34(5):9612-9619.
[36]SUN Z,VASHISHTH S,SANYAL S,et al.A Re-evaluation of Knowledge Graph Completion Methods[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.Online:Association for Computational Linguistics,2020:5516-5522.
[1] CHEN Heng, WANG Wei-mei, LI Guan-yu, SHI Yi-ming. Knowledge Graph Completion Model Using Quaternion as Relational Rotation [J]. Computer Science, 2021, 48(5): 225-231.
[2] HANG Ting-ting, FENG Jun, LU Jia-min. Knowledge Graph Construction Techniques:Taxonomy,Survey and Future Directions [J]. Computer Science, 2021, 48(2): 175-189.
[3] ZHENG Hong-liang, HOU Xue-hui, SONG Xiao-ying, PANG Kuo, ZOU Li. Approach for Knowledge Reasoning Based on Hesitate Fuzzy Credibility [J]. Computer Science, 2019, 46(1): 131-137.
[4] CHEN Hui and MA Ya-ping. Uncertain Extension of Description Logic SROIQ(D) for Uncertain Knowledge Representation [J]. Computer Science, 2016, 43(Z11): 88-92.
[5] MENG Hua, YUAN Ya-yan, CHU Jie-lei and WANG Hong-jun. Topological Characterization of AGM Belief Contraction Operator [J]. Computer Science, 2016, 43(9): 87-90.
[6] ZHOU Ru-qi, CHEN Yi-qun and FENG Jia-li. Knowledge Reasoning Method Based on Extended Fuzzy Petri Net [J]. Computer Science, 2016, 43(3): 75-79.
[7] LU Fang-fang and WANG Jie. Improved Method to Process Inconsistent Answer Set Program [J]. Computer Science, 2015, 42(Z6): 542-545.
[8] CHEN Min, ZHAO Shu-liang, GUO Xiao-bo, LI Xiao-chao and LIU Meng-meng. New Vis-Meta Graph Knowledge Representation for Association Rules [J]. Computer Science, 2015, 42(3): 218-223.
[9] ZHOU Ru-qi and FENG Jia-li. Attribute Granular Reasoning Based on Attribute Petri Net [J]. Computer Science, 2014, 41(8): 101-105.
[10] LIU Bin,YAO Li,HAO Zhi-yong and GONG Yong. Research on Argument Ontology for Computational Argumentation [J]. Computer Science, 2014, 41(2): 226-231.
[11] CHANG Liang,CHEN Li-min. Action Theory Based on the Dynamic Description Logic DDL [J]. Computer Science, 2011, 38(7): 203-208.
[12] LI Zhan-shan,HAN Wen-cheng,GUO Ting. Retrospect and Prospect of Decomposition Technology [J]. Computer Science, 2011, 38(10): 29-33.
[13] YANG Jin-song LING Pei-liang. Design and Implement on a Backward Reasoning Algorithm Based on Fuzzy Petri Net [J]. Computer Science, 2009, 36(12): 158-160.
[14] LIU Zong-tian,HUANG Mei-li,ZHOU Wen,ZHONG Zhao-man,FU Jian-feng,SHAH Jian-fang,ZHI Hui-lai. Research on Event-oriented Ontology Model [J]. Computer Science, 2009, 36(11): 189-192.
[15] . [J]. Computer Science, 2008, 35(8): 258-261.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!