计算机科学

计算机科学 ›› 2025, Vol. 52 ›› Issue (1): 94-101.doi: 10.11896/jsjkx.240600170

• 大语言模型技术研究及应用 • 上一篇    下一篇

大语言模型驱动的多元关系知识图谱补全方法

刘畅成, 桑磊, 李炜, 张以文   

  1. 安徽大学计算机科学与技术学院 合肥 230601
  • 收稿日期:2027-06-28 修回日期:2024-09-03 出版日期:2025-01-15 发布日期:2025-01-09
  • 通讯作者: 李炜(le998@sohu.com)
  • 作者简介:(charlesliu@stu.ahu.edu.cn)

Large Language Model Driven Multi-relational Knowledge Graph Completion Method

LIU Changcheng, SANG Lei, LI Wei, ZHANG Yiwen   

  1. College of Computer Science and Technology,Anhui University,Hefei 230601,China
  • Received:2027-06-28 Revised:2024-09-03 Online:2025-01-15 Published:2025-01-09
  • About author:LIU Changcheng,born in 1999,master,is a member of CCF(No.U6154G).His main research interests include large language models and data mining.
    LI Wei,born in 1969,Ph.D,professor.Her main research interests include software engineering,virtual reality human-computer interaction,and data mining.

PDF (PC)

245

摘要： 知识图谱通过将复杂的互联网信息转化为易于理解的结构化形式,极大地提高了信息的可访问性。知识图谱补全技术进一步增强了知识图谱的信息完整性,显著提升了智能问答和推荐系统等通用领域应用的性能与用户体验。然而,现有的知识图谱补全方法大多专注于关系类型较少和简单语义情景下的三元组实例,未能充分利用知识图谱在处理多元关系和复杂语义方面的潜力。针对此问题,提出了一种由大语言模型(LLM)驱动的多元关系知识图谱补全方法。将 LLM 的深层语言理解能力与知识图谱的结构特性相结合,有效捕捉多元关系,理解复杂语义情景。此外,还引入了一种基于思维链的提示工程策略,旨在提高补全任务的准确性。该方法在两个公开知识图谱数据集上的实验结果都取得了显著的提升。

关键词: 知识图谱, 大语言模型, 知识图谱补全, 多元关系, 候选集构建, 思维链提示

Abstract: Knowledge graphs transform complex Internet information into an easily understandable structured format,significantly enhancing the accessibility of information.Knowledge graph completion(KGC) techniques further enhance the completeness of knowledge graphs,markedly improved the performance and user experience of general domain applications such as intelligent question answering and recommendation systems by enhancing the information completeness of knowledge graphs.However,most existing KGC methods focus on triple instances in scenarios with fewer types of relationships and simpler semantics,failing to fully leverage the potential of graphs in handling multi-relational and complex semantics.In response to this issue,we propose a method for multi-relational knowledge graph completion driven by large language model(LLM).By combining the deep linguistic understanding capabilities of LLM with the structural characteristics of knowledge graphs,this method effectively captures complex semantic scenarios and comprehends multi-relational relationships.Additionally,we introduce a chain-of-thought based prompting engineering strategy,aiming to enhancing the accuracy of the completion tasks.Experimental results on two public knowledge graph datasets have demonstrated the significant performance improvements of this method.

Key words: Knowledge graph, Large language model, Knowledge graph completion, Multi-relational, Candidate set construction, Chain-of-thought prompt

中图分类号: 

  • TP311
[1]CHEN X,JIA S,XIANG Y.A review:Knowledge reasoningover knowledge graph [J].Expert Systems with Applications,2020,141:112948.
[2]JI S,PAN S,CAMBRIA E,et al.A survey on knowledgegraphs:Representation,acquisition,and applications[J].IEEE Transactions on Neural Networks and Learning Systems,2022,33(2):494-514.
[3]ANNERVAZ K M,CHOWDHURY S B R,DUKKIPATI A.Learning beyond datasets:Knowledge graph augmented neural networks fornatural language processing[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Volume 1(Long Papers).New Orleans,Louisiana:Association for Computational Linguistics,2018:313-322.
[4]WANG W J,YU Y F.Automatic completion algorithm for mis-sing links in nowledge graph considering data sparsity[J].Journal of Jilin University(Engineering and Technology Edition),2022,52(6):1428-1433.
[5]CHEN Z,WANG Y,ZHAO B,et al.Knowledge graph completion:A review[J].IEEE Access,2020,8:192435-192456.
[6]BORDES A,USUNIER N,GARCIA-DURAN A,et al.Translating Embeddings for Modeling Multi-relational Data [C]//Proceedings of Advances in Neural Information Processing Systems.Red Hook,NY:Curran Associates,Inc.,2013.
[7]WANG Z,ZHANG J,FENG J,et al.Knowledge graph embedding by translating on hyperplanes[C]//AAAI’14:Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence.Québec City,Québec,Canada:AAAI Press,2014:1112-1119.
[8]WANG L,MA C,FENG X,et al.A survey on large languagemodel based autonomous agents [J].Frontiers of Computer Science,2024,18(6):186345.
[9]TOUVRON H,MARTIN L,STONE K,et al.Llama 2:Openfoundation and finetuned chatmodels[J].arXiv:2307.09288,2023.
[10]BAI J,BAI S,CHU Y,et al.Qwen technical report[J].arXiv:2309.16609,2023.
[11]ZHANG Y,CHEN Z,ZHANG W,et al.Making large language models perform better in knowledge graph completion[J].ar-Xiv:2310.06671,2023.
[12]LI Q,WANG G,LIU J,et al.Explainability for Large Language Models:A Survey [J].ACM Transactions on Intelligent Systems and Technology,2024,15(2):20.
[13]DENG S,WANGC,LI Z,et al.Construction and applications of billion-scale pre-trained multimodal business knowledge graph[C]//2023 IEEE 39th International Conference on Data Engineering(ICDE).Anaheim:IEEE Press,2023:2988-3002.
[14]HOYT C T,BERRENDORF M,GALKIN M,et al.A unified framework for rankbased evaluation metrics for link prediction in knowledge graphs[J].arXiv:2203.07544,2022.
[15]LIN Y,LIU Z,SUN M,et al.Learning entity and relation embeddings for knowledge graph completion[C]//Proceedings of the AAAI Conference on Artificial Intelligence.Washington:AAAI Press,2015.
[16]JI G,HE S,XU L,et al.Knowledge graph e-mbedding via dynamic mapping matrix [C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing(Volume 1:Long Papers).Beijing,China:Association for Computational Linguistics,2015:687-696.
[17]NICKEL M,TRESP V,KRIEGEL H P.A three-way model for collective learning on multi-relational data[C]//ICML’11:Proceedings of the 28th International Conference on International Conference on Machine Learning.Madison,WI,USA:Omnipress,2011:809-816.
[18]YANG B,TAU YIH W,HE X,et al.Embedding entities and relations for learning and inference in knowledge bases[J].arXiv:1412.6575,2015.
[19]TROUILLON T,WELBL J,RIEDEL S,et al.Complex embed-dings for simple link prediction [C]//Proceedings of The 33rd International Conference on Machine Learning.New York,USA:PMLR,2016:2071-2080.
[20]DETTMERS T,MINERVINI P,STENETOR-P P,et al.Convolutional 2d knowledge graph embeddings[C]//Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence and Thirtieth Innovative Applications of Artificial Intelligence Conference and Eighth AAAI Symposium on Educational Advances in Artificial Intelligence.New Orleans,LA:AAAI Press,2018.
[21]CAO Z,XU Q,YANG Z,et al.Geometry int-eraction knowledge graph embeddings[C]//Proceedings of the AAAI Conference on Artificial Intelligence,2022,36(5):5521-5529.
[22]ZHANG Y,ZHANG W.Cause:Towards causal knowledgegraph embedding[C]//Knowledge Graph and Semantic Computing:Knowledge Graph Empowers Artificial General Intelligence.Singapore:Springer Nature Singapore,2023:17- 28.
[23]SCARSELLI F,GORI M,TSOI A C,et al.The graph neural network model[J].IEEE Transactions on Neural Networks,2009,20(1):61-80.
[24]SCHLICHTKRULL M,KIPF T N,BLOEM P,et al.Modeling relational data with graph convolutional networks[C]//Proceedings of the 15th International Conference on The Semantic Web:ESWC 2018.Berlin,Heidelberg:Springer-Verlag,2018:593-607.
[25]VASHISHTH S,SANYAL S,NITIN V,et al.Composition-based multi-relational graph convolutional networks[J].arXiv:1911.03082,2020.
[26]DAI G,WANG X,ZOU X,et al.Mrgat:Multi-relational graph attention network for knowledge graph completion[J].Neural Networks,2022,154:234-245.
[27]LI R,CAO Y,ZHU Q,et al.How does knowledge graph embedding extrapolate to unseen data:A semantic evidence view[J].Proceedings of the AAAI Conference on Artificial Intelligence,2022,36(5):5781-5791.
[28]ZHANG X,ZHANG C,GUO J,et al.Graph attention network with dynamic representationof relations for knowledge graph completion [J].Expert Systems with Applications,2023,219:119616.
[29]LU X,WANG L,JIANG Z,et al.MMKRL:A robust embedding approach for multi-modal knowledge graph representation lear-ning [J].Applied Intelligence,2022,52(7):7480-7497.
[30]XIE R,LIU Z,JIA J,et al.Representation learning of knowledge graphs with entity descriptions[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2016.
[31]YANG B X,LUO X D,SUN K L.Recent Progress on Machine Translation Based on Pre-trained Language Models[J].Compu-ter Science,2024,51(S1):230700112-8.
[32]GENG M,WANG S,DONG D,et al.Large Language Models are Few-Shot Summarizers:Multi-Intent Comment Generation via In-Context Learning [C]//Proceedings of the IEEE/ACM 46th International Conference on Software Engineering.New York,NY,USA:Association for Computing Machinery,2024.
[33]ZHANG Z,HAN X,LIU Z,et al.Ernie:Enhanced language representation with informativeentities[J].arXiv:1905.07129,2019.
[34]DEVLIN J,CHANG M W,LEE K,et al.Bert:Pre-training of deep bidirectional transformersfor language understanding[J].arXiv:1810.04805,2019.
[35]LIU W,ZHOU P,ZHAO Z,et al.K-BERT:Enabling language representation with knowledge graph[J].Proceedings of the AAAI Conference on Artificial Intelligence.2020,34(3):2901-2908.
[36]YAO L,MAO C,LUO Y.Kg-bert:Bert for knowledge graphcompletion[J].arXiv:1909.03193,2019.
[37]FEI H,REN Y,ZHANG Y,et al.Enriching contextualized language model from knowledge graph for biomedical information extraction [J].Briefings in Bioinformatics,2020,22(3):bbaa110.
[38]WEI Y,HUANG Q,ZHANG Y,et al.Kicgpt:Large language model with knowledge in context for knowledge graph completion[C]//Findings of the Association for Computational Linguistics:EMNLP 2023.Singapore:Association for Computa-tional Linguistics,2023:8667-8683.
[39]GUAN L,LIN Y,LIN H,et al.Mitigating Large LanguageModel Hallucinations via Autonomous Knowledge Graph-Based Retrofitting [C]//Proceedings of the AAAI Conference on Artificial Intelligence.2024:18126-18134.
[40]YANG L,CHEN H,LI Z,et al.Give us the Facts:Enhancing Large Language Models With Knowledge Graphs for Fact-Aware Language Modeling [J].IEEE Transactions on Know-ledge and Data Engineering,2024,36(7):3091-3110.
[41]PAN S,LUO L,WANG Y,et al.Unifying large language mo-dels and knowledge graphs:A roadmap[J].arXiv:2306.08302,2024.
[42]WEI J,WANG X,SCHUURMANS D,et al.Chain-of-thoughtprompting elicits reasoning in large language models[C]//Advances in Neural Information Processing Systems.Curran Associates,Inc.,2022:24824-24837.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发