计算机科学 ›› 2023, Vol. 50 ›› Issue (4): 149-158.doi: 10.11896/jsjkx.211200175
申秋慧1, 张宏军2, 徐有为1, 王航1, 程恺2
SHEN Qiuhui1, ZHANG Hongjun2, XU Youwei1, WANG Hang1, CHENG Kai2
摘要: 表达方式丰富直观的知识图谱得到了大量学者的关注。在知识图谱嵌入方面已积累了大量研究,其成果在电商、金融、医药、交通、智能问答等领域发挥了重要的作用。其中,损失函数在知识图谱嵌入模型的训练阶段起到了非常关键的作用。在现有知识图谱嵌入研究的基础上,根据基础损失函数把模型中使用的损失函数梳理为合页损失、逻辑回归损失、交叉熵损失、对数似然损失、负采样损失和均方误差损失六大类,并逐类详细分析了损失函数的原型公式、物理含义和其在知识图谱嵌入模型中的扩展、演变及应用。在此基础上,对静态和动态两大知识图谱场景中各种损失函数的使用情况、效率和收敛性进行了综合分析评价;根据分析结果,结合知识图谱的发展应用趋势和损失函数现状,对损失函数的未来研究方向进行了探讨。
中图分类号:
[1]BORDES A,USUNIER N,GARCIA-DURAN A,et al.Translating embeddings for modeling multi-relational data[C]//Neural Information Processing Systems(NIPS).2013:1-9. [2]WANG Z,ZHANG J,FENG J,et al.Knowledge graph embedding by translating on hyperplanes[C]//AAAI Conference on Artificial Intelligence.2014:1112-1119. [3]SUN Z,VASHISHTH S,SANYAL S,et al.A Re-evaluation of Knowledge Graph Completion Methods[EB/OL].(2020-07-08)[2021-12-13].https://arxiv.org/abs/1911.03903v3. [4]NICKEL M,TRESP V,KRIEGEL H P.A three-way model for collective learning on multi-relational data[C]//International Conference on International Conference on Machine Learning.2011:809-816. [5]HAYASHI K,SHIMBO M.On the Equivalence of Holographicand Complex Embeddings for Link Prediction[C]//Proceedings of the 55th Annual Meeting of the Association for Computa-tional Linguistics(ACL).2017:554-559. [6]NICKEL M,ROSASCO L,POGGIO T.Holographic embed-dings of knowledge graphs[C]//AAAI Conference on Artificial Intelligence.2016:1955-1961. [7]TROUILLON T,WELBL J,RIEDEL S,et al.Complex embeddings for simple link prediction[C]//International Conference on Machine Learning.PMLR,2016:2071-2080. [8]TROUILLON T,NICKEL M.Complex and Holographic Em-beddings of Knowledge Graphs:A Comparison[EB/OL].(2017-07-23)[2021-12-13].https://arxiv.org/abs/1707.01475. [9]BORDES A,GLOROT X,WESTON J,et al.Joint learning ofwords and meaning representations for open-text semantic parsing[C]//Artificial Intelligence and Statistics.PMLR,2012:127-135. [10]BORDES A,WESTON J,COLLOBERT R,et al.Learningstructured embeddings of knowledge bases[C]//AAAI Confe-rence on Artificial Intelligence.2011:301-306. [11]GARCÍA-DURÁN A,BORDES A,USUNIER N.Effectiveblending of two and three-way interactions for modeling multi-relational data[C]//Joint European Conference on Machine Learning and Knowledge Discovery in Databases.2014:434-449. [12]YANG B,YIH W,HE X,et al.Embedding entities and relations for learning and inference in knowledge bases[C]//International Conference on Learning Representations(ICLR).2015:1-13. [13]FENG J,HUANG M,WANG M,et al.Knowledge graph em-bedding by flexible translation[C]//International Conference on Principles of Knowledge Representation and Reasoning.2016:557-560. [14]LIN Y,LIU Z,SUN M,et al.Learning entity and relation embeddings for knowledge graph completion[C]//AAAI Confe-rence on Artificial Intelligence.2015:2181-2187. [15]XIE R,LIU Z,SUN M.Representation learning of knowledgegraphs with hierarchical types[C]//Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence.2016:2965-2971. [16]XIE R,LIU Z,JIA J,et al.Representation learning of knowledge graphs with entity descriptions[C]//AAAI Conference on Artificial Intelligence.2016:2659-2665. [17]XIE R,LIU Z,LUAN H,et al.Image-embodied knowledge representation learning[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence.2017:3140-3146. [18]DASGUPTA S,RAY S N,TALUKDAR P.Hyte:Hyperplanebased temporally aware knowledge graph embedding[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing(EMNLP).2018:2001-2011. [19]JI G,HE S,XU L,et al.Knowledge graph embedding via dy-namic mapping matrix[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing.2015:687-696. [20]JI G,LIU K,HE S,et al.Knowledge graph completion withadaptive sparse transfer matrix[C]//AAAI Conference on Artificial Intelligence.2016:985-991. [21]XIE Q,MA X,DAI Z,et al.An Interpretable Knowledge Transfer Model for Knowledge Base Completion[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.2017:950-962. [22]XIAO H,HUANG M,ZHU X.From one point to a manifold:knowledge graph embedding for precise link prediction[C]//Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence.2016:1315-1321. [23]HE S,LIU K,JI G,et al.Learning to represent knowledgegraphs with gaussian embedding[C]//Proceedings of the 24th ACM International on Conference on Information and Know-ledge Management.2015:623-632. [24]NGUYEN D Q,SIRTS K,QU L,et al.STransE:a novel embedding model of entities and relationships in knowledge bases[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2016:460-466. [25]LIN Y,LIU Z,LUAN H,et al.Modeling Relation Paths forRepresentation Learning of Knowledge Bases[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.2015:705-714. [26]FAN M,ZHOU Q,CHANG E,et al.Transition-based know-ledge graph embedding with relational mapping properties[C]//Proceedings of the 28th Pacific AsiaConference on Language,Information and Computing.2014:328-337. [27]EBISU T,ICHISE R.Toruse:Knowledge graph embedding on a lie group[C]//AAAI Conference on Artificial Intelligence.2018:1819-1826. [28]LEBLAYJ,CHEKOL M W.Deriving validity time in knowledge graph[C]//Proceedings of the 26th International World Wide Web Conference.2018:1771-1776. [29]XIAO H,HUANG M,HAO Y,et al.TransA:An adaptive approach for knowledge graph embedding[EB/OL].(2015-09-28)[2021-12-13].https://arxiv.org/abs/1509.05490. [30]QIAN W,FU C,ZHU Y,et al.Translating embeddings forknowledge graph completion with relation attention mechanism[C]//Proceedings of the 27th International Joint Conference on Artificial Intelligence.2018:4286-4292. [31]SOCHER R,CHEN D,MANNING C D,et al.Reasoning with neural tensor networks for knowledge base completion[C]//International Conference on Neural Information Processing Systems-Volume 1.2013:926-934. [32]CHAMI I,WOLF A,JUAN D C,et al.Low-Dimensional Hy-perbolic Knowledge Graph Embeddings[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:6901-6914. [33]VU T,NGUYEN T D,NGUYEN D Q,et al.A capsule network-based embedding model for knowledge graph completion and search personalization[C]//Proceedings of the 2019 Confe-rence of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Vo-lume 1(Long and Short Papers).2019:2180-2189. [34]LIU H,WU Y,YANG Y.Analogical inference for multi-rela-tional embeddings[C]//Proceedings of the 34th International Conference on Machine Learning-Volume 70.2017:2168-2178. [35]KAZEMI S M,POOLE D.SimplE embedding for link prediction in knowledge graphs[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems.2018:4289-4300. [36]ZHANG S,TAY Y,YAO L,et al.Quaternion knowledge graph embeddings[EB/OL].(2019-10-31)[2021-12-13].https://arxiv.org/abs/1904.10281. [37]NGUYEN T D,NGUYEN D Q,PHUNG D.A Novel Embed-ding Model for Knowledge Base Completion Based on Convolutional Neural Network[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Volume 2(Short Papers).2018:327-333. [38]XU C,LI R.Relation Embedding with Dihedral Group inKnowledge Graph[C]//Proceedings of the 57th Annual Mee-ting of the Association for Computational Linguistics.2019:263-272. [39]BALAZEVIC I,ALLEN C,HOSPEDALES T.TuckER:Tensor Factorization for Knowledge Graph Completion[C]//Procee-dings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing(EMNLP-IJCNLP).2019:5188-5197. [40]BALAZEVIC I,ALLEN C,HOSPEDALES T.Multi-relational poincaré graph embeddings[J].Advances in Neural Information Processing Systems,2019,32:4463-4473. [41]DETTMERS T,MINERVINI P,STENETORP P,et al.Convolutional 2d knowledge graph embeddings[C]//AAAI Confe-rence on Artificial Intelligence.2018:1811-1818. [42]SHANG C,TANG Y,HUANG J,et al.End-to-end structure-aware convolutional networks for knowledge base completion[C]//Conference on Artificial Intelligence.2019:3060-3067. [43]BALAEVIĆ I,ALLEN C,HOSPEDALES T M.Hypernetwork knowledge graph embeddings[C]//International Conference on Artificial Neural Networks.Cham:Springer,2019:553-565. [44]SCHLICHTKRULL M,KIPF T N,BLOEM P,et al.Modeling relational data with graph convolutional networks[C]//Euro-pean Semantic Web Conference.Cham:Springer,2018:593-607. [45]JIANG X,WANG Q,WANG B.Adaptive convolution for multi-relational learning[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Volume 1(Long and Short Papers).2019:978-987. [46]PEZESHKPOUR P,CHEN L,SINGH S.Embedding Multimodal Relational Data for Knowledge Base Completion[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.2018:3208-3218. [47]ZHU C,CHEN M,FAN C,et al.Learning from History:Mode-ling Temporal Knowledge Graphs with Sequential Copy-Generation Networks[EB/OL].(2021-03-05)[2021-12-13].https://arxiv.org/abs/2012.08492. [48]YAO L,MAO C,LUO Y.KG-BERT:BERT for knowledgegraph completion[EB/OL].(2019-09-11)[2021-12-13].https://arxiv.org/abs/1909.03193v2. [49]NICKEL M,KIELA D.Poincaré embeddings for learning hierarchical representations[C]//Proceedings of the 31st Interna-tional Conference on Neural Information Processing Systems.2017:6341-6350. [50]SHI B,WENINGER T.ProjE:Embedding projection for know-ledge graph completion[C]//AAAI Conference on Artificial Intelligence.2017:1236-1242. [51]XUE Y,YUAN Y,XU Z,et al.Expanding holographic embeddings for knowledge completion[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems.2018:4496-4506. [52]LIN Y,LIU Z,SUN M.Knowledge representation learning with entities,attributes and relations[C]//Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence.2016:2866-2872. [53]GARCIA-DURAN A,NIEPERT M.Kblrn:End-to-end learning of knowledge base representations with latent,relational,and numerical features[EB/OL].(2018-06-11)[2021-12-13].https://arxiv.org/abs/1709.04676. [54]QU M,TANG J.Probabilistic logic neural networks for reaso-ning[EB/OL].(2019-10-29)[2021-12-13].https://arxiv.org/abs/1906.08495. [55]XIAO H,HUANG M,ZHU X.TransG:A Generative Model for Knowledge Graph Embedding[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics(Volume 1:Long Papers).2016:2316-2325. [56]ZHANG W,PAUDEL B,ZHANG W,et al.Interaction embeddings for prediction and explanation in knowledge graphs[C]//Proceedings of the Twelfth ACM International Conference on Web Search and Data Mining.2019:96-104. [57]TRIVEDI R,FARAJTABAR M,WANG Y,et al.Know-Evolve:Deep Reasoning in Temporal Knowledge Graphs[EB/OL].(2017-06-21)[2021-12-13].https://arxiv.org/abs/1705.05742v1. [58]SADEGHIAN A,ARMANDPOUR M,COLAS A,et al.Chro-noR:Rotation Based Temporal Knowledge Graph Embedding[EB/OL].(2021-03-18)[2021-12-13].https://arxiv.org/abs/2103.10379. [59]JIN W,QU M,JIN X,et al.Recurrent Event Network:Auto regressive Structure Inference over Temporal Knowledge Graphs[EB/OL].(2020-10-06)[2021-12-13].https://arxiv.org/abs/1904.05530v4. [60]JENATTON R,LE ROUX N,BORDES A,et al.A latent factor model for highly multi-relational data[C]//Advances in Neural Information Processing Systems 25(NIPS 2012).2012:3176-3184. [61]LIU Q,JIANG H,EVDOKIMOV A,et al.Probabilistic reaso-ning via deep learning:Neural association models[EB/OL].(2016-08-03)[2021-12-13].https://arxiv.org/abs/1603.07704. [62]GUTMANN M U,HYVÄRINEN A.Noise-contrastive estimation of unnormalized statistical models,with applications to na-tural image statistics[J].The Journal of Machine Learning Research,2012(13):307-361. [63]ANDRIY M,YEE W T.A fast and simple algorithm for training neural probabilistic language models[EB/OL].(2012-06-27)[2021-12-13].https://arxiv.org/abs/1206.6426. [64]MIKOLOV T,SUTSKEVER I,CHEN K,et al.Distributed representations of words and phrases and their compositionality[C]//Conference and Workshop on Neural Information Proces-sing Systems(NIPS).2013:3111-3119. [65]GUO L,SUN Z,HU W.Learning to exploit long-term relational dependencies in knowledge graphs[C]//International Confe-rence on Machine Learning.PMLR,2019:2505-2514. [66]SUN Z,DENG Z H,NIE J Y,et al.RotatE:Knowledge Graph Embedding by Relational Rotation in Complex Space[C]//International Conference on Learning Representations.2018:1-18. [67]ZHANG Z,CAI J,ZHANG Y,et al.Learning hierarchy-aware knowledge graph embeddings for link prediction[C]//AAAI Conference on Artificial Intelligence.2020:3065-3072. [68]CHANG K W,YIH W,YANG B,et al.Typed tensor decomposition of knowledge bases for relation extraction[C]//Procee-dings of the 2014 Conference on Empirical Methods in Natural Language Processing(EMNLP).2014:1568-1579. [69]KAZEMI S M,GOEL R,JAIN K,et al.Representation Learning for Dynamic Graphs:A Survey[J].Journal of Machine Learning Research,2020(21):1-73. [70]LIU Z Y,SUN M S,LIN Y K,et al.Knowledge Representation Learning:A Review[J].Journal of Computer Research and Development,2016,53(2):247-261. [71]WANG Q,MAO Z,WANG B,et al.Knowledge graph embedding:A survey of approaches and applications[J].IEEE Tran-sactions on Knowledge and Data Engineering,2017,29(12):2724-2743. [72]PAULHEIM H.Knowledge graph refinement:A survey of approaches and evaluation methods[J].Semantic Web,2017,8(3):489-508. [73]CAI H,ZHENG V W,CHANG K C.A comprehensive survey of graph embedding:Problems,techniques,and applications[J].IEEE Transactions on Knowledge and Data Engineering,2018,30(9):1616-1637. [74]LIN Y,HAN X,XIE R,et al.Knowledge representation lear-ning:A quantitative review[EB/OL].(2018-12-28)[2021-12-13].https://arxiv.org/abs/1812.10901. [75]GESESE G A,BISWAS R,SACK H.A Comprehensive Survey of Knowledge Graph Embeddings with Literals:Techniques and Applications[C]//Proceedings of DL4KG2019-Workshop on Deep Learning for Knowledge Graphs.2019:131. [76]CHEN X,JIA S,XIANG Y.A review:Knowledge reasoningover knowledge graph[J].Expert Systems with Applications,2020,141(5):1-21. [77]TAY Y,TUAN L A,PHAN M C,et al.Multi-task neural network for non-discrete attribute prediction in knowledge graphs[C]//Proceedings of the 2017 ACM on Conference on Information and Knowledge Management.2017:1029-1038. [78]CHEN X,CHEN M,SHI W,et al.Embedding uncertain know-ledge graphs[C]//AAAI Conference on Artificial Intelligence.2019:3363-3370. [79]ALI M,BERRENDORF M,HOYT C T,et al.Bringing Light Into the Dark:A Large-scale Evaluation of Knowledge Graph Embedding Models Under a Unified Framework[EB/OL].(2021-11-01)[2021-12-13].https://arxiv.org/abs/2006.13365. |
[1] | 栗书敬, 黄增峰. 混合曲率空间用于多关系异构知识图谱链接补全 Mixed-curve for Link Completion of Multi-relational Heterogeneous Knowledge Graphs 计算机科学, 2023, 50(4): 172-180. https://doi.org/10.11896/jsjkx.220500135 |
[2] | 马廷淮, 孙圣杰, 荣欢, 钱敏峰. 基于动态记忆和双层重构强化的知识图谱至文本转译模型 Knowledge Graph-to-Text Model Based on Dynamic Memory and Two-layer Reconstruction Reinforcement 计算机科学, 2023, 50(3): 12-22. https://doi.org/10.11896/jsjkx.220700111 |
[3] | 汪璟玢, 赖晓连, 林新宇, 杨心逸. 基于关系约束的上下文感知时态知识图谱补全 Context-aware Temporal Knowledge Graph Completion Based on Relation Constraints 计算机科学, 2023, 50(3): 23-33. https://doi.org/10.11896/jsjkx.220400255 |
[4] | 陈富强, 寇嘉敏, 苏利敏, 李克. 基于图神经网络的多信息优化实体对齐模型 Multi-information Optimized Entity Alignment Model Based on Graph Neural Network 计算机科学, 2023, 50(3): 34-41. https://doi.org/10.11896/jsjkx.220700242 |
[5] | 刘昕炜, 陶传奇. 一种静态分析与知识图谱结合的Java冗余代码检测方法 Method of Java Redundant Code Detection Based on Static Analysis and Knowledge Graph 计算机科学, 2023, 50(3): 65-71. https://doi.org/10.11896/jsjkx.220700240 |
[6] | 陈姝睿, 梁子然, 饶洋辉. 细粒度语义知识图谱增强的中文OOV词嵌入学习 Fine-grained Semantic Knowledge Graph Enhanced Chinese OOV Word Embedding Learning 计算机科学, 2023, 50(3): 72-82. https://doi.org/10.11896/jsjkx.220700249 |
[7] | 蒋川宇, 韩翔宇, 杨文蕊, 吕博涵, 黄小欧, 谢夏, 谷阳. 医学知识图谱研究与应用综述 Survey of Medical Knowledge Graph Research and Application 计算机科学, 2023, 50(3): 83-93. https://doi.org/10.11896/jsjkx.220700241 |
[8] | 李志飞, 赵月, 张龑. 基于表示学习的知识图谱推理研究综述 Survey of Knowledge Graph Reasoning Based on Representation Learning 计算机科学, 2023, 50(3): 94-113. https://doi.org/10.11896/jsjkx.220900136 |
[9] | 王晓飞, 樊学强, 李章维. 基于迁移学习和多视图特征融合提高RNA碱基相互作用预测 Improving RNA Base Interactions Prediction Based on Transfer Learning and Multi-view Feature Fusion 计算机科学, 2023, 50(3): 164-172. https://doi.org/10.11896/jsjkx.211200186 |
[10] | 刘泽京, 邬楠, 黄抚群, 宋友. 基于知识图谱与协同过滤混合策略的在线编程评测系统题目推荐模型 Hybrid Programming Task Recommendation Model Based on Knowledge Graph and Collaborative Filtering for Online Judge 计算机科学, 2023, 50(2): 106-114. https://doi.org/10.11896/jsjkx.211200105 |
[11] | 李俊林, 欧阳智, 杜逆索. 基于改进区域候选网络的场景文本检测 Scene Text Detection with Improved Region Proposal Network 计算机科学, 2023, 50(2): 201-208. https://doi.org/10.11896/jsjkx.211000191 |
[12] | 单中原, 杨恺, 赵俊峰, 王亚沙, 徐涌鑫. 一种增量式本体模型与数据模式映射的图谱实例模型构建演化方法 Ontology-Schema Mapping Based Incremental Entity Model Construction and Evolution Approach of Knowledge Graph 计算机科学, 2023, 50(1): 18-24. https://doi.org/10.11896/jsjkx.220500205 |
[13] | 荣欢, 钱敏峰, 马廷淮, 孙圣杰. 基于先验知识图谱的多代理被遮挡目标类别推理模型 Novel Class Reasoning Model Towards Covered Area in Given Image Based on InformedKnowledge Graph Reasoning and Multi-agent Collaboration 计算机科学, 2023, 50(1): 243-252. https://doi.org/10.11896/jsjkx.220700112 |
[14] | 徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺. 时序知识图谱表示学习 Temporal Knowledge Graph Representation Learning 计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204 |
[15] | 饶志双, 贾真, 张凡, 李天瑞. 基于Key-Value关联记忆网络的知识图谱问答方法 Key-Value Relational Memory Networks for Question Answering over Knowledge Graph 计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277 |
|