STransH:一种改进的基于翻译模型的知识表示模型

doi:10.11896/j.issn.1002-137X.2019.09.026

摘要/Abstract

摘要： 最近,以深度学习为代表的表示学习技术受到广泛关注。表示学习旨在将研究对象的语义信息表示为低维稠密实值向量。因此,一系列知识表示模型被提出,其中基于翻译模型的经典方法TransE不仅模型复杂度低、计算效率高,而且具有良好的知识表达能力。但是,TransE方法在处理自反、一对多、多对一和多对多等复杂关系时存在局限性。鉴于此,文中提出一种改进的知识表示模型STransH,分别在实体空间和关系空间建模,并采用单层神经网络的非线性操作来加强实体和关系的语义联系。同时,受TransH模型的启发,引入投影到特定关系超平面的机制,使得实体在不同的关系中有不同的角色。在模型训练时,通过替换语义相似实体来提高生成负例的质量。最后,在公开的数据集FB15K和WN18上进行链接预测实验,分析和验证了所提方法的有效性。相比于TransE和TransH模型,STransH在各项性能指标上均取得了较大提升,其Hits@10和三元组分类准确率分别提高近10%。

关键词: 表示学习, 链接预测, 三元组分类, 知识图谱

Abstract: Recently,representation learning technology represented by deep learning has attracted many attentions in natural language processing,computer vision and speech recognition.Representation learning aims to project the interested objects into a low-dimensional,dense and real-valued semantic space.To this end,a number of models and methods were proposed for knowledge embedding.Among them,TransE is a classic translation-based method with low model complexity,high computational efficiency and favorable knowledge representation ability.However,it has limitations in dealing with complex relations including reflexive,one-to-many,many-to-one and many-to-many relations.In light of this,this paper proposed a revised translation-based method for knowledge graph representation,namely STransH.In this method,firstly,entity and relation embeddings are built in separate entity space and relation space,and then the non-linear operation of single-layer network layer is adopted to enhance the semantic connection between entity and relation.Inspired by TransH,this paper introduced the relation-oriented hyperspace model,thus projecting head and tail entities to the hyperspace of a given relation for distinction.Besides,it also proposed a simple trick to improve the quality of negative triplets.At last,it conducted extensive experiments on link prediction and triplet classification on benchmark datasets like WordNet and Freebase.Experimental results show that STransH performs significant improvements over TransE and TransH compared with TransE and TransH,and its Hits@10 and triplet classification accuracy are increased by nearly 10% respectively.

Key words: Knowledge graph, Link prediction, Representation learning, Triplet classification

中图分类号:

TP391

陈晓军, 向阳. STransH:一种改进的基于翻译模型的知识表示模型[J]. 计算机科学, 2019, 46(9): 184-189. https://doi.org/10.11896/j.issn.1002-137X.2019.09.026

CHEN Xiao-jun, XIANG Yang. STransH:A Revised Translation-based Model for Knowledge Representation[J]. Computer Science, 2019, 46(9): 184-189. https://doi.org/10.11896/j.issn.1002-137X.2019.09.026

参考文献

[1]MILLER G.Wordnet-a Lexical Database for English [J].Communications of the Acm,1995,38(11):39-41.
[2]BOLLACKER K,EVANS C,PARITOSH P,et al.Freebase:a collaboratively created graph database for structuring human knowledge[C]//Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data.New York:ACM,2008:1247-1250.
[3]BENGIO Y,COURVILLE A,VINCENT P.Representationlearning:a review and new perspectives [J].IEEE Transactions on Pattern Analysis & Machine Intelligence,2013,35(8):1798-1828.
[4]MIKOLOV T,SUTSKEVER I,CHEN K,et al.DistributedRepresentations of Words and Phrases and their Compositionality [J].Advances in Neural Information Processing Systems,2013,26:3111-3119.
[5]BORDES A,WESTON J,COLLOBERT R,et al.LearningStructured Embeddings of Knowledge Bases[C]//Proceedings of the Twenty-Fifth AAAI Conference on Artificial Intelligence.Palo Alto:AAAI Press,2011:301-306.
[6]SOCHER R,CHEN D,MANNING C D,et al.Reasoning with neural tensor networks for knowledge base completion[C]//Neural Information Processing Systems 2013.Lake Tahoe:NIPS,2013:926-934.
[7]BORDES A,GLOROT X,WESTON J,et al.A semantic matching energy function for learning with multi-relational data [J].Machine Learning,2014,94(2):233-259.
[8]BORDES A,USUNIER N,GARCIA-DURAN A,et al.Translating Embeddings for Modeling Multi-relational Data[C]//Proceedings of Neural Information Processing Systems 2013.Massachusetts:MIT Press,2013:2787-2795.
[9]NGUYEN D Q,SIRTS K,QU L,et al.STransE:a novel embedding model of entities and relationships in knowledge bases[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics.San Diego:ACL,2016:460-466.
[10]WANG Z,ZHAN J,FENG J,et al.Knowledge graph embedding by translating on hyperplanes[C]//Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence.Palo Alto:AAAI Press,2014:1112-1119.
[11]JIANG T W,QIN B,LIU T.Open Domain Knowledge Reaso-ning for Chinese Based on Representation Learning[J].Journal of Chinese Information Processing,2018,32(2):34-41.(in Chinese)姜天文,秦兵,刘挺.基于表示学习的开放域中文知识推理 [J].中文信息学报,2018,32(2):34-41.
[12]LIN Y,LIU Z,ZHU X,et al.Learning entity and relation embeddings for knowledge graph completion[C]//Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence.Palo Alto:AAAI Press,2015:2181-2187.
[13]LIN Y,LIU Z,LUAN H,et al.Modeling Relation Paths forRepresentation Learning of Knowledge Bases[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.Lisbon:ACL,2015:705-714.
[14]XIAO H,HUANG M,HAO Y,et al.TransA:An adaptive approach for knowledge graph embedding [J].arXiv:1509.05490,2015.
[15]JI G,HE S,XU L,et al.Knowledge graph embedding via dynamic mapping matrix[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing.Beijing:ACL,2015:687-696.
[16]JENATTON R,ROUXN L,BORDES A,et al.A latent factor model for highly multi-relational data[C]//Proceedings of Neural Information Processing Systems 2012.Massachusetts:MIT Press,2012:3167-3175.
[17]NICKEL M,TRESP V,KRIEGELH P.A Three-Way Model for Collective Learning on Multi-Relational Data[C]//Proceedings of the 28th International Conference on Machine Learning.New York:ACM,2011:809-816.
[18]AN B,HAN X P,SUN L,et al.Triple Classification Based on Synthesized Features for Knowledge Based[J].Journal of Chinese Information Processing,2016,30(6):84-89.(in Chinese)安波,韩先培,孙乐,等.基于分布式表示和多特征融合的知识库三元组分类 [J].中文信息学报,2016,30(6):84-89.

相关文章 15

[1]	宋杰, 梁美玉, 薛哲, 杜军平, 寇菲菲. 基于无监督集群级的科技论文异质图节点表示学习方法 Scientific Paper Heterogeneous Graph Node Representation Learning Method Based onUnsupervised Clustering Level 计算机科学, 2022, 49(9): 64-69. https://doi.org/10.11896/jsjkx.220500196
[2]	黄丽, 朱焱, 李春平. 基于异构网络表征学习的作者学术行为预测 Author’s Academic Behavior Prediction Based on Heterogeneous Network Representation Learning 计算机科学, 2022, 49(9): 76-82. https://doi.org/10.11896/jsjkx.210900078
[3]	徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺. 时序知识图谱表示学习 Temporal Knowledge Graph Representation Learning 计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[4]	饶志双, 贾真, 张凡, 李天瑞. 基于Key-Value关联记忆网络的知识图谱问答方法 Key-Value Relational Memory Networks for Question Answering over Knowledge Graph 计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[5]	吴子仪, 李邵梅, 姜梦函, 张建朋. 基于自注意力模型的本体对齐方法 Ontology Alignment Method Based on Self-attention 计算机科学, 2022, 49(9): 215-220. https://doi.org/10.11896/jsjkx.210700190
[6]	孔世明, 冯永, 张嘉云. 融合知识图谱的多层次传承影响力计算与泛化研究 Multi-level Inheritance Influence Calculation and Generalization Based on Knowledge Graph 计算机科学, 2022, 49(9): 221-227. https://doi.org/10.11896/jsjkx.210700144
[7]	秦琪琦, 张月琴, 王润泽, 张泽华. 基于知识图谱的层次粒化推荐方法 Hierarchical Granulation Recommendation Method Based on Knowledge Graph 计算机科学, 2022, 49(8): 64-69. https://doi.org/10.11896/jsjkx.210600111
[8]	王杰, 李晓楠, 李冠宇. 基于自适应注意力机制的知识图谱补全算法 Adaptive Attention-based Knowledge Graph Completion 计算机科学, 2022, 49(7): 204-211. https://doi.org/10.11896/jsjkx.210400129
[9]	黄璞, 杜旭然, 沈阳阳, 杨章静. 基于局部正则二次线性重构表示的人脸识别 Face Recognition Based on Locality Regularized Double Linear Reconstruction Representation 计算机科学, 2022, 49(6A): 407-411. https://doi.org/10.11896/jsjkx.210700018
[10]	马瑞新, 李泽阳, 陈志奎, 赵亮. 知识图谱推理研究综述 Review of Reasoning on Knowledge Graph 计算机科学, 2022, 49(6A): 74-85. https://doi.org/10.11896/jsjkx.210100122
[11]	邓凯, 杨频, 李益洲, 杨星, 曾凡瑞, 张振毓. 一种可快速迁移的领域知识图谱构建方法 Fast and Transmissible Domain Knowledge Graph Construction Method 计算机科学, 2022, 49(6A): 100-108. https://doi.org/10.11896/jsjkx.210900018
[12]	杜晓明, 袁清波, 杨帆, 姚奕, 蒋祥. 军事指控保障领域命名实体识别语料库的构建 Construction of Named Entity Recognition Corpus in Field of Military Command and Control Support 计算机科学, 2022, 49(6A): 133-139. https://doi.org/10.11896/jsjkx.210400132
[13]	熊中敏, 舒贵文, 郭怀宇. 融合用户偏好的图神经网络推荐模型 Graph Neural Network Recommendation Model Integrating User Preferences 计算机科学, 2022, 49(6): 165-171. https://doi.org/10.11896/jsjkx.210400276
[14]	钟将, 尹红, 张剑. 基于学术知识图谱的辅助创新技术研究 Academic Knowledge Graph-based Research for Auxiliary Innovation Technology 计算机科学, 2022, 49(5): 194-199. https://doi.org/10.11896/jsjkx.210400195
[15]	朱敏, 梁朝晖, 姚林, 王翔坤, 曹梦琦. 学术引用信息可视化方法综述 Survey of Visualization Methods on Academic Citation Information 计算机科学, 2022, 49(4): 88-99. https://doi.org/10.11896/jsjkx.210300219

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed