基于贝叶斯规则的具有层次注意力的知识补全

doi:10.11896/jsjkx.221000056

Abstract

Abstract: As artificial intelligence in the big data era,knowledge graphs are widely used in many fields.Knowledge graphs gene-rally suffer from incompleteness and sparsity.As a sub-task of knowledge acquisition,knowledge completion aims to predict mis-sing links from known triples in the knowledge base.However,existing knowledge completion methods generally ignore the auxi-liary role of entity type jointly with neighborhood information,which can improve the knowledge completion accuracy.There are other problems such as feature information closely encodes into the objective function,and integration operations depend on the training process highly.To this end,a Bayesian rule-based knowledge completion method with hierarchical attention is proposed.Firstly,it regards entity type and neighborhood information as hierarchical structures,groups by relationship.It calculates each type information's attention weights independently.Then the entity types and neighborhood information encoding are regarded as the prior probability.The instance information encoding as likelihood probability.The two are combined according to the Bayesian rule.Experimental results show that the mean reciprocal rank(MRR ) metric in the FB15k dataset improves 14.4% over ConvE and 10.7% over TKRL.The MRR metric in the FB15k-237 dataset improves 2.1% over TACT.In the FB15k,FB15k-237 and YAGO26K-906 datasets,its Hits@1 reaches 77.5%,73.8% and 95.1% respectively,which demonstrates the introduction of type information and neighborhood information with hierarchical structure can embed richer and more accurate descriptive information for entities,and thus improve the accuracy of knowledge completion.

Key words: Bayesian rule, Entity type, Hierarchical attention, Knowledge graph completion

CLC Number:

TP311

SHAN Xiaohuan, ZHAO Xue, CHEN Tingwei. Bayesian Rule-based Knowledge Completion with Hierarchical Attention[J].Computer Science, 2023, 50(11): 234-240.

References

[1]HUANG X,ZHANG J,LI D,et al.Knowledge graph embedding based question answering[C]//Proceedings of the Twelfth ACM International Conference on Web Search and Data Mi-ning.2019:105-113.
[2]WANG X,HE X,CAO Y,et al.Kgat:Knowledge graph attention network for recommendation[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Disco-very & Data Mining.2019:950-958.
[3]KAPANIPATHI P,THOST V,PATEL S S,et al.Infusingknowledge into the textual entailment task using graph convolutional networks[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:8074-8081.
[4]WANG S,DU Z J,MENG X F.Research progress of large-scale knowledge graph completion technology[J].Chinese Science:Information Science,2020,50(4):551-575.
[5]BORDES A,USUNIER N,GARCIA-DURAN A,et al.Translating embeddings for modeling multi-relational data[C]//Proceedings of the Advances in Neural Information Processing Systems.2013:2787-2795.
[6]YANG B,YIH W T,HE X,et al.Embedding entities and relations for learning and inference in knowledge bases[C]//Proceedings of the International Conference on Learning Representations.2015.
[7]TROUILLON T,WELBL J,RIEDEL S,et al.Complex embeddings for simple link prediction[C]//Proceedings of the 33th International Conference on Machine Learning.2016:2071-2080.
[8]SUN Z,DENG Z H,NIE J Y,et al.Rotate:Knowledge graph embedding by relational rotation in complex space[C]//Proceedings of the International Conference on Learning Representations.2019.
[9]DETTMERS T,MINERVINI P,STENETROP P,et al.Convolutional 2dknowledge graph embeddings[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2018:1811-1818.
[10]WANG Z,ZHANG J,FENG J,et al.Knowledge graph embedding by translating on hyperplanes[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2014:1112-1119.
[11]LIN Y,LIU Z,SUN M,et al.Learning entity and relation embeddings for knowledge graph completion[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2015:2181-2187.
[12]NICKEL M,TRESP V,KRIEGEL H P.A three-way model for collective learning on multi-relational data[C]//Proceedings of the International Conference on Machine Learning.2011:809-816.
[13]WANG J,LI X N,LI G Y.Knowledge Graph Completion Algorithm Based on Adaptive Attention Mechanism[J].Computer Science,2022,49(7):204-211.
[14]ZHANG S,TAY Y,YAO L,et al.Quaternion knowledge graph embeddings[C]//Proceedings of the Advances in Neural Information Processing Systems.2019:2731-2741.
[15]XIE R,LIU Z,SUN M.Representation Learning of Knowledge Graphs with Hierarchical Types[C]//Proceedings of the International Joint Conference on Artificial Intelligence.2016:2965-2971.
[16]MA S,DING J,JIA W,et al.TransT:Type-based multiple embedding representations for knowledge graph completion[C]//Proceedings of the Joint European Conference on Machine Learning and Knowledge Discovery in Databases.2017:717-733.
[17]HAO J,CHEN M,YU W,et al.Universal representation lear-ning of knowledge bases by jointly embedding instances and ontological concepts[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mi-ning.2019:1709-1719.
[18]ZHANG Z,CAI J,ZHANG Y,et al.Learning hierarchy-aware knowledge graph embeddings for link prediction[C]//Procee-dings of the Thirty-Fourth AAAI Conference on Artificial Intelligence.2020:3065-3072.
[19]CHEN J,HE H,WU F,et al.Topology-Aware Correlations Between Relations for Inductive Link Prediction in Knowledge Graphs[C]//Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence.2021:6271-6278.
[20]ZHANG Y,ZHANG X,WANG J,et al.Generalized RelationLearning with Semantic Correlation Awareness for Link Prediction[C]//Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence.2021:4679-4687.
[21]WANG C,ZHOU X,PAN S,et al.Exploring Relational Semantics for Inductive Knowledge Graph Completion[C]//Procee-dings of the Thirty-Sixth AAAI Conference on Artificial Intelligence.2022.
[22]CAO Z,XU Q,YANG Z,et al.Geometry Interaction Knowledge Graph Embeddings[C]//Proceedings of the Thirty-Sixth AAAI Conference on Artificial Intelligence.2022.
[23]SCHLICHTKRULL M,KIPF T N,BLOEM P,et al.Modeling relational data with graph convolutional networks[C]//Proceedings of the Extended Semantic Web Conference.2018:593-607.
[24]BANSAL T,JUAN D C,RAVI S,et al.A2n:Attending toneighbors for knowledge graph inference[C]//Proceedings of the Association for Computational Linguistics.2019:4387-4392.
[25]NATHANI D,CHAUHAN J,SHARMA C,et al.Learning attention-based embeddings for relation prediction in knowledge graphs[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.2019:4719-4723.
[26]QIN X,SHEIKH N,REINWALD B,et al.Relation-aware graph attention model with adaptive self-adversarial training[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2021:5774-5782.
[27]KINGMA D P,BA J L.Adam:A method forstochastic optimization[C]//Proceedings of the International Conference on Lear-ning Representations.2014.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Bayesian Rule-based Knowledge Completion with Hierarchical Attention

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 5

Metrics

Comments

Recommended 0

[1]	CHEN Yujue, HU He, LI Qiang. Construction of Badminton Knowledge Graph Completion Model Based on Deep Learning [J]. Computer Science, 2023, 50(11A): 220900205-6.
[2]	LIN Xueyuan, E Haihong , SONG Wenyu, LUO Haoran, SONG Meina. QubitE:Qubit Embedding for Knowledge Graph Completion [J]. Computer Science, 2023, 50(11): 201-209.
[3]	WANG Jie, LI Xiao-nan, LI Guan-yu. Adaptive Attention-based Knowledge Graph Completion [J]. Computer Science, 2022, 49(7): 204-211.
[4]	CHEN Heng, WANG Wei-mei, LI Guan-yu, SHI Yi-ming. Knowledge Graph Completion Model Using Quaternion as Relational Rotation [J]. Computer Science, 2021, 48(5): 225-231.
[5]	QU Qiang, WU Xin-jie and CHEN Xue-bo. Bayesian Rule-based Background Model for Object Device-free Localization [J]. Computer Science, 2016, 43(6): 127-130.