Computer Science ›› 2026, Vol. 53 ›› Issue (4): 393-405.doi: 10.11896/jsjkx.250400050

• Artificial Intelligence • Previous Articles     Next Articles

Enhancing Temporal Knowledge Graph Reasoning Method with Graph Information Bottleneck and Transformer

XIN Yichen1, LI Shichong1, CHEN Bin1, CHENG Zhangtao1, LI Ye1,2, ZHOU Fan1   

  1. 1 School of Information and Software Engineering, University of Electronic Science and Technology of China, Chengdu 610054, China
    2 Kash Institute of Electronics and Information Industry, Kash, Xinjiang 844099, China
  • Received:2025-04-11 Revised:2025-07-04 Online:2026-04-15 Published:2026-04-08
  • About author:XIN Yichen,born in 1981,Ph.D candidate.His main research interests include data mining and knowledge graph.
    LI Ye,born in 1993,Ph.D,associate researcher.His main research interests include deep learning and spatio-temporal data mining.
  • Supported by:
    National Natural Science Foundation of China(62176043,62072077,U22A2097) and Xinjiang Uygur Autonomous Region Key Research and Development Program(2024B03041).

Abstract: Temporal knowledge graphs(TKGs) dynamically record event knowledge in the form of quadruples(subject,relation,object,timestamp),effectively capturing the dynamic evolution of knowledge in the real world.As a result,they have been widely applied in various domains such as recommender systems,large language models,and knowledge-based question answering.However,their inherent incompleteness poses significant challenges for further development and application.Temporal knowledge graph reasoning aims to predict missing event knowledge in TKGs,and has thus attracted considerable attention from both academia and industry.Existing methods for TKG reasoning mainly focus on extracting structural information within graph snapshots and modeling temporal dependencies between them.Nonetheless,they still suffer from two major limitations:1)insufficient handling of noise and redundancy present in the snapshots during the modeling process;2)an overreliance on local temporal patterns within short time windows,while ignoring global temporal dependencies across the entire TKG.To address these issues,this paper proposes GIBformer,a novel temporal knowledge graph reasoning framework that integrates the graph information bottleneck principle with a Transformer architecture.Specifically,it first introduces the graph information bottleneck to compress structural information in TKGs,preserving key information that is highly relevant to downstream prediction tasks while effectively filtering out noise and redundancy.Then,a Transformer with multi-head attention is employed to capture global temporal dependencies across snapshots,while also incorporating local temporal dynamics to enhance the prediction of missing event knowledge.Extensive experiments conducted on four widely-used benchmark datasets demonstrate the effectiveness of the proposed model.

Key words: Temporal knowledge graph, Graph information bottleneck, Transformer, Temporal knowledge graph reasoning

CLC Number: 

  • TP391
[1]CHEN J,CHEN J,YU Z.Incorporating structured common-sense knowledge in story completion[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2019:6244-6251.
[2]WU S,SUN F,ZHANG W,et al.Graph neural networks in re-commender systems:a survey[J].ACM Computing Surveys,2022,55(5):1-37.
[3]CHEN B,SHEN H,CHENG Z,et al.Unveiling cross-modalconsistency:Taming inter-and intra-modal noise for robust multi-modal knowledge graph completion[J].Information Processing & Management,2026,63(2):104472.
[4]CHEN B,YANG Y,CHENG Z,et al.Semantic duality in hypergraphs:Uncertainty-aware bipolar evidence aggregation for temporal knowledge graph reasoning[J].Expert Systems with Applications,2026,299:130162.
[5]CHEN M Q,ZHANG C L,YU H.Temporal knowledge graph reasoning model incorporating cross-time commonality features[J].Journal of Chongqing University of Posts and Telecommunications(Natural Science Edition),2025,37(5):708-716.
[6]SCHLICHTKRULL M,KIPF T N,BLOEM P,et al.Modeling relational data with graph convolutional networks[C]//The Semantic Web:15th International Conference.Springer,2018:593-607.
[7]LIANG K,MENG L,LIU M,et al.A survey of knowledgegraph reasoning on graph types:Static,dynamic,and multi-modal[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2024,46(12):9456-9478.
[8]CHEN B,WU J,LIU X,et al.Enhancing Temporal Knowledge Graph for Future Event Prediction with Long-Term Dense Graph[J].Tsinghua Science and Technology,2026,31(1):621-638.
[9]JIN W,QU M,JIN X,et al.Recurrent Event Network:Autoregressive Structure Inferenceover Temporal Knowledge Graphs[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing(EMNLP).2020:6669-6683.
[10]LIU Y,MA Y,HILDEBRANDT M,et al.Tlogic:Temporal lo-gical rules for explainable link forecasting on temporal knowledge graphs[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2022:4120-4127.
[11]TISHBY N,PEREIRA F C,BIALEK W.The information bottleneck method[EB/OL].https://www.princeton.edu/~wbialek/our_papers/tishby+al_99.pdf.
[12]VASWANI A,SHAZEER N,PARMAR N,et al.Attention is all you need[C]//Proceedings of the 31st International Confe-rence on Neural Information Processing Systems.2017:6000-6010.
[13]CHEN B,XIAO C,ZHOU F.Natural evolution-based dual-level aggregation for temporal knowledge graph reasoning[C]//Findings of the association for computational linguistics:EMNLP 2024.2024:9274-9284.
[14]LEBLAY J,CHEKOL M W.Deriving validity time in know-ledge graph[C]//Companion Proceedings of the Web Confe-rence 2018.2018:1771-1776.
[15]BORDES A,USUNIER N,GARCIA-DURAN A,et al.Translating embeddings for modeling multi-relational data[C]//Proceedings of the 27th International Confefence on Neural Information Processing Systems.2013:2785-2795.
[16]CHEN K,WANG Y,LI Y,et al.Rotateqvs:Representing temporal information as rotations in quaternion vector space for temporal knowledge graph completion[J].arXiv:2203.07993,2022.
[17]DASGUPTA S S,RAY S N,TALUKDAR P.Hyte:Hyper-plane-based temporally aware knowledge graph embedding[C]//Proceedings of the 2018 Conference on Empirical Me-thods in Natural Language Processing.2018:2001-2011.
[18]WANG J,WANG B,GAO J,et al.QDN:A quadruplet distributor network for temporal knowledge graph completion[J].IEEE Transactions on Neural Networks and Learning Systems,2024,35(10):14018-14030.
[19]LI Z,JIN X,LI W,et al.Temporal knowledge graph reasoningbased on evolutional representation learning[C]//Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval.2021:408-417.
[20]DENG S,RANGWALA H,NING Y.Dynamic knowledge graph based multi-event forecasting[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Disco-very & Data Mining.2020:1585-1595.
[21]LI Y,SUN S,ZHAO J.TiRGN:Time-Guided Recurrent Graph Network with Local-Global Historical Patterns for Temporal Knowledge Graph Reasoning[C]//IJCAI.2022:2152-2158.
[22]YANG L,WU F,ZHENG Z,et al.Heterogeneous Graph Information Bottleneck[C]//IJCAI.2021:1638-1645.
[23]XU Y,OU J,XU H,et al.Temporal knowledge graph reasoning with historical contrastive learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2023:4765-4773.
[24]HAN Z,CHEN P,MA Y,et al.Explainable subgraph reasoning for forecasting on temporal knowledge graphs[C]//Internatio-nal Conference on Learning Representations.2020.
[25]TISHBY N,ZASLAVSKY N.Deep learning and the informa-tion bottleneck principle[C]//2015 IEEE Information Theory Workshop(ITW).IEEE,2015:1-5.
[26]SUN Q,LI J,PENG H,et al.Graph structure learning with va-riational information bottleneck[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2022:4165-4174.
[27]YU J,XU T,RONG Y,et al.Recognizing predictive substructures with subgraph information bottleneck[J].IEEE Transactions on Pattern Analysis and Machine Intelligence,2021,46(3):1650-1663.
[28]WU T,REN H,LI P,et al.Graph information bottleneck[J].Advances in Neural Information Processing Systems,2020,33:20437-20448.
[29]YUAN H,SUN Q,FU X,et al.Dynamic graph information bottleneck[C]//Proceedings of the ACM Web Conference 2024.2024:469-480.
[30]HU Z,GUTIÉRREZ-BASULTO V,XIANG Z,et al.Trans-former-based entity typing in knowledge graphs[J].arXiv:2210.11151,2022.
[31]XU Z,YE P,CHEN H,et al.Ruleformer:Context-aware rule mining over knowledge graph[C]//Proceedings of the 29th International Conference on Computational Linguistics.2022:2551-2560.
[32]HAN Z,LIAO R,LIU B,et al.Enhanced temporal knowledgeembeddings with contextualized language representations[C]//ICLR.2022.
[33]SUN H,GENG S,ZHONG J,et al.Graph hawkes transformer for extrapolated reasoning on temporal know-ledge graphs[C]//Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing.2022:7481-7493.
[34]GAO Y,HE Y,KAN Z,et al.Learning joint structural and temporal contextualized knowledge embeddings for temporal know-ledge graph completion[C]//Findings of the Association for Computational Linguistics:ACL 2023.2023:417-430.
[35]JANG E,GU S,POOLE B.Categorical reparameterization with gumbel-softmax[J].arXiv:1611.01144,2016.
[36]ZHU C,CHEN M,FAN C,et al.Learning from history:Modeling temporal knowledge graphs with sequential copy-generation networks[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2021:4732-4740.
[37]KAZEMI S M,GOEL R,EGHBALI S,et al.Time2vec:Lear-ning a vector representation of time[J].arXiv:1907.05321,2019.
[38]SHANG C,TANG Y,HUANG J,et al.End-to-end structure-aware convolutional networks for knowledge base completion[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2019:3060-3067.
[39]GARCIA-DURAN A,DUMANČIĆ S,NIEPERT M.Learning Sequence Encoders for Temporal Knowledge Graph Completion[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.2018:4816-4821.
[40]LEETARU K,SCHRODT P A.Gdelt:Global data on events,location,and tone,1979-2012[C]//ISA Annual Convention.Citeseer,2013:1-49.
[41]LI Z,JIN X,LI W,et al.Temporal knowledge graph reasoning based on evolutional representation learning[C]//Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval.2021:408-417.
[42]LI Z,GUAN S,JIN X,et al.Complex Evolutional PatternLearning for Temporal Knowledge Graph Reasoning[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics.2022:290-296.
[43]DETTMERS T,MINERVINI P,STENETORP P,et al.Con-volutional 2d knowledge graph embeddings[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2018.
[44]JIANG T,LIU T,GE T,et al.Towards time-aware knowledge graph completion[C]//Proceedings of COLING 2016,the 26th International Conference on Computational Linguistics.2016:1715-1724.
[45]HAN Z,DING Z,MA Y,et al.Learning neural ordinary equations for forecasting future links on temporal knowledge graphs[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing.2021:8352-8364.
[46]LIU K,ZHAO F,XU G,et al.RETIA:relation-entity twin-interact aggregation for temporal knowledge graph extrapolation[C]//2023 IEEE 39th International Conference on Data Engineering(ICDE).IEEE,2023:1761-1774.
[47]LIU Z,TAN L,LI M,et al.SiMFy:A Simple Yet Effective Approach for Temporal Knowledge Graph Reasoning[C]//Fin-dings of the Association for Computational Linguistics:EMNLP 2023.2023:3825-3836.
[1] CHEN Han, XU Zefeng, JIANG Jiu, FAN Fan, ZHANG Junjian, HE Chu, WANG Wenwei. Large Language Model and Deep Network Based Cognitive Assessment Automatic Diagnosis [J]. Computer Science, 2026, 53(3): 41-51.
[2] LI Zequn, DING Fei. Fatigue Driving Detection Based on Dual-branch Fusion and Segmented Domain AdaptationTransfer Learning [J]. Computer Science, 2026, 53(3): 78-87.
[3] YU Ding, LI Zhangwei. Prediction Method of RNA Secondary Structure Based on Transformer Architecture [J]. Computer Science, 2026, 53(3): 375-382.
[4] LI Jiahao, JING Junchang, XU Qian, LIU Dong. GTKT:Knowledge Tracing Model Integrating Connectivism Learning and Multi-layer TemporalGraph Transformer [J]. Computer Science, 2026, 53(2): 78-88.
[5] PAN Jian, WANG Xuhao. Time Series Forecasting Model Integrating Multi-scale Features and Attention Mechanism [J]. Computer Science, 2026, 53(2): 180-186.
[6] LIU Chenhong, LI Fenglian, YANG Jia, WANG Suzhe, CHEN Guijun. Boundary-focused Multi-scale Feature Fusion Network for Stroke Lesion Segmentation [J]. Computer Science, 2026, 53(2): 264-272.
[7] WANG Cheng, JIN Cheng. KAN-based Unsupervised Multivariate Time Series Anomaly Detection Network [J]. Computer Science, 2026, 53(1): 89-96.
[8] HU Hailong, XU Xiangwei, LI Yaqian. Drug Combination Recommendation Model Based on Dynamic Disease Modeling [J]. Computer Science, 2025, 52(9): 96-105.
[9] DENG Jiayan, TIAN Shirui, LIU Xiangli, OUYANG Hongwei, JIAO Yunjia, DUAN Mingxing. Trajectory Prediction Method Based on Multi-stage Pedestrian Feature Mining [J]. Computer Science, 2025, 52(9): 241-248.
[10] DING Zhengze, NIE Rencan, LI Jintao, SU Huaping, XU Hang. MTFuse:An Infrared and Visible Image Fusion Network Based on Mamba and Transformer [J]. Computer Science, 2025, 52(8): 188-194.
[11] LIU Huayong, XU Minghui. Hash Image Retrieval Based on Mixed Attention and Polarization Asymmetric Loss [J]. Computer Science, 2025, 52(8): 204-213.
[12] LIU Yajun, JI Qingge. Pedestrian Trajectory Prediction Based on Motion Patterns and Time-Frequency Domain Fusion [J]. Computer Science, 2025, 52(7): 92-102.
[13] HUANG Xingyu, WANG Lihui, TANG Kun, CHENG Xinyu, ZHANG Jian, YE Chen. EFormer:Efficient Transformer for Medical Image Registration Based on Frequency Division and Board Attention [J]. Computer Science, 2025, 52(7): 151-160.
[14] WANG Youkang, CHENG Chunling. Multimodal Sentiment Analysis Model Based on Cross-modal Unidirectional Weighting [J]. Computer Science, 2025, 52(7): 226-232.
[15] LONG Xiao, HUANG Wei, HU Kai. Bi-MI ViT:Bi-directional Multi-level Interaction Vision Transformer for Lung CT ImageClassification [J]. Computer Science, 2025, 52(6A): 240700183-6.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!