Computer Science ›› 2025, Vol. 52 ›› Issue (8): 308-316.doi: 10.11896/jsjkx.240900170

• Artificial Intelligence • Previous Articles     Next Articles

Multi-defendant Legal Judgment Prediction with Multi-turn LLM and Criminal Knowledge Graph

WANG Dongsheng   

  1. School of Information Management for Law,China University of Political Science and Law,Beijing 102249,China
  • Received:2024-09-29 Revised:2025-01-22 Online:2025-08-15 Published:2025-08-08
  • About author:WANG Dongsheng,born in 1992,Ph.D,lecturer.His main research interests include natural language processing and knowledge graph.
  • Supported by:
    China University of Political Science and Law Research Innovation Project(24KYGH013) and Fundamental Research Funds for the Central Universities.

Abstract: Some studies use advanced Large Language Model(LLM) technologies to understand legal facts and predict the defendant's charges,prison term and other judgment results.For further in-depth research,this paper chooses the more complex task of predicting legal judgments for multiple defendants,which is more challenging than predicting for a single defendant.Specifically,upgrading the interaction with LLM from a single-turn to multi-turn process to enhance LLM's understanding of criminal cases.In addition, two types of crime Knowledge Graphs(KGs) are construted to describe the case.The criminal relationship knowledge graph depicts the relationships of assistance between the defendants,while the sentencing circumstance knowledge graph represents the core criminal details of the case.Through crime knowledge graphs,a retrieval system is designed to provide LLM with references for similar case judgments.In experiments on predicting legal judgments for multiple defendants, the prediction results of the proposed method are better than the comparison methods,which shows that the designs of multi-turn LLM interactions and crime knowledge graphs are effective.

Key words: Multi-defendant, Legal judgment prediction, Large language model, Criminal knowledge graph

CLC Number: 

  • TP391.1
[1]TONG S X,YUAN J L,ZHANG P L,et al.Legal Judgment Prediction via graph boosting with constraints[J].Information Processing and Management,2024,61(2):103663.
[2]ZHANG C Y,QU H,CUI C R,et al.Process Supervision Based Sequence Multi-task Method for Legal Judgement Prediction[J].Computer Science,2021,48(3):227-232.
[3]GAN L L,LI B K,KUANG K,et al.Exploiting Contrastive Learning and Numerical Evidence for Confusing Legal Judgment Prediction[C]//Conference on Empirical Methods in Natural Language Processing.2023:12174-12185.
[4]FENG G Y,QIN Y B,HUANG R Z,et al.Criminal Action Graph:A semantic representation model of judgement documents for legal charge prediction[J].Information Processing and Management,2024,60(5):103421.
[5]SUKANYA G,JAYARAJU P.Modified Hierarchical-Attention Network model for legal judgment predictions[J].Data & Knowledge Engineering,2023,147:102203.
[6]ZHAO Q H,GAO T H,GUO N.LA-MGFM:A legal judgment prediction method via sememe-enhanced graph neural networks and multi-graph fusion mechanism[J].Information Processing and Management,2023,60(5):103455.
[7]XIAO C J,HU X Y,LIU Z Y,et al.Lawformer:A pre-trained language model for Chinese legal long documents[J].AI Open,2021,2:79-84.
[8]ZHOU Z,SHI J X,SONG P X,et al.LawGPT:A Chinese Legal Knowledge-Enhanced Large Language Model[J].arXiv:2406.04614,2024.
[9]PAN S P,LU T,GU N,et al.Charge Prediction for Multi-defendant Cases with Multi-scale Attention[C]//Computer Supported Cooperative Work and Social Computing.2019:766-777.
[10]LYU Y G,HAO J T,WANG Z H,et al.Multi-Defendant Legal Judgment Prediction via Hierarchical Reasoning[C]//Findings of the Association for Computational Linguistics.2023:2198-2209.
[11]QIN W C,CAO Z L,YU W J,et al.Explicitly Integrating Judgment Prediction with Legal Document Retrieval:A Law-Guided Generative Approach[C]//The 47th International ACM SIGIR Conference on Research and Development in Information Retrieval.2024:2210-2220.
[12]KORT F.Predicting Supreme Court Decisions Mathematically:A Quantitative Analysis of the “Right to Counsel” Cases[J].The American Political Science Review,1957,51(1):1-12.
[13]FENG Y,LI C Y,NG V.Legal Judgment Prediction via Event Extraction with Constraints[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics.2022:648-664.
[14]LIU Y F,WU Y Q,ZHANG Y T,et al.ML-LJP:Multi-LawAware Legal Judgment Prediction[C]//The 46th International ACM SIGIR Conference on Research and Development in Information Retrieval.2023:1023-1034.
[15]CHALKIDIS I,FERGADIOTIS M,MALAKASIOTIS P,et al.LEGAL-BERT:The Muppets straight out of Law School[J].arXiv:2010.02559,2020.
[16]BELTAGY L,PETERS M,COHAN A.Longformer:The Long-Document Transformer[J].arXiv:2004.05150,2020.
[17]OUYANG L,WU J,JIANG X,et al.Training language models to follow instructions with human feedback[C]//Annual Conference on Neural Information Processing Systems 2022.2022:27730-27744.
[18]SUN Q,HUANG K,YANG X C,et al.Consistency GuidedKnowledge Retrieval and Denoising in LLMs for Zero-shot Do-cument-level Relation Triplet Extraction[C]//The ACM on Web Conference.2024:4407-4416.
[19]JI B J,DUAN X Y,QIU Z Y,et al.Submodular-based In-context Example Selection for LLMs-based Machine Translation[C]//The 2024 Joint International Conference on ComputationalLinguistics,Language Resources and Evaluation.2024:15398-15409.
[20]AGRAWAL G,PAL K,DENG Y L,et al.CyberQ:Generating Questions and Answers for Cybersecurity Education Using Knowledge Graph-Augmented LLMs[C]//Thirty-Eighth AAAI Conference on Artificial Intelligence.2024:23164-23172.
[21]HUANG Q Z,TAO M X,AN Z W,et al.Lawyer LLaMA Tech-nical Report[J].arXiv:2305.15062,2023.
[22]WU Y Q,ZHOU S Y,LIU Y F,et al.Precedent-Enhanced Legal Judgment Prediction with LLM and Domain-Model Collaboration[C]//The 2023 Conference on Empirical Methods in Natural Language Processing.2023:12060-12075.
[23]SHUI R H,CAO Y X,WANG X,et al.A Comprehensive Evaluation of Large Language Models on Legal Judgment Prediction[C]//Findings of the Association for Computational Linguistics:EMNLP 2023.2023:7337-7348.
[24]DENG C L,MAO K L,ZHANG Y Y,et al.Enabling Discriminative Reasoning in LLMs for Legal Judgment Prediction[J].arXiv:2407.01964,2024.
[25]WEI J,WANG X Z,SCHUURMANS D,et al.Chain-of-Thought Prompting Elicits Reasoning in Large Language Models[C]//Annual Conference on Neural Information Processing Systems.2022:24824-24837.
[26]ZHANG Z P,WANG A Q,ZHANG Y,et al.Relation pruning and discriminative sampling over knowledge graph for long-tail recommendation[J].Information Sciences,2024,679:120871.
[27]LIU H Z,ZHU Y,WU Z H.Knowledge Graph-Based Behavior Denoising and Preference Learning for Sequential Recommendation[J].IEEE Transactions on Knowledge and Data Enginee-ring,2024,36(6):2490-2503.
[28]HU Y L,ZOU F T,HAN J J,et al.LLM-TIKG:Threat intelligence knowledge graph construction utilizing large language model[J].Computers & Security,2024,145:103999.
[29]FAN Z L,CHEN C C.CuPe-KG:Cultural perspective-basedknowledge graph construction of tourism resources via pretrained language models[J].Information Processing and Ma-nagement,2024,61(2):103646.
[30]ZHANG Y X,ZHAI S L,MENG Y,et al.Event is more valuable than you think:Improving the Similar Legal Case Retrieval via event knowledge[J].Information Processing and Management,2024,61(4):103729.
[31]CORREIA F,ALMEIDA A,et al.Fine-grained legal entity annotation:A case study on the Brazilian Supreme Court[J].Information Processing and Management,2022,59(1):102794.
[32]LIU D G,DU W H,LI L,et al.Augmenting Legal JudgmentPrediction with Contrastive Case Relations[C]//The 29th International Conference on Computational Linguistics.2022:2658-2667.
[33]YANG W M,JIA W J,ZHOU X J,et al.Legal Judgment Prediction via Multi-Perspective Bi-Feedback Network[C]//The Twenty-Eighth International Joint Conference on Artificial Intelligence.2019:4085-4091.
[34]XU N,WANG P H,CHEN L,et al.Distinguish Confusing Law Articles for Legal Judgment Prediction[C]//The 58th Annual Meeting of the Association for Computational Linguistics.2020:3086-3095.
[35]YUE L N,LIU Q,JIN B B,et al.NeurJudge:A Circumstance-aware Neural Framework for Legal Judgment Prediction[C]//The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval.2021:973-982.
[36]CUI Y M,CHE W X,LIU T,et al.Pre-Training With Whole Word Masking for Chinese BERT[J].IEEE/ACM Transactions on Audio,Speech and Language Processing,2021,29:3504-3514.
[37]XUE L T,CONSTANT N,ROBERTS A,et al.mT5:A Massively Multilingual Pre-trained Text-to-Text Transformer[C]//The 2021 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2021:483-498.
[1] WANG Limei, HAN Linrui, DU Zuwei, ZHENG Ri, SHI Jianzhong, LIU Yiqun. Privacy Policy Compliance Detection Method for Mobile Application Based on Large LanguageModel [J]. Computer Science, 2025, 52(8): 1-16.
[2] LI Maolin, LIN Jiajie, YANG Zhenguo. Confidence-guided Prompt Learning for Multimodal Aspect-level Sentiment Analysis [J]. Computer Science, 2025, 52(7): 241-247.
[3] CHEN Jinyin, XI Changkun, ZHENG Haibin, GAO Ming, ZHANG Tianxin. Survey of Security Research on Multimodal Large Language Models [J]. Computer Science, 2025, 52(7): 315-341.
[4] TU Ji, XIAO Wendong, TU Wenji, LI Lijian. Application of Large Language Models in Medical Education:Current Situation,Challenges and Future [J]. Computer Science, 2025, 52(6A): 240400121-6.
[5] LI Bo, MO Xian. Application of Large Language Models in Recommendation System [J]. Computer Science, 2025, 52(6A): 240400097-7.
[6] ZOU Rui, YANG Jian, ZHANG Kai. Low-resource Vietnamese Speech Synthesis Based on Phoneme Large Language Model andDiffusion Model [J]. Computer Science, 2025, 52(6A): 240700138-6.
[7] ZHOU Lei, SHI Huaifeng, YANG Kai, WANG Rui, LIU Chaofan. Intelligent Prediction of Network Traffic Based on Large Language Model [J]. Computer Science, 2025, 52(6A): 241100058-7.
[8] BAI Yuntian, HAO Wenning, JIN Dawei. Study on Open-domain Question Answering Methods Based on Retrieval-augmented Generation [J]. Computer Science, 2025, 52(6A): 240800141-7.
[9] ZHANG Le, CHE Chao, LIANG Yan. Hallucinations Proactive Relief in Diabetes Q&A LLM [J]. Computer Science, 2025, 52(6A): 240700182-10.
[10] YIN Baosheng, ZONG Chen. Research on Semantic Fusion of Chinese Polysemous Words Based on Large LanguageModel [J]. Computer Science, 2025, 52(6A): 240400139-7.
[11] HU Caishun. Study on Named Entity Recognition Algorithms in Audit Domain Based on Large LanguageModels [J]. Computer Science, 2025, 52(6A): 240700190-4.
[12] ZHAO Zheyu, WANG Zhongqing, WANG Hongling. Commodity Attribute Classification Method Based on Dual Pre-training [J]. Computer Science, 2025, 52(6A): 240500127-8.
[13] GAO Hongkui, MA Ruixiang, BAO Qihao, XIA Shaojie, QU Chongxiao. Research on Hybrid Retrieval-augmented Dual-tower Model [J]. Computer Science, 2025, 52(6): 324-329.
[14] CHEN Xuhao, HU Sipeng, LIU Hongchao, LIU Boran, TANG Dan, ZHAO Di. Research on LLM Vector Dot Product Acceleration Based on RISC-V Matrix Instruction Set Extension [J]. Computer Science, 2025, 52(5): 83-90.
[15] CONG Yingnan, HAN Linrui, MA Jiayu, ZHU Jinqing. Research on Intelligent Judgment of Criminal Cases Based on Large Language Models [J]. Computer Science, 2025, 52(5): 248-259.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!