Computer Science

Computer Science ›› 2025, Vol. 52 ›› Issue (5): 248-259.doi: 10.11896/jsjkx.241100100

• Artificial Intelligence • Previous Articles     Next Articles

Research on Intelligent Judgment of Criminal Cases Based on Large Language Models

CONG Yingnan1, HAN Linrui2,3, MA Jiayu4, ZHU Jinqing5   

  1. 1 Business School,China University of Political Science and Law,Beijing 100088,China
    2 Ministry of Education Laboratory of Philosophy and Social Sciences－The Data Law Lab,China University of Political Science and Law,Beijing 100088,China
    3 The Institute for Data Law,China University of Political Science and Law,Beijing 100088,China
    4 School of Law,Tsinghua University,Beijing 100084,China
    5 Beijing Bytedance Network Technology Co. Ltd.,Beijing 100043,China
  • Received:2024-11-18 Revised:2025-03-06 Online:2025-05-15 Published:2025-05-12
  • About author:CONG Yingnan,born in 1985,Ph.D,associate professor,Ph.D supervisor,is a member of CCF(No.J0079M).His main research interests include big data on business and law,artificial intelligence,blockchain,Fin-tech,Reg-tech and complex system.
    ZHU Jinqing,born in 1984,postgraduate,engineer,is a member of CCF(No.D0034M).His main research interests include database systems,content data analysis,artificial intelligence,and knowledge graphs.
  • Supported by:
    Program for Young Innovative Research Team in China University of Political Science and Law(25CXTD04),2022 National Key R&D Program of China“Social Governance and Smart Society Technology Support” Key Special Project(2022YFC3303000) and General Project of Humanities and Social Sciences Research of the Ministry of Education(22YJC190003).

PDF (PC)

71

Abstract: The intelligentization of criminal case trials has been a hot research topic in the development of digital courts.In the conventional method based on natural language processing,the model directly predicts the final judgment based on the facts of the case.However,when dealing with complex criminal cases,the model may fail to identify the logical dependencies between legal elements and to clearly present the legal reasoning process.The intelligent criminal case trial method based on large language models proposed in this paper follows the approach of “annotating case corpus-pre-training large language model-reinforcing trial logic”.The first step is to annotate the legal elements of the case such as subjects,objects,subjective elements,and objective elements by combining automated annotating with manual correction and create a structured reasoning dataset.The second step is to use ChatGLM3-6b-32k as the foundational large language model for incremental pre-training based on the GLM pre-training framework.The last step is to fine-tune the parameter and increase legal knowledge using the LoRA parameter-efficient fine-tuning strategy and large language model retrieval enhancement technology,thereby reinforcing the trial logic.Experimental results indicate that,compared to Qwen-7B-Chat and Baichuan2-7B-Chat,the ChatGLM3-6b-32k model exhibits superior performance after supervised fine-tuning.The introduction of judicial syllogism significantly enhances the logicality of the judgment texts,ma-king them closer to the reasoning of human judges.In the tasks of charge prediction and sentencing prediction,the model created using this method shows a significant improvement in accuracy compared to the MTL-Fusion,Lawformer,and BERT models.In addition,compared to Legal-BERT and CaseLawBERT,which are trained on European and American legal texts,the ChatGLM3-6b-32k model better suits the trial logic of Chinese criminal cases and demonstrates stronger capabilities in handling long texts.This paper not only explores the application of large language models in intelligent criminal case trials,but also provides valuable references for research on large language models in justice.

Key words: Digital court, Legal judgement prediction, Judicial syllogism, Large language model, Parameter-efficient fine-tuning

CLC Number: 

  • TP183
[1]CUI J,SHEN X,WEN S.A Survey on Legal Judgment Prediction:Datasets,Metrics,Models and Challenges[J].IEEE Access,2023,11:102050-102071.
[2]YUE L,LIU Q,WU H,et al.Circumstances enhanced Criminal Court View Generation[C]//Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval.2021:1855-1859.
[3]WU Y,KUANG K,ZHANG Y,et al.De-Biased Court's View Generation with Causality[C]//Proceedings of the 2020 Confe-rence on Empirical Methods in Natural Language Processing.2020:763-780.
[4]YE H,JIANG X,LUO Z,et al.Interpretable Charge Predictions for Criminal Cases:Learning to Generate Court Views from Fact Descriptions[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics.2018:1854-1864.
[5]SILVER N.The Signal and the Noise:Why So Many Predictions Fail-But Some Don't[M].Penguin,2012.
[6]DENG W,PEI J,KONG K,et al.Syllogistic Reasoning for Legal Judgment Analysis[C]//Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing.2023:13997-14009.
[7]ALETRAS N,TSARAPATSANIS D,PREOTIUC-PIETRO D,et al.Predicting judicial decisions of the European Court of Human Rights:a Natural Language ProcessingPerspective[J].PeerJ Computer Science,2016,2:e93.
[8]KORT F.Predicting Supreme Court Decisions Mathematically:A Quantitative Analysis of the “Right to Counsel” Cases[J].American Political Science Review,1957,51(1):1-12.
[9]NAGEL S.Applying Correlation Analysis to Case Prediction[J].Texas Law Review,1963,42:1006.
[10]LAWLOR R.What Computers Can Do:Analysis and Prediction of Judicial Decisions[J].American Bar Association Journal,1963,49(4):337-344.
[11]KEOWN R.Mathematical Models For Legal Prediction,2 Computer L.J.829(1980)[J].UIC John Marshall Journal of Information Technology & Privacy Law,1980,2(1):29.
[12]SUSSKIND R.Expert Systems in Law:A Jurisprudential Ap-proach to Artificial Intelligence and Legal Reasoning[J].The Modern Law Review,1986,49(2):168-194.
[13]LIU C L,LIAO T M.Classifying Criminal Charges in Chinese for Web-Based Legal Services[C]//Asia-pacific Web Confe-rence.2005:64-75.
[14]BOELLA G,DI CARO L,HUMPHREYS L.Using Classification to Support Legal Knowledge Engineers in the Eunomos Legal Document Management System[C]//Fifth International Workshop on Juris-informatics.2011.
[15]GONÇALVES T,QUARESMA P.Evaluating PreprocessingTechniques in a Text Classification Problem[C]//Proceedings of the Conference of the Brazilian Computer Society.2005.
[16]LIN W,KUO T,CHANG T,et al.Exploiting Machine Learning Models for Chinese Legal Documents Labeling,Case Classification,and Sentencing Prediction[C]//Proceedings of ROCLING.2012:140.
[17]LIU C,HSIEH C.Exploring Phrase-based Classification of Judicial Documents for Criminal Charges in Chinese[C]//Procee-dings of the 16th International Conference on Foundations of Intelligent Systems.2006:681-690.
[18]LIU C,CHANG C,HO J.Case Instance Generation and Refinement for Case-Based Criminal Summary Judgments in Chinese.[J].Journal of Information Science and Engineering,2004,20(4):783-800.
[19]KIANMEHR K,ALHAJJ R.Crime Hot-Spots Prediction Using Support Vector Machine[C]//IEEE International Conference on Computer Systems and Applications.2006:952-959.
[20]SULEA O,ZAMPIERI M,MALMASI S,et al.Exploring theUse of Text Classification in the Legal Domain[C]//Procee-dings of the Second Workshop on Automated Semantic Analysis of Information in Legal Texts co-located with the 16th International Conference on Artificial Intelligence and Law.2017:2143.
[21]KATZ D.Quantitative Legal Prediction-or-How I Learned toStop Worrying and Start Preparing for the Data Driven Future of the Legal Services Industry[J].Emory Law Journal,2013,62(4):909.
[22]ZELEZNIKOW J,STRANIERI A.The Split-up System:In-tegrating Neural Networks and Rule-Based Reasoning in the Legal Domain[C]//Proceedings of the 5th International Conference on Artificial Intelligence and Law.1995:185-194.
[23]HUANG N,HE J,SUN J,et al.Improved Lawformer-basedApproach for Forecasting Crimes[J].Journal of Beijing University of Technology,2019,45(8):742-748.
[24]LUO B,FENG Y,XU J,et al.Learning to Predict Charges for Criminal Cases with Legal Basis[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Proces-sing.2017:2727-2736.
[25]HU Z,LI X,TU C,et al.Few-Shot Charge Prediction with Discriminative Legal Attributes[C]//Proceedings of the 27th International Conference on Computational Linguistics.2018:487-498.
[26]ZHAO J,GUAN Z,XU C,et al.Charge Prediction by Constitutive Elements Matching of Crimes[C]//International Joint Conferences on Artificial Intelligence Organization.2022:4517-4523.
[27]CHEN Y,LIU Y,HO W.A text mining approach to assist the general public in the retrieval of legal documents[J].Journal of the American Society for Information Science and Technology,2013,64(2):280-290.
[28]KIM M,XU Y,GOEBEL R.Legal Question Answering Using Ranking SVM and Syntactic/Semantic Similarity[C]//New Frontiers in Artificial Intelligence.2015:244-258.
[29]ZHONG H,GUO Z,TU C,et al.Legal Judgment Prediction via Topological Learning[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.2018:3540-3549.
[30]YUE L,LIU Q,JIN B,et al.NeurJudge:A Circumstance-aware Neural Framework for Legal Judgment Prediction[C]//Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval.2021:973-982.
[31]PETERS M,NEUMANN M,IYYER M,et al.Deep Contextualized Word Representations[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics.2018:2227-2237.
[32]DEVLIN J,CHANG M,LEE K,et al.BERT:Pre-training ofDeep Bidirectional Transformers for Language Understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics.2019:4171-4186.
[33]LIU Y,OTT M,GOYAL N,et al.A robustly optimized BERT pre-training approach with post-training[C]//China National Conference on Chinese Computational Linguistics.Cham:Springer International Publishing,2021:471-484.
[34]CHALKIDIS I,FERGADIOTIS M,MALAKASIOTIS P,et al.LEGAL-BERT:The Muppets straight out of Law School[C]//Findings of the Association for Computational Linguistics.2020:2898-2904.
[35]ZHENG L,GUHA N,ANDERSON B,et al.When does pre-training help? assessing self-supervised learning for law and the CaseHOLD dataset of 53,000+ legal holdings[C]//Procee-dings of the Eighteenth International Conference on Artificial Intelligence and Law.2021:159-168.
[36]XIAO C,HU X,LIU Z,et al.Lawformer:APre-trained Lan-guage Model for Chinese Legal Long Documents[J].AI Open,2021,2:79-84.
[37]BROWN T,MANN B,RYDER N,et al.Language Models are Few-Shot Learners[J].Advances in Neural Information Processing Systems,2020,33:1877-1901.
[38]KOJIMA T,GU S,REID M,et al.Large Language Models are Zero-Shot Reasoners[J].Advances in Neural Information Processing Systems,2022,35:22199-22213.
[39]ZELIKMAN E,WU Y,MU J,et al.STaR:self-taught reasoner bootstrapping reasoning with reasoning[C]//Proceedings of the 36th International Conference on Neural Information Processing Systems.2024:15476-15488.
[40]TRAUTMANN D,PETROVA A,SCHILDER F.Legal Prompt Engineering for Multilingual Legal Judgement Prediction[J].arXiv:2212.02199,2022.
[41]YU F,QUARTEY L,SCHILDER F.Legal Prompting:Teaching a Language Model to think like a Lawyer[J].arXiv:2212.01326,2022.
[42]JIANG C,YANG X.Legal Syllogism Prompting:TeachingLarge Language Models for Legal Judgment Prediction[C]//Proceedings of the Nineteenth International Conference on Artificial Intelligence and Law.2023:417-421.
[43]HUANG Q,TAO M,ZHANG C,et al.Lawyer LLaMA Technical Report[J].arXiv:2305.15062,2023.
[44]CHEN S,HOU Y,CUI Y,et al.Recall and Learn:Fine-tuning Deep Pretrained Language Models with Less Forgetting[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing.2020:7870-7881.
[45]RADFORD A,NARASIMHAN K,SALIMANS T,et al.Improving Language Understanding by Generative Pre-Training[EB/OL].[2025-03-11].https://www.mikecaptain.com/resources/pdf/GPT-1.pdf.
[46]DEEPA M.Bidirectional Encoder Representations from Trans-formers(BERT) Language Model for Sentiment Analysis Task[J].Turkish Journal of Computer and Mathematics Education(TURCOMAT),2021,12(7):1708-1721.
[47]RAFFEL C,SHAZEER N,ROBERTS A,et al.Exploring theLimits of Transfer Learning with a Unified Text-to-Text Transformer[J].Journal of Machine Learning Research,2020,21(140):1-67.
[48]DU Z,QIAN Y,LIU X,et al.GLM:General Language Model Pretraining with Autoregressive Blank Infilling[C]//Procee-dings of the 60th Annual Meeting of the Association for Computational Linguistics.2022:320-335.
[49]WEN S,QIAN L,HU H,et al.Review of Research Progress on Question-Answering Techniques Based on Large Language Models[J].Data Analysis and Knowledge Discovery,2024,8(6):16-29.
[50]HOULSBY N,GIURGIU A,JASTRZEBSKI S,et al.Parame-ter-Efficient Transfer Learning for NLP[C]//Proceedings of the 36th International Conference on Machine Learning.PMLR,2019:2790-2799.
[51]LI X L,LIANG P.Prefix-Tuning:Optimizing ContinuousPrompts for Generation[J].arXiv:2101.00190,2021.
[52]HU E J,SHEN Y,WALLIS P,et al.LoRA:Low-Rank Adaptation of Large Language Models[J].arXiv:2106.09685,2021.
[53]ZHANG Q T,WANG Y C,WANG H X,et al.A Comprehensive Review of Large Language Model Fine-tuning[J].ComputerEngineering and Applications,2024,60(17):17-33.
[54]ZHAO Y,HE J W,ZHU S C,et al.Security of Large Language Models:Current Status and Challenges[J].Computer Science,2024,51(1):68-71.
[55]LEWIS P,PEREZ E,PIKTUS A,et al.Retrieval-AugmentedGeneration for Knowledge-Intensive NLP Tasks[J].Advances in Neural Information Processing Systems,2020,33:9459-9474.
[56]XIAO C,ZHONG H,GUO Z,et al.CAIL2018:A Large-Scale Legal Dataset for Judgment Prediction[J].arXiv:1807.02478,2018.
[57]CHENG F D.Legal Consultation Data and Corpus from China Law Network:Replication Data for Design and Research of Text Classification System[EB/OL].[2025-03-11].https://doi.org/10.18170/DVN/OLO4G8.
[58]ZHONG H,XIAO C,TU C,et al.JEC-QA:a Legal-domainQuestion Answering Dataset[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:9701-9708.
[59]PAPINENI K,ROUKOS S,WARD T,et al.BLEU:a Method for Automatic Evaluation of Machine Translation[C]//Procee-dings of the 40th Annual Meeting of the Association for Computational Linguistics.2002:311-318.
[60]LIN C Y.ROUGE:A Package for Automatic Evaluation ofSummaries[C]//Text Summarization Branches Out.Barcelona,Spain:Association for Computational Linguistics,2004:74-81.
[61]XU L,LI A,ZHU L,et al.Superclue:A Comprehensive Chinese Large Language Model Benchmark[J].arXiv:2307.15020,2023.
[62]FEI Z,SHEN X,ZHU D,et al.LawBench:Benchmarking Legal Knowledge of Large Language Models[J].arXiv:2309.16289,2023.
[63]BAI J,BAI S,CHU Y,et al.Qwen Technical Report[J].arXiv:2309.16609,2023.
[64]YANG A,XIAO B,WANG B,et al.Baichuan 2:Open Large-scale Language Models[J].arXiv:2309.10305,2023.
[65]LIU Z L,ZHANG M S,ZHENG R R,et al.Multi-task LearningModel for Legal Judgment Predictions with Charge Keywords[J].Journal of Tsinghua University(Science and Technology),2019,59(7):497-504.
[66]FAN A M,WANG Y C.Multi Task Intelligent Legal Judgment Method Based on BERT Model[J].Microelectronics & Compu-ter,2022,39(9):107-114.
[67]MA C S.Law Towards a Digital Society[M].Beijing:Law Press China,2021.
[68]ZHENG Y.The Embedding of Artificial Intelligence in the Rule of Law:A Jurisprudential Reflection of Intelligent Justice[J].Jianghai Academic Journal,2024,(3):172-180,256.
[1] CHEN Xuhao, HU Sipeng, LIU Hongchao, LIU Boran, TANG Dan, ZHAO Di. Research on LLM Vector Dot Product Acceleration Based on RISC-V Matrix Instruction Set Extension [J]. Computer Science, 2025, 52(5): 83-90.
[2] ZHU Shucheng, HUO Hongying, WANG Weikang, LIU Ying, LIU Pengyuan. Automatic Optimization and Evaluation of Prompt Fairness Based on Large Language Model Itself [J]. Computer Science, 2025, 52(4): 240-248.
[3] SONG Xingnuo, WANG Congyan, CHEN Mingkai. Survey on 3D Scene Reconstruction Techniques in Metaverse [J]. Computer Science, 2025, 52(3): 17-32.
[4] CHENG Dawei, WU Jiaxuan, LI Jiangtong, DING Zhijun, JIANG Changjun. Study on Evaluation Framework of Large Language Model’s Financial Scenario Capability [J]. Computer Science, 2025, 52(3): 239-247.
[5] HUANG Xueqin, ZHANG Sheng, ZHU Xianqiang, ZHANG Qianzhen, ZHU Cheng. Generative Task Network:New Paradigm for Autonomic Task Planning and Execution Based on LLM [J]. Computer Science, 2025, 52(3): 248-259.
[6] XU Siyao, ZENG Jianjun, ZHANG Weiyan, YE Qi, ZHU Yan. Dependency Parsing for Chinese Electronic Medical Record Enhanced by Dual-scale Collaboration of Large and Small Language Models [J]. Computer Science, 2025, 52(2): 253-260.
[7] ZENG Zefan, HU Xingchen, CHENG Qing, SI Yuehang, LIU Zhong. Survey of Research on Knowledge Graph Based on Pre-trained Language Models [J]. Computer Science, 2025, 52(1): 1-33.
[8] DUN Jingbo, LI Zhuo. Survey on Transmission Optimization Technologies for Federated Large Language Model Training [J]. Computer Science, 2025, 52(1): 42-55.
[9] ZHENG Mingqi, CHEN Xiaohui, LIU Bing, ZHANG Bing, ZHANG Ran. Survey of Chain-of-Thought Generation and Enhancement Methods in Prompt Learning [J]. Computer Science, 2025, 52(1): 56-64.
[10] LI Tingting, WANG Qi, WANG Jiakang, XU Yongjun. SWARM-LLM:An Unmanned Swarm Task Planning System Based on Large Language Models [J]. Computer Science, 2025, 52(1): 72-79.
[11] YAN Yusong, ZHOU Yuan, WANG Cong, KONG Shengqi, WANG Quan, LI Minne, WANG Zhiyuan. COA Generation Based on Pre-trained Large Language Models [J]. Computer Science, 2025, 52(1): 80-86.
[12] CHENG Zhiyu, CHEN Xinglin, WANG Jing, ZHOU Zhongyuan, ZHANG Zhizheng. Retrieval-augmented Generative Intelligence Question Answering Technology Based on Knowledge Graph [J]. Computer Science, 2025, 52(1): 87-93.
[13] LIU Changcheng, SANG Lei, LI Wei, ZHANG Yiwen. Large Language Model Driven Multi-relational Knowledge Graph Completion Method [J]. Computer Science, 2025, 52(1): 94-101.
[14] LIU Yumeng, ZHAO Yijing, WANG Bicong, WANG Chao, ZHANG Baomin. Advances in SQL Intelligent Synthesis Technology [J]. Computer Science, 2024, 51(7): 40-48.
[15] LI Zhanqi, WU Xinwei, ZHANG Lei, LIU Quanzhou, XIE Hui, XIONG Deyi. Automatic Test Case Generation Method for Automotive Electronic Control System Verification [J]. Computer Science, 2024, 51(12): 63-70.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.