Computer Science ›› 2026, Vol. 53 ›› Issue (10): 295-300.doi: 10.11896/jsjkx.250900130

• Artificial Intelligence • Previous Articles     Next Articles

CLERAG:Retrieval-augmented Generation Framework for Legal QA with Cognitive-levelEvaluation

ZHAO Xunan, ZHAO Zhuofeng   

  1. School of Artificial Intelligence and Computer Science,North China University of Technology,Beijing 100144, China
  • Received:2025-09-21 Revised:2025-12-25 Published:2026-10-13
  • About author:ZHAO Xunan,born in 2000,postgra-duate,is a member of CCF(No.A02977G).Her main research interest is large-scale model engineering applications.
    ZHAO Zhuofeng,born in 1977,Ph.D,researcher,Ph.D supervisor,is a member of CCF(No.08923D).His main research interests include large-scale model engineering applications and embodied intelligence.
  • Supported by:
    National Key Research and Development Program of China(2023YFC3107900) and Guangzhou Key Research and Development Program(202206030009).

Abstract: With the growing use of large language models in legal question answering,existing methods still struggle with semantic understanding,provision matching,and reasoning.To address these issues,this paper proposes CLERAG,a retrieval-augmented generation method based on cognitive-level evaluation.Inspired by cognitive outsourcing,CLERAG structures semantic understanding and provision filtering as cooperative processes with external models.It consists of two components:cognitive-oriented query enhancement,which employs standardized rewriting and keyword expansion to align user queries with legal expressions,and a “divergent-convergent” evaluation that combines semantic scoring and a verification model to identify the most applicable provisions.Experimental results on CivGuideQA and LawBench show that CLERAG consistently outperforms baseline methods,achieving a 13.30% improvement in ROUGE-L over standard RAG on CivGuideQA.

Key words: Legal question answering, Retrieval-augmented generation, Query enhancement, Large language model, Cognitive-level evaluation

CLC Number: 

  • TP181
[1] AN Z W,LAI Y X,FENG Y S.Natural Language Understan-ding for Legal Text:A Review[J].Journal of Chinese Information Processing,2022,36(8):1-11.
[2] ZHANG H,ZHENG W H,DOU Z C,et al.Integrating Multi-Layer Structure Information of Law for Legal Judgement Prediction[J].Computer Engineering and Applications,2023,59(3):253-263.
[3] ACHIAM J,ADLER S,AGARWAL S,et al.Gpt-4 technical report[J].arXiv:2303.08774,2023.
[4] BAI J,BAI S,CHU Y,et al.Qwen technical report [J].arXiv:2309.16609,2023.
[5] BROWN T,MANN B,RYDER N,et al.Language models are few-shot learners[J].Advances in Neural Information Proces-sing Systems,2020,33:1877-1901.
[6] JI Z,LEE N,FRIESKE R,et al.Survey of hallucination in natural language generation[J].ACM Computing Surveys,2023,55(12):1-38.
[7] LEWIS P,PEREZ E,PIKTUS A,et al.Retrieval-augmentedgeneration for knowledge-intensive nlp tasks[J].Advances in Neural Information Processing Systems,2020,33:9459-9474.
[8] MA X,GONG Y,HE P,et al.Query rewriting in retrieval-augmented large language models[C]//Proceedings of the 2023 Conference on Empirical Methods in Natural Language Proces-sing.2023:5303-5315.
[9] LI R,WANG Y,WEN Z,et al.Different paths to the same destination:Diversifying LLMs generation for multi-hop open-domain question answering[J].Knowledge-Based Systems,2025,309:112789.
[10] ASAI A,WU Z,WANG Y,et al.Self-rag:Learning to retrieve,generate,and critique through self-reflection[J].arXiv:2310.11511,2023.
[11] BOSTROM N,SANDBERG A.Cognitive enhancement:me-thods,ethics,regulatory challenges[J].Science and Engineering Ethics,2009,15(3):311-341.
[12] WANG F Z,TANG X Y,YU S Q.Cognitive outsourcing based on generative artificial intelligence:An Analysis of interactive behavioral patterns and cognitive structural features[J].Acta Psychologica Sinica,2025,57(6):967.
[13] FEI Z,SHEN X,ZHU D,et al.Lawbench:Benchmarking legal knowledge of large language models[C]//Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing.ACL,2024:7933-7962.
[14] WANG Y,SHEN X,HUANG Z,et al.cLegal-QA:a Chinese legal question answering with natural language generation me-thods[J].Complex & Intelligent Systems,2025,11(1):77.
[15] DENG W,PEI J,KONG K,et al.Syllogistic reasoning for legal judgment analysis[C]//Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing.2023:13997-14009.
[16] ZHOU Z,SHI J X,SONG P X,et al.Lawgpt:A chinese legal knowledge-enhanced large language model[J].arXiv:2406.04614,2024.
[17] LI H,AI Q,CHEN J,et al.SAILER:structure-aware pre-trained language model for legal case retrieval[C]//Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval.2023:1035-1044.
[18] HUANG Q,TAO M,ZHANG C,et al.Lawyer llama technical report[J].arXiv:2305.15062,2023.
[19] GUU K,LEE K,TUNG Z,et al.Retrieval augmented language model pre-training[C]//International Conference on Machine Learning.PMLR,2020:3929-3938.
[20] MAO S,JIANG Y,CHEN B,et al.RaFe:Ranking Feedback Improves Query Rewriting for RAG[C]//Findings of the Association for Computational Linguistics(EMNLP 2024).ACL,2024:884-901.
[21] WANG Y,LI P,SUN M,et al.Self-Knowledge Guided Retrieval Augmentation for Large Language Models [C]//Findings of the Association for Computational Linguistics(EMNLP 2023).ACL,2023:10303-10315.
[22] GAO L,MA X,LIN J,et al.Precise zero-shot dense retrieval without relevance labels[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics.ACL,2023:1762-1777.
[23] SHI W,MIN S Y.REPLUG:Retrieval-Augmented Black-Box Language Models[C]//Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.ACL,2024:8371-8384.
[24] KARPUKHIN V,OGUZ B,MIN S,et al.Dense Passage Retrieval for Open-Domain Question Answering[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing(EMNLP).ACL,2020:6769-6781.
[25] XIONG L,XIONG C,LI Y,et al.Approximate nearest neighbor negative contrastive learning for dense text retrieval [J].arXiv.2007.00808,2020.
[26] KIM K,LEE J Y.RE-RAG:Improving Open-Domain QA Performance and Interpretability with Relevance Estimator in Retrieval-Augmented Generation[C]//Proceedings of the 2024 Conference on Empirical Methods in Natural Language Proces-sing.ACL,2024:22149-22161.
[1] HE Jiaojun, LI Xin. Review of Graph Learning Based on Large Language Models:Methods,Benchmarks and Advances [J]. Computer Science, 2026, 53(9): 1-15.
[2] LI Luozheng, LI Lingbo, YUAN Quan. Survey of Chinese Datasets for LLM Safety Alignment:Landscape and Prospects [J]. Computer Science, 2026, 53(9): 16-23.
[3] AN Ran, CHENG Kai, SHAO Tianhao, LI Changyuan, CHEN Yan. Framework,Techniques and Challenges of Task Planning Agent Based on Large Language Model [J]. Computer Science, 2026, 53(9): 24-37.
[4] LI Zhennan, QIAN Jiayan, WANG Xinzhi, ZHANG Hui. Technology Risk Structure Recognition Based on Multi-granularity Semantic Dual Reflection [J]. Computer Science, 2026, 53(9): 395-404.
[5] XIA Yi, ZHOU Gang, ZHANG Kaixiang, LAN Mingjing, LI Zhufeng, SU Benrong, HE Haofeng, FENG Shizhong. Survey on Collaborative-driven Knowledge Reasoning Technologies Integrating Large Language Models and Knowledge Graphs [J]. Computer Science, 2026, 53(9): 38-54.
[6] ZHANG Rongjie, PANG Xiongwen, WANG Fengling. Review of Large Language Model-based Time Series Modeling via Fine-tuning and AgentArchitecture [J]. Computer Science, 2026, 53(9): 55-70.
[7] CHEN Hai, FAN Changjun, SUN Boliang, CHENG Jie, KONG Junfeng, YUAN Yuchen. Order-based,LLM-augmented Kill-chain for Time-sensitive Target Engagement [J]. Computer Science, 2026, 53(9): 71-81.
[8] GUO Yuyang, SHI Lei, LIU Huan, DONG Yixiang, LI Rui. Domain-adapted and Dynamically Retrieval-augmented Approach for Large-scale History Discipline Model Construction [J]. Computer Science, 2026, 53(9): 92-100.
[9] LEI Kunhan, TAO Jiuyang, LI Chenyu, ZHONG Qi, YE Linfa. Method for Generating Operational Action Plans Based on Large Language Model Guided byChain-of-Thought [J]. Computer Science, 2026, 53(9): 101-109.
[10] JIA Zishuo, ZHANG Jian’ge, HE Haofeng, FENG Shizhong, LIU Yilin. Survey on Mutually Augmenting Technologies and Applications of Large Models and KnowledgeGraphs [J]. Computer Science, 2026, 53(8): 219-228.
[11] LIU Jing. Review of Music Artificial Intelligence Driven by Large Language Models [J]. Computer Science, 2026, 53(8): 229-244.
[12] ZHANG Haoran, HAO Wenning, JIN Dawei, CHENG Kai, LIU Junyang. Agentic Retrieval Augmented Generation Framework Based on Retrieval Task Planning and Reflection Mechanism [J]. Computer Science, 2026, 53(8): 285-297.
[13] ZHANG Zhonglin, XIA Hang. LSQ-RAG:Retrieval-enhanced Generation Framework Based on LLM-enhanced Ranker [J]. Computer Science, 2026, 53(8): 298-306.
[14] HAN Linrui, SONG Gaojie, ZHENG Ri, LI Bing, CUI Yan. Automated Judicial Document Generation for Criminal Cases Based on Multi-agent Collaboration [J]. Computer Science, 2026, 53(8): 336-356.
[15] FAN Ruxin, SUN Baicai, GONG Lina, YAO Xiangjuan, GONG Dunwei. Test Case Generation and Prioritization for Program Fault Diagnosis Based on Large LanguageModel [J]. Computer Science, 2026, 53(8): 365-374.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!