Computer Science

Computer Science ›› 2026, Vol. 53 ›› Issue (5): 299-308.doi: 10.11896/jsjkx.250600023

• Artificial Intelligence • Previous Articles     Next Articles

EC-MIIP:Efficient Fine-tuning Small-parameter Large Language Model for Intellectual Property

LIU Xukai1, LIU Yang1,2, HUANG Haozhen3   

  1. 1 Department of Inteligent Science and Information Law, East China University of Political and Law, Shanghai 201620, China
    2 China Institute for Smart Court, Shanghai Jiao Tong University, Shanghai 200030, China
    3 School of Economic Law, East China University of Political Science and Law, Shanghai 201620, China
  • Received:2025-06-03 Revised:2025-08-13 Published:2026-05-08
  • About author:LIU Xukai,born in 2006,undergra-duate. His main research interests include large language models,agents and smart justice.
    LIU Yang,born in 1983,Ph.D,lecturer.Her main research interests include na-tural language processing and artificial intelligence.
  • Supported by:
    National Key Research and Development Program of China(2023YFC3306100,2023YFC3306105,2023YFC3306103) and Shanghai Philosophy and Social Science Planning Project(2023EFX011).

PDF (PC)

190

Abstract: In recent years,large language models have been developing rapidly,demonstrating excellent capabilities in several na-tural language processing tasks,and providing strong technical support in the field of intelligent justice.Combining model pre-training and fine-tuning techniques,this paper constructs a database of MIPLD(Micro-model Intellectual Property Learning Direction) intellectual property directions under small parameters,and realizes an algorithmic framework for distributed pre-training according to the characteristics of the discipline of law and the characteristics of the intellectual property systems.Subsequently,based on the database of MIPLD,high-quality fine-tuned Q&A pairs of each direction are constructed,and EC-MIIP,an intellectual property problem analysis model with high capacity density under small parameters,is realized,which can be used for tasks such as intellectual property doctrine quizzing,analysis the nature of the act,judicial case analysis,and legal document writing.Experimental results show that EC-MIIP performs better than Owen3-4B,Qwen3 full-parameter and Deepseek-R1 full-parameter models.This study not only explores the application of large language models in the intellectual property domain,but also provides a reference for realizing the applicability of small parameter models in the judicial domain.

Key words: Large language model, Model pre-training and fine-tuning, Intelligent law, Intellectual property, Small parameter

CLC Number: 

  • TP311
[1]LI Y,YAN Y,TONG Z,et al.Efficient fine-tuning of small-parameter large language models for biomedical bilingual multi-task applications[J].Applied Soft Computing,2025,175:113084.
[2]NGUYEN C,SHEN X,APONTE R,et al.Asurvey of small language models[J].arXiv:2410.2001,2024.
[3]DEEPSEEK-AI,GUO D,YANG D,et al.DeepSeek-R1:In-centivizingreasoning capability in LLMs via reinforcement learning[J].arXiv:2501.12948,2025.
[4]YANG A,YANG B,HUI B,et al.Qwen2technical report[J].arXiv:2407.10671,2024.
[5]YANG A,LI A,YANG B,et al.Qwen technical report[J].ar-Xiv:2505.09388,2025.
[6]COLOMBO P,PIRES T,BOUDIAF M,et al.SaulLM-7B:Apioneeringlarge language model for law[J].arXiv:2403.03883,2024.
[7]CUI J,NING M,LI Z,et al.Chatlaw:Amulti-agent collaborative legal assistant with knowledge graph enhanced mixture-of-experts large language model[J].arXiv:2306.16092,2023.
[8]LEE J S.Evaluating generative patent language models[J].World Patent Information,2023,72:102173.
[9]NI S,TAN M,BAI Y,et al.MoZIP:A multilingual benchmark to evaluate large language models in intellectual property[J].arXiv:2402.16389,2024.
[10]LEE J S.Instruct Patent GPT:training patent language models to follow instructions with human feedback[J].Artificial Intelligence and Law,2024,33:739-782.
[11]BAI Z,ZHANG R,CHEN L,et al.Patent GPT:a large language model for intellectual property[J].arXiv:2404.18255,2024.
[12]LV K,YANG Y,LIU T,et al.Full Parameter Fine-tuning for Large Language Models with Limited Resources[C]//Procee-dings of the 62nd Annual Meeting of the Association for Computational Linguistics.2024:8187-8198.
[13]HAN Z,GAO C,LIU J,et al.Parameter-efficient fine-tuning for large models:A Comprehensive Survey[J].arXiv:2403.14608,2024.
[14]HU E,SHEN Y,WALLIS P,et al.LoRA:Low-rank adaptation of large language models[C]//ICLR.2022.
[15]DETTMERS T,PAGNONI A,HOLTZMAN A,et al.QLoRA:Efficientfinetuning of quantized LLMs[J].Advances in Neural Information Processing Systems,2023,36:10088-10115.
[16]LAMY-POIRIER J.Layered gradient accumulation and modular pipeline parallelism:fast and efficient training of large language models[J].arXiv:2106.02679,2021.
[17]JOHNSON O,XINGYING C,KHAW K,et al.Ps-CALR:Perio-dic-shift cosine annealing learning rate for deep neural networks[J].IEEE Access,2023,11:139171-139186,.
[18]DONG Q,LI L,DAI D,Asurvey on in-context learning[J].ar-Xiv:2301.00234,2024.
[19]GUNASERKAR S,ZHANG Y,ANEJA J,et al.Textbooksare all you need[J].arXiv:2306.11644,2023.
[20]LZACARD G,LEWIS P,LOMELI M,et al.Atlas:Few-shotlearning with retrieval augmented language models[J].Journal of Machine Learning Research,2023,24(251):1-43.
[21]WEI J,WANG X,SCHUURMANS D,et al.Chain-of-thoughtprompting elicits reasoning in large language models[C]//NeurlPS 2022.2022
[22]FEI Z,SGEN X,ZHU D,et al.Lawbench:Benchmarking legal knowledge of large language models[J].arXiv:2309.16289,2023.
[23]DAI Y,FENG D,HUANG J,et al.Laiw:A chinese legal large language models benchmark[J].arXiv:2310.05620,2023.
[24]LI H,CHEN Y,AI Q,et al.LexEval:Acomprehensive Chinese legal benchmark for evaluating large language models[J].arXiv:2409.20288,2024.
[25]CHALKIDIS I,FERGADIOTIS M,PRODROMOS M P,et al.LEGAL-BERT:Themuppets straight out of Law School[C]//Findings of the Association for Computational Linguistics(EMNLP2020).2020:2898-2904.
[26]ZHENG L,GUHA N,ANDERSON B,et al.When does pre-training help? Assessing self-supervised learning for law and the CaseHOLD dataset of 53 000＋Legal Holdings[C]//Procee-dings of the Eighteenth International Conference on Artificial Intelligence and Law(ICAIL’21).2021:159-168.
[1] JI Wendi, WANG Yongquan, SHEN Yicheng. Boosting Generative Rule Extraction via Negative-aware Approach [J]. Computer Science, 2026, 53(5): 276-285.
[2] HAN Linrui, ZHENG Ri, CONG Yingnan. Explainable Sentencing Prediction Method Driven by Sentencing Rule Knowledge Graph [J]. Computer Science, 2026, 53(5): 286-298.
[3] WANG Shenghui, LI Teng. Innovative Automated Scoring Based on Large Language Models [J]. Computer Science, 2026, 53(5): 90-98.
[4] LIU Suyi, LIU Qi, GAO Weibo. Agent4Stu:Efficient LLM-based Student Answer Behavior Simulation Agent [J]. Computer Science, 2026, 53(4): 347-355.
[5] WU Qiaorui, LUO Li, ZHAO Cairong. LLM-augmented Training Framework with Cycle-Consistency Constraints [J]. Computer Science, 2026, 53(4): 377-383.
[6] HU Junjie, CHEN Yujie, HU Yikun, WEN Cheng, CAO Jialun, MA Zhi, SU Jie, SUN Weidi, TIAN Cong, QIN Shengchao. Formal Theorem Proving Empowered by Large Language Model:Survey and Perspectives [J]. Computer Science, 2026, 53(4): 1-23.
[7] WU Xianjie, LI Tongliang, LI Zhoujun. Survey of Table Question Answering Research [J]. Computer Science, 2026, 53(3): 295-306.
[8] XU Cheng, LIU Yuxuan, WANG Xin, ZHANG Cheng, YAO Dengfeng, YUAN Jiazheng. Review of Speech Disorder Assessment Methods Driven by Large Language Models [J]. Computer Science, 2026, 53(3): 307-320.
[9] LI Wenli, FENG Xiaonian, QIAN Tieyun. Few-shot Continuous Toxicity Detection Based on Large Language Model Augmentation [J]. Computer Science, 2026, 53(3): 321-330.
[10] WANG Zhibin, LI Shipeng, ZHOU Yuhang, LI Xue, ZHANG Zhonghui, JIANG Zhiwei, GU Rong, TIAN Chen, CHEN Guihai, ZHONG Sheng. Optimization of Service Level Objectives and System Level Metrics in Large Language ModelServing System [J]. Computer Science, 2026, 53(3): 23-32.
[11] ZHOU Yueyuan, LU Guanze, XIANG Jiawei, ZHANG Jiawei, SHAO En, HE Xin. Training System for Large Language Models Based on Adaptive Transpose on Hygon DCU [J]. Computer Science, 2026, 53(3): 33-40.
[12] CHEN Han, XU Zefeng, JIANG Jiu, FAN Fan, ZHANG Junjian, HE Chu, WANG Wenwei. Large Language Model and Deep Network Based Cognitive Assessment Automatic Diagnosis [J]. Computer Science, 2026, 53(3): 41-51.
[13] CHEN Yuyin, LI Guanfeng, QIN Jing, XIAO Yuhang. Survey on Complex Logical Query Methods in Knowledge Graphs [J]. Computer Science, 2026, 53(2): 273-288.
[14] GUO Luxiang, WANG Yueyu, LI Qianyue, LI Shasha, LIU Xiaodong, JI Bin, YU Jie. Comprehensive Survey of LLM-based Agent Operating Systems [J]. Computer Science, 2026, 53(1): 1-11.
[15] LIU Lilong, LIU Guoming, QI Baoyuan, DENG Xueshan, XUE Dizhan, QIAN Shengsheng. Efficient Inference Techniques of Large Models in Real-world Applications:A Comprehensive Survey [J]. Computer Science, 2026, 53(1): 12-28.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.