计算机科学

计算机科学 ›› 2026, Vol. 53 ›› Issue (5): 299-308.doi: 10.11896/jsjkx.250600023

• 人工智能 • 上一篇    下一篇

EC-MIIP:基于高效微调的知识产权小参数大语言模型

刘旭凯1, 刘洋1,2, 黄浩桢3   

  1. 1 华东政法大学智能科学与信息法学系 上海 201620
    2 上海交通大学智慧司法研究院 上海 200030
    3 华东政法大学经济法学院 上海 201620
  • 收稿日期:2025-06-03 修回日期:2025-08-13 发布日期:2026-05-08
  • 通讯作者: 刘洋(lyang@ecupl.edu.cn)
  • 作者简介:(liuxukaiqaq@163.com)
  • 基金资助:
    国家重点研发计划(2023YFC3306100,2023YFC3306105,2023YFC3306103);上海市哲学社会科学规划课题(2023EFX011)

EC-MIIP:Efficient Fine-tuning Small-parameter Large Language Model for Intellectual Property

LIU Xukai1, LIU Yang1,2, HUANG Haozhen3   

  1. 1 Department of Inteligent Science and Information Law, East China University of Political and Law, Shanghai 201620, China
    2 China Institute for Smart Court, Shanghai Jiao Tong University, Shanghai 200030, China
    3 School of Economic Law, East China University of Political Science and Law, Shanghai 201620, China
  • Received:2025-06-03 Revised:2025-08-13 Online:2026-05-08
  • About author:LIU Xukai,born in 2006,undergra-duate. His main research interests include large language models,agents and smart justice.
    LIU Yang,born in 1983,Ph.D,lecturer.Her main research interests include na-tural language processing and artificial intelligence.
  • Supported by:
    National Key Research and Development Program of China(2023YFC3306100,2023YFC3306105,2023YFC3306103) and Shanghai Philosophy and Social Science Planning Project(2023EFX011).

PDF (PC)

224

摘要： 近年来,大语言模型发展迅速,在多个自然语言处理任务上展现了出色的能力,也为智慧司法领域提供了强大的技术支持。对此,结合模型预训练和微调技术,构建了MIPLD(Micro-model Intellectual Property Learning Direction)知识产权数据库,并根据法学学科特性和知识产权门类特性,搭建分布预训练的算法框架。随后基于MIPLD数据库,构建了知识产权领域多方向的高质量微调问答对,实现了在小参数下拥有高能力密度的知识产权问题分析大语言模型EC-MIIP,该模型适用于知识产权的学理问答、行为性质分析、司法案例解析以及法律文书撰写等任务。实验结果表明,与Qwen3-4B、Qwen3全参和Deepseek-R1全参模型相比,EC-MIIP的性能更优。该研究不仅探索了大语言模型在知识产权领域中的应用,还为实现小参数模型在司法领域的适用提供了参考。

关键词: 大语言模型, 模型预训练和微调, 智慧法律, 知识产权, 小参数

Abstract: In recent years,large language models have been developing rapidly,demonstrating excellent capabilities in several na-tural language processing tasks,and providing strong technical support in the field of intelligent justice.Combining model pre-training and fine-tuning techniques,this paper constructs a database of MIPLD(Micro-model Intellectual Property Learning Direction) intellectual property directions under small parameters,and realizes an algorithmic framework for distributed pre-training according to the characteristics of the discipline of law and the characteristics of the intellectual property systems.Subsequently,based on the database of MIPLD,high-quality fine-tuned Q&A pairs of each direction are constructed,and EC-MIIP,an intellectual property problem analysis model with high capacity density under small parameters,is realized,which can be used for tasks such as intellectual property doctrine quizzing,analysis the nature of the act,judicial case analysis,and legal document writing.Experimental results show that EC-MIIP performs better than Owen3-4B,Qwen3 full-parameter and Deepseek-R1 full-parameter models.This study not only explores the application of large language models in the intellectual property domain,but also provides a reference for realizing the applicability of small parameter models in the judicial domain.

Key words: Large language model, Model pre-training and fine-tuning, Intelligent law, Intellectual property, Small parameter

中图分类号: 

  • TP311
[1]LI Y,YAN Y,TONG Z,et al.Efficient fine-tuning of small-parameter large language models for biomedical bilingual multi-task applications[J].Applied Soft Computing,2025,175:113084.
[2]NGUYEN C,SHEN X,APONTE R,et al.Asurvey of small language models[J].arXiv:2410.2001,2024.
[3]DEEPSEEK-AI,GUO D,YANG D,et al.DeepSeek-R1:In-centivizingreasoning capability in LLMs via reinforcement learning[J].arXiv:2501.12948,2025.
[4]YANG A,YANG B,HUI B,et al.Qwen2technical report[J].arXiv:2407.10671,2024.
[5]YANG A,LI A,YANG B,et al.Qwen technical report[J].ar-Xiv:2505.09388,2025.
[6]COLOMBO P,PIRES T,BOUDIAF M,et al.SaulLM-7B:Apioneeringlarge language model for law[J].arXiv:2403.03883,2024.
[7]CUI J,NING M,LI Z,et al.Chatlaw:Amulti-agent collaborative legal assistant with knowledge graph enhanced mixture-of-experts large language model[J].arXiv:2306.16092,2023.
[8]LEE J S.Evaluating generative patent language models[J].World Patent Information,2023,72:102173.
[9]NI S,TAN M,BAI Y,et al.MoZIP:A multilingual benchmark to evaluate large language models in intellectual property[J].arXiv:2402.16389,2024.
[10]LEE J S.Instruct Patent GPT:training patent language models to follow instructions with human feedback[J].Artificial Intelligence and Law,2024,33:739-782.
[11]BAI Z,ZHANG R,CHEN L,et al.Patent GPT:a large language model for intellectual property[J].arXiv:2404.18255,2024.
[12]LV K,YANG Y,LIU T,et al.Full Parameter Fine-tuning for Large Language Models with Limited Resources[C]//Procee-dings of the 62nd Annual Meeting of the Association for Computational Linguistics.2024:8187-8198.
[13]HAN Z,GAO C,LIU J,et al.Parameter-efficient fine-tuning for large models:A Comprehensive Survey[J].arXiv:2403.14608,2024.
[14]HU E,SHEN Y,WALLIS P,et al.LoRA:Low-rank adaptation of large language models[C]//ICLR.2022.
[15]DETTMERS T,PAGNONI A,HOLTZMAN A,et al.QLoRA:Efficientfinetuning of quantized LLMs[J].Advances in Neural Information Processing Systems,2023,36:10088-10115.
[16]LAMY-POIRIER J.Layered gradient accumulation and modular pipeline parallelism:fast and efficient training of large language models[J].arXiv:2106.02679,2021.
[17]JOHNSON O,XINGYING C,KHAW K,et al.Ps-CALR:Perio-dic-shift cosine annealing learning rate for deep neural networks[J].IEEE Access,2023,11:139171-139186,.
[18]DONG Q,LI L,DAI D,Asurvey on in-context learning[J].ar-Xiv:2301.00234,2024.
[19]GUNASERKAR S,ZHANG Y,ANEJA J,et al.Textbooksare all you need[J].arXiv:2306.11644,2023.
[20]LZACARD G,LEWIS P,LOMELI M,et al.Atlas:Few-shotlearning with retrieval augmented language models[J].Journal of Machine Learning Research,2023,24(251):1-43.
[21]WEI J,WANG X,SCHUURMANS D,et al.Chain-of-thoughtprompting elicits reasoning in large language models[C]//NeurlPS 2022.2022
[22]FEI Z,SGEN X,ZHU D,et al.Lawbench:Benchmarking legal knowledge of large language models[J].arXiv:2309.16289,2023.
[23]DAI Y,FENG D,HUANG J,et al.Laiw:A chinese legal large language models benchmark[J].arXiv:2310.05620,2023.
[24]LI H,CHEN Y,AI Q,et al.LexEval:Acomprehensive Chinese legal benchmark for evaluating large language models[J].arXiv:2409.20288,2024.
[25]CHALKIDIS I,FERGADIOTIS M,PRODROMOS M P,et al.LEGAL-BERT:Themuppets straight out of Law School[C]//Findings of the Association for Computational Linguistics(EMNLP2020).2020:2898-2904.
[26]ZHENG L,GUHA N,ANDERSON B,et al.When does pre-training help? Assessing self-supervised learning for law and the CaseHOLD dataset of 53 000＋Legal Holdings[C]//Procee-dings of the Eighteenth International Conference on Artificial Intelligence and Law(ICAIL’21).2021:159-168.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市两江新区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发