计算机科学

计算机科学 ›› 2025, Vol. 52 ›› Issue (2): 253-260.doi: 10.11896/jsjkx.231200054

• 人工智能 • 上一篇    下一篇

基于大小语言模型协同增强的中文电子病历依存句法分析

许思遥1, 曾健骏2, 张维彦2, 叶琪2, 朱焱1   

  1. 1 华东理工大学数学学院 上海 200237
    2 华东理工大学信息科学与工程学院 上海 200237
  • 收稿日期:2023-12-07 修回日期:2024-04-28 出版日期:2025-02-15 发布日期:2025-02-17
  • 通讯作者: 朱焱(zhuygraph@ecust.edu.cn)
  • 作者简介:(18817535379@163.com)
  • 基金资助:
    上海市促进产业高质量发展专项资金(2021-GZL-RGZN-01018)

Dependency Parsing for Chinese Electronic Medical Record Enhanced by Dual-scale Collaboration of Large and Small Language Models

XU Siyao1, ZENG Jianjun2, ZHANG Weiyan2, YE Qi2, ZHU Yan1   

  1. 1 School of Mathematics,East China University of Science and Technology,Shanghai 200237,China
    2 School of Information Science and Engineering,East China University of Science and Technology,Shanghai 200237,China
  • Received:2023-12-07 Revised:2024-04-28 Online:2025-02-15 Published:2025-02-17
  • About author:XU Siyao,born in 2000,postgraduate.Her main research interests include na-tural language processing and depen-dency parsing.
    ZHU Yan,born in 1984,Ph.D,associate professor.Her main research interest is graph theory and its applications.
  • Supported by:
    Shanghai Municipal Special Fund for Promoting High-quality Development of Industries (2021-GZL-RGZN-01018).

PDF (PC)

181

摘要： 依存句法分析是一项重要的自然语言处理任务,其目标是识别句子中词与词之间的依存关系。但在面向中文医疗电子病历的依存句法分析中,现有的研究存在以下问题:当出现缺省指示语法结构的成分和修饰成分位置多样的情况时,当前的通用解析器无法准确分析。针对该问题,提出基于大小语言模型协同增强的中文电子病历依存句法分析方法。首先,分析中文电子病历的语言特征,提出通过成分补全指示医疗文本中的特殊语法结构。然后,利用通用解析器进行依存句法分析,对于解析后的语法图,利用大语言模型的先验语法知识进行自动修正。此外,所提方法将重点放在缩小医疗文本与通用文本之间的特征分布差异上,故不受医疗领域缺少标注数据的限制。针对中文电子病历的依存句法分析,标注了444条测试样本,并对所提方法进行验证。实验表明该方法能有效地对中文电子病历进行依存分析,基于少量标注语料,LAS指标可达92.42,UAS指标可达94.60,并且在不同科室的中文电子病历上也能够达到同样显著的效果。

关键词: 自然语言处理, 依存句法分析, 中文电子病历, 大语言模型, 协同增强

Abstract: Dependency parsing is a crucial task in natural language processing,aiming to identify the syntactic dependencies between words in a sentence.However,existing research on dependency parsing for Chinese electronic medical records faces follo-wing problems:current general-purpose parsers are unable to accurately analyze the situation when there is a lack of components indicative of grammatical structure and a variety of positions of modifiers.To address these issues,this paper proposes a method based on a dual-scale collaborative enhancement of large and small language models for dependency parsing of Chinese electronic medical records.Specifically,we first analyze the linguistic features of Chinese electronic medical records,and propose component completion to indicate special grammatical structures in medical texts.Subsequently,we utilize a generic parser for dependency parsing,for the parsed syntactic graph,we employ the prior grammatical knowledge of a large language model to modify it automatically.In addition,since our approach focuses on narrowing the feature distribution gap between medical and generic texts,it is not constrained by the lack of annotated data in the medical domain.This study annotates 444 samples for dependency parsing of Chinese electronic medical records,which validates our method.Experimental results demonstrate the effectiveness of our approach in parsing Chinese electronic medical records,achieving LAS and UAS metrics of 92.42 and 94.60 in the scenario with little data.The proposed method also shows significant performance in various departments.

Key words: Natural language processing, Dependency parsing, Chinese electronic medical records, Large language model, Collaborative enhancement

中图分类号: 

  • TP391
[1]EISNER.Bilexical Grammars and their Cubic-Time Parsing Algorithms [J].Springer Netherlands,2000,10(7):29-61.
[2]CHEN W L,ZHANG M,ZHANG Y.Semi-supervised FeatureTransformation for Dependency Parsing[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language.2013:1303-1313.
[3]TIMOTHY D,CHRISTOPHER M.Deep Biaffine Attention for Neural Dependency Parsing[C]//Proceedings of the 2017 International Conference on Learning Representations.2017:1-8.
[4]CHEN D Q,MANNING.A fast and accurate dependency parser using neural networks[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language.2014:740-750.
[5]WEISS D,ALBERTI C,COLLINS M.Structured training forneural network transition-based parsing.[C]//Proceedings of Annual Meeting of the Association for Computational Linguistics.2015.
[6]DYER C,BALLESTEROS M,WANG L,et al.Transition-based dependency parsing with stack long short-term memory[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language.2015.
[7]KIPERWASSER E,GOLDBERG Y.Simple and accurate de-pendency parsing using bidirectional LSTM feature representations[C]//Proceedings of Annual Meeting of the Association for Computational Linguistics.2016:313-327.
[8]DOZAT T,MANNING C.Deep biaffine attention for neural dependency parsing[J]. arXiv:1611.01734,2016.
[9]MRINI K,DERNONCOURT F.Rethinking self-attention:To-wards interpretability in neural parsing [C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language.2020:731-742.
[10]BADA M,PYYSALO S,CIOSICI M,et al.Craft SharedTasks 2019 Overview－Integrated Structure,Semantics,and Coreference[C]//Proceedings of the 5th Workshop on BioNLP Open Shared Tasks.2019:174-184.
[11]NGO TM,KANERVA J,GINTER F,et al.Neural Dependency Parsing of Biomedical Text:TurkuNLP entry in the CRAFT structural annotation task[C]//Proceedings of the 5th Workshop on BioNLP Open Shared Tasks.2019:206-215.
[12]JANG Z P,GUAN Y.A Fusion Model for Chinese Electronic Medical Record Parsing [J].ACTA Automatica Sinica,2019,45(2):276-288.
[13]KOPF A,KILCHER Y,RUTTE D,et al.Open AssistantConversations-Democratizing Large Language Model Alignment[C]//Proceedings of the 2023 Conference and Workshop on Neural Information Processing Systems.2023:1-13.
[14]WEI J,TAY Y,RISHI B,et al.Emergent Abilities of Large Language Models [J].arXiv:2206.07682,2022.
[15]SUN X F,DONG L F.Pushing the Limits of ChatGPT on NLP Tasks[J].arXiv:2306.09719,2023.
[16]SCHICK T,SCHÜTZE H.Exploiting Cloze-questions for Few-shot Text Classification and Natural Language Inference[C]//Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics:Main Volume.2021:255-269.
[17]GUNTER T D,TERRY N P.The Emergence of National Electronic Health Record Architectures in the United States and Australia:Models,Costs,and Questions [J].Journal of Medical Internet Research,2005,7(1):e3.
[18]YEH C L,CHEN Y C.Zero Anaphora Resolution in Chinese with Shallow Parsing [J].Journal of Chinese Language and Computing,2007,17(1):41-56.
[19]JIANG M,HUANG Y,FAN J W,et al. Parsing Clinical Text:How Good Are the State-of-the-art Parsers?[J] BMC Medical Informatics and Decision Making,2015,15(S1):1-6.
[20]SHI J L,LUO X Y.Construction of a Treebank of LearnersChinese [J].Journal of Chinese Information Processing,2022,36(1):39-46.
[21]CHE W,FENG Y,QIN L,et al.N-LTP:An Open-source Neural Language Technology Platform for Chinese[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language.2021:42-49.
[22]PLANK B,ALONSO H M,AGIĆ Ž,et al.Do dependencyparsing metrics correlate with human judgments?[C]//Proceedings of the 19th Conference on Computational Natural Language Learning.2015:315-320.
[23]HAN H,CHOI J D.The Stem Cell Hypothesis:Dilemma behind Multi-Task Learning with Transformer Encoders[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language.2021:5555-5577.
[24]ZHANG S,WANG L,SUN K,et al.A practical Chinese dependency parser based on a large-scale dataset [J]. arXiv:2009.00901,2020.
[25]ZHANG Y,CUI L.Siren's Song in the AI Ocean:A Survey on Hallucination in Large Language Models [J].arXiv:2309.01219,2023.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发