计算机科学

计算机科学 ›› 2025, Vol. 52 ›› Issue (6A): 240700143-6.doi: 10.11896/jsjkx.240700143

• 人工智能 • 上一篇    下一篇

基于预训练模型和双向二维卷积的命名实体识别算法

林楠, 刘志慧, 杨聪   

  1. 郑州大学网络空间安全学院 郑州 450003
  • 出版日期:2025-06-16 发布日期:2025-06-12
  • 通讯作者: 杨聪(wangyuanyc@zzu.edu.cn)
  • 作者简介:(linnan@zzu.edu.cn)
  • 基金资助:
    河南省高等学校重点科研项目(22A520042);郑州市协同创新重大专项(20XTZ06013)

Named Entity Recognition Algorithm Based on Pre-training Model and Bidirectional TwoDimensional Convolution

LIN Nan, LIU Zhihui, YANG Cong   

  1. School of Cyberspace Security,Zhengzhou University,Zhengzhou 450003,China
  • Online:2025-06-16 Published:2025-06-12
  • About author:LIN Nan,born in 1973,professor.Her main research interests include intelligent systems and artificial intelligence.
    YANG Cong,born in 1987,assistant professor.His main research interests include deep neural networks,computer vision and medical image processing.
  • Supported by:
    Key Scientific Research Project of Henan Universities(22A520042) and Zhengzhou Collaborative Innovation Major Project(20XTZ06013).

PDF (PC)

36

摘要： 针对命名实体识别在处理嵌套结构时语义信息逐层减弱的问题,提出了一种基于预训练模型和双向二维卷积的命名实体识别算法BAM-TDNN。该算法首先通过四词嵌入策略即BERT、距离、局部和注意力嵌入,来提取语句中的不同层次语义特征,将多个层次的语义特征转换为二维语义表示,以更好地捕捉嵌套结构之间的语义信息;其次,采用Bi-TDNN模型学习语句中实体的长距离语义依赖关系,扩展跨度表示的感受野,提取嵌套实体间更准确的语义信息,更好地理解嵌套实体之间的语义关联。通过在4个公共数据集上进行评估,实验结果表明,所提出的命名实体识别算法在多个实体识别数据集上均取得了良好的性能。BAM-TDNN在ACE2005数据集上的精确率、召回率和F1值分别为86.83%,87.93% 和 86.83%,在GENIA数据集上的精确率、召回率和F1值分别为86.52%,82.37% 和 84.36%,在CoNLL2003数据集上的精确率、召回率和F1值分别为92.24%,93.72% 和 91.97%等。

关键词: 命名实体识别, 四词嵌入策略, BERT, Bi-TDNN

Abstract: A named entity recognition algorithm BAM-TDNN based on bidirectional two-dimensional convolution and pre-training model is proposed to address the problem of semantic information weakening layer by layer when processing nested structures in named entity recognition.This algorithm first uses four word embedding strategies,namely BERT,distance,locality,and attention embedding,to extract semantic features at different levels within a sentence,and converts semantic features at multiple levels into two-dimensional semantic representations,better capturing semantic information between nested structures.Secondly,the Bi TDNN model is used to learn the long-range semantic dependencies of entities in sentences,expand the receptive field of span representation,provide more accurate semantic information between nested entities,and better understand the semantic associations between nested entities.Through evaluation on four public datasets,experimental results show that the proposed named entity recognition algorithm has achieved good performance on multiple entity recognition datasets.The accuracy,recall,and F1 value of BAM-TDNN on the ACE2005 dataset is 86.83%,87.93%,and 86.83%,respectively.The accuracy,recall,and F1 value on the GENIA dataset is 86.52%,82.37%,and 84.36%,respectively.The accuracy,recall,and F1 value on the CoNLL2003 dataset is 92.24%,93.72%,and 91.97%,respectively.

Key words: Named entity recognition, Four-word embedding strategy, BERT, Bi-TDNN

中图分类号: 

  • TP391
[1]OZCELIK O,TORAMAN C.Named entity recognition in Turkish:A comparative study with detailed error analysis [J].Information Processing & Management,2022,59(6):103065.
[2]LIU X F,TAN K W,DONG S B.Multi-granularity sequential neural network for document-level biomedical relation extraction.Information Processing & Management [J].Information Processing & Management,2021,58(6):102718.
[3]XIA Y,LAN M J,LOU J Y,et al.Iterative rule-guided reasoning over sparse knowledge graphs with deep reinforcement learning [J].Information Processing & Management,2022,59(5):103040.
[4]DINARELLI M,ROSSET S.Models cascade for tree-structurednamed entity detection [C]//Proceedings of 5th International Joint Conference on Natural Language Processing.2011:1269-1278.
[5]JU M Z,MIWA M,ANANIADOU S.A neural layered model for nested named entity recognition [C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2018:1446-1459.
[6]FISHER J,VLACHOS A.Merge and label:A novel neural network architecture for nested NER [C]//Proceedings of the57th Annual Meeting of the Association for Computational Linguistics.2019:5840-5850.
[7]STRAKOVA J,STRAKA M,HAJIC J.Neural architectures for nested NER through linearization[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.2019:5326-5331.
[8]SOHRAB M G,MIWA M.Deep exhaustive model for nestednamed entity recognition [C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.2018:2843-2849.
[9]KATIYAR A,CARDIE C.Nested named entity recognition revisited[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2018:861-871.
[10]YU J T,BOHNET B,POESIO M.Named entity recognition asdependency parsing [C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:6470-6476.
[11]LI J Y,FEI H,LIU J,et al.Unified named entity recognition as word-word relation classification [C]//Proceedings of the AAAI Conference on Artificial Intelligence.2022:10965-10973.
[12]WANG J,LI D,CHEN G,et al.Pyramid:A layered model for nested named entity recognition [C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:5918-5928.
[13]YANG S L,TU K W.Bottom-up constituency parsing and nested named entity recognition with pointer networks [C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2022:2403-2416.
[14]OUCHI H,SUZUKI J,KOBAYASHI S,et al.Instance-basedlearning of span representations:A case study through named entity recognition [EB/OL].(2020-04-29)[2020-07-10].arXiv:2004.14514.
[15]SUN L,SUN Y X,JI F,et al.Joint learning of token context and span feature for span-based nested NER[C]//IEEE/ACM Transactions on Audio,Speech,and Language Processing.2020:2720-2730.
[16]LU W,ROTH D.Joint mention extraction and classificationwith mention hypergraphs [C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.2015:857-867.
[17]MUIS A O,LU W.Labeling gaps between words:Recognizing overlapping mentions with mention separators [C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing.2017:2608-2618.
[18]ZHANG Y,ZHOU H Q,LI Z H.Fast and accurate neural CRF constituency parsing [C]//Proceedings of the Twenty-ninth International Conference on International Joint Conferences on Artificial Intelligence.2021:4046-4053.
[19]FU Y,TAN C Q,CHEN M S,et al.Nested named entity recognition with partially-observed treecrfs [C]//Proceedings of the AAAI Conference on Artificial Intelligence.2021:12839-12847.
[20]LOU C,YWANG S L,TU K W.Nested named entity recognition as latent lexicalized constituency parsing [C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics.2022:6183-6198.
[21]GENG R,CHEN Y,HUANG R,et al.Planarized sentence representation for nested named entity recognition[C]//Information Processing and Management.2023,60:103352.
[22]LIU R B,WEI J,JIA C Y,et al.Modulating language modelswith emotions [C]//Findings of the Association for Computational Linguistics:ACL(IJCNLP 2021).2021:4332-4339.
[23]WALKER C,STRASSEL S,MEDERO J,et al.ACE 2005 multilingual training corpus[J].Linguistic Data Consortium,2006,57:45.
[24]KIM J,OHTA T,TATEISI Y,et al.GENIA corpus－a semantically annotated corpus for bio-textmining[J].Bioinformatics,2003,19(1):180-182.
[25]SANG E,DE M.Introduction to the CoNLL-2003 shared task:Language-independent named entity recognition [C]//Proceedings of the Seventh Conference on Natural Language Learning at HLT(NAACL 2003).2003:142-147.
[26]ZHANG Y,YANG J.Chinese NER using lattice LSTM [C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.2018:1554-1564.
[27]SHEN Y L,MA X Y,TAN Z Q,et al.Locate and label:A two-stage identifier for nested named entity recognition [EB/OL].(2021-01-01)[2024-03-01].arXiv:2105.06804.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发