计算机科学

计算机科学 ›› 2024, Vol. 51 ›› Issue (6A): 230700206-8.doi: 10.11896/jsjkx.230700206

• 人工智能 • 上一篇    下一篇

面向工艺实体识别的双向神经概率转换器

李瑞婷, 王裴岩, 王立帮, 杨丹清忻   

  1. 沈阳航空航天大学计算机学院 沈阳 110136
  • 发布日期:2024-06-06
  • 通讯作者: 王裴岩(wangpy@sau.edu.cn)
  • 作者简介:(1473326715@qq.com)
  • 基金资助:
    辽宁省应用基础研究计划(2022JH2/101300248);全国科技名词审定委员会科研项目(YB2022015);国家自然科学基金(U1908216)

Bidirectional Neural Probabilistic Transducer for Process Text Entity Recognition

LI Ruiting, WANG Peiyan, WANG Libang, YANG Danqingxin   

  1. School of Computer Science,Shenyang Aerospace University,Shenyang 110136,China
  • Published:2024-06-06
  • About author:LI Ruiting,born in 1999,postgraduate,is a student member of CCF(No.O9179G).Her main research interests include artificial intelligence and know-ledge engineering.
    WANG Peiyan,born in 1983,senior engineer,is a member of CCF(No.33066M).His main research interests include naturallanguage processing,machine learning and knowledge enginee-ring.
  • Supported by:
    Applied Basic Research Program of Liaoning Province(2022JH2/101300248),Research Programs of China National Committee for Terminology in Science and Technology(YB2022015) and National Natural Science Foundation of China(U1908216).

PDF (PC)

239

摘要： 工艺实体识别旨在识别出产品制造中所遵照或是产生的文本中蕴含的零件、材料、属性和属性值等实体。目前,工艺等领域实体识别大多加入词典或正则规则等领域实体先验知识,修正神经网络模型识别结果或是生成预识别特征加入模型中。但上述方法未能实现领域实体识别的先验知识与神经网络模型统一建模,领域知识的加入没有减小模型训练代价,仍需大量标注数据。为解决上述问题,提出了面向工艺实体识别的双向神经概率转换器(Bi-NPT),将工艺实体识别先验知识建模为正则规则,然后将正则规则转化为参数化的概率有限状态转换器,使得模型在训练前带有实体识别的先验知识,同时具有可训练性。通过在标注数据上的训练,模型能够习得正则规则未覆盖实体的识别能力。实验结果表明,提出的Bi-NPT在未训练的情况下与正则规则实体识别效果相当,这表明未经过训练的初始模型即携带了实体识别知识。在小样本条件下,Bi-NPT优于PER,Template-based BART和NNShot方法;在充足样本条件下,Bi-NPT优于BiLSTM与TENER等方法。

关键词: 工艺文本, 实体识别, 正则规则, 概率有限状态转换器

Abstract: Process text entity recognition aims to recognize entities such as parts,materials,attributes and attribute values from texts generated or associated with the manufacturing process of products.Recently,in most domain-specific entity recognition tasks,such as process domain,prior knowledge in the form of dictionaries or rules is used to adjust neural network model results or generate pre-recognized features to incorporate into the model.However,these methods do not realize the integration of domain entity recognition knowledge and neural network models.Furthermore,the addition of domain knowledge does not reduce the training cost of the model and still need a large amount of labeled data.To address these challenges,this paper proposes a bidirectional neural probabilistic transducer(Bi-NPT) for process text entity recognition.This approach models the domain-specific prior knowledge for process text entity recognition as regular rules,and then converts these rules into a parameterized probabilistic finite state transducer.This method makes the model carry entity recognition prior knowledge before training,while being traina-ble.The model acquires the ability to recognize entities not covered by the regular rules by training on labeled data.Experimental results demonstrate that the proposed Bi-NPT performs comparably to regular rule-based entity recognition without training,suggesting that the untrained initial model already has possess entity recognition knowledge.Additionally,Bi-NPT outperforms other methods such as PER,Template-based BART,NNShot in few-shot and BiLSTM,TENER in rich-resource scenarios.

Key words: Process text, Entity recognition, Regular rules, Probabilistic finite state transducer

中图分类号: 

  • TP391
[1]ZHANG N N,WANG P Y,ZHANG G P.Named Entity DeepLearning Recognition Method for Process Operation Description Text[J].Computer Applications and Software,2019,36(11):188-195,261.
[2]JIA M,WANG P Y,ZHANG G P,et al.Named Entity Recongnition for Process Text[J].Journal of Chinese Information Processing,2022,36(3):54-63.
[3]SIAMI-NAMINI S,TAVAKOLI N,NAMIN A S.The perfor-mance of LSTM and BiLSTM in forecasting time series[C]//2019 IEEE International Conference on Big Data(Big Data).IEEE,2019:3285-3292.
[4]YAN H,DENG B C,LI X N,et al.TENER:adapting transfor-mer encoder for named entity recognition[J].arXiv:1911.04474,2019.
[5]LI X N,YAN H,QIU X P,et al.FLAT:Chinese NER using flat-lattice transformer[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:6836-6842.
[6]MA J H,WANG L Q,YAO S.Named Entity Recognition for Chemical Resource Text[J].Journal of Zhengzhou University(Natural Science Edition),2018,50(4):14-20.
[7]FENG Y T,ZHANG H J,HAO W N.Named Entity Recognition for Military Text[J].Journal of Computer Science,2015,42(7):15-18.
[8]ZHU S L.Deep Learing Based Uyghur Named Entities Recognition[J].Journal of Computer Engineering And Design,2019,40(10):2874-2878,2890.
[9]CUI L,WU Y,LIU J,et al.Template-Based Named Entity Re-cognition Using BART[C]//Findings of the Association for Computational Linguistics:ACL(IJCNLP 2021).2021:1835-1845.
[10]YANG Y,KATIYAR A.Simple and Effective Few-Shot Named Entity Recognition with Structured Nearest Neighbor Learning[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing(EMNLP).2020:6365-6375.
[11]LIU H B,ZHANG D M,XIONG S F,et al.Named Entity Re-cognition of Wheat Diseases and Pests fusing ALBERT and Rules[J].Journal of Frontiers of Computer Science and Technology,2023,17(6):1395-1404.
[12]CUI S G,CHEN J Y,LI X H.Named Entity Recognition for Chinese Electronic Medical Record by Fusing Semantic and Boundary Information[J].Journal of University of Electronic Science and Technology of China,2022,51(4):565-571.
[13]ZHANG Z W,CHEN J Y,GAO K N,et al.SVR-BIGRU-CRFBased Chinese Named Entity Recognition for Education Domain[J].Journal of Chinese Information Processing,2022,36(7):114-122.
[14]LIU W P,ZHANG B,CHEN W R,et al.Military Named Entity Recognition Based on Transfer Representation Learning[J].Command Information System and Technology,2020,62(2):68-73.
[15]PENG H,SCHWARTZ R,THOMSON S,et al.Rational Recurrences[J].arXiv:1808.09357,2018.
[16]LIN C,ZHU H,GORMLEY M R,et al.Neural finite-statetransducers:Beyond rational relations[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Volume 1(Long and Short Papers).2019:272-283.
[17]RASTOGI P,COTTERELL R,EISNER J.Weighting Finite-State Transductions With Neural Context[C]//Proceedings of NAACL-HLT.2016:623-633.
[18]JIANG C Y,ZHAO Y G,CHU S B,et al.Cold-start and interpretability:Turning regular expressions into trainable recurrent neural networks[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing(EMNLP).2020:3193-3207.
[19]JIANG C Y,JIN Z J,TU K W.Neuralizing Regular Expressions for Slot Filling[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing.2021:9481-9498.
[20]SAKUMA Y,MINAMIDE Y,VORONKOV A.Translatingregular expression matching into transducers[J].Journal of Applied Logic,2012,10(1):32-51.
[21]THOMPSON K.Programming techniques:Regular expressionsearch algorithm[J].Communications of the ACM,1968,11(6):419-422.
[22]HOPCROFT J.An n log n algorithm for minimizing states in a finite automaton[M]//Theory of machines and computations.Academic Press,1971:189-196.
[23]FRANCISCO C,COLIN D.Computational complexity of pro-blems on probabilistic grammars and transducers[C]//International Colloquium on Grammatical Inference,2000:15-24.
[24]COHEN J.A coefficient of agreement for nominal scales[J].Educational and Psychological Measurement,1960,20(1):37-46.
[25]LIU A T,XIAO W,ZHU H,et al.QaNER:Prompting question answering models for few-shot named entity recognition[J].arXiv:2203.01543,2022.
[26]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[J].arXiv:1706.03762,2017.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发