Computer Science ›› 2024, Vol. 51 ›› Issue (6A): 230700206-8.doi: 10.11896/jsjkx.230700206

• Artificial Intelligenc • Previous Articles     Next Articles

Bidirectional Neural Probabilistic Transducer for Process Text Entity Recognition

LI Ruiting, WANG Peiyan, WANG Libang, YANG Danqingxin   

  1. School of Computer Science,Shenyang Aerospace University,Shenyang 110136,China
  • Published:2024-06-06
  • About author:LI Ruiting,born in 1999,postgraduate,is a student member of CCF(No.O9179G).Her main research interests include artificial intelligence and know-ledge engineering.
    WANG Peiyan,born in 1983,senior engineer,is a member of CCF(No.33066M).His main research interests include naturallanguage processing,machine learning and knowledge enginee-ring.
  • Supported by:
    Applied Basic Research Program of Liaoning Province(2022JH2/101300248),Research Programs of China National Committee for Terminology in Science and Technology(YB2022015) and National Natural Science Foundation of China(U1908216).

Abstract: Process text entity recognition aims to recognize entities such as parts,materials,attributes and attribute values from texts generated or associated with the manufacturing process of products.Recently,in most domain-specific entity recognition tasks,such as process domain,prior knowledge in the form of dictionaries or rules is used to adjust neural network model results or generate pre-recognized features to incorporate into the model.However,these methods do not realize the integration of domain entity recognition knowledge and neural network models.Furthermore,the addition of domain knowledge does not reduce the training cost of the model and still need a large amount of labeled data.To address these challenges,this paper proposes a bidirectional neural probabilistic transducer(Bi-NPT) for process text entity recognition.This approach models the domain-specific prior knowledge for process text entity recognition as regular rules,and then converts these rules into a parameterized probabilistic finite state transducer.This method makes the model carry entity recognition prior knowledge before training,while being traina-ble.The model acquires the ability to recognize entities not covered by the regular rules by training on labeled data.Experimental results demonstrate that the proposed Bi-NPT performs comparably to regular rule-based entity recognition without training,suggesting that the untrained initial model already has possess entity recognition knowledge.Additionally,Bi-NPT outperforms other methods such as PER,Template-based BART,NNShot in few-shot and BiLSTM,TENER in rich-resource scenarios.

Key words: Process text, Entity recognition, Regular rules, Probabilistic finite state transducer

CLC Number: 

  • TP391
[1]ZHANG N N,WANG P Y,ZHANG G P.Named Entity DeepLearning Recognition Method for Process Operation Description Text[J].Computer Applications and Software,2019,36(11):188-195,261.
[2]JIA M,WANG P Y,ZHANG G P,et al.Named Entity Recongnition for Process Text[J].Journal of Chinese Information Processing,2022,36(3):54-63.
[3]SIAMI-NAMINI S,TAVAKOLI N,NAMIN A S.The perfor-mance of LSTM and BiLSTM in forecasting time series[C]//2019 IEEE International Conference on Big Data(Big Data).IEEE,2019:3285-3292.
[4]YAN H,DENG B C,LI X N,et al.TENER:adapting transfor-mer encoder for named entity recognition[J].arXiv:1911.04474,2019.
[5]LI X N,YAN H,QIU X P,et al.FLAT:Chinese NER using flat-lattice transformer[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:6836-6842.
[6]MA J H,WANG L Q,YAO S.Named Entity Recognition for Chemical Resource Text[J].Journal of Zhengzhou University(Natural Science Edition),2018,50(4):14-20.
[7]FENG Y T,ZHANG H J,HAO W N.Named Entity Recognition for Military Text[J].Journal of Computer Science,2015,42(7):15-18.
[8]ZHU S L.Deep Learing Based Uyghur Named Entities Recognition[J].Journal of Computer Engineering And Design,2019,40(10):2874-2878,2890.
[9]CUI L,WU Y,LIU J,et al.Template-Based Named Entity Re-cognition Using BART[C]//Findings of the Association for Computational Linguistics:ACL(IJCNLP 2021).2021:1835-1845.
[10]YANG Y,KATIYAR A.Simple and Effective Few-Shot Named Entity Recognition with Structured Nearest Neighbor Learning[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing(EMNLP).2020:6365-6375.
[11]LIU H B,ZHANG D M,XIONG S F,et al.Named Entity Re-cognition of Wheat Diseases and Pests fusing ALBERT and Rules[J].Journal of Frontiers of Computer Science and Technology,2023,17(6):1395-1404.
[12]CUI S G,CHEN J Y,LI X H.Named Entity Recognition for Chinese Electronic Medical Record by Fusing Semantic and Boundary Information[J].Journal of University of Electronic Science and Technology of China,2022,51(4):565-571.
[13]ZHANG Z W,CHEN J Y,GAO K N,et al.SVR-BIGRU-CRFBased Chinese Named Entity Recognition for Education Domain[J].Journal of Chinese Information Processing,2022,36(7):114-122.
[14]LIU W P,ZHANG B,CHEN W R,et al.Military Named Entity Recognition Based on Transfer Representation Learning[J].Command Information System and Technology,2020,62(2):68-73.
[15]PENG H,SCHWARTZ R,THOMSON S,et al.Rational Recurrences[J].arXiv:1808.09357,2018.
[16]LIN C,ZHU H,GORMLEY M R,et al.Neural finite-statetransducers:Beyond rational relations[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Volume 1(Long and Short Papers).2019:272-283.
[17]RASTOGI P,COTTERELL R,EISNER J.Weighting Finite-State Transductions With Neural Context[C]//Proceedings of NAACL-HLT.2016:623-633.
[18]JIANG C Y,ZHAO Y G,CHU S B,et al.Cold-start and interpretability:Turning regular expressions into trainable recurrent neural networks[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing(EMNLP).2020:3193-3207.
[19]JIANG C Y,JIN Z J,TU K W.Neuralizing Regular Expressions for Slot Filling[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing.2021:9481-9498.
[20]SAKUMA Y,MINAMIDE Y,VORONKOV A.Translatingregular expression matching into transducers[J].Journal of Applied Logic,2012,10(1):32-51.
[21]THOMPSON K.Programming techniques:Regular expressionsearch algorithm[J].Communications of the ACM,1968,11(6):419-422.
[22]HOPCROFT J.An n log n algorithm for minimizing states in a finite automaton[M]//Theory of machines and computations.Academic Press,1971:189-196.
[23]FRANCISCO C,COLIN D.Computational complexity of pro-blems on probabilistic grammars and transducers[C]//International Colloquium on Grammatical Inference,2000:15-24.
[24]COHEN J.A coefficient of agreement for nominal scales[J].Educational and Psychological Measurement,1960,20(1):37-46.
[25]LIU A T,XIAO W,ZHU H,et al.QaNER:Prompting question answering models for few-shot named entity recognition[J].arXiv:2203.01543,2022.
[26]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[J].arXiv:1706.03762,2017.
[1] YIN Baosheng, ZHOU Peng. Chinese Medical Named Entity Recognition with Label Knowledge [J]. Computer Science, 2024, 51(6A): 230500203-7.
[2] LAI Xin, LI Sining, LIANG Changsheng, ZHANG Hengyan. Ontology-driven Study on Information Structuring of Aeronautical Information Tables [J]. Computer Science, 2024, 51(6A): 230800150-7.
[3] YU Bihui, TAN Shuyue, WEI Jingxuan, SUN Linzhuang, BU Liping, ZHAO Yiman. Vision-enhanced Multimodal Named Entity Recognition Based on Contrastive Learning [J]. Computer Science, 2024, 51(6): 198-205.
[4] LIAO Meng, JIA Zhen, LI Tianrui. Chinese Named Entity Recognition Based on Label Information Fusion and Multi-task Learning [J]. Computer Science, 2024, 51(3): 198-204.
[5] LUO Yuanyuan, YANG Chunming, LI Bo, ZHANG Hui, ZHAO Xujian. Chinese Medical Named Entity Recognition Method Incorporating Machine ReadingComprehension [J]. Computer Science, 2023, 50(9): 287-294.
[6] GAO Xiang, WANG Shi, ZHU Junwu, LIANG Mingxuan, LI Yang, JIAO Zhixiang. Overview of Named Entity Recognition Tasks [J]. Computer Science, 2023, 50(6A): 220200119-8.
[7] GAO Xiang, TANG Jiqiang, ZHU Junwu, LIANG Mingxuan, LI Yang. Study on Named Entity Recognition Method Based on Knowledge Graph Enhancement [J]. Computer Science, 2023, 50(6A): 220700153-6.
[8] HUANG Jiange, JIA Zhen, ZHANG Fan, LI Tianrui. Chinese Medical Named Entity Recognition Based on Multi-feature Embedding [J]. Computer Science, 2023, 50(6): 243-250.
[9] LIU Pan, GUO Yanming, LEI Jun, LAO Mingrui, LI Guohui. Study on Chinese Named Entity Extraction Rules Based on Boundary Location and Correction [J]. Computer Science, 2023, 50(3): 276-281.
[10] ZHANG Xianggang, LYU Yunlian, ZENG Jing, ZHANG Ting. Gait Recognition Based on Inertial Sensor [J]. Computer Science, 2023, 50(2): 254-266.
[11] QIAN Taiyu, CHEN Yifei, PANG Bowen. Audit Text Named Entity Recognition Based on MacBERT and Adversarial Training [J]. Computer Science, 2023, 50(11A): 230200083-6.
[12] DING Hongxin, ZOU Peinie, ZHAO Junfeng, WANG Yasha. Active Learning-based Text Entity and Relation Joint Extraction Method [J]. Computer Science, 2023, 50(10): 126-134.
[13] ZHANG Rujia, DAI Lu, GUO Peng, WANG Bang. Chinese Nested Named Entity Recognition Algorithm Based on Segmentation Attention andBoundary-aware [J]. Computer Science, 2023, 50(1): 213-220.
[14] DU Xiao-ming, YUAN Qing-bo, YANG Fan, YAO Yi, JIANG Xiang. Construction of Named Entity Recognition Corpus in Field of Military Command and Control Support [J]. Computer Science, 2022, 49(6A): 133-139.
[15] WEI Ru-ming, CHEN Ruo-yu, LI Han, LIU Xu-hong. Analysis of Technology Trends Based on Deep Learning and Text Measurement [J]. Computer Science, 2022, 49(11A): 211100119-6.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!