Computer Science ›› 2023, Vol. 50 ›› Issue (5): 262-269.doi: 10.11896/jsjkx.220400126

• Artificial Intelligence • Previous Articles     Next Articles

Convolutional Network Entity Missing Detection Method Combined with Gated Mechanism

YE Han, LI Xin, SUN Haichun   

  1. School of Information and Cyber Security,People's Public Security University of China,Beijing 102623,China
  • Received:2022-04-12 Revised:2022-09-08 Online:2023-05-15 Published:2023-05-06
  • About author:YE Han,born in 1999,postgraduate.His main research interests include na-tural language processing and deep learning.
    LI Xin,born in 1977,Ph.D,associate professor.His main research interests include big data processing and information communication.
  • Supported by:
    Ministry of Public Security Technology Research Program(2020JSYJC22,2021JSZ09).

Abstract: The adequacy of the entity information directly affects the applications that depend on textual entity information,while conventional entity recognition models can only identify the existing entities.The task of the entity missing detection,defined as a sequence labeling task,aims at finding the location where the entity is missing.In order to construct training dataset,three corres-ponding methods are proposed.We introduce an entity missing detection method combining the convolutional neural network with the gated mechanism and the pre-trained language model.Experiments show that the F1 scores of this model are 80.45% for the PER entity,83.02% for the ORG entity,and 86.75% for the LOC entity.The model performance exceeds the other LSTM-based named entity recognition model.It is found that there is a correlation between the accuracy of the model and the word frequency of the annotated characters.

Key words: Gated mechanism, Abnormal detection, Pre-trained language model, Convolutional neural network

CLC Number: 

  • TP391
[1]FAN M,FENG C,GUO L,et al.Product-Aware HelpfulnessPrediction of Online Reviews[C]//The World Wide Web Conference(WWW '19).ACM Press,2019:2715-2721.
[2]YANG Y,CHEN C,BAO F S.Aspect-Based Helpfulness Prediction for Online Product Reviews[C]//2016 IEEE 28th International Conference on Tools with Artificial Intelligence(IC-TAI).IEEE,2016:836-843.
[3]ALIKANIOTIS D,YANNAKOUDAKIS H,REI M.Automatic Text Scoring Using Neural Networks[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics(Volume 1:Long Papers).Association for Computa-tional Linguistics,2016:715-725.
[4]TAY Y,PHAN M C,TUAN L A,et al.SkipFlow:Incorporating Neural Coherence Features for End-to-End Automatic Text Scoring[C]//Thirty-Second AAAI Conference on Artificial Intelligence.Association for the Advancement of Artificial Intelligence,2018:5948-5955.
[5]SUN F,ZHANG J.Research on Grammar Checking SystemUsing Computer Big Data and Convolutional Neural Network Constructing Classification Model[J].Journal of Physics:Conference Series,2021,1952(4):042097,1-9.
[6]HAO S,HAO G.A Research on Online Grammar Checker Sys-tem Based on Neural Network Model[J].Journal of Physics:Conference Series,2020,1651(1):012135,1-8.
[7]LAMPLE G,BALLESTEROS M,SUBRAMANIAN S,et al.Neural Architectures for Named Entity Recognition[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Association for Computational Linguistics,2016:260-270.
[8]LIU Y,GOPALAKRISHNAN V.An Overview and Evaluation of Recent Machine Learning Imputation Methods Using Cardiac Imaging Data[J].Data,2017,2(1):8,1-15.
[9]BIESSMANN F,RUKAT T,SCHMIDT P,et al.DataWig:Missing Value Imputation for Tables[J].Journal of Machine Learning Research,2019,20(175):1-6.
[10]LI F,GUI Z,WU H,et al.Big enterprise registration data imputation:Supporting spatiotemporal analysis of industries in China[J].Computers,Environment and Urban Systems,2018,70:9-23.
[11]GRAVES A,MOHAMED A R,HINTON G.Speech Recogni-tion with Deep Recurrent Neural Networks[C]//2013 IEEE International Conference on Acoustics,Speech and Signal Proces-sing.IEEE,2013:6645-6649.
[12]MA X,HOVY E.End-to-end Sequence Labeling via Bi-direc-tional LSTM-CNNs-CRF[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics(Vo-lume 1:Long Papers).Association for Computational Linguistics,2016:1064-1074.
[13]DEVLIN J,CHANG M W,LEE K,et al.BERT:Pre-training of Deep Bidirectional Transformers for Language Understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies,Volume 1(Long and Short Papers).2019:4171-4186.
[14]LI X,ZHANG H,ZHOU X H.Chinese clinical named entityrecognition with variant neural structures based on BERT me-thods[J].Journal of Biomedical Informatics,2020,107:103422,1-7.
[15]LIU S,YANG H,LI J,et al.Chinese Named Entity Recognition Method in History and Culture Field Based on BERT[J].International Journal of Computational Intelligence Systems,2021,14(1):163.
[16]FU J,LIU P,ZHANG Q.Rethinking Generalization of Neural Models:A Named Entity Recognition Case Study[C]//Procee-dings of the AAAI Conference on Artificial Intelligence.2020:7732-7739.
[17]AGARWAL O,YANG Y,WALLACE B C,et al.Interpretability Analysis for Named Entity Recognition to Understand System Predictions and How They Can Improve[J].Computational Linguistics,2021,47(1):117-140.
[18]CHEN H,HE B.Automated Essay Scoring by Maximizing Human-Machine Agreement[C]//Proceedings of the 2013 Confe-rence on Empirical Methods in Natural Language Processing.Association for Computational Linguistics,2013:1741-1752.
[19]TAGHIPOUR K,NG H T.A Neural Approach to Automated Essay Scoring[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.Association for Computational Linguistics,2016:1882-1891.
[20]KUMARASWAMY R,WAZALWAR A,KHOT T,et al.Anomaly Detection in Text:The Value of Domain Knowledge[C]//Proceedings of the Twenty-Eighth International Florida Artificial Intelligence Research Society Conference.Association for the Advancement of Artificial Intelligence,2015:225-228.
[21]CICHOSZ P.Unsupervised modeling anomaly detection in discussion forums posts using global vectors for text representation[J].Natural Language Engineering,2020,26(5):551-578.
[22]RUFF L,ZEMLYANSKIY Y,VANDERMEULEN R,et al.Self-Attentive,Multi-Context One-Class Classification for Unsupervised Anomaly Detection on Text[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.Association for Computational Linguistics,2019:4061-4071.
[23]RUFF L,KAUFFMANN J R,VANDERMEULEN R A,et al.A Unifying Review of Deep and Shallow Anomaly Detection[J].Proceedings of the IEEE,2021,109(5):756-795.
[24]DAUPHIN Y N,FAN A,AULI M,et al.Language Modelingwith Gated Convolutional Networks[C]//Proceedings of the 34th International Conference on Machine Learning.2017:933-941.
[25]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isAll You Need[C]//The 31st International Conference on Neural Information Processing Systems.Curran Associates Inc.,2017:6000-6010.
[26]LEVOW G A.The Third International Chinese Language Processing Bakeoff:Word Segmentation and Named Entity Recognition[C]//Proceedings of the 5th SIGHAN Workshop on Chinese Language Processing.Association for Computational Linguistics.2006:108-117.
[27]HE K,ZHANG X,REN S,et al.Deep Residual Learning forImage Recognition[C]//the IEEE Conference on Computer Vision and Pattern Recognition.IEEE,2016:770-778.
[28]KINGMA D P,BA J.Adam:A Method for Stochastic Optimization[C]//3rd International Conference on Learning Representations.2015.
[1] LI Han, HOU Shoulu, TONG Qiang, CHEN Tongtong, YANG Qimin, LIU Xiulei. Entity Relation Extraction Method in Weapon Field Based on DCNN and GLU [J]. Computer Science, 2023, 50(6A): 220200112-7.
[2] HUANG Yujiao, CHEN Mingkai, ZHENG Yuan, FAN Xinggang, XIAO Jie, LONG Haixia. Text Classification Based on Weakened Graph Convolutional Networks [J]. Computer Science, 2023, 50(6A): 220700039-5.
[3] LUO Ruiqi, YAN Jinlin, HU Xinrong, DING Lei. EEG Emotion Recognition Based on Multiple Directed Weighted Graph and ConvolutionalNeural Network [J]. Computer Science, 2023, 50(6A): 220600128-8.
[4] LUO Huilan, LONG Jun, LIANG Miaomiao. Attentional Feature Fusion Approach for Siamese Network Based Object Tracking [J]. Computer Science, 2023, 50(6A): 220300237-9.
[5] XIONG Haojie, WEI Yi. Study on Multibeam Sonar Elevation Data Prediction Based on Improved CNN-BP [J]. Computer Science, 2023, 50(6A): 220100161-4.
[6] XU Changqian, WANG Dong, SU Feng, ZHANG Jun, BIAN Haifeng, LI Long. Image Recognition Method of Transmission Line Safety Risk Assessment Based on MultidimensionalData Coupling [J]. Computer Science, 2023, 50(6A): 220500032-6.
[7] WANG Jinwei, ZENG Kehui, ZHANG Jiawei, LUO Xiangyang, MA Bin. GAN-generated Face Detection Based on Space-Frequency Convolutional Neural Network [J]. Computer Science, 2023, 50(6): 216-224.
[8] ZHANG Xue, ZHAO Hui. Sentiment Analysis Based on Multi-event Semantic Enhancement [J]. Computer Science, 2023, 50(5): 238-247.
[9] WANG Lin, MENG Zuqiang, YANG Lina. Chinese Sentiment Analysis Based on CNN-BiLSTM Model of Multi-level and Multi-scale Feature Extraction [J]. Computer Science, 2023, 50(5): 248-254.
[10] CHANG Liwei, LIU Xiujuan, QIAN Yuhua, GENG Haijun, LAI Yuping. Multi-source Fusion Network Security Situation Awareness Model Based on Convolutional Neural Network [J]. Computer Science, 2023, 50(5): 382-389.
[11] SHAO Yunfei, SONG You, WANG Baohui. Study on Degree of Node Based Personalized Propagation of Neural Predictions forSocial Networks [J]. Computer Science, 2023, 50(4): 16-21.
[12] CAO Chenyang, YANG Xiaodong, DUAN Pengsong. WiDoor:Close-range Contactless Human Identification Approach [J]. Computer Science, 2023, 50(4): 388-396.
[13] WANG Xiaofei, FAN Xueqiang, LI Zhangwei. Improving RNA Base Interactions Prediction Based on Transfer Learning and Multi-view Feature Fusion [J]. Computer Science, 2023, 50(3): 164-172.
[14] MEI Pengcheng, YANG Jibin, ZHANG Qiang, HUANG Xiang. Sound Event Joint Estimation Method Based on Three-dimension Convolution [J]. Computer Science, 2023, 50(3): 191-198.
[15] HUAN Zhigang, JIANG Guoquan, ZHANG Yujian, LIU Liu, LIU Shanshan. Employing Gated Mechanism to Incorporate Multi-features into Chinese Event Coreference Resolution [J]. Computer Science, 2023, 50(3): 291-297.
Full text



No Suggested Reading articles found!