计算机科学 ›› 2020, Vol. 47 ›› Issue (2): 245-250.doi: 10.11896/jsjkx.190500063
古雪梅1,刘嘉勇1,程芃森1,2,何祥1
GU Xue-mei1,LIU Jia-yong1,CHENG Peng-sen1,2,HE Xiang1
摘要: 针对推文中恶意软件名称识别任务存在的文本简短、非正式、实体类别单一以及实体歧义等问题,提出了一种基于BERT-BiLSTM-Self-attention-CRF的实体识别方法,以实现推文中恶意软件名称的自动识别。在BiLSTM-CRF模型的基础上,利用BERT模型编码单词语境信息,提升词嵌入的上下文语义质量,增强原有模型的语义消歧能力;同时,借助Self-attention机制学习单词间关系和句子结构特征,利用加权表征帮助单一类别实体的解码,以提升恶意软件名称实体的识别效果。通过构建包含恶意软件名称实体的推文标记数据集进行实验测试,结果表明,提出的方法可以实现更好的性能,其精确率、召回率、F1值分别为86.38%,84.73%,85.55%,相较于基线模型BiLSTM-CRF,F1值提升了12.61%。
中图分类号:
[1]MITTAL S,DAS P K,MULWAD V,et al.Cybertwitter:Using twitter to generate alerts for cybersecurity threats and vulnerabilities[C]∥Proceedings of the 2016 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mi-ning.IEEE Press,2016:860-867. [2]DERCZYNSKI L,MAYNARD D,RIZZO G,et al.Analysis of named entity recognition and linking for tweets[J].Information Processing & Management,2015,51(2):32-49. [3]LE N T,MALLEK F,SADAT F.Uqam-ntl:Named entity re-cognition in twitter messages[C]∥Proceedings of the 2nd Workshop on Noisy User-generated Text (WNUT).2016:197-202. [4]MAHMOOD T,MUJTABA G,SHUIB L,et al.Public bus commuter assistance through the named entity recognition of twitter feeds and intelligent route finding[J].IET Intelligent Transport Systems,2017,11(8):521-529. [5]LIU X,ZHANG S,WEI F,et al.Recognizing named entities in tweets[C]∥Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics:Human Language Technologies-Volume 1.Association for Computational Linguistics,2011:359-367. [6]RITTER A,CLARK S,ETZIONI O.Named entity recognition in tweets:an experimental study[C]∥Proceedings of the Conference on Empirical Methods in Natural Language Processing.Association for Computational Linguistics,2011:1524-1534. [7]OKUR E,DEMIR H,ÖZGÜR A.Named entity recognition on twitter for turkish using semi-supervised learning with word embeddings[J].arXiv:1810.08732,2018. [8]ZHANG Q,FU J,LIU X,et al.Adaptive co-attention network for named entity recognition in tweets[C]∥Thirty-Second AAAI Conference on Artificial Intelligence.2018. [9]LIMSOPATHAM N,COLLIER N H.Bidirectional LSTM for named entity recognition in Twitter messages[C]∥Proceedings of the 2nd Workshop on Noisy User-generated Text (WNUT).2016:197-202. [10]DEVLIN J,CHANG M W,LEE K,et al.Bert:Pre-training of deep bidirectional transformers for language understanding[J].arXiv:1810.04805,2018. [11]VASWANI A,SHAZEER N,PARMAR N,et al.Attention is all you need[C]∥Advances in Neural Information Processing Systems.2017:5998-6008. [12]SHEN T,ZHOU T,LONG G,et al.Disan:Directional self-attention network for rnn/cnn-free language understanding[C]∥Thirty-Second AAAI Conference on Artificial Intelligence.2018. [13]TAN Z,WANG M,XIE J,et al.Deep semantic role labeling with self-attention[C]∥Thirty-Second AAAI Conference on Artificial Intelligence.2018. [14]CAO P,CHEN Y,LIU K,et al.Adversarial Transfer Learning for Chinese Named Entity Recognition with Self-Attention Mechanism[C]∥Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.2018:182-192. [15]MEFTAH S,SEMMAR N.A neural network model for part-of-speech tagging of social media texts[C]∥Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC-2018).2018. [16]JANSSON P,LIU S.Distributed representation,lda topic mo-delling and deep learning for emerging named entity recognition from social media[C]∥Proceedings of the 3rd Workshop on Noisy User-generated Text.2017:154-159. [17]GUPTA D,EKBAL A,BHATTACHARYYA P.A Deep Neural Network based Approach for Entity Extraction in Code-Mixed Indian Social Media Text[C]∥Proceedings of the Ele-venth International Conference on Language Resources and Ev-aluation (LREC-2018).2018. [18]LAFFERTY J,MCCALLUM A,PEREIRA F C N.Conditional random fields:Probabilistic models for segmenting and labeling sequence data[C]∥Proceedings of International Conference on Machine Learning.2001:282-289. [19]BELAININE B,FONSECA A,SADAT F.Named entity recognition and hashtag decomposition to improve the classification of tweets[C]∥Proceedings of the 2nd Workshop on Noisy User-generated Text (WNUT).2016:102-111. [20]BOJANOWSKI P,GRAVE E,JOULIN A,et al.Enriching word vectors with subword information[J].Transactions of the Association for Computational Linguistics,2017,5:135-146. [21]PENNINGTON J,SOCHER R,MANNING C.Glove:Global vectors for word representation[C]∥Proceedings of the 2014 Conference on Empirical Methods in Natural Language Proces-sing (EMNLP).2014:1532-1543. |
[1] | 王卫红, 陈骁, 吴炜, 高星宇. 高分影像复杂背景下的城市水体自动提取方法 Method of Automatically Extracting Urban Water Bodies from High-resolution Images with Complex Background 计算机科学, 2019, 46(11): 277-283. https://doi.org/10.11896/jsjkx.181001985 |
|