Computer Science ›› 2018, Vol. 45 ›› Issue (6A): 91-96.

• Intelligent Computing • Previous Articles     Next Articles

Automatic Keyword Extraction Based on BiLSTM-CRF

CHEN Wei1,WU You-zheng2,CHEN Wen-liang1,ZHANG Min1   

  1. School of Computer Sciences and Technology,Soochow University,Suzhou,Jiangsu 215006,China1
    IQIYI Artificial Intelligence Research Group,Beijng 100080,China2
  • Online:2018-06-20 Published:2018-08-03

Abstract: Automatic keyword extraction is an important task of natural language processing (NLP),which provides technical support for personalized recommendation,online shopping and other applications.For the task,a new keyword extraction method based on bidirectional long short-term memory network and conditional random field (BiLSTM-CRF) was proposed.In the method,the extraction task is regarded as the sequence labeling problem.Firstly,the input text is represented as a low-dimensional,high-density vector.Then,a classification algorithm is used to predict the tags of the words.Finally,a CRF layer is used to decode the whole sequence to get the tagging result.Experiments were conducted on large scale real data,and the results show that this way can improve about 1% compared with the base system.

Key words: Conditional random field, Keyword extraction, Long short-term memory network, Natural language processing

CLC Number: 

  • TP391
[1]刘知远.基于文档主题结构的关键词抽取方法研究[D].北京:清华大学,2011.
[2]MARUJO L,WANG L,TRANCOSO I,et al.Automatic keyword extraction on twitter[C]∥Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Short Papers).USA:ACL,2015:637-643.
[3]GOLLAPALLI S D,LI X L,YANG P.Incorporating Expert Knowledge into Keyphrase Extraction[C]∥Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17).California:AAAI,2017:3180-3187.
[4]TURNEY P D.Learning Algorithms for Keyphrase Extraction[J].Information Retrieval,2000,2(4):303-336.
[5]WU W,ZHANG B,OSTENDORF M.Automatic generation of personalized annotation tags for twitter users[C]∥Human Language Technologies:The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics(HLT’10).USA:ACL,2010:689-692.
[6]ZHAO W X,JIANG J,HE J,et al.Topical keyphrase extraction from twitter[C]∥Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics:Human Language Technologies(HLT’11).USA:ACL,2011:379-388.
[7]BELLAACHIA A,AL-DHELAAN M.Ne-rank:A novel graph-based keyphrase extraction in twitter[C]∥The 2012 IEEE/WIC/ACM International Joint Conferences on Web Intelligence and Intelligent Agent Technology(WI-IAT’12).Washington,DC:IEEE Computer Society,2012:372-379.
[8]RILOFF E,LEHNERT W.Information extraction as a basis for high-precision text classification[J].ACM Transactions on Information Systems(TOIS),1994,12(3):296-333.
[9]WITTEN I H,PAYNTER G W,FRANK E,et al.Kea:practical automatic keyphrase extraction[C]∥4th ACM Conference on Digital Libraries(DL’99).New York:ACM,1999:254-255.
[10]MEDELYAN O,PERRONE V,WITTEN I H.Subject metadata support powered by maui[C]∥10th Annual Joint Conference on Digital Libraries(JCDL’10).New York:ACM,2010:407-408.
[11]WANG C,LI S J.Corankbayes:Bayesian learning to rank under the co-training framework and its application in keyphrase extraction[C]∥20th ACM International Conference on Information and Knowledge Management(CIKM’11).New York:ACM,2011:2241-2244.
[12]FRANK E,PAYNTER G W,WITTEN I H,et al.Domain-specific Keyphrase Extraction[C]∥Proceedings of IJCAI.California:AAAI,1999:668-673.
[13]HULTH A.Improved Automatic Keyword Extraction Given More Linguistic Knowledge[C]∥Proceedings of EMNLP.USA:ACL,2003:216-223.
[14]HULTH A,KARLGREN J,JONSSON A,et al.Automatic keyword extraction using domain knowledge[C]∥2nd International Conference on Computational Linguistics and Intelligent Text Processing.Mexico City:Springer-verlag,2001:472-482.
[15]KIM S N,KAN M Y.Re-examining automatic keyphrase extraction approaches in scientific articles[C]∥Proceedings of the ACL-IJCNLP Workshop on Multiword Expressions.USA:ACL,2009:9-16.
[16]LOPEZ P,ROMARY L.HUMB:Automatic key term extraction from scientific articles in GROBID[C]∥Proceedings of the 5th International Workshop on Semantic Evaluation.Sweden:ACM,2010:248-251.
[17]JIANG X,HU Y H,LI H.A ranking approach to keyphrase extraction[C]∥32nd International ACM SIGIR Conference on Research and Development in Information Retrieval.New York:ACM,2009:756-757.
[18]HUANG Z H,XU W,YU K.Bidirectional LSTM-CRF Models for Sequence Tagging(arXiv)(Version1.0)[OL].https://arxiv.org/abs/1508.01991.
[19]BENGIO Y,DUCHARME R,VINCENT P,et al.A neural probabilistic language model[J].Journal of Machine Learning Research,2003,3(6):1137-1155.
[20]COLLOBERT R,WESTON J,BOTTOU L,et al.Natural language processing (almost) from scratch[J].Journal of Machine Learning Research,2011,12(1):2493-2537.
[21]MIKOLOV T,YIH W T,ZWEIG G.Linguistic regularities in continuous space word representations[C]∥NAACL-HLT.USA:ACL,2013:746-751.
[22]LEVY O,GOLDBERG Y,DAGAN I.Improving distributional similarity with lessons learned from word embeddings[J].Transactions of the Association for Computational Linguistics,2015,75(3):211-225.
[23]LAMPLE G,BALLESTEROS M,SUBRAMANIAN S,et al. Neural Architectures for Named Entity Recognition (arXiv)(Version3.0)[OL].https://arxiv.org/abs/1603.01360.
[24]LAFFERTY F,MCCALLUM A,PEREIRA F.Conditional Random Fields:Probabilistic models for segmenting and labeling sequence data[C]∥Proceedings of ICML-2001.New York:ACM,2001:282-289.
[1] YAN Jia-dan, JIA Cai-yan. Text Classification Method Based on Information Fusion of Dual-graph Neural Network [J]. Computer Science, 2022, 49(8): 230-236.
[2] WANG Xin-tong, WANG Xuan, SUN Zhi-xin. Network Traffic Anomaly Detection Method Based on Multi-scale Memory Residual Network [J]. Computer Science, 2022, 49(8): 314-322.
[3] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[4] LI Xiao-wei, SHU Hui, GUANG Yan, ZHAI Yi, YANG Zi-ji. Survey of the Application of Natural Language Processing for Resume Analysis [J]. Computer Science, 2022, 49(6A): 66-73.
[5] WANG Shan, XU Chu-yi, SHI Chun-xiang, ZHANG Ying. Study on Cloud Classification Method of Satellite Cloud Images Based on CNN-LSTM [J]. Computer Science, 2022, 49(6A): 675-679.
[6] PAN Zhi-hao, ZENG Bi, LIAO Wen-xiong, WEI Peng-fei, WEN Song. Interactive Attention Graph Convolutional Networks for Aspect-based Sentiment Classification [J]. Computer Science, 2022, 49(3): 294-300.
[7] ZHANG Hu, BAI Ping. Graph Convolutional Networks with Long-distance Words Dependency in Sentences for Short Text Classification [J]. Computer Science, 2022, 49(2): 279-284.
[8] CHEN Zhi-yi, SUI Jie. DeepFM and Convolutional Neural Networks Ensembles for Multimodal Rumor Detection [J]. Computer Science, 2022, 49(1): 101-107.
[9] WANG Li-mei, ZHU Xu-guang, WANG De-jia, ZHANG Yong, XING Chun-xiao. Study on Judicial Data Classification Method Based on Natural Language Processing Technologies [J]. Computer Science, 2021, 48(8): 80-85.
[10] PENG Bin, LI Zheng, LIU Yong, WU Yong-hao. Automatic Code Comments Generation Method Based on Convolutional Neural Network [J]. Computer Science, 2021, 48(12): 117-124.
[11] WU Yu, LI Zhou-jun. Survey on Retrieval-based Chatbots [J]. Computer Science, 2021, 48(12): 278-285.
[12] ZHANG Ning, FANG Jing-wen, ZHAO Yu-xuan. Bitcoin Price Forecast Based on Mixed LSTM Model [J]. Computer Science, 2021, 48(11A): 39-45.
[13] ZHANG Yu-shuai, ZHAO Huan, LI Bo. Semantic Slot Filling Based on BERT and BiLSTM [J]. Computer Science, 2021, 48(1): 247-252.
[14] TONG Xin, WANG Bin-jun, WANG Run-zheng, PAN Xiao-qin. Survey on Adversarial Sample of Deep Learning Towards Natural Language Processing [J]. Computer Science, 2021, 48(1): 258-267.
[15] LU Long-long, CHEN Tong, PAN Min-xue, ZHANG Tian. CodeSearcher:Code Query Using Functional Descriptions in Natural Languages [J]. Computer Science, 2020, 47(9): 1-9.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!