Computer Science ›› 2020, Vol. 47 ›› Issue (1): 193-198.doi: 10.11896/jsjkx.181202261

• Artificial Intelligence • Previous Articles     Next Articles

Chinese Short Text Keyphrase Extraction Model Based on Attention

YANG Dan-hao,WU Yue-xin,FAN Chun-xiao   

  1. (School of Electronic Engineering,Beijing University of Posts and Telecommunications,Beijing 100089,China)
  • Received:2018-12-05 Published:2020-01-19
  • About author:YANG Dan-hao,born in 1994,master.His main research interests include natural language processing;FAN Chun-xiao,born in 1962,professor.Her main research interests artificial intelligence and internet of things.

Abstract: Keyphrase extraction technology is a research hotspot in the field of natural language processing.In the current keyphrase extraction algorithm,the deep learning method seldom takes into account the characteristics of Chinese,the information of Chinese character granularity is not fully utilized,and the extraction effect of Chinese short text keyworks still has a large improvement space.In order to improve the effect of the keyphrase extraction for short text,a model for automatic keyphrase extraction abstracts was proposed,namely BAST model,which combines the bidirectional long short-term memory and attention mechanism based on sequence tagging model.Firstly, word vectors in the word granularity and character vectors in the character granularity are used to represent input text information.Secondly,the BAST model is trained,text features are extracted by using BiLSTM and attention mechanism,and the label of each word is classified.Finally,the character vector model is used to correct the extraction results of the word vector model.The experimental results show that the F1-measure of the BAST model reaches 66.93% on 8159 abstract data,which is 2.08% higher than that of the BiLSTM-CRF(Bidirectional Long Shoft-Term Memory and Conditional Random Field) algorithm,and is further improved than other traditional keyphrase extraction algorithms.The innovation of the model lies in the combination of the extraction results of the word vector and the character vector model.The model makes full use of the characteristics of the Chinese text information and can effectively extract keyphrases from the short text,and extraction effect is further improved.

Key words: Attention mechanism, Word embedding, Character embedding, Keyphrase extraction, LSTM

CLC Number: 

  • TP391
[1]GOLLAPALLI S,CARAGRA C.Extracting Keyphrases from Research Papers Using Citation Networks [C]∥ Proceedings of the National Conference on Artificial Intelligence.Quebec:AAAI Press,2014:1629-1635.
[2]FLORESCU C,CARAGEA C.Positionrank:An unsupervised approach to keyphrase extraction from scholarly documents[C]∥Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Vancouver,Canada,2017:1105-1115.
[3]HASAN K,NG V.Automatic keyphrase extraction:A survey of the state of the art[C]∥Proceedings of the 27th International Conference on Computational Linguistics.Baltimore,Maryland,2014:1262-1273.
[4]LI G,WANG H.Improved automatic keyword extraction based on textrank using domain knowledge[C]∥ Proceedings of the 2014 Natural Language Processing and Chinese Computing.Berlin:Springer-Verlag,2014:403-413.
[5]BOUGOUIN A,BOUDIN F,DAILLE B.TopicRank:Graph- Based Topic Ranking for Keyphrase Extraction[C]∥Procee-dings of theInternational Joint Conference on Natural Language Processing.Nagoya,Japan,2013:543-551.
[6]TENEVA N,CHENG W.Salience rank:efficient keyphrase extraction with topic modeling[C]∥Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Vancouver,Canada,2017:530-535.
[7]FLORESCU C,CARAGEA C.A Position-Biased PageRank Algorithm for Keyphrase Extraction[C]∥Proceedings of the American Association for Artificial Intelligence.San Francisco:AAAI Press,2017:4923-4924.
[8]ZHANG C,WANG H,LIU Y,et al.Automatic keyword extraction from documents using conditional random fields[J].Journal of Computational Information Systems,2008,4(3):1169-1180.
[9]HADDOUD M,MOKHRARI A,LECROQ T,et al.Accurate Keyphrase Extraction from Scientific Papers by Mining Linguistic Information[C]∥Proceedings of The Workshop on Mining Scientific Papers:Computational Linguistics and Bibliometrics.Istanbul,Turkey:CEUR-WS,2015:12-17.
[10]ONAN A,KORUKOGLU S,BULUT H.Ensemble of keyword extraction methods and classifiers in text classification[J].Expert Systems with Applications,2016,57(3):232-247.
[11]GOLLAPALLI S,LI X,YANG P.Incorporating expert know- ledge into keyphrase extraction[C]∥ Processings of the American Association for Artificial Intelligence.San Francisco:AAAI Press,2017:3180-3187.
[12]ZHANG Q,WANG Y,GONG Y,et al.Keyphrase Extraction Using Deep Recurrent Neural Networks on Twitter[C]∥ Proceedings of Conference on Empirical Methods in Natural Language Processing.Stroudsburg,PA:Association for Computational Linguistics,2016:836-845.
[13]REKIA K,ZHANG Y,ZHANG W,et al.CCG Supertagging via Bidirectional LSTM-CRF Neural Architecture[J].Neurocomputing,2017,283(12):31-37.
[14]MOURAD G.Character-level neural network for biomedical named entity recognition[J].Journal of Biomedical Informatics,2017,70(5):85-91.
[15]ANDREJ Z,YORAM B,PASHA M,et al.Neural Named Entity Recognition Using a Self-Attention Mechanism[C]∥Procee-dings of International Conference on TOOLS with Artificial Intelligence.Boston:IEEE Computer Society,2017:652-656.
[16]SI Y,XIAO Y,XU J,et al.Recurrent neural network language model with vector-space word representations[C]∥Proceedings of the International Conference on Learning Representations.Beijing:International Institute of Acoustics and Vibrations,2014:3024-3031.
[17]SUNDERMEYER M,SCHLUTER R,NEY H.LSTM Neural Networks for Language Modeling[C]∥Proceedings of the 13th Annual Conference of the International Speech Communication Association Interspeech.Portland,OR,2012:194-197.
[18]GRAVES A,SCHMIDHUBER J.Framewise phoneme classification with bidirectional LSTM and other neural network architectures[J].Nrural Networks,2005,18(5):602-610.
[19]FENG S,LIU S,YANG N,et al.Improving attention modeling with implicit distortion and fertility for machine translation[C]∥Proceedings of 26th International Conference on Computational Linguistics.Osaka,Japan,2016:3082-3092.
[20]TAN Z,WANG M,XIE J,et al.Deep Semantic Role Labeling with Self-Attention[C]∥Proceedings of the American Association for Artificial Intelligence.San Francisco:AAAI Press,2017:4923-4924.
[1] ZHAO Jia-qi, WANG Han-zheng, ZHOU Yong, ZHANG Di, ZHOU Zi-yuan. Remote Sensing Image Description Generation Method Based on Attention and Multi-scale Feature Enhancement [J]. Computer Science, 2021, 48(1): 190-196.
[2] LIU Yang, JIN Zhong. Fine-grained Image Recognition Method Combining with Non-local and Multi-region Attention Mechanism [J]. Computer Science, 2021, 48(1): 197-203.
[3] WANG Rui-ping, JIA Zhen, LIU Chang, CHEN Ze-wei, LI Tian-rui. Deep Interest Factorization Machine Network Based on DeepFM [J]. Computer Science, 2021, 48(1): 226-232.
[4] ZHANG Yu-shuai, ZHAO Huan, LI Bo. Semantic Slot Filling Based on BERT and BiLSTM [J]. Computer Science, 2021, 48(1): 247-252.
[5] WANG Run-zheng, GAO Jian, HUANG Shu-hua, TONG Xin. Malicious Code Family Detection Method Based on Knowledge Distillation [J]. Computer Science, 2021, 48(1): 280-286.
[6] TIAN Ye, SHOU Li-dan, CHEN Ke, LUO Xin-yuan, CHEN Gang. Natural Language Interface for Databases with Content-based Table Column Embeddings [J]. Computer Science, 2020, 47(9): 60-66.
[7] CUI Tong-tong, WANG Gui-ling, GAO Jing. Ship Trajectory Classification Method Based on 1DCNN-LSTM [J]. Computer Science, 2020, 47(9): 175-184.
[8] PAN Zu-jiang, LIU Ning, ZHANG Wei, WANG Jian-yong. MTHAM:Multitask Disease Progression Modeling Based on Hierarchical Attention Mechanism [J]. Computer Science, 2020, 47(9): 185-189.
[9] ZHAO Wei, LIN Yu-ming, WANG Chao-qiang, CAI Guo-yong. Opinion Word-pairs Collaborative Extraction Based on Dependency Relation Analysis [J]. Computer Science, 2020, 47(8): 164-170.
[10] WANG Jiao-jin, JIAN Mu-wei, LIU Xiang-yu, LIN Pei-guang, GEN Lei-lei, CUI Chao-ran, YIN Yi-long. Video Saliency Detection Based on 3D Full ConvLSTM Neural Network [J]. Computer Science, 2020, 47(8): 195-201.
[11] CHENG Jing, LIU Na-na, MIN Ke-rui, KANG Yu, WANG Xin, ZHOU Yang-fan. Word Embedding Optimization for Low-frequency Words with Applications in Short-text Classification [J]. Computer Science, 2020, 47(8): 255-260.
[12] YUAN Ye, HE Xiao-ge, ZHU Ding-kun, WANG Fu-lee, XIE Hao-ran, WANG Jun, WEI Ming-qiang, GUO Yan-wen. Survey of Visual Image Saliency Detection [J]. Computer Science, 2020, 47(7): 84-91.
[13] WANG Wen-dao, WANG Run-ze, WEI Xin-lei, QI Yun-liang, MA Yi-de. Automatic Recognition of ECG Based on Stacked Bidirectional LSTM [J]. Computer Science, 2020, 47(7): 118-124.
[14] LIU Yan, WEN Jing. Complex Scene Text Detection Based on Attention Mechanism [J]. Computer Science, 2020, 47(7): 135-140.
[15] ZHANG Ying, ZHANG Yi-fei, WANG Zhong-qing and WANG Hong-ling. Automatic Summarization Method Based on Primary and Secondary Relation Feature [J]. Computer Science, 2020, 47(6A): 6-11.
Full text



[1] LEI Li-hui and WANG Jing. Parallelization of LTL Model Checking Based on Possibility Measure[J]. Computer Science, 2018, 45(4): 71 -75 .
[2] SUN Qi, JIN Yan, HE Kun and XU Ling-xuan. Hybrid Evolutionary Algorithm for Solving Mixed Capacitated General Routing Problem[J]. Computer Science, 2018, 45(4): 76 -82 .
[3] ZHANG Jia-nan and XIAO Ming-yu. Approximation Algorithm for Weighted Mixed Domination Problem[J]. Computer Science, 2018, 45(4): 83 -88 .
[4] WU Jian-hui, HUANG Zhong-xiang, LI Wu, WU Jian-hui, PENG Xin and ZHANG Sheng. Robustness Optimization of Sequence Decision in Urban Road Construction[J]. Computer Science, 2018, 45(4): 89 -93 .
[5] SHI Wen-jun, WU Ji-gang and LUO Yu-chun. Fast and Efficient Scheduling Algorithms for Mobile Cloud Offloading[J]. Computer Science, 2018, 45(4): 94 -99 .
[6] ZHOU Yan-ping and YE Qiao-lin. L1-norm Distance Based Least Squares Twin Support Vector Machine[J]. Computer Science, 2018, 45(4): 100 -105 .
[7] LIU Bo-yi, TANG Xiang-yan and CHENG Jie-ren. Recognition Method for Corn Borer Based on Templates Matching in Muliple Growth Periods[J]. Computer Science, 2018, 45(4): 106 -111 .
[8] GENG Hai-jun, SHI Xin-gang, WANG Zhi-liang, YIN Xia and YIN Shao-ping. Energy-efficient Intra-domain Routing Algorithm Based on Directed Acyclic Graph[J]. Computer Science, 2018, 45(4): 112 -116 .
[9] CUI Qiong, LI Jian-hua, WANG Hong and NAN Ming-li. Resilience Analysis Model of Networked Command Information System Based on Node Repairability[J]. Computer Science, 2018, 45(4): 117 -121 .
[10] WANG Zhen-chao, HOU Huan-huan and LIAN Rui. Path Optimization Scheme for Restraining Degree of Disorder in CMT[J]. Computer Science, 2018, 45(4): 122 -125 .