基于Word2Vec的一种文档向量表示

doi:10.11896/j.issn.1002-137X.2016.06.043

摘要/Abstract

摘要： 在文本分类中,如何运用word2vec词向量高效地表达一篇文档一直是一个难点。目前,将word2vec模型与聚类算法结合形成的doc2vec模型能有效地表达文档信息。但是,这种方法很少考虑单个词对整篇文档的影响力。为了解决这个问题,利用TF-IDF算法计算每篇文档中词的权重,并结合word2vec词向量生成文档向量,最后将其应用于中文文档分类。在搜狗中文语料库上的实验验证了新方法的有效性。

关键词: TF-IDF,word2vec,doc2vec,文本分类

Abstract: In text classification issues,it is difficult to express a document efficiently by the word vector of word2vec.At present,doc2vec built on the combination of word2vec and clustering algorithm can express the information of document very well.However,this method rarely considers a single word’s influence for the entire document.To solve this pro-blem,in this paper, TF-IDF algorithm was used to calculate the right weight of words in documents,and word2vec was combined to generate document vectors,which were used for Chinese text classification.Experiments on the Sogou Chinese corpus laboratory demonstrate the efficiency of this newly proposed algorithm.

Key words: TF-IDF,Word2vec,Doc2vec,Text classification

唐明,朱磊,邹显春. 基于Word2Vec的一种文档向量表示[J]. 计算机科学, 2016, 43(6): 214-217. https://doi.org/10.11896/j.issn.1002-137X.2016.06.043

TANG Ming, ZHU Lei and ZOU Xian-chun. Document Vector Representation Based on Word2Vec[J]. Computer Science, 2016, 43(6): 214-217. https://doi.org/10.11896/j.issn.1002-137X.2016.06.043

参考文献

[1] Baeza-Yates R,Ribeiro-Neto B.Modern Information Retrieval[M].New York:ACM press,1999
[2] Manning C D,Schütze H.Foundations of Statistical NaturalLanguage Processing [M].Cambridge:MIT press,1999
[3] Hwang M,Choi C,Youn B,et al.Word Sense Disambiguation Based on Relation Structure[C]∥International Conference on Advanced Language Processing and Web Information Technology.2008:15-20
[4] Wang X,Mccallum A,Wei X.Topical N-Grams:Phrase andTopic Discovery,with an Application to Information Retrieval [C]∥IEEE International Conference on Data Mining.IEEE Computer Society,2007:697-702
[5] Haruechaiyasak C,Jitkrittum W,Sangkeettrakarn C,et al.Im-plementing News Article Category Browsing Based on Text Categorization Technique [C]∥2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology.IEEE Computer Society,2008:143-146
[6] Mikolov T,Sutskever I,Chen K,et al.Distributed Representations of Words and Phrases and their Compositionality [J].Advances in Neural Information Processing Systems,2013,26:3111-3119
[7] Mikolov T,Chen K,Corrado G,et al.Efficient Estimation of Word Representations in Vector Space [C]∥ICLR 2013.2013
[8] Joachims T.A Probabilistic Analysis of the Rocchio Algorithm with TFIDF for Text Categorization [M].Springer US,1997:143-151
[9] Hinton G E.Learning distributed representations of concepts[C]∥Proceedings of CogSci.1986:1-12
[10] Socher R,Bauer J,Manning C D,et al.Parsing with Compositional Vector Grammars [C]∥Meeting of the Association for Computational Linguistics.2013:455-465
[11] Socher R,Perelygin A,Wu J Y,et al.Recursive Deep Models for Semantic Compositionality over a Sentiment Treebank [C]∥Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP).2013:1631-1642
[12] Sun Y,Lin L,Yang N,et al.Radical-Enhanced Chinese Character Embedding [J].Lecture Notes in Computer Science,2014,8835:279-286
[13] Mansur M,Pei W,Chang B.Feature-based Neural LanguageModel and Chinese Word Segmentation [C]∥IJCNLP.2013:1271-1277
[14] Zheng X,Chen H,Xu T.Deep Learning for Chinese Word Segmentation and POS Tagging [C]∥EMNLP.2013:647-657
[15] Tang D,Wei F,Yang N,et al.Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification [C]∥ACL.2014:1555-1565
[16] Zhang M,Zhang Y,Che W,et al.Chinese Parsing ExploitingCharacters [C]∥ACL.2013:125-134
[17] Xing C,Wang D,Zhang X,et al.Document Classification with Distributions of Word Vectors [C]∥2014 Annual Summit and Conference Asia-Pacific Signal and Information Processing Association(APSIPA).IEEE,2014:1-5
[18] Kim H K,Kim H,Cho S.Bag-of-Concepts:Comprehending Do-cument Representation through Clustering Words in Distributed Representation.http://dm.snu.ac.kr/static/docs/TR/SNUDM-TR-2015-05.pdf
[19] Le Q V,Mikolov T.Distributed Representations of Sentences and Documents [J].Eprint Arxiv,2014,4:1188-1196
[20] Morin F,Bengio Y.Hierarchical Probabilistic Neural NetworkLanguage Model [J].Aistats.2005,5:246-252
[21] Mnih A,Hinton G E.A Scalable Hierarchical Distributed Language Model [C]∥Advances in Neural Information Processing Systems.2009:1081-1088
[22] Mikolov T,Yih W,Zweig G.Linguistic Regularities in Conti-nuous Space Word Representations [C]∥HLT-NAACL.2013:746-751
[23] Santana L E A,De Oliveira D F,Canuto A M P,et al.A Comparative Analysis of Feature Selection Mmethods for Ensembles with Different Combination Methods [C]∥International Joint Conference on Neural Networks,2007(IJCNN 2007).IEEE,2007:643-648
[24] Forman G.An Extensive Empirical Study of Feature SelectionMetrics for Text Classification [J].The Journal of Machine Learning Research,2003,3:1289-1305
[25] 搜狗.文本分类语料库.http://www.sogou.com/labs/dl/c.html
[26] Gensim.Topic Modelling for Humans.http://radimre-hurek.com/gensim
[27] Bengio Y,Schwenk H,Senécal J S,et al.Neural ProbabilisticLanguage Models [M]∥Innovations in Machine Learning.Springer Berlin Heidelberg,2006
[28] Mnih A,Hinton G.Three New Graphical Models for Statistical Language Modelling [C]∥Proceedings of the 24th International Conference on Machine Learning.ACM,2007:641-648

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed