Computer Science ›› 2016, Vol. 43 ›› Issue (6): 214-217.doi: 10.11896/j.issn.1002-137X.2016.06.043

Previous Articles     Next Articles

Document Vector Representation Based on Word2Vec

TANG Ming, ZHU Lei and ZOU Xian-chun   

  • Online:2018-12-01 Published:2018-12-01

Abstract: In text classification issues,it is difficult to express a document efficiently by the word vector of word2vec.At present,doc2vec built on the combination of word2vec and clustering algorithm can express the information of document very well.However,this method rarely considers a single word’s influence for the entire document.To solve this pro-blem,in this paper, TF-IDF algorithm was used to calculate the right weight of words in documents,and word2vec was combined to generate document vectors,which were used for Chinese text classification.Experiments on the Sogou Chinese corpus laboratory demonstrate the efficiency of this newly proposed algorithm.

Key words: TF-IDF,Word2vec,Doc2vec,Text classification

[1] Baeza-Yates R,Ribeiro-Neto B.Modern Information Retrieval[M].New York:ACM press,1999
[2] Manning C D,Schütze H.Foundations of Statistical NaturalLanguage Processing [M].Cambridge:MIT press,1999
[3] Hwang M,Choi C,Youn B,et al.Word Sense Disambiguation Based on Relation Structure[C]∥International Conference on Advanced Language Processing and Web Information Technology.2008:15-20
[4] Wang X,Mccallum A,Wei X.Topical N-Grams:Phrase andTopic Discovery,with an Application to Information Retrieval [C]∥IEEE International Conference on Data Mining.IEEE Computer Society,2007:697-702
[5] Haruechaiyasak C,Jitkrittum W,Sangkeettrakarn C,et al.Im-plementing News Article Category Browsing Based on Text Categorization Technique [C]∥2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology.IEEE Computer Society,2008:143-146
[6] Mikolov T,Sutskever I,Chen K,et al.Distributed Representations of Words and Phrases and their Compositionality [J].Advances in Neural Information Processing Systems,2013,26:3111-3119
[7] Mikolov T,Chen K,Corrado G,et al.Efficient Estimation of Word Representations in Vector Space [C]∥ICLR 2013.2013
[8] Joachims T.A Probabilistic Analysis of the Rocchio Algorithm with TFIDF for Text Categorization [M].Springer US,1997:143-151
[9] Hinton G E.Learning distributed representations of concepts[C]∥Proceedings of CogSci.1986:1-12
[10] Socher R,Bauer J,Manning C D,et al.Parsing with Compositional Vector Grammars [C]∥Meeting of the Association for Computational Linguistics.2013:455-465
[11] Socher R,Perelygin A,Wu J Y,et al.Recursive Deep Models for Semantic Compositionality over a Sentiment Treebank [C]∥Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP).2013:1631-1642
[12] Sun Y,Lin L,Yang N,et al.Radical-Enhanced Chinese Character Embedding [J].Lecture Notes in Computer Science,2014,8835:279-286
[13] Mansur M,Pei W,Chang B.Feature-based Neural LanguageModel and Chinese Word Segmentation [C]∥IJCNLP.2013:1271-1277
[14] Zheng X,Chen H,Xu T.Deep Learning for Chinese Word Segmentation and POS Tagging [C]∥EMNLP.2013:647-657
[15] Tang D,Wei F,Yang N,et al.Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification [C]∥ACL.2014:1555-1565
[16] Zhang M,Zhang Y,Che W,et al.Chinese Parsing ExploitingCharacters [C]∥ACL.2013:125-134
[17] Xing C,Wang D,Zhang X,et al.Document Classification with Distributions of Word Vectors [C]∥2014 Annual Summit and Conference Asia-Pacific Signal and Information Processing Association(APSIPA).IEEE,2014:1-5
[18] Kim H K,Kim H,Cho S.Bag-of-Concepts:Comprehending Do-cument Representation through Clustering Words in Distributed Representation.http://dm.snu.ac.kr/static/docs/TR/SNUDM-TR-2015-05.pdf
[19] Le Q V,Mikolov T.Distributed Representations of Sentences and Documents [J].Eprint Arxiv,2014,4:1188-1196
[20] Morin F,Bengio Y.Hierarchical Probabilistic Neural NetworkLanguage Model [J].Aistats.2005,5:246-252
[21] Mnih A,Hinton G E.A Scalable Hierarchical Distributed Language Model [C]∥Advances in Neural Information Processing Systems.2009:1081-1088
[22] Mikolov T,Yih W,Zweig G.Linguistic Regularities in Conti-nuous Space Word Representations [C]∥HLT-NAACL.2013:746-751
[23] Santana L E A,De Oliveira D F,Canuto A M P,et al.A Comparative Analysis of Feature Selection Mmethods for Ensembles with Different Combination Methods [C]∥International Joint Conference on Neural Networks,2007(IJCNN 2007).IEEE,2007:643-648
[24] Forman G.An Extensive Empirical Study of Feature SelectionMetrics for Text Classification [J].The Journal of Machine Learning Research,2003,3:1289-1305
[25] 搜狗.文本分类语料库.http://www.sogou.com/labs/dl/c.html
[26] Gensim.Topic Modelling for Humans.http://radimre-hurek.com/gensim
[27] Bengio Y,Schwenk H,Senécal J S,et al.Neural ProbabilisticLanguage Models [M]∥Innovations in Machine Learning.Springer Berlin Heidelberg,2006
[28] Mnih A,Hinton G.Three New Graphical Models for Statistical Language Modelling [C]∥Proceedings of the 24th International Conference on Machine Learning.ACM,2007:641-648

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!