计算机科学 ›› 2018, Vol. 45 ›› Issue (4): 208-214.doi: 10.11896/j.issn.1002-137X.2018.04.035
张景,朱国宾
ZHANG Jing and ZHU Guo-bin
摘要: Stack Overflow是一个热门的国外编程问答网站,通过对该网站编程提问帖的问题文本进行文本语义挖掘,能获析用户关注的编程热点。由于研究对象所代表的短文本信息具有高维性及分布不均的特点,易导致主题获取不明晰。文中提出一种基于LDA(Latent Dirichlet Allocation)主题模型的CBOW-LDA建模方法,该方法对目标语料进行相似词聚类后再完成主题建模,能有效降低文本输入维度,使主题分布更明确。采集Stack Overflow网站上2010-2015年的问题帖数据集POST,并对其进行实验,同等主题数下采用文本建模中衡量模型性能的评价指标困惑度(Perplexity)来度量算法在不同数据集容量维度下的性能。结果表明,与现有的基于词频权重的词量化主题建模TF-LDA方法相比,CBOW-LDA方法的困惑度更低,在实验语料下的困惑度降低约4.87%,证明了所提算法的性能更好。采用CBOW-LDA方法对Stack Overflow进行热点挖掘,同时使用TF-LDA方法进行对比实验,建立手工标注的标准评测集对两种方法获取的热门主题和热搜词汇进行查全率、查准率及F1值的判定,结果证实CBOW-LDA表现更佳,其热点挖掘效果较好。由实验结果可知,Java为该编程网站提问帖中最热门的主题,而C和Javascript则为该网站用户提问中被提及得最频繁的词汇。
[1] BLEI D M,NG A Y,JORDAN M I.Latent dirichlet allocation [J].Journal of Machine Learning Research,2003,3:993-1022. [2] MIAO Z,CHEN K,FANG Y,et al.Cost-Effective Online Trending Topic Detection and Popularity Prediction in Microblogging[J].Acm Transactions on Information Systems,2016,35(3):18. [3] LEE Y J,YEH Y R,WANG Y C F.Anomaly Detection via Online Oversampling Principal Component Analysis[J].IEEE Transactions on Knowledge & Data Engineering,2013,25(7):1460-1470. [4] WU L,HOI S C H,YU N.Semantics-preserving bag-of-words models and applications [J].IEEE Transactions on Image Processing A Publication of the IEEE Signal Processing Society,2010,19(7):1908-1920. [5] RAMAGE D,DUMAIS S T,LIEBLING D J.Characterizing microblogs with topic models[C]∥Fourth International Confe-rence on Weblogs and Social Media.Menlo Park:AAAI Press,2010:130-137. [6] LEE C H,CHIEN T F.Leveraging microblogging big data with a modified density-based clustering approach for event awareness and topic ranking [J].Journal of Information Science,2013,39(4):523-543. [7] MIKOLOV T,Language Modeling for Speech Recognition[D].Brno:Brno University of Technology,2007. [8] MIKOLOV T,KOPECYK J,BURGRT L,et al.Neural network based language models for highly inflective languages[C]∥IEEE International Conference on Acoustics,Speech and Signal Processing.Taipei:IEEE,2009:4725-4728. [9] TOMAS M,CHEN K,CORRADO G.Efficient estimation ofword representations in vector space[EB/OL].(2013-08-18) [2013-09-07].http://arxiv.org /abs /1301.3781. [10] PEGHOTY.word2vec中的数学原理[EB/OL].http://blog.csdn.net/itplus/article/details/37969979. [11] POST数据集来源网址[EB/OL].http://meta.stackexchange.com/questions/2677/database-schema-documentation-for-the-public-data-dump-and-sede. [12] GUO L T,LI Y,MU D J.A Method of Topic Discovery Based on LDA Theme Model [J].Journal of Northwestern Polytechnical University,2016,4(1):698-702. [13] HUANG B,YANG Y,MAHMOOD A,et al.Microblog TopicDetection Based on LDA Model and Single-Pass Clustering[M]∥Rough Sets and Current Trends in Computing.Springer Berlin Heidelberg,2012:166-171. [14] GUPTA M,KUMAR P,BHASKER B.Clustering of users on microblogging social media:A rough set based approach[C]∥International Conference on Data Science and Engineering.IEEE,2017:1-6. |
No related articles found! |
|