Computer Science ›› 2018, Vol. 45 ›› Issue (9): 243-247.doi: 10.11896/j.issn.1002-137X.2018.09.040

• Artificial Intelligence • Previous Articles     Next Articles

Approach of Stance Detection in Micro-blog Based on Transfer Learning and Multi-representation

ZHOU Yan-fang, ZHOU Gang, LU Zhong-lei   

  1. State Key Laboratory of Mathematical Engineering and Advanced Computing,Zhengzhou 450001,China
  • Received:2017-08-10 Online:2018-09-20 Published:2018-10-10

Abstract: Stance detection aims to identify users’ opinion towards a particular target.Aiming at the problem that exi-sting methods are often difficult to overcome the lack of labeled data and the error caused by word segmentation of Chinese text,this paper presented a transfer learning method and a hybrid model of character-level and word-level features.Firstly,character-level and the word-level features are inputted to deep neural network and the outputs of both are concatenated to reproduce the missing semantic information caused by word segmentation.Then,a topic classification model(parent model) is trained with a large external micro-blog data to obtain the effective sentence feature representation.Next,some of parent model’s parametersare used to initialize stance detection model and the knowledge transferred from auxiliary data can be used to enhance semantic representation ability of sentences.Finally,the labeled data are used to fine tune the child model andtrain classifiers.Experiment on NLPCC-2016 Task 4 proves that F1 value of proposed method achieves 72.2%,which is better than the best one of participating teams.The results show that this approach can improve the stance detection performance and alleviate the influence caused by word segmentation.

Key words: Deep learning, Micro-blog, Stance detection, Transfer learning

CLC Number: 

  • TP391
[1]THOMAS M,PANG B,LEE L.Get out the vote:determining support or opposition from congressional floor-debate transcripts[C]∥Conference on Empirical Methods in Natural Language Processing.Association for Computational Linguistics,2006:327-335.
[2]SOMASUNDARAN S,WIEBE J.Recognizing Stances in Online Debates[C]∥Joint Conference of the 47th Annual Metting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP.2009:116-124.
[3]MURAKAMI A,RAYMOND R.Support or oppose?:classif-ying positions in online debates from reply activities and opinion expressions[C]∥International Conference on Computational Linguistics:Posters.Association for Computational Linguistics,2010:869-875.
[4]WALKER M A,ANAND P,ABBOTT R,et al.Stance classification using dialogic properties of persuasion[C]∥Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2013:592-596.
[5]RAJADESINGAN A,LIU H.Identifying Users with Opposing Opinions in Twitter Debates[M]∥Social Computing,Behavio-ral-Cultural Modeling and Prediction.Springer International Publishing,2016:153-160.
[6]MIKOLOV T,LE Q V,SUTSKEVER I.Exploiting Similarities among Languages for Machine Translation[J].arXiv preprint arXiv:1309.4168.2013.
[7]PAN S J,YANG Q.A survey on transfer learning[J].IEEE Transactions on Data Engineering,2010,22(10):1345-1359.
[8]SCHÖLKOPF B,PLATT J,HOFMANN T.Analysis of Representations for Domain Adaptation[C]∥International Conference on Neural Information Processing Systems.MIT Press,2006:137-144.
[9]ERHAN D,BENGIO Y,COURVILLE A C,et al.Why Does Unsupervised Pre-training Help Deep Learning?[J].Journal of Machine Learning Research,2010,11(3):625-660.
[10]GRAVES A.Supervised Sequence Labelling with Recurrent
Neural Networks[OL].http://mediatum.ub.tum.de/doc/673554/file.pdf.
[11]BENGIO Y,SIMARD P,FRASCONI P.Learning long-term dependencies with gradient descent is difficult[J].IEEE Transactions on Neural Networks,2002,5(2):157-166.
[12]HOCHREITER S,SCHMIDHUBER J.Long Short-Term Me-mory[J].Neural Computation,1997,9(8):1735.
[13]SCHUSTER M,PALIWAL K K.Bidirectional re-current neural networks[J].IEEE Transactions on Signal Processing,1997,45(11):2673-2681.
[14]GRAVES A,JAITLY N,MOHAMED A R.Hybrid speech re-cognition with Deep Bidirectional LSTM[C]∥Automatic Speech Recognition and Understanding.IEEE,2014:273-278.
[15]BAHDANAU D,CHO K,BENGIO Y.Neural Machine Translation by Jointly Learning to Align and Translate[C]∥3rd International Conference on Learning Representation.2015.
[16]YOSINSKI J,CLUNE J,BENGIO Y,et al.How transferable are features in deep neural networks?[J].EprintArxiv,2014,27:3320-3328.
[17]KIROS R,ZHU Y,SALAKHUTDINOV R,et al.Skip-Thought Vectors[OL].http://www.cs.toronto.edu/~zemel/documents/skipThought.pdf.
[18]DAI A M,LE Q V.Semi-supervised Sequence Learning[C]∥International Conference on Neural Internation Processing Systems.2015:3079-3087.
[19]HILL F,CHO K,KORHONEN A.Learning Distributed Representations of Sentences from Unlabelled Data[C]∥Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2016:1367-1377.
[20]WESTON J,CHOPRA S,ADAMS K.#TagSpace:Semantic
Embeddings from Hashtags[C]∥Conference on Empirical Methods in Natural Language Processing.2014:1822-1827.
[21]YANG W,SONG J J,TANG J Q.A Study on the Classification Approach for Chinese MicroBlog Subjective and Objective Sentences[J].Journal of Chongqing Institute of Technology,2013,27(1):51-56.(in Chinses)
杨武,宋静静,唐继强.中文微博情感分析中主客观句分类方法[J].重庆理工大学学报(自然科学),2013,27(1):51-56.
[1] XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[2] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[3] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[4] WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[5] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[6] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[7] FANG Yi-qiu, ZHANG Zhen-kun, GE Jun-wei. Cross-domain Recommendation Algorithm Based on Self-attention Mechanism and Transfer Learning [J]. Computer Science, 2022, 49(8): 70-77.
[8] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[9] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[10] ZHOU Hui, SHI Hao-chen, TU Yao-feng, HUANG Sheng-jun. Robust Deep Neural Network Learning Based on Active Sampling [J]. Computer Science, 2022, 49(7): 164-169.
[11] SU Dan-ning, CAO Gui-tao, WANG Yan-nan, WANG Hong, REN He. Survey of Deep Learning for Radar Emitter Identification Based on Small Sample [J]. Computer Science, 2022, 49(7): 226-235.
[12] HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[13] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[14] LIU Wei-ye, LU Hui-min, LI Yu-peng, MA Ning. Survey on Finger Vein Recognition Research [J]. Computer Science, 2022, 49(6A): 1-11.
[15] SUN Fu-quan, CUI Zhi-qing, ZOU Peng, ZHANG Kun. Brain Tumor Segmentation Algorithm Based on Multi-scale Features [J]. Computer Science, 2022, 49(6A): 12-16.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!