基于图神经网络和依存句法分析的文本分类

doi:10.11896/jsjkx.220300195

摘要/Abstract

摘要： 文本分类被广泛应用于新闻分类、话题标记和情感分析等语言处理场景中,是自然语言处理中的一个基本而重要的任务。目前的文本分类模型一般没有同时考虑文本单词的共现关系和文本自身的句法特性,从而限制了文本分类的效果。因此,提出了一个基于图卷积神经网络的文本分类模型(Mix-GCN)。首先基于文本单词之间的共现关系和句法依存关系,将文本数据构建成文本共现图和句法依存图;接着,利用GCN模型对文本图和句法依赖图进行表示学习,得到单词的嵌入向量;然后通过图池化方法以及自适应融合的方法得到文本的嵌入向量;最后通过图分类方法完成文本分类。Mix-GCN模型同时考虑了文本中相邻单词之间的关系和文本单词之间存在的句法依存关系,提升了文本分类性能。在6个基准数据集上与8种知名文本分类方法进行了比较,实验结果表明Mix-GCN具有良好的文本分类效果。

关键词: 文本分类, 图神经网络, 依存句法分析, 图分类

Abstract: Text classification is a basic and important task in natural language processing.It is widely used in language processing scenarios such as news classification,topic tagging and sentiment analysis.The current text classification models generally do not consider the co-occurrence relationship of text words and the syntactic characteristics of the text itself,thus limiting the effect of text classification.Therefore,a text classification model based on graph convolutional neural network(Mix-GCN) is proposed.Firstly,based on the co-occurrence relationship and syntactic dependency between text words,the text data is constructed into a text co-occurrence graph and a syntactic dependency graph.Then the GCN model is used to perform representation learning on the text graph and syntactic dependency graph,and the embedding vector of the word is obtained.Then the embedding vector of the text is obtained by graph pooling method and adaptive fusion method,and the text classification is completed by the graph classification method.Mix-GCN model simultaneously considers the relationship between adjacent words in the text and the syntactic dependencies existing between text words,which improves the performance of text classification.On 6 benchmark datasets,compared to 8 well-known text classification methods,experimental results show that Mix-GCN has a good text classification effect.

Key words: Text classification, Graph neural network, Dependency parsing, Graph classification

中图分类号:

TP391

杨旭华, 金鑫, 陶进, 毛剑飞. 基于图神经网络和依存句法分析的文本分类[J]. 计算机科学, 2022, 49(12): 293-300. https://doi.org/10.11896/jsjkx.220300195

YANG Xu-hua, JIN Xin, TAO Jin, MAO Jian-fei. Text Classification Based on Graph Neural Networks and Dependency Parsing[J]. Computer Science, 2022, 49(12): 293-300. https://doi.org/10.11896/jsjkx.220300195

参考文献

[1]ROY P K,SINGH J P,BANERJEE S.Deep learning to filter SMS Spam[J].Future Generation Computer Systems,2020,102:524-533.
[2]MINAEE S,KALCHBRENNER N,CAMBRIA E,et al.DeepLearning--based Text Classification:A Comprehensive Review[J].ACM Computing Surveys,2021,54(3):1-40.
[3]XIONG S F,JI D H.A short text sentiment topic model for product review analysis[J].Acta Automatica Sinica,2016,42(8):1227-1237.
[4]JIN L,HE T T.Chinese text classification model based on improved TF-IDF and ABLCNN[J].Computer Science,2021,48(11A):170-175.
[5]HARRELL F E.Regression modeling strategies:with applications to linear models,logistic and ordinal regression,and survival analysis[M].New York:Springer,2015.
[6]RISH I.An empirical study of the naive Bayes classifier[C]//IJCAI 2001 Workshop on Empirical Methods in Artificial Intelligence.2001:41-46.
[7]KELLER J M,GRAY M R,GIVENS J A.A fuzzy k-nearest neighbor algorithm[J].IEEE Transactions on Systems,Man,and Cybernetics,1985(4):580-585.
[8]SANCHEZ AV D.Advanced support vector machines and kernel methods[J].Neurocomputing,2003,55(1/2):5-20.
[9]MIKOLOV T,CHEN K,CORRADO G,et al.Efficient estimation of word representations in vector space[J].arXiv:1301.3781,2013.
[10]KIM Y.Convolutional neural networks for sentence classification[J].arXiv:1408.5882,2014.
[11]JOHNSON R,TONG Z.Deep Pyramid Convolutional NeuralNetworks for Text Categorization[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.2017:562-570.
[12]MIKOLOV T,KARAFIAT M,BURGET L,et al.Recurrentneural network based language modeI[C]//Proceedings of the Annual Conference of the International Speech Communication Association.2010:1045-1048.
[13]ZHANG Y,LIU Q,SONG L.Sentence-State LSTM for Text Representation[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.2018:317-327.
[14]YANG M,ZHAO W,YE J,et al.Investigating capsule networks with dynamic routing for text classification[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.2018:3110-3119.
[15]LECUN Y,BENGIO Y,HINTON G.Deep learning[J].Nature,2015,521(7553):436-444.
[16]JOULIN A,GRAVE E,BOJANOWSKI P,et al.Bag of Tricks for Efficient Text Classification[C]//Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics:Volume 2,Short Papers.2017:427-431.
[17]ZHANG X,ZHAO J,LECUN Y.Character-level convolutional networks for text classification[J].Advances in Neural Information Processing Systems,2015,28:649-657.
[18]TANG D,QIN B,LIU T.Document modeling with gated recurrent neural network for sentiment classification[C]//Procee-dings of the Conference on Empirical Methods in Natural Language Processing.2015:1422-1432.
[19]LAI S,XU L,LIU K,et al.Recurrent convolutional neural networks for text classification[C]//Twenty-ninth AAAI Confe-rence on Artificial Intelligence.2015:2267-2273.
[20]WANG R,LI Z,CAO J,et al.Convolutional Recurrent Neural Networks for Text Classification[C]//2019 International Joint Conference on Neural Networks.2019:1-6.
[21]KIPF T N,WELLING M.Semi-supervised classification withgraph convolutional networks[J].arXiv:1609.02907,2016.
[22]GAO H,CHEN Y,JI S.Learning Graph Pooling and HybridConvolutional Operations for Text Representations[C]//World Wide Web Conference.2019:2743-2749.
[23]YAO L,MAO C,LUO Y.Graph convolutional networks fortext classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2019:7370-7377.
[24]PENG H,LI J,HE Y,et al.Large-scale hierarchical text classification with recursively regularized deep graph-cnn[C]//Proceedings of the 2018 World Wide Web Conference.2018:1063-1072.
[25]WU M,PAN S,ZHU X,et al.Domain-adversarial graph neural networks for text classification[C]//2019 IEEE International Conference on Data Mining.IEEE,2019:648-657.
[26]IVANOVA A,OEPEN S,ØVRELID L.Survey on parsing three dependency representations for English[C]//51st Annual Mee-ting of the Association for Computational Linguistics Procee-dings of the Student Research Workshop.2013:31-37.
[27]CHEN M,WEINBERGER K Q,SHA F.An alternative textrepresentation to TF-IDF and Bag-of-Words[J].arXiv:1301.6770,2013.
[28]ZHU Y,LI L,LUO L.Learning to classify short text with topic model and external knowledge[C]//International Conference on Knowledge Science,Engineering and Management.2013:493-503.
[29]MA X,JIN R,PAIK J Y,et al.LargeScale Text Classification with Efficient Word Embedding[C]//International Conference on Mobile and Wireless Technology.2017:465-469.
[30]BENGIO Y,DUCHARME R,VINCENT P,et al.A neuralprobabilistic language model[J].The Journal of Machine Lear-ning Research,2003,3:1137-1155.
[31]LE Q,MIKOLOV T.Distributed representations of sentencesand documents[C]//International Conference on Machine Learning.PMLR,2014:1188-1196.
[32]PENNINGTON J,SOCHER R,MANNING C D.Glove:Global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Proces-sing.2014:1532-1543.
[33]TAN T,PHIENTHRAKUL T.Sentiment Classification Using Document Embeddings Trained with Cosine Similarity[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics:Student Research Workshop.2019:407-414.
[34]PEROZZI B,AL-RFOU R,SKIENA S.Deepwalk:Online lear-ning of social representations[C]//Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Disco-very and Data Mining.2014:701-710.
[35]GOYAL P,FERRARA E.Graph embedding techniques,applica-tions,and performance:A survey[J].Knowledge-Based Systems,2018,151:78-94.
[36]ZHANG R,LI J T.A Survey of Research on Scene Segmentation Algorithms Based on Deep Learning[J].Computer Research and Development,2020,57(4):859-875.
[37]FEI H,LI F,LI B,et al.Encoder-decoder based unified semantic role labeling with label-aware syntax[C]//Proceedings of AAAI National Conference on Artificial Intelligence.2021:1479-1488.
[38]LI Y,JIN R,LUO Y.Classifying relations in clinical narratives using segment graph convolutional and recurrent neural networks(Seg-GCRNs)[J].Journal of the American Medical Informatics Association,2019,26(3):262-268.
[39]BASTINGS J,TITOV I,AZIZ W,et al.Graph convolutional encoders for syntax-aware neural machine translation[J].arXiv:1704.04675,2017.
[40]VELICKOVIC P,CUCURULL G,CASANOVA A,et al.Graph attention networks[J].arXiv:1710.10903,2017.
[41]HAMILTON W,YING Z,LESKOVEC J.Inductive representation learning on large graphs[J].arXiv:1706.02216,2017.
[42]KIM D,OH A.How to find your friendly neighborhood:Graph attention design with self-supervision[J].arXiv:2204.04879,2022.
[43]ALON U,YAHAV E.On the bottleneck of graph neural networks and its practical implications[J].arXiv:2006.05205,2020.
[44]ZHAO T,LIU Y,NEVES L,et al.Data Augmentation forGraph Neural Networks[C]//Proceedings of the AAAI Confe-rence on Artificial Intelligence.2021:11015-11023.
[45]LUONG M T,PHAM H,MANNING C D.Effective approaches to attention-based neural machine translation[J].arXiv:1508.04025,2015.
[46]CHO K,COURVILLE A,BENGIO Y.Describing multimediacontent using attention-based encoder-decoder networks[J].IEEE Transactions on Multimedia,2015,17(11):1875-1886.
[47]ZENG Y F,LAN T,WU Z F,et al.Aspect-level sentiment classification model based on dual-memory attention[J].Chinese Journal of Computers,2019,8:1845-1857.
[48]SUKHBAATAR S,WESTON J,FERGUS R.End-to-end me-mory networks[C]//Proceedings of the Advances in Neural Information Processing Systems.2015:2440-2448.
[49]WANG W,PAN S J,DAHLMEIER D,et al.Coupled multi-la-yer attentions for co-extraction of aspect and opinion terms[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2017.
[50]YANG Z,YANG D,DYER C,et al.Hierarchical attention networks for document classification[C]//Proceedings of the 2016 CONFERENce of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2016:1480-1489.
[51]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[C]//Proceedings of the Advances in Neural Information Processing Systems.2017:5998-6008.
[52]SUN Z,HUANG S,WEI H R,et al.Generating diverse translation by manipulating multi-head attention[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:8976-8983.
[53]MISRA D,NALAMADA T,ARASANIPALAI A U,et al.Rotate to attend:Convolutional triplet attention module[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.2021:3139-3148.
[54]WU Z,PAN S,CHEN F,et al.A comprehensive survey ongraph neural networks[J].IEEE Transactions on Neural Networks and Learning Systems,2020,32(1):4-24.
[55]DAI Y,SHOU L,GONG M,et al.Graph Fusion Network for Text Classification[J].Knowledge-Based Systems,2022,236:107659.
[56]GUO B,ZHANG C,LIU J,et al.Improving text classification with weighted word embeddings via a multi-channel TextCNN model[J].Neurocomputing,2019,363:366-374.
[57]LIU H,CHEN G,LI P,et al.Multi-label text classification via joint learning from label embedding and label correlation[J].Neurocomputing,2021,460:385-398.
[58]XU J,CAI Y,WU X,et al.Incorporating context-relevant concepts into convolutional neural networks for short text classification[J].Neurocomputing,2020,386:42-53.
[59]WU Y,LI J,WU J,et al.Siamese capsule networks with global and local features for text classification[J].Neurocomputing,2020,390:88-98.
[60]SHEN D,WANG G,WANG W,et al.Baseline needs more love:On simple word-embedding-based models and associated pooling mechanisms[J].arXiv:1805.09843,2018.

相关文章 15

[1]	周芳泉, 成卫青. 基于全局增强图神经网络的序列推荐 Sequence Recommendation Based on Global Enhanced Graph Neural Network 计算机科学, 2022, 49(9): 55-63. https://doi.org/10.11896/jsjkx.210700085
[2]	武红鑫, 韩萌, 陈志强, 张喜龙, 李慕航. 监督和半监督学习下的多标签分类综述 Survey of Multi-label Classification Based on Supervised and Semi-supervised Learning 计算机科学, 2022, 49(8): 12-25. https://doi.org/10.11896/jsjkx.210700111
[3]	檀莹莹, 王俊丽, 张超波. 基于图卷积神经网络的文本分类方法研究综述 Review of Text Classification Methods Based on Graph Convolutional Network 计算机科学, 2022, 49(8): 205-216. https://doi.org/10.11896/jsjkx.210800064
[4]	闫佳丹, 贾彩燕. 基于双图神经网络信息融合的文本分类方法 Text Classification Method Based on Information Fusion of Dual-graph Neural Network 计算机科学, 2022, 49(8): 230-236. https://doi.org/10.11896/jsjkx.210600042
[5]	郝志荣, 陈龙, 黄嘉成. 面向文本分类的类别区分式通用对抗攻击方法 Class Discriminative Universal Adversarial Attack for Text Classification 计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[6]	齐秀秀, 王佳昊, 李文雄, 周帆. 基于概率元学习的矩阵补全预测融合算法 Fusion Algorithm for Matrix Completion Prediction Based on Probabilistic Meta-learning 计算机科学, 2022, 49(7): 18-24. https://doi.org/10.11896/jsjkx.210600126
[7]	杨炳新, 郭艳蓉, 郝世杰, 洪日昌. 基于数据增广和模型集成策略的图神经网络在抑郁症识别上的应用 Application of Graph Neural Network Based on Data Augmentation and Model Ensemble in Depression Recognition 计算机科学, 2022, 49(7): 57-63. https://doi.org/10.11896/jsjkx.210800070
[8]	邓凯, 杨频, 李益洲, 杨星, 曾凡瑞, 张振毓. 一种可快速迁移的领域知识图谱构建方法 Fast and Transmissible Domain Knowledge Graph Construction Method 计算机科学, 2022, 49(6A): 100-108. https://doi.org/10.11896/jsjkx.210900018
[9]	康雁, 吴志伟, 寇勇奇, 张兰, 谢思宇, 李浩. 融合Bert和图卷积的深度集成学习软件需求分类 Deep Integrated Learning Software Requirement Classification Fusing Bert and Graph Convolution 计算机科学, 2022, 49(6A): 150-158. https://doi.org/10.11896/jsjkx.210500065
[10]	邵欣欣. TI-FastText自动商品分类算法 TI-FastText Automatic Goods Classification Algorithm 计算机科学, 2022, 49(6A): 206-210. https://doi.org/10.11896/jsjkx.210500089
[11]	熊中敏, 舒贵文, 郭怀宇. 融合用户偏好的图神经网络推荐模型 Graph Neural Network Recommendation Model Integrating User Preferences 计算机科学, 2022, 49(6): 165-171. https://doi.org/10.11896/jsjkx.210400276
[12]	邓朝阳, 仲国强, 王栋. 基于注意力门控图神经网络的文本分类 Text Classification Based on Attention Gated Graph Neural Network 计算机科学, 2022, 49(6): 326-334. https://doi.org/10.11896/jsjkx.210400218
[13]	余皑欣, 冯秀芳, 孙静宇. 结合物品相似性的社交信任推荐算法 Social Trust Recommendation Algorithm Combining Item Similarity 计算机科学, 2022, 49(5): 144-151. https://doi.org/10.11896/jsjkx.210300217
[14]	李勇, 吴京鹏, 张钟颖, 张强. 融合快速注意力机制的节点无特征网络链路预测算法 Link Prediction for Node Featureless Networks Based on Faster Attention Mechanism 计算机科学, 2022, 49(4): 43-48. https://doi.org/10.11896/jsjkx.210800276
[15]	曹合心, 赵亮, 李雪峰. 图神经网络在Text-to-SQL解析中的技术研究 Technical Research of Graph Neural Network for Text-to-SQL Parsing 计算机科学, 2022, 49(4): 110-115. https://doi.org/10.11896/jsjkx.210200173

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed