计算机科学

计算机科学 ›› 2024, Vol. 51 ›› Issue (6A): 230900018-5.doi: 10.11896/jsjkx.230900018

• 人工智能 • 上一篇    下一篇

基于不变图卷积神经网络的文本分类

黄瑞1, 徐计2   

  1. 1 贵州大学计算机科学与技术学院 贵阳 550025
    2 贵州大学省部共建公共大数据国家重点实验室 贵阳 550025
  • 发布日期:2024-06-06
  • 通讯作者: 徐计(jixu@gzu.edu.cn)
  • 作者简介:(gs.huangr21@gzu.edu.cn)
  • 基金资助:
    国家自然科学基金(61966005,62366008)

Text Classification Based on Invariant Graph Convolutional Neural Networks

HUANG Rui1, XU Ji2   

  1. 1 College of Computer Science and Technology,Guizhou University,Guiyang 550025,China
    2 State Key Laboratory of Public Big Data,Guizhou University,Guiyang 550025,China
  • Published:2024-06-06
  • About author:HUANG Rui,born in 1999,postgra-duate,is a member of CCF(No.N8705G).Her main research interests include machine learning and natural language processing.
    XU Ji,born in 1979,Ph.D,professor,is a member of CCF(No.12919M).His main research interests include data mining,granular computing and machine learning.
  • Supported by:
    National Natural Science Foundation of China(61966005,62366008).

PDF (PC)

305

摘要： 文本分类是自然语言处理中一个基本而又重要的任务,近年来,图神经网络被越来越多地应用于文本分类中。然而,使用图神经网络的图表示学习在涉及文本分类的任务中不能很好地满足新词的归纳学习,其一般假设训练和测试数据来自相同的分布,但现实中这个假设经常不成立。为了克服这些问题,文中提出了Invariant-GCN,用于通过GCN进行归纳文本分类。首先为每个文档构建单个图,使用GCN根据其局部结构学习细粒度的单词表示,这可以有效地为新文档中没见过的单词生成嵌入进而将单词节点作为文档嵌入合并;然后提取最大限度地保留不变类内信息的期望子图,使用这些子图进行学习不受分布变化的影响;最后通过图分类方法完成文本分类。在4个基准数据集上与5种分类方法进行了比较,实验结果表明Invariant-GCN具有良好的文本分类效果。

关键词: 文本分类, 图卷积神经网络, 因果学习, 文本图构建

Abstract: Text classification is a basic and important task in natural language processing,and graph neural networks have been applied to this task in recent years.However,the graph representation learning using graph neural networks can not well satisfy the generalization learning of new words in the task involving text classification.It is generally assumed that training and testing data come from the same distribution,which is often invalid in reality.To overcome these problems,this paper puts forward the Invariant-GCN,which is used for text categorization by GCN reported.First,to build a single figure for each document,use GCN to learn fine-grained word representation according to its local structure,which can effectivelygenerate embeddings for words not seen in the new document and then merge the word nodes as document embeddings.And then extract the maximum limit retained within the same class information expectations subgraph,use the graph to study is not affected by the distribution change.Finally,the text classification is completed by graph classification method.In four benchmark datasets,the the Invariant-GCN is compared with five classification methods,and the experimental results show that it has a good effect of text categorization.

Key words: Text classification, Graph convolutional neural network, Casual learning, Text graph construction

中图分类号: 

  • TP391
[1]ROY P K,SINGH J P,BANERJEE S.Deep learning to filterSMS Spam[J].Future Generation Computer Systems,2020,102:524-533.
[2]ZHANG L,WANG S,LIU B.Deep learning for sentiment analysis:A survey[J].Wiley Interdisciplinary Reviews:Data Mining and Knowledge Discovery,2018,8(4):e1253.
[3]ZENG Z,HE K,YAN Y,et al.Modeling Discriminative Repre-sentations for Out-of-Domain Detection with SupervisedCon-trastive Learning[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing(Volume 2:Short Papers).2021:870-878.
[4]JOHNSON R,ZHANG T.Deep pyramid convolutional neural networks for text categorization[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics(Volume 1:Long Papers).2017:562-570.
[5]ZHANG Y,LIU Q,SONG L.Sentence-State LSTM for Text Representation[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics(Volume 1:Long Papers).2018:317-327.
[6]WAN S,PAN S,YANG J,et al.Contrastive and generativegraph convolutional networks for graph-based semi-supervised learning[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2021,35(11):10049-10057.
[7]AHUJA K,CABALLERO E,ZHANG D,et al.Invariance principle meets information bottleneck for out-of-distribution gene-ralization[J].Advances in Neural Information Processing Systems,2021,34:3438-3450.
[8]MCCALLUM A,NIGAM K.A comparison of event models fornaive bayes text classification[C]//AAAI-98 Workshop on Learning for Text Categorization.1998:41-48.
[9]JOACHIMS T.Text categorization with support vector ma-chines:Learning with many relevant features[C]//European Conference on Machine Learning.Berlin,Springer,1998:137-142.
[10]COVER T,HART P.Nearest neighbor pattern classification[J].IEEE Transactions on Information Theory,1967,13(1):21-27.
[11]KIM Y.Convolutional Neural Networks for Sentence Classification[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing(EMNLP).Association for Computational Linguistics,2014.
[12]LIU P,QIU X,HUANG X.Recurrent neural network for text classification with multi-task learning[C]//Proceedings of the Twenty-Fifth International Joint Conference on Artificial Intelligence.2016:2873-2879.
[13]WANG P,XU B,XU J,et al.Semantic expansion using word embedding clustering and convolutional neural network for improving short text classification[J].Neurocomputing,2016,174:806-814.
[14]YAO L,MAO C,LUO Y.Graph convolutional networks fortext classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2019:7370-7377.
[15]HUANG L,MA D,LI S,et al.Text Level Graph Neural Network for Text Classification[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing(EMNLP-IJCNLP).2019:3444-3450.
[16]ZHANG Y,YU X,CUI Z,et al.Every Document Owns ItsStructure:Inductive Text Classification via Graph Neural Networks[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.2020:334-339.
[17]LIU X,YOU X,ZHANG X,et al.Tensor graph convolutional networks for text classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:8409-8416.
[18]JOULIN A,GRAVE É,BOJANOWSKI P,et al.Bag of Tricks for Efficient Text Classification[C]//Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics:Volume 2,Short Papers.2017:427-431.
[19]SHEN D,WANG G,WANG W,et al.Baseline Needs MoreLove:On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics(Vo-lume 1:Long Papers).Association for Computational Linguistics,2018.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发