计算机科学 ›› 2024, Vol. 51 ›› Issue (7): 303-309.doi: 10.11896/jsjkx.230400164
魏子昂, 彭舰, 黄飞虎, 琚生根
WEI Ziang, PENG Jian, HUANG Feihu, JU Shenggen
摘要： 文本分类是自然语言处理中的一个重要问题,其目的是将标签分配给输入的文档。在文本分类任务中,单词间的共现关系提供了文本特性及词汇分布的重要视角,而词嵌入信息能提供丰富的语义信息,并对全局词汇交互和潜在语义关系造成影响。然而,过去的研究未能有效整合这两方面,或过度关注其中一方面。在这样的背景下,文中提出了一种新的方法,用于自适应地融合这两类信息,在考虑结构关系和嵌入信息的同时,找到一个合理的平衡以提高模型效果。该模型首先从词汇共现模式和语义嵌入信息的角度将文本数据构建成文本共现图和文本嵌入图,利用图卷积来增强节点嵌入,图池化层融合节点嵌入并识别保留重要性更高的节点,遵循分层池化模式并按层学习文档级表示,并引入门控融合模块对两个图的嵌入进行自适应的融合。在5个公开的文本分类数据集上进行了大量实验,结果表明了HTGNN在文本分类任务上的优异性能。
中图分类号:
| [1]KOLANU N,BROWN A S,BEECH A,et al.Natural languageprocessing of radiology reports for the identification of patients with fracture[J].Archives of Osteoporosis,2021,16:1-8. [2]JACOVI A,SHALOM O S,GOLDBERG Y.Understandingconvolutional neural networks for text classification[J].arXiv:1809.08037,2018. [3]KALCHBRENNER N,GREFENSTETTE E,BLUNSOM P.A convolutional neural network for modelling sentences[J].arXiv:1404.2188,2014. [4]MIKOLOV T,KARAFIÁT M,BURGET L,et al.Recurrentneural network based language model [C]//Interspeech.2010:1045-1048. [5]YANG Z,YANG D,DYER C,et al.Hierarchical attention networks for document classification[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2016:1480-1489. [6]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[C]//Advances in Neural Information Processing Systems,2017:6000-6010. [7]YAO L,MAO C,LUO Y.Graph convolutional networks fortext classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2019:7370-7377. [8]HUANG L,MA D,LI S,et al.Text level graph neural network for text classification[J].arXiv:1910.02356,2019. [9]WANG Y,WANG S,YAO Q,et al.Hierarchical heterogeneous graph representation learning for short text classification[J].arXiv:2111.00180,2021. [10]CHEN Y Z,LIU X S,SUN L T,et al.Social Network Influence Prediction Algorithm Based on Graph Neural Network[J].Journal of Nanjing University(Natural Science),2022,58(3):386-397. [11]MA Y Q,CAI M L,CHEN M,et al.Drug Interaction Prediction Method Based on Graph Neural Network[J].Computer Know-ledge and Technology,2022,18(18):61-63. [12]WANG S,LI Y,ZHANG J,et al.Pm2.5-gnn:A domain know-ledge enhanced graph neural network for pm2.5 forecasting [C]//Proceedings of the 28th International Conference on Advances in Geographic Information Systems.2020:163-166. [13]GILMER J,SCHOENHOLZ S S,RILEY P F,et al.Neural message passing for quantum chemistry[C]//International Confe-rence on Cachine Learning.PMLR,2017:1263-1272. [14]ZHANG M,CUI Z,NEUMANN M,et al.An end-to-end deep learning architecture for graph classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2018. [15]YING Z,YOU J,MORRIS C,et al.Hierarchical graph representation learning with differentiable pooling[C]//Advances in Neural Information Processing Systems,2018:4805-4815. [16]MA Y,WANG S,AGGARWAL C C,et al.Graph convolutional networks with eigenpooling [C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery Data Mining.2019:723-731. [17]LEE J,LEE I,KANG J.Self-attention graph pooling [C]//International Conference on Machine Learning.PMLR,2019:3734-3743. [18]ZHU Z L,RAO Y,WU Y,et al.Research Progress of Attention Mechanism in Deep Learning[J].Journal of Chinese Information Processing,2019,33(6):1-11. [19]LIU G,GUO J.Bidirectional LSTM with attention mechanism and convolutional layer for text classification[J].Neurocompu-ting,2019,337:325338. [20]SHEN T,ZHOU T,LONG G,et al.Disan:Directional self-attention network for rnn/cnn-free language understanding [C]//Proceedings of the AAAI conference on Artificial Intelligence.2018. [21]VELIKOVI P,CUCURULL G,CASANOVA A,et al.Graph attention networks[J].arXiv:1710.10903,2017. [22]ZHANG J,SHI X,XIE J,et al.Gaan:Gated attention networks for learning on large and spatiotemporal graphs[J].arXiv:1803.07294,2018. [23]KINGMA D P,BA J.Adam:A method for stochastic optimization[J].arXiv:1412.6980,2014. [24]GLOROT X,BENGIO Y.Understanding the difficulty of trai-ning deep feedforward neural networks[C]//Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics.JMLR Workshop and Conference Proceedings,2010:249-256. [25]HINTON G E,SRIVASTAVA N,KRIZHEVSKY A,et al.Improving neural networks by preventing co-adaptation of feature detectors[J].arXiv:1207.0580,2012. [26]JOULIN A,GRAVE E,BOJANOWSKI P,et al.Bag of tricks for efficient text classification[J].arXiv:1607.01759,2016. [27]SHEN D,WANG G,WANG W,et al.Baseline needs more love:On simple word-embedding-based models and associated pooling mechanisms[J].arXiv:1805.09843,2018. [28]LIU X,YOU X,ZHANG X,et al.Tensor graph convolutional networks for text classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:8409-8416. [29]RANJAN E,SANYAL S,TALUKDAR P.Asap:Adaptivestructure aware pooling for learning hierarchical graph representations [C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:5470-5477. [30]KIPF T N,WELLING M.Semi-supervised classification withgraph convolutional networks[J].arXiv:1609.02907,2016. [31]HAMILTON W,YING Z,LESKOVEC J.Inductive representation learning on large graphs[C]//Advances in Neural Information Processing Systems.2017:1025-1035. [32]QIAO L,ZHANG L,CHEN S,et al.Data-driven graph construction and graph learning:A review[J].Neurocomputing,2018,312:336-351. |
|
||
首页