基于图神经网络和依存句法分析的文本分类

doi:10.11896/jsjkx.220300195

Abstract

Abstract: Text classification is a basic and important task in natural language processing.It is widely used in language processing scenarios such as news classification,topic tagging and sentiment analysis.The current text classification models generally do not consider the co-occurrence relationship of text words and the syntactic characteristics of the text itself,thus limiting the effect of text classification.Therefore,a text classification model based on graph convolutional neural network(Mix-GCN) is proposed.Firstly,based on the co-occurrence relationship and syntactic dependency between text words,the text data is constructed into a text co-occurrence graph and a syntactic dependency graph.Then the GCN model is used to perform representation learning on the text graph and syntactic dependency graph,and the embedding vector of the word is obtained.Then the embedding vector of the text is obtained by graph pooling method and adaptive fusion method,and the text classification is completed by the graph classification method.Mix-GCN model simultaneously considers the relationship between adjacent words in the text and the syntactic dependencies existing between text words,which improves the performance of text classification.On 6 benchmark datasets,compared to 8 well-known text classification methods,experimental results show that Mix-GCN has a good text classification effect.

Key words: Text classification, Graph neural network, Dependency parsing, Graph classification

CLC Number:

TP391

YANG Xu-hua, JIN Xin, TAO Jin, MAO Jian-fei. Text Classification Based on Graph Neural Networks and Dependency Parsing[J].Computer Science, 2022, 49(12): 293-300.

References

[1]ROY P K,SINGH J P,BANERJEE S.Deep learning to filter SMS Spam[J].Future Generation Computer Systems,2020,102:524-533.
[2]MINAEE S,KALCHBRENNER N,CAMBRIA E,et al.DeepLearning--based Text Classification:A Comprehensive Review[J].ACM Computing Surveys,2021,54(3):1-40.
[3]XIONG S F,JI D H.A short text sentiment topic model for product review analysis[J].Acta Automatica Sinica,2016,42(8):1227-1237.
[4]JIN L,HE T T.Chinese text classification model based on improved TF-IDF and ABLCNN[J].Computer Science,2021,48(11A):170-175.
[5]HARRELL F E.Regression modeling strategies:with applications to linear models,logistic and ordinal regression,and survival analysis[M].New York:Springer,2015.
[6]RISH I.An empirical study of the naive Bayes classifier[C]//IJCAI 2001 Workshop on Empirical Methods in Artificial Intelligence.2001:41-46.
[7]KELLER J M,GRAY M R,GIVENS J A.A fuzzy k-nearest neighbor algorithm[J].IEEE Transactions on Systems,Man,and Cybernetics,1985(4):580-585.
[8]SANCHEZ AV D.Advanced support vector machines and kernel methods[J].Neurocomputing,2003,55(1/2):5-20.
[9]MIKOLOV T,CHEN K,CORRADO G,et al.Efficient estimation of word representations in vector space[J].arXiv:1301.3781,2013.
[10]KIM Y.Convolutional neural networks for sentence classification[J].arXiv:1408.5882,2014.
[11]JOHNSON R,TONG Z.Deep Pyramid Convolutional NeuralNetworks for Text Categorization[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.2017:562-570.
[12]MIKOLOV T,KARAFIAT M,BURGET L,et al.Recurrentneural network based language modeI[C]//Proceedings of the Annual Conference of the International Speech Communication Association.2010:1045-1048.
[13]ZHANG Y,LIU Q,SONG L.Sentence-State LSTM for Text Representation[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.2018:317-327.
[14]YANG M,ZHAO W,YE J,et al.Investigating capsule networks with dynamic routing for text classification[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.2018:3110-3119.
[15]LECUN Y,BENGIO Y,HINTON G.Deep learning[J].Nature,2015,521(7553):436-444.
[16]JOULIN A,GRAVE E,BOJANOWSKI P,et al.Bag of Tricks for Efficient Text Classification[C]//Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics:Volume 2,Short Papers.2017:427-431.
[17]ZHANG X,ZHAO J,LECUN Y.Character-level convolutional networks for text classification[J].Advances in Neural Information Processing Systems,2015,28:649-657.
[18]TANG D,QIN B,LIU T.Document modeling with gated recurrent neural network for sentiment classification[C]//Procee-dings of the Conference on Empirical Methods in Natural Language Processing.2015:1422-1432.
[19]LAI S,XU L,LIU K,et al.Recurrent convolutional neural networks for text classification[C]//Twenty-ninth AAAI Confe-rence on Artificial Intelligence.2015:2267-2273.
[20]WANG R,LI Z,CAO J,et al.Convolutional Recurrent Neural Networks for Text Classification[C]//2019 International Joint Conference on Neural Networks.2019:1-6.
[21]KIPF T N,WELLING M.Semi-supervised classification withgraph convolutional networks[J].arXiv:1609.02907,2016.
[22]GAO H,CHEN Y,JI S.Learning Graph Pooling and HybridConvolutional Operations for Text Representations[C]//World Wide Web Conference.2019:2743-2749.
[23]YAO L,MAO C,LUO Y.Graph convolutional networks fortext classification[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2019:7370-7377.
[24]PENG H,LI J,HE Y,et al.Large-scale hierarchical text classification with recursively regularized deep graph-cnn[C]//Proceedings of the 2018 World Wide Web Conference.2018:1063-1072.
[25]WU M,PAN S,ZHU X,et al.Domain-adversarial graph neural networks for text classification[C]//2019 IEEE International Conference on Data Mining.IEEE,2019:648-657.
[26]IVANOVA A,OEPEN S,ØVRELID L.Survey on parsing three dependency representations for English[C]//51st Annual Mee-ting of the Association for Computational Linguistics Procee-dings of the Student Research Workshop.2013:31-37.
[27]CHEN M,WEINBERGER K Q,SHA F.An alternative textrepresentation to TF-IDF and Bag-of-Words[J].arXiv:1301.6770,2013.
[28]ZHU Y,LI L,LUO L.Learning to classify short text with topic model and external knowledge[C]//International Conference on Knowledge Science,Engineering and Management.2013:493-503.
[29]MA X,JIN R,PAIK J Y,et al.LargeScale Text Classification with Efficient Word Embedding[C]//International Conference on Mobile and Wireless Technology.2017:465-469.
[30]BENGIO Y,DUCHARME R,VINCENT P,et al.A neuralprobabilistic language model[J].The Journal of Machine Lear-ning Research,2003,3:1137-1155.
[31]LE Q,MIKOLOV T.Distributed representations of sentencesand documents[C]//International Conference on Machine Learning.PMLR,2014:1188-1196.
[32]PENNINGTON J,SOCHER R,MANNING C D.Glove:Global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Proces-sing.2014:1532-1543.
[33]TAN T,PHIENTHRAKUL T.Sentiment Classification Using Document Embeddings Trained with Cosine Similarity[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics:Student Research Workshop.2019:407-414.
[34]PEROZZI B,AL-RFOU R,SKIENA S.Deepwalk:Online lear-ning of social representations[C]//Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Disco-very and Data Mining.2014:701-710.
[35]GOYAL P,FERRARA E.Graph embedding techniques,applica-tions,and performance:A survey[J].Knowledge-Based Systems,2018,151:78-94.
[36]ZHANG R,LI J T.A Survey of Research on Scene Segmentation Algorithms Based on Deep Learning[J].Computer Research and Development,2020,57(4):859-875.
[37]FEI H,LI F,LI B,et al.Encoder-decoder based unified semantic role labeling with label-aware syntax[C]//Proceedings of AAAI National Conference on Artificial Intelligence.2021:1479-1488.
[38]LI Y,JIN R,LUO Y.Classifying relations in clinical narratives using segment graph convolutional and recurrent neural networks(Seg-GCRNs)[J].Journal of the American Medical Informatics Association,2019,26(3):262-268.
[39]BASTINGS J,TITOV I,AZIZ W,et al.Graph convolutional encoders for syntax-aware neural machine translation[J].arXiv:1704.04675,2017.
[40]VELICKOVIC P,CUCURULL G,CASANOVA A,et al.Graph attention networks[J].arXiv:1710.10903,2017.
[41]HAMILTON W,YING Z,LESKOVEC J.Inductive representation learning on large graphs[J].arXiv:1706.02216,2017.
[42]KIM D,OH A.How to find your friendly neighborhood:Graph attention design with self-supervision[J].arXiv:2204.04879,2022.
[43]ALON U,YAHAV E.On the bottleneck of graph neural networks and its practical implications[J].arXiv:2006.05205,2020.
[44]ZHAO T,LIU Y,NEVES L,et al.Data Augmentation forGraph Neural Networks[C]//Proceedings of the AAAI Confe-rence on Artificial Intelligence.2021:11015-11023.
[45]LUONG M T,PHAM H,MANNING C D.Effective approaches to attention-based neural machine translation[J].arXiv:1508.04025,2015.
[46]CHO K,COURVILLE A,BENGIO Y.Describing multimediacontent using attention-based encoder-decoder networks[J].IEEE Transactions on Multimedia,2015,17(11):1875-1886.
[47]ZENG Y F,LAN T,WU Z F,et al.Aspect-level sentiment classification model based on dual-memory attention[J].Chinese Journal of Computers,2019,8:1845-1857.
[48]SUKHBAATAR S,WESTON J,FERGUS R.End-to-end me-mory networks[C]//Proceedings of the Advances in Neural Information Processing Systems.2015:2440-2448.
[49]WANG W,PAN S J,DAHLMEIER D,et al.Coupled multi-la-yer attentions for co-extraction of aspect and opinion terms[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2017.
[50]YANG Z,YANG D,DYER C,et al.Hierarchical attention networks for document classification[C]//Proceedings of the 2016 CONFERENce of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2016:1480-1489.
[51]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isall you need[C]//Proceedings of the Advances in Neural Information Processing Systems.2017:5998-6008.
[52]SUN Z,HUANG S,WEI H R,et al.Generating diverse translation by manipulating multi-head attention[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2020:8976-8983.
[53]MISRA D,NALAMADA T,ARASANIPALAI A U,et al.Rotate to attend:Convolutional triplet attention module[C]//Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision.2021:3139-3148.
[54]WU Z,PAN S,CHEN F,et al.A comprehensive survey ongraph neural networks[J].IEEE Transactions on Neural Networks and Learning Systems,2020,32(1):4-24.
[55]DAI Y,SHOU L,GONG M,et al.Graph Fusion Network for Text Classification[J].Knowledge-Based Systems,2022,236:107659.
[56]GUO B,ZHANG C,LIU J,et al.Improving text classification with weighted word embeddings via a multi-channel TextCNN model[J].Neurocomputing,2019,363:366-374.
[57]LIU H,CHEN G,LI P,et al.Multi-label text classification via joint learning from label embedding and label correlation[J].Neurocomputing,2021,460:385-398.
[58]XU J,CAI Y,WU X,et al.Incorporating context-relevant concepts into convolutional neural networks for short text classification[J].Neurocomputing,2020,386:42-53.
[59]WU Y,LI J,WU J,et al.Siamese capsule networks with global and local features for text classification[J].Neurocomputing,2020,390:88-98.
[60]SHEN D,WANG G,WANG W,et al.Baseline needs more love:On simple word-embedding-based models and associated pooling mechanisms[J].arXiv:1805.09843,2018.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Text Classification Based on Graph Neural Networks and Dependency Parsing

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 15

Metrics

Comments

Recommended 0

[1]	ZHOU Fang-quan, CHENG Wei-qing. Sequence Recommendation Based on Global Enhanced Graph Neural Network [J]. Computer Science, 2022, 49(9): 55-63.
[2]	WU Hong-xin, HAN Meng, CHEN Zhi-qiang, ZHANG Xi-long, LI Mu-hang. Survey of Multi-label Classification Based on Supervised and Semi-supervised Learning [J]. Computer Science, 2022, 49(8): 12-25.
[3]	TAN Ying-ying, WANG Jun-li, ZHANG Chao-bo. Review of Text Classification Methods Based on Graph Convolutional Network [J]. Computer Science, 2022, 49(8): 205-216.
[4]	YAN Jia-dan, JIA Cai-yan. Text Classification Method Based on Information Fusion of Dual-graph Neural Network [J]. Computer Science, 2022, 49(8): 230-236.
[5]	HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[6]	QI Xiu-xiu, WANG Jia-hao, LI Wen-xiong, ZHOU Fan. Fusion Algorithm for Matrix Completion Prediction Based on Probabilistic Meta-learning [J]. Computer Science, 2022, 49(7): 18-24.
[7]	YANG Bing-xin, GUO Yan-rong, HAO Shi-jie, Hong Ri-chang. Application of Graph Neural Network Based on Data Augmentation and Model Ensemble in Depression Recognition [J]. Computer Science, 2022, 49(7): 57-63.
[8]	DENG Kai, YANG Pin, LI Yi-zhou, YANG Xing, ZENG Fan-rui, ZHANG Zhen-yu. Fast and Transmissible Domain Knowledge Graph Construction Method [J]. Computer Science, 2022, 49(6A): 100-108.
[9]	KANG Yan, WU Zhi-wei, KOU Yong-qi, ZHANG Lan, XIE Si-yu, LI Hao. Deep Integrated Learning Software Requirement Classification Fusing Bert and Graph Convolution [J]. Computer Science, 2022, 49(6A): 150-158.
[10]	SHAO Xin-xin. TI-FastText Automatic Goods Classification Algorithm [J]. Computer Science, 2022, 49(6A): 206-210.
[11]	XIONG Zhong-min, SHU Gui-wen, GUO Huai-yu. Graph Neural Network Recommendation Model Integrating User Preferences [J]. Computer Science, 2022, 49(6): 165-171.
[12]	DENG Zhao-yang, ZHONG Guo-qiang, WANG Dong. Text Classification Based on Attention Gated Graph Neural Network [J]. Computer Science, 2022, 49(6): 326-334.
[13]	YU Ai-xin, FENG Xiu-fang, SUN Jing-yu. Social Trust Recommendation Algorithm Combining Item Similarity [J]. Computer Science, 2022, 49(5): 144-151.
[14]	LI Yong, WU Jing-peng, ZHANG Zhong-ying, ZHANG Qiang. Link Prediction for Node Featureless Networks Based on Faster Attention Mechanism [J]. Computer Science, 2022, 49(4): 43-48.
[15]	CAO He-xin, ZHAO Liang, LI Xue-feng. Technical Research of Graph Neural Network for Text-to-SQL Parsing [J]. Computer Science, 2022, 49(4): 110-115.