计算机科学 ›› 2021, Vol. 48 ›› Issue (6A): 295-298.doi: 10.11896/jsjkx.200500019

• 智能计算 • 上一篇    下一篇

基于Transformer模型与关系词特征的汉语因果类复句关系自动识别

杨进才, 曹元, 胡泉, 沈显君   

  1. 华中师范大学计算机学院 武汉430079
  • 出版日期:2021-06-10 发布日期:2021-06-17
  • 通讯作者: 杨进才(jcyang@mail.ccnu.edu.cn)
  • 基金资助:
    国家社科基金(19BYY092)

Relation Classification of Chinese Causal Compound Sentences Based on Transformer Model and Relational Word Feature

YANG Jin-cai, CAO Yuan, HU Quan, SHEN Xian-jun   

  1. School of Computer Science,Central China Normal University,Wuhan 430079,China
  • Online:2021-06-10 Published:2021-06-17
  • About author:YANG Jin-cai,born in 1967,Ph.D,professor,Ph.D supervisor,is a member of China Computer Federation.His main research interests include advanced database and information system,Chinese information processing,artificial intelligence and natural language processing.
  • Supported by:
    National Social Science Fundation of China(19BYY092).

摘要: 汉语复句的语义关系丰富而复杂,复句关系自动识别是对复句语义关系的判别,是分析复句所表达意义的重要环节。因果类复句是使用最多的汉语复句,文中以二句式有标因果类复句为研究对象,通过深度学习的方法自动挖掘复句隐含的特征,同时融合了关系词这一语言学研究的显著知识。将word2vec词向量与one-hot编码的关系词特征结合作为模型的输入,利用卷积神经网络作为前馈层的transformer模型来对因果复句关系进行识别。采用文中的方法对因果类复句关系类别进行识别,实验结果的F1值达到92.13%,优于现有的对比模型,表明了该方法的有效性。

关键词: transformer模型, 词向量, 关系识别, 深度学习, 因果复句

Abstract: Chinese compound sentences have rich and complicated semantic relations.Recognition of relation category of a Chinese compound sentence is judgment of semantic relations of the sentence,and it is very important to analyze the meaning of compound sentences.Causal compound sentences are the most frequently used sentences in Chinese article.In this paper,a corpus of causal compound sentences with two clauses is taken as research object.A deep learning method is used in order to mine the hidden features of compound sentences automatically.At the same time,this paperintegrates significant linguistics knowledge of relational words in the proposed model.Combining word vector of word2vec and relation word feature of one hot coding as input to the model,a transformer model using convolutional neural network as feedforward layer is exploited to identify the relation category of causal compound sentences.Using our model,the F1 value of the experiment reaches 92.13%,which is better than the compa-rative model.

Key words: Causal compound sentences, Deep learning, Relation classification, Transformer model, Word vector

中图分类号: 

  • TP391
[1] JOTY S,GUZMÁN F,MÀRQUEZ L I,et al.Using Discourse Structure for Machine Translation Evaluation[C]//ACL Procee-dings of the 45th Annual Meeting of the Association for Computational Linguistics.2014:687-698.
[2] HUTTUNEN S,VIHAVAINEN A,VON ETTER P,et al.Re-levance Prediction in Information Extraction using Discourse and Lexical Features[C]//Proceedings of the 18th Nordic Confe-rence of Computational Linguistics(NODALIDA 2011).2011:114-121.
[3] VERBERNE S,BOVES L,OOSTDIJK N,et al.Evaluating discourse-based answer extraction for why -question answering[C] //ACM.2007.
[4] RONG L H,ZHOU G D.An Overview of the Study on Chinese Cause-effect Complex Sentence [J].Journal of Changchun Normal University,2011,30(9):47-51.
[5] YANG J C,CHEN Z Z,SHENG X J,et al.Automatic recognition of relation category of non-saturated compound sentences with two clauses [J].Application Research of Computers,2017,34(10):2950-2953.
[6] YANG J C,CHEN Z Z,SHENG X J,et al.Word Semantic Relevancy Computation and Categories Identification[J].Application Research of Computers,2017,44(5):280-284.
[7] HUANG H,CHANG T,CHEN H,et al.Interpretation of Chinese Discourse Connectives for Explicit Discourse Relation Re-cognition[C]//the 25th International Conference on Computational Linguistics.2014,15(1):632-643.
[8] XI X F,ZHOU G D.A Survey on Deep Learning for NaturalLanguage Processing[J].Acta Automatica Sinica,2016,42(10):1445-1465.
[9] RÖNNQVIST S,SCHENK N,CHIARCOS C.A RecurrentNeural Model with Attention for the Recognition of Chinese Implicit Discourse Relations[EB/OL].[2017-04-26].https://arxiv.org/abs/1704.08092.
[10] ZHOU Y,XUE N.The Chinese Discourse TreeBank:a Chinese corpus annotated with discourse relations[J].Language Resources and Evaluation,2015,49(2):397-431.
[11] QIN L,ZHANG Z,ZHAO H.Shallow Discourse Parsing Using Convolutional Neural Network[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.2016:70-77.
[12] XING F Y.A Research on Chinese Compound Sentences[M].Beijing:Commercial Press,2003.
[13] VASWANI A,SHAZEER N.Attention Is All You Need[C] //31st Conference on Neural Information Processing Systems.2017.
[14] IOFFE S,SZEGEDY C.Batch Normalization:Accelerating Deep Network Training by Reducing Internal Covariate Shift[EB/OL].[2015-03-02].https://arxiv.org/abs/1502.03167.
[15] MIKOLOV T,SUTSKEVER I,CHEN K,et al.DistributedRepresentations of Words and Phrases and their Compositionality[J].Advances in Neural Information Processing Systems,2013,26:3111-3119.
[16] SRIVASTAVA N,HINTON G,KRIZHEVSKY A,et al.Dropout:A Simple Way to Prevent Neural Networks from Overfitting[J].Journal of Machine Learning Research,2014,15(1):1929-1958.
[17] CARUANA R,LAWRENCE S,GILES L.Overfitting in Neural Nets:Backpropagation,Conjugate Gradient,and Early Stopping[C]//Proceedings of the 13th International Conference on Neural Information Processing Systems(NIPS'00).2001:381-387.
[18] CARO L D,GRELLA M.Sentiment analysis via dependencyparsing[J].Computer Standards & Interfaces,2013,35(5):442-453.
[19] NIEPERT M,AHMED M,KUTZKOV K.Learning Convolu-tional Neural Networks for Graphs [EB/OL].[2016-05-17].https://arxiv.org/abs/1605.05273.
[1] 饶志双, 贾真, 张凡, 李天瑞.
基于Key-Value关联记忆网络的知识图谱问答方法
Key-Value Relational Memory Networks for Question Answering over Knowledge Graph
计算机科学, 2022, 49(9): 202-207. https://doi.org/10.11896/jsjkx.220300277
[2] 汤凌韬, 王迪, 张鲁飞, 刘盛云.
基于安全多方计算和差分隐私的联邦学习方案
Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy
计算机科学, 2022, 49(9): 297-305. https://doi.org/10.11896/jsjkx.210800108
[3] 徐涌鑫, 赵俊峰, 王亚沙, 谢冰, 杨恺.
时序知识图谱表示学习
Temporal Knowledge Graph Representation Learning
计算机科学, 2022, 49(9): 162-171. https://doi.org/10.11896/jsjkx.220500204
[4] 王剑, 彭雨琦, 赵宇斐, 杨健.
基于深度学习的社交网络舆情信息抽取方法综述
Survey of Social Network Public Opinion Information Extraction Based on Deep Learning
计算机科学, 2022, 49(8): 279-293. https://doi.org/10.11896/jsjkx.220300099
[5] 郝志荣, 陈龙, 黄嘉成.
面向文本分类的类别区分式通用对抗攻击方法
Class Discriminative Universal Adversarial Attack for Text Classification
计算机科学, 2022, 49(8): 323-329. https://doi.org/10.11896/jsjkx.220200077
[6] 姜梦函, 李邵梅, 郑洪浩, 张建朋.
基于改进位置编码的谣言检测模型
Rumor Detection Model Based on Improved Position Embedding
计算机科学, 2022, 49(8): 330-335. https://doi.org/10.11896/jsjkx.210600046
[7] 孙奇, 吉根林, 张杰.
基于非局部注意力生成对抗网络的视频异常事件检测方法
Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection
计算机科学, 2022, 49(8): 172-177. https://doi.org/10.11896/jsjkx.210600061
[8] 胡艳羽, 赵龙, 董祥军.
一种用于癌症分类的两阶段深度特征选择提取算法
Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification
计算机科学, 2022, 49(7): 73-78. https://doi.org/10.11896/jsjkx.210500092
[9] 程成, 降爱莲.
基于多路径特征提取的实时语义分割方法
Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction
计算机科学, 2022, 49(7): 120-126. https://doi.org/10.11896/jsjkx.210500157
[10] 侯钰涛, 阿布都克力木·阿布力孜, 哈里旦木·阿布都克里木.
中文预训练模型研究进展
Advances in Chinese Pre-training Models
计算机科学, 2022, 49(7): 148-163. https://doi.org/10.11896/jsjkx.211200018
[11] 周慧, 施皓晨, 屠要峰, 黄圣君.
基于主动采样的深度鲁棒神经网络学习
Robust Deep Neural Network Learning Based on Active Sampling
计算机科学, 2022, 49(7): 164-169. https://doi.org/10.11896/jsjkx.210600044
[12] 苏丹宁, 曹桂涛, 王燕楠, 王宏, 任赫.
小样本雷达辐射源识别的深度学习方法综述
Survey of Deep Learning for Radar Emitter Identification Based on Small Sample
计算机科学, 2022, 49(7): 226-235. https://doi.org/10.11896/jsjkx.210600138
[13] 姜胜腾, 张亦弛, 罗鹏, 刘月玲, 曹阔, 赵海涛, 魏急波.
语义通信系统的性能度量指标分析
Analysis of Performance Metrics of Semantic Communication Systems
计算机科学, 2022, 49(7): 236-241. https://doi.org/10.11896/jsjkx.211200071
[14] 王君锋, 刘凡, 杨赛, 吕坦悦, 陈峙宇, 许峰.
基于多源迁移学习的大坝裂缝检测
Dam Crack Detection Based on Multi-source Transfer Learning
计算机科学, 2022, 49(6A): 319-324. https://doi.org/10.11896/jsjkx.210500124
[15] 楚玉春, 龚航, 王学芳, 刘培顺.
基于YOLOv4的目标检测知识蒸馏算法研究
Study on Knowledge Distillation of Target Detection Algorithm Based on YOLOv4
计算机科学, 2022, 49(6A): 337-344. https://doi.org/10.11896/jsjkx.210600204
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!