Computer Science

Computer Science ›› 2020, Vol. 47 ›› Issue (4): 157-163.doi: 10.11896/jsjkx.190300115

• Artificial Intelligence • Previous Articles     Next Articles

Survey of Implicit Discourse Relation Recognition Based on Deep Learning

HU Chao-wen1, YANG Ya-lian2, WU Chang-xing1   

  1. 1 Virtual Reality and Interactive Techniques Institute,East China Jiaotong University,Nanchang 330013,China;
    2 School of Software,East China Jiaotong University,Nanchang 330013,China
  • Received:2019-03-24 Online:2020-04-15 Published:2020-04-15
  • Contact: WU Chang-xing,born in 1981,Ph.D,lecturer,is a member of CCF.His main research interests include nature language processing and machine learning.
  • About author:HU Chao-wen,born in 1993,master candidate.His main research interests include deep learning and nature language processing.
  • Supported by:
    This work was supported by the National Natural Science Foundation of China(61866012),Natural Science Foundation of Jiangxi Province (20181BAB202012) and Science and Technology Research Project of Jiangxi Education Department(GJJ180329)

PDF (PC)

1373

Abstract: Implicit discourse relation recognition is still a challenging task in natural language processing.It aims to discover the semantic relations (such as transition) between two arguments (e.g.clauses or sentences) where discourse connectives are absent.In recent years,with the extensive application of deep learning in natural language processing,various methods based on deep learning have achieved promising results on implicit discourse relation recognition.Their performance is much better than that of previous methods based on manual features.This paper discussed recent implicit discourse recognition methods in three categories:argument encoding based methods,argument interaction based methods and semi-supervised methods with explicit discourse data.Results on the PDTB data set show that,by explicitly modeling the semantic relation between words or text spans in two arguments,the performance of argument interaction based methods is significantly better than that of argument encoding based methods,and by incorporating explicit discourse data,the semi-supervised methods can effectively alleviate the problem of data sparsity,and then further improve the recognition performance.Lastly,this paper analyzed the major problems faced at pre-sent,and pointed out the possible research directions.

Key words: Deep learning, Implicit discourse relation recognition, Natural language processing

CLC Number: 

  • TP391.1
[1]PITLER E,NENKOVA A.Revisiting Readability:A UnifiedFramework for Predicting Text Quality[C]//Proceedings of the 2008 Conference on Empirical Methods in Natural Language Processing.Honolulu,Hawaii:Association for Computational Linguistics,2008:186-195.
[2]LIN Z,NG H T,KAN M Y.Automatically Evaluating Text Coherence Using Discourse Relations[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics:Human Language Technologies-Volume 1.Association for Computational Linguistics,2011:997-1006.
[3]PITLER E,NENKOVA A.Using Syntax to Disambiguate Explicit Discourse Connectives in Text[C]//Proceedings of the ACL-IJCNLP 2009.Suntec,Singapore:Association for Computational Linguistics,2009:13-16.
[4]LI Y C,SUN J,ZHOU G D.Automatic Recognition and Classification on Chinese Discourse Connective[J].Acta Scientiarum Naturalium Universitatis Pekinensis,2015,51(2):307-314.
[5]PRASAD R,DINESH N,LEE A,et al.The Penn DiscourseTreeBank 2.0[C]//Proceedings of the 6th Conference of the Language Resources and Evaluation.2008:2961-2968.
[6]RUTHERFORD A,DEMBERG V,XUE N.A Systematic Studyof Neural Discourse Models for Implicit Discourse Relation[C]//Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics.2017:281-291.
[7]LAN M,WANG J,WU Y,et al.Multi-task Attention-basedNeural Networks for Implicit Discourse Relationship Representation and Identification[C]//Proceedings of the 2017 Confe-rence on Empirical Methods in Natural Language Processing.Copenhagen,Denmark:Association for Computational Linguistics,2017:1299-1308.
[8]WU C X.Semi-supervised Implicit Discourse Relation Recognition[D].Xiamen:Xiamen Uuniversity,2017.
[9]TU M,ZHOU Y,ZONG C.Enhancing Grammatical Cohesion:Generating Transitional Expressions for SMT[C]//Proceedings of the 52nd Annual Meeting of the Association for Computatio-nal Linguistics.2014:850-860.
[10]ZHOU L,LI B,GAO W,et al.Unsupervised Discovery of Discourse Relations for Eliminating Intra-sentence Polarity Ambiguities[C]//Proceedings of the Conference on Empirical Me-thods in Natural Language Processing.Association for Computational Linguistics,2011:162-171.
[11]VERBERNE S,BOVES L,OOSTDIJK N,et al.Evaluating Discourse-based Answer Extraction for Why Question Answering[C]//Proceedings of the 30th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR ’07).Amsterdam,ACM Press,2007:735.
[12]MARCU D,ECHIHABI A.An Unsupervised Approach to Recognizing Discourse Relations[C]//Proceedings of 40th Annual Meeting of The Association for Computational Linguistics.Philadelphia,Pennsylvania,USA:Association for Computational Linguistics,2002:368-375.
[13]RUTHERFORD A,XUE N.Discovering Implicit Discourse Relations Through Brown Cluster Pair Representation and Coreference Patterns[C]//Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics.Gothenburg,Sweden:Association for Computational Linguistics,2014:645-654.
[14]WELLNER B,PUSTEJOVSKY J,HAVASI C,et al.Classification of Discourse Coherence Relations:An Exploratory Study using Multiple Knowledge Sources[C]//Proceedings of the 7th SIGdial Workshop on Discourse and Dialogue.Sydney,Austra-lia:Association for Computational Linguistics,2006:117-125.
[15]LIN Z,KAN M Y,NG H T.Recognizing Implicit Discourse Relations in the Penn Discourse Treebank[C]//Proceedings of the 2009 Conference on Empirical Methods in Natural Language Processing.Singapore:Association for Computational Linguistics,2009:343-351.
[16]PITLER E,LOUIS A,NENKOVA A.Automatic Sense Prediction for Implicit Discourse Relations in Text[C]//Proceedings of the Joint Conference of the 47th Annual Meeting of the ACL and the 4th International Joint Conference on Natural Language Processing of the AFNLP.Suntec,Singapore:Association for Computational Linguistics,2009:683-691.
[17]XU F,ZHU Q M,ZHOU G D.Implicit Discourse Relation Recognition Based on Tree Kernel[J].Acta Journal of Software,2013,24(5):1022-1035.
[18]ZHANG M Y,SONG Y,QIN B,et al.Chinese Discourse Relation Recognition[J].Journal of Chinese Information processing,2013,27(6):51-58.
[19]BAI H,ZHAO H.Deep Enhanced Representation for ImplicitDiscourse Relation Recognition[C]//Proceedings of the 27th International Conference on Computational Linguistics.Santa Fe,New Mexico,USA:Association for Computational Linguistics,2018:571-583.
[20]DAI Z,HUANG R.Improving Implicit Discourse Relation Classification by Modeling Inter-dependencies of Discourse Units in a Paragraph[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.New Orleans,Louisiana:Association for Computational Linguistics,2018:141-151.
[21]ZHANG B,SU J,XIONG D,et al.Shallow Convolutional Neural Network for Implicit Discourse Relation Recognition[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.Lisbon,Portugal:Association for Computational Linguistics,2015:2230-2235.
[22]SAK H,SENIOR A,BEAUFAYS F.Long Short-term Memory Recurrent Neural Network Architectures for Large Scale Acoustic Modeling[C]//Fifteenth Annual Conference of the International Speech Communication Association.2014:338-342.
[23]JI Y,EISENSTEIN J.One Vector is Not Enough:Entity-Augmented Distributed Semantics for Discourse Relations[J].Transactions of the Association for Computational Linguistics,2015,3:329-344.
[24]WANG Y,LI S,YANG J,et al.Tag-Enhanced Tree-Structured Neural Networks for Implicit Discourse Relation Classification[C]//Proceedings of the Eighth International Joint Conference on Natural Language Processing.2017:496-505.
[25]TAI K S,SOCHER R,MANNING C D.Improved SemanticRepresentations From Tree-Structured Long Short-Term Memory Networks[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing.Beijing,China:Association for Computational Linguistics,2015:1556-1566.
[26]QIN L,ZHANG Z,ZHAO H.Implicit Discourse Relation Recognition with Context-aware Character-enhanced Embeddings[C]//the 26th International Conference on Computational Linguistics.Osaka,Japan:The COLING 2016 Organizing Committee,2016:1914-1924.
[27]ZHANG B,XIONG D,SU J,et al.Learning Better Discourse Representation for Implicit Discourse Relation Recognition via Attention Networks[J].Neurocomputing,2018,275:1241-1249.
[28]KISHIMOTO Y,MURAWAKI Y,KUROHASHI S.A Knowledge-Augmented Neural Network Model for Implicit Discourse Relation Classification[C]//Proceedings of the 27th International Conference on Computational Linguistics.Santa Fe,New Mexico,USA:Association for Computational Linguistics,2018:584-595.
[29]CHEN J,ZHANG Q,LIU P,et al.Implicit Discourse Relation Detection via a Deep Architecture with Gated Relevance Network[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.Berlin,Germany:Association for Computational Linguistics,2016:1726-1735.
[30]LEI W,WANG X,LIU M,et al.SWIM:A Simple Word Interaction Model for Implicit Discourse Relation Recognition[C]//International Joint Conferences on Artificial Intelligence Organization.2017:4026-4032.
[31]LIU Y,LI S,ZHANG X,et al.Implicit Discourse Relation Classification via Multi-task Neural Networks[C]//Thirtieth AAAI Conference on Artificial Intelligence.2016:2750-2756.
[32]LIU Y,LI S.Recognizing Implicit Discourse Relations via Repeated Reading:Neural Networks with Multi-Level Attention[C]//Association for Computational Linguistics.2016:1224-1233.
[33]RÖNNQVIST S,SCHENK N,CHIARCOS C.A RecurrentNeural Model with Attention for the Recognition of Chinese Implicit Discourse Relations[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Vancouver,Canada:Association for Computational Linguistics,2017:256-262.
[34]WU W,WANG H,LIU T,et al.Phrase-level Self-AttentionNetworks for Universal Sentence Encoding[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.Brussels,Belgium:Association for Computational Linguistics,2018:3729-3738.
[35]SPORLEDER C,LASCARIDES A.Using Automatically Labelled Examples to Classify Rhetorical Relations:an Assessment[J].Natural Language Engineering,2008,14(3):369-416.
[36]RUTHERFORD A,XUE N.Improving the Inference of Implicit Discourse Relations Via Classifying Explicit Discourse Connectives[C]//Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2015:799-808.
[37]WU C,SHI X,SU J,et al.Co-training for Implicit Discourse Relation Recognition Based on Manual and Distributed Features[J].Neural Processing Letters,2017,46(1):233-250.
[38]XU Y,HONG Y,RUAN H,et al.Using Active Learning to Expand Training Data for Implicit Discourse Relation Recognition[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.Brussels,Belgium:Association for Computational Linguistics,2018:725-731.
[39]CARLSON L,MARCU D,OKUROWSKI M E.Building a discourse-tagged corpus in the framework of rhetorical structure theory//Current and new directions in discourse and dialogue.Dordrecht:Springer,2003:85-112.
[40]WU C,SHI X D,CHEN Y,et al.Bilingually-constrained Synthetic Data for Implicit Discourse Relation Recognition[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.Austin,Texas:Association for Computational Linguistics,2016:2306-2312.
[41]PAN S J,YANG Q.A Survey on Transfer Learning[J].IEEE Transactions on Knowledge and Data Engineering,2010,22(10):1345-1359.
[42]MIKOLOV T,SUTSKEVER I,CHEN K,et al.DistributedRepresentations of Words and Phrases and Their Compositio-nality[C]//Advances in Neural Information Processing Systems.2013:3111-3119.
[43]PENNINGTON J,SOCHER R,MANNING C.Glove:GlobalVectors for Word Representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Proces-sing (EMNLP).Doha,Qatar:Association for Computational Linguistics,2014:1532-1543.
[44]PETERS M,NEUMANN M,IYYER M,et al.Deep Contextualized Word Representations[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.New Orleans,Louisiana:Association for Computational Linguistics,2018:2227-2237.
[45]RADFORD A,NARASIMHAN K,SALIMANS T,et al.Improving Language Understanding by Generative Pre-training[OL].https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language unsupervised/ language understanding paper.pdf,2018.
[46]DEVLIN J,CHANG M W,LEE K,et al.BERT:Pre-training of Deep Bidirectional Transformers for Language Understanding[J].arXiv:1810.04805,2018.
[47]BRAUD C,DENIS P.Learning Connective-based Word Representations for Implicit Discourse Relation Identification[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.Austin,Texas:Association for Computational Linguistics,2016:203-213.
[48]TURNEY P D,PANTEL P.From Frequency to Meaning:Vector Space Models of Semantics[J].Journal of Artificial Intelligence Research,2010,37:141-188.
[49]WU C,SHI X D,CHEN Y,et al.Improving Implicit Discourse Relation Recognition with Discourse-specific Word Embeddings[C]//Association for Computational Linguistics.2017:269-274.
[50]PARK J,CARDIE C.Improving Implicit Discourse RelationRecognition Through Feature Set Optimization[C]// Procee-dings of the 13th Annual Meeting of the Special Interest Group on Discourse and Dialogue.Association for Computational Linguistics,2012:108-112.
[51]MCCANN B,BRADBURY J,XIONG C,et al.Learned inTranslation:Contextualized Word Vectors[C]//Advances in Neural Information Processing Systems.2017:6294-6305.
[52]HOWARD J,RUDER S.Universal Language Model Fine-tuning for Text Classification[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics.2018:328-339.
[53]PARK J,CARDIE C.Building Chinese Discourse Corpus with Connective-driven Dependency Tree Structure[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP).Doha,Qatar:Association for Computational Linguistics,2014:2105-2114.
[1] XU Yong-xin, ZHAO Jun-feng, WANG Ya-sha, XIE Bing, YANG Kai. Temporal Knowledge Graph Representation Learning [J]. Computer Science, 2022, 49(9): 162-171.
[2] RAO Zhi-shuang, JIA Zhen, ZHANG Fan, LI Tian-rui. Key-Value Relational Memory Networks for Question Answering over Knowledge Graph [J]. Computer Science, 2022, 49(9): 202-207.
[3] TANG Ling-tao, WANG Di, ZHANG Lu-fei, LIU Sheng-yun. Federated Learning Scheme Based on Secure Multi-party Computation and Differential Privacy [J]. Computer Science, 2022, 49(9): 297-305.
[4] WANG Jian, PENG Yu-qi, ZHAO Yu-fei, YANG Jian. Survey of Social Network Public Opinion Information Extraction Based on Deep Learning [J]. Computer Science, 2022, 49(8): 279-293.
[5] HAO Zhi-rong, CHEN Long, HUANG Jia-cheng. Class Discriminative Universal Adversarial Attack for Text Classification [J]. Computer Science, 2022, 49(8): 323-329.
[6] JIANG Meng-han, LI Shao-mei, ZHENG Hong-hao, ZHANG Jian-peng. Rumor Detection Model Based on Improved Position Embedding [J]. Computer Science, 2022, 49(8): 330-335.
[7] SUN Qi, JI Gen-lin, ZHANG Jie. Non-local Attention Based Generative Adversarial Network for Video Abnormal Event Detection [J]. Computer Science, 2022, 49(8): 172-177.
[8] YAN Jia-dan, JIA Cai-yan. Text Classification Method Based on Information Fusion of Dual-graph Neural Network [J]. Computer Science, 2022, 49(8): 230-236.
[9] HOU Yu-tao, ABULIZI Abudukelimu, ABUDUKELIMU Halidanmu. Advances in Chinese Pre-training Models [J]. Computer Science, 2022, 49(7): 148-163.
[10] ZHOU Hui, SHI Hao-chen, TU Yao-feng, HUANG Sheng-jun. Robust Deep Neural Network Learning Based on Active Sampling [J]. Computer Science, 2022, 49(7): 164-169.
[11] SU Dan-ning, CAO Gui-tao, WANG Yan-nan, WANG Hong, REN He. Survey of Deep Learning for Radar Emitter Identification Based on Small Sample [J]. Computer Science, 2022, 49(7): 226-235.
[12] HU Yan-yu, ZHAO Long, DONG Xiang-jun. Two-stage Deep Feature Selection Extraction Algorithm for Cancer Classification [J]. Computer Science, 2022, 49(7): 73-78.
[13] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[14] WANG Jun-feng, LIU Fan, YANG Sai, LYU Tan-yue, CHEN Zhi-yu, XU Feng. Dam Crack Detection Based on Multi-source Transfer Learning [J]. Computer Science, 2022, 49(6A): 319-324.
[15] CHU Yu-chun, GONG Hang, Wang Xue-fang, LIU Pei-shun. Study on Knowledge Distillation of Target Detection Algorithm Based on YOLOv4 [J]. Computer Science, 2022, 49(6A): 337-344.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.