Computer Science ›› 2019, Vol. 46 ›› Issue (7): 7-12.doi: 10.11896/j.issn.1002-137X.2019.07.002

• Surveys • Previous Articles     Next Articles

Survey on Deep-learning-based Machine Reading Comprehension

LI Zhou-jun,WANG Chang-bao   

  1. (School of Computer Science and Engineering,Beihang University,Beijing 100191,China)
  • Received:2018-08-28 Online:2019-07-15 Published:2019-07-15

Abstract: Natural language processing is the key to achieving artificial intelligence.Machine reading comprehension,as the crown jewel in the field of natural language processing,has always been the focus of research in the field.With the rapid development of deep learning and neural network in recent years,machine reading comprehension has made great progress.Firstly,the research background and development history of machine reading comprehension were introduced.Then,by reviewing the important progress in the development of word vector,attention mechanism and answer prediction,the problems in recent research related to machine reading comprehension were proposed.Finally,the outlook of machine reading comprehension was discussed.

Key words: Natural language processing, Machine reading comprehension, Deep learning, Word vector, Attention mechanism

CLC Number: 

  • TP391
[1] HERMANN K M,KOCISKY T,GREFENSTETTE E,et al. Teaching machines to read and comprehend[C]∥Advances in Neural Information Processing Systems.Cambridge:MIT Press,2015:1693-1701.
[2] RAJPURKAR P,ZHANG J,LOPYREV K,et al.SQuAD: 100 000+ Questions for Machine Comprehension of Text[C]∥Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.Stroudsburg:ACL,2016:2383-2392.
[3] NGUYEN T,ROSENBERG M,SONG X,et al.MS MARCO:A human generated machine reading comprehension dataset [DB/OL].[2018-08-28].https://arxiv.org/abs/1611.09268.
[4] HIRSCHMAN L,LIGHT M,BRECK E,et al.Deep read:A reading comprehension system[C]∥Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics.Stroudsburg:ACL,1999:325-332.
[5] RILOFF E,THELEN M.A rule-based question answering system for reading comprehension tests[C]∥Proceedings of the 2000 ANLP/NAACL Workshop on Reading comprehension tests as evaluation for computer-based language understanding sytems-Volume 6.Stroudsburg:ACL,2000:13-19.
[6] POON H,CHRISTENSEN J,DOMINGOS P,et al.Machine reading at the university of washington[C]∥Proceedings of the NAACL HLT 2010 First International Workshop on Forma-lisms and Methodology for Learning by Reading.Association for Computational Linguistics.Stroudsburg:ACL,2010:87-95.
[7] RICHARDSON M,BURGES C J C,RENSHAW E.Mctest:A challenge dataset for the open-domain machine comprehension of text[C]∥Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing.Stroudsburg:ACL,2013:193-203.
[8] KADLEC R,SCHMID M,BAJGAR O,et al.Text understan- ding with the attention sum reader network [DB/OL].[2018-08-28].https://arxiv.org/abs/1603.01547.
[9] CHEN D,BOLTON J,MANNING C D.A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task[C]∥Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).Stroudsburg:ACL,2016:2358-2367.
[10] CUI Y,CHEN Z,WEI S,et al.Attention-over-Attention Neural Networks for Reading Comprehension[C]∥Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).Stroudsburg:ACL,2017:593-602.
[11] DHINGRA B,LIU H,YANG Z,et al.Gated-Attention Readers for Text Comprehension[C]∥Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Vo-lume 1:Long Papers).Stroudsburg:ACL,2017:1832-1846.
[12] WANG S,JIANG J.Machine comprehension using match-lstm and answer pointer [DB/OL].[2018-08-28].https://ar-xiv.org/abs/1608.07905.
[13] WANG S,JIANG J.Learning Natural Language Inference with LSTM[C]∥Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Stroudsburg:ACL,2016:1442-1451.
[14] VINYALS O,FORTUNATO M,JAITLY N.Pointer networks[C]∥Advances in Neural Information Processing Systems.Cambridge:MIT Press,2015:2692-2700.
[15] SEO,MINJOON,et al.Bidirectional attention flow for machine comprehension [DB/OL].[2018-08-28].https://arxiv.org/abs/1611.01603.
[16] XIONG C,ZHONG V,SOCHER R.Dynamic coattention network for question answering:U.S.Patent Application 15/421,193[P].2018-05-10.
[17] WANG W,YANG N,WEI F,et al.Gated self-matching networks for reading comprehension and question answering[C]∥Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).Stroudsburg:ACL,2017:189-198.
[18] JOSHI M,CHOI E,WELD D,et al.TriviaQA:A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension[C]∥Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).Stroudsburg:ACL,2017:1601-1611.
[19] HE W,LIU K,LYU Y,et al.DuReader:a Chinese Machine Reading Comprehension Dataset from Real-world Applications[DB/OL].[2018-08-28].https://arxiv.org/abs/1711.05073.
[20] TAN C,WEI F,YANG N,et al.S-net:From answer extraction to answer generation for machine reading comprehension [DB/OL].[2018-08-28].https://arxiv.org/abs/1706.04815.
[21] CLARK C,GARDNER M.Simple and Effective Multi-Para- graph Reading Comprehension[C]∥Meeting of the Association for Computational Linguistics.Stroudsburg:ACL,2018:845-855.
[22] WANG Y,LIU K,LIU J,et al.Multi-Passage Machine Reading Comprehension with Cross-Passage Answer Verification[C]∥Meeting of the Association for Computational Linguistics.Stroudsburg:ACL,2018:1918-1927.
[23] KOCISKY T,SCHWARZ J,BLUNSOM P,et al.The Narra- tiveQA Reading Comprehension Challenge[J].Transactions of the Association for Computational Linguistics,2018,6:317-328.
[24] DEERWESTER S,DUMAIS S T,FURNAS G W,et al.Indexing by latent semantic analysis[J].Journal of the American society for Information Science,1990,41(6):391-407.
[25] LUND K,BURGESS C.Producing high-dimensional semantic spaces from lexical co-occurrence[J].Behavior Research Me-thods,Instruments,& Computers,1996,28(2):203-208.
[26] ROHDE D L T,GONNERMAN L M,PLAUT D C.An improved model of semantic similarity based on lexical co-occurrence[J].Communications of the ACM,2006,8(627-633):116.
[27] BENGIO Y,DUCHARME R,VINCENT P,et al.A neural probabilistic language model[J].Journal of Machine Learning Research,2003,3:1137-1155.
[28] MIKOLOV T,SUTSKEVER I,CHEN K,et al.Distributed representations of words and phrases and their compositionality[C]∥dAdvancesf in Neural Information Processing Systems.Cambridge:MIT Press,2013:3111-3119.
[29] MIKOLOV T,CHEN K,CORRADO G,et al.Efficient estimation of word representations in vector space[J].arXiv:1301.3781,2013.
[30] PENNINGTON J,SOCHER R,MANNING C.Glove:Global vectors for word representation[C]∥Proceedings of the 2014 Conference on Empirical Methods in Natural Language Proces-sing (EMNLP).Stroudsburg:ACL,2014:1532-1543.
[31] MELAMUD O,GOLDBERGER J,DAGAN I.context2vec: Learning generic context embedding with bidirectional lstm[C]∥Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning.Stroudsburg:ACL,2016:51-61.
[32] MCCANN B,BRADBURY J,XIONG C,et al.Learned in translation:Contextualized word vectors[C]∥Advances in Neural Information Processing Systems.Cambridge:MIT Press,2017:6294-6305.
[33] PETERS M,NEUMANN M,IYYER M,et al.Deep Contextua- lized Word Representations[C]∥Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies(Vo-lume 1:Long Papers).Stroudsburg:ACL,2018:2227-2237.
[34] RENSINK R A.The dynamic representation of scenes[J].Vi- sual cognition,2000,7(1/2/3):17-42.
[35] MNIH V,HEESS N,GRAVES A.Recurrent models of visual attention[C]∥Advances in Neural Information Processing Systems.Stroudsburg:ACL,2014:2204-2212.
[36] BAHDANAU D,CHO K,BENGIO Y.Neural machine translation by jointly learning to align and translate [DB/OL].[2018-08-28].https://arxiv.org/abs/1409.0473.
[37] CUI Y,LIU T,CHEN Z,et al.Consensus Attention-based Neural Networks for Chinese Reading Comprehension[C]∥Proceedings of COLING 2016,the 26th International Conference on Computational Linguistics:Technical Papers.Pisa:ACM,2016:1777-1786.
[38] CUI Y,CHEN Z,WEI S,et al.Attention-over-Attention Neural Networks for Reading Comprehension[C]∥Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).Stroudsburg:ACL,2017:593-602.
[39] WANG S,JIANG J.Machine comprehension using match-lstm and answer pointer [DB/OL].[2018-08-28].https://arxiv.org/abs/1608.07905.
[1] ZHAO Jia-qi, WANG Han-zheng, ZHOU Yong, ZHANG Di, ZHOU Zi-yuan. Remote Sensing Image Description Generation Method Based on Attention and Multi-scale Feature Enhancement [J]. Computer Science, 2021, 48(1): 190-196.
[2] LIU Yang, JIN Zhong. Fine-grained Image Recognition Method Combining with Non-local and Multi-region Attention Mechanism [J]. Computer Science, 2021, 48(1): 197-203.
[3] WANG Rui-ping, JIA Zhen, LIU Chang, CHEN Ze-wei, LI Tian-rui. Deep Interest Factorization Machine Network Based on DeepFM [J]. Computer Science, 2021, 48(1): 226-232.
[4] YU Wen-jia, DING Shi-fei. Conditional Generative Adversarial Network Based on Self-attention Mechanism [J]. Computer Science, 2021, 48(1): 241-246.
[5] TONG Xin, WANG Bin-jun, WANG Run-zheng, PAN Xiao-qin. Survey on Adversarial Sample of Deep Learning Towards Natural Language Processing [J]. Computer Science, 2021, 48(1): 258-267.
[6] WANG Run-zheng, GAO Jian, HUANG Shu-hua, TONG Xin. Malicious Code Family Detection Method Based on Knowledge Distillation [J]. Computer Science, 2021, 48(1): 280-286.
[7] LU Long-long, CHEN Tong, PAN Min-xue, ZHANG Tian. CodeSearcher:Code Query Using Functional Descriptions in Natural Languages [J]. Computer Science, 2020, 47(9): 1-9.
[8] DING Yu, WEI Hao, PAN Zhi-song, LIU Xin. Survey of Network Representation Learning [J]. Computer Science, 2020, 47(9): 52-59.
[9] TIAN Ye, SHOU Li-dan, CHEN Ke, LUO Xin-yuan, CHEN Gang. Natural Language Interface for Databases with Content-based Table Column Embeddings [J]. Computer Science, 2020, 47(9): 60-66.
[10] HE Xin, XU Juan, JIN Ying-ying. Action-related Network:Towards Modeling Complete Changeable Action [J]. Computer Science, 2020, 47(9): 123-128.
[11] YE Ya-nan, CHI Jing, YU Zhi-ping, ZHAN Yu-liand ZHANG Cai-ming. Expression Animation Synthesis Based on Improved CycleGan Model and Region Segmentation [J]. Computer Science, 2020, 47(9): 142-149.
[12] DENG Liang, XU Geng-lin, LI Meng-jie, CHEN Zhang-jin. Fast Face Recognition Based on Deep Learning and Multiple Hash Similarity Weighting [J]. Computer Science, 2020, 47(9): 163-168.
[13] PAN Zu-jiang, LIU Ning, ZHANG Wei, WANG Jian-yong. MTHAM:Multitask Disease Progression Modeling Based on Hierarchical Attention Mechanism [J]. Computer Science, 2020, 47(9): 185-189.
[14] BAO Yu-xuan, LU Tian-liang, DU Yan-hui. Overview of Deepfake Video Detection Technology [J]. Computer Science, 2020, 47(9): 283-292.
[15] ZHAO Wei, LIN Yu-ming, WANG Chao-qiang, CAI Guo-yong. Opinion Word-pairs Collaborative Extraction Based on Dependency Relation Analysis [J]. Computer Science, 2020, 47(8): 164-170.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] LEI Li-hui and WANG Jing. Parallelization of LTL Model Checking Based on Possibility Measure[J]. Computer Science, 2018, 45(4): 71 -75 .
[2] SUN Qi, JIN Yan, HE Kun and XU Ling-xuan. Hybrid Evolutionary Algorithm for Solving Mixed Capacitated General Routing Problem[J]. Computer Science, 2018, 45(4): 76 -82 .
[3] ZHANG Jia-nan and XIAO Ming-yu. Approximation Algorithm for Weighted Mixed Domination Problem[J]. Computer Science, 2018, 45(4): 83 -88 .
[4] WU Jian-hui, HUANG Zhong-xiang, LI Wu, WU Jian-hui, PENG Xin and ZHANG Sheng. Robustness Optimization of Sequence Decision in Urban Road Construction[J]. Computer Science, 2018, 45(4): 89 -93 .
[5] SHI Wen-jun, WU Ji-gang and LUO Yu-chun. Fast and Efficient Scheduling Algorithms for Mobile Cloud Offloading[J]. Computer Science, 2018, 45(4): 94 -99 .
[6] ZHOU Yan-ping and YE Qiao-lin. L1-norm Distance Based Least Squares Twin Support Vector Machine[J]. Computer Science, 2018, 45(4): 100 -105 .
[7] LIU Bo-yi, TANG Xiang-yan and CHENG Jie-ren. Recognition Method for Corn Borer Based on Templates Matching in Muliple Growth Periods[J]. Computer Science, 2018, 45(4): 106 -111 .
[8] GENG Hai-jun, SHI Xin-gang, WANG Zhi-liang, YIN Xia and YIN Shao-ping. Energy-efficient Intra-domain Routing Algorithm Based on Directed Acyclic Graph[J]. Computer Science, 2018, 45(4): 112 -116 .
[9] CUI Qiong, LI Jian-hua, WANG Hong and NAN Ming-li. Resilience Analysis Model of Networked Command Information System Based on Node Repairability[J]. Computer Science, 2018, 45(4): 117 -121 .
[10] WANG Zhen-chao, HOU Huan-huan and LIAN Rui. Path Optimization Scheme for Restraining Degree of Disorder in CMT[J]. Computer Science, 2018, 45(4): 122 -125 .