Computer Science ›› 2019, Vol. 46 ›› Issue (7): 7-12.doi: 10.11896/j.issn.1002-137X.2019.07.002

• Surveys • Previous Articles     Next Articles

Survey on Deep-learning-based Machine Reading Comprehension

LI Zhou-jun,WANG Chang-bao   

  1. (School of Computer Science and Engineering,Beihang University,Beijing 100191,China)
  • Received:2018-08-28 Online:2019-07-15 Published:2019-07-15

Abstract: Natural language processing is the key to achieving artificial intelligence.Machine reading comprehension,as the crown jewel in the field of natural language processing,has always been the focus of research in the field.With the rapid development of deep learning and neural network in recent years,machine reading comprehension has made great progress.Firstly,the research background and development history of machine reading comprehension were introduced.Then,by reviewing the important progress in the development of word vector,attention mechanism and answer prediction,the problems in recent research related to machine reading comprehension were proposed.Finally,the outlook of machine reading comprehension was discussed.

Key words: Natural language processing, Machine reading comprehension, Deep learning, Word vector, Attention mechanism

CLC Number: 

  • TP391
[1] HERMANN K M,KOCISKY T,GREFENSTETTE E,et al. Teaching machines to read and comprehend[C]∥Advances in Neural Information Processing Systems.Cambridge:MIT Press,2015:1693-1701.
[2] RAJPURKAR P,ZHANG J,LOPYREV K,et al.SQuAD: 100 000+ Questions for Machine Comprehension of Text[C]∥Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.Stroudsburg:ACL,2016:2383-2392.
[3] NGUYEN T,ROSENBERG M,SONG X,et al.MS MARCO:A human generated machine reading comprehension dataset [DB/OL].[2018-08-28].https://arxiv.org/abs/1611.09268.
[4] HIRSCHMAN L,LIGHT M,BRECK E,et al.Deep read:A reading comprehension system[C]∥Proceedings of the 37th annual meeting of the Association for Computational Linguistics on Computational Linguistics.Stroudsburg:ACL,1999:325-332.
[5] RILOFF E,THELEN M.A rule-based question answering system for reading comprehension tests[C]∥Proceedings of the 2000 ANLP/NAACL Workshop on Reading comprehension tests as evaluation for computer-based language understanding sytems-Volume 6.Stroudsburg:ACL,2000:13-19.
[6] POON H,CHRISTENSEN J,DOMINGOS P,et al.Machine reading at the university of washington[C]∥Proceedings of the NAACL HLT 2010 First International Workshop on Forma-lisms and Methodology for Learning by Reading.Association for Computational Linguistics.Stroudsburg:ACL,2010:87-95.
[7] RICHARDSON M,BURGES C J C,RENSHAW E.Mctest:A challenge dataset for the open-domain machine comprehension of text[C]∥Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing.Stroudsburg:ACL,2013:193-203.
[8] KADLEC R,SCHMID M,BAJGAR O,et al.Text understan- ding with the attention sum reader network [DB/OL].[2018-08-28].https://arxiv.org/abs/1603.01547.
[9] CHEN D,BOLTON J,MANNING C D.A Thorough Examination of the CNN/Daily Mail Reading Comprehension Task[C]∥Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).Stroudsburg:ACL,2016:2358-2367.
[10] CUI Y,CHEN Z,WEI S,et al.Attention-over-Attention Neural Networks for Reading Comprehension[C]∥Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).Stroudsburg:ACL,2017:593-602.
[11] DHINGRA B,LIU H,YANG Z,et al.Gated-Attention Readers for Text Comprehension[C]∥Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Vo-lume 1:Long Papers).Stroudsburg:ACL,2017:1832-1846.
[12] WANG S,JIANG J.Machine comprehension using match-lstm and answer pointer [DB/OL].[2018-08-28].https://ar-xiv.org/abs/1608.07905.
[13] WANG S,JIANG J.Learning Natural Language Inference with LSTM[C]∥Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Stroudsburg:ACL,2016:1442-1451.
[14] VINYALS O,FORTUNATO M,JAITLY N.Pointer networks[C]∥Advances in Neural Information Processing Systems.Cambridge:MIT Press,2015:2692-2700.
[15] SEO,MINJOON,et al.Bidirectional attention flow for machine comprehension [DB/OL].[2018-08-28].https://arxiv.org/abs/1611.01603.
[16] XIONG C,ZHONG V,SOCHER R.Dynamic coattention network for question answering:U.S.Patent Application 15/421,193[P].2018-05-10.
[17] WANG W,YANG N,WEI F,et al.Gated self-matching networks for reading comprehension and question answering[C]∥Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).Stroudsburg:ACL,2017:189-198.
[18] JOSHI M,CHOI E,WELD D,et al.TriviaQA:A Large Scale Distantly Supervised Challenge Dataset for Reading Comprehension[C]∥Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).Stroudsburg:ACL,2017:1601-1611.
[19] HE W,LIU K,LYU Y,et al.DuReader:a Chinese Machine Reading Comprehension Dataset from Real-world Applications[DB/OL].[2018-08-28].https://arxiv.org/abs/1711.05073.
[20] TAN C,WEI F,YANG N,et al.S-net:From answer extraction to answer generation for machine reading comprehension [DB/OL].[2018-08-28].https://arxiv.org/abs/1706.04815.
[21] CLARK C,GARDNER M.Simple and Effective Multi-Para- graph Reading Comprehension[C]∥Meeting of the Association for Computational Linguistics.Stroudsburg:ACL,2018:845-855.
[22] WANG Y,LIU K,LIU J,et al.Multi-Passage Machine Reading Comprehension with Cross-Passage Answer Verification[C]∥Meeting of the Association for Computational Linguistics.Stroudsburg:ACL,2018:1918-1927.
[23] KOCISKY T,SCHWARZ J,BLUNSOM P,et al.The Narra- tiveQA Reading Comprehension Challenge[J].Transactions of the Association for Computational Linguistics,2018,6:317-328.
[24] DEERWESTER S,DUMAIS S T,FURNAS G W,et al.Indexing by latent semantic analysis[J].Journal of the American society for Information Science,1990,41(6):391-407.
[25] LUND K,BURGESS C.Producing high-dimensional semantic spaces from lexical co-occurrence[J].Behavior Research Me-thods,Instruments,& Computers,1996,28(2):203-208.
[26] ROHDE D L T,GONNERMAN L M,PLAUT D C.An improved model of semantic similarity based on lexical co-occurrence[J].Communications of the ACM,2006,8(627-633):116.
[27] BENGIO Y,DUCHARME R,VINCENT P,et al.A neural probabilistic language model[J].Journal of Machine Learning Research,2003,3:1137-1155.
[28] MIKOLOV T,SUTSKEVER I,CHEN K,et al.Distributed representations of words and phrases and their compositionality[C]∥dAdvancesf in Neural Information Processing Systems.Cambridge:MIT Press,2013:3111-3119.
[29] MIKOLOV T,CHEN K,CORRADO G,et al.Efficient estimation of word representations in vector space[J].arXiv:1301.3781,2013.
[30] PENNINGTON J,SOCHER R,MANNING C.Glove:Global vectors for word representation[C]∥Proceedings of the 2014 Conference on Empirical Methods in Natural Language Proces-sing (EMNLP).Stroudsburg:ACL,2014:1532-1543.
[31] MELAMUD O,GOLDBERGER J,DAGAN I.context2vec: Learning generic context embedding with bidirectional lstm[C]∥Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning.Stroudsburg:ACL,2016:51-61.
[32] MCCANN B,BRADBURY J,XIONG C,et al.Learned in translation:Contextualized word vectors[C]∥Advances in Neural Information Processing Systems.Cambridge:MIT Press,2017:6294-6305.
[33] PETERS M,NEUMANN M,IYYER M,et al.Deep Contextua- lized Word Representations[C]∥Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies(Vo-lume 1:Long Papers).Stroudsburg:ACL,2018:2227-2237.
[34] RENSINK R A.The dynamic representation of scenes[J].Vi- sual cognition,2000,7(1/2/3):17-42.
[35] MNIH V,HEESS N,GRAVES A.Recurrent models of visual attention[C]∥Advances in Neural Information Processing Systems.Stroudsburg:ACL,2014:2204-2212.
[36] BAHDANAU D,CHO K,BENGIO Y.Neural machine translation by jointly learning to align and translate [DB/OL].[2018-08-28].https://arxiv.org/abs/1409.0473.
[37] CUI Y,LIU T,CHEN Z,et al.Consensus Attention-based Neural Networks for Chinese Reading Comprehension[C]∥Proceedings of COLING 2016,the 26th International Conference on Computational Linguistics:Technical Papers.Pisa:ACM,2016:1777-1786.
[38] CUI Y,CHEN Z,WEI S,et al.Attention-over-Attention Neural Networks for Reading Comprehension[C]∥Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1:Long Papers).Stroudsburg:ACL,2017:593-602.
[39] WANG S,JIANG J.Machine comprehension using match-lstm and answer pointer [DB/OL].[2018-08-28].https://arxiv.org/abs/1608.07905.
[1] ZHOU Yan, ZENG Fan-zhi, WU Chen, LUO Yue, LIU Zi-qin. 3D Shape Feature Extraction Method Based on Deep Learning [J]. Computer Science, 2019, 46(9): 47-58.
[2] MA Lu, PEI Wei, ZHU Yong-ying, WANG Chun-li, WANG Peng-qian. Fall Action Recognition Based on Deep Learning [J]. Computer Science, 2019, 46(9): 106-112.
[3] LI Qing-hua, LI Cui-ping, ZHANG Jing, CHEN Hong, WANG Shao-qing. Survey of Compressed Deep Neural Network [J]. Computer Science, 2019, 46(9): 1-14.
[4] WANG Yan-ran, CHEN Qing-liang, WU Jun-jun. Research on Image Semantic Segmentation for Complex Environments [J]. Computer Science, 2019, 46(9): 36-46.
[5] SUN Zhong-feng, WANG Jing. RCNN-BGRU-HN Network Model for Aspect-based Sentiment Analysis [J]. Computer Science, 2019, 46(9): 223-228.
[6] SHI Chun-dan, QIN Lin. Chinese Named Entity Recognition Method Based on BGRU-CRF [J]. Computer Science, 2019, 46(9): 237-242.
[7] MIAO Yong-wei, LI Gao-yi, BAO Chen, ZHANG Xu-dong, PENG Si-long. Image Localized Style Transfer Based on Convolutional Neural Network [J]. Computer Science, 2019, 46(9): 259-264.
[8] DENG Cun-bin, YU Hui-qun, FAN Gui-sheng. Integrating Dynamic Collaborative Filtering and Deep Learning for Recommendation [J]. Computer Science, 2019, 46(8): 28-34.
[9] DU Wei, DING Shi-fei. Overview on Multi-agent Reinforcement Learning [J]. Computer Science, 2019, 46(8): 1-8.
[10] GUO Xu, ZHU Jing-hua. Deep Neural Network Recommendation Model Based on User Vectorization Representation and Attention Mechanism [J]. Computer Science, 2019, 46(8): 111-115.
[11] ZHANG Yi-jie, LI Pei-feng, ZHU Qiao-ming. Event Temporal Relation Classification Method Based on Self-attention Mechanism [J]. Computer Science, 2019, 46(8): 244-248.
[12] FANG Jie, LI Pei-feng, ZHU Qiao-ming. Employing Multi-attention Mechanism to Resolve Event Coreference [J]. Computer Science, 2019, 46(8): 277-281.
[13] WANG Le-le,WANG Bin-qiang,LIU Jian-gang,ZHANG Jian-hui,MIAO Qi-guang. Study on Malicious Program Detection Based on Recurrent Neural Network [J]. Computer Science, 2019, 46(7): 86-90.
[14] SHEN Chen-lin, ZHANG Lu, WU Liang-qing, LI Shou-shan. Sentiment Classification Towards Question-Answering Based on Bidirectional Attention Mechanism [J]. Computer Science, 2019, 46(7): 151-156.
[15] ZHANG Lin-na,CHEN Jian-qiang,CHEN Xiao-ling,CEN Yi-gang,KAN Shi-chao. Lightweight SSD Network for Real-time Object Detection in Automotive Videos [J]. Computer Science, 2019, 46(7): 233-237.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
[1] LEI Li-hui and WANG Jing. Parallelization of LTL Model Checking Based on Possibility Measure[J]. Computer Science, 2018, 45(4): 71 -75, 88 .
[2] XIA Qing-xun and ZHUANG Yi. Remote Attestation Mechanism Based on Locality Principle[J]. Computer Science, 2018, 45(4): 148 -151, 162 .
[3] LI Bai-shen, LI Ling-zhi, SUN Yong and ZHU Yan-qin. Intranet Defense Algorithm Based on Pseudo Boosting Decision Tree[J]. Computer Science, 2018, 45(4): 157 -162 .
[4] ZHANG Jia-nan and XIAO Ming-yu. Approximation Algorithm for Weighted Mixed Domination Problem[J]. Computer Science, 2018, 45(4): 83 -88 .
[5] WU Jian-hui, HUANG Zhong-xiang, LI Wu, WU Jian-hui, PENG Xin and ZHANG Sheng. Robustness Optimization of Sequence Decision in Urban Road Construction[J]. Computer Science, 2018, 45(4): 89 -93 .
[6] LIU Bo-yi, TANG Xiang-yan and CHENG Jie-ren. Recognition Method for Corn Borer Based on Templates Matching in Muliple Growth Periods[J]. Computer Science, 2018, 45(4): 106 -111, 142 .
[7] WANG Zhen-wu, LV Xiao-hua and HAN Xiao-hui. Survey of Terrain LOD Technology Based on Quadtree Segmentation[J]. Computer Science, 2018, 45(4): 34 -45 .
[8] WANG Shuai, LIU Juan, BI Yao-yao, CHEN Zhe, ZHENG Qun-hua and DUAN Hui-fang. Automatic Recognition of Breast Gland Based on Two-step Clustering and Random Forest[J]. Computer Science, 2018, 45(3): 247 -252 .
[9] WANG Zhen-chao, HOU Huan-huan and LIAN Rui. Path Optimization Scheme for Restraining Degree of Disorder in CMT[J]. Computer Science, 2018, 45(4): 122 -125 .
[10] SHI Chao, XIE Zai-peng, LIU Han and LV Xin. Optimization of Container Deployment Strategy Based on Stable Matching[J]. Computer Science, 2018, 45(4): 131 -136 .