Computer Science ›› 2017, Vol. 44 ›› Issue (Z6): 92-96.doi: 10.11896/j.issn.1002-137X.2017.6A.019

Previous Articles     Next Articles

Attention of Bilinear Function Based Bi-LSTM Model for Machine Reading Comprehension

LIU Fei-long, HAO Wen-ning, CHEN Gang, JIN Da-wei and SONG Jia-xing   

  • Online:2017-12-01 Published:2018-12-01

Abstract: With the wild usage of deep learning in machine reading comprehension in the past few years,machine rea-ding comprehension has developed rapidly.In order to improve machine reading comprehension’s semantic comprehension and inference abilities,an attention of bilinear function based Bi-LSTM model was proposed,which has good performance in extracting semantics of questions,candidates and articles,and producing the correct answers.We tested the model on CET-4 and CET-6 listening text materials.The results show that the accuracy rate of word-level input is about 2% higher than sentence-level input.Besides,the accuracy rate can increase about 8% after adding infe-rence structure with multi-layer attention.

Key words: Deep learning,Machine reading comprehension,Attention,Bi-LSTM

[1] BURGES C J C.Towards the machine comprehension of text:An essay:MSR-TR-2013-125 [R].2013.
[2] BORDES A,USUNIER N,C HOPRA S,et al.Large-scale simple question answering with memory networks[J].arXiv preprint arXiv:1506.02075,2015.
[3] KUMAR A,IRSOY O,SU J,et al.Ask me anything:Dynamic memory networks for natural language processing[J].arXiv preprint arXiv:1506.07285,2015.
[4] COLLOBERT R,WESTON J,BOTTOU L,et al.Natural language processing (almost) from scratch[J].Journal of Machine Learning Research,2011,12(8):2493-2537.
[5] SUKHBAATAR S,WESTON J,F ERGUS R.End-to-end memory networks[C]∥Advances in Neural Information Processing Systems.2015:2440-2448.
[6] KALCHBRENNER N,GREFENSTENTTE E,BLUNSOM P.A convolutional neural network for modelling sentences[J].arXiv preprint arXiv:1404.2188,2014.
[7] 尹宝才,王文通,王立春.深度学习研究综述[J].北京工业大学学报,2015,41(1):48-59.
[8] RUSH A M,CHOPRA S,WESTON J.A neural attention model for abstractive sentence summarization[J].arXiv preprint ar-Xiv:1509.00685,2015.
[9] CHEN D,BOLTON J,MANNING C D.A thorough examination of the cnn/daily mail reading comprehension task[J].arXiv preprint arXiv:1606.02858,2016.
[10] HOCHREITER S,SCHMIDHUBER J.Long short-term memory[J].Neural Computation,1997,9(8):1735-1780.
[11] BAHDANAU D,CHO K,BENGIO Y.Neural machine translation by jointly learning to align and translate[J].arXiv preprint arXiv:1409.0473,2014.
[12] KADLEC R,SCHMOID M,BAJGAR O,et al.Text Under-standing with the Attention Sum Reader Network[J].arXiv preprint arXiv:1603.01547,2016.
[13] HERMANN K M,KOCISKY T,GREFENSTETTE E,et al.Teaching machines to read and comprehend[C]∥Advances in Neural Information Processing Systems.2015:1693-1701.
[14] 北京大学数学系前代数小组.高等代数(第四版)[M].北京:高等教育出版社,2013.
[15] LUONG M T,PHAM H,MANNING C D.Effective approaches to attention-based neural machine translation[J].arXiv preprint arXiv:1508.04025,2015.
[16] http://nlp.stanford.edu/data/glove.6B.zip.
[17] PENNINGTON J,SOCHER R,Manning C D.Glove:GlobalVectors for Word Representation[C]∥EMNLP.2014,14:1532-1543.
[18] WESTON J,CHOPRA S,BORDES A.Memory networks[J].arXiv preprint arXiv:1410.3916,2014.

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!