Computer Science

Computer Science ›› 2019, Vol. 46 ›› Issue (5): 198-202.doi: 10.11896/j.issn.1002-137X.2019.05.030

Previous Articles     Next Articles

Neural Machine Translation Inclined to Close Neighbor Association

WANG Kun, DUAN Xiang-yu   

  1. (School of Computer Science & Technology,Soochow University,Suzhou,Jiangsu 215006,China)
  • Received:2018-04-18 Revised:2018-09-01 Published:2019-05-15

PDF (PC)

849

Abstract: The existing neural machine translation model only considers the relevance of the target end corresponding to the source end when modeling the sequences,and does not model the source end association and the target end association.In this paper,the source and target associations were modeled separately,and a reasonable loss function was designed.The source-hidden layer is more related to its neighboring K word-hidden layers.The target-side hidden layer is more related to its historical M word-hidden layers.The experimental results on the large-scale Chinese-English dataset show that compared with the neural machine translation which only considers the relevance of the target end to the source,the proposed method can construct a better neighbor correlation representation and improve the translation qua-lity of the machine translation system.

Key words: Attention machanism, Close neighbor association, Machine translation

CLC Number: 

  • TP391
[1]LI Y C,XIONG D Y,ZHANG M.A survey of neural machine translation[OL].http://cjc.ict.ac.cn/online/bfpub/lyc-20171229152034.pdf.
[2]BAHDANAU D,CHO K,BENGIO Y.Neural Machine Translation by Jointly Learning to Align and Translate[J]. arXiv preprint arXiv:1409.0473,2014.
[3]SUTSKEVER I,VINYALS O,LE Q V.Sequence to Sequence Learning with Neural Networks[C]∥Advances in Neural Information Processing Systems.2014:3104-3112.
[4]LUONGM T,PHAM H,MANNING C D.Effective Approaches to Attention-based Neural Machine Translation[J].arXiv preprint arXiv:1508.04025,2015.
[5]LIUL M,UTIYAMA M,FINCH A,et al.Neural MachineTranslationwith Supervised Attention[J].arXiv preprint arXiv:1609.04186,2016.
[6]MI H T,WANG Z G,ITTYCHERIAH A.Supervised Atten-tions for Neural Machine Translation[J].arXiv preprint arXiv:1608.00112,2016.
[7]CHEN W H,MATUSOV E,KHADIVI S,et al.Guided Alignment Training for Topic-aware Neural Machine Translation[J].arXiv preprint arXiv:1607.01628,2016.
[8]GEHRING J,AULI M,GRANGIER D,et al.ConvolutionalSequence to Sequence Learning[J].arXiv preprint arXiv:1705.03122,2017.
[9]VASWANI A,SHAZEER N,PARMAR N,et al.Attention is All You Need [C]∥Advances in Neural Information Processing Systems.2017:5998-6008.
[10]MIKOLOV T,CHEN K,CORRADO G,et al.Efficient estimation of word representations in vectorspace[J].arXiv preprint arXiv:1301.3781,2013.
[11]HOCHREITER S,SCHMIDHUBER J.Long Short-term Me-mory[J].Neural Computation,1997,9(8):1735-1780.
[12]CHO K,VAN MERRIENBOER B,GULERHRE C,et al.
Learning Phrase Representations using RNN Encoder-decoder for Statistical Machine Translation [J].arXiv preprint arXiv:1406.1078,2014.
[13]NEUBIG G,DYER C,GOLDBERG Y,et al.Dynet:The Dy-namic Neural Network Toolkit[J].arXiv preprint arXiv:1701.03980,2017.
[14]NEUBIG G.lamtram:A Toolkit for Language and Translation Modeling using Neural Networks[OL].http://www.github.com/neubig/lamtram.
[15]KINGMA D,BA J.Adam:A Method For Stochastic Optimization[J].arXiv preprint arXiv:1412.6980,2014.
[1] DONG Zhen-heng, REN Wei-ping, YOU Xin-dong, LYU Xue-qiang. Machine Translation Method Integrating New Energy Terminology Knowledge [J]. Computer Science, 2022, 49(6): 305-312.
[2] LIU Jun-peng, SU Jin-song, HUANG De-gen. Incorporating Language-specific Adapter into Multilingual Neural Machine Translation [J]. Computer Science, 2022, 49(1): 17-23.
[3] YU Dong, XIE Wan-ying, GU Shu-hao, FENG Yang. Similarity-based Curriculum Learning for Multilingual Neural Machine Translation [J]. Computer Science, 2022, 49(1): 24-30.
[4] HOU Hong-xu, SUN Shuo, WU Nier. Survey of Mongolian-Chinese Neural Machine Translation [J]. Computer Science, 2022, 49(1): 31-40.
[5] LIU Yan, XIONG De-yi. Construction Method of Parallel Corpus for Minority Language Machine Translation [J]. Computer Science, 2022, 49(1): 41-46.
[6] LIU Chuang, XIONG De-yi. Survey of Multilingual Question Answering [J]. Computer Science, 2022, 49(1): 65-72.
[7] NING Qiu-yi, SHI Xiao-jing, DUAN Xiang-yu, ZHANG Min. Unsupervised Domain Adaptation Based on Style Aware [J]. Computer Science, 2022, 49(1): 271-278.
[8] LIU Xiao-die. Recognition and Transformation for Complex Noun Phrases Based on Boundary Perception [J]. Computer Science, 2021, 48(6A): 299-305.
[9] GUO Dan, TANG Shen-geng, HONG Ri-chang, WANG Meng. Review of Sign Language Recognition, Translation and Generation [J]. Computer Science, 2021, 48(3): 60-70.
[10] ZHOU Xiao-shi, ZHANG Zi-wei, WEN Juan. Natural Language Steganography Based on Neural Machine Translation [J]. Computer Science, 2021, 48(11A): 557-564.
[11] QIAO Bo-wen,LI Jun-hui. Neural Machine Translation Combining Source Semantic Roles [J]. Computer Science, 2020, 47(2): 163-168.
[12] JI Ming-xuan, SONG Yu-rong. New Machine Translation Model Based on Logarithmic Position Representation and Self-attention [J]. Computer Science, 2020, 47(11A): 86-91.
[13] WANG Qi, DUAN Xiang-yu. Neural Machine Translation Based on Attention Convolution [J]. Computer Science, 2018, 45(11): 226-230.
[14] YANG Zhi-zhuo. Supervised WSD Method Based on Context Translation [J]. Computer Science, 2017, 44(4): 252-255.
[15] LI Jin-ting, HOU Hong-xu, WU Jing, WANG Hong-bin and FAN Wen-ting. Effect of Preprocessing on Corpus of Mongolian-Chinese Statistical Machine Translation [J]. Computer Science, 2017, 44(10): 259-264.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.