Computer Science ›› 2020, Vol. 47 ›› Issue (2): 163-168.doi: 10.11896/jsjkx.190100048

• Artificial Intelligence • Previous Articles     Next Articles

Neural Machine Translation Combining Source Semantic Roles

QIAO Bo-wen,LI Jun-hui   

  1. (School of Computer Science and Technology,Soochow University,Suzhou,Jiangsu 215006,China)
  • Received:2019-01-07 Online:2020-02-15 Published:2020-03-18
  • About author:QIAO Bo-wen,born in 1994,postgra-duate.His main research interests include machine translation and so on;LI Jun-hui,born in 1983,Ph.D,asso-ciate professor.His main research inte-rests include machine translation and natural language processing.
  • Supported by:
    This work was supported by the National Natural Science Foundation of China (61876120).

Abstract: With the rapid development of deep learning in recent years,neural machine translation combining deep learning has gradually replaced statistical machine translation and becomes the mainstream machine translation method in the academic circle.However,the traditional neural machine translation regards the source-side sentence as a word sequence and does not take into account the implicit semantic information of sentences,resulting in the inconsistency between the translation results and source-side semantics.To solve this problem,some linguistic knowledges,such as syntax and semantics,are applied to neural machine translation and achieve good experimental results.Semantic roles can also be used to express the semantic information of sentences and have a certain application value in neural machine translation.This paper proposed two neural machine translation encoding models that incorporate semantic role information of sentences.On the one hand,semantic role played by labels are added to the word sequences to mark the semantic role played by each wordin the sentence.The semantic role labels and source-side words together constitute the word sequence.On the other hand,by constructing the semantic role tree of source sentences,the position information of each word in the semantic role tree is obtained,which is spliced with the word vector as a feature vector to form a word vector containing semantic role information.Experimental results on large-scale Chinese-English translation show that,compared with the baseline system,the two methods proposed in this paper not only improve 0.9 BLEU points and 0.72 BLEU points on average in all test sets respectively,but also improve performance in other evaluation indexes,such as TER (Translation Edit Rate) and RIBES (Rank-based Intuitive Bilingual Evaluation Score).Further experimental analysis shows that the proposed neural machine translation encoding models combining semantic roles have better translation effect on long sentences and translation adequacy than the baseline system.

Key words: Encoding model, Neural machine translation, Semantic feature, Semantic role labeling

CLC Number: 

  • TP391
[1]SUTSKEVER I,VINYALS O,LE Q V.Sequence to sequence learning with neural networks[C]∥Advances in neural information processing systems.Massachusetts:MIT Press,2014:3104-3112.
[2]BAHDANAU D,CHO K,BENGIO Y.Neural Machine Translation by Jointly Learning to Align and Translate[C]∥Procee-dings of the 3rd International Conference on Learning Representations.San Diego,CA,USA:ICLR,2015:1-15.
[3]LI Y C,XIONG D Y,ZHANG M.A Survey of Neural Machine Translation[J].Chinese Journal of Computers,2018,41(12):2734-2755.
[4]GILDEA D,JURAFSKY D.Automatic Labeling of Semantic Roles[C]∥Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics.Hong Kong,China:Association for Computational Linguistics,2000:512-520.
[5]WU D K,FUNG P.Semantic Roles for SMT:A Hybrid Two-Pass Model[C]∥Proceedings of the 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Boulder,Co-lorado:Association for Computational Linguistics,2009:13-16.
[6]LIU D,GILDEA D.Semantic Role Features for Machine Translation[C]∥Proceedings of the 23rd International Conference on Computational Linguistics.Association for Computational Linguistics,Beijing,2010:716-724.
[7]BAZRAFSHAN M,GILDEA D.Semantic Roles for String to Tree Machine Translation[C]∥Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics.Sofia,Bulgaria:Association for Computational Linguistics,2013:419-423.
[8]GAO Q,VOGEL S.Corpus Expansion for Statistical Machine Translation with Semantic Role Label Substitution Rules[C]∥Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics.Portland,Oregon:Association for Computational Linguistics,2011:294-298.
[9]XIONG D Y,ZHANG M,LI H Z.Modeling the Translation of Predicate-Argument Structure for SMT[C]∥Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics.Jeju,Republic of Korea:Association for Computational Linguistics,2012:902-911.
[10]GAO Q,VOGEL S.Utilizing Target-Side Semantic Role Labels to Assist Hierarchical Phrase-based Machine Translation[C]∥Pro-ceedings of SSST-5,Fifth Workshop on Syntax,Semantics and Structure in Statistical Translation.Portland,Oregon,USA:Association for Computational Linguistics,2011:107-115.
[11]LI J H,RESNIK P,DAUMÉ H.Modeling Syntactic and Semantic Structures in Hierarchical Phrase-based Translation[C]∥Proceedings of the 2013 Annual Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Atlanta,Georgia:Association for Computational Linguistics,2013:540-549.
[12]LI J H,MARTON Y,RESNIK P,et al.A Unified Model for Soft Linguistic Reordering Constrains in Statistical Machine Translation[C]∥Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics.Baltimore,Maryland,USA:Association for Computational Linguistics,2014:1123-1133.
[13]SENNRICH R,HADDOW B.Linguistic Input Features Improve Neural Machine Translation[C]∥Proceedings of the First Conference on Machine Translation.Berlin,Germany:Association for Computational Linguistics,2016:83-91.
[14]LI J H,XIONG D Y,TU Z P,et al.Modeling Source Syntax for Neural Machine Translation[C]∥Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Vancouver,Canada:Association for Computational Linguistics,2017:688-697.
[15]ERIGUCHI A,HASHIMOTO K,TSURUOKA Y.Tree-to-Sequence Attentional Neural Machine Translation[C]∥Procee-dings of the 54th Annual Meeting of the Association for Computational Linguistics.Berlin,Germany:Association for Computational Linguistics,2016:823-833.
[16]CHEN H D,HUANG S J,CHIANG D,et al.Improved Neural Machine Translation with a Syntax-Aware Encoder and Decoder[C]∥Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Vancouver,Canada:Association for Computational Linguistics,2017:1936-1945.
[17]CHEN K H,WANG R,UTIYAMA M,et al.Neural Machine Translation with Source Dependency Representation[C]∥Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing.Copenhagen,Denmark:Association for Computational Linguistics,2017:2846-2852.
[18]WU S Z,ZHOU M,ZHANG D D.Improved Neural Machine Translation with Source Syntax[C]∥Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence.Melbourne,Australia:IJCAI,2017:4179-4185.
[19]AHARONI R,GOLDBERG Y.Towards String-to-Tree Neural Machine Translation[C]∥Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Vancouver,Canada:Association for Computational Linguistics,2017:132-140.
[20]MORISHITA M,SUZUKI J,NAGATA M.Improving NeuralMachine Translation by Incorporating Hierarchical Subword Features[C]∥Proceedings of the 27th International Conference on Computational Linguistics.Santa Fe,New-Mexico,USA:COLING,2018:618-629.
[21]XIONG D Y,LI J H,WANG X,et al.Neural Machine Translation with Constraints[J].Scientia Sinica Informationis,2018,48(5):574-588.
[22]WANG Q,DUAN X Y.Neural Machine Translation Based on Attention Convolution[J].Computer Science,2018,45(11):226-230.
[23]CHO K,MERRIENBOER B V,BAHDANAU D.On the Properties of Neural Machine Translation:Encoder-Decoder Approaches[C]∥Proceedings of SSST-8,Eighth Workshop on Syntax,Semantics and Structure in Statistical Translation.Doha,Qatar:Association for Computational Linguistics,2014:103-111.
[24]CHUNG J,GULCEHRE C,CHO K,et al.Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling[C]∥Proceedings of the Twenty-eighth Conference on Neural Information Processing Systems.Montreal,Quebec,Canada:NIPS,2014:1-9.
[25]ZEILER M D.An Adaptive Learning Rate Method[J].arXiv:1212.5701.
[26]PETROV S,KLEIN D.Improved Inference for Unlexicalized Parsing[C]∥Proceedings of the 2007 Annual Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Rochester,NY:Association for Computational Linguistics,2007:404-411.
[27]LI J H,ZHOU G D,HWEE T N.Joint Syntactic and Semantic Parsing of Chinese[C]∥Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics.Uppsala,Sweden:Association for Computational Linguistics,2010:1108-1117.
[28]PAPINENI K,ROUKOS S,WARD T,et al.BLEU:a Method for Automatic Evaluation of Machine Translation[C]∥Procee-dings of the 40th Annual Meeting of the Association for Computational Linguistics.Philadelphia:Association for Computational Linguistics,2002:311-318.
[29]SNOVER M,DORR B,SCHWARTZ R,et al.A Study of Translation Edit Rate with Targeted Human Annotation[C]∥Proceedings of Association for Machine Translation in the Americas.2006:231-231.
[30]ISOZAKI H,HIRAO T,DUH K,et al.Automatic Evaluation of Translation Quality for Distant Language Pairs[C]∥Procee-dings of the 2010 Conference on Empirical Methods in Natural Language Processing.MIT,Massachusetts:Association for Computational Linguistics,2010:944-952.
[1] DENG Wei-bin, ZHU Kun, LI Yun-bo, HU Feng. FMNN:Text Classification Model Fused with Multiple Neural Networks [J]. Computer Science, 2022, 49(3): 281-287.
[2] LIU Jun-peng, SU Jin-song, HUANG De-gen. Incorporating Language-specific Adapter into Multilingual Neural Machine Translation [J]. Computer Science, 2022, 49(1): 17-23.
[3] LIU Yan, XIONG De-yi. Construction Method of Parallel Corpus for Minority Language Machine Translation [J]. Computer Science, 2022, 49(1): 41-46.
[4] WANG Shi-hao, WANG Zhong-qing, LI Shou-shan, ZHOU Guo-dong. Event Argument Extraction Using Gated Graph Convolution and Dynamic Dependency Pooling [J]. Computer Science, 2021, 48(11A): 52-56.
[5] GAO Nan,LI Li-juan,Wei-william LEE,ZHU Jian-ming. Keywords Extraction Method Based on Semantic Feature Fusion [J]. Computer Science, 2020, 47(3): 110-115.
[6] XIE Nian-nian, ZENG Fan-ping, ZHOU Ming-song, QIN Xiao-xia, LV Cheng-cheng, CHEN Zhao. Android Malware Detection with Multi-dimensional Sensitive Features [J]. Computer Science, 2019, 46(2): 95-101.
[7] QIU Shao-jian, CAIZi-yi, LU Lu. Cost-sensitive Convolutional Neural Network Model for Software Defect Prediction [J]. Computer Science, 2019, 46(11): 156-160.
[8] LIU Ying, ZHANG Shuai, GE Yu-xiang, WANG Fu-ping, LI Da-xiang. Survey of Tire Pattern Image Retrieval Techniques [J]. Computer Science, 2018, 45(12): 52-60.
[9] WANG Qi, DUAN Xiang-yu. Neural Machine Translation Based on Attention Convolution [J]. Computer Science, 2018, 45(11): 226-230.
[10] JIANG Zheng and CHENG Chun-ling. Improved HOG Face Feature Extraction Algorithm Based on Haar Characteristics [J]. Computer Science, 2017, 44(1): 303-307.
[11] ZHANG Dong-wen, YANG Peng-fei and XU Yun-feng. Research of Chinese Comments Sentiment Classification Based on Word2vec and SVMperf [J]. Computer Science, 2016, 43(Z6): 418-421.
[12] . Research on Collaborative Semantic Feature Modeling System [J]. Computer Science, 2012, 39(2): 280-282.
[13] JIN Ying-hao,SUN Li-juan. Research of Representation Based on Feature Semantic for Models [J]. Computer Science, 2011, 38(1): 286-289.
[14] SUN Li-juan,JIN Ying-hao. Strategy of Feature Interaction Based on the Sufficiency Principle [J]. Computer Science, 2010, 37(8): 270-272.
[15] . [J]. Computer Science, 2008, 35(3): 155-157.
Full text



No Suggested Reading articles found!