融合语义角色的神经机器翻译

doi:10.11896/jsjkx.190100048

摘要/Abstract

摘要： 近年来,深度学习取得了重大突破,融合深度学习技术的神经机器翻译逐渐取代统计机器翻译,成为学术界主流的机器翻译方法。然而,传统的神经机器翻译将源端句子看作一个词序列,没有考虑句子的隐含语义信息,使得翻译结果与源端语义不一致。为了解决这个问题,一些语言学知识如句法、语义等被相继应用于神经机器翻译,并取得了不错的实验效果。语义角色也可用于表达句子语义信息,在神经机器翻译中具有一定的应用价值。文中提出了两种融合句子语义角色信息的神经机器翻译编码模型,一方面,在句子词序列中添加语义角色标签,标记每段词序列在句子中担当的语义角色,语义角色标签与源端词汇共同构成句子词序列;另一方面,通过构建源端句子的语义角色树,获取每个词在该语义角色树中的位置信息,将其作为特征向量与词向量进行拼接,构成含语义角色信息的词向量。在大规模中-英翻译任务上的实验结果表明,相较基准系统,文中提出的两种方法分别在所有测试集上平均提高了0.9和0.72个BLEU点,在其他评测指标如TER(Translation Edit Rate)和RIBES(Rank-based Intuitive Bilingual Evaluation Score)上也有不同程度的性能提升。进一步的实验分析显示,相较基准系统,文中提出的融合语义角色的神经机器翻译编码模型具有更佳的长句翻译效果和翻译充分性。

关键词: 编码模型, 神经机器翻译, 语义角色标注, 语义特征

Abstract: With the rapid development of deep learning in recent years,neural machine translation combining deep learning has gradually replaced statistical machine translation and becomes the mainstream machine translation method in the academic circle.However,the traditional neural machine translation regards the source-side sentence as a word sequence and does not take into account the implicit semantic information of sentences,resulting in the inconsistency between the translation results and source-side semantics.To solve this problem,some linguistic knowledges,such as syntax and semantics,are applied to neural machine translation and achieve good experimental results.Semantic roles can also be used to express the semantic information of sentences and have a certain application value in neural machine translation.This paper proposed two neural machine translation encoding models that incorporate semantic role information of sentences.On the one hand,semantic role played by labels are added to the word sequences to mark the semantic role played by each wordin the sentence.The semantic role labels and source-side words together constitute the word sequence.On the other hand,by constructing the semantic role tree of source sentences,the position information of each word in the semantic role tree is obtained,which is spliced with the word vector as a feature vector to form a word vector containing semantic role information.Experimental results on large-scale Chinese-English translation show that,compared with the baseline system,the two methods proposed in this paper not only improve 0.9 BLEU points and 0.72 BLEU points on average in all test sets respectively,but also improve performance in other evaluation indexes,such as TER (Translation Edit Rate) and RIBES (Rank-based Intuitive Bilingual Evaluation Score).Further experimental analysis shows that the proposed neural machine translation encoding models combining semantic roles have better translation effect on long sentences and translation adequacy than the baseline system.

Key words: Encoding model, Neural machine translation, Semantic feature, Semantic role labeling

中图分类号:

TP391

乔博文,李军辉. 融合语义角色的神经机器翻译[J]. 计算机科学, 2020, 47(2): 163-168. https://doi.org/10.11896/jsjkx.190100048

QIAO Bo-wen,LI Jun-hui. Neural Machine Translation Combining Source Semantic Roles[J]. Computer Science, 2020, 47(2): 163-168. https://doi.org/10.11896/jsjkx.190100048

参考文献

[1]SUTSKEVER I,VINYALS O,LE Q V.Sequence to sequence learning with neural networks[C]∥Advances in neural information processing systems.Massachusetts:MIT Press,2014:3104-3112.
[2]BAHDANAU D,CHO K,BENGIO Y.Neural Machine Translation by Jointly Learning to Align and Translate[C]∥Procee-dings of the 3rd International Conference on Learning Representations.San Diego,CA,USA:ICLR,2015:1-15.
[3]LI Y C,XIONG D Y,ZHANG M.A Survey of Neural Machine Translation[J].Chinese Journal of Computers,2018,41(12):2734-2755.
[4]GILDEA D,JURAFSKY D.Automatic Labeling of Semantic Roles[C]∥Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics.Hong Kong,China:Association for Computational Linguistics,2000:512-520.
[5]WU D K,FUNG P.Semantic Roles for SMT:A Hybrid Two-Pass Model[C]∥Proceedings of the 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Boulder,Co-lorado:Association for Computational Linguistics,2009:13-16.
[6]LIU D,GILDEA D.Semantic Role Features for Machine Translation[C]∥Proceedings of the 23rd International Conference on Computational Linguistics.Association for Computational Linguistics,Beijing,2010:716-724.
[7]BAZRAFSHAN M,GILDEA D.Semantic Roles for String to Tree Machine Translation[C]∥Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics.Sofia,Bulgaria:Association for Computational Linguistics,2013:419-423.
[8]GAO Q,VOGEL S.Corpus Expansion for Statistical Machine Translation with Semantic Role Label Substitution Rules[C]∥Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics.Portland,Oregon:Association for Computational Linguistics,2011:294-298.
[9]XIONG D Y,ZHANG M,LI H Z.Modeling the Translation of Predicate-Argument Structure for SMT[C]∥Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics.Jeju,Republic of Korea:Association for Computational Linguistics,2012:902-911.
[10]GAO Q,VOGEL S.Utilizing Target-Side Semantic Role Labels to Assist Hierarchical Phrase-based Machine Translation[C]∥Pro-ceedings of SSST-5,Fifth Workshop on Syntax,Semantics and Structure in Statistical Translation.Portland,Oregon,USA:Association for Computational Linguistics,2011:107-115.
[11]LI J H,RESNIK P,DAUMÉ H.Modeling Syntactic and Semantic Structures in Hierarchical Phrase-based Translation[C]∥Proceedings of the 2013 Annual Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Atlanta,Georgia:Association for Computational Linguistics,2013:540-549.
[12]LI J H,MARTON Y,RESNIK P,et al.A Unified Model for Soft Linguistic Reordering Constrains in Statistical Machine Translation[C]∥Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics.Baltimore,Maryland,USA:Association for Computational Linguistics,2014:1123-1133.
[13]SENNRICH R,HADDOW B.Linguistic Input Features Improve Neural Machine Translation[C]∥Proceedings of the First Conference on Machine Translation.Berlin,Germany:Association for Computational Linguistics,2016:83-91.
[14]LI J H,XIONG D Y,TU Z P,et al.Modeling Source Syntax for Neural Machine Translation[C]∥Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Vancouver,Canada:Association for Computational Linguistics,2017:688-697.
[15]ERIGUCHI A,HASHIMOTO K,TSURUOKA Y.Tree-to-Sequence Attentional Neural Machine Translation[C]∥Procee-dings of the 54th Annual Meeting of the Association for Computational Linguistics.Berlin,Germany:Association for Computational Linguistics,2016:823-833.
[16]CHEN H D,HUANG S J,CHIANG D,et al.Improved Neural Machine Translation with a Syntax-Aware Encoder and Decoder[C]∥Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Vancouver,Canada:Association for Computational Linguistics,2017:1936-1945.
[17]CHEN K H,WANG R,UTIYAMA M,et al.Neural Machine Translation with Source Dependency Representation[C]∥Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing.Copenhagen,Denmark:Association for Computational Linguistics,2017:2846-2852.
[18]WU S Z,ZHOU M,ZHANG D D.Improved Neural Machine Translation with Source Syntax[C]∥Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence.Melbourne,Australia:IJCAI,2017:4179-4185.
[19]AHARONI R,GOLDBERG Y.Towards String-to-Tree Neural Machine Translation[C]∥Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Vancouver,Canada:Association for Computational Linguistics,2017:132-140.
[20]MORISHITA M,SUZUKI J,NAGATA M.Improving NeuralMachine Translation by Incorporating Hierarchical Subword Features[C]∥Proceedings of the 27th International Conference on Computational Linguistics.Santa Fe,New-Mexico,USA:COLING,2018:618-629.
[21]XIONG D Y,LI J H,WANG X,et al.Neural Machine Translation with Constraints[J].Scientia Sinica Informationis,2018,48(5):574-588.
[22]WANG Q,DUAN X Y.Neural Machine Translation Based on Attention Convolution[J].Computer Science,2018,45(11):226-230.
[23]CHO K,MERRIENBOER B V,BAHDANAU D.On the Properties of Neural Machine Translation:Encoder-Decoder Approaches[C]∥Proceedings of SSST-8,Eighth Workshop on Syntax,Semantics and Structure in Statistical Translation.Doha,Qatar:Association for Computational Linguistics,2014:103-111.
[24]CHUNG J,GULCEHRE C,CHO K,et al.Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling[C]∥Proceedings of the Twenty-eighth Conference on Neural Information Processing Systems.Montreal,Quebec,Canada:NIPS,2014:1-9.
[25]ZEILER M D.An Adaptive Learning Rate Method[J].arXiv:1212.5701.
[26]PETROV S,KLEIN D.Improved Inference for Unlexicalized Parsing[C]∥Proceedings of the 2007 Annual Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.Rochester,NY:Association for Computational Linguistics,2007:404-411.
[27]LI J H,ZHOU G D,HWEE T N.Joint Syntactic and Semantic Parsing of Chinese[C]∥Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics.Uppsala,Sweden:Association for Computational Linguistics,2010:1108-1117.
[28]PAPINENI K,ROUKOS S,WARD T,et al.BLEU:a Method for Automatic Evaluation of Machine Translation[C]∥Procee-dings of the 40th Annual Meeting of the Association for Computational Linguistics.Philadelphia:Association for Computational Linguistics,2002:311-318.
[29]SNOVER M,DORR B,SCHWARTZ R,et al.A Study of Translation Edit Rate with Targeted Human Annotation[C]∥Proceedings of Association for Machine Translation in the Americas.2006:231-231.
[30]ISOZAKI H,HIRAO T,DUH K,et al.Automatic Evaluation of Translation Quality for Distant Language Pairs[C]∥Procee-dings of the 2010 Conference on Empirical Methods in Natural Language Processing.MIT,Massachusetts:Association for Computational Linguistics,2010:944-952.

相关文章 15

[1]	邓维斌, 朱坤, 李云波, 胡峰. FMNN:融合多神经网络的文本分类模型 FMNN:Text Classification Model Fused with Multiple Neural Networks 计算机科学, 2022, 49(3): 281-287. https://doi.org/10.11896/jsjkx.210200090
[2]	刘俊鹏, 苏劲松, 黄德根. 融合特定语言适配模块的多语言神经机器翻译 Incorporating Language-specific Adapter into Multilingual Neural Machine Translation 计算机科学, 2022, 49(1): 17-23. https://doi.org/10.11896/jsjkx.210900005
[3]	刘妍, 熊德意. 面向小语种机器翻译的平行语料库构建方法 Construction Method of Parallel Corpus for Minority Language Machine Translation 计算机科学, 2022, 49(1): 41-46. https://doi.org/10.11896/jsjkx.210900012
[4]	王士浩, 王中卿, 李寿山, 周国栋. 基于门控图卷积与动态依存池化的事件论元抽取 Event Argument Extraction Using Gated Graph Convolution and Dynamic Dependency Pooling 计算机科学, 2021, 48(11A): 52-56. https://doi.org/10.11896/jsjkx.201200259
[5]	高楠,李利娟,李伟,祝建明. 融合语义特征的关键词提取方法 Keywords Extraction Method Based on Semantic Feature Fusion 计算机科学, 2020, 47(3): 110-115. https://doi.org/10.11896/jsjkx.190700041
[6]	谢念念, 曾凡平, 周明松, 秦晓霞, 吕成成, 陈钊. 多维敏感特征的Android恶意应用检测 Android Malware Detection with Multi-dimensional Sensitive Features 计算机科学, 2019, 46(2): 95-101. https://doi.org/10.11896/j.issn.1002-137X.2019.02.015
[7]	邱少健, 蔡子仪, 陆璐. 基于卷积神经网络的代价敏感软件缺陷预测模型 Cost-sensitive Convolutional Neural Network Model for Software Defect Prediction 计算机科学, 2019, 46(11): 156-160. https://doi.org/10.11896/jsjkx.191100502C
[8]	刘颖, 张帅, 葛瑜祥, 王富平, 李大湘. 轮胎花纹图像检索技术综述 Survey of Tire Pattern Image Retrieval Techniques 计算机科学, 2018, 45(12): 52-60. https://doi.org/10.11896／j.issn.1002-137X.2018.12.007
[9]	汪琪, 段湘煜. 基于注意力卷积的神经机器翻译 Neural Machine Translation Based on Attention Convolution 计算机科学, 2018, 45(11): 226-230. https://doi.org/10.11896／j.issn.1002-137X.2018.11.035
[10]	张冬雯,杨鹏飞,许云峰. 基于word2vec和SVMperf的中文评论情感分类研究 Research of Chinese Comments Sentiment Classification Based on Word2vec and SVMperf 计算机科学, 2016, 43(Z6): 418-421. https://doi.org/10.11896/j.issn.1002-137X.2016.6A.099
[11]	金瑛浩，孙立镌. 协同语义特征建模技术研究 Research on Collaborative Semantic Feature Modeling System 计算机科学, 2012, 39(2): 280-282.
[12]	金瑛浩,孙立镌. 基于特征语义的模型表示法研究 Research of Representation Based on Feature Semantic for Models 计算机科学, 2011, 38(1): 286-289.
[13]	孙立镌，金瑛浩. 基于充分性原理的特征交互检测策略 Strategy of Feature Interaction Based on the Sufficiency Principle 计算机科学, 2010, 37(8): 270-272.
[14]	. 事件信息抽取中语义角色标注研究计算机科学, 2008, 35(3): 155-157.
[15]	任家东岳丽文. SXBP：基于Pri—order编码的XML文档存储方法计算机科学, 2007, 34(4): 116-118.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed