基于跨语言术语注意力机制的译文质量估计方法

doi:10.11896/jsjkx.250200007

Abstract

Abstract: Translation quality estimation(QE) refers to the assessment of the quality of machine translations in the absence of a reference translation.Existing QE systems perform well in general domains but poorly in specific domains(e.g.,engineering,medicine,law) that contain a large number of specialised terms because they focus on evaluating the semantic similarity between the original text and the translated text,and lack sensitivity to translation bias of specialised terms.In order to solve this pro-blem,this paper proposes a translation quality estimation method based on cross-lingual term attention mechanism.Firstly,a prompt template is designed to guide GPT to complete the recognition of bilingual terms.Secondly,the sentence representation is obtained using the sentence encoding module,and then the enhanced sentence representation is obtained by explicitly fusing the bilingual term information.Then,the bilingual cross-lingual representation is generated using the cross attention mechanism and the semantic similarity value is computed as a term feature.Finally,a Knowledge Enhancement Layer(KEL) is introduced into the QE model to fuse the term features with the neural features output from the model,which is processed by a feed-forward neural network to obtain the predicted scores of the model.Experimental results on English-Chinese engineering literature dataset show that the proposed method improves the main metric of Spearman correlation coefficient by 3.77 percentage points,the auxiliary metrics of Pearson correlation coefficient and Kendall correlation coefficient by 3.07 percentage points and 4.45 percentage points,when compared to the state-of-the-art methods.

Key words: Machine translation quality estimation, Cross attention mechanism, Feature fusion, Middle layer

CLC Number:

TP391

WANG Xueni, YE Na, ZHANG Guiping. Translation Quality Estimation Based on Cross-lingual Term Attention Mechanism[J].Computer Science, 2025, 52(11A): 250200007-9.

References

[1]SPECIAL L,RAJ D,TURCHI M.Machine translation evaluation versus quality estimation[J].Machine Translation,2010,24(1):39-50.
[2]RANASINGHE T,ORĂSAN C,MITKOV R.TransQuest:Translation Quality Estimation with Cross-lingual Transfor-mers[C]//Proceedings of the 28th International Conference on Computational Linguistics.2020:5070-5081.
[3]CONNEAU A,KHANDELWAL K,GOYAL N,et al.Unsupervised cross-lingual representation learning at scale[J].arXiv:1911.02116,2019.
[4]REI R,STEWART C,FARINHA A C,et al.COMET:A Neural Framework for MT Evaluation[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Proces-sing(EMNLP).2020:2685-2702.
[5]ZERVA C,BLAIN F,REI R,et al.Findings of the WMT 2022 Shared Task on Quality Estimation[C]//Proceedings of the Seventh Conference on Machine Translation(WMT).2022:69-99.
[6]FREITAG M,REI R,MATHUR N,et al.Results of theWMT21 metrics shared task:Evaluating metrics with expert-based human evaluations on TED and news domain[C]//Proceedings of the Sixth Conference on Machine Translation.2021:733-774.
[7]RUO L,HE X,LI M.A review of research on sentence level machine translation quality estimation [J].Intelligence Engineering,2022,8(2):34-50.
[8]SPECIA L,SHAH K,DE SOUZA J G C,et al.QuEst-A translation quality estimation framework[C]//Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics:System Demonstrations.2013:79-84.
[9]HARDMEIER C,NIVRE J,TIEDEMANN J.Tree kernels formachine translation quality estimation [C]//Seventh Workshop on Statistical Machine Translation,Montréal,Canada,June 7-8,2012.Association for Computational Linguistics,2012:109-113.
[10]RUBINO R,FOSTER J,WAGNER J,et al.DCU-Symantecsubmission for the WMT 2012 Quality Estimation Task [C]//Proceedings of the Seventh Workshop on Statistical Machine Translation.2012:138-144.
[11]SPECIA L,GIMÉNEZ J.Combining confidence estimation and reference-based metrics for segment-level MT evaluation [C]//The Ninth Conference of the Association for Machine Translation in the Americas.2010.
[12]DENG H,XIONG D.A Review of Quality Estimation in Ma-chine Translation Translation [J].Chinese Journal of Information Science,2022,36(11):20-37.
[13]SHAH K,BOUGARES F,BARRAULT L,et al.Shef-lium-nn:Sentence level quality estimation with neural network features [C]//Proceedings of the First Conference on Machine Translation:Volume 2,Shared Task Papers.2016:838-842.
[14]SCHWENK H.Continuous space translation models for phrase-based statistical machine translation [C]//Proceedings of COLING 2012:Posters.2012:1071-1080.
[15]CHEN Z,LI M,WANG M.Sentence level translation quality estimation based on neural network features [J].Computer Research and Development,2017,54(8):1804-1812.
[16]YE N,LI T,CAI D,et al.Improving Neural Translation Quality Estimation by Using Dependency Syntax Relationships [J].Chinese Journal of Information Science,2021,35(9):46-57.
[17]KIM H,LEE J H.A recurrent neural networks approach for estimating the quality of machine translation output [C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2016:494-498.
[18]SUN X,ZHU C,ZHAO T.Machine Translation Quality Estimation Algorithm Integrating Translation Knowledge [J].Intelligent Computers and Applications,2019,9(2):279-283.
[19]LI P,LI M,QIU B,et al.Research on Translation QualityEstimation Method Integrating BERT Context Word Vectors [J].Chinese Journal of Information Science,2020,34(3):56-63.
[20]DEVLIN J,CHANG MW,LEE K,et al.BERT:Pre-training of Deep Bidirectional Transformers for Language Understanding [J].arXiv:1810.04805,2018.
[21]KIM H,LEE J H,NA S H.Predictor-estimator using multilevel task learning with stack propagation for neural quality estimation [C]//Proceedings of the Second Conference on Machine Translation.2017:562-568.
[22]LIU L,FUJITA A,UTIYAMA M,et al.Translation quality estimation using only bilingual corpora [J].IEEE/ACM Transactions on Audio,Speech,and Language Processing,2017,25(9):1762-1772.
[23]LI S,BI X,LIU T,et al.Information Dropping Data Augmentation for Machine Translation Quality Estimation[J].IEEE/ACM Transactions on Audio,Speech,and Language Processing,2024,32:2112-2124.
[24]WU H,YANG M,WANG J,et al.Target oriented data generation for quality estimation of machine translation [C]//Natural Language Processing and Chinese Computing:8th CCF International Conference(NLPCC 2019).Springer International Publishing,2019:393-405.
[25]KEPLER F,TRÉNOZ J,TRIVISO M,et al.Unbabel’s Partici-pation in the WMT19 Translation Quality Estimation Shared Task [J].arXiv:1907.10352,2019.
[26]SNOVER M,DORR B,SCHWARTZ R,et al.A study of translation edit rate with targeted human annotation [C]//Procee-dings of the 7th Conference of the Association for Machine Translation in the Americas:Technical Papers.2006:223-231.
[27]YE N,LI J.A k-Nearest Neighbor Approach for Domain-Speci-fic Translation Quality Estimation [C]//China Conference on Machine Translation.Singapore:Springer Nature Singapore,2023:69-80.
[28]DE SOUZA J G C,TURCHI M,NEGRI M.Machine translation quality estimation across domains [C]//The 25th International Conference on Computational Linguistics:Technical Papers(COLING 2014).2014:409-420.
[29]DE SOUZA J G C,NEGRI M,RICCI E,et al.Online multitask learning for machine translation quality estimation [C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing(Volume 1:Long Papers).2015:219-228.
[30]SHARAMI J P R,SHTERIONOV D,BLAIN F,et al.Tailoring Domain Adaptation for Machine Translation Quality Estimation [C]//Proceedings of the 24th Annual Conference of the European Association for Machine Translation.2023:9-20.
[31]WEI X,ZHANG T,LI Y,et al.Multi-modality Cross Attention Network for Image and Sentence Matching [C]//Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition.2020:10941-10950.
[32]LI J,SELVARAJU R R,GOTMARE A D,et al.Align beforeFuse:Vision and Language Representation Learning with Momentum Distillation[J].arXiv:2107.07651,2021.
[33]LAI S,YANG Z,MENG F,et al.Cross-Align:Modeling Deep Cross-lingual Interactions for Word Alignment[C]//Procee-dings of the 2022 Conference on Empirical Methods in Natural Language Processing.2022:3715-3725.
[34]OUYANG L,WU J,JIANG X,et al.Training language models to follow instructions with human feedback[J].arXiv:2203.02155,2022.
[35]AMRHEIN C,SENNRICH R.Identifying weaknesses in ma-chine translation metrics through minimum Bayes risk decoding:A case study for COMET [J].arXiv:2202.05148,2022.
[36]GLUSHKOVA T,ZERVA C,MARTINS A F T.BLEU Meets COMET:Combining Lexical and Neural Metrics Towards Robust Machine Translation Evaluation [C]//Proceedings of the 24th Annual Conference of the European Association for Machine Translation.2023:47-58.
[37]GUO L,WANG L,DANG J,et al.A feature fusion method based on extreme learning machine for speech emotion recognition [C]//IEEE International Conference on Acoustics,Speech and Signal Processing(ICASSP 2018).IEEE,2018:2666-2670.
[38]DING N,TIAN S,YU L.A multimodal fusion method for sarcasm detection based on late fusion [J].Multimedia Tools and Applications,2022,81(6):8597-8616.
[39]MOURA J,VERA M,VAN STIGT D,et al.Ist-unbabel partici-pation in the WMT20 quality estimation shared task [C]//Proceedings of the Fifth Conference on Machine Translation.2020:1029-1036.
[40]SHIMANAKA H,KAJIWARA T,KOMACHI M.RUSE:Regressor using sentence embeddings for automatic machine translation evaluation [C]//Proceedings of the Third Conference on Machine Translation:Shared Task Papers.2018:751-758.
[41]HUANG H,DI H,LI C,et al.BJTU-Toshiba’s Submission toWMT22 Quality Estimation Shared Task [C]//Proceedings of the Seventh Conference on Machine Translation(WMT).2022:621-626.
[42]LAMPLE G,CONNEAU A.Cross-lingual language model pretraining [J].arXiv:1901.07291,2019.
[43]TANG Y,TRAN C,LI X,et al.Multilingual translation with extensible multilingual pretraining and finetuning [J].arXiv:2008.00401,2020.
[44]WOLF T,DEBUT L,SANH V,et al.Transformers:State-of-the-art natural language processing[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing:System Demonstrations.2020:38-45.
[45]GENG X,ZHANG Y,HUANG S,et al.Njunlp’s participation for the wmt2022 quality estimation shared task[C]//Procee-dings of the Seventh Conference on Machine Translation(WMT).2022:615-620.

Metrics

Viewed

Full text

Abstract

Cited

Shared

Discussed

Comments

Recommended 0

No Suggested Reading articles found!

Translation Quality Estimation Based on Cross-lingual Term Attention Mechanism

PDF (PC)

Abstract

Cite this article

share this article

References

Related Articles 15

Metrics

Comments

Recommended 0

[1]	LUO Chi, LU Lingyun, LIU Fei. Partial Differential Equation Solving Method Based on Locally Enhanced Fourier NeuralOperators [J]. Computer Science, 2025, 52(9): 144-151.
[2]	GUO Husheng, ZHANG Xufei, SUN Yujie, WANG Wenjian. Continuously Evolution Streaming Graph Neural Network [J]. Computer Science, 2025, 52(8): 118-126.
[3]	LUO Xuyang, TAN Zhiyi. Knowledge-aware Graph Refinement Network for Recommendation [J]. Computer Science, 2025, 52(7): 103-109.
[4]	LIU Chengzhuang, ZHAI Sulan, LIU Haiqing, WANG Kunpeng. Weakly-aligned RGBT Salient Object Detection Based on Multi-modal Feature Alignment [J]. Computer Science, 2025, 52(7): 142-150.
[5]	XU Yongwei, REN Haopan, WANG Pengfei. Object Detection Algorithm Based on YOLOv8 Enhancement and Its Application Norms [J]. Computer Science, 2025, 52(7): 189-200.
[6]	FANG Chunying, HE Yuankun, WU Anxin. Emotion Recognition Based on Brain Network Connectivity and EEG Microstates [J]. Computer Science, 2025, 52(7): 201-209.
[7]	SHI Xincheng, WANG Baohui, YU Litao, DU Hui. Study on Segmentation Algorithm of Lower Limb Bone Anatomical Structure Based on 3D CTImages [J]. Computer Science, 2025, 52(6A): 240500119-9.
[8]	LI Weirong, YIN Jibin. FB-TimesNet:An Improved Multimodal Emotion Recognition Method Based on TimesNet [J]. Computer Science, 2025, 52(6A): 240900046-8.
[9]	XU Yutao, TANG Shouguo. Visual Question Answering Integrating Visual Common Sense Features and Gated Counting Module [J]. Computer Science, 2025, 52(6A): 240800086-7.
[10]	WANG Rui, TANG Zhanjun. Multi-feature Fusion and Ensemble Learning-based Wind Turbine Blade Defect Detection Method [J]. Computer Science, 2025, 52(6A): 240900138-8.
[11]	LI Mingjie, HU Yi, YI Zhengming. Flame Image Enhancement with Few Samples Based on Style Weight Modulation Technique [J]. Computer Science, 2025, 52(6A): 240500129-7.
[12]	WANG Rong , ZOU Shuping, HAO Pengfei, GUO Jiawei, SHU Peng. Sand Dust Image Enhancement Method Based on Multi-cascaded Attention Interaction [J]. Computer Science, 2025, 52(6A): 240800048-7.
[13]	JIN Lu, LIU Mingkun, ZHANG Chunhong, CHEN Kefei, LUO Yaqiong, LI Bo. Pedestrian Re-identification Based on Spatial Transformation and Multi-scale Feature Fusion [J]. Computer Science, 2025, 52(6A): 240800156-7.
[14]	ZHANG Yongyu, GUO Chenjuan, WEI Hanyue. Deep Learning Stock Price Probability Prediction Based on Multi-modal Feature Wavelet Decomposition [J]. Computer Science, 2025, 52(6A): 240600140-11.
[15]	SHEN Xinyang, WANG Shanmin, SUN Yubao. Depression Recognition Based on Speech Corpus Alignment and Adaptive Fusion [J]. Computer Science, 2025, 52(6): 219-227.