Computer Science

Computer Science ›› 2022, Vol. 49 ›› Issue (11): 221-227.doi: 10.11896/jsjkx.210900247

• Artificial Intelligence • Previous Articles     Next Articles

Incorporating Part of Speech and Tonal Features for Vietnamese Grammatical Error Detection

ZHANG Zhou, ZHU Jun-guo, YU Zheng-tao   

  1. School of Information Engineering and Automation,Kunming University of Science and Technology,Kunming 650500,China
    Yunnan Key Laboratory of Artificial Intelligence,Kunming University of Science and Technology,Kunming 650500,China
  • Received:2021-09-28 Revised:2022-03-22 Online:2022-11-15 Published:2022-11-03
  • About author:ZHANG Zhou,born in 1992,postgra-duate.His main research interests include natural language processing and grammatical error correction.
    ZHU Jun-guo,born in 1982,Ph.D,lecturer,is a member of China Computer Federation.His main research interests include natural language processing and machine translation.
  • Supported by:
    National Natural Science Foundation of China(62166022,61732005,61866020),Yunnan Provincial Major Science and Technology Special Plan(202002AD080001,202103AA080015),General Project of Yunnan Provincial Department of Science and Technology(202101AT070077) and People Training Project of Yunnan Province(KKSY201903018).

PDF (PC)

597

Abstract: The BERT pre-trained language model removes the tones of the syllables when segmenting Vietnamese words,which leads to the loss of some semantic information during the training process of grammatical error detection model.To address this problem,an approach combining part of speech and tonal features is proposed to complete the semantic information of the input syllables.Grammatical error detection task confronts the problem of insufficient training data due to the scarcity of labeled Vietnamese data.To address this problem,a data augmentation algorithm is designed to generate a large number of error texts from the correct corpus.Experimental results on Vietnamese Wikipedia and news corpus show that the proposed method achieves the highest F0.5 and F1 score on the test set,which proves it improves the detection performance.Both the proposed method and the baseline model method have a gradual improvement with the increasing scales of the generated data,which proves that the proposed data augmentation algorithm is effective.

Key words: Pre-trained language model, Vietnamese grammatical error detection, Feature fusion, Data augmentation

CLC Number: 

  • TP391
[1]MADI N,AL-KHALIFA H S.Grammatical Error CheckingSystems:A Review of Approaches and Emerging Directions[C]//Proceedings of the Thirteenth International Conference on Digital Information Management.2019:142-147.
[2]YIN C,WU M.Survey on N-gram Model [J].Computer Systems and Applications,2018,27(10):33-38.
[3]PETERS M,NEUMANN M,IYYER M,et al.Deep Contextua-lized Word Representations[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2018:2227-2237.
[4]DEVLIN J,CHANG M W,LEE K,et al.BERT:Pre-training of Deep Bidirectional Transformers for Language Understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:Human Language Technologies.2019:4171-4186.
[5]MACDONALD N,FRASE L,GINGRICH P,et al.The Writer’sWorkbench:Computer Aids for Text Analysis[J].IEEE Tran-sactions on Communications,1982,30(1):105-110.
[6]FOSTER J,VOGEL C.Parsing Ill-Formed Text Using an Error Grammar[J].Artificial Intelligence Review,2004,21(3):269-291.
[7]TETREAULT J R,CHODOROW M.The Ups and Downs ofPreposition Error Detection in ESL Writing[C]//Proceedings of the 22nd International Conference on Computational Linguistics.2008:865-872.
[8]NGUYEN P H,NGO T D,PHAN D A,et al.Vietnamese Spel-ling Detection and Correction Using Bi-gram,Minimum Edit Distance,SoundEx Algorithms with Some Additional Heuristics[C]//Proceedings of 2008 International Conference on Research,Innovation and Vision for the Future.IEEE,2008:96-102.
[9]HUONG N,DANG T T,NGUYEN T T,et al.Using Large N-gram for Vietnamese Spell Checking[C]//Proceedings of the Sixth International Conference on Knowledge and Systems Engineering.2015:617-627.
[10]MIKOLOV T,CHEN K,CORRADO G,et al.Efficient Estimation of Word Representations in Vector Space[C]//Proceedings of International Conference on Learning Representations.2013:1-12.
[11]REI M.Semi-supervised Multitask Learning for Sequence Labeling[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.2017:2121-2130.
[12]REI M,YANNAKOUDAKIS H.Compositional Sequence Labeling Models for Error Detection in Learner Writing[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics.2016:1181-1191.
[13]HOCHREITER S,SCHMIDHUBER J.Long Short-Term Me-mory[J].Neural Computation,1997,9(8):1735-1780.
[14]PISLAR M,REI M.Seeing Both the Forest and the Trees:Multi-head Attention for Joint Classification on Different Compositional Levels[C]//Proceedings of the 28th International Conference on Computational Linguistics.2020:3761-3775.
[15]MASAHIRO K,MAMORU K.Multi-Head Multi-Layer Attention to Deep Language Representations for Grammatical Error Detection[J].Computación y Sistemas,2019,23(3):883-891.
[16]ZHANG J H.Combining GCN and Transformer for ChineseGrammatical Error Detection[J].arXiv:2105.09085,2021.
[17]KIPF T N,WELLING M.Semi-Supervised Classification withGraph Convolutional Networks[C]//Proceedings of 5th International Conference on Learning Representation.2017:1-14.
[18]RAO G Q,YANG E H,ZHANG B L.Overview of NLPTEA-2020 Shared Task for Chinese Grammatical Error Diagnosis[C]//Proceedings of the 6th Workshop on Natural Language Processing Techniques for Educational Applications.2020:25-35.
[19]PIRES T,SCHLINGER E,GARRETTE D.How Multilingual is Multilingual BERT?[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.2019:4996-5001.
[20]VASWANI A,SHAZEER N,PARMAR N,et al.Attention isAll You Need[C]//Proceedings of Advances in Neural Information Processing Systems.2017:5998-6008.
[21]TAN Z C,XU F Y,LIN L.Basic Vietnamese[M].Beijing:World Publishing Corporation,2013:3-95.
[22]PENNINGTON J,SOCHER R,MANNING C D.GloVe:Global Vectors for Word Representation [C]//Proceedings of the Conference on Empirical Methods in Natural Language Processing.2014:1532-1543.
[23]STRUBELL E,VERGA P,BELANGER D,et al.Fast and Accurate Entity Recognition with Iterated Dilated Convolutions[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing.2017:2670-2680.
[1] YANG Bing-xin, GUO Yan-rong, HAO Shi-jie, Hong Ri-chang. Application of Graph Neural Network Based on Data Augmentation and Model Ensemble in Depression Recognition [J]. Computer Science, 2022, 49(7): 57-63.
[2] ZHANG Ying-tao, ZHANG Jie, ZHANG Rui, ZHANG Wen-qiang. Photorealistic Style Transfer Guided by Global Information [J]. Computer Science, 2022, 49(7): 100-105.
[3] CHENG Cheng, JIANG Ai-lian. Real-time Semantic Segmentation Method Based on Multi-path Feature Extraction [J]. Computer Science, 2022, 49(7): 120-126.
[4] YU Shu-hao, ZHOU Hui, YE Chun-yang, WANG Tai-zheng. SDFA:Study on Ship Trajectory Clustering Method Based on Multi-feature Fusion [J]. Computer Science, 2022, 49(6A): 256-260.
[5] YANG Yue, FENG Tao, LIANG Hong, YANG Yang. Image Arbitrary Style Transfer via Criss-cross Attention [J]. Computer Science, 2022, 49(6A): 345-352.
[6] WANG Jian-ming, CHEN Xiang-yu, YANG Zi-zhong, SHI Chen-yang, ZHANG Yu-hang, QIAN Zheng-kun. Influence of Different Data Augmentation Methods on Model Recognition Accuracy [J]. Computer Science, 2022, 49(6A): 418-423.
[7] CHEN Yong-ping, ZHU Jian-qing, XIE Yi, WU Han-xiao, ZENG Huan-qiang. Real-time Helmet Detection Algorithm Based on Circumcircle Radius Difference Loss [J]. Computer Science, 2022, 49(6A): 424-428.
[8] SUN Jie-qi, LI Ya-feng, ZHANG Wen-bo, LIU Peng-hui. Dual-field Feature Fusion Deep Convolutional Neural Network Based on Discrete Wavelet Transformation [J]. Computer Science, 2022, 49(6A): 434-440.
[9] LAN Ling-xiang, CHI Ming-min. Remote Sensing Change Detection Based on Feature Fusion and Attention Network [J]. Computer Science, 2022, 49(6): 193-198.
[10] FAN Xin-nan, ZHAO Zhong-xin, YAN Wei, YAN Xi-jun, SHI Peng-fei. Multi-scale Feature Fusion Image Dehazing Algorithm Combined with Attention Mechanism [J]. Computer Science, 2022, 49(5): 50-57.
[11] LI Fa-guang, YILIHAMU·Yaermaimaiti. Real-time Detection Model of Insulator Defect Based on Improved CenterNet [J]. Computer Science, 2022, 49(5): 84-91.
[12] DONG Qi-da, WANG Zhe, WU Song-yang. Feature Fusion Framework Combining Attention Mechanism and Geometric Information [J]. Computer Science, 2022, 49(5): 129-134.
[13] LI Peng-zu, LI Yao, Ibegbu Nnamdi JULIAN, SUN Chao, GUO Hao, CHEN Jun-jie. Construction and Classification of Brain Function Hypernetwork Based on Overlapping Group Lasso with Multi-feature Fusion [J]. Computer Science, 2022, 49(5): 206-211.
[14] XU Hua-jie, QIN Yuan-zhuo, YANG Yang. Scene Recognition Method Based on Multi-level Feature Fusion and Attention Module [J]. Computer Science, 2022, 49(4): 209-214.
[15] GAO Xin-yue, TIAN Han-min. Droplet Segmentation Method Based on Improved U-Net Network [J]. Computer Science, 2022, 49(4): 227-232.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
Add:18# Honghu West Rd., Yubei ,Chongqing,China    Post Code: 401121
Tel: 023-63500828/023-67039612/023-67039623
E-mail: jsjkx12@163.com
Powered by Beijing Magtech Co., Ltd.