Computer Science ›› 2016, Vol. 43 ›› Issue (10): 200-205.doi: 10.11896/j.issn.1002-137X.2016.10.038

Previous Articles     Next Articles

Study of Automatic Proofreading Method for Non-multi-character Word Error in Chinese Text

LIU Liang-liang and CAO Cun-gen   

  • Online:2018-12-01 Published:2018-12-01

Abstract: Aiming at the insufficiency existing in current automatic proofreading method of Chinese non-multi-character word error,a method based on fuzzy segmentation of automatic error-detecting and automatic proofreading for ‘non-multi-character word error’ in Chinese texts was proposed.Firstly,we used exact matching algorithm for exact word segmentation and fuzzy matching algorithm for Chinese strings to full fuzzy segmentation,and built the word graph.In order to get the best segmentation results,a method based on an improved language model is used to solve the shortest path.After that,the ‘non-multi-character word errors’ in Chinese texts are achieved to be automatic error-detected and automatic proofreaded.The test set has 20000 sentences of domain question answering log which include 664 non-multi-character word errors.Experiments show that the method proposed in our paper can detect the ‘non-multi-character word error’ effectively including character substitution errors,deletion errors and insertion errors.The error-detection recall rate is 75.9% and the error-detection accuracy rate is 85%.The proposed method combines automatic error-detecting and automatic error-correction together.

Key words: Non-word error,Non-multi-character word error,Fuzzy matching,Word graph

[1] Kukich K.Techniques for automatically correcting words in text[J].ACM Computing Surveys (CSUR),1992,24(4):377-439
[2] Zhang Yang-sen,Cao Yuan-da,Yu Shi-wen.A Hybrid Model of Combining Rule-based and Statistics-based Approaches for Automatic Detecting Errors in Chinese Text[J].J ournal of Chinese Information Processing,2006,20:1-7(in Chinese) 张仰森,曹元大,俞士汶.基于规则与统计相结合的中文文本自动查错模型与算法[J].中文信息学报,2006,20:1-7
[3] Shi De-sheng,Wang Liang-zhi,Chen Zhi-da,et al.A Statistics-based Approach for Automatic Detecting Errors in Chinese Text[J].Computer and Communications,1992,8:19-26(in Chinese) 施得胜,王良志,陈志达,等.基于统计的中文错字侦测法[J].电脑与通讯,1992,8:19-26
[4] Zhang Zhao-huang.Automatic Error Detection and Correction of Chinese Text[J].Communications of COLIPS,1994,4(2):143-149(in Chinese) 张照煌.中文错别字自动订正方法初探[J].Communications of COLIPS,1994,4(2):143-149
[5] Zhang L,Zhou M,Huang C,et al.Multifeature-based approach to automatic error detection and correction of Chinese text[C]∥Proceedings of the First Workshop on Natural Language Processing and Neural Networks.2000:2744-2748
[6] Ma Jin-shan,Zhang Yu,Liu Ting,et al.Detecting Chinese Text Errors Based on Trigram and Dependency Parsing[J].Journal of the China Society for Scientific and Technical Information,2005,23(6):723-728(in Chinese) 马金山,张宇,刘挺,等.利用三元模型及依存分析查找中文文本错误[J].情报学报,2005,23(6):723-728
[7] Zhang Yang-sen,Cao Yuan-da,Yu Shi-wen.A Hybrid Model of Combining Rule-based and Statistics-based Approaches for Automatic Detecting Errors in Chinese Text[J].J ournal of Chinese Information Processing,2006,20(4):1-7(in Chinese) 张仰森,曹元大,俞士汶.基于规则与统计相结合的中文文本自动查错模型与算法[J].中文信息学报,2006,20(4):1-7
[8] Zhang Yang-sen,Ding Bing-qing.Automatic Errors Detecting of Chinese Texts Based on the Bi-neighborship[J].Journal of Chinese Information Processing,2001,15(3):36-43(in Chinese) 张仰森,丁冰青.基于二元接续关系检查的字词级自动查错方法[J].中文信息学报,2001,15(3):36-43
[9] Zhang Lei,Sun Mao-song,Zhou Ming,et al.Automatic Chinese Text Error Correction Approach Based on Fast Approximate Chinese Word-Matching Algorithm[C]∥Procceding of the 3rd World Congress on Intelligent Control and Automation.2000:2739-2743(in Chinese) 张磊,孙茂松,周明,等.基于快速模糊词匹配算法的中文自动校对方法[C]∥第三届全球智能控制与自动化大会.2000:2739-2743
[10] Liu Liang-liang,Wang Shi,Wang Dong-sheng,et al.AutomaticText Error Detection in Domain Question Answering[J].Journal of Chinese Information Processing,2013,27(3):77-83(in Chinese) 刘亮亮,王石,王东升,等.领域问答系统中的文本错误自动发现方法[J].中文信息学报,2013,27(3):77-83
[11] 朱德熙.语法讲义[M].商务印书馆,1982
[12] Zhang Hua-ping,Liu Qun.Model of Chinese Words Rough Segmentation Based on N-Shortest-Paths Method[J].Journal of Chinese Information Processing,2002,6(5):1-7(in Chinese) 张华平,刘群.基于N-最短路径方法的中文词语粗分模型[J].中文信息学报,2002,6(5):1-7
[13] 丰强泽,曹存根.语音查询中的辨音方法[M].Google Patents.CN Patent App.CN02,0,272
[14] 王石,王卫民,符建辉.一种汉字字形认知相似度计算方法[M].Google Patents,28.CN Patent App.CN 201,0,205,7
[15] Wang Si-li,Zhang Hua-ping,Wang Bin.Research of Optimization on Double-Array Trie and its Application[J].Journal of Chinese Information Processing,2006,20:24-30(in Chinese) 王思力,张华平,王斌.双数组Trie树算法优化及其应用研究[J].中文信息学报,2006,20:24-30

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!