计算机科学 ›› 2023, Vol. 50 ›› Issue (3): 315-322.doi: 10.11896/jsjkx.220100137

• 人工智能 • 上一篇    下一篇

基于多视角建模的汉语议论文写作质量评估方法

贺亚琼, 蒋峰, 褚晓敏, 李培峰   

  1. 苏州大学计算机科学与技术学院 江苏 苏州 215006
  • 收稿日期:2022-01-14 修回日期:2022-08-14 出版日期:2023-03-15 发布日期:2023-03-15
  • 通讯作者: 李培峰(pfli@suda.edu.cn)
  • 作者简介:(20204227070@stu.suda.edu.cn)
  • 基金资助:
    国家自然科学基金(61836007,62006167);江苏省高校优势学科建设工程资助项目

Chinese Argumentative Writing Quality Evaluation Based on Multi-perspective Modeling

HE Yaqiong, JIANG Feng, CHU Xiaomin, LI Peifeng   

  1. School of Computer Science and Technology,Soochow University,Suzhou,Jiangsu 215006,China
  • Received:2022-01-14 Revised:2022-08-14 Online:2023-03-15 Published:2023-03-15
  • About author:HE Yaqiong,born in 1997,postgra-duate,is a member of China Computer Federation.His main research interests include natural language processing and so on.
    LI Peifeng,born in 1971,Ph.D,professor,Ph.D supervisor,is a member of China Computer Federation.His main research interests include natural language processing and machine learning.
  • Supported by:
    National Natural Science Foundation of China(61836007,62006167) and Priority Academic Program Development of Jiangsu Higher Education Institutions(PAPD).

摘要: 自动作文评分是一项代替人工为学生作文进行等级评分的任务,其中丰富的语义、严密的组织和合理的逻辑是重要的考虑因素。已有的研究大多数只从语义或组织等视角出发评估作文的质量,未考虑如逻辑等更高层次的因素。因此,文中提出了一个多视角评价框架(Multi-perspective Evaluation Framework,MPE),从语义表达、组织结构和整体逻辑3个方面对学生议论文进行了客观、可靠的评价。具体来说,多视角评价框架首先利用预训练模型编码句子并获得由低到高3个层次的语义信息,来评估文章的语义表达;其次,框架将句子功能识别与段落功能识别相结合,用于评估文章的组织结构;然后,通过计算段落之间的连贯性来评估文章的整体逻辑;最后,该框架综合这3个方面的评估特征,对作文评分。实验结果表明,所提出的多视角评价框架能够有效地对不同质量的作文进行评分,优于所有基准系统。

关键词: 多视角, 作文评分, 议论文, XLNet, 全局连贯性

Abstract: Automated essay scoring is a task that replaces manual grading for students’ essays,where rich semantics,rigorous organization,and reasonable logic are important considering factors.Most previous studies only consider the semantics or organization of the essay from a single perspective,lacking considering higher-level factors such as logic.Therefore,this paper proposes a multi-perspective evaluation framework(MPE) to more objective and reliable evaluate the essay from semantics,organization,and logic.MPE first utilizes the pre-trained model to encode sentence and obtain three levels semantic information to evaluate the essay's semantic expression.Then,it combines sentence function identification and paragraph function identification to evaluate the essay′s organization.Moreover,MPE evaluates the essay's logic by calculating the coherence between paragraphs.Finally,the framework scores the essay by integrating these three evaluation perspectives.Experimental results show that the proposed multi-perspective evaluation framework can effectively score the essays at various qualities,outperforming all the baselines.

Key words: Multi-perspective, Essay score, Argumentation, XLNet, Global coherence

中图分类号: 

  • TP391
[1]MESGAR M,STRUBE M.A neural local coherence model fortext quality assessment[C]//Proceedings of the 2018 Confe-rence on Empirical Methods in Natural Language Processing.2018:4328-4339.
[2]LIU J,XU Y,ZHU Y.Automated essay scoring based on two-stage learning[J].arXiv:1901.07744,2019.
[3]YANG Y,ZHONG J.Automated essay scoring via example-based learning[C]//International Conference on Web Enginee-ring.Cham:Springer,2021:201-208.
[4]CHEN H,HE B.Automated essay scoring by maximizing human-machine agreement [C]//Proceedings of the 2013 Confe-rence on Empirical Methods in Natural Language Processing.2013:1741-1752.
[5]SOMASUNDARAN S,BURSTEIN J,CHODOROW M.Lexical chaining for measuring discourse coherence quality in test-taker essays[C]//The 25th International Conference on Computa-tional Linguistics:Technical papers(COLING 2014).2014:950-961.
[6]YANNAOUDAKIS H,BRISCOE T,MEDLOCK B.A newdataset and method for automatically grading ESOL texts[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics:Human Language Technologies.2011:180-189.
[7]PHANDI P,CHAI K M A,NG H T.Flexible domain adaptation for automated essay scoring using correlated linear regression[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.2015:431-439.
[8]ALIKANIOTIS D,YANNAOUDAKIS H,REI M.Automatictext scoring using neural networks[J].arXiv:1606.04289,2016.
[9]TAGHIPOUR K,NG H T.A neural approach to automated essay scoring[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.2016:1882-1891.
[10]DONG F,ZHANG Y.Automatic features for essay scoring-an empirical study[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.2016:1072-1077.
[11]DONG F,ZHANG Y,YANG J.Attention-based recurrent convolutional neural network for automatic essay scoring[C]//Proceedings of the 21st Conference on Computational Natural Language Learning(CoNLL 2017).2017:153-162.
[12]SOMASUNDARAN S,FLOR M,CHODOROW M,et al.To-wards evaluating narrative quality in student writing[J/OL].Transactions of the Association for Computational Linguistics,2018,6:91-106.https://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00007/43428/Towards-Evaluating-Narrative-Qua-lity-In-Student.
[13]PERSING I,NG V.Modeling stance in student essays[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics(Volume 1:Long Papers).2016:2174-2184.
[14]MATHIAS S,BHATTACHARYYA P.Thank “Goodness”! A Way to Measure Style in Student Essays[C]//Proceedings of the 5th Workshop on Natural Language Processing Techniques for Educational Applications.2018:35-41.
[15]KE Z,INAMDAR H,LIN H,et al.Give me more feedback II:Annotating thesis strength and related attributes in student essays[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.2019:3994-4004.
[16]PERSING I,DAVIS A,NG V.Modeling organization in student essays[C]//Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing.2010:229-239.
[17]SONG W,SONG Z,LIU L,et al.Hierarchical Multi-task Lear-ning for Organization Evaluation of Argumentative Student Essays[C]//IJCAI.2020:3875-3881.
[18]CHEN Y.Convolutional neural network for sentence classification[D].Canadian:University of Waterloo,2015.
[19]SHI X J,CHEN Z,WANG H,et al.Convolutional LSTM network:A machine learning approach for precipitation nowcasting[C]//Advances in Neural Information Processing Systems.2015:802-810.
[20]LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-basedlearning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324.
[21]WON Y.The Prediction of Writing Scores Using Vocabulary Features in ESL University Students’ Essays[J].Modern English Education Society,2019,20(4):31-40.
[22]LE Q,MIKOLOV T.Distributed representations of sentences and documents[C]//International Conference on Machine Learning.PMLR,2014:1188-1196.
[23]LIU C,ZHAO S,VOLKOVS M.Unsupervised document embedding with cnns[J].arXiv:1711.04168,2017.
[24]WU L,YEN I E H,XU K,et al.Word mover's embedding:From word2vec to document embedding[J].arXiv:1811.01713,2018.
[25]YANG Z,DAI Z,YANG Y,et al.XLNet:Generalized autore-gressive pretraining for language understaning[J/OL].Advances in neural information processing systems,2019,32.https://proceedings.neurips.cc/paper/2019/hash/dc6a7e655d7e5840e66733e9ee67cc69-Abstract.html.
[26]ATTALI Y,BURSTEIN J.Automated essay scoring with e-ra-ter© V.2[J/OL].The Journal of Technology,Learning and Assessment,2006,4(3).https://ejournals.bc.edu/index.php/jtla/article/view/1650.
[27]LIANG M C.A study of coherence in EFL learners’ writtenproduction [J].Modern Foreign Languages,2006,29(3):284-292.
[28]LOUIS A,NENKOVA A.A coherence model based on syntactic patterns[C]//Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning.2012:1157-1168.
[29]MILTSAKAKI E,KUKICH K.Evaluation of text coherence for electronic essay scoring systems[J].Natural Language Engineering,2004,10(1):25-55.
[30]LIAO D,XU J,LI G,et al.Hierarchical Coherence Modeling for Document Quality Assessment[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2021,35(15):13353-13361.
[31]XU W C.Cohesion,coherence and quality in English composition[J].Journal of GuangZhou University,2000(5):71-75.
[32]MA G G.A comparative analysis of the linguistic features ofEnglish composition between Chinese and American College Students[J].Foreign Language Teaching Research,2002,34(5):345-350.
[33]ZHU Y S.Halliday's standard of discourse coherence is misunderstood by the outside world and its own shortcomings[J].Foreign Language Teaching and Research,1997(1):23-27.
[34]MCNAMARA D S,LOUWERSE M M,GRAESSER A C.CohMetrix:Automated cohesion and coherence scoresto predict text readability an-d facilitate comprehension[R].Technical report,Institute for Intelligent Systems,University of Memphis,Memphis,TN,2002.
[35]VASWANIA,SHAZEER N,PARMAR N,et al.Attention is all you need[C]//Advances in Neural Information Processing Systems.2017:5998-6008.
[36]乐乐.问题[OL].http://www.leleketang.com/zuowen/287886.shtml.
[1] 李斌, 万源.
基于相似度矩阵学习和矩阵校正的无监督多视角特征选择
Unsupervised Multi-view Feature Selection Based on Similarity Matrix Learning and Matrix Alignment
计算机科学, 2022, 49(8): 86-96. https://doi.org/10.11896/jsjkx.210700124
[2] 陈佳舟, 赵熠波, 徐阳辉, 马骥, 金灵枫, 秦绪佳.
三维城市场景中的小物体检测
Small Object Detection in 3D Urban Scenes
计算机科学, 2022, 49(6): 238-244. https://doi.org/10.11896/jsjkx.210400174
[3] 冷佳旭, 谭明圮, 胡波, 高新波.
基于隐式视角转换的视频异常检测
Video Anomaly Detection Based on Implicit View Transformation
计算机科学, 2022, 49(2): 142-148. https://doi.org/10.11896/jsjkx.210900266
[4] 张帆, 贺文琪, 姬红兵, 李丹萍, 王磊.
基于块对角化表示的多视角字典对学习
Multi-view Dictionary-pair Learning Based on Block-diagonal Representation
计算机科学, 2021, 48(1): 233-240. https://doi.org/10.11896/jsjkx.200800211
[5] 孟翰, 吴际, 胡京徽, 刘超, 杨海燕, 孙新颖.
硬件系统自动化测试的多视角建模及案例研究
Modeling in Multiple Views and Industrial Case Study of Automatic Test for Hardware System
计算机科学, 2018, 45(9): 75-80. https://doi.org/10.11896/j.issn.1002-137X.2018.09.011
[6] 温雯, 陈颖, 蔡瑞初, 郝志峰, 王丽娟.
基于多视角多标签学习的读者情绪分类
Emotion Classification for Readers Based on Multi-view Multi-label Learning
计算机科学, 2018, 45(8): 191-197. https://doi.org/10.11896/j.issn.1002-137X.2018.08.034
[7] 苏若, 吴际, 刘超, 杨海燕.
基于多视角卡牌模型的需求缺陷检测
Requirement Defect Detection Based on Multi-view Card Model
计算机科学, 2018, 45(10): 183-188. https://doi.org/10.11896/j.issn.1002-137X.2018.10.034
[8] 费鹏,林鸿飞,杨亮,徐博,古丽孜热·艾尼外.
一种用于构建用户画像的多视角融合框架
Multi-view Ensemble Framework for Constructing User Profile
计算机科学, 2018, 45(1): 179-182. https://doi.org/10.11896/j.issn.1002-137X.2018.01.031
[9] 刘冬,秦瑞,陈曦,李庆.
3D车载环视全景生成方法
Generation of Three-dimention Vehicle Panorama
计算机科学, 2017, 44(4): 302-305. https://doi.org/10.11896/j.issn.1002-137X.2017.04.061
[10] 杜琳琳,朱振峰,段红帅,赵耀.
LSPSA:基于局部结构保持的共享子空间分析
Local Structure Preserved Shared-subspace Analysis
计算机科学, 2014, 41(10): 67-71. https://doi.org/10.11896/j.issn.1002-137X.2014.10.015
[11] 尹维冲,路通.
基于多分类器融合的多视角目标检测算法
Novel Framework for Multi-view Object Detection through Combining Multiple Classifiers
计算机科学, 2013, 40(7): 266-269.
[12] 李闪闪,曹存根.
事件前提和后果常识知识分析方法研究
Commonsense Knowledge Analysis Approach Based on Event Preconditions and Effects
计算机科学, 2013, 40(4): 185-192.
[13] 柴艳妹,韩文英,刘灿涛,李海峰.
融合理论在步态识别中的应用研究
Study on Application of Fusion Theory in Gait Recognition
计算机科学, 2012, 39(12): 272-277.
[14] 陈嘉佳 毛新军.
多主体系统开发环境的结构化评价框架SEF及评价结果

计算机科学, 2005, 32(5): 200-205.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!