计算机科学 ›› 2023, Vol. 50 ›› Issue (3): 315-322.doi: 10.11896/jsjkx.220100137
贺亚琼, 蒋峰, 褚晓敏, 李培峰
HE Yaqiong, JIANG Feng, CHU Xiaomin, LI Peifeng
摘要: 自动作文评分是一项代替人工为学生作文进行等级评分的任务,其中丰富的语义、严密的组织和合理的逻辑是重要的考虑因素。已有的研究大多数只从语义或组织等视角出发评估作文的质量,未考虑如逻辑等更高层次的因素。因此,文中提出了一个多视角评价框架(Multi-perspective Evaluation Framework,MPE),从语义表达、组织结构和整体逻辑3个方面对学生议论文进行了客观、可靠的评价。具体来说,多视角评价框架首先利用预训练模型编码句子并获得由低到高3个层次的语义信息,来评估文章的语义表达;其次,框架将句子功能识别与段落功能识别相结合,用于评估文章的组织结构;然后,通过计算段落之间的连贯性来评估文章的整体逻辑;最后,该框架综合这3个方面的评估特征,对作文评分。实验结果表明,所提出的多视角评价框架能够有效地对不同质量的作文进行评分,优于所有基准系统。
中图分类号:
[1]MESGAR M,STRUBE M.A neural local coherence model fortext quality assessment[C]//Proceedings of the 2018 Confe-rence on Empirical Methods in Natural Language Processing.2018:4328-4339. [2]LIU J,XU Y,ZHU Y.Automated essay scoring based on two-stage learning[J].arXiv:1901.07744,2019. [3]YANG Y,ZHONG J.Automated essay scoring via example-based learning[C]//International Conference on Web Enginee-ring.Cham:Springer,2021:201-208. [4]CHEN H,HE B.Automated essay scoring by maximizing human-machine agreement [C]//Proceedings of the 2013 Confe-rence on Empirical Methods in Natural Language Processing.2013:1741-1752. [5]SOMASUNDARAN S,BURSTEIN J,CHODOROW M.Lexical chaining for measuring discourse coherence quality in test-taker essays[C]//The 25th International Conference on Computa-tional Linguistics:Technical papers(COLING 2014).2014:950-961. [6]YANNAOUDAKIS H,BRISCOE T,MEDLOCK B.A newdataset and method for automatically grading ESOL texts[C]//Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics:Human Language Technologies.2011:180-189. [7]PHANDI P,CHAI K M A,NG H T.Flexible domain adaptation for automated essay scoring using correlated linear regression[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing.2015:431-439. [8]ALIKANIOTIS D,YANNAOUDAKIS H,REI M.Automatictext scoring using neural networks[J].arXiv:1606.04289,2016. [9]TAGHIPOUR K,NG H T.A neural approach to automated essay scoring[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.2016:1882-1891. [10]DONG F,ZHANG Y.Automatic features for essay scoring-an empirical study[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.2016:1072-1077. [11]DONG F,ZHANG Y,YANG J.Attention-based recurrent convolutional neural network for automatic essay scoring[C]//Proceedings of the 21st Conference on Computational Natural Language Learning(CoNLL 2017).2017:153-162. [12]SOMASUNDARAN S,FLOR M,CHODOROW M,et al.To-wards evaluating narrative quality in student writing[J/OL].Transactions of the Association for Computational Linguistics,2018,6:91-106.https://direct.mit.edu/tacl/article/doi/10.1162/tacl_a_00007/43428/Towards-Evaluating-Narrative-Qua-lity-In-Student. [13]PERSING I,NG V.Modeling stance in student essays[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics(Volume 1:Long Papers).2016:2174-2184. [14]MATHIAS S,BHATTACHARYYA P.Thank “Goodness”! A Way to Measure Style in Student Essays[C]//Proceedings of the 5th Workshop on Natural Language Processing Techniques for Educational Applications.2018:35-41. [15]KE Z,INAMDAR H,LIN H,et al.Give me more feedback II:Annotating thesis strength and related attributes in student essays[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.2019:3994-4004. [16]PERSING I,DAVIS A,NG V.Modeling organization in student essays[C]//Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing.2010:229-239. [17]SONG W,SONG Z,LIU L,et al.Hierarchical Multi-task Lear-ning for Organization Evaluation of Argumentative Student Essays[C]//IJCAI.2020:3875-3881. [18]CHEN Y.Convolutional neural network for sentence classification[D].Canadian:University of Waterloo,2015. [19]SHI X J,CHEN Z,WANG H,et al.Convolutional LSTM network:A machine learning approach for precipitation nowcasting[C]//Advances in Neural Information Processing Systems.2015:802-810. [20]LECUN Y,BOTTOU L,BENGIO Y,et al.Gradient-basedlearning applied to document recognition[J].Proceedings of the IEEE,1998,86(11):2278-2324. [21]WON Y.The Prediction of Writing Scores Using Vocabulary Features in ESL University Students’ Essays[J].Modern English Education Society,2019,20(4):31-40. [22]LE Q,MIKOLOV T.Distributed representations of sentences and documents[C]//International Conference on Machine Learning.PMLR,2014:1188-1196. [23]LIU C,ZHAO S,VOLKOVS M.Unsupervised document embedding with cnns[J].arXiv:1711.04168,2017. [24]WU L,YEN I E H,XU K,et al.Word mover's embedding:From word2vec to document embedding[J].arXiv:1811.01713,2018. [25]YANG Z,DAI Z,YANG Y,et al.XLNet:Generalized autore-gressive pretraining for language understaning[J/OL].Advances in neural information processing systems,2019,32.https://proceedings.neurips.cc/paper/2019/hash/dc6a7e655d7e5840e66733e9ee67cc69-Abstract.html. [26]ATTALI Y,BURSTEIN J.Automated essay scoring with e-ra-ter© V.2[J/OL].The Journal of Technology,Learning and Assessment,2006,4(3).https://ejournals.bc.edu/index.php/jtla/article/view/1650. [27]LIANG M C.A study of coherence in EFL learners’ writtenproduction [J].Modern Foreign Languages,2006,29(3):284-292. [28]LOUIS A,NENKOVA A.A coherence model based on syntactic patterns[C]//Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning.2012:1157-1168. [29]MILTSAKAKI E,KUKICH K.Evaluation of text coherence for electronic essay scoring systems[J].Natural Language Engineering,2004,10(1):25-55. [30]LIAO D,XU J,LI G,et al.Hierarchical Coherence Modeling for Document Quality Assessment[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2021,35(15):13353-13361. [31]XU W C.Cohesion,coherence and quality in English composition[J].Journal of GuangZhou University,2000(5):71-75. [32]MA G G.A comparative analysis of the linguistic features ofEnglish composition between Chinese and American College Students[J].Foreign Language Teaching Research,2002,34(5):345-350. [33]ZHU Y S.Halliday's standard of discourse coherence is misunderstood by the outside world and its own shortcomings[J].Foreign Language Teaching and Research,1997(1):23-27. [34]MCNAMARA D S,LOUWERSE M M,GRAESSER A C.CohMetrix:Automated cohesion and coherence scoresto predict text readability an-d facilitate comprehension[R].Technical report,Institute for Intelligent Systems,University of Memphis,Memphis,TN,2002. [35]VASWANIA,SHAZEER N,PARMAR N,et al.Attention is all you need[C]//Advances in Neural Information Processing Systems.2017:5998-6008. [36]乐乐.问题[OL].http://www.leleketang.com/zuowen/287886.shtml. |
[1] | 李斌, 万源. 基于相似度矩阵学习和矩阵校正的无监督多视角特征选择 Unsupervised Multi-view Feature Selection Based on Similarity Matrix Learning and Matrix Alignment 计算机科学, 2022, 49(8): 86-96. https://doi.org/10.11896/jsjkx.210700124 |
[2] | 陈佳舟, 赵熠波, 徐阳辉, 马骥, 金灵枫, 秦绪佳. 三维城市场景中的小物体检测 Small Object Detection in 3D Urban Scenes 计算机科学, 2022, 49(6): 238-244. https://doi.org/10.11896/jsjkx.210400174 |
[3] | 冷佳旭, 谭明圮, 胡波, 高新波. 基于隐式视角转换的视频异常检测 Video Anomaly Detection Based on Implicit View Transformation 计算机科学, 2022, 49(2): 142-148. https://doi.org/10.11896/jsjkx.210900266 |
[4] | 张帆, 贺文琪, 姬红兵, 李丹萍, 王磊. 基于块对角化表示的多视角字典对学习 Multi-view Dictionary-pair Learning Based on Block-diagonal Representation 计算机科学, 2021, 48(1): 233-240. https://doi.org/10.11896/jsjkx.200800211 |
[5] | 孟翰, 吴际, 胡京徽, 刘超, 杨海燕, 孙新颖. 硬件系统自动化测试的多视角建模及案例研究 Modeling in Multiple Views and Industrial Case Study of Automatic Test for Hardware System 计算机科学, 2018, 45(9): 75-80. https://doi.org/10.11896/j.issn.1002-137X.2018.09.011 |
[6] | 温雯, 陈颖, 蔡瑞初, 郝志峰, 王丽娟. 基于多视角多标签学习的读者情绪分类 Emotion Classification for Readers Based on Multi-view Multi-label Learning 计算机科学, 2018, 45(8): 191-197. https://doi.org/10.11896/j.issn.1002-137X.2018.08.034 |
[7] | 苏若, 吴际, 刘超, 杨海燕. 基于多视角卡牌模型的需求缺陷检测 Requirement Defect Detection Based on Multi-view Card Model 计算机科学, 2018, 45(10): 183-188. https://doi.org/10.11896/j.issn.1002-137X.2018.10.034 |
[8] | 费鹏,林鸿飞,杨亮,徐博,古丽孜热·艾尼外. 一种用于构建用户画像的多视角融合框架 Multi-view Ensemble Framework for Constructing User Profile 计算机科学, 2018, 45(1): 179-182. https://doi.org/10.11896/j.issn.1002-137X.2018.01.031 |
[9] | 刘冬,秦瑞,陈曦,李庆. 3D车载环视全景生成方法 Generation of Three-dimention Vehicle Panorama 计算机科学, 2017, 44(4): 302-305. https://doi.org/10.11896/j.issn.1002-137X.2017.04.061 |
[10] | 杜琳琳,朱振峰,段红帅,赵耀. LSPSA:基于局部结构保持的共享子空间分析 Local Structure Preserved Shared-subspace Analysis 计算机科学, 2014, 41(10): 67-71. https://doi.org/10.11896/j.issn.1002-137X.2014.10.015 |
[11] | 尹维冲,路通. 基于多分类器融合的多视角目标检测算法 Novel Framework for Multi-view Object Detection through Combining Multiple Classifiers 计算机科学, 2013, 40(7): 266-269. |
[12] | 李闪闪,曹存根. 事件前提和后果常识知识分析方法研究 Commonsense Knowledge Analysis Approach Based on Event Preconditions and Effects 计算机科学, 2013, 40(4): 185-192. |
[13] | 柴艳妹,韩文英,刘灿涛,李海峰. 融合理论在步态识别中的应用研究 Study on Application of Fusion Theory in Gait Recognition 计算机科学, 2012, 39(12): 272-277. |
[14] | 陈嘉佳 毛新军. 多主体系统开发环境的结构化评价框架SEF及评价结果 计算机科学, 2005, 32(5): 200-205. |
|