关键词: 语言模型, 学科知识, 选择题评估

Abstract: Subject knowledge evaluation methods for pre-trained language models based on Multiple Choice Questions(MCQ) offer rapid,quantitative evaluation of model knowledge.However,their reliability is compromised by irrelevant factors such as option order and length,raising robustness concerns.To address this challenge,an analytical framework for evaluating subject knowledge of pre-trained language models using MCQ is proposed.This framework formalizes MCQ evaluation into two mo-dules:prompting and parsing,systematically investigating the impact of various MCQ evaluation methods on evaluation outcomes.The robustness of different prompting and parsing techniques is analyzed through experiments on Chinese and English subject knowledge evaluation datasets.Based on these findings,a rewriting-enhanced parsing method is introduced that employs pre-trained language models to rewrite model responses,effectively overcoming the limitations of traditional rule-based parsing when handling non-standard replies.By integrating rewriting and rule-based parsing,this approach enhances both answer extraction accuracy and evaluation process robustness,offering a novel and effective strategy for language model evaluation.

Key words: Language model,Subject knowledge,Multiple choice evaluation