Abstract: Code quality evaluation is the process of autonomously measuring and scoring code quality for specific tasks.Existing evaluation techniques can be broadly divided into match-based and execution-based approaches.The former emphasizes static textual comparison with reference code,making it difficult to assess semantic differences in program behavior comprehensively.The latter depends on large numbers of manually or semi-automatically constructed test cases,which are costly to build and have limi-ted coverage.With the rapid development of large language models(LLMs),an increasing number of studies are using LLMs as automatic evaluators.However,in realistic scenarios where complete test suites and high-quality reference code are often unavailable,how to mitigate hallucinations in LLM reasoning and obtain automated evaluation results that are highly consistent with expert judgments and actual execution behavior remains a key open problem.To this end,this paper proposes SDEval,a semantic-driven execution framework for code quality evaluation,together with its fine-grained scoring variant SDEval-FG.Firstly,the framework performs recursive semantic understanding,decomposing complex programs into fine-grained semantic units and conducting layered reasoning to strengthen comprehension of control-flow and data-flow relationships,capture logical dependencies among code fragments,and suppress hallucinations.It then constructs a dynamic,semantic-driven execution perspective through automatic test-case generation and completion,enabling the LLM to consider both semantic correctness and functional completeness,thereby bridging the gap between code text and actual execution behavior.Compared with existing methods that mainly aim to improve test-case generation and coverage,SDEval shifts the evaluation focus from generating high-quality test cases to interpretable,defect-category-oriented scoring,thus enhancing its correlation with human expert judgments.Experimental results show that on the HumanEval dataset,SDEval improves the Pearson correlation coefficient by 11.86% over the execution-based ChatTester,and on the CodePair dataset,the SDEval-FG variant achieves a 31.4% improvement in Pearson correlation over the CoT-prompting baseline.

Key words: Code quality evaluation, Code semantic representation, Test case generation, Defect identification, Large language models