Computer Science ›› 2026, Vol. 53 ›› Issue (6): 59-68.doi: 10.11896/jsjkx.250600150

• Intelligent Education Technology • Previous Articles     Next Articles

Automatic Knowledge Point Annotation for Student Code Based on Multi-agent Collaboration:A Case Study of C Language

LIU Jiaqi, GAO Zhizezhang, MENG Xianjia, SUN Xia, FENG Jun   

  1. College of Computer Science,Northwest University,Xi'an 710127,China
  • Received:2025-06-24 Revised:2025-09-04 Online:2026-06-15 Published:2026-06-09
  • About author:LIU Jiaqi,born in 2000,postgraduate.His main research interest is intelligent education.
    FENG Jun,born in 1972,Ph.D,professor,Ph.D supervisor,is an advanced member of CCF(No.10834S).Her main research interests include intelligent information processing and so on.
  • Supported by:
    Research Program on Teacher Education Reform and Teacher Development of Shaanxi Province(SJS2023ZD030) and Talent Fostering Program of Northwest University(JX2024068).

Abstract: In intelligent education systems,knowledge point annotation is a key module for organizing teaching resources,enabling personalized recommendations,and modeling students' cognitive states.However,traditional exercise-oriented approaches to knowledge point annotation have limitations in reflecting the individual differences exhibited by students in programming learning processes.To address this issue,this paper proposes an automatic student code knowledge point annotation method based on multi-agent collaboration.This method shifts the focus from exercise-driven to code-driven annotation,constructing a three-level knowledge framework encompassing statement layer,code-block layer,and function layer.It also introduces a collaborative system composed of three agents-knowledge annotation,task analysis,and integrated feedback-with internal self-inspection and iterative optimization capabilities.Experimental evaluation is conducted on 363 student code submissions from an introductory programming course.The system demonstrates strong interpretability and group analysis capability in real-world educational scena-rios,effectively revealing students' knowledge mastery status and common cognitive deficiencies.Moreover,the study employs a LLM-based peer-review mechanism for performance assessment.Results indicate that the multi-agent collaborative approach outperforms methods based directly on LLM across five evaluation dimensions(completeness,accuracy,reasonableness,error identification ability,and educational guidance),with significantly more selections as the preferred solution.This research achieves automated and interpretable knowledge point annotation for student code,providing technical support and practical foundations for fine-grained student modeling,personalized assessment,and other downstream tasks.

Key words: Programming education, Knowledge point annotation, Student code, Large language model, Multi-agent

CLC Number: 

  • TP399
[1]CHEN J M,CHEN M C,SUN Y S.A tag based learning approach to knowledge acquisition for constructing prior know-ledge and enhancing student reading comprehension[J].Compu-ters & Education,2014,70:256-268.
[2]ZHONG M,DING R.Design of a personalized recommendation system for learning resources based on collaborative filtering[J].International Journal of Circuits,Systems and Signal Processing,2022,16(1):122-31.
[3]LIU Z,GUO T,LIANG Q,et al.Deep Learning Based Know-ledge Tracing:A Review,A Tool and Empirical Studies[J].IEEE Transactions on Knowledge and Data Engineering,2025,37(8):4512-4536.
[4]CHAO P Y.Exploring students' computational practice,design and performance of problem-solving through a visual programming environment[J].Computers & Education,2016,95:202-215.
[5]POULSEN S,KULKARNI S,HERMAN G,et al.EfficientFeedback and Partial Credit Grading for Proof Blocks Problems[C]//International Conference on Artificial Intelligence in Education.Cham:Springer Nature Switzerland,2023:502-514.
[6]ALON U,ZILBERSTEIN M,LEVY O,et al.code2vec:Lear-ning distributed representations of code[J].Proceedings of the ACM on Programming Languages,2019,3(POPL):1-29.
[7]GAO Z,YAN H,LIU J,et al.Tracing distinct learning trajectories in introductory programming course:a sequence analysis of score,engagement,and code metrics for novice computer science vs.math cohorts[J].International Journal of STEM Education,2025,12(1):1-26.
[8]SHI Y,CHI M,BARNES T,et al.Evaluating Multi-Knowledge Component Interpretability of Deep Knowledge Tracing Models in Programming[C]//Proceedings of the 17th International Conference on Educational Data Mining.2024:288-295.
[9]WU Y,ZHU H,WANG C,et al.Programming knowledge tracing based on heterogeneous graph representation[J].Know-ledge-Based Systems,2024,300:112161.
[10]HAN S,WANG Y,LU X.Errorclr:semantic error classifica-tion,localization and repair for introductory programming assignments[C]//Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval.2023:1345-1354.
[11]NGUYEN H,ALLAN V.Using GPT-4 to provide tiered,formative code feedback[C]//Proceedings of the 55th ACM Technical Symposium on Computer Science Education V.1.2024:958-964.
[12]RATNER A,BACH S H,EHRENBERG H,et al.Snorkel:Ra-pid training data creation with weak supervision[C]//Procee-dings of the VLDB Endowment.International Conference on VeryLarge Data Bases.2017:269.
[13]GUO S,LATIF E,ZHOU Y,et al.Using Generative AI and Multi-Agents to Provide Automatic Feedback[J].arXiv:2411.07407,2024.
[14]LI X,WANG S,ZENG S,et al.A survey on LLM-based multi-agent systems:workflow,infrastructure,and challenges[J].Vi-cinagearth,2024,1(1):9.
[15]GUO T,CHEN X,WANG Y,et al.Large language model based multi-agents:A survey of progress and challenges[J].arXiv:2402.01680,2024.
[16]ZHANG Z,ZHANG L D,YU J,et al.Simulating classroomeducation with llm-empowered agents[J].arXiv:2406.19226,2024.
[17]SONG T,ZHANG H,XIAO Y.A High-Quality Generation Approach for Educational Programming Projects Using LLM[J].IEEE Transactions on Learning Technologies,2024(17):2242-2255.
[18]LI H,XU T,CHANG E,et al.Knowledge tagging with large language model based multi-agent system[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2025:28775-28782.
[19]LI H,XU T,TANG J,et al.Automate knowledge concept tagging on math questions with LLMs[J].arXiv:2403.17281,2024.
[20]SUN B,ZHU Y,XIAO Y,et al.Automatic question taggingwith deep neural networks[J].IEEE Transactions on Learning Technologies,2018,12(1):29-43.
[21]LIU Q,HUANG Z,YIN Y,et al.Ekt:Exercise-aware know-ledge tracing for student performance prediction[J].IEEE Transactions on Knowledge and Data Engineering,2019,33(1):100-115.
[22]HUANG Z,LIU Q,GAO W,et al.Neural mathematical solver with enhanced formula structure[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval.2020:1729-1732.
[23]WANG J,LI H,DU X,et al.S-kmn:Integrating semantic features learning and knowledge mapping network for automatic quiz question annotation[J].Journal of King Saud University-Computer and Information Sciences,2023,35(7):101594.
[24]DING Z,WANG X,WU Y,et al.Tagging knowledge concepts for math problems based on multi-label text classification[J].Expert Systems with Applications,2025,267:126232.
[25]HUANG T,HU S,YANG H,et al.PQSCT:Pseudo-siameseBERT for concept tagging with both questions and solutions[J].IEEE Transactions on Learning Technologies,2023,16(5):831-846.
[26]MOORE S,SCHMUCKER R,MITCHELL T,et al.Automated generation and tagging of knowledge components from multiple-choice questions[C]//Proceedings of the Eleventh ACM Confe-rence on Learning@ scale.2024:122-133.
[27]LI H,XU T,TANG J,et al.Knowledge tagging system on math questions via llms with flexible demonstration retriever[J].ar-Xiv:2406.13885,2024.
[28]MEHTA N,TERUEL M,DENG X,et al.Improving Grounded Language Understanding in a Collaborative Environment by Interacting with Agents Through Help Feedback[C]//Findings of the Association for Computational Linguistics:EACL 2024.2024:1306-1321.
[29]LAGAKIS P,DEMETRIADIS S.EvaAI:a multi-agent frame-work leveraging large language models for enhanced automated grading[C]//International Conference on Intelligent Tutoring Systems.Cham:Springer Nature Switzerland,2024:378-385.
[30]YANG K,CHU Y,DARWIN T,et al.Content knowledge identification with multi-agent large language models(LLMs)[C]//International Conference on Artificial Intelligence in Education.Cham:Springer Nature Switzerland,2024:284-292.
[31]FENG J,WU T,LU X,et al.Exploration and Practice of Agile Teaching Methods in the Basic Programming Course [J].Journal of Computer Education,2019(8):81-87.
[32]LI R,KANG J,LIU Q,et al.MGS3:A Multi-Granularity Self-Supervised Code Search Framework[C]//Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.1.2025:695-706.
[33]MADAAN A,TANDON N,GUPTA P,et al.Self-refine:Iterative refinement with self-feedback[J].Advances in Neural Information Processing Systems,2023,36:46534-46594.
[34]ZHENG L,CHIANG W L,SHENG Y,et al.Judging llm-as-a-judge with mt-bench and chatbot arena[J].Advances in Neural Information Processing Systems,2023,36:46595-46623.
[1] SHI Hongxu, LIU Yi, LIU Kun. Survey of Recommendation Systems Based on Large Language Models [J]. Computer Science, 2026, 53(6): 281-303.
[2] JI Wendi, WANG Yongquan, SHEN Yicheng. Boosting Generative Rule Extraction via Negative-aware Approach [J]. Computer Science, 2026, 53(5): 276-285.
[3] HAN Linrui, ZHENG Ri, CONG Yingnan. Explainable Sentencing Prediction Method Driven by Sentencing Rule Knowledge Graph [J]. Computer Science, 2026, 53(5): 286-298.
[4] LIU Xukai, LIU Yang, HUANG Haozhen. EC-MIIP:Efficient Fine-tuning Small-parameter Large Language Model for Intellectual Property [J]. Computer Science, 2026, 53(5): 299-308.
[5] WANG Shenghui, LI Teng. Innovative Automated Scoring Based on Large Language Models [J]. Computer Science, 2026, 53(5): 90-98.
[6] HU Junjie, CHEN Yujie, HU Yikun, WEN Cheng, CAO Jialun, MA Zhi, SU Jie, SUN Weidi, TIAN Cong, QIN Shengchao. Formal Theorem Proving Empowered by Large Language Model:Survey and Perspectives [J]. Computer Science, 2026, 53(4): 1-23.
[7] LIU Suyi, LIU Qi, GAO Weibo. Agent4Stu:Efficient LLM-based Student Answer Behavior Simulation Agent [J]. Computer Science, 2026, 53(4): 347-355.
[8] WU Qiaorui, LUO Li, ZHAO Cairong. LLM-augmented Training Framework with Cycle-Consistency Constraints [J]. Computer Science, 2026, 53(4): 377-383.
[9] WU Xianjie, LI Tongliang, LI Zhoujun. Survey of Table Question Answering Research [J]. Computer Science, 2026, 53(3): 295-306.
[10] XU Cheng, LIU Yuxuan, WANG Xin, ZHANG Cheng, YAO Dengfeng, YUAN Jiazheng. Review of Speech Disorder Assessment Methods Driven by Large Language Models [J]. Computer Science, 2026, 53(3): 307-320.
[11] LI Wenli, FENG Xiaonian, QIAN Tieyun. Few-shot Continuous Toxicity Detection Based on Large Language Model Augmentation [J]. Computer Science, 2026, 53(3): 321-330.
[12] WANG Zhibin, LI Shipeng, ZHOU Yuhang, LI Xue, ZHANG Zhonghui, JIANG Zhiwei, GU Rong, TIAN Chen, CHEN Guihai, ZHONG Sheng. Optimization of Service Level Objectives and System Level Metrics in Large Language ModelServing System [J]. Computer Science, 2026, 53(3): 23-32.
[13] ZHOU Yueyuan, LU Guanze, XIANG Jiawei, ZHANG Jiawei, SHAO En, HE Xin. Training System for Large Language Models Based on Adaptive Transpose on Hygon DCU [J]. Computer Science, 2026, 53(3): 33-40.
[14] CHEN Han, XU Zefeng, JIANG Jiu, FAN Fan, ZHANG Junjian, HE Chu, WANG Wenwei. Large Language Model and Deep Network Based Cognitive Assessment Automatic Diagnosis [J]. Computer Science, 2026, 53(3): 41-51.
[15] XU Jiawen, ZHENG Yungui, ZHOU Wei, XU Yaoqiang, HU Huiqi, ZHOU Xuan. SQL-MARS:Text-to-SQL Structured Data Recommendation System for Ambiguous UserRequirements [J]. Computer Science, 2026, 53(3): 52-63.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!