Computer Science ›› 2026, Vol. 53 ›› Issue (10): 396-412.doi: 10.11896/jsjkx.251200012

• Computer Software • Previous Articles     Next Articles

Automatic Code Refactoring:LLM-based Prompt Evolution Approach

ZHANG Yang, FAN Zishuo, WU Shaoguang   

  1. School of Information Science and Engineering,Hebei University of Science and Technology,Shijiazhuang 050018, China
  • Received:2025-12-01 Revised:2026-03-23 Published:2026-10-13
  • About author:ZHANG Yang,born in 1980,Ph.D,professor,is a distinguished member of CCF(No.41854D).His main research interests include software refactoring and intelligent software.
    WU Shaoguang,born in 1988,Ph.D,lecturer.His main research interest is intelligent software.
  • Supported by:
    Hebei Natural Science Foundation(F2023208001) and Hebei Provincial Overseas High-level Talent Foundation(C20230358).

Abstract: Aiming at the poor code-refactoring performance of existing LLM-driven methods caused by rigid prompts and insufficient adaptability to intricate code patterns,this study proposes a self-feedback code refactoring method based on LLM,named RefPro;this method constructs a self-feedback prompt and code optimization method based on the idea of evolutionary algorithms.Specifically,RefPro treats prompt templates as evolvable individuals;in each iteration,the LLM generates optimized prompts and refactored code according to the prompts in the current population,and the code is evaluated to obtain a fitness score,which is used as feedback to drive the optimization and mutation of prompts(meta-mutation),thereby updating the population.Finally,highquality prompts are generated through multiround iteration to obtain the refactored code.To verify the effectiveness of the RefPro method,multiple open-source datasets such as HumanEval,CoderEval,and LC-325 are selected to evaluate RefPro.Experimental results show that RefPro can reduce the cyclomatic complexity of code by an average of 24.85% and increase the PyLint score by an average of 4.31,which indicates that RefPro can effectively optimize the structural complexity and quality of code.

Key words: Large language models, Code refactoring, Prompt evolution, Automatic

CLC Number: 

  • TP311
[1] FOWLER M.Refactoring:improving the design of existing code[M].Boston:Addison-Wesley,1999:16-50.
[2] DING Y,MIN M J,KAISERG,et al.Cycle:Learning to self-refine the code generation[C]//Proceedings of the ACM on Programming Languages.2024:392-418.
[3] MADAAN A,TANDON N,GUPTA P,et al.Self-refine:Iterative refinement with self-feedback[J].Advances in Neural Information Processing Systems,2023,36:46534-46594.
[4] FUJII K,NAKAMURA T,LOEMM,et al.Continual pre-training for cross-lingual llm adaptation:Enhancing japanese language capabilities[J].arXiv:2404.17790,2024.
[5] BERABI B,GRONSKIY A,RAYCHEV V,et al.Deepcode AIfix:Fixing security vulnerabilities with large language models[J].arXiv:2402.13291,2024.
[6] KHATTAB O,SINGHVI A,MAHESHWARI P,et al.Dspy:Compiling declarative language model calls into self-improving pipelines[J].arXiv:2310.03714,2023.
[7] WHITE J,FU Q,HAYS S,et al.A prompt pattern catalog to enhance prompt engineering with chatGPT[J].arXiv:2302.11382,2023.
[8] BROWN T,MANN B,RYDER N,et al.Language models are few-shot learners[J].Advances in Neural Information Proces-sing Systems,2020,33:1877-1901.
[9] GUO Q Y,WANG R,GUO J L,et al.Connecting large language models with evolutionary algorithms yields powerful prompt optimizers[J].arXiv:2309.08532,2023.
[10] ACHIAM J,ADLER S,AGARWAL S,et al.GPT-4 technicalreport[J].arXiv:2303.08774,2023.
[11] SCHULMAN J,ZOPH B,KIM C,et al.ChatGPT:Optimizing language models for dialogue[J].OpenAI blog,2022,2(4).
[12] CHEN M,TWOREK J,JUN H,et al.Evaluating large language models trained on code[J].arXiv:2107.03374,2021.
[13] LI R,ALLAL L B,ZI Y,et al.Starcoder:may the source be with you![J].arXiv:2305.06161,2023.
[14] LOZHKOV A,LI R,ALLAL L B,et al.Starcoder 2 and the stack v2:The next generation[J].arXiv:2402.19173,2024.
[15] SHIRAFUJI A,ODA Y,SUZUKIJ,et al.Refactoring programs using large language models with few-shot examples[C]//2023 30th Asia-Pacific Software Engineering Conference(APSEC).IEEE,2023:151-160.
[16] JIN M,SHAHRIAR S,TUFANOM,et al.Inferfix:End-to-end program repair with LLM[C]//Proceedings of the 31st ACM Joint European Software Engineering Conference and Sympo-sium on the Foundations of Software Engineering.2023:1646-1656.
[17] CORDEIRO J,NOEI S,ZOU Y.An empirical study on the code refactoring capability of large language models[J].arXiv:2411.02320,2024.
[18] FAN R X,SUN B C,GONGL N,et al.Test Case Generation and Ranking for Program Fault Diagnosis Based on Large Language Models[J].Computer Science,2026,53(8):365-374.
[19] TORNHILL A,BORG M,HAGATULAH N,et al.ACE:automated technical debt remediation with validated large language model refactorings[C]//Proceedings of the 33rd ACM International Conference on the Foundations of Software Engineering.2025:1318-1324.
[20] ZHANG Y,LI Y L,MEREDITH G,et al.Move method refactoring recommendation based on deep learning and LLM-generated information[J].Information Sciences,2025,697:121753.
[21] XU Z M,JIANG Y J,ZHANG Y X,et al.Long Method Decomposition Based on Large Language Model [J].Journal of Software,2025,36(6):2501-2514.
[22] XIA P,ZHANG Y J,QI J.Multi agent collaborative code gene-ration technology driven by big language models [J].Computer Science,2025,52(S2):771-779.
[23] WHITE J,HAYS S,FU Q,et al.ChatGPT prompt patterns for improving code quality,refactoring,requirements elicitation,and software design[M]//Generative AI for Effective Software Development.Cham:Springer,2024:71-108.
[24] SIDDEEQ S,WASEEM M,RASHEED Z,et al.LLM-basedMulti-Agent System for Intelligent Refactoring of Haskell Code[J].arXiv:2506.19481,2025.
[25] CORDEIRO J,NOEI S,ZOU Y.LLM-Driven Code Refactoring:Opportunities and Limitations[C]//2025 IEEE/ACM Second IDE Workshop(IDE).IEEE,2025:32-36.
[26] WEI J,WANG X,SCHUURMANS D,et al.Chain-of-thoughtprompting elicits reasoning in large language models[J].Advances in Neural Information Processing Systems,2022,35:24824-24837.
[27] ZHOU Y C,MURESANU A I,HAN Z,et al.Large language models are human-level prompt engineers[C]//The Eleventh International Conference on Learning Representations.2022.
[28] WANG X,WEI J,SCHUURMANSD,et al.Self-consistency improves chain of thought reasoning in language models[J].arXiv:2203.11171,2022.
[29] ZHOU D,SCHÄRLI N,HOU L,et al.Least-to-most prompting enables complex reasoning in large language models[J].arXiv:2205.10625,2022.
[30] KOJIMA T,GU S S,REID M,et al.Large language models are zero-shot reasoners[J].Advances in Neural Information Processing Systems,2022,35:22199-22213.
[31] YAO S Y,YU D,ZHAO J,et al.Tree of thoughts:Deliberateproblem solving with large language models[J].Advances in Neural Information Processing Systems,2023,36:11809-11822.
[32] BESTA M,BLACH N,KUBICEK A,et al.Graph of thoughts:Solving elaborate problems with large language models[C]//Proceedings of the AAAI Conference on Artificial Intelligence.2024:17682-17690.
[33] WANG L,XU W,LAN Y,et al.Plan-and-solve prompting:Improving zero-shot chain-of-thought reasoning by large language models[J].arXiv:2305.04091,2023.
[34] ZHANG N Y,LI L,CHEN X,et al.Differentiable promptmakes pre-trained language models better few-shot learners[J].arXiv:2108.13161,2021.
[35] MEYERSON E,NELSON M J,BRADLEY H,et al.Language model crossover:Variation through few-shot prompting[J].ACM Transactions on Evolutionary Learning,2024,4(4):1-40.
[36] CHEN A,DOHAN D,SO D.Evoprompting:Language modelsfor code-level neural architecture search[J].Advances in Neural Information Processing Systems,2023,36:7787-7817.
[37] YANG C R,WANG X Z,LU Y F,et al.Large Language Models as Optimizers[C]//International Conference on Learning Representations 2024(ICLR 2024).2024.
[38] LUKASCZYK S,FRASER G.Pynguin:Automated unit testgeneration for python[C]//Proceedings of the ACM/IEEE 44th International Conference on Software Engineering:Companion Proceedings.2022:168-172.
[39] FRASER G,ARCURI A.Evosuite:automatic test suite genera-tion for object-oriented software[C]//Proceedings of the 19th ACM SIGSOFT Symposium and the 13th European Conference on Foundations of Software Engineering.2011:416-419.
[40] ILL G K,KEMERER C F.Cyclomatic complexity density and software maintenance productivity[J].IEEE Transactions on Software Engineering,1991,17(12):1284-1288.
[41] LOPES M,HORA A.How and why we end up with complex methods:a multi-language study[J].Empirical Software Engineering,2022,27(5):115.
[42] SHI C F,YANG H,CAI D,et al.A thorough examination of decoding methods in the era of LLM[J].arXiv:2402.06925,2024.
[43] LIPOWSKI A,LIPOWSKA D.Roulette-wheel selection via stochastic acceptance[J].Physica A:Statistical Mechanics and Its Applications,2012,391(6):2193-2196.
[44] FERNANDO C,BANARSE D,MICHALEWSKI H,et al.Promptbreeder:Self-referential self-improvement via prompt evolution[J].arXiv:2309.16797,2023.
[45] DAWKINS R.The evolution of evolvability[M]//ArtificialLife.Routledge,2019:201-220.
[46] PAYNE J L,WAGNER A.The causes of evolvability and their evolution[J].Nature Reviews Genetics,2019,20(1):24-38.
[47] GAJEWSKI A,CLUNE J,STANLEY K O,et al.Evolvability ES:scalable and direct optimization of evolvability[C]//Proceedings of the Genetic and Evolutionary Computation Confe-rence.2019:107-115.
[48] ZHENG Q,XIA X,ZOU X,et al.Codegeex:A pre-trained model for code generation with multil-ingual benchmarking on humaneval-x[C]//Proceedi-ngs of the 29th ACM SIGKDD Confe-rence on Knowledge Discovery and Data Mining.2023:5673-5684.
[49] CHOI J,AN G,YOO S.Iterative refactoring of real-world open-source programs with large language models[C]//International Symposium on Search Based Software Engineering.Cham:Springer,2024:49-55.
[1] HE Jiaojun, LI Xin. Review of Graph Learning Based on Large Language Models:Methods,Benchmarks and Advances [J]. Computer Science, 2026, 53(9): 1-15.
[2] LI Luozheng, LI Lingbo, YUAN Quan. Survey of Chinese Datasets for LLM Safety Alignment:Landscape and Prospects [J]. Computer Science, 2026, 53(9): 16-23.
[3] ZHANG Rongjie, PANG Xiongwen, WANG Fengling. Review of Large Language Model-based Time Series Modeling via Fine-tuning and AgentArchitecture [J]. Computer Science, 2026, 53(9): 55-70.
[4] GUO Yuyang, SHI Lei, LIU Huan, DONG Yixiang, LI Rui. Domain-adapted and Dynamically Retrieval-augmented Approach for Large-scale History Discipline Model Construction [J]. Computer Science, 2026, 53(9): 92-100.
[5] LI Zhennan, QIAN Jiayan, WANG Xinzhi, ZHANG Hui. Technology Risk Structure Recognition Based on Multi-granularity Semantic Dual Reflection [J]. Computer Science, 2026, 53(9): 395-404.
[6] LIU Jing. Review of Music Artificial Intelligence Driven by Large Language Models [J]. Computer Science, 2026, 53(8): 229-244.
[7] ZHANG Haoran, HAO Wenning, JIN Dawei, CHENG Kai, LIU Junyang. Agentic Retrieval Augmented Generation Framework Based on Retrieval Task Planning and Reflection Mechanism [J]. Computer Science, 2026, 53(8): 285-297.
[8] CHEN Zhixiang, XIE Zhipeng. Event Causal Data Augmentation Method Based on Large Language Model [J]. Computer Science, 2026, 53(7): 125-131.
[9] XU Rui, LIU Jin, LIU Xudong, GUAN Jian, DONG Wei. Exploring the Generalization Ability of Prompt-based Large Language Models for TextClassification [J]. Computer Science, 2026, 53(6A): 250400092-7.
[10] WEI Qing, ZHANG Yupeng, LIU Shaoxun, ZHANG Jinfeng, ZHANG Yuezhong, CHEN Haoyang. Fuzzing Driver Generation Based on Large Language Models [J]. Computer Science, 2026, 53(6A): 250400113-8.
[11] ZHANG Yongyu, GUO Chenjuan, FEI Xueqin, LI Feng. Study on Financial Text Sentiment Analysis Method Based on Large Language Models with Market Feedback Supervision [J]. Computer Science, 2026, 53(6A): 250500073-14.
[12] SHI Hongxu, LIU Yi, LIU Kun. Survey of Recommendation Systems Based on Large Language Models [J]. Computer Science, 2026, 53(6): 281-303.
[13] WANG Shenghui, LI Teng. Innovative Automated Scoring Based on Large Language Models [J]. Computer Science, 2026, 53(5): 90-98.
[14] HU Junjie, CHEN Yujie, HU Yikun, WEN Cheng, CAO Jialun, MA Zhi, SU Jie, SUN Weidi, TIAN Cong, QIN Shengchao. Formal Theorem Proving Empowered by Large Language Model:Survey and Perspectives [J]. Computer Science, 2026, 53(4): 1-23.
[15] LIU Suyi, LIU Qi, GAO Weibo. Agent4Stu:Efficient LLM-based Student Answer Behavior Simulation Agent [J]. Computer Science, 2026, 53(4): 347-355.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!