计算机科学 ›› 2026, Vol. 53 ›› Issue (10): 345-355.doi: 10.11896/jsjkx.250900141
潘天乐, 薛雯耀, 任庆玮, 王轶辰
PAN Tianle, XUE Wenyao, REN Qingwei, WANG Yichen
摘要: 如果航空、航天等领域的安全关键软件质量得不到保障,会造成极大的危害性事故。软件测试,是保障软件质量的主要方式。单元测试作为软件测试的基础环节,是保证安全关键软件质量的必要步骤。单元测试的核心环节是测试用例设计,传统单元测试用例设计方法有人工编写和自动生成两类。大语言模型(LLM)等人工智能技术的发展,为单元测试用例设计提供了全新的智能生成思路。对于单元测试用例设计这类复杂任务,通用大语言模型通常不具备较高的适配度,使用数据集训练大语言模型是一种能有效改善模型任务表现且被广泛使用的方式。目前,面向模型训练的单元测试数据集大多使用静态分析等传统方式构建,存在映射关系有误、缺乏统一的定义与评估标准以及全面性和逻辑性不足的问题。针对上述问题,提出了结构化的单元测试数据集的全面定义和基于LLM的单元测试数据集的构建方法UTDCM-LLM。方法包括基于人工的单元测试数据点的构建、基于思维链的大语言模型提示词设计、基于LLM的单元测试数据集智能批量生成和基于静态检查的数据集评估四大环节。此外,还设计了基于数据集的模型微调实验。实验成功验证了单元测试数据集在模型训练上的有效性和UTDCM-LLM方法的可行性。
中图分类号:
| [1] LEVESON N G.Engineering a safer world:Systems thinkingapplied to safety[M].Cambridge:MIT Press,2016. [2] ZHANG X H,HE Y Q.Overview of software testing methods[J].Science & Technology Vision,2012(4):35-37. [3] MOHD-SHAFIE M L,NASIR WAN KADIR W M,KHATIB-SYARBINIM,et al.An EFSM-Based Test Data Generation Approach in Model-Based Testing[J].Computers,Materials & Continua,2022,71(3):4337-4354. [4] SHANJ H,JIANG Y,SUN P.Research Progress in SoftwareTesting[J].Acta Scientiarum Naturalium Universitatis Peki-nensis,2005(1):134-145. [5] BIRD D L,MUNOZC U.Automatic generation of random self-checking test cases[J].IBM systems journal,1983,22(3):229-245. [6] CHEN T Y,LEUNG H,MAK I K.Adaptive random testing[C]//Annual Asian Computing Science Conference.Berlin:Springer,2004:320-329. [7] SAHOO R R,RAY M.PSO based test case generation for criti-cal path using improved combined fitness function[J].Journal of King Saud University-Computer and Information Sciences,2020,32(4):479-490. [8] BIDGOLI A M,HAGHIGHI H.Augmenting ant colony optimi-zation with adaptive random testing to cover prime paths[J].Journal of Systems and Software,2020,161:110495. [9] YUAN Z,LOU Y,LIU M,et al.No more manual tests? evalua-ting and improving chatgpt for unit test generation[J].arXiv:2305.04207,2023. [10] LI T O,ZONG W,WANG Y,et al.Nuances are the key:Unloc-king chatgpt to find failure-inducing tests with differential prompting[C]//2023 38th IEEE/ACM International Confe-rence on Automated Software Engineering(ASE).IEEE,2023:14-26. [11] KARANJAI R,HUSSAIN A,RABIN M R I,et al.Harnessing the power of llms:Automating unit test generation for high-performance computing[J].arXiv:2407.05202,2024. [12] UDUM U,KUMAR R,VASUMATHI D.Software Defect Prediction Using Temporal Transformer Graph Network with Newton-Raphson Optimization[J].Journal of Artificial Intelligence and Technology,2025(5):441-452. [13] HE Y,HUANG J,RONG Y,et al.Unitsyn:A large-scale dataset capable of enhancing the prowess of large language models for program testing[C]//Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis.2024:1061-1072. [14] JAIN K,SYNNAEVE G,ROZIÈRE B.Testgeneval:A realworld unit test generation and test completion benchmark[J].arXiv:2410.00752,2024. [15] TUFANO M,DENG S K,SUNDARESAN N,et al.MethodsTest:A dataset of focal methods mapped to test cases[C]//Proceedings of the 19th International Conference on Mining Software Repositories.2022:299-303. [16] LI B N,WANG B,CHEN M D,et al.Unit Test Generation for CPrograms via Large Language Models[J].Aerospace Contrd and Application,2025,51(1):115-122. [17] JOHNSON L A.DO-178B:Software considerations in airbornesystems and equipment certification[J].Crosstalk,October,1998,199:11-20. [18] SIAMI NAMIN A,ANDREWS J H,MURDOCHD J.Sufficient mutation operators for measuring test effectiveness[C]//Proceedings of the 30th International Conference on Software Engineering.2008:351-360. [19] National Standardization Administration.Systems and software engineering—Software testing-Part 2:Test Processes:GB/T 38634.2-2020 [S].Beijing:Standards Press of China,2020. [20] WEI J,WANG X,SCHUURMANS D,et al.Chain-of-thoughtprompting elicits reasoning in large language models[J].Advances in Neural Information Processing Systems,2022,35:24824-24837. [21] WHITE J,FU Q,HAYS S,et al.A prompt pattern catalog toenhance prompt engineering with chatgpt[J].arXiv:2302.11382,2023. [22] LIU A,FENG B,XUE B,et al.Deepseek-v3 technical report[J].arXiv:2412.19437,2024. [23] WANG B.Research on Evaluation Index System and Evaluation Method of Software Test Cases [J].Ship Electronic Enginee-ring,2020,40(11):112-115. [24] LIZ Z,LI H J.Research on dataset quality assessment model and method for large model training[J].Digital Economy,2024(12):26-30. [25] YANG A,LI A,YANG B,et al.Qwen3 technical report[J].arXiv:2505.09388,2025. [26] HOU X,ZHAO Y,WANG S,et al.Model context protocol(mcp):Landscape,security threats,and future research directions[J].arXiv:2503.23278,2025. |
|
||