Computer Science ›› 2026, Vol. 53 ›› Issue (10): 345-355.doi: 10.11896/jsjkx.250900141

• Computer Software • Previous Articles     Next Articles

Intelligent Dataset Generation Method for Safety-critical Software Unit Testing

PAN Tianle, XUE Wenyao, REN Qingwei, WANG Yichen   

  1. School of Reliability and Systems Engineering,Beihang University,Beijing 100191, China
  • Received:2025-09-24 Revised:2025-12-19 Published:2026-10-13
  • About author:PAN Tianle,born in 2003,postgra-duate.His main research interests include software intelligent testing technology,and so on.
    WANG Yichen,born in 1977,associate professor.His main research interests include model-based software testing,software quality evaluation,and so on.

Abstract: For safety-critical software in domains such as aviation and aerospace,inadequate software quality can lead to highly hazardous accidents.Software testing is the primary means of ensuring software quality,and unit testing-being the foundational phase of software testing-is a necessary step for guaranteeing the quality of safety-critical software.The core component of unit testing is test case design.Traditional approaches to unit test case design fall into two categories:manual writing and automatic generation.Recent advances in artificial intelligence,particularly large language model(LLM),have introduced new intelligent strategies for generating unit test cases.However,general-purpose LLMs typically lack sufficient adaptability for complex tasks like unit test case design.A widely adopted and effective approach to improving model performance is training LLMs on specia-lized datasets.Currently,most unit test datasets used for model training are constructed using traditional techniques such as static analysis.These datasets often suffer from incorrect code-test mappings,lack standardized definitions and evaluation criteria,and are insufficient in both comprehensiveness and logical coherence.To address these issues,a structured,comprehensive definition for unit test datasets and a novel unit test dataset construction method by LLM named UTDCM-LLM are proposed.This method consists of four main components:manual construction of unit test data points,design of chain-of-thought prompts for LLMs,intelligent batch generation of unit test dataset using LLMs,and dataset evaluation based on static analysis.In addition,a model fine-tuning experiment based on the constructed dataset are conducted.The experiment successfully validates both the effectiveness of the dataset in model training and the feasibility of the UTDCM-LLM approach.

Key words: Safety-critical software, Unit testing, Large language model, Dataset, Model training

CLC Number: 

  • TP391
[1] LEVESON N G.Engineering a safer world:Systems thinkingapplied to safety[M].Cambridge:MIT Press,2016.
[2] ZHANG X H,HE Y Q.Overview of software testing methods[J].Science & Technology Vision,2012(4):35-37.
[3] MOHD-SHAFIE M L,NASIR WAN KADIR W M,KHATIB-SYARBINIM,et al.An EFSM-Based Test Data Generation Approach in Model-Based Testing[J].Computers,Materials & Continua,2022,71(3):4337-4354.
[4] SHANJ H,JIANG Y,SUN P.Research Progress in SoftwareTesting[J].Acta Scientiarum Naturalium Universitatis Peki-nensis,2005(1):134-145.
[5] BIRD D L,MUNOZC U.Automatic generation of random self-checking test cases[J].IBM systems journal,1983,22(3):229-245.
[6] CHEN T Y,LEUNG H,MAK I K.Adaptive random testing[C]//Annual Asian Computing Science Conference.Berlin:Springer,2004:320-329.
[7] SAHOO R R,RAY M.PSO based test case generation for criti-cal path using improved combined fitness function[J].Journal of King Saud University-Computer and Information Sciences,2020,32(4):479-490.
[8] BIDGOLI A M,HAGHIGHI H.Augmenting ant colony optimi-zation with adaptive random testing to cover prime paths[J].Journal of Systems and Software,2020,161:110495.
[9] YUAN Z,LOU Y,LIU M,et al.No more manual tests? evalua-ting and improving chatgpt for unit test generation[J].arXiv:2305.04207,2023.
[10] LI T O,ZONG W,WANG Y,et al.Nuances are the key:Unloc-king chatgpt to find failure-inducing tests with differential prompting[C]//2023 38th IEEE/ACM International Confe-rence on Automated Software Engineering(ASE).IEEE,2023:14-26.
[11] KARANJAI R,HUSSAIN A,RABIN M R I,et al.Harnessing the power of llms:Automating unit test generation for high-performance computing[J].arXiv:2407.05202,2024.
[12] UDUM U,KUMAR R,VASUMATHI D.Software Defect Prediction Using Temporal Transformer Graph Network with Newton-Raphson Optimization[J].Journal of Artificial Intelligence and Technology,2025(5):441-452.
[13] HE Y,HUANG J,RONG Y,et al.Unitsyn:A large-scale dataset capable of enhancing the prowess of large language models for program testing[C]//Proceedings of the 33rd ACM SIGSOFT International Symposium on Software Testing and Analysis.2024:1061-1072.
[14] JAIN K,SYNNAEVE G,ROZIÈRE B.Testgeneval:A realworld unit test generation and test completion benchmark[J].arXiv:2410.00752,2024.
[15] TUFANO M,DENG S K,SUNDARESAN N,et al.MethodsTest:A dataset of focal methods mapped to test cases[C]//Proceedings of the 19th International Conference on Mining Software Repositories.2022:299-303.
[16] LI B N,WANG B,CHEN M D,et al.Unit Test Generation for CPrograms via Large Language Models[J].Aerospace Contrd and Application,2025,51(1):115-122.
[17] JOHNSON L A.DO-178B:Software considerations in airbornesystems and equipment certification[J].Crosstalk,October,1998,199:11-20.
[18] SIAMI NAMIN A,ANDREWS J H,MURDOCHD J.Sufficient mutation operators for measuring test effectiveness[C]//Proceedings of the 30th International Conference on Software Engineering.2008:351-360.
[19] National Standardization Administration.Systems and software engineering—Software testing-Part 2:Test Processes:GB/T 38634.2-2020 [S].Beijing:Standards Press of China,2020.
[20] WEI J,WANG X,SCHUURMANS D,et al.Chain-of-thoughtprompting elicits reasoning in large language models[J].Advances in Neural Information Processing Systems,2022,35:24824-24837.
[21] WHITE J,FU Q,HAYS S,et al.A prompt pattern catalog toenhance prompt engineering with chatgpt[J].arXiv:2302.11382,2023.
[22] LIU A,FENG B,XUE B,et al.Deepseek-v3 technical report[J].arXiv:2412.19437,2024.
[23] WANG B.Research on Evaluation Index System and Evaluation Method of Software Test Cases [J].Ship Electronic Enginee-ring,2020,40(11):112-115.
[24] LIZ Z,LI H J.Research on dataset quality assessment model and method for large model training[J].Digital Economy,2024(12):26-30.
[25] YANG A,LI A,YANG B,et al.Qwen3 technical report[J].arXiv:2505.09388,2025.
[26] HOU X,ZHAO Y,WANG S,et al.Model context protocol(mcp):Landscape,security threats,and future research directions[J].arXiv:2503.23278,2025.
[1] XIA Yi, ZHOU Gang, ZHANG Kaixiang, LAN Mingjing, LI Zhufeng, SU Benrong, HE Haofeng, FENG Shizhong. Survey on Collaborative-driven Knowledge Reasoning Technologies Integrating Large Language Models and Knowledge Graphs [J]. Computer Science, 2026, 53(9): 38-54.
[2] ZHANG Rongjie, PANG Xiongwen, WANG Fengling. Review of Large Language Model-based Time Series Modeling via Fine-tuning and AgentArchitecture [J]. Computer Science, 2026, 53(9): 55-70.
[3] CHEN Hai, FAN Changjun, SUN Boliang, CHENG Jie, KONG Junfeng, YUAN Yuchen. Order-based,LLM-augmented Kill-chain for Time-sensitive Target Engagement [J]. Computer Science, 2026, 53(9): 71-81.
[4] GUO Yuyang, SHI Lei, LIU Huan, DONG Yixiang, LI Rui. Domain-adapted and Dynamically Retrieval-augmented Approach for Large-scale History Discipline Model Construction [J]. Computer Science, 2026, 53(9): 92-100.
[5] LEI Kunhan, TAO Jiuyang, LI Chenyu, ZHONG Qi, YE Linfa. Method for Generating Operational Action Plans Based on Large Language Model Guided byChain-of-Thought [J]. Computer Science, 2026, 53(9): 101-109.
[6] HE Jiaojun, LI Xin. Review of Graph Learning Based on Large Language Models:Methods,Benchmarks and Advances [J]. Computer Science, 2026, 53(9): 1-15.
[7] LI Luozheng, LI Lingbo, YUAN Quan. Survey of Chinese Datasets for LLM Safety Alignment:Landscape and Prospects [J]. Computer Science, 2026, 53(9): 16-23.
[8] AN Ran, CHENG Kai, SHAO Tianhao, LI Changyuan, CHEN Yan. Framework,Techniques and Challenges of Task Planning Agent Based on Large Language Model [J]. Computer Science, 2026, 53(9): 24-37.
[9] LI Zhennan, QIAN Jiayan, WANG Xinzhi, ZHANG Hui. Technology Risk Structure Recognition Based on Multi-granularity Semantic Dual Reflection [J]. Computer Science, 2026, 53(9): 395-404.
[10] LING Yun, LI Haodong. Localization of Diffusion-based Image Editing via Inversion-Reconstruction [J]. Computer Science, 2026, 53(8): 182-190.
[11] JIA Zishuo, ZHANG Jian’ge, HE Haofeng, FENG Shizhong, LIU Yilin. Survey on Mutually Augmenting Technologies and Applications of Large Models and KnowledgeGraphs [J]. Computer Science, 2026, 53(8): 219-228.
[12] LIU Jing. Review of Music Artificial Intelligence Driven by Large Language Models [J]. Computer Science, 2026, 53(8): 229-244.
[13] ZHANG Haoran, HAO Wenning, JIN Dawei, CHENG Kai, LIU Junyang. Agentic Retrieval Augmented Generation Framework Based on Retrieval Task Planning and Reflection Mechanism [J]. Computer Science, 2026, 53(8): 285-297.
[14] ZHANG Zhonglin, XIA Hang. LSQ-RAG:Retrieval-enhanced Generation Framework Based on LLM-enhanced Ranker [J]. Computer Science, 2026, 53(8): 298-306.
[15] FAN Ruxin, SUN Baicai, GONG Lina, YAO Xiangjuan, GONG Dunwei. Test Case Generation and Prioritization for Program Fault Diagnosis Based on Large LanguageModel [J]. Computer Science, 2026, 53(8): 365-374.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!