计算机科学

计算机科学 ›› 2025, Vol. 52 ›› Issue (11): 339-348.doi: 10.11896/jsjkx.240900006

• 计算机软件 • 上一篇    下一篇

基于神经元覆盖指标的测试用例生成优化研究

肖子勤, 史涯晴, 曲豫宾   

  1. 陆军工程大学指挥控制工程学院 南京 210007
  • 收稿日期:2024-09-02 修回日期:2024-11-27 出版日期:2025-11-15 发布日期:2025-11-06
  • 通讯作者: 史涯晴(cuterabbitlele@qq.com)
  • 作者简介:(1561294988@qq.com)

Research on Optimization of Test Case Generation Based on Neuron Coverage Index

XIAO Ziqin, SHI Yaqing, QU Yubin   

  1. College of Command and Control Engineering,Army Engineering University,Nanjing 210007,China
  • Received:2024-09-02 Revised:2024-11-27 Online:2025-11-15 Published:2025-11-06
  • About author:XIAO Ziqi,born in 1999,postgraduate.Her main research interests include intelligent software testing and deep learning model testing.
    SHI Yaqing,born in 1981,Ph.D,professor,master's supervisor,is a member of CCF(No.49805M).Her main research interest is intelligent software testing.

PDF (PC)

15

摘要： 深度神经网络(Deep Neural Networks,DNNs)已在诸多领域实现广泛应用,因其复杂性和不确定性,对其进行测试显得尤为重要。传统的测试方法过于依赖单一指标,无法全面揭示深度神经网络的完整行为模式。因此,需综合考量不同的覆盖指标,以便更全面地评估模型性能。结合6种多粒度的深度神经网络覆盖指标,优化模糊测试的变异策略和种子选择等步骤,生成高质量且高覆盖率的测试用例。在MNIST和CIFAR10数据集上对4种不同复杂性的模型进行实验,将原始训练集和新生成的有效测试用例合并用于重训练模型,以提高分类准确率。实验结果显示,该方法可以显著提高覆盖率,并通过自适应重训练优化模型提高了分类准确率。

关键词: 神经网络, 图像分类, 模糊测试, 变异策略, 测试用例生成

Abstract: DNNs have been widely applied in many fields,and testing them is particularly important due to their complexity and uncertainty.Traditional testing methods rely too much on a single indicator and cannot fully reveal the complete behavioral patterns of deep neural networks.Therefore,it is necessary to comprehensively consider different coverage indicators to more comprehensively evaluate the performance of the model.It combines six multi-granularity deep neural network coverage metrics,optimizes the mutation strategy and seed selection steps of fuzzy testing,generates high-quality and high-coverage test cases.Experi-ments are conducted on four models of different complexities on the MNIST and CIFAR10 datasets.The original training set and newly generated effective test cases are combined for retraining the model to classification accuracy.The experimental results show that this method can significantly improve coverage and classification accuracy by optimizing the model through adaptive retraining.

Key words: Neural networks, Image classification, Fuzzy testing, Mutation strategies, Test case generation

中图分类号: 

  • TP311.5
[1]YANG Z,SHI J,ASYROFI M H,et al.Revisiting neuron covera-ge metrics and quality of deep neural networks[C]//2022 IEEE International Conference on Software Analysis,Evolution and Engineering(SANER).2022:408-419.
[2]XIE X,LI T,WANG J,et al.NPC:Neuron path coverage viacharacterizing decision logic of deep neural networks [J].ACM Transactions on Software Engineering and Methodology,2022,31(3):1-27.
[3]AGHABABAEYAN Z,ABDELLATIF M,BRIAND L,et al.Black-box testing of deep neural networks through test case diversity [J].IEEE Transactions on Software Engineering,2023,49(5):3182-3204.
[4]FAHMY H,PASTORE F,BRIAND L.HUDD:A tool to debug DNNs for safety analysis [C]//Proceedings of the ACM/IEEE 44th International Conference on Software Engineering:Companion Proceedings.2022:100-104.
[5]PEI K,CAO Y,YANG J,et al.DeepXplore:automated white box testing of deep learning systems [C]//26th Symposium on Operating Systems Principles.2017:1-18.
[6]XIE X,MA L,JUEFEIXU F,et al.DeepHunter:a coverageguided fuzz testing framework for deep neural networks [C]//28th ACM SIGSOFT Inter-national Symposium on Software Testing and Analysis.2019:146-157.
[7]MA L,XU J F,ZHANG F Y,et al.DeepGauge:multi granularity testing criteria for deep learning systems [C]//33rd ACM/IEEE International Conference on Automated Software Engineering.2018:120-131.
[8]DU X,XIE X,LI Y,et al.Deepcruiser:Automated guided testing for stateful deep learning systems[J].arXiv:1812.05339,2018.
[9]YI Z B,LI S S,MA J,et al.Towards an Efficient and Robust Adversarial Attack Against Neural Text Classifier[J].Internatio-nal Journal of Pattern Recognition and Artificial Intelligence,2022,36(11):2253007.
[10]MA L,XU J F,XUE M H,et al.Deepct:Tomographic combinatorial testing for deep learning systems[C]//2019 IEEE 26th International Conference on Software Analysis,Evolution and Engineering(SANER).IEEE,2019:614-618.
[11]GUO H,TAO C,HUANG Z.Multi-objective white-box test input selection for deep neural network model enhancement [C]//2023 IEEE 34th International Symposium on Software Reliability Engineering.2023:521-532.
[12]YUAN Y,PANG Q,WANG S.Revisiting neuron coverage for DNN testing:A layer wise and distribution-aware criterion[C]//2023 IEEE/ACM 45th International Conference on Software Engineering.2023:1200-1212.
[13]KANG D.Bridging fuzz testing and metamorphic testing forclassification of machine learning [C]//Proceedings of the 30th IEEE International Conference on Consumer Electronics(ICCE 2022).2022:1-2.
[14]LI Z,MA X,XU C,et al.Structural coverage criteria for neural networks could be misleading[C]//2019 IEEE/ACM 41st International Conference on Software Engineering:New Ideas and Emerging Results(ICSE-NIER).IEEE,2019:89-92.
[15]WANG L,XIE X,DU X,et al.DistXplore:Distribution-guided testing for evaluating and enhancing deep learning systems [C]//Proceedings of the 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering.2023:68-80.
[16]XIAO D,LIU Z,YUAN Y,et al.Metamorphic testing of deep learning compilers [C]//Proceedings of the ACM on Measurement and Analysis of Computing Systems.2022:1-28.
[17]HU Q,GUO Y,XIE X,et al.Test optimization in DNN testing:A survey [J].ACM Transactions on Software Engineering and Methodology,2024,1(22):1-41.
[18]ATTAOUI M O,FAHMY H,PASTORE F,et al.DNN explanation for safety analysis:An empirical evaluation of clustering-based approaches [J].ACM Transactions on Software Enginee-ring and Methodology,2023,10(41):16-57.
[19]KAUR K,SINGH E J.Reducing SSIM(structural similarity index measure) using improved edge detection technique on grey scale images[J].International Journal for Research in Applied Science and Engineering Technology,2020,8(9):504-508.
[20]YANG Z,SHI J,ASYROFI M H,et al.Revisiting neuron coverage metrics and quality of deep neural networks[C]//2022 IEEE International Conference on Software Analysis,Evolution and Reengineering.2022:408-419.
[21]ODENA A,OLSSON C,Andersen D,et al.Tensorfuzz:Debugging neural networks with coverage-guided fuzzing[C]//International Conference on Machine Learning.PMLR,2019:4901-4911.
[22]TIAN Y,PEI K,JANA S,et al.Deeptest:Automated testing of deep-neural-network-driven autonomous cars[C]//Proceedings of the 40th International Conference on Software Engineering.ACM,2018:303-314.
[23]YU J,DUAN S,YE X.A white-box testing for deep neural networks based on neuron coverage[J].IEEE Transactions on Neural Networks and Learning Systems,2022,34(11):9185-9197.
[24]WANG Z,YAN M,LIU S,et al.A Review of Deep Neural Network Testing Research [J].Journal of Software,2020,31(5):1255-1275.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发