Computer Science ›› 2024, Vol. 51 ›› Issue (12): 46-52.doi: 10.11896/jsjkx.231000147

• Computer Software • Previous Articles     Next Articles

Regression Test Case Prioritization Approach Based on Deep Learning

ZHANG Lizheng, YANG Qiuhui, LI Xingjia, DAI Shengxin   

  1. College of Computer Science, Sichuan University, Chengdu 610065, China
  • Received:2023-10-20 Revised:2024-03-16 Online:2024-12-15 Published:2024-12-10
  • About author:ZHANG Lizheng,born in 2000,postgraduate,is a member of CCF(No.Q6918G).His main research interests include software quality assurance and testing and automated program repair.
    YANG Qiuhui,born in 1970,Ph.D,associate professor.Her main research interests include software automation testing and software project management.
  • Supported by:
    National Natural Science Foundation of China(62302323) and Sichuan Science and Technology Program(2023NSFSC1413,2023YFG0117).

Abstract: Prioritizing test cases in regression testing can expedite the detection of code defects,save testing time and resources,and enhance testing efficiency.However,existing test case prioritization methods often fail to consider both code change information and test case execution history simultaneously,and they do not adequately account for differences in the length of test case execution history,resulting in poor prioritization outcomes.To address these issues,this paper introduces a deep learning-based approach for prioritizing regression test cases.Initially,it constructs classification models based on code change information and historical execution data separately.Subsequently,it identifies classes affected by code changes using inter-class relationship graphs and classifies test cases belonging to these classes,as well as those that have recently exposed defects.Finally,it employs classification models and heuristic sorting method to prioritize the test cases,followed by merging the sorted results through an iterative process.Experimental results on 6 projects selected from the preprocessed RTPTorrent dataset demonstrate that:1)in scenarios without time constraints,the proposed approach achieves impressive prioritization results across all projects,with an APFD of 0.972 on the cloudify project;2)under time-constrained conditions,the proposed approach outperforms popular existing prioritization methods in terms of NAPFD metrics.

Key words: Test case prioritization, Deep learning, Interclass relation graph, Classification model, Categorization sorting

CLC Number: 

  • TP311.5
[1]MARIJAN D.Comparative study of machine learning test case prioritization for continuous integrationtesting[J].Software Quality Journal,2023,28(4):1-24.
[2]MARCHETTO A,SCANNIELLO G,SUSI A.Combining code and requirements coverage with execution cost for test suite reduction[J].IEEE Transactions on Software Engineering,2017,45(4):363-390.
[3]KHATIBSYARBINI M,ISA M A,JAWAWI D N A,et al.Test case prioritization approaches in regression testing:A systematic literature review[J].Information and Software Technology,2018,93(C):74-93.
[4]LIMA J A,VERGILIO S R.A multi-armed bandit approach for test case prioritization in continuous integration environments[J].IEEE Transactions on Software Engineering,2020,48(2):453-65.
[5]LOU Y,CHEN J,ZHANG L,et al.A survey on regression test-case prioritization[M]//Advances in Computers.Elsevier,2019:1-46.
[6]SIQUEIRA V,MIRANDA B.Investigating the Adoption ofHistory-based Prioritization in the Context of Manual Testing in a Real Industrial Setting[C]//48th Euromicro Conference on Software Engineering and Advanced Applications(SEAA).IEEE,2022:141-148.
[7]LI F,ZHOU J,LI Y,et al.AGA:An Accelerated Greedy Additional Algorithm for Test Case Prioritization[J].IEEE Transactions on Software Engineering,2022,48(12):5102-5119.
[8]DI N D,PANICHELLA A,ZAIDMAN A,et al.A Test Case Prioritization Genetic Algorithm Guided by the Hypervolume Indicator[J].IEEE Transactions on Software Engineering,2020,46(6):674-696.
[9]YANG L,CHEN J,YOU H,et al.Can Code RepresentationBoost IR-Based Test Case Prioritization?[C]//IEEE 34th International Symposium on Software Reliability Engineering(ISSRE).IEEE,2023:240-251.
[10]AZIZI M.QRTest:Automatic Query Reformulation for InformationRetrieval Based Regression Test Case Prioritization[C]//IEEE International Conference on Software Testing,Verification and Validation Workshops(ICSTW).IEEE,2021:254-262.
[11]LIN C T,YUAN S H,INTASARA J.A Learning-to-RankBased Approach for Improving Regression Test Case Prioritization[C]//28th Asia-Pacific Software Engineering Conference(APSEC).IEEE,2021:576-577.
[12]SHARMA S,CHANDE S V.Optimizing test case prioritization using machine learning algorithms[J].Journal of Autonomous Intelligence,2023,6(2):661-676.
[13]SHARIF A,MARIJAN D,Liaaen M.DeepOrder:Deep learning for test case prioritization in continuous integration testing[C]//IEEE International Conference on Software Maintenance and Evolution(ICSME).IEEE,2021:525-534.
[14]XIAO L,MIAO H,SHI T,et al.LSTM-based deep learning for spatial-temporal software testing[J].Distributed and Parallel Databases,2020,38(3):687-712.
[15]ABDELKARIM M,ELADAWI R.TCP-Net:Test Case Prioritization using End-to-End Deep Neural Networks[C]//IEEE International Conference on Software Testing,Verification and Validation Workshops(ICSTW).IEEE,2022:122-129.
[16]SPIEKER H,GOTLIEB A,MARIJAN D,et al.Reinforcement learning for automatic test case prioritization and selection in continuous integration[C]//Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis.2017:12-22.
[17]WU Z,YANG Y,LI Z,et al.A time window based reinforcement learning reward for test case prioritization in continuous integration[C]//Proceedings of the 11th Asia-Pacific Sympo-sium on Internetware.2019:1-6.
[18]YANG Y,PAN C,LI Z,et al.Adaptive Reward Computation in Reinforcement Learning-Based Continuous Integration Testing[J].IEEE Access,2021,9:36674-36688.
[19]ZHAO Y F,HAO D.Test Case Prioritization Technique in Continuous Integration Based on Reinforcement Learning[J].Journal of Software,2023,34(6):2708-2726.
[20]ROTHERMEL G,UNTCH R H,CHU C,et al.Prioritizing test cases for regression testing[J].IEEE Transactions on software engineering,2001,27(10):929-948.
[21]NOOR T B,HEMMATI H.A similarity-based approach for test case prioritization using historical failure data[C]//IEEE 26th International Symposium on Software Reliability Engineering(ISSRE).IEEE,2015:58-68.
[22]MATTIS T,REIN P,DÜRSCH F,et al.RTPTorrent:An open-source dataset for evaluating regression test pri-oritization[C]//Proceedings of the 17th International Conference on Mining Software Repositories.2020:385-396.
[23]XIAO L.Research on Test Case Prioritization Technique Based on History Execution Information in Continuous Integration[D].Shanghai:Shanghai University,2020.
[24]MACHALICA M,SAMYLKIN A,PORTH M,et al.Predictivetest selection[C]//IEEE/ACM 41st International Conference on Software Engineering:Software Engineering in Practice(ICSE-SEIP).IEEE,2019:91-100.
[25]MEMON A,GAO Z,NGUYEN B,et al.Taming Google-scalecontinuous testing[C]//IEEE/ACM 39th International Confe-rence on Software Engineering:Software Engineering in Practice Track(ICSE-SEIP).IEEE,2017:233-242.
[26]FENG Z,GUO D,TANG D,et al.CodeBERT:A Pre-Trained Model for Programming and Natural Languages[C]//Findings of the Association for Computational Linguistics.EMNLP,2020:1536-1547.
[27]ADEM K,KILICARSLAN S,COMERT O.Classification anddiagnosis of cervical cancer with softmax classification with stacked autoencoder[J].Expert Systems with Applications,2019,115:557-564.
[28]LIN T Y,GOYAL P,GIRSHICK R,et al. Focal loss for dense object detection[C]//Proceedings of the IEEEInternational Conference on Computer Vision.IEEE,2017:2980-2988.
[29]ORSO A,SHI N,HARROLD M J.Scaling regression testing to large software systems[J].ACM SIGSOFT Software Enginee-ring Notes,2004,29(6):241-251.
[30]ROTHERMEL G,UNTCH R H,CHU C,et al. Test case prioritization: An empirical study[C]//Proceedings IEEE International Conference on Software Maintenance.IEEE,1999:179-188.
[31]QU X,COHEN M B,WOOLF K M.Combinatorial interactionregression testing:A study of test case generation and prioritization[C]//IEEE International Conference on Software Maintenance.IEEE,2007:255-264.
[32]BAGHERZADEH M,KAHANI N,BRIAND L.Reinforcement learning for test case prioritization[J].IEEE Transactions on Software Engineering,2021,48(8):2836-2856.
[1] DU Yu, YU Zishu, PENG Xiaohui, XU Zhiwei. Padding Load:Load Reducing Cluster Resource Waste and Deep Learning Training Costs [J]. Computer Science, 2024, 51(9): 71-79.
[2] XU Jinlong, GUI Zhonghua, LI Jia'nan, LI Yingying, HAN Lin. FP8 Quantization and Inference Memory Optimization Based on MLIR [J]. Computer Science, 2024, 51(9): 112-120.
[3] SUN Yumo, LI Xinhang, ZHAO Wenjie, ZHU Li, LIANG Ya’nan. Driving Towards Intelligent Future:The Application of Deep Learning in Rail Transit Innovation [J]. Computer Science, 2024, 51(8): 1-10.
[4] KONG Lingchao, LIU Guozhu. Review of Outlier Detection Algorithms [J]. Computer Science, 2024, 51(8): 20-33.
[5] TANG Ruiqi, XIAO Ting, CHI Ziqiu, WANG Zhe. Few-shot Image Classification Based on Pseudo-label Dependence Enhancement and NoiseInterferenceReduction [J]. Computer Science, 2024, 51(8): 152-159.
[6] XIAO Xiao, BAI Zhengyao, LI Zekai, LIU Xuheng, DU Jiajin. Parallel Multi-scale with Attention Mechanism for Point Cloud Upsampling [J]. Computer Science, 2024, 51(8): 183-191.
[7] ZHANG Junsan, CHENG Ming, SHEN Xiuxuan, LIU Yuxue, WANG Leiquan. Diversified Label Matrix Based Medical Image Report Generation [J]. Computer Science, 2024, 51(8): 200-208.
[8] GUO Fangyuan, JI Genlin. Video Anomaly Detection Method Based on Dual Discriminators and Pseudo Video Generation [J]. Computer Science, 2024, 51(8): 217-223.
[9] CHEN Siyu, MA Hailong, ZHANG Jianhui. Encrypted Traffic Classification of CNN and BiGRU Based on Self-attention [J]. Computer Science, 2024, 51(8): 396-402.
[10] YANG Heng, LIU Qinrang, FAN Wang, PEI Xue, WEI Shuai, WANG Xuan. Study on Deep Learning Automatic Scheduling Optimization Based on Feature Importance [J]. Computer Science, 2024, 51(7): 22-28.
[11] LI Jiaying, LIANG Yudong, LI Shaoji, ZHANG Kunpeng, ZHANG Chao. Study on Algorithm of Depth Image Super-resolution Guided by High-frequency Information ofColor Images [J]. Computer Science, 2024, 51(7): 197-205.
[12] SHI Dianxi, GAO Yunqi, SONG Linna, LIU Zhe, ZHOU Chenlei, CHEN Ying. Deep-Init:Non Joint Initialization Method for Visual Inertial Odometry Based on Deep Learning [J]. Computer Science, 2024, 51(7): 327-336.
[13] FAN Yi, HU Tao, YI Peng. Host Anomaly Detection Framework Based on Multifaceted Information Fusion of SemanticFeatures for System Calls [J]. Computer Science, 2024, 51(7): 380-388.
[14] GAN Run, WEI Xianglin, WANG Chao, WANG Bin, WANG Min, FAN Jianhua. Backdoor Attack Method in Autoencoder End-to-End Communication System [J]. Computer Science, 2024, 51(7): 413-421.
[15] HUANG Haixin, CAI Mingqi, WANG Yuyao. Review of Point Cloud Semantic Segmentation Based on Graph Convolutional Neural Networks [J]. Computer Science, 2024, 51(6A): 230400196-7.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!