计算机科学

计算机科学 ›› 2024, Vol. 51 ›› Issue (12): 46-52.doi: 10.11896/jsjkx.231000147

• 计算机软件 • 上一篇    下一篇

基于深度学习的回归测试用例优先级排序方法

张李政, 杨秋辉, 李兴佳, 代声馨   

  1. 四川大学计算机学院 成都 610065
  • 收稿日期:2023-10-20 修回日期:2024-03-16 出版日期:2024-12-15 发布日期:2024-12-10
  • 通讯作者: 杨秋辉(yangqiuhui@scu.edu.cn)
  • 作者简介:(zleo0824@gmail.com)
  • 基金资助:
    国家自然科学基金(62302323);四川省科技计划资助项目(2023NSFSC1413,2023YFG0117)

Regression Test Case Prioritization Approach Based on Deep Learning

ZHANG Lizheng, YANG Qiuhui, LI Xingjia, DAI Shengxin   

  1. College of Computer Science, Sichuan University, Chengdu 610065, China
  • Received:2023-10-20 Revised:2024-03-16 Online:2024-12-15 Published:2024-12-10
  • About author:ZHANG Lizheng,born in 2000,postgraduate,is a member of CCF(No.Q6918G).His main research interests include software quality assurance and testing and automated program repair.
    YANG Qiuhui,born in 1970,Ph.D,associate professor.Her main research interests include software automation testing and software project management.
  • Supported by:
    National Natural Science Foundation of China(62302323) and Sichuan Science and Technology Program(2023NSFSC1413,2023YFG0117).

PDF (PC)

213

摘要： 在回归测试中对测试用例排序可以更快地发现代码缺陷,节约测试时间和资源,提高测试效率。现有的测试用例排序方法没有同时考虑代码的变更信息以及测试用例的历史执行信息,也没有考虑不同测试用例执行历史长短的区别,因此排序效果不佳。针对这些问题,提出基于深度学习的回归测试用例优先级排序方法。首先分别构建基于代码变更信息和历史执行信息的分类模型;然后基于类间关系图识别受代码变更影响的类,对这些类的测试用例以及近期执行发现缺陷的测试用例进行分类,使用分类模型和启发式排序方法对测试用例分类进行排序;最后通过交替排序融合排序结果。在RTPTorrent数据集上选取6个项目进行实验,结果表明:1)在无时间约束时,所提方法在所有项目上都取得了不错的排序效果,在cloudify项目上的APFD指标达到0.972;2)在有时间约束时,所提方法的NAPFD指标超过了目前主流的排序方案。

关键词: 测试用例排序, 深度学习, 类间关系图, 分类模型, 分类排序

Abstract: Prioritizing test cases in regression testing can expedite the detection of code defects,save testing time and resources,and enhance testing efficiency.However,existing test case prioritization methods often fail to consider both code change information and test case execution history simultaneously,and they do not adequately account for differences in the length of test case execution history,resulting in poor prioritization outcomes.To address these issues,this paper introduces a deep learning-based approach for prioritizing regression test cases.Initially,it constructs classification models based on code change information and historical execution data separately.Subsequently,it identifies classes affected by code changes using inter-class relationship graphs and classifies test cases belonging to these classes,as well as those that have recently exposed defects.Finally,it employs classification models and heuristic sorting method to prioritize the test cases,followed by merging the sorted results through an iterative process.Experimental results on 6 projects selected from the preprocessed RTPTorrent dataset demonstrate that:1)in scenarios without time constraints,the proposed approach achieves impressive prioritization results across all projects,with an APFD of 0.972 on the cloudify project;2)under time-constrained conditions,the proposed approach outperforms popular existing prioritization methods in terms of NAPFD metrics.

Key words: Test case prioritization, Deep learning, Interclass relation graph, Classification model, Categorization sorting

中图分类号: 

  • TP311.5
[1]MARIJAN D.Comparative study of machine learning test case prioritization for continuous integrationtesting[J].Software Quality Journal,2023,28(4):1-24.
[2]MARCHETTO A,SCANNIELLO G,SUSI A.Combining code and requirements coverage with execution cost for test suite reduction[J].IEEE Transactions on Software Engineering,2017,45(4):363-390.
[3]KHATIBSYARBINI M,ISA M A,JAWAWI D N A,et al.Test case prioritization approaches in regression testing:A systematic literature review[J].Information and Software Technology,2018,93(C):74-93.
[4]LIMA J A,VERGILIO S R.A multi-armed bandit approach for test case prioritization in continuous integration environments[J].IEEE Transactions on Software Engineering,2020,48(2):453-65.
[5]LOU Y,CHEN J,ZHANG L,et al.A survey on regression test-case prioritization[M]//Advances in Computers.Elsevier,2019:1-46.
[6]SIQUEIRA V,MIRANDA B.Investigating the Adoption ofHistory-based Prioritization in the Context of Manual Testing in a Real Industrial Setting[C]//48th Euromicro Conference on Software Engineering and Advanced Applications(SEAA).IEEE,2022:141-148.
[7]LI F,ZHOU J,LI Y,et al.AGA:An Accelerated Greedy Additional Algorithm for Test Case Prioritization[J].IEEE Transactions on Software Engineering,2022,48(12):5102-5119.
[8]DI N D,PANICHELLA A,ZAIDMAN A,et al.A Test Case Prioritization Genetic Algorithm Guided by the Hypervolume Indicator[J].IEEE Transactions on Software Engineering,2020,46(6):674-696.
[9]YANG L,CHEN J,YOU H,et al.Can Code RepresentationBoost IR-Based Test Case Prioritization?[C]//IEEE 34th International Symposium on Software Reliability Engineering(ISSRE).IEEE,2023:240-251.
[10]AZIZI M.QRTest:Automatic Query Reformulation for InformationRetrieval Based Regression Test Case Prioritization[C]//IEEE International Conference on Software Testing,Verification and Validation Workshops(ICSTW).IEEE,2021:254-262.
[11]LIN C T,YUAN S H,INTASARA J.A Learning-to-RankBased Approach for Improving Regression Test Case Prioritization[C]//28th Asia-Pacific Software Engineering Conference(APSEC).IEEE,2021:576-577.
[12]SHARMA S,CHANDE S V.Optimizing test case prioritization using machine learning algorithms[J].Journal of Autonomous Intelligence,2023,6(2):661-676.
[13]SHARIF A,MARIJAN D,Liaaen M.DeepOrder:Deep learning for test case prioritization in continuous integration testing[C]//IEEE International Conference on Software Maintenance and Evolution(ICSME).IEEE,2021:525-534.
[14]XIAO L,MIAO H,SHI T,et al.LSTM-based deep learning for spatial-temporal software testing[J].Distributed and Parallel Databases,2020,38(3):687-712.
[15]ABDELKARIM M,ELADAWI R.TCP-Net:Test Case Prioritization using End-to-End Deep Neural Networks[C]//IEEE International Conference on Software Testing,Verification and Validation Workshops(ICSTW).IEEE,2022:122-129.
[16]SPIEKER H,GOTLIEB A,MARIJAN D,et al.Reinforcement learning for automatic test case prioritization and selection in continuous integration[C]//Proceedings of the 26th ACM SIGSOFT International Symposium on Software Testing and Analysis.2017:12-22.
[17]WU Z,YANG Y,LI Z,et al.A time window based reinforcement learning reward for test case prioritization in continuous integration[C]//Proceedings of the 11th Asia-Pacific Sympo-sium on Internetware.2019:1-6.
[18]YANG Y,PAN C,LI Z,et al.Adaptive Reward Computation in Reinforcement Learning-Based Continuous Integration Testing[J].IEEE Access,2021,9:36674-36688.
[19]ZHAO Y F,HAO D.Test Case Prioritization Technique in Continuous Integration Based on Reinforcement Learning[J].Journal of Software,2023,34(6):2708-2726.
[20]ROTHERMEL G,UNTCH R H,CHU C,et al.Prioritizing test cases for regression testing[J].IEEE Transactions on software engineering,2001,27(10):929-948.
[21]NOOR T B,HEMMATI H.A similarity-based approach for test case prioritization using historical failure data[C]//IEEE 26th International Symposium on Software Reliability Engineering(ISSRE).IEEE,2015:58-68.
[22]MATTIS T,REIN P,DÜRSCH F,et al.RTPTorrent:An open-source dataset for evaluating regression test pri-oritization[C]//Proceedings of the 17th International Conference on Mining Software Repositories.2020:385-396.
[23]XIAO L.Research on Test Case Prioritization Technique Based on History Execution Information in Continuous Integration[D].Shanghai:Shanghai University,2020.
[24]MACHALICA M,SAMYLKIN A,PORTH M,et al.Predictivetest selection[C]//IEEE/ACM 41st International Conference on Software Engineering:Software Engineering in Practice(ICSE-SEIP).IEEE,2019:91-100.
[25]MEMON A,GAO Z,NGUYEN B,et al.Taming Google-scalecontinuous testing[C]//IEEE/ACM 39th International Confe-rence on Software Engineering:Software Engineering in Practice Track(ICSE-SEIP).IEEE,2017:233-242.
[26]FENG Z,GUO D,TANG D,et al.CodeBERT:A Pre-Trained Model for Programming and Natural Languages[C]//Findings of the Association for Computational Linguistics.EMNLP,2020:1536-1547.
[27]ADEM K,KILICARSLAN S,COMERT O.Classification anddiagnosis of cervical cancer with softmax classification with stacked autoencoder[J].Expert Systems with Applications,2019,115:557-564.
[28]LIN T Y,GOYAL P,GIRSHICK R,et al. Focal loss for dense object detection[C]//Proceedings of the IEEEInternational Conference on Computer Vision.IEEE,2017:2980-2988.
[29]ORSO A,SHI N,HARROLD M J.Scaling regression testing to large software systems[J].ACM SIGSOFT Software Enginee-ring Notes,2004,29(6):241-251.
[30]ROTHERMEL G,UNTCH R H,CHU C,et al. Test case prioritization: An empirical study[C]//Proceedings IEEE International Conference on Software Maintenance.IEEE,1999:179-188.
[31]QU X,COHEN M B,WOOLF K M.Combinatorial interactionregression testing:A study of test case generation and prioritization[C]//IEEE International Conference on Software Maintenance.IEEE,2007:255-264.
[32]BAGHERZADEH M,KAHANI N,BRIAND L.Reinforcement learning for test case prioritization[J].IEEE Transactions on Software Engineering,2021,48(8):2836-2856.
Viewed
Full text


Abstract

Cited
  Shared   
  Discussed   
No Suggested Reading articles found!
渝ICP备16013121号-4
地址：重庆市渝北区洪湖西路18号    邮编：401121  
电话：023-63500828（编辑部） 023-67039612（会议/专题） 023-67039623（财务/发票）
E-mail：jsjkx12@163.com
本系统由北京玛格泰克科技发展有限公司设计开发