Computer Science ›› 2014, Vol. 41 ›› Issue (9): 196-204.doi: 10.11896/j.issn.1002-137X.2014.09.037

Previous Articles     Next Articles

Systematic Review of Test Suite Minimization for Regression Testing

CHEN Xiang,GU Qing,CHEN Dao-xu and JIANG Zheng-zheng   

  • Online:2018-11-14 Published:2018-11-14

Abstract: Test suite minimization (TSM) is a hot and difficult issue in regression testing search.It aims to identify and remove redundant test cases,and reducing the cost of regression testing.The reduced test suite can satisfy the same test requirements as the original test suite.In this paper,we reviewed the existing research work of TSM.We firstly classified existing TSM approaches into two categories:source code based and model based.In source code based approaches,we analyzed and summarized traditional TSM approaches and fault detection ability concerned TSM approaches.In modelbased approaches,we mainly analyzed and summarized EFSM based approaches.We secondly summarized empirical subjects,evaluation metrics,and empirical results of previous empirical studies.We thirdly summarized the successful applications of TSM in some specific testing domains,such as GUI application testing,Web Application testing,and fault localization.We finally gave some future work for this hot research topic.

Key words: Regression testing,Test suite minimization,Linear programming,Greedy algorithm,Meta-heuristic search,Multi-objective optimization,Empirical study

[1] Rothermel G,Untch R J,Chu C.Prioritizing Test Cases for Regression Testing[J].IEEE Transactions on Software Enginee-ring, 2001,27(10):929-948
[2] Harrold M,Orso A.Retesting Software during Development andMaintenance[C]∥Proceedings of Frontiers of Software Maintenance.Beijing:IEEE Press,2008:99-108
[3] Beizer B.Software Testing Techniques[M].New York:Van No strand Reinhold,1990
[4] Leung H,White L.Insights into Regression Testing[C]∥Proceedings of the International Conference on Software Maintenance.Pittsburg:ACM Press,1989:60-69
[5] Malishevsky A G,Rothermel G,Elbaum S.Modeling the Cost-benefits Tradeoffs for Regression Testing Techniques[C]∥Proceedings of the International Conference on Software Maintenance.Montreal:IEEE Press,2002204-213
[6] Do H,Rothermel G.An Empirical Study of Regression Testing Techniques incorporating Context and Lifetime Factors and Improved Cost-benefit Models[C]∥Proceedings of the Symposium on the Foundations of Software Engineering.Graz:Springer Press,2006:141-151
[7] Do H,Rothermel G.Using Sensitivity Analysis to Create Sim-plified Economic Models for Regression Testing[C]∥Procee-dings of the International Symposium on Software Testing and Analysis.Seattle:ACM Press,2008:51-62
[8] Harrold M J,Gupta R,Soffa M L.A Methodology for Controlling the Size of a Test Suite[J].ACM Transactions on Software Engineering and Methodology,1993,2(3):270-285
[9] Zhu H,Hall P,May J.Software Unit Test Coverage and Adequacy[J].ACM Computing Survey,1997,29(4):366-427
[10] Black J,Melachrinoudis E,Kaeli D.Bi-criteria Models for All-uses Test Suite Reduction[C]∥Proceedings of the International Conference on Software Engineering.Edinburgh:IEEE Press,2004:106-115
[11] Zhong H,Zhang L,Mei H.An Experimental Study of Four Typi-cal Test Suite Reduction Techniques[J].Information and Software Technology,2008,50(6):534-546
[12] Zhang L M,Marinov D,Zhang L,Khurshid S.An EmpiricalStudy of Junit Test-Suite Reduction[C]∥Proceedings of the International Symposium on Software Reliability Engineering.Hiroshima:IEEE Press,2011:170-179
[13] Chen T Y,Lau M F.A New Heuristic for Test Suite Reduction[J].Information and Software Technology,1998,40(5/6):347-354
[14] Chen T Y,Lau M F.On the Divide-and-conquer Approach towards Test Suite Reduction[J].Information Sciences,2003,152(1):89-119
[15] Jones J A,Harrold M J.Test-Suite Reduction and Prioritization for Modified Condition/Decision Coverage[J].IEEE Transactions on Software Engineering,2003,29(3):195-209
[16] Tallam S,Gupta N.A Concept Analysis Inspired Greedy Algorithm for Test Suite Minimization[C]∥Proceedings of the 6th workshop on Program Analysis for Software Tools and Engineering.Lisbon:ACM Press,2005:35-42
[17] Mansour N,El-Fakih K.Simulated Annealing and Genetic Algorithms for Optimal Regression Testing[J].Journal of Software Maintenance:Research and Practice,1999,11(1):19-34
[18] Jeffrey D,Gupta N.Test Suite Reduction with Selective Redundancy[C]∥Proceedings of the International Conference on Software Maintenance.Budapest:IEEE Press,2005:549-558
[19] Jeffrey D,Gupta N.Improving Fault Detection Capability by Selectively Retaining Test Cases during Test Suite Reduction[J].IEEE Transactions on Software Engineering,2007,33(2):108-123
[20] Hsu H Y,Orso A.MINTS:A General Framework and Tool for Supporting Test-suite Minimization[C]∥Proceedings of the International Conference on Software Engineering.Vancouver:IEEE Press,2009:419-429
[21] Yoo S,Harman M.Pareto Efficient Multi-objective Test CaseSelection[C]∥Proceedings of the International Symposium on Software Testing and Analysis.London:ACM Press,2007:140-150
[22] Yoo S,Harman M.Using Hybrid Algorithm for Pareto Efficient Multi-objective Test Suite Minimization[J].Journal of System and Software,2010,83(4):689-701
[23] Harder M,Mellen J,Ernst M D.Improving Test Suites via Opera-tional Abstraction[C]∥Proceedings of the International Confe-rence on Software Engineering.Portland:IEEE Press,2003:60-71
[24] McMaster S,Memon A M.Call Stack Coverage for Test Suite Reduction[C]∥Proceedings of the International Conference on Software Maintenance.Budapest:IEEE Press,2005:539-548
[25] Hao D,Zhang L,Wu X,et al.On-demand Test Suite Reduction[C]∥Proceedings of the International Conference on Software Engineering.Zurich:IEEE Press,2012.738-748
[26] Marre M,Bertolino A.Using Spanning Sets for Coverage Testing[J].IEEE Transactions on Software Engineering,2003,29(11):974-984
[27] 章晓芳,徐宝文,聂长海,等.一种基于测试需求约简的测试用例集优化方法[J].软件学报,2007,8(4):821-831
[28] Chen Z Y,Xu B W,Zhang X F,et al.A Novel Approach for Test Suite Reduction based on Requirement Relation Contraction[C]∥Proceedings of the ACM Symposium on Applied Computing.Fortaleza:ACM Press,2008:390-394
[29] 顾庆,唐宝,陈道蓄.一种面向测试需求部分覆盖的测试用例集约简技术[J].计算机学报,2011(5):879-889
[30] Zhang L J,Chen X,Gu Q,et al.CATESR:Change-aware Test Suite Reduction based on Partial Coverage of Test Requirements[C]∥Proceedings of International Conference on Software Engineering & Knowledge Engineering.San Francisco Bay:Knowledge Systems Institute Graduate School,2012:217-224
[31] Chen X,Zhang L J,Gu Q,et al.A Test Suite Reduction Ap-proach based on Pairwise Interaction of Requirements[C]∥Proceedings of the Symposium on Applied Computing.TaiChung:ACM Press,2011:1390-1397
[32] Vaysburg B,Tahat L H,Korel B.Dependence Analysis in Reduction of Requirement based Test Suites[C]∥Proceedings of the International Symposium on Software Testing and Analysis.Roma:ACM Press,2002:107-111
[33] Korel B,Tahat L,Vaysburg B.Model based Regression TestReduction using Dependence Analysis[C]∥Proceedings of the International Conference on Software Maintenance.Montreal:IEEE Press,2002:214-225
[34] Chen Y,Probert R L,Ural H.Regression Test Suite Reduction using Extended Dependence Analysis[C]∥Proceedings of the International Workshop on Software Quality Assurance.Dubrovnik:ACM Press,2007:62-69
[35] Do H,Elbaum S,Rothermel G.Supporting Controlled Experi-mentation with Testing Techniques:an Infrastructure and its Potential Impact[J].Empirical Software Engineering,2005,10(4):405-435
[36] Hutchins M,Foster H,Goradia T,et al.Experiments on the Effectiveness of Dataflow and Control-flow-based Test Adequacy Criteria[C]∥Proceedings of the International Conference on Software Engineering.Sorrento:ACM Press,1994:191-200
[37] McMaster S,Memon A M.Fault Detection Probability Analysis for Coverage-Based Test Suite Reduction[C]∥Proceedings of the International Conference on Software Maintenance.Paris:IEEE Press,2007:335-344
[38] Chen T Y,Lau M.A Simulation Study on some Heuristics for Test Suite Reduction[J].Information and Software Technology,1998,40(13):777-787
[39] Wong W E,Horgan J R,London S,et al.Effect of Test Set Mini-mization on Fault Detection Effectiveness[J].Software Practice and Experience,1998,28(4):347-369
[40] Wong W E,Horgan J R,Mathur A P,et al.Test Set Size Minimization and Fault Detection Effectiveness:A Case Study in a Space Application[J].The Journal of Systems and Software,1999,48(2):79-89
[41] Rothermel G,Harrold M J,Ostrin J,et al.An Empirical Study of the Effects of Minimization on the Fault Detection Capabilities of Test Suites[C]∥Proceedings of International Conference on Software Maintenance.Maryland:IEEE Press,1998:34-43
[42] Rothermel G,Harrold M,Ronne J,et al.Empirical Studies of Test Suite Reduction[J].Software Testing,Verification,and Reliability,2002,4(2):219-249
[43] McMaster S,Memon A M.Call Stack Coverage for GUI Test-Suite Reduction[C]∥Proceedings of International Symposium on Software Reliability Engineering.North Carolina:IEEE Press,2006:33-44
[44] McMaster S,Memon A F.Call-Stack Coverage for GUI TestSuite Reduction[J].IEEE Transactions on Software Enginee-ring,2008,34(1):99-115
[45] Sprenkle S,Sampath S,Gibson E,et al.An Empirical Comparison of Test Suite Reduction Techniques for User-session-based Testing of Web Applications[C]∥Proceedings of International Conference on Software Maintenance.Budapest:IEEE Press,2005:587-596
[46] Sampath S,Bryce R.Improving the Effectiveness of Test Suite Reduction for User-Session-Based Testing of Web Applications[J].Information and Software Technology,2012,54(7):724-738
[47] Yu Y,Jones J A,Harrold M J.An Empirical Study of theEffects of Test-suite Reduction on Fault Localization[C]∥Proceedings of the International Conference on Software Enginee-ring.Leipzig:ACM Press,2008:201-210
[48] Zhang X,Gu Q,Chen X,et al.A Study of Relative Redundancy in Test-suite Reduction while Retaining or Improving Fault-localization Effectiveness[C]∥Proceedings of the ACM Symposiumon Applied Computing.Sierre:ACM Press,2010:2229-2236
[49] Harman M.The Current State and Future of Search Based Software Engineering[C]∥Proceedings of Workshop on the Future of Software Engineering.Minneapolis:ACM Press,2007:342-357
[50] 陈翔,顾庆.变异测试:原理、优化和应用[J].计算机科学与探索,2012,6(12):1057-1075
[51] Andrews J H,Briand L C,Labiche Y.Is Mutation an Appropria-te Tool for Testing Experiments[C]∥Proceedings of International Conference on Software Engineering.St Louis:ACM Press,2005:402-411

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!