计算机科学 ›› 2013, Vol. 40 ›› Issue (6): 8-15.

• 综述 • 上一篇    下一篇

回归测试中测试用例集扩充技术研究进展

陈翔,顾庆,陈道蓄   

  1. 南通大学计算机科学与技术学院 南通226019;南京大学软件新技术国家重点实验室 南京210093;南京大学软件新技术国家重点实验室 南京210093
  • 出版日期:2018-11-16 发布日期:2018-11-16
  • 基金资助:
    本文受国家自然科学基金(60873027,61202006),江苏省高校自然科学研究项目(12KJB520014),南通市应用研究计划项目(BK2012023),南京大学计算机软件新技术国家重点实验室开放课题(KFKT2012B29)资助

Research Advances in Test Suite Augmentation for Regression Testing

CHEN Xiang,GU Qing and CHEN Dao-xu   

  • Online:2018-11-16 Published:2018-11-16

摘要: 测试用例集扩充问题(Test Suite Augmentation)是回归测试研究的一个最新研究热点。在完成代码修改影响分析后,对已有测试用例集的充分性进行评估,若不充分则设计新的测试用例,以确保对代码修改的充分测试。但到目前为止,国内外学者并未对该研究问题的已有研究成果进行系统总结和展望。首先介绍了测试用例集扩充问题的研究背景和问题描述,然后总结出研究框架并对已有研究工作进行分类和系统比较,接着对常用评测数据集和评测指标进行了分析,最后对该问题值得关注的未来研究方向进行展望。

关键词: 回归测试,测试用例集扩充,测试覆盖准则,动态符号执行,演化测试

Abstract: Test suite augmentation is a hot research issue in regression testing.After code change impact analysis,we evaluated the adequacy of existing test suite.If not adequate,we will design new test cases to adequately test the code change.Until now,no researchers have given a survey for this research topic.We firstly introduced the research background and problem description of this topic.Secondly we summarized a framework of this topic and made a systematic classification and comparison for existing research work.Thirdly we introduced the common used experiment subjects and evaluation metrics.Lastly we gave some suggestion on potential future work for this research area.

Key words: Regression testing,Test suite augmentation,Test coverage criterion,Dynamic symbolic execution,Evolutionary testing

[2] Leung H,White L.Insights into Regression Testing [C]∥Proceedings of the International Conference on Software Maintena-ce.1989:60-69
[3] Harrold M J,Orso A.Retesting Software during Development and Maintenance [C]∥Proceedings of Frontiers of Software Maintenance.2008:99-108
[4] Yoo S,Harman M.Regression Testing Minimization,Selectionand Prioritization:a Survey [J].Software Testing,Verification & Reliability,2012,22(2):67-120
[5] 章晓芳,陈林,徐宝文,等.测试用例集约简问题研究及其进展 [J].计算机科学与探索,2008(3):235-247
[6] 屈波,聂长海,徐宝文.回归测试中测试用例优先级技术研究综述 [J].计算机科学与探索,2009(3):225-233
[7] Santelices R,Chittimalli PK,Apiwattanapong,et al.Test-suite Augmentation for Evolving Software [C]∥Proceedings of the International Conference on Automated Software Engineering.2008:218-227
[8] Xu Z,Kim Y,Kim M,et al.Directed Test Suite Augmentation:Techniques and Tradeoffs [C]∥Proceedings of the InternationalSymposium on Foundations of Software Engineering.2010:257-266
[9] Person S,Dwyer M B,Elbaum S,et al.Differential Symbolic Exe-cution [C]∥Proceedings of the International Symposium on Foundations of Software Engineering.2008:226-237
[10] Rothermel G,Harrold M J.Selecting Tests and Identifying Test Coverage Requirements for Modified Software [C]∥Procee-dings of the International Symposium on Software Testing and Analysis.1994:169-184
[11] Taneja K,Xie T,Tillmann N,et al.eXpress:Guided Path Exploration for Regression Test Generation [C]∥Proceedings of the International Symposium on Software Testing and Analysis.2011:1-11
[12] Qi D,Roychoudhury A,Liang Z.Test Generation to ExposeChanges in Evolving Programs [C]∥Proceedings of the International Conference on Automated Software Engineering.2010:397-406
[13] Santelices R,Harrold M J.Applying Aggressive Propagation-based Strategies for Testing Changes [C]∥Proceedings of the International Conference on Software Testing.2011:11-20
[14] Xu Z H,Kim Y,Kim M,et al.A Hybrid Directed Test Suite Augmentation Technique [C]∥Proceedings of the International Symposium on Software Reliability Engineering.2011:150-159
[15] Xu Z,Cohen M B,Rothermel G.Factors Affecting the Use of Genetic Algorithms in Test Suite Augmentation [C]∥Procee-dings of the Annual Conference on Genetic and Evolutionary Computation.2010:1365-1372
[16] Xu Z,Rothermel G.Directed Test Suite Augmentation [C]∥Proceedings of the Asia-Pacific Software Engineering Confe-rence.2009:406-413
[17] Apiwattanapong T,Santelices R,Chittimalli PK,et al.Matrix:Maintenance-oriented Testing Requirement Identifier and Exa-miner [C]∥Proceedings of the Testing:Academic and Industrial Conference-Practice and Research Techniques.2006:137-146
[18] Binkley D.Semantics Guided Regression Test Cost Reduction[J].IEEE Transactions on Software Engineering,1997,23:498-516
[19] Daniel B,Jagannath V,Dig D,et al.Reassert:Suggesting Repair for Broken Unit Tests [C]∥Proceedings of the International Conference on Automated Software Engineering.2009:433-444
[20] Daniel B,Gvero T,Marinov D.On Test Repair using Symbolic Execution [C]∥Proceedings of the International Symposium on Software Testing and Analysis.2010:207-218
[21] Rothermel G,Harrold M.A Safe,Efficient Regression Test Selection Technique [J].ACM Transactions on Software Engineering and Methodology,1997,6(2):173-210
[22] Rothermel G,Harrold M.Empirical Studies of a Safe Regression Test Selection Technique [J].IEEE Transactions on Software Engineering,1998,24(6):401-419
[23] Rothermel G,Harrold M,Dedhia J.Regression Test Selectionfor C++ Software [J].Software Testing,Verification and Relia-bility,2000,10(2):77-109
[24] Harrold M,Gupta R,Soffa M.A Methodology for Controlling the Size of a Test Suite [J].ACM Transactions on Software Engineering and Methodology,1993,2(3):270-285
[25] Chen T,Lau M.A New Heuristic for Test Suite Reduction [J].Information and Software Technology,1998,40(5/6):347-354
[26] Hsu H,Orso A.MINTS:A General Framework and Tool for Supporting Test-suite Minimization [C]∥Proceedings of the International Conference on Software Engineering.2009:419-429
[27] 章晓芳,徐宝文,聂长海,等.一种基于测试需求约简的测试用例集优化方法[J].软件学报,2007,18(4):821-831
[28] 顾庆,唐宝,陈道蓄.一种面向测试需求部分覆盖的测试用例集约简技术[J].计算机学报,2011,34(5):879-888
[29] Chen X,Zhang L,Gu Q,et al.A Test Suite Reduction Approach based on Pairwise Interaction of Requirements [C]∥Procee-dings of the ACM Symposium on Applied Computing.2011:1395-1402
[30] Zhang L,Chen X,Gu Q,et al.CATESR:Change-aware Test Suite Reduction Based on Partial Coverage of Test Requirements [C]∥Proceedings of the International Conference on Software Engineering and Knowledge Engineering.2012:217-224
[31] Rothermel G,Untch R,Chu C.Prioritizing Test Cases for Regression Testing [J].IEEE Transactions on Software Enginee-ring,2001,27(10):929-948
[32] Elbaum S,Malishevsky A,Rothermel G.Test Case Prioritiza-tion:A Family of Empirical Studies [J].IEEE Transactions on Software Engineering,2002,28(2):159-182
[33] Li Z,Harman M,Hierons R.Search Algorithms for Regression Test Case Prioritization [J].IEEE Transactions on Software Engineering,2007,33(4):225-237
[34] Zhang L,Hou S,Guo C,et al.Time-Aware Test-Case Prioritization using Integer Linear Programming [C]∥Proceedings of the International Symposium on Software Testing and Analysis.2009:213-224
[35] Do H,Mirarab S,Tahvildari L,et al.The Effects of Time Constraints on Test Case Prioritization:A Series of Controlled Experiments [J].IEEE Transactions on Software Engineering,2010,36(5):593-617
[36] Do H,Rothermel G.An Empirical Study of Regression Testing Techniques incorporating Context and Lifetime Factors and Improved Cost-benefit Models [C]∥Proceedings of the Symposiumon the Foundations of Software Engineering.2006:141-151
[37] Malishevsky A G,Rothermel G,Elbaum S.Modeling the Cost-benefits Tradeoffs for Regression Testing Techniques [C]∥Proceedings of the International Conference on Software Maintenance.2002:204-213
[38] Do H,Rothermel G.Using Sensitivity Analysis to Create Simplified Economic Models for Regression Testing [C]∥Procee-dings of the International Symposium on Software Testing and Analysis.2008:51-62
[39] Voas J.PIE:A Dynamic Failure-based Technique [J].IEEETransactions on Software Engineering,1992,18(8):352-357
[40] Zhu H,Hall P,May J.Software Unit Test Coverage and Adequacy [J].ACM Computing Survey,1997,29(4):366-427
[41] Ferrante J,Ottenstein K,Warren J.The Program DependenceGraph and its Use in Optimization [J].ACM Transactions on Programming Languages and Systems,1987,9(3):319-349
[42] Weiser M.Program Slicing [J].IEEE Transactions on Software Engineering,1984,10(4):352-357
[43] Bohner S,Arnold R.Software Change Impact Analysis [M].Los Alamitos,1996
[44] Sen K,Marinov D,Agha G.CUTE:a Concolic Unit Testing Engine for C [C]∥Proceedings of the International Symposium on Foundations of Software Engineering.2005:263-272
[45] Godefroid P,Klarlund N,Sen K.DART:Directed AutomatedRandom Testing [C]∥Proceedings of the Conference on Programming Language Design and Implementation.2005:213-223
[46] Tillmann N,Halleux J.Pex-white Box Test Generation for.NET[C]∥Proceedings of the International Conference on Tests and Proofs.2008:134-153
[47] McMinn P.Search-based Software Test Data Generation:a Survey [J].Software Testing,Verification and Reliability,2004,14(2):105-156
[48] 陈翔,顾庆,王子元,等.一种基于粒子群优化的成对组合测试算法框架 [J].软件学报,2011,22(12):2879-2893
[49] Apiwattanapong T,Orso A,Harrold M J.A Differencing Algorithm for Object-oriented Programs [C]∥Proceedings of the International Conference on Automated Software Engineering.2004:2-13
[50] http://www.cc.gatech.edu/aristotle/Tools/jaba.html
[51] Seesing A,Orso A.InsECTJ:A Generic Instrumentation Frame-work for Collecting Dynamic Information within Eclipse [C]∥Proceedings of the eclipse Technology eXchange Workshop at OOPSLA.2005:49-53
[52] http://www.sable.mcgill.ca/soot
[53] http://sofya.unl.edu
[54] http://bitblaze.cs.berkeley.edu
[55] http://babelfish.arc.nasa.gov/trac/jpf
[56] Do H,Elbaum S,Rothermel G.Supporting Controlled Experi-mentation with Testing Techniques:An Infrastructure and its Potential Impact [J].Empirical Software Engineering,2005,10(4):405-435
[57] http://stringtopathgeometry.codeplex.com
[58] http://code.google.com/p/structorian
[59] http://sourceforge.net/projects/j2cstranslator
[60] Sun X,Li B,Zhang S,et al.Using Lattice of Class and Method Dependence for Change Impact Analysis of Object Oriented Programs [C]∥Proceedings of the Symposium on Applied Computing.2011:1444-1449
[61] Inkumsah K,Xie T.Improving Structural Testing of Object-Oriented Programs via Integrating Evolutionary Testing and Symbolic Execution [C]∥Proceedings of the International Confe-rence on Automated Software Engineering.2008:297-306
[62] Baars A,Harman M,Hassoun Y,et al.Symbolic Search-based Testing [C]∥Proceedings of the International Conference on Automated Software Engineering.2011:53-62
[63] Briand LC,Labiche Y,Buist K,et al.Automating Impact Analysis and Regression Test Selection based on UML Designs [C]∥Proceedings of the International Conference on Software Maintenance.2002:252-261
[64] Briand LC,Labiche Y,He S.Automating Regression Test Selection based on UML Designs [J].Journal of Information and Software Technology,2009,51(1):16-30
[65] Wu Y,Offutt J.Maintaining Evolving Component-based Soft-ware with UML [C]∥Proceedings of the European Conference on Software Maintenance and Reengineering.2003:133-142
[66] Yu Y,Jones J,Harrold M.An Empirical Study of the Effects of Test-suite Reduction on Fault Localization [C]∥Proceedings of the International Conference on Software Engineering.2008:201-210
[67] Artzi S,Dolby J,Tip F,et al.Directed Test Generation for Effective Fault Localization [C]∥Proceedings of the International Symposium on Software Testing and Analysis.2010:49-60
[68] Zhang X,Gu Q,Chen X,et al.A Study of Relative Redundancy in Test-suite Reduction while Retaining or Improving Fault Localization Effectiveness [C]∥Proceedings of the Symposium on Applied Computing.2010:2229-2236

No related articles found!
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!