计算机科学 ›› 2020, Vol. 47 ›› Issue (9): 24-30.doi: 10.11896/jsjkx.200200116

• 计算机软件* 上一篇    下一篇

面向Java的Randoop自动化单元测试生成工具性能分析

刘芳1, 洪玫2, 王潇1, 郭丹1, 杨正卉1, 黄小丹1   

  1. 1 四川大学计算机学院 成都610065
    2 四川大学软件学院 成都610065
  • 收稿日期:2020-02-26 发布日期:2020-09-10
  • 通讯作者: 洪玫(hongmei@scu.edu.cn)
  • 作者简介:756493201@qq.com

Performance Analysis of Randoop Automated Unit Test Generation Tool for Java

LIU Fang1, HONG Mei2, WANG Xiao1, GUO Dan1, YANG Zheng-hui1, HUANG Xiao-dan1   

  1. 1 School of Computer Science,Sichuan University,Chengdu 610065,China
    2 School of Software Engineering,Sichuan University,Chengdu 610065,China
  • Received:2020-02-26 Published:2020-09-10
  • About author:LIU Fang,born in 1995,postgraduate.Her main research interests include automated unit test generation tools and mutation testing.
    HONG Mei,born in 1963,master’s degree,professor,is a member of China Computer Federation.Her main research interests include software engineering and software automation testing.

摘要: 自动化单元测试是现代软件开发研究的热点。Randoop自动化单元测试用例生成工具针对Java和.NET代码,基于反馈指导随机生成测试用例,在业界应用广泛。为了有效使用Randoop进行自动化测试,采用经验软件工程的方法,通过实验分析Randoop的性能特点;实验选取4个有代表性的Java开源项目,分析Randoop生成测试用例的代码覆盖率和变异体检测能力,以及它们与时间成本和被测类源代码的代码结构之间的关系。实验发现,Randoop可以在短时间内生成有效的测试用例,生成测试用例的性能随时间增加而上升,并在测试用例生成时间为120s时趋于稳定,其平均变异体覆盖率达55.59%,且平均变异体杀死率为28.15%。Randoop生成的测试用例的性能与被测类源代码的代码结构和复杂度存在一定的关系。该研究为软件测试人员有效使用Randoop工具提供了有价值的参考。

关键词: Randoop, 变异分析, 测试用例自动生成, 代码覆盖率, 面向对象自动化单元测试

Abstract: Automated unit testing is a hotspot in modern software development research.Randoop,an automated unit test cases generation tool,is designed for Java and.NET code,and generates test cases based on feedback guidance.It is widely used in the industry.In order to effectively use Randoop for automated testing,this paper uses empirical software engineering methods toana-lyze the performance characteristics of Randoop through experiments.Four representative Java open source projects are selected to analyze the code coverage of Randoop-generated test cases and the ability to detect mutants,and the relationship of the effectiveness of randoop with the time cost and the source code structure.The experiment find that Randoop can generate valid test cases in a short time.With the increase of generation time,the performance of Randoop generation test cases increases,and tends to be stable when the tests generation time is 120s,with an average mutants coverage of 55.59% and an average mutants kill rate of 28.15%.The performance of the test cases generated by Randoop is related to the structure and complexity of the source code of the tested classes.This paper provides a valuable reference for software testers to effectively use the Randoop tool.

Key words: Automatic generation of test cases, Code coverage, Mutation analysis, Object-oriented automation unit testing, Randoop

中图分类号: 

  • TP311.5
[1] Randoop官网介绍 [EB/OL].[2019-06-12].https://randoop.github.io/ randoop/.
[2] RUEDA U,VOS T E J,PRASETYA I S W B.Unit Testing Tool Competition-Round Three[C]//2015 IEEE/ACM 8th International Workshop on Search-Based Software Testing (SBST).ACM,2015.
[3] RUEDA U,JUST R,GALEOTTI J P,et al.Unit Testing Tool Competition — Round Four[C]//2016 IEEE/ACM 9th International Workshop on Search-Based Software Testing (SBST).Austin,TX,2016:19-28.
[4] PANICHELLA A,MOLINA U R.Java unit testing tool competition— fifth round[C]//10th International Workshop on Search-Based Software Testing.IEEE P,2017:32-38.
[5] RUEDA MOLINA U,KIFETEW F,PANICHELLA A.Javaunit testing tool competition— sixth round[C]//2018 IEEE/ACM 11th International Workshop on Search-Based Software Testing (SBST).Gothenburg,2018:22-29.
[6] SHAMSHIRI S,JUST R,ROJAS J M,et al.Do automatically generated unit tests find real faults? An empirical study of effectiveness and challenges[C]//International Conference on Automated Software Engineering (ASE).IEEE,2015:201-211.
[7] KOCHHAR P S,THUNG F,LO D.Code coverage and testsuite effectiveness:Empirical study with real bugs in large systems[C]//IEEE International Conference on Software Analysis,Evolution and Reengineering.IEEE,2015:560-564.
[8] JALALI D,INOZEMTSEVA L,ERNST M D,et al.Are mu-.tants a valid substitute for real faults in software testing? [C]//ACM Sigsoft International Symposium on Foundations of Software Engineering.ACM,2014:654-665.
[9] Defects4j官网介绍 [EB/OL].[2019-06-12].https://github.com/rjust/defects4j.
[10] PAN N G,ZENG F P,CAO Q.Automatic Generation and Reduction of Random Test Case[J].Journal of Chinese Computer Systems,2011,32(10):2035-2040.
[11] PACHECO C,ERNST M D.Randoop:feedback-directed ran-dom testing for Java[C]//Companion To the,ACM Sigplan Conference on Object-Oriented Programming Systems and Applications Companion.ACM,2007:815-816.
[12] Cobertura官网介绍[EB/OL].[2019-06-12].http://cobertura.github.io/cobertura/.
[13] Major官网介绍[EB/OL].[2019-06-12].http://mutation-testing.org/.
[14] JUST R,SCHWEIGGERT F,KAPFHAMMER G M.MAJOR:An efficient and extensible tool for mutation analysis in a Java compiler[C]//26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011).ACM,2011:612-615.
[15] JALALI D,ERNST M D.Defects4J:a database of existing faults to enable controlled testing studies for Java programs[C]//International Symposium on Software Testing and Analysis.ACM,2014:437-440.
[16] 相关性系数的解释 [EB/OL].[2019-06-13].https://en.wikipedia.org/wiki/Pearson_correlation_coefficient.
[17] ALMASI M M,HEMMATI H,FRASER G,et al.An industrial evaluation of unit test generation:finding real faults in a financial application[C]//2017 IEEE/ACM 39th International Conference on Software Engineering:Software Engineering in Practice Track (ICSE-SEIP).ACM,2017:263-272.
[18] RAMLER R,WINKLER D,SCHMIDT M.Random test casegeneration and manual unit testing:substitute or complement in retrofitting tests for legacy code?[C]//2012 38th EUROMICRO Conference on Software Engineering and Advanced Applications (SEAA).IEEE Computer Society,2012:286-293.
[1] 孙昌爱, 张守峰, 朱维忠.
一种基于变异分析的BPEL程序故障定位技术
Mutation Based Fault Localization Technique for BPEL Programs
计算机科学, 2021, 48(1): 301-307. https://doi.org/10.11896/jsjkx.200900051
[2] 王林鑫,王微微,赵瑞莲,李征.
面向语句的MBFL变异体约减策略
MBFL with Statement-oriented Mutant Reduction Strategy
计算机科学, 2017, 44(11): 175-180. https://doi.org/10.11896/j.issn.1002-137X.2017.11.026
[3] 龚沛,耿楚瑶,郭俊霞,赵瑞莲.
针对基于变异错误定位的一种动态变异执行策略
Dynamic Mutation Execution Strategy for Mutation-based Fault Localization
计算机科学, 2016, 43(2): 199-203. https://doi.org/10.11896/j.issn.1002-137X.2016.02.043
[4] 谭鑫,彭耀鹏,杨帅,郑炜.
基于SPEA2+SDE算法的测试用例自动生成技术研究
Automated Test Case Generation Based on SPEA2+SDE
计算机科学, 2015, 42(Z11): 450-453.
[5] 姜鹏,陈鑫,李宣东.
一种面向信息物理融合系统安全攸关场景的测试用例自动生成方法
Method to Automatic Testcase Generation toward Safety Critical Scenarios of Cyber-physical Systems
计算机科学, 2014, 41(11): 124-127. https://doi.org/10.11896/j.issn.1002-137X.2014.11.024
[6] 杨博 唐祝寿 朱浩谨 沈备军 林九川.
基于静态数据流分析的Android应用权限检测方法
Method of Android Applications Permission Detection Based on Static Dataflow Analysis
计算机科学, 2012, 39(Z11): 16-18.
[7] 马春燕,朱怡安,陆伟.
Web服务自动化测试技术
Automatic Test for Web Services
计算机科学, 2012, 39(2): 170-174.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!