Computer Science ›› 2020, Vol. 47 ›› Issue (9): 24-30.doi: 10.11896/jsjkx.200200116

• Computer Software • Previous Articles     Next Articles

Performance Analysis of Randoop Automated Unit Test Generation Tool for Java

LIU Fang1, HONG Mei2, WANG Xiao1, GUO Dan1, YANG Zheng-hui1, HUANG Xiao-dan1   

  1. 1 School of Computer Science,Sichuan University,Chengdu 610065,China
    2 School of Software Engineering,Sichuan University,Chengdu 610065,China
  • Received:2020-02-26 Published:2020-09-10
  • About author:LIU Fang,born in 1995,postgraduate.Her main research interests include automated unit test generation tools and mutation testing.
    HONG Mei,born in 1963,master’s degree,professor,is a member of China Computer Federation.Her main research interests include software engineering and software automation testing.

Abstract: Automated unit testing is a hotspot in modern software development research.Randoop,an automated unit test cases generation tool,is designed for Java and.NET code,and generates test cases based on feedback guidance.It is widely used in the industry.In order to effectively use Randoop for automated testing,this paper uses empirical software engineering methods toana-lyze the performance characteristics of Randoop through experiments.Four representative Java open source projects are selected to analyze the code coverage of Randoop-generated test cases and the ability to detect mutants,and the relationship of the effectiveness of randoop with the time cost and the source code structure.The experiment find that Randoop can generate valid test cases in a short time.With the increase of generation time,the performance of Randoop generation test cases increases,and tends to be stable when the tests generation time is 120s,with an average mutants coverage of 55.59% and an average mutants kill rate of 28.15%.The performance of the test cases generated by Randoop is related to the structure and complexity of the source code of the tested classes.This paper provides a valuable reference for software testers to effectively use the Randoop tool.

Key words: Automatic generation of test cases, Code coverage, Mutation analysis, Object-oriented automation unit testing, Randoop

CLC Number: 

  • TP311.5
[1] Randoop官网介绍 [EB/OL].[2019-06-12].https://randoop.github.io/ randoop/.
[2] RUEDA U,VOS T E J,PRASETYA I S W B.Unit Testing Tool Competition-Round Three[C]//2015 IEEE/ACM 8th International Workshop on Search-Based Software Testing (SBST).ACM,2015.
[3] RUEDA U,JUST R,GALEOTTI J P,et al.Unit Testing Tool Competition — Round Four[C]//2016 IEEE/ACM 9th International Workshop on Search-Based Software Testing (SBST).Austin,TX,2016:19-28.
[4] PANICHELLA A,MOLINA U R.Java unit testing tool competition— fifth round[C]//10th International Workshop on Search-Based Software Testing.IEEE P,2017:32-38.
[5] RUEDA MOLINA U,KIFETEW F,PANICHELLA A.Javaunit testing tool competition— sixth round[C]//2018 IEEE/ACM 11th International Workshop on Search-Based Software Testing (SBST).Gothenburg,2018:22-29.
[6] SHAMSHIRI S,JUST R,ROJAS J M,et al.Do automatically generated unit tests find real faults? An empirical study of effectiveness and challenges[C]//International Conference on Automated Software Engineering (ASE).IEEE,2015:201-211.
[7] KOCHHAR P S,THUNG F,LO D.Code coverage and testsuite effectiveness:Empirical study with real bugs in large systems[C]//IEEE International Conference on Software Analysis,Evolution and Reengineering.IEEE,2015:560-564.
[8] JALALI D,INOZEMTSEVA L,ERNST M D,et al.Are mu-.tants a valid substitute for real faults in software testing? [C]//ACM Sigsoft International Symposium on Foundations of Software Engineering.ACM,2014:654-665.
[9] Defects4j官网介绍 [EB/OL].[2019-06-12].https://github.com/rjust/defects4j.
[10] PAN N G,ZENG F P,CAO Q.Automatic Generation and Reduction of Random Test Case[J].Journal of Chinese Computer Systems,2011,32(10):2035-2040.
[11] PACHECO C,ERNST M D.Randoop:feedback-directed ran-dom testing for Java[C]//Companion To the,ACM Sigplan Conference on Object-Oriented Programming Systems and Applications Companion.ACM,2007:815-816.
[12] Cobertura官网介绍[EB/OL].[2019-06-12].http://cobertura.github.io/cobertura/.
[13] Major官网介绍[EB/OL].[2019-06-12].http://mutation-testing.org/.
[14] JUST R,SCHWEIGGERT F,KAPFHAMMER G M.MAJOR:An efficient and extensible tool for mutation analysis in a Java compiler[C]//26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011).ACM,2011:612-615.
[15] JALALI D,ERNST M D.Defects4J:a database of existing faults to enable controlled testing studies for Java programs[C]//International Symposium on Software Testing and Analysis.ACM,2014:437-440.
[16] 相关性系数的解释 [EB/OL].[2019-06-13].https://en.wikipedia.org/wiki/Pearson_correlation_coefficient.
[17] ALMASI M M,HEMMATI H,FRASER G,et al.An industrial evaluation of unit test generation:finding real faults in a financial application[C]//2017 IEEE/ACM 39th International Conference on Software Engineering:Software Engineering in Practice Track (ICSE-SEIP).ACM,2017:263-272.
[18] RAMLER R,WINKLER D,SCHMIDT M.Random test casegeneration and manual unit testing:substitute or complement in retrofitting tests for legacy code?[C]//2012 38th EUROMICRO Conference on Software Engineering and Advanced Applications (SEAA).IEEE Computer Society,2012:286-293.
[1] SUN Chang-ai, ZHANG Shou-feng, ZHU Wei-zhong. Mutation Based Fault Localization Technique for BPEL Programs [J]. Computer Science, 2021, 48(1): 301-307.
[2] WANG Lin-xin, WANG Wei-wei, ZHAO Rui-lian and LI Zheng. MBFL with Statement-oriented Mutant Reduction Strategy [J]. Computer Science, 2017, 44(11): 175-180.
[3] GONG Pei, GENG Chu-yao, GUO Jun-xia and ZHAO Rui-lian. Dynamic Mutation Execution Strategy for Mutation-based Fault Localization [J]. Computer Science, 2016, 43(2): 199-203.
[4] . Method of Android Applications Permission Detection Based on Static Dataflow Analysis [J]. Computer Science, 2012, 39(Z11): 16-18.
Viewed
Full text


Abstract

Cited

  Shared   
  Discussed   
No Suggested Reading articles found!