Random vs GA Study (view page)
To support replication, this web page contains the experimental material used for the paper titled "Random or Genetic Algorithm Search for Object-Oriented Test Suite Generation?", accepted for publication at GECCO 2015.
View / Download GECCO 2015 Slides.
View / Download PDF of the paper.
Achieving high structural coverage is an important aim in software testing. Several search-based techniques have proved successful at automatically generating tests that achieve high coverage. However, despite the well-established arguments behind using evolutionary search algorithms (e.g., genetic algorithms) in preference to random search, it remains an open question whether the benefits can actually be observed in practice when generating unit test suites for object-oriented classes. In this paper, we report an empirical study on the effects of using a genetic algorithm (GA) to generate test suites over generating test suites incrementally with random search, by applying the EvoSuite unit test suite generator to 1,000 classes randomly selected from the SF110 corpus of open source projects. Surprisingly, the results show little difference between the coverage achieved by test suites generated with evolutionary search compared to those generated using random search. A detailed analysis reveals that the genetic algorithm covers more branches of the type where standard fitness functions provide guidance. In practice, however, we observed that the vast majority of branches in the analyzed projects provide no such guidance.
List of the 1000 classes taken from SF110.
EvoSuite along with the implementation of both GA and Random-Search algorithms used in this study are open source and available on Github. For more information about EvoSuite and the documentation of the tool, please visit evosuite.org.
@inproceedings{shamshiri2015random,
author = "Shamshiri, Sina and Rojas, Jos\'{e} Miguel and Fraser, Gordon and McMinn, Phil",
title = "Random or Genetic Algorithm Search for Object-Oriented Test Suite Generation?",
booktitle = "Genetic and Evolutionary Computation Conference (GECCO 2015)",
pages = "1367--1374",
year = "2015",
publisher = "ACM"
}
Reference: Sina Shamshiri, José Miguel Rojas, Gordon Fraser and Phil McMinn. Random or Genetic Algorithm Search for Object-Oriented Test Suite Generation? Genetic and Evolutionary Computation Conference (GECCO 2015), pp. 1367–1374, 2015.