The idea of this repository is to test the performance of Gradle and other technologies in different scenarios (for unit and integration tests)! The goal is to provide reproducable, reliable and accurate results and metrics that can be used to make better decitions.
WARNING: Currently the results are NOT reliable when comparing between "ten_tests" and non "ten_tests" results! See reddit discussion
- Clone this repository
git clone https://github.com/nbrugger-tgm/gradle-testing-performance
cd gradle-testing-performance
./gradlew run
(this will take some time (~10 mins atm) since - well it will execute ~200k test combinations)- open the json in
build/report.json
it includes all results and also calculated metrics
This is what is planned this is WIP! Feel free to contribute new scenarios (especially the ones listed) See later on how to contribute and create scenarios!
Scenarios
- Unit tests (
./gradlew :unit-only:run
) - Integration tests (with pre-running DB)
- Integration tests (with testcontainers)
- Integration tests (with liquibase provisioning using a pre-running DB)
- small changeset (~10 migrations)
- medium changeset (~50 migrations)
- large changeset (~200 migrations)
- Integration tests (with liquibase provisioning using testcontainers)
- small changeset (~10 migrations)
- Mockito tests
- Micronaut tests
- Small context (20 services/components/beans) all used
- Medium context (70 services/components/beans) all used
- Medium context (70 services/components/beans) 20 used
- Large context (400 services/components/beans) all used
- Large context (400 services/components/beans) 20 used
Factors
- Test class count
- 1
- 50
- 200
- Test method count
- 1
- 10
- Gradle submodule count
- 1
- 5
- 20
This is done in order to better understand how the performance is affected by different factors and how they interact with each other. For example, Does testcontainers starts a new container for every gradle submodule or only once for all of them?
This also causes a very large number of tests and very high execution time, but it is worth it! It will result in a very large data mass that can be analyzed and used to make better decisions!