Select a portion of the collected tests, so you can run differents parts of your test suite in differents instances to scale horizontally.
Suppose you have a big, slow test suite, but you can trigger several CI workers to run different portions of it, in a sake lazy/simple way to parallelize it.
A basic, obvious way to do that is to explictily collect from different directories/modules
- worker1:
pytest tests/a
(100 tests, ~4 minutes to finish) - worker2:
pytest tests/b
(20 tests, ~1 minute to finish) - worker3:
pytest tests/c tests/d
(30 tests, ~1 minute to finish)
The problem is that directory tests/a may have a lot more tests that tests/c plus test/d,
so worker1
takes a lot more to finish.
With pytest-portion
you can still split the tests in different instances, but letting
the extension makes the selection in a more balanced way.
- worker1:
pytest --portion 1/3 tests
(first 50 tests, ~2 minutes) - worker2:
pytest --portion 2/3 tests
(next 50 tests, ~2 minutes) - worker3:
pytest --portion 3/3 tests
(last 50 tests, ~2 minutes)
In this case, the tests of all the directories are collected, but only a third (a different one!) of them will be actually executed on each worker.
Note this balance is by number of tests, so if there is very slow tests in a particular portion, the duration may not be expected.
For a fine tuning, you could pass the portion in a more explicit way:
- worker1:
pytest --portion 0:0.5 tests
(first half, 1st to 75th test) - worker2:
pytest --portion 0.5:0.8 tests
(next 30%, from 76th to 125º) - worker3:
pytest --portion 0.8:1 tests
(last 20%)
You can install "pytest-portion" via pip from PyPI:
$ pip install pytest-portion
Pass --portion <i/n>
where:
n
is the number of portionsi
is the i-th portion to select (1 <= i <= n
)
Note
If the number of tests collected is not divisible by n, the last portion will contain the rest. For instance, if you have test_1, test_2 and test_3, --portion 1/2 will run the first one, and --portion 2/2 the last 2.
Alternatively --portion start:end
where:
start
andend
are the coefficient (between 0 and 1) that represent the segment of the collected tests
to select.
Contributions are very welcome. Please ensure the coverage at least stays the same before you submit a pull request.
Distributed under the terms of the BSD-3 license, "pytest-portion" is free and open source software
If you encounter any problems, please file an issue along with a detailed description.
- Thanks to ShipHero for give me some time to develop this.
- I used cookiecutter along with @hackebrot's cookiecutter-pytest-plugin template for the boilerplate code of this package. Thanks!