Make testing tox-centric
scasagrande opened this issue · 4 comments
Over the past few years, the Python community seems to be using a tox-centric workflow when it comes to testing. This helps to ensure that the installation process is functional, that the installed code is tested against, and a consistency between testing locally and in CI.
An example of how this is done can be found implemented in the pylint source code: https://github.com/PyCQA/pylint
More reading can be found here: https://blog.ionelmc.ro/2014/05/25/python-packaging/
Tox is definitely neat for locally testing against different Python versions, and for the reasons you outline.
What I personally don't like about tox is that it does not mesh well (yet?) with a conda-based workflow, that's why I have not moved to tox in my Python projects. I remember from previous investigations that the limitation was mainly that the way tox pulls in different interpreter versions into separate environment is not really foreseen in Anaconda.
For me personally, testing locally against the present Python (using pytest), and testing other versions in CI seems to be good enough so far.
Agreed, tox is great for testing multiple versions and installation. I have started to adopt nox (https://nox.thea.codes/en/stable/), which is similar but uses a python setup file.
https://medium.com/@cjolowicz/hypermodern-python-d44485d9d769
This is the reading that made me test out nox and start using it. Also interesting because there's a whole section on poetry, might be useful for #287
@bilderbuchi yeah I should clarify, executing pytest like we do now will continue to work, the idea however is to elevate tox testing to be the authority for pass/fail. One of the first big uses for this is to help my work in #279 . With numpy becoming an optional dep, the ability to create and test different environments is key.
I think what's left is to update the development docs.