Switch to pytest
Closed this issue · 3 comments
The tests are now performed using nose inside tox envs. The problem is that nose is not maintained anymore. Pytest also offers nice testing features and should be adopted instead.
The goal of this issue is to track the work required and discussion related to the switch to pytest.
It is also a move I wanted to make for some point.
I had minor issues with nose, the wrapping of coverage is not complete, and the error presentation was better in pytest. I did not know for the non-maintenance but it also adds to the list.
Requirements
For such a change to be merged, I would require to keep the same features level:
Currently, jenkins build expects output to be coverage.xml and nosetests.xml.
Output in xml jenkins compatible is required. Of course the output file will change and it would be good to have it clear in the commit message, what changed when switching to pytest to upgrade the build configuration accordingly.
I expect cover (or any directory) to contain html coverage output as its really convenient when developing.
Verify that coverage and number of executed tests are the same (or greater!), I have options in setup.cfg for nosetests to try to detect as many tests as possible. There might be options to put for pytest to also detect all tests, like doctests for example.
If all the dependencies are installed, running pytest without options outside of tox should run the tests with the correct configuration.
By default, it tries to run tests into the .tox directory for example, which is bad...
Also integration tests should be ignored by default.
After a build, produced results are correctly ignored by git.
Nice features to have
Another switch that could be interesting, is also look into pytest to run pylint, pep8 with pytest wrapping instead of setup.py.
I don't care about an output file, the format is parsed from standard output by jenkins. Message format should be kept the same to be recognized.
In tox, add an option for pytest to say for example xml outputs to be in python_version_coverage.xml and same for tests.
It would also be a nice regardless of pytest/nose, its just that if someone looks into the test system, it could be the good time to add it.
Agreed with all the above comment in general.
For doctest, one can use --doctest-modules. For lint, pep8, there's indeed dedicated python packages extending pytest with corresponding options (--lint, --pep8) but also for flake8.
in setup.cfg, the idea would be to add [tool:pytest] section as described in the pytest documentation. There one can use addopts to set the required options.
It also seems that pytest can work with the tox.ini file.
I can have a look at that.