/python-standard

Primary LanguageJinja

Python Project Copier Template

This template produces a Python project. The most "interesting" choices it makes are:

  • Use nox to run various code checkers.
  • Use virtue to run the unit tests.
  • Use pyproject.toml-based setuptools to build a package.
  • 100% code coverage on every unit test run.

All other tooling choices are either vanilla, or a consequence of using these tools.

Quickstart

Initialize a project using copier and nox:

$ copier gh:moshez/python-standard.git <TARGET_DIRECTORY>
$ cd TARGET_DIRECTORY
$ nox -e refresh_deps

Note: Running nox -e refresh_deps produces requirements-*.txt files, which are essential to the other sessions in nox.

Working With the Project

Testing

The configuration assumes you care about testing. It uses virtue to run the tests.

Since virtue assumes tests are importable code, put tests under your project: PROJECT_NAME/tests/.... An example test is already included, which checks the version number.

The virtue runner is only a runner: it does not have test cases or assertions built in. In order to write test cases, use unittest.TestCase. The hamcrest library is already included in the test dependencies, and can be used for writing assertions.

Managing dependencies

Add unpinned dependencies to pyproject.toml. After doing that, run

$ nox -e refresh_deps

This will recreate the pinned files. Note that all other sessions only use the pinned files.

Pinning is useful: a nox session will not fail just because a new Python package has been uploaded to PyPI. However, this does require periodic repinning.

There are services that will automatically suggest a patch for repinning. If you do not use those, you can create a branch yourself:

$ git checkout -b update-dependencies
$ nox -e refresh_deps

Create a Pull Request/Merge Request from the branch as you would for any other branch.

Releasing

The version number is managed in pyproject.toml. There is no internal tool used to bump the version number.

Running nox -e build or nox will produce a wheel. This wheel can be uploaded to a Python packaging index of your choice.

Documenting

The documentation is set up to use sphinx. It automatically ignores Jupyter-created temporary files, to allow including notebooks in a natural way.

It suggests a quick start guide, followed by an API reference. Adding new sections is sometimes a good idea.

Continuous Integration

The continuous integration configured is GitHub Actions. This is a relatively simple configuration.

It assumes that each interpreter, on each operating system, achieves full coverage. It also runs the tests only on Ubuntu.

Guiding principles

Note: This section explains Moshe's personal opinions.

The choice of tooling is guided by one principle: maximize ergonomics, minimize magic.

Even after years of using tox, I sometimes find it non-trivial to do stuff. How do you minimize copy paste while setting up each command exactly right? With nox, the syntax, and semantics, are Python code.

The pytest runner has a lot of magic. It finds fixtures, has complicated test finding algorithms, and, most famously, fancy assert rewrite logic.

The hamcrest library is an explicit "fancy assert" library. It is based on regular Python functions and Python objects, and allows explicit control on how to make the assertions useful.

With that out of the way, virtue focuses on running tests, not finding them. Tests have to be Python modules, with all the regular semantics that comes with that.

For "fixtures", it is possible to define functions in test-helper modules explicitly. With unittest.TestCase's support for setUp and addCleanup, it is possible to do setup and teardown.

One loss in ergonomics is that now even simple tests need to be in a TestCase. The benefit in reducing the magic is one that offsets it for me.

Detailed Python Ecosystem Stack

  • Uses nox as its checker runner.
  • For packaging:
  • For tests:
    • virtue as test runner.
    • hamcrest as assertion library.
    • coverage for coverage testing. Coverage is set to fail below 100%. Explicitly mark untested code with # pragma: no coverage.
  • For static checking:
    • black for automatically fixable errors.
    • flake8 for other issues.
    • mypy for type checking.
  • For documentation:
  • For continuous integration: