This repository aims to provide a starting template for Python projects containing the most important configuration files (which have to be tailored for your project!) and an initial directory structure separating source code and tests.
I tried to incorporate most "best practices" but in the end, most of the design choices and tools are just personal preferences.
The following tools are used: black, covdefaults, coverage, pre-commit, pylint, pytest, tox
All of the source code goes into the src/<project-name> directory. Here, some dunder files can be found:
- __version__.py: just the version number as string, used by config files
- __init__.py: entry point for program/library
- __main__.py: same as
__init__.pyallowing calls viapython -m <prog>
Testing is done with pytest and tox. All tests go into the test directory. Pytest automatically finds all directories
and modules as well as functions and classes within these matching test_*.py/*_test.py files, Test* classes, and test_*
functions and methods (automatic test discovery).
The conftest.py file is sort of a setup file that can be used to create additional configurations/hooks (small example) and setup code (fixtures) for all tests.
The test environment for pytest is setup with the setup.cfg and/or pyproject.toml file. tox needs extra configuration
which can be found in the deps section of tox.ini. Some projects also use a requirements-tests.txt file that lists
all test dependencies and is also given in the deps section with deps = -rrequirements-tests.txt.
Furthermore, to run pytest from tox, the commands section must be given. Here, additional options for the code coverage report
from the coverage tool are given.
When to use pytest, coverage and tox?
Personally, I mostly use just pytest without coverage to test in my working environment with pytest -svv test or a specific
test module. Before committing, however, it is a good idea to check if your code also runs in different environments, which is where
tox comes in. Running just tox, will test in all environments specified in tox.ini's envlist and may take some
time. Certain environments can be selected with tox -e py37. Note that tox must be able to find a Python interpreter for
each version given in the envlist.
How to provide the Python interpreters for tox.
Unfortunately, this does not directly work with something like a conda environment but you can setup the environments and provide a symlink to a directory which is in your path.
mamba create --name "py311" python=3.11 -y
ln -s /opt/miniforge3/envs/py311/bin/python3.11 ~/bin/python3.11Finally, some handy features of pytest you should be aware of:
- fixtures: common setup for multiple tests (e.g., reading file or database connection)
- parametrize: multiple test cases for single function
- expected fails: testing if the code handles wrong inputs (
with pytest.raises(Exception): ...or@pytest.mark.xfail) - check for test pollution by randomizing the order of tests (pytest-plugin)
Packaging is done with setuptools, which is configured through the pyproject.toml and/or setup.cfg/setup.py files.
pyproject.toml vs.
setup.cfg vs
setup.py
The setup.py file is a Python script and configuration is passed through keyword arguments of setuptools.setup(). This is not recommended due to possible security and parsing issues. The same setup can be accomplished in a declarative style within setup.cfg and setup.py remains mostly empty only calling setuptools.setup().
The pyproject.toml file aims to unify configuration files including various tools like black or pytest. For packaging, it is very similar to setup.cfg. However, pyproject.toml has not been adopted as the default yet and many projects still use setup.cfg to declare the packaging setup. Note that setup.py is not necessary if a pyproject.toml is present.
- minimal build specification to use with setuptools
- configuration of other tools (black, pytest, mypy, ...)
- declarative configuration for setuptools
- metadata: must at least contain name and version
- options: package discovery, dependencies
- additional setup required for
src/layout
- additional setup required for
- options.extras_require: optional dependencies (dev tools, docs, ...)
- options.package_data: inclusion of other, non-Python files (marker files, data, ...)
- alternative:
MANIFEST.in
- alternative:
- options.entry_points: entry point for command line interface
- can also hold configuration of other tools
The package can be installed with pip install . or something like pip install .[dev] to also install additional dependencies specified in setup.cfg's options.extras_require. Pass the -e flag for editable mode, which loads the package from the source directory, i.e., changing the source code does not require a new installation.
Important! CI in private repositories is generally limited (to x minutes of execution time).
...
Tools
Websites