di-mi-ta/python-project-template

Python ML Project Template

Python Project Template

A Python project template using conda to manage virtual environments and pip-tools for dependency management.

The template uses pre-commit hooks to run code style and code quality checks for each Git commit and provides several invoke commands for common development tasks.

The following tools are used for project development:

• black for code formatting,
• flake8 for code style checks,
• mypy for type checking,
• pytest for test execution.

Setup

Setup on a local machine

To set up the development environment on a single machine run the following commands:

# setup conda environment
conda env create -f environment.yml

# activate conda environment
conda activate python-project-template

# install dependencies
invoke pip-install

# install pre-commit hook
invoke precommit-install

This sequence of commands creates the project environment using conda, compiles the python dependencies specified in the requirement files and installs the dependencies into the newly created environment. Afterwards the pre-commit hooks are installed and executed automatically upon each Git commit.

Setup using Remote Interpreter

If a remote interpreter is employed (e.g. by using a PyCharm SSH interpreter), the project environment can be installed on the remote machine using the following commands:

# create conda environment
conda env create -f environment.yml

# activate environment
conda activate python-project-template

# install dependencies
invoke pip-install

To run the Git pre-commit hooks in a local environment, the following commands can be used locally:

# create and activate build tools environment
conda create -n build-tools python=3.9
conda activate build-tools
# install build tool dependencies
pip install pre-commit
pip install invoke

Install project pre-commit hooks using:

invoke precommit-install

Usage

The project template provides invoke tasks to execute common development tasks.

Execute code formatting, style and type checks using:

invoke code-check
# or
invoke cc

Run the unit tests using:

invoke unit-test
# or
invoke ut

Run the integration tests using:

invoke integration-test
# or
invoke it

The complete test suite can be executed by running:

invoke test

To create a distribution package use the following command:

invoke build

After the build process has finished the sdist and wheel packages are available in the /dist folder.