Template for a python library or module
Table of contents generated with markdown-toc --no-firsth1 .\README.md
Replace this section with "About project name"
This provides an example for a module that can be installed with pip for use in another module or script.
In here you'll find some minimal GitHub workflows configurations, code and tests to seed a python project.
The template is compatible with python 2 and 3 for easy adoption.
Do we have significant release testing focused on this repository? If yes, use git-flow. Otherwise use github-flow.
This repository follows git-flow. Released code is on the master branch. PRs are typically merged into develop. For release code, crate a release/<description> branch from develop and open a PR into master.
This way we can make bugfixes to the code that is prepared for release while continuing new feature development that can merge into develop. Tagging each release candidate on the release/ branch and each releas on master allows us to go back and make patches where needed if we are not ready to make a new minor release based on develop.
This repository follows github-flow. Released code is kept on the main branch. New features are developed on seperate branches, but these branches should be short lived. PRs are created from the feature branch into the main branch. Significant test coverage provides confidence that newly merged features do not break the build. Additionally, downstream dependencies are able to pin versions of this repository allowing for control over what gets released in customer applications.
Setup pre-commit to run before each commit.
pip install pre-commit
pre-commit installpre-commit performs the checks and actions defined in .pre-commit-config.yaml.
It's also a good idea to run the unit tests before commiting. See Running tests.
It's encouraged to create a branch for work in progress. You can use the format <user name>/<description> for short term or experimental work. Pushing the branch may inspire others over invite feedback. Since we have many people working remotely, it also helps communicate active effort and avoid duplicative work. Since it's your own branch, feel free to rebase force push anything or anything else you want to do with that branch.
<type>/<description> Is used for shared development branches and development that's intended to merge back into develop. <type> Comes from the types listed in the .gitmessage file. On these branches, use the .gitmessage template to create informative commit messages. These can help speed along review. And links back to the Jira ticket help others understand the context without needing to duplicate the what's written in the ticket.
Please, organize your commits into small self-contained commits with descriptive messages (see .gitmessage). This helps reviewers quickly give relevant feedback, ensuring the PR is approved as quickly as it can be.
Use merge commits to merge in changes from the PR in order to maintain history and maintain the context of review comments. This can also help if we need to run git-bisect, sine the commits will be smaller, it is easier to pinpoint the change that resulted in a particular bug.
The easiest way to run tests is with tox.
Simply run tox and it will run all of the tests for each environment both supported by the module and installed on the developer's machine.
In the below example tests passed for python 2.7 and 3.9. Python3.8 was not installed and therefore those tests were not run.
py27: commands succeeded
ERROR: py38: InterpreterNotFound: python3.8
py39: commands succeeded
report: commands succeeded
Tox can take a little while to run since it sets up each environment for each run of the tests. PyTest is useful to get a shorter feedback loop.
Follow these steps to run tests using pytest, replacing <ver> with the appropriate python version.
- Set up a virtual environment
python<ver> -m virtualenv env - Activate the virtual environment
- Linux/Mac
. env/bin/activate - Windows
.\env\Scripts\Activate
- Linux/Mac
- Install development dependencies
pip install -r requirements.dev.txt - Install this repo in editable mode
pip install -e . - Run the tests
pytest
This results in the following output:
3 passed in 0.07 seconds
After the virtual environment is set up (steps 1-4 above), calling pytest is all you need.
It can be useful to run a watch window in a terminal while working on the project to keep the tests running on each save. watch -c pytest --color=yes will keep the color highlighting and the tests running in a loop.
Define the direct dependencies for the library in the requirements.txt but avoid pinning the specific version. This can result in dependency conflicts in downstream consumers of the library. In general, applications should pin dependencies in the requirements.txt file, whil libraries should not. For testing, though, it may still be useful to pin dependencies for repeatable testing.
https://modelpredict.com/wht-requirements-txt-is-not-enough
Periodically update the dependencies as follows:
python2 -m virtualenv env
. env/bin/activate
pip install -r requirements.txt
pip install -r requirements.dev.in
pip freeze > requirements.dev.txtIf python 2 compatability is desired, make sure to perform the above steps using python 2.
New repositories may want to consider using pip-compile: https://github.com/jazzband/pip-tools#updating-requirements
It uses setup.cfg with a minimal setup.py to minimimize redundancy. While a lot of customation can be performed in setup.py, it is not recommended.
https://packaging.python.org/tutorials/packaging-projects/#configuring-metadata
Tests also run using GitHub actions: https://github.com/features/actions