cookiecutter-numengo

Cookiecutter template for a python library for complete setup configuration from cookiecutter .

Notes:

This is a fork from the excellent https://github.com/ionelmc/cookiecutter-pylibrary Most of the code and documentation from Ionel Cristian Maries "https://blog.ionelmc.ro" with adaptations from Cedric ROMAN
More options are included in the cookie cutter to allow a complete project setup from a cookie cutter template.

Therefore, it s more intended for people using an input config file rather than command line interface. User can define in the configuration file advanced settings such as python environments and dependencies to install, keywords, etc... Again, the idea is to be able to provide a project configuration file which will allow to generate completely the setup.py file, and ideally the cookiecutter can be used to regenerate the project each time the configuration change to update all environments. Further code generation can be achieved with the command line tools.

It will automatically create the project structure and default files, and set up all environments and create a Visual Studio

project .pyproj properly configured with all Tox environments detected.

It installs command line tools to recreate visual studio projects, create new files according to a template, clean

code, run tests, build distributions, etc...

This is largely designed to address this blog post about packaging python libraries.
- ... and it will save you from packaging pitfalls.
There's a bare library using this template (if you're curious about the final result): https://github.com/ionelmc/python-nameless.

Table of Contents

Features
Requirements
Usage and options
- Developing the project
- Releasing the project
  - Version management
  - Building and uploading
Changelog
Questions & answers
Not Exactly What You Want?

Features

This is an "all inclusive" sort of template.

Choice of various licenses.
Tox for managing test environments for Python 2.7, 3.3, PyPy etc.
Pytest or Nose for testing Python 2.7, 3.3, PyPy etc.
Optional support for creating a tests matrix out of dependencies and python versions.
Travis-CI and AppVeyor for continuous testing.
Coveralls or Codecov for coverage tracking (using Tox).
Documentation with Sphinx, ready for ReadTheDocs.
Configurations for:
- isort
- bumpversion
Support for C extensions (including coverage measurement for the C code). See c_extension_support.
Packaging and code quality checks. This template comes with a tox environment (check) that will:
- Check if your README.rst is valid.
- Check if the MANIFEST.in has any issues.
- Run flake8 (a combo of PEP8, pyflakes and McCabe checks) or pylama

Requirements

Projects using this template have these minimal dependencies:

Cookiecutter - just for creating the project
Tox - for running the tests
Setuptools - for building the package, wheels etc. Now-days Setuptools is widely available, it shouldn't pose a problem :)

To get quickly started on a new system, just install setuptools and then install pip. That's the bare minimum to required install Tox and Cookiecutter. To install them, just run this in your shell or command prompt:

pip install tox cookiecutter

Usage and options

This template is more involved than the regular cookiecutter-pypackage.

First generate your project:

cookiecutter gh:numengo/cc-py-setup

You will be asked for these fields:

Template variable	Default	Description
`full_name`	"John Doe"	Main author of this library or application (used in `AUTHORS.rst` and `setup.py`). Can be set in your `~/.cookiecutterrc` config file.
`email`	"johndoe@gmail.com"	Contact email of the author (used in `AUTHORS.rst` and `setup.py`). Can be set in your `~/.cookiecutterrc` config file.
`website`	"https://blog.johndoe.com"	Website of the author (used in `AUTHORS.rst`). Can be set in your `~/.cookiecutterrc` config file.
`github_username`	"johndoe"	GitHub user name of this project (used for GitHub link). Can be set in your `~/.cookiecutterrc` config file.
`project_name`	"Nameless"	Verbose project name, used in headings (docs, readme, etc).
`repo_name`	"python-nameless"	Repository name on GitHub (and project's root directory name).
`package_name`	"nameless"	Python package name (whatever you would import).
`distribution_name`	"nameless"	PyPI distribution name (what you would `pip install`).
`short_description`	"An example package [...]"	One line description of the project (used in `README.rst` and `setup.py`).
`release_date`	"today"	Release date of the project (ISO 8601 format) default to today (used in `CHANGELOG.rst`).
`year`	"now"	Copyright year (used in Sphinx `conf.py`).
`version`	"0.1.0"	Release version (see `.bumpversion.cfg` and in Sphinx `conf.py`).
`c_extension_support`	"no"	Support C extensions (will slighly change the outputted `setup.py`). Available options: `"yes"` - to generate a Python C extension `"cffi"` - to generate CFFI bindings against a C library `"cython"` - to generate a Cython extension
`c_extension_optional`	"no"	Make C extensions optional (will allow your package to install even if extensions can't be compiled)
`test_matrix_configurator`	"no"	Enable the test matrix generator script. If you don't have a huge number of test environments then probably you don't need this.
`test_matrix_separate_coverage`	"no"	Enable this to have a separate env for measuring coverage. Indicated if you want to run doctests or collect tests from `src` with pytest. Note that `test_matrix_separate_coverage == 'no'` only works if you also have `test_matrix_configurator == 'no'`.
`test_runner`	"pytest"	Test runner to use. Available options: `pytest` or `nose`.
`linter`	"flake8"	Linter to use for `tox -e check`. Available options: `flake8` or `pylama`
`command_line_interface`	"plain"	Option to enable a CLI (a bin/executable file). Available options: `plain` - a very simple command. `argparse` - a command implemented with `argparse`. `click` - a command implemented with click - which you can use to build more complex commands. `no` - no CLI at all.
`command_line_interface_bin_name`	"nameless"	Name of the CLI bin/executable file (set the console script name in `setup.py`).
`license`	"BSD license"	License to use. Available options: BSD license MIT license ISC license Apache Software License 2.0 What license to pick? https://choosealicense.com/
`coveralls`	"no"	Enable pushing coverage data to Coveralls and add badge in `README.rst`.
`codecov`	"yes"	Enable pushing coverage data to Codecov and add badge in `README.rst`. Note: Doesn't support pushing C extension coverage yet.
`landscape`	"no"	Add a Landscape badge in `README.rst`.
`scrutinizer`	"no"	Add a Scrutinizer badge in `README.rst`.
`codacy`	"no"	Add a Codacy badge in `README.rst`. Note: After importing the project in Codacy, find the hexadecimal project ID from settings and replace it in badge URL
`codeclimate`	"no"	Add a CodeClimate badge in `README.rst`.
`sphinx_theme`	"sphinx-rtd-theme"	What Sphinx theme to use. Suggested alternative: sphinx-py3doc-enhanced-theme <https://pypi.python.org/pypi/sphinx_py3doc_enhanced_theme> for a responsive theme based on the Python 3 documentation.
`sphinx_doctest`	"no"	Set to `"yes"` if you want to enable doctesting in the docs environment. Works best with `test_matrix_separate_coverage == 'no'`. Read more about doctest support in Sphinx.
`travis`	"yes"	If you want the Travis-CI badge and configuration.
`appveyor`	"yes"	If you want the AppVeyor badge and configuration.
`requiresio`	"yes"	If you want the requires.io badge and configuration.

The testing (tox.ini and .travis.yml) configuration is generated from templates. For your convenience there's an initial bootstrap tox.ini, to get the initial generation going just run:

tox

You can later regenerate tox.ini and .travis.yml by running (if you enabled the test_matrix_configurator option):

tox -e bootstrap

After this you can create the initial repository (make sure you create an empty Github project):

git init .
git add .
git commit -m "Initial skel."
git remote add origin git@github.com:ionelmc/python-nameless.git
git push -u origin master

Then:

Enable the repository in your Travis CI account.
Enable the repository in your Coveralls account.
Add the repo to your ReadTheDocs account + turn on the ReadTheDocs service hook. Don't forget to enable virtualenv and specify docs/requirements.txt as the requirements file in Advanced Settings.

Developing the project

To run all the tests, just run:

tox

To see all the tox environments:

tox -l

To only build the docs:

tox -e docs

To build and verify that the built package is proper and other code QA checks:

tox -e check

Releasing the project

Before releasing your package on PyPI you should have all the tox environments passing.

Version management

This template provides a basic bumpversion configuration. It's as simple as running:

bumpversion patch to increase version from 1.0.0 to 1.0.1.
bumpversion minor to increase version from 1.0.0 to 1.1.0.
bumpversion major to increase version from 1.0.0 to 2.0.0.

You should read Semantic Versioning 2.0.0 before bumping versions.

Building and uploading

Before building dists make sure you got a clean build area:

rm -rf build
rm -rf src/*.egg-info

Note:

Dirty build or egg-info dirs can cause problems: missing or stale files in the resulting dist or strange and confusing errors. Avoid having them around.

Then you should check that you got no packaging issues:

tox -e check

And then you can build the sdist, and if possible, the bdist_wheel too:

python setup.py clean --all sdist bdist_wheel

To make a release of the project on PyPI, assuming you got some distributions in dist/, the most simple usage is:

twine register dist/*
twine upload --skip-existing dist/*.whl dist/*.gz dist/*.zip

In ZSH you can use this to upload everything in dist/ that ain't a linux-specific wheel (you may need setopt extended_glob):

twine upload --skip-existing dist/*.(whl|gz|zip)~dist/*linux*.whl

For making and uploading manylinux1 wheels you can use this contraption:

docker run --rm -itv $(pwd):/code quay.io/pypa/manylinux1_x86_64 bash -c 'set -eux; cd code; rm -rf wheelhouse; for variant in /opt/python/*; do rm -rf dist build *.egg-info && $variant/bin/python setup.py clean --all bdist_wheel; auditwheel repair dist/*.whl; done; rm -rf dist build *.egg-info'
twine upload --skip-existing wheelhouse/*.whl
docker run --rm -itv $(pwd):/code quay.io/pypa/manylinux1_i686 bash -c 'set -eux; cd code; rm -rf wheelhouse; for variant in /opt/python/*; do rm -rf dist build *.egg-info && $variant/bin/python setup.py clean --all bdist_wheel; auditwheel repair dist/*.whl; done; rm -rf dist build *.egg-info'
twine upload --skip-existing wheelhouse/*.whl

Note:

twine is a tool that you can use to securely upload your releases to PyPI. You can still use the old python setup.py register sdist bdist_wheel upload but it's not very secure - your PyPI password will be sent over plaintext.

Changelog

See CHANGELOG.rst.

Questions & answers

There's no Makefile?

Sorry, no Makefile yet. The Tox environments stand for whatever you'd have in a Makefile.

Why does tox.ini have a passenv = *?

Tox 2.0 changes the way it runs subprocesses - it no longer passes all the environment variables by default. This causes all sorts of problems if you want to run/use any of these with Tox: SSH Agents, Browsers (for Selenium), Appengine SDK, VC Compiler and so on.

cc-py-setup errs on the side of convenience here. You can always remove passenv = * if you like the strictness.

Why is the version stored in several files (pkg/__init__.py, setup.py, docs/conf.py)?

We cannot use a metadata/version file [1] because this template is to be used with both distributions of packages (dirs with __init__.py) and modules (simple .py files that go straigh in site-packages). There's no good place for that extra file if you're distributing modules.

But this isn't so bad - bumpversion manages the version string quite neatly.

[1]	Example, an `__about__.py` file.

Not Exactly What You Want?

No way, this is the best. 😜

If you have criticism or suggestions please open up an Issue or Pull Request.