heroku/heroku-buildpack-python

Possible out-of-date setuptools /wheel versions

jackkinsella opened this issue · 4 comments

jackkinsella commented

When deploying to Heroku I ran into the following issue when trying to install a package (git+https://github.com/jplehmann/django-hijack) via requirements.txt

remote:          Running setup.py install for django-hijack: finished with status 'error'
remote:          error: subprocess-exited-with-error
remote:
remote:          × Running setup.py install for django-hijack did not run successfully.
remote:          │ exit code: 1
remote:          ╰─> [8 lines of output]
remote:              /app/.heroku/python/lib/python3.10/site-packages/setuptools/installer.py:27: SetuptoolsDeprecationWarning: setuptools.installer is deprecated. Requirements should be satisfied by a PEP 517 installer.
remote:                warnings.warn(
remote:              usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
remote:                 or: setup.py --help [cmd1 cmd2 ...]
remote:                 or: setup.py --help-commands
remote:                 or: setup.py cmd --help
remote:
remote:              error: option --single-version-externally-managed not recognized
remote:              [end of output]
remote:
remote:          note: This error originates from a subprocess, and is likely not a problem with pip.
remote:        error: legacy-install-failure
remote:
remote:        × Encountered error while trying to install package.
remote:        ╰─> django-hijack
remote:
remote:        note: This is an issue with the package mentioned above, not pip.
remote:        hint: See above for output from the failure.
remote:  !     Push rejected, failed to compile Python app.

An hour earlier, I encountered this exact same problem on my development machine, but this was solved by running

pip install -U setuptools
pip install -U wheel

My python version is 3.10.6

I tried requiring specific minimum setuptools/wheel packages in my requirements.txt and deployed again, but this didn't solve the issue.

jackkinsella commented

Reproduction steps:

heroku run bash
pip install git+https://github.com/jplehmann/django-hijack
edmorley commented

Hi!

Thank you for the detailed report + steps to reproduce - makes a big difference :-)

The Python buildpack currently installs:

  • setuptools v63.4.3 (intentionally not the latest [1], but released pretty recently: changelog)
  • wheel v0.37.1 (which is the latest release: PyPI)

Whilst the setuptools version isn't the latest, it's already newer than what comes in the official Python 3.10 Docker image:

$ docker run --rm -it python:3.10.8-slim pip list
Package    Version
---------- -------
pip        22.2.2
setuptools 63.2.0
wheel      0.37.1

However the issue you are encountering is unrelated to the setuptools version.

What's happening is that:

  • The package is being installed from VCS (so needs to be built from a source distribution, in comparison to say a package on PyPI where a pre-built wheel has been uploaded to save time on installation)
  • Pip first tries to build this sdist using the setup.py bdist_wheel method, however this fails with an error (that's earlier in the logs than the log output included in the OP)
  • Pip then falls back to the legacy setuptools setup.py install method (which is why there is a deprecation warning, since that approach generally shouldn't be used), which then seems to not like the --single-version-externally-managed parameter that's being passed in somewhere (by something used by the django-hijack project; the Python buildpack doesn't set that flag)

When I try running pip install -vv git+https://github.com/jplehmann/django-hijack on a Heroku one-off dyno the original error (higher up in the logs) is:

...
  running compile_translations
  running command: msgfmt -c -o build/lib/hijack/locale/pt_BR/LC_MESSAGES/django.mo hijack/locale/pt_BR/LC_MESSAGES/django.po
  error: [Errno 2] No such file or directory: 'msgfmt'
  error: subprocess-exited-with-error
...

The reason for this error in a one-off dyno is that the msgfmt tool is part of the gettext package, and that package is only installed in the build-time stack image:
https://devcenter.heroku.com/articles/stack-packages

(In general pip install is not intended to be run in a one-off dyno, though it can be useful as a good place to start when debugging.)

When I try via a Heroku app build (which runs using the build time stack image, so has the msgfmt tool), the build gets further, but fails with:

$ mkdir testcase && cd $_
$ git init && heroku create
$ echo 'git+https://github.com/jplehmann/django-hijack' > requirements.txt
$ git add -A && git commit -m '.' && git push heroku main
...
remote:              running compile_translations
remote:              running command: msgfmt -c -o build/lib/hijack/locale/fr/LC_MESSAGES/django.mo hijack/locale/fr/LC_MESSAGES/django.po
remote:              running command: msgfmt -c -o build/lib/hijack/locale/da/LC_MESSAGES/django.mo hijack/locale/da/LC_MESSAGES/django.po
remote:              running command: msgfmt -c -o build/lib/hijack/locale/es/LC_MESSAGES/django.mo hijack/locale/es/LC_MESSAGES/django.po
remote:              running command: msgfmt -c -o build/lib/hijack/locale/cs/LC_MESSAGES/django.mo hijack/locale/cs/LC_MESSAGES/django.po
remote:              running command: msgfmt -c -o build/lib/hijack/locale/nl/LC_MESSAGES/django.mo hijack/locale/nl/LC_MESSAGES/django.po
remote:              running command: msgfmt -c -o build/lib/hijack/locale/de/LC_MESSAGES/django.mo hijack/locale/de/LC_MESSAGES/django.po
remote:              running command: msgfmt -c -o build/lib/hijack/locale/sk/LC_MESSAGES/django.mo hijack/locale/sk/LC_MESSAGES/django.po
remote:              running command: msgfmt -c -o build/lib/hijack/locale/ru/LC_MESSAGES/django.mo hijack/locale/ru/LC_MESSAGES/django.po
remote:              running command: msgfmt -c -o build/lib/hijack/locale/pt_BR/LC_MESSAGES/django.mo hijack/locale/pt_BR/LC_MESSAGES/django.po
remote:              running compile_scss
remote:              running command: npm ci
remote:              error: [Errno 2] No such file or directory: 'npm'
remote:              [end of output]
remote:
remote:          note: This error originates from a subprocess, and is likely not a problem with pip.
remote:          ERROR: Failed building wheel for django-hijack

This is the real cause of the package not installing - it needs a Node.js installation.

To make sure one is present, create a package.json, then add the Node.js buildpack, setting it to run before the Python buildpack.

For example:

$ mkdir testcase && cd $_
$ git init && heroku create
$ heroku buildpacks:set --index 1 heroku/nodejs
Buildpack set. Next release on blooming-everglades-92960 will use heroku/nodejs.
$ heroku buildpacks:set --index 2 heroku/python
Buildpack set. Next release on blooming-everglades-92960 will use:
  1. heroku/nodejs
  2. heroku/python
$ echo 'git+https://github.com/jplehmann/django-hijack' > requirements.txt
$ echo '{}' > package.json
$ git add -A && git commit -m '.' && git push heroku main
...
remote: -----> Installing requirements with pip
remote:        Collecting git+https://github.com/jplehmann/django-hijack (from -r /tmp/build_14c3fc25/requirements.txt (line 1))
remote:          Cloning https://github.com/jplehmann/django-hijack to /tmp/pip-req-build-ylwp8ko0
remote:          Running command git clone --filter=blob:none --quiet https://github.com/jplehmann/django-hijack /tmp/pip-req-build-ylwp8ko0
remote:          Resolved https://github.com/jplehmann/django-hijack to commit 7dc81adc3207cca1ab5efa9a2f38f2402ddc8d25
remote:          Preparing metadata (setup.py): started
remote:          Preparing metadata (setup.py): finished with status 'done'
remote:        Collecting django>=3.2
remote:          Downloading Django-4.1.2-py3-none-any.whl (8.1 MB)
remote:        Collecting asgiref<4,>=3.5.2
remote:          Downloading asgiref-3.5.2-py3-none-any.whl (22 kB)
remote:        Collecting sqlparse>=0.2.2
remote:          Downloading sqlparse-0.4.3-py3-none-any.whl (42 kB)
remote:        Building wheels for collected packages: django-hijack
remote:          Building wheel for django-hijack (setup.py): started
remote:          Building wheel for django-hijack (setup.py): finished with status 'done'
remote:          Created wheel for django-hijack: filename=django_hijack-0.1.dev749+g7dc81ad-py3-none-any.whl size=35523 sha256=048275ca86b7144fbc73f8edf17631bd66924df92de58e8b4889e3c85a361ced
remote:          Stored in directory: /tmp/pip-ephem-wheel-cache-m2zwx716/wheels/2b/45/17/c314aec2b5af25744ed05c6e931d41570986c4d731dcc1b8db
remote:        Successfully built django-hijack
remote:        Installing collected packages: sqlparse, asgiref, django, django-hijack
remote:        Successfully installed asgiref-3.5.2 django-4.1.2 django-hijack-0.1.dev749+g7dc81ad sqlparse-0.4.3
remote: -----> Skipping Django collectstatic since no manage.py file found.
remote: -----> Discovering process types
remote:        Procfile declares types -> (none)
remote:
remote: -----> Compressing...
remote:        Done: 59.7M
remote: -----> Launching...
remote:        Released v3

Longer term I would recommend that you migrate django-hijack from setup.py to pyproject.toml which I believe will prevent pip/setuptools from doing the confusing fallback behaviour that caused the earlier errors to be missed in the logs. See:
https://setuptools.pypa.io/en/stable/userguide/pyproject_config.html

Hope that helps!

[1] The buildpack is intentionally not using setuptools v64+ yet since its new editable install behaviour contains both breaking changes (by design) and also some bugs (examples). Over time, hopefully those bugs will be fixed, and also users/the ecosystem will adapt to the intentionally breaking design changes (eg the issues that affect flat package layouts, where the src isn't nested) to the point where we can update to a newer setuptools release.

jackkinsella commented

hi Ed,

I wanted to thank you for an incredibly helpful message. It not only prompted me how to figure out the problem, but it also taught me a lot about pip and package distribution. If we ever cross paths, let me buy you lunch!

In case this is useful for others, here's the path I took to get around this issue:

  • As I didn't want to install an extra buildpack on Heroku (we already have four, and they are sensitive to load order and slow), the best solution was to ensure that the compilation step happened off of Heroku. This would avoid the dependencies issue.
  • The way to do this was to run python setup.py bdist_wheel --universal within the fork of the library on my machine. This output a wheel file in the ./dist folder. (The universal flag means "all architectures", which makes sense for my library since it's just Python, JavaScript, CSS, and some translation files -- this won't be true if you have C code etc.)
  • I could then add this .whl file to my repo and tell requirements.txt to install from that exact path
edmorley commented

You are most welcome :-)