pypa/pip

Respect top-level "constraints" file when populating isolated build environments

mwchase opened this issue · 5 comments

mwchase commented

Environment

  • pip version: 20.1.1
  • Python version: 3.8.2
  • OS: Linux

pip freeze output:

attrs==19.3.0
certifi==2020.4.5.2
chardet==3.0.4
docutils==0.16
flit==2.3.0
flit-core==2.3.0
idna==2.9
punq==0.4.1
pytoml==0.1.21
requests==2.23.0
urllib3==1.25.9

Description

Installing a package using a PEP 517 backend with build isolation enabled results in an install_requirements call using the return value of backend.get_requires_for_build_wheel(), which generates and runs a pip install command. Constraints files passed to the outermost pip install command are not reflected in the generated command.

This means that, at least when using flit, a local install with build isolation will fail if it's relying on a constraints file to specify paths to dependencies that are only available locally.

(The other build backends I checked don't require the runtime requirements in the isolated environment. I brought this up in the flit repo first in pypa/flit#354)

Expected behavior

Given two projects, a dependency and a dependent, with the dependent using flit, and the dependency not present in an index, the command pip install -c path/to/constraints.txt path/to/dependent, where the constraints file has a line like name-of-dependency @ file:///absolute/path/to/dependency should succeed.

How to Reproduce

  1. Create a "dependency" project with a name not present in pypi
  2. Create a "dependent" project that depends on the dependency, using flit
  3. Create a constraints.txt file that at least contains a direct reference linking the name of the dependency to its absolute path
    4 Run pip install -c constraints.txt ./dependent, assuming the constraints file and the project directories are next to each other

Output

$ ls
constraints.txt  dependency_bleh  dependent
$ cat constraints.txt 
dependency_bleh @ file:///absolute/path/of/pwd/dependency_bleh
dependent @ file:///absolute/path/of/pwd/dependent
$ pip install -c constraints.txt  ./dependent
Processing ./dependent
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... error
  ERROR: Command errored out with exit status 1:
   command: <venv>/bin/python3.8 <venv>/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-67rnvdf5/normal --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- dependency_bleh
       cwd: None
  Complete output (2 lines):
  ERROR: Could not find a version that satisfies the requirement dependency_bleh (from versions: none)
  ERROR: No matching distribution found for dependency_bleh
  ----------------------------------------
ERROR: Command errored out with exit status 1: <venv>/bin/python3.8 <venv>/lib/python3.8/site-packages/pip install --ignore-installed --no-user --prefix /tmp/pip-build-env-67rnvdf5/normal --no-warn-script-location --no-binary :none: --only-binary :none: -i https://pypi.org/simple -- dependency_bleh Check the logs for full command output.
mwchase commented

PIP_CONSTRAINT=constraints.txt pip install dependent turns out to be an acceptable workaround for my purposes. Probably better than the other workarounds I've used since I filed this.

EDIT: Also, FWIW, if the version can be calculated just by AST analysis, flit now works with the command I gave above. After I remembered this, I tested with the old command, then verified both commands against a setup that deliberately broke flit's AST analysis. When AST analysis fails, the original command fails, and the workaround does, in fact, work.

mwchase commented

I should have kept up with the behavior of the new resolver. It appears that #8210 in a sense, breaks both the problem and the solution. I'll investigate more comprehensive workarounds that are outside the scope of this issue.

I suppose this issue should be left open, because it's still relevant to the intended behavior of constraints files.

apljungquist commented

It appears another consequence of this is that hash-checking is not enabled for build system requirements, not even if --require-hashes is set on the primary pip command. This is particularly surprising given the emphasis on secure default behaviors w.r.t. hash checking described in the documentation.

Using the workaround suggested earlier hashes are checked also for build system requirements, assuming hashes are specified for at least one package in the constraints.

chaoflow commented

Could somebody clarify what the status is with recent pip? Grepping for PIP_CONSTRAINT produces no results.

As @apljungquist already pointed out, hashes must be checked also for build dependencies. Is there anything to help make this?