pypa/pip

When $TMPDIR is within $PWD, pip wheel dies on infinite recursion

hroncok opened this issue · 2 comments

Environment

  • pip version: master, 20.0.2, 19.3.1... possibly more
  • Python version: 3.9.0a4, 3.8.2, 3.7.7... possibly all
  • OS: Linux (Fedora)

Description

During debugging the problem described in #7555 (comment) we ahve realized that simply setting $TMPDIR to $PWD/.tmp would make our problems go away. However, when doing that, pip wheel tries to copy the entire $PWD to $TMPDIR/pip-req-build-... and it dies out on recursion:

Traceback (most recent call last):
  File "/usr/lib/python3.8/site-packages/pip/_internal/cli/base_command.py", line 186, in _main
    status = self.run(options, args)
  File "/usr/lib/python3.8/site-packages/pip/_internal/commands/wheel.py", line 161, in run
    resolver.resolve(requirement_set)
  File "/usr/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 178, in resolve
    discovered_reqs.extend(self._resolve_one(requirement_set, req))
  File "/usr/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 336, in _resolve_one
    abstract_dist = self._get_abstract_dist_for(req_to_install)
  File "/usr/lib/python3.8/site-packages/pip/_internal/legacy_resolve.py", line 285, in _get_abstract_dist_for
    abstract_dist = self.preparer.prepare_linked_requirement(req)
  File "/usr/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 486, in prepare_linked_requirement
    local_path = unpack_url(
  File "/usr/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 284, in unpack_url
    return unpack_file_url(link, location, download_dir, hashes=hashes)
  File "/usr/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 230, in unpack_file_url
    _copy_source_tree(link_path, location)
  File "/usr/lib/python3.8/site-packages/pip/_internal/operations/prepare.py", line 213, in _copy_source_tree
    shutil.copytree(source, target, **kwargs)
  File "/usr/lib64/python3.8/shutil.py", line 554, in copytree
    return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
  File "/usr/lib64/python3.8/shutil.py", line 510, in _copytree
    raise Error(errors)
shutil.Error: [('/builddir/build/BUILD/mistune-0.8.3/.../.tmp/pip-req-build-0pdab7qx/.tmp/pip-req-build-0pdab7qx/__pycache__/mistune.cpython-38.pyc', '/builddir/build/BUILD/mistune-0.8.3/.../.tmp/pip-req-build-0pdab7qx/.tmp/pip-req-build-0pdab7qx/.tmp/pip-req-build-0pdab7qx/__pycache__/mistune.cpython-38.pyc', "[Errno 36] File name too long: '/builddir/build/BUILD/mistune-0.8.3/.../.tmp/pip-req-build-0pdab7qx/.tmp/pip-req-build-0pdab7qx/.tmp/pip-req-build-0pdab7qx/__pycache__/mistune.cpython-38.pyc'"), ...]

Expected behavior

The target directory itself should be ignored when copying to it.

How to Reproduce

$ export TMPDIR=${PWD}/.tmp
$ mkdir ${TMPDIR}
$ pip wheel .  # with an extension module

Output

+ /usr/bin/python3 -m pip wheel --wheel-dir ./pyproject-macros-wheeldir --no-deps --use-pep517 --no-build-isolation --disable-pip-version-check --no-clean --progress-bar off --verbose .
Created temporary directory: /builddir/build/BUILD/mistune-0.8.3/.tmp/pip-ephem-wheel-cache-40ziqhu0
Created temporary directory: /builddir/build/BUILD/mistune-0.8.3/.tmp/pip-req-tracker-kg6rp8lh
Initialized build tracking at /builddir/build/BUILD/mistune-0.8.3/.tmp/pip-req-tracker-kg6rp8lh
Created build tracker: /builddir/build/BUILD/mistune-0.8.3/.tmp/pip-req-tracker-kg6rp8lh
Entered build tracker: /builddir/build/BUILD/mistune-0.8.3/.tmp/pip-req-tracker-kg6rp8lh
Created temporary directory: /builddir/build/BUILD/mistune-0.8.3/.tmp/pip-wheel-d6bw8m93
Processing /builddir/build/BUILD/mistune-0.8.3
  Created temporary directory: /builddir/build/BUILD/mistune-0.8.3/.tmp/pip-req-build-0pdab7qx
Removed build tracker: '/builddir/build/BUILD/mistune-0.8.3/.tmp/pip-req-tracker-kg6rp8lh'
ERROR: Exception:
(see above)

I have a fix ready and will submit a Pull Request shorty.