pypa/pip

DistutilsOptionError with --target and prefix or exec-prefix

jaraco opened this issue · 9 comments

jaraco commented
  • Pip version: 9.0.1
  • Python version: 2.7.12
  • Operating System: macOS 10.12.1

Description:

Having installed Python 2.7 using Homebrew, I'm unable to install packages using `--target'

What I've run:

$ python2.7 -m pip install -t foo requests
Collecting requests
  Using cached requests-2.11.1-py2.py3-none-any.whl
Installing collected packages: requests
Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/pip/basecommand.py", line 215, in main
    status = self.run(options, args)
  File "/usr/local/lib/python2.7/site-packages/pip/commands/install.py", line 342, in run
    prefix=options.prefix_path,
  File "/usr/local/lib/python2.7/site-packages/pip/req/req_set.py", line 784, in install
    **kwargs
  File "/usr/local/lib/python2.7/site-packages/pip/req/req_install.py", line 851, in install
    self.move_wheel_files(self.source_dir, root=root, prefix=prefix)
  File "/usr/local/lib/python2.7/site-packages/pip/req/req_install.py", line 1064, in move_wheel_files
    isolated=self.isolated,
  File "/usr/local/lib/python2.7/site-packages/pip/wheel.py", line 247, in move_wheel_files
    prefix=prefix,
  File "/usr/local/lib/python2.7/site-packages/pip/locations.py", line 153, in distutils_scheme
    i.finalize_options()
  File "/usr/local/Cellar/python/2.7.12_2/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/command/install.py", line 264, in finalize_options
    "must supply either home or prefix/exec-prefix -- not both"
DistutilsOptionError: must supply either home or prefix/exec-prefix -- not both

Obviously, I didn't supply home nor prefix, so the error message is confusing.

dstufft commented

Closing to move a bunch of related issues to a single issue: #4390.

jaraco commented

This issue still exists.

$ python2.7 -m pip -V
pip 18.1 from /usr/local/lib/python2.7/site-packages/pip (python 2.7)
$ python2.7 -m pip install --target out requests
Collecting requests
  Downloading https://files.pythonhosted.org/packages/ff/17/5cbb026005115301a8fb2f9b0e3e8d32313142fe8b617070e7baad20554f/requests-2.20.1-py2.py3-none-any.whl (57kB)
    100% |████████████████████████████████| 61kB 400kB/s
Collecting certifi>=2017.4.17 (from requests)
  Using cached https://files.pythonhosted.org/packages/56/9d/1d02dd80bc4cd955f98980f28c5ee2200e1209292d5f9e9cc8d030d18655/certifi-2018.10.15-py2.py3-none-any.whl
Collecting idna<2.8,>=2.5 (from requests)
  Using cached https://files.pythonhosted.org/packages/4b/2a/0276479a4b3caeb8a8c1af2f8e4355746a97fab05a372e4a2c6a6b876165/idna-2.7-py2.py3-none-any.whl
Collecting urllib3<1.25,>=1.21.1 (from requests)
  Using cached https://files.pythonhosted.org/packages/62/00/ee1d7de624db8ba7090d1226aebefab96a2c71cd5cfa7629d6ad3f61b79e/urllib3-1.24.1-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from requests)
  Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Installing collected packages: certifi, idna, urllib3, chardet, requests
Exception:
Traceback (most recent call last):
  File "/usr/local/lib/python2.7/site-packages/pip/_internal/cli/base_command.py", line 143, in main
    status = self.run(options, args)
  File "/usr/local/lib/python2.7/site-packages/pip/_internal/commands/install.py", line 366, in run
    use_user_site=options.use_user_site,
  File "/usr/local/lib/python2.7/site-packages/pip/_internal/req/__init__.py", line 49, in install_given_reqs
    **kwargs
  File "/usr/local/lib/python2.7/site-packages/pip/_internal/req/req_install.py", line 760, in install
    use_user_site=use_user_site, pycompile=pycompile,
  File "/usr/local/lib/python2.7/site-packages/pip/_internal/req/req_install.py", line 382, in move_wheel_files
    warn_script_location=warn_script_location,
  File "/usr/local/lib/python2.7/site-packages/pip/_internal/wheel.py", line 215, in move_wheel_files
    prefix=prefix,
  File "/usr/local/lib/python2.7/site-packages/pip/_internal/locations.py", line 165, in distutils_scheme
    i.finalize_options()
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/command/install.py", line 264, in finalize_options
    "must supply either home or prefix/exec-prefix -- not both"
DistutilsOptionError: must supply either home or prefix/exec-prefix -- not both

Note, I have not specified prefix nor home. There's something about the Homebrew install of Python that causes it to set the prefix. I think it's this:

$ cat /usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/distutils.cfg
[install]
prefix=/usr/local

[build_ext]
include_dirs=/usr/local/include:/usr/local/opt/openssl/include:/usr/local/opt/sqlite/include
library_dirs=/usr/local/lib:/usr/local/opt/openssl/lib:/usr/local/opt/sqlite/lib

As a result, pip install --target is simply broken on vanilla Homebrew installs (or any install that sets a prefix).

benoit-pierre commented

Does it work when using pip --isolated install --target?

jaraco commented

Does it work with --isolated?

No, same error.

benoit-pierre commented

From the code, it looks like --isolated will ignore distutils user configuration, but the global one will still get picked up.

benoit-pierre commented

I think, similarly to when user is used, distutils_scheme should ignore the install prefix when home is used:

 src/pip/_internal/locations.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git i/src/pip/_internal/locations.py w/src/pip/_internal/locations.py
index 89a3656b..72fe7c6f 100644
--- i/src/pip/_internal/locations.py
+++ w/src/pip/_internal/locations.py
@@ -170,7 +170,7 @@ def distutils_scheme(dist_name, user=False, home=None, root=None,
     # ideally, we'd prefer a scheme class that has no side-effects.
     assert not (user and prefix), "user={} prefix={}".format(user, prefix)
     i.user = user or i.user
-    if user:
+    if user or home:
         i.prefix = ""
     i.prefix = prefix or i.prefix
     i.home = home or i.home
benoit-pierre commented

The assert line should probably be updated too.

jaraco commented

Confirmed - that one patch does address the issue.

pip 18.1 $ python2.7 -m pip -V
pip 18.1 from /Users/jaraco/code/public/pypa/pip/src/pip (python 2.7)
pip 18.1 $ python2.7 -m pip install --target out requests
Collecting requests
  Using cached https://files.pythonhosted.org/packages/ff/17/5cbb026005115301a8fb2f9b0e3e8d32313142fe8b617070e7baad20554f/requests-2.20.1-py2.py3-none-any.whl
Collecting idna<2.8,>=2.5 (from requests)
  Using cached https://files.pythonhosted.org/packages/4b/2a/0276479a4b3caeb8a8c1af2f8e4355746a97fab05a372e4a2c6a6b876165/idna-2.7-py2.py3-none-any.whl
Collecting urllib3<1.25,>=1.21.1 (from requests)
  Using cached https://files.pythonhosted.org/packages/62/00/ee1d7de624db8ba7090d1226aebefab96a2c71cd5cfa7629d6ad3f61b79e/urllib3-1.24.1-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from requests)
  Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests)
  Using cached https://files.pythonhosted.org/packages/56/9d/1d02dd80bc4cd955f98980f28c5ee2200e1209292d5f9e9cc8d030d18655/certifi-2018.10.15-py2.py3-none-any.whl
Installing collected packages: idna, urllib3, chardet, certifi, requests
Exception:
Traceback (most recent call last):
  File "/Users/jaraco/code/public/pypa/pip/src/pip/_internal/cli/base_command.py", line 143, in main
    status = self.run(options, args)
  File "/Users/jaraco/code/public/pypa/pip/src/pip/_internal/commands/install.py", line 366, in run
    use_user_site=options.use_user_site,
  File "/Users/jaraco/code/public/pypa/pip/src/pip/_internal/req/__init__.py", line 49, in install_given_reqs
    **kwargs
  File "/Users/jaraco/code/public/pypa/pip/src/pip/_internal/req/req_install.py", line 760, in install
    use_user_site=use_user_site, pycompile=pycompile,
  File "/Users/jaraco/code/public/pypa/pip/src/pip/_internal/req/req_install.py", line 382, in move_wheel_files
    warn_script_location=warn_script_location,
  File "/Users/jaraco/code/public/pypa/pip/src/pip/_internal/wheel.py", line 215, in move_wheel_files
    prefix=prefix,
  File "/Users/jaraco/code/public/pypa/pip/src/pip/_internal/locations.py", line 165, in distutils_scheme
    i.finalize_options()
  File "/usr/local/Cellar/python@2/2.7.15_1/Frameworks/Python.framework/Versions/2.7/lib/python2.7/distutils/command/install.py", line 264, in finalize_options
    "must supply either home or prefix/exec-prefix -- not both"
DistutilsOptionError: must supply either home or prefix/exec-prefix -- not both
pip 18.1 $ git stash pop
HEAD detached at 18.1
Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git checkout -- <file>..." to discard changes in working directory)

        modified:   src/pip/_internal/locations.py

no changes added to commit (use "git add" and/or "git commit -a")
Dropped refs/stash@{0} (2c685ea2bf964f8475f09552556b800a693f1201)
pip 18.1 $ git diff
diff --git a/src/pip/_internal/locations.py b/src/pip/_internal/locations.py
index 183aaa39..a4c79135 100644
--- a/src/pip/_internal/locations.py
+++ b/src/pip/_internal/locations.py
@@ -157,7 +157,7 @@ def distutils_scheme(dist_name, user=False, home=None, root=None,
     # ideally, we'd prefer a scheme class that has no side-effects.
     assert not (user and prefix), "user={} prefix={}".format(user, prefix)
     i.user = user or i.user
-    if user:
+    if user or home:
         i.prefix = ""
     i.prefix = prefix or i.prefix
     i.home = home or i.home
pip 18.1 $ python2.7 -m pip install --target out requests
Collecting requests
  Using cached https://files.pythonhosted.org/packages/ff/17/5cbb026005115301a8fb2f9b0e3e8d32313142fe8b617070e7baad20554f/requests-2.20.1-py2.py3-none-any.whl
Collecting idna<2.8,>=2.5 (from requests)
  Using cached https://files.pythonhosted.org/packages/4b/2a/0276479a4b3caeb8a8c1af2f8e4355746a97fab05a372e4a2c6a6b876165/idna-2.7-py2.py3-none-any.whl
Collecting urllib3<1.25,>=1.21.1 (from requests)
  Using cached https://files.pythonhosted.org/packages/62/00/ee1d7de624db8ba7090d1226aebefab96a2c71cd5cfa7629d6ad3f61b79e/urllib3-1.24.1-py2.py3-none-any.whl
Collecting chardet<3.1.0,>=3.0.2 (from requests)
  Using cached https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl
Collecting certifi>=2017.4.17 (from requests)
  Using cached https://files.pythonhosted.org/packages/56/9d/1d02dd80bc4cd955f98980f28c5ee2200e1209292d5f9e9cc8d030d18655/certifi-2018.10.15-py2.py3-none-any.whl
Installing collected packages: idna, urllib3, chardet, certifi, requests
Successfully installed certifi-2018.10.15 chardet-3.0.4 idna-2.7 requests-2.20.1 urllib3-1.24.1