Using --target with --editable results in "internal" error
piotr-dobrogost opened this issue · 22 comments
It seems like --target
is not supported when using --editable
at the same time. It would be nice to have it working. Meanwhile there should be some better diagnostic as
c:\>pip install -t z:\1 -e git+https://github.com/kennethreitz/requests.git#egg=requests
results in
Obtaining requests from git+https://github.com/kennethreitz/requests.git#egg=requests
Updating c:\python\virtualenv\test\src\requests clone
Running setup.py egg_info for package requests
Downloading/unpacking certifi>=0.0.7 (from requests)
Running setup.py egg_info for package certifi
Downloading/unpacking oauthlib>=0.1.0,<0.2.0 (from requests)
Running setup.py egg_info for package oauthlib
Downloading/unpacking chardet>=1.0.0 (from requests)
Running setup.py egg_info for package chardet
Downloading/unpacking rsa (from oauthlib>=0.1.0,<0.2.0->requests)
Running setup.py egg_info for package rsa
warning: no files found matching 'README'
Downloading/unpacking pyasn1>=0.0.13 (from rsa->oauthlib>=0.1.0,<0.2.0->requests)
Running setup.py egg_info for package pyasn1
Installing collected packages: requests, certifi, oauthlib, chardet, rsa, pyasn1
Running setup.py develop for requests
usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: -c --help [cmd1 cmd2 ...]
or: -c --help-commands
or: -c cmd --help
error: option --home not recognized
Complete output from command c:\python\virtualenv\test\Scripts\python.exe -c "import setuptools; __file__='c:\\python\\virtualenv\\test\\src\\requests\\setup.py'; exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" develop --no-deps --home=c:\users\piotr\appdata\local\temp\tmp3abskl:
usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
or: -c --help [cmd1 cmd2 ...]
or: -c --help-commands
or: -c cmd --help
error: option --home not recognized
----------------------------------------
Command c:\python\virtualenv\test\Scripts\python.exe -c "import setuptools; __file__='c:\\python\\virtualenv\\test\\src\\requests\\setup.py'; exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" develop --no-deps --home=c:\users\piotr\appdata\local\temp\tmp3abskl failed with error code 1 in c:\python\virtualenv\test\src\requests
Storing complete log in C:\Users\Piotr\AppData\Roaming\pip\pip.log
Experiencing this error with pip install -t dir -e git+git://github.com/shazow/urllib3.git@f088037#egg=urllib3
as well.
i also ran into this error right now.
This seems to happen on any package if --target
and --editable
are combined:
pip calls setup.py develop --home ...
, but --home
is only applicable with setup.py install
.
So, in the end i found out that using the --src
option with editable packages instead of --target
does what i want.
Maybe --target
should have the same effect as --src
when --editable
is specified or it could present the user with an error message and maybe point the user to --src
.
Maybe --target should have the same effect as --src when --editable is specified
IMO the expected behavior would create / update the easy_install.pth in the target
directory.
I just experienced the same issue. As suggested by @jezdez you can work around this by doing the following (without using --editable that is):
git+ssh://user@github.com/some-user/some-repo.git#egg=Foo
I'm seeing a similar issue here myself:
Cleaning up...
Exception:
Traceback (most recent call last):
File "/efs/dev/bti/pip/1.3.1-build001/install/exec/2.7/lib/python/pip-1.3.1-py2.7.egg/pip/basecommand.py", line 139, in main
status = self.run(options, args)
File "/efs/dev/bti/pip/1.3.1-build001/install/exec/2.7/lib/python/pip-1.3.1-py2.7.egg/pip/commands/install.py", line 291, in run
for item in os.listdir(lib_dir):
OSError: [Errno 2] No such file or directory: '/tmp/tmppjdGHI/lib/python'
line 290 in pip/commands/install.py is:
lib_dir = home_lib(temp_target_dir)
From what I can trace, pip has already cleaned this path up in pip/req.py line 1194 where it removes the temporary source.
requirement.remove_temporary_source()
I can see leaving that clean-up in there as is but wrapping it with an exists or try block so the install can continue on. Does anyone see that as a problem?
@tima I had the same problem with the lib directory. Pip HEAD supposedly fixed this but at least on CentOS the issue still persists. In this particular case there is a lib64 directory instead of a lib one.
I have the same issue using --target
(but not --editable
).
Here's my traceback—
Exception:
Traceback (most recent call last):
File "/Users/beaum/homebrew/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg/pip/basecommand.py", line 139, in main
status = self.run(options, args)
File "/Users/beaum/homebrew/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/pip-1.3.1-py2.7.egg/pip/commands/install.py", line 294, in run
for item in os.listdir(lib_dir):
OSError: [Errno 2] No such file or directory: '/var/folders/00/1hyxr000h01000cxqpysvccm0063vq/T/tmpc_E_Bl/lib/python'
+1,
Downloading PyYAML-3.10.tar.gz (241kB): 241kB downloaded Running setup.py egg_info for package pyyaml skipping 'ext/_yaml.c' Cython extension (up-to-date) Installing collected packages: krcore, sympy, pyparsing, pyyaml Running setup.py develop for krcore usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] or: -c --help [cmd1 cmd2 ...] or: -c --help-commands or: -c cmd --help error: option --home not recognized Complete output from command /usr/bin/python -c "import setuptools; __file__='/tmp/krapp/src/krcore/setup.py'; exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" develop --no-deps --home=/tmp/tmpvKaRYp: usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...] or: -c --help [cmd1 cmd2 ...] or: -c --help-commands or: -c cmd --help error: option --home not recognized ---------------------------------------- Cleaning up... Command /usr/bin/python -c "import setuptools; __file__='/tmp/krapp/src/krcore/setup.py'; exec(compile(open(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" develop --no-deps --home=/tmp/tmpvKaRYp failed with error code 1 in /tmp/krapp/src/krcore
I get the 'error: option --home not recognized' when using --target with -r (--requirements)
👍
Following @yorickpeterse and @jezdez workaround caused:
error: must supply either home or prefix/exec-prefix -- not both
--editable and --target options don't work together
I have been hitting the same issue...
I am trying to use pip to install python packages in a custom location (not a virtualenv) and I need one of them (the one I'm working on) to be editable.
This is likely related to pypa/setuptools#392
And since one pip command can trigger both setup.py develop
(for editable pkg) and setup.py install
(dependencies), the only (very ugly) workaround that I can think about is to issue 2 commands :
- one for pkg with
--no-deps
- one for dependencies only (no simple way here...)
It would be much cleaner if there was no need to change other pip command line options whether --editable
is passed or not.
So I guess there are two ways to get this working :
- making setuptools support
--home
ie. fixing pypa/setuptools#392. probably tricky, reading the details of that issue ? - having pip abstract setuptools details and implement
--target
behavior differently depending if it callssetup.py develop
orsetup.py install
. maybe simpler ?
having pip abstract setuptools details and implement
--target
behavior differently depending if it callssetup.py develop
orsetup.py install
. maybe simpler ?
The biggest difficulty with this option is that pip is working to abstract away the details of the build system (setuptools) so special-case workarounds for setuptools issues work against that goal.
In the end I managed to do what I wanted, using --prefix
, and without using any custom --install-option
, so I can finally use the same options whether I pass --editable
or not.
However I had to somehow adapt my existing (debian) layout to match pip default (site-packages), since there is no pip option to specify the layout...
Maybe a feature to consider adding?
This needs --target option
label.
could someone please share what the best workaround is to use both -e and -t options? Are you suggesting to use distutils since it supports the '--home' option?
any news?
I am still using --editable
with --prefix
(instead of --target
) which does the job for me. But I am stuck at pip<9.0.0 because of the difference between --prefix
and --target
. more details in #4243
if you want to use the --prefix option instead of --target you need to have the PYTHONPATH
(pointing to the to-be-created site-packages dir) set correctly before you can call pip -t . --prefix myprefix
. Is there an elegant way to circumvent this?
I'd add that this is an issue for Google Appengine developers who need to install their dependencies in a directory in the app root but also need to deploy an editable dependency in a local checkout as part of a QA process. As it sits, the editable would need to be manually symlinked which is cumbersome.
This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.