pypa/pip

The --target option clashes with other command line flags and config files

dstufft opened this issue ยท 28 comments

The --target option clashes with several other command like flags and config files like --user and distutils setup. Ideally we should handle this far more gracefully.

See also #3826, #4106, #562, #4139

halms commented

I have made an attempt to fix the behavior of --target in PR #4111.

#4111 was rebased to #4557

jedie commented

Current pull request is: #4557 isn't it?

The bug disappears with the latest pip 18.0.

This related issue still appears with pip 18.0
pypa/setuptools#392

Can confirm, this issue still persists.

Getting this error when installing with pipenv a local -e module:

  Running setup.py develop for Flask-Serverless
    Complete output from command /Users/cyber/.virtualenvs/jbwidgets-Jr4Btlt8/bin/python3.7 -c "import setuptools, tokenize;__file__=
'/Users/cyber/dev/flask-serverless/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close(
);exec(compile(code, __file__, 'exec'))" --no-user-cfg develop --no-deps --home=/private/var/folders/n8/rmzyc9yx2zvbc978_p6y_by40000g
n/T/pip-target-1lg_9i3s:
    usage: -c [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
       or: -c --help [cmd1 cmd2 ...]
       or: -c --help-commands
       or: -c cmd --help
    
    error: option --home not recognized

It still doesn't work.

Does anyone have any advice on how to work around this issue? It's particularly problematic when dealing with private repositories (since the -e flag breaks things), and this issue essentially makes it impossible to work with private repositories on serverless setups.

I have the same issue which breaks the deployment of serverless functions. The workaround with setting the prefix= to blanck in ~/.pydistutils.cfg does not work. Does anyone have another idea?

#6008 might fix some of this. Not sure what needs to be done on that or if it just needs to be reviewed?

rdlou commented

Is there an update for this? its pretty essential for creating zips to deploy on aws lambda

Solved this for the time being on Ubuntu 18.04.2 LTS with
setting the prefix= to blank in /root/.pydistutils.cfg
and sudo pip install <Package> --target <dir>

any update on this issue? I'm still getting error: option --home not recognize when I run the command

pip install \
  --target . \
  --editable git+ssh://git@github.com/myOrg/myPackage@master#egg=my-package

Is there any workaround? I'm having the same issue. pip install -r requirements with '--target' fails.

@joinsito It is impossible to suggest workarounds without information on your runtime configurations and environment.

@joinsito I was trying to package libraries for AWS Lambda and was getting this issue. Their documentation had a helpful suggestion:

In order for --target to work on Debian-based systems like Ubuntu, you may also need to pass the --system flag to prevent distutils errors.

So by adding the --system flag to the list of arguments, I was able to resolve my particular issue

@lorenzoLectrefy Maybe this could be a workaround for your case:

pip wheel \
  --src . \
  --editable git+ssh://git@github.com/myOrg/myPackage@master#egg=my-package
zooba commented

I just posted (half) a PR that enables at least some of this. Not sure if it's what the team had in mind, but I'm open to making changes.

PR #8555

This is also covered by #7828.

zooba commented

This is also covered by #7828.

Not quite. My change makes it so that --target on the command line will override the PIP_USER environment variable (also --prefix, which now I think about it isn't quite right?).

But #7828 is a good looking change to take as well. We'll just have to update the "check for incompatible options" part of it to allow CLI flags to override other defaults.

I had to combine the --target and --editable options when I needed to package my custom code shared among my AWS Lambda Functions as a Lambda layer, and ended up with an error error: option --home not recognized.

My workaround so far is

  1. Create a wheel with --editable,

    pip wheel --editable .

    Suppose the output is my-module-x.y.z.whl.

  2. Install from the wheel created at the step 1 (i.e., my-module-x.y.z.whl) with --target.

    pip install --target ../package/python my-module-x.y.z.whl
dwt commented

I don't pretend to understand all the history in this bug - but it seems sensible to convert the --target $folder option to --install-dir=$folder, when calling setup.py develop instead of providing --home would provide a good enough workaround for this? Is there any reason not to do this?

time available and the fact that until recently this was using easy_install which is not compatible with --home

dwt commented

@RonnyPfannschmidt I've created a pull request that implements the fix by renaming the argument handed to setup.py. What do you think?

Looks like a breaking change

dwt commented

@RonnyPfannschmidt I'm not sure I understand - the behaviour was broken before, as in using --target together with --editable exploded while installing. Is that what you mean?

This is not yet fixed since we still need some changes in setuptools, but Iโ€™m closing since thereโ€™s nothing more to do at pipโ€™s side. Kindly continue the discussion on setuptoolsโ€™s issue tracker, and open new issues if it turns out pip needs to do something more about this.