Support URL and path dependencies without a package name
charliermarsh opened this issue · 37 comments
Apparently this works with pip?
git+https://github.com/pallets/flask.git@refs/pull/5313/head
As opposed to:
flask @ git+https://github.com/pallets/flask.git@refs/pull/5313/head
We could decide not to support these. I don't know if they're spec-compliant.
\cc @konstin - do you know if these are spec-compliant? Or legacy?
They are iirc not part of PEP 621 and PEP 508, but they are supported in many places (pip install https://... makes sense)
Sadly, adds a lot of complexity.
Oh this is the syntax I use every time I install a package from git using pip 😬
I'm not sure if we really need to support it in a requirements file? Unless it's common in the wild. Generally it seems reasonable to have a nice error suggesting inclusion of the package name.
If we support it at all, then it’s not too hard to support it in a requirements file too.
Although, I guess it lets us skip supporting it in the parser. Anyway, we probably do need to support this long-term if it’s allowed in pyproject.toml.
But yeah we should support them :)
@konstin - Do you think we should make name optional on Requirement, or convert Requirement to an enum? Names can only be omitted when the version specific is a URL. Similarly, you can't have extras without a name. Feels like all of this could be encoded in the type system.
We should add a RequirementsTxtRequirement which allows not having a name (or where name can be a url), i'd keep PEP 508 Requirement as-is.
I'll give this another try.
Would you mind waiting until #587 is merged? Editables will profit from this, but doing it now would clash badly.
Will do!
It seems like this is also an issue for installing a local whl file?
% uv pip install ./streamlit-1.31.1-py2.py3-none-any.whl
error: Failed to parse `./streamlit-1.31.1-py2.py3-none-any.whl`
Caused by: Expected package name starting with an alphanumeric character, found '.'
./streamlit-1.31.1-py2.py3-none-any.whl
^
% uv pip install "streamlit @ ./streamlit-1.31.1-py2.py3-none-any.whl"
Resolved 40 packages in 680ms
Downloaded 40 packages in 1.74s
Installed 40 packages in 97ms
...BTW uv seems awesome!! Thank you!!!
Yup, all the same issue! Although we can probably support local wheels "trivially" since the file name is required to be encoded in the wheel.
This is probably the biggest blocker to experimenting with uv in Meltano since dependencies of the type git+https://... are very widely used by plugins.
👍 I want to support this, requires a bunch of internal refactors but definitely doable.
Hopefully not to add to the confusion - I've been running scenarios against integrating with pdm.
While investigating my own feature request, pdm-project/pdm#2641 I discovered pdm is handling requirements.txt-style exports in a way that falls back to pip's behavior...and works with uv if I remove the #egg= fragments.
Here's some quirks I was able to discover. Being able to do uv v --seed for running through was very helpful, I use the .venv/bin/pip for wherever pip install is used. I believe these quirks would be universal, anywhere there's a python package with a pyproject.toml.
I'm using the pdm-example-monorepo below in the latest Python Docker image under a freshly made user in their home directory.
When in a git directory:
pip will end up resolving an editable install (pip install -e 'file://'$PWD'relative/path') of a local path that's inside a git directory differently than one that is not.
When inside a git directory, pip will include an #egg= fragment along with another fragment param for its relative directory, subdirectory.
e.g.;
# replace .venv
$ uv v --seed
# install the package as editable with relative path
$ .venv/bin/pip install -e 'file://'$PWD'packages/pkg-second'
# ...
# show pip installation
$ .venv/bin/pip freeze | grep pkg-core
-e git+https://github.com/pdm-project/pdm-example-monorepo@b85261555ea063e68eaf744e225804149c27e64f#egg=pkg_core&subdirectory=packages/pkg-coreWhen it's not a git directory, and it's still an editable install:
# replace .venv
$ uv v --seed
# get rid of .git, so it's not detected as a git repository
$ mv .git .gitold
# install the package as editable with relative installation, again
$ .venv/bin/pip install -e 'file://'$PWD'/packages/pkg-core'
# ...
# show pip installation
$ .venv/bin/pip freeze | grep pkg-core
# Editable install with no version control (pkg-core==0.1.0+editable)
-e /home/python/pdm-example-monorepo/packages/pkg-coreNon-editable installs for comparison
There's also this way that will lend itself to the next example - pkg-name @ file:///..., which cannot be installed as editable, regardless of git directory or not:
# replace .venv
$ uv v --seed
# install relative package, non-editable, pkg @ file://... way
$ .venv/bin/pip install 'pkg-core @ file://'$PWD'/packages/pkg-core'
# ...
# freezes the same way whether in a git repository or not
$ venv/bin/pip freeze | grep pkg-core
pkg-core @ file:///home/python/pdm-example-monorepo/packages/pkg-coreAnd lastly - pip install 'file://... will result in the same package-name @ file://... expansion regardless if it's in a git directory or not 😅
# replace .venv
$ uv v --seed
# install non-editable, just 'file://...'
$ .venv/bin/pip install 'file://'$PWD'/packages/pkg-core'
# ...
# freezes the same way whether in git repository or not
$ .venv/bin/pip freeze | grep pkg-core
pkg-core @ file:///home/python/pdm-example-monorepo/packages/pkg-coreWe have a similar use case; we want to install a package in a local directory without editable.
We have several Python packages in a mono repo. Sometimes we need to create a python venv, install those packages into the venv, then pack the venv and distribute it to CI or production environment.
So we want to be able to install a local package like this:
uv pip install ./path/to/package/dirWe don't want to use -e here because the venv will be redistributed.
@kkpattern - that is supported, but you need to include the package name, like “ uv pip install ‘foo @ ./path/to/package/dir’”. We’ll lift this restriction in time, I’m mostly focused on correctness issues right now.
@charliermarsh Yes. We figured that out. We have already switched to uv on our main branch. Massive speed up. Thanks for this awesome tool. (And ruff too!)
I could able to install the package as mentioned above ways. but
uv pip sync requirements.txt is failing. I am running uv pip freeze > requirements.txt to generate the txt file.
error: Couldn't parse requirement in `requirements.txt` at position 384
Caused by: Trailing `(from https://<url>/<package>-0.7.1-py3-none-any.whl)` is not allowed
@shekharidentv - do you mind opening a new issue for this? We may want to strip those URLs from freeze.
👍 for this feature. I'm liking UV so much that I'm using pip install path/to/my.whl (which is slow) and then uv pip install after.
@jkgenser note you can just provide a package name e.g. uv pip install "foo@path/to/my.whl" in the meantime.
@jkgenser note you can just provide a package name e.g.
uv pip install "foo@path/to/my.whl"in the meantime.
Wow amazing thank you that worked! I was having trouble because I saw other examples in github issues that referred to "package @ path/to/my.whl" and it wasn't working me. However removing the spaces was what was needed.
How to install optional deps?
uv pip install xxx@./xxx[yy] dont work and says Distribution not found
How to install optional deps?
uv pip install xxx@./xxx[yy]dont work
I'd guess uv pip install package[extra]@location.
Loving uv so far!
Will supporting this also allow the very basic uv pip install . (non-editable) to work?
Yup!
Publicly committing to this one.
I've a similar issue with a requirements.txt file that contains a package installed as:
-e git+https://github.com/xxx/package.git@commit_id#egg=package package
at the moment uv pip install fails because -e seems to work only for local files
error: Unsupported URL (expected a `file://` scheme) in requirements.txt
Will this syntax be supported? There is an alternative way to install a specific package in requirements in current folder from a github repo?
Thanks
You can do package @ git+https://github.com/xxx/package.git@commit_id, right?
Yes, but in that case it'll be installed under python path and not under current folder.. also with standard pip the behavior is different
We don't support editable installs for Git and URL dependencies. I'd suggest just cloning the repo and installing it as a file-based editable install (-e ./package).
Okay, this is now in review.
These have all merged. Will be supported in the next release.
Thank you so much for making this, makes a world of a difference !