pip compile --no-deps fails with extras

[mitsuhiko]

It looks like the --no-deps mode somehow fails with pip-compile when extras are specified.

$ echo 'flask[dotenv]' | uv pip compile - --no-deps
thread 'main' panicked at crates/uv-resolver/src/resolution.rs:136:29:
Every package should have metadata: NameVersion(PackageName("flask"), "3.0.2")
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

vs

$ echo 'flask[dotenv]' | uv pip compile -
# This file was autogenerated by uv via the following command:
#    uv pip compile -
blinker==1.7.0
    # via flask
click==8.1.7
    # via flask
flask==3.0.2
itsdangerous==2.1.2
    # via flask
jinja2==3.1.3
    # via flask
markupsafe==2.1.5
    # via
    #   jinja2
    #   werkzeug
python-dotenv==1.0.1
    # via flask
werkzeug==3.0.1
    # via flask

Happens with uv 0.1.6 and earlier.

[charliermarsh]

Fixing now.

[charliermarsh]

Should be fixed in v0.1.7 (out now).

[rachtsingh]

Is this fixed? I think the final result should be flask[dotenv]==3.0.2, at least that's what pip-compile tells me:

~$ echo 'flask[dotenv]' | pip-compile - --output-file test.in
WARNING: --strip-extras is becoming the default in version 8.0.0. To silence this warning, either use --strip-extras to opt into the new default or use --no-strip-extras to retain the existing behavior.
#
# This file is autogenerated by pip-compile with Python 3.10
# by the following command:
#
#    pip-compile --output-file=test.in -
#
blinker==1.7.0
    # via flask
click==8.1.7
    # via flask
flask[dotenv]==3.0.2
    # via -r -
itsdangerous==2.1.2
    # via flask
jinja2==3.1.3
    # via flask
markupsafe==2.1.5
    # via
    #   jinja2
    #   werkzeug
python-dotenv==1.0.1
    # via flask
werkzeug==3.0.1
    # via flask

with uv (0.1.10):

~ ❯ echo 'flask[dotenv]' | uv pip compile -
Resolved 8 packages in 8ms
# This file was autogenerated by uv via the following command:
#    uv pip compile -
blinker==1.7.0
    # via flask
click==8.1.7
    # via flask
flask==3.0.2
itsdangerous==2.1.2
    # via flask
jinja2==3.1.3
    # via flask
markupsafe==2.1.5
    # via
    #   jinja2
    #   werkzeug
python-dotenv==1.0.1
    # via flask
werkzeug==3.0.1
    # via flask

I think this is the cause of astral-sh/rye#745, and a related JAX issue I filed with rye.

[rachtsingh]

Oh, I see - python-dotenv is included in both places, which is why uv pip install flask[dotenv] works correctly, but rye add flask[dotenv] breaks because it tries to desugar the feature into flask==3.0.2 and python-dotenv==1.0.1, and rye only matches against the first.

I'm not familiar enough with the implications and uv's goals to say this isn't what uv wants, but I would personally prefer if uv pip compile did not expand flask[dotenv] to flask + python-dotenv, at least from rye's perspective (for example if I type rye add flask[dotenv] I want flask[dotenv]==3.0.2 in my pyproject.toml. Maybe there's a higher level API rye could be using instead (I think currently spawning a subprocess)? Let me know if you want me to create a new issue for visibility/clarity.

[charliermarsh]

I believe what uv is doing is correct. It's right to omit the [dotenv] in the output file, since it doesn't have any meaning (the "extras" are already included by way of including the dependencies themselves). You can see with the --strip-extras is becoming the default in version 8.0.0. warning that pip-tools is moving to that default soon (IIUC).

It seems like the issue here is around rye add with extras, which it's using to lock the correct version of a single dependency. I think rye either needs to re-add the extra with some manual parsing, or we'd need a flag or another workflow for it.

[charliermarsh]

I tried rye add "flask[dotenv]" and it added the "right" dependency:

--- a/pyproject.toml
+++ b/pyproject.toml
@@ -30,6 +30,9 @@ classifiers = [
   "Topic :: Software Development :: Libraries",
 ]
 readme = "README.md"
+dependencies = [
+    "flask[dotenv]>=3.0.2",
+]

[rachtsingh]

Huh, I have a different result (maybe it's a zsh issue?):

~/proj/test-proj main ?5 ❯ rye add "flask[dotenv]"
Initializing new virtualenv in /home/singhrac/proj/test-proj/.venv
Python version: cpython@3.12.1
Added flask>=3.0.2 as regular dependency
Reusing already existing virtualenv
Generating production lockfile: /home/singhrac/proj/test-proj/requirements.lock
Generating dev lockfile: /home/singhrac/proj/test-proj/requirements-dev.lock
Installing dependencies
   Built file:///home/singhrac/proj/test-proj                                                                                                                  Built 1 editable in 202ms
Resolved 1 package in 1ms
Downloaded 1 package in 35ms
Installed 8 packages in 7ms
 + blinker==1.7.0
 + click==8.1.7
 + flask==3.0.2
 + itsdangerous==2.1.2
 + jinja2==3.1.3
 + markupsafe==2.1.5
 + test-proj==0.1.0 (from file:///home/singhrac/proj/test-proj)
 + werkzeug==3.0.1
Done!
~/proj/test-proj main ?7 ❯ cat pyproject.toml
[project]
name = "test-proj"
version = "0.1.0"
description = "Add your description here"
authors = [
    { name = "Rachit Singh", email = "rachitsingh@outlook.com" }
]
dependencies = [
    "flask>=3.0.2",
]
readme = "README.md"
requires-python = ">= 3.8"

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[tool.rye]
managed = true
dev-dependencies = []

[tool.hatch.metadata]
allow-direct-references = true

[tool.hatch.build.targets.wheel]
packages = ["src/test_proj"]

Do you have rye configured to use uv by default (i.e. rye config --set-bool behavior.use-uv=true)? I think it works that way when that flag is off (when it uses pip-compile + its semantics, which as you pointed out is going to be deprecated).

[charliermarsh]

Thanks, I had to set the Rye config! It's a Rye issue. I'll open one there.

[charliermarsh]

@rachtsingh - Tracking as astral-sh/rye#745 (existing issue in Rye).