PERF: optional pip install?

Question

PERF: optional pip install?

henryiii opened this issue 9 months ago · 12 comments

I've been looking at UV, and wondered if we could get a speed up with virtualenv/venv by checking for pip in build's environment. If pip is present and of reasonable version, we could skip installing it into the isolated environments and use --python to run the outer pip command against the inner environment. That saves at least 200ms per environment, and maybe a lot more if pip has to be downloaded. If pip is not present, then we do what we do now.

Thoughts?

Answer 1 · 2024-02-18T14:22:54.000Z

Sounds reasonable. We could also look into supporting uv venv as an alternative backend.

Answer 2 · 2024-02-18T17:08:42.000Z

astral-sh/uv#1510 It would be nice if we could get info about the created venv back as json. We want a couple of paths.

Answer 3 · 2024-02-18T22:50:39.000Z

Didn't you need that same information for wntrblm/nox#762 - how did you make it work?

Answer 4 · 2024-02-19T17:55:33.000Z

I believe that hard codes the bin/Script search based on platform.

Answer 5 · 2024-02-19T18:53:49.000Z

Might be good enough for us too :) Alternatively, we could use the same logic we do for the stdlib venv module.

Answer 6 · 2024-02-19T20:36:55.000Z

How would we handle opting-in/out? Should we add a --backend=uv option (along with venv and virtualenv)? Or an environment variable? (Or both?). I don't think we want to automatically switch based on the presence of a uv command, like we do with virtualenv being an installed module.

Answer 7 · 2024-02-19T21:01:32.000Z

I would be in favour of a CLI option, but I know we've been reluctant to add more options. The default can be kept dynamic between venv and virtualenv and would be reflected in the CLI's help text, which I think is a plus. Prolly a good idea to call the new option something other than "backend" though to avoid confusion with build backend - --venv-impl(ementation) perhaps?

Answer 8 · 2024-02-19T21:05:46.000Z

I meant to mention that "backend" was just a placeholder because I was unimaginative, but forgot. What about --venv=venv, --venv=virtualenv, --venv=uv? With the default basically being --venv=virtualenv|venv.

Answer 9 · 2024-02-19T21:15:59.000Z

I'm okay with that. How would the new option interact with -n? Would they be mutually exclusive? Should --venv accept none with -n becoming a synonym of --venv=none?

Answer 10 · 2024-02-19T21:21:42.000Z

Why can't we just embed creating the virtualenv and not worry about virtualenv and instead just have controls over the installer? We'd need to write out:

.venv
├── bin
│   ├── python -> /usr/local/opt/python@3.12/bin/python3.12
│   ├── python3 -> python
│   └── python3.12 -> python
├── lib
│   └── python3.12
│       └── site-packages
│           ├── _virtualenv.pth
│           └── _virtualenv.py
└── pyvenv.cfg

Doesn't seem that hard - I think that's basically what uv is doing. Unless there's something tricky I'm missing. Then we could drop venv, which would be nice. We don't need activators (which we turn off anyway), etc.

Then we just set the installer command (pip vs. uv pip). That also doesn't interfere with -n. :)

Answer 11 · 2024-02-19T21:28:58.000Z

Ahh, I guess the problem then is that we really do need external pip if we were to make it ourselves (or ensurepip), we do leave embedded pip turned on.

Answer 12 · 2024-02-19T21:32:13.000Z

I'm okay with that. How would the new option interact with -n? Would they be mutually exclusive? Should --venv accept none with -n becoming a synonym of --venv=none?

That makes sense, I think.