Surprising solution chosen by 2020-resolver
Closed this issue · 12 comments
We have this situation
- pythran 0.9.5 requires "gast"
There was a new release of gast (0.4.0) but no released pythran versions are compatible with the new gast so the most recent pythran version (0.9.6) pinned the gast version to 0.3.3.
- pythran 0.9.6 requires gast~=0.3.3
- transonic 0.4.4 is compatible with both gast 0.3.3 and 0.4.0 so it only requires "gast"
when installing transonic and pythran with pip, the correct versions to install should be
transonic 0.4.0, pythran 0.9.6 and gast 0.3.3
However, pip install transonic pythran --use-feature=2020-resolver chooses
transonic 0.4.0, pythran 0.9.5 and gast 0.4.0
I understand this choice, which seems to be OK but it's not actually working because, even though pythran 0.9.5 officially requires "gast" (any versions of gast), it is not compatible with gast 0.4.0.
The log of what I did
$ python -m venv /tmp/venv_pip
$ source-bash /tmp/venv_pip/bin/activate
(venv_pip) $ pip install --upgrade pip
Collecting pip
Using cached https://files.pythonhosted.org/packages/5a/4a/39400ff9b36e719bdf8f31c99fe1fa7842a42fa77432e584f707a5080063/pip-20.2.2-py2.py3-none-any.whl
Installing collected packages: pip
Found existing installation: pip 19.2.3
Uninstalling pip-19.2.3:
Successfully uninstalled pip-19.2.3
Successfully installed pip-20.2.2
(venv_pip) $ pip install -U "pip @ https://github.com/pypa/pip/archive/master.zip"
Collecting pip@ https://github.com/pypa/pip/archive/master.zip
Downloading https://github.com/pypa/pip/archive/master.zip (9.1 MB)
|████████████████████████████████| 9.1 MB 1.7 MB/s
Installing build dependencies ... done
Getting requirements to build wheel ... done
Preparing wheel metadata ... done
Building wheels for collected packages: pip
Building wheel for pip (PEP 517) ... done
Created wheel for pip: filename=pip-20.3.dev0-py2.py3-none-any.whl size=1504298 sha256=0da4eac67eca2e681405b90a81e75dbc4354166b1ddcf3a9bb9b4efd179f6cb9
Stored in directory: /tmp/pip-ephem-wheel-cache-57w93zk8/wheels/b2/f0/ae/286fb76d950bd0a0d20bcabbda0f56531389ed5030f038f6b9
Successfully built pip
Installing collected packages: pip
Attempting uninstall: pip
Found existing installation: pip 20.2.2
Uninstalling pip-20.2.2:
Successfully uninstalled pip-20.2.2
Successfully installed pip-20.3.dev0
(venv_pip) $ pip --version
pip 20.3.dev0 from /tmp/venv_pip/lib/python3.8/site-packages/pip (python 3.8)
(venv_pip) $ pip install transonic pythran --use-feature=2020-resolver
Collecting transonic
Downloading transonic-0.4.4-py3-none-any.whl (82 kB)
|████████████████████████████████| 82 kB 764 kB/s
Collecting astunparse>=1.6.3
Using cached astunparse-1.6.3-py2.py3-none-any.whl (12 kB)
Collecting beniget
Using cached beniget-0.3.0-py3-none-any.whl (9.3 kB)
Collecting gast
Using cached gast-0.4.0-py3-none-any.whl (9.8 kB)
Collecting six<2.0,>=1.6.1
Using cached six-1.15.0-py2.py3-none-any.whl (10 kB)
Collecting wheel<1.0,>=0.23.0
Using cached wheel-0.35.1-py2.py3-none-any.whl (33 kB)
Collecting pythran
Using cached pythran-0.9.6-py3-none-any.whl (4.4 MB)
Processing ./.cache/pip/wheels/e4/66/c1/1a5e2c8a2bb67eedaa1e9a2e004176d7116c6fdf2ec5a3e51b/pythran-0.9.5-py3-none-any.whl
Collecting ply>=3.4
Using cached ply-3.11-py2.py3-none-any.whl (49 kB)
Collecting networkx>=2
Using cached networkx-2.5-py3-none-any.whl (1.6 MB)
Collecting decorator
Using cached decorator-4.4.2-py2.py3-none-any.whl (9.2 kB)
Collecting numpy
Using cached numpy-1.19.1-cp38-cp38-manylinux2010_x86_64.whl (14.5 MB)
Processing ./.cache/pip/wheels/5f/b4/93/a30fa45720240e1106155083bac98146542dd614e1e6ce12da/autopep8-1.5.4-py2.py3-none-any.whl
Collecting pycodestyle>=2.6.0
Using cached pycodestyle-2.6.0-py2.py3-none-any.whl (41 kB)
Collecting toml
Using cached toml-0.10.1-py2.py3-none-any.whl (19 kB)
Installing collected packages: wheel, toml, six, pycodestyle, gast, decorator, ply, numpy, networkx, beniget, autopep8, astunparse, transonic, pythran
Successfully installed astunparse-1.6.3 autopep8-1.5.4 beniget-0.3.0 decorator-4.4.2 gast-0.4.0 networkx-2.5 numpy-1.19.1 ply-3.11 pycodestyle-2.6.0 pythran-0.9.5 six-1.15.0 toml-0.10.1 transonic-0.4.4 wheel-0.35.1
The resolver can only work with what the packages declare, and would always need further instructions from the user, since even if it knows the current solution is wrong, it can’t possibly know the solution you expect is correct.
This is unfortunately a difficult problem. There was a discussion (IIRC on Twitter but can’t find right now) to develop a separate database to provide curated dependency information to retroactively fix the dependency declaration after a package is released (so pip can tell pythran 0.9.5 is not actually compatibile with gast 0.4.0 even if the declaration in package metadata says so), but that requires much administrative resources that PyPA do not have right now, unfortunately.
One thing pip probably can do is to support a way to let the user say “don’t ever install this package combination even if their dependency declaration allows for it,” so you can do something like
$ echo '{"conflicts": [["pythran<0.9.6", "gast>=0.4.0"]]}' > conflicts.json
$ pip install --conflicts=./conflicts.json pythranand the resolver would avoid that combination (thus landing on the “correct” solution).
(Personal speculation: Would it actually work as a startup idea to provide maintenance work to a giant incompatibility mapping? So customers can pip install stuff faster and more accurately. Sounds like many commercial entities would be interested?)
What pip can do though, is to give the users choices and/or provide a way to specify requirement priority (e.g. does one prefer the more updated gast or pythran?). Personally as an user I'd prefer to the prior solution, basically in the form of
- I give pip a list of requirements
- pip proceed into resolution it and hand me the first result it can find
- I choose between installing, trying to resolve for a better one or canceling the installation
- If I chose to cancel, pip quits
- If I chose to re-resolve, go back to step 2
- Install the confirm resolution results
This will basically allow:
- Users to be aware of additional packages to be installed
- Automated resolution by hand that best corporate the power of machines and human (what a way to describe software 😄)
Edit: I understand that the confirmation thingy would change pip's UX down to conceptual level, but if we have any plan to support smart uninstallation and upgrading, allow users to intervene is likely to be the best thing we can do for less trivial cases.
I feel the interactive approach would be too overwhelming and only confuses users most of the time. The “resolution plan” is straightforward enough here, but gets complex very quickly when there are more possible solutions with more packages and deeper dependencies (since users don’t care about most of the priority choices the resolver makes). And even accepting that, this only switches the complaint to become why pip not shows this better solution before the other ones I don’t want.
IMO it would be better UX the other way around, for users to provide the priority information to pip install. Combining with a confirmation prompt after resolution (apt and conda etc. do that IIRC) so the user can abort and revise the priority information if they don’t like the plan pip generates. But how that priority information can be described is a whole other difficult problem.
From the information that pip has (dependencies and release dates), a human can guess that pythran 0.9.5 could be incompatible with gast 0.4.0 (because (i) pythran 0.9.6 is marked as only compatible with gast 0.3.3, (ii) gast 0.4 has been released after pythran 0.9.5 and (iii) 0.9.6 > 0.9.5). Therefore it seems to me that a human can guess that the solution (transonic 0.4.0, pythran 0.9.6 and gast 0.3.3) is better than the solution (transonic 0.4.0, pythran 0.9.5 and gast 0.4.0).
So I'm wondering if pip could order different solutions and to choose the "best one".
In this case, the solution with pythran 0.9.6 could also be considered as "better" because it includes a newer version of a package directly asked by the user.
@uranusjr, I see your point; still not all the time the users can be aware of all possible solutions, and there should be a way to get all alternatives. I'm not exactly sure if we should have that as pip's default behavior (e.g. in Debian apt/apt-get does not show alternatives but aptitude does). Users will always complain (source: I am an user of multiple things myself), but making something possible yet inconvenient is much better than impossible/clueless. I'm not pushing on this though because I know that the dependency resolution algo is not stabilized (there's a near future plan for pubgrub?) and that there are many concerns to be resolved before we want to bang our heads against the wall trying to sort solution alternatives intuitively.
@paugier, I suppose some human could imply that, some would imply otherwise, depending on the development of the projects. For instance, A 4.2 could be using something that is removed in B 34, but A 4.1 doesn't, so both (A 4.2, B 33) and (A 4.1, B 34) would be correct solutions.
Does the new resolver even know there are multiple solutions? As far as I'm aware, it currently just stops after the first valid solve and uses it. Checking for alternatives could mean doing a complete solve on a potentially huge dependency tree (for example, if there is only one valid answer, confirming that there isn't a second requires checking everything).
Once the user knows that the original solve is wrong, it's easy¹ to add a requirement like gast < 0.4.0 to fix the issue. The problem is basically 100% around UX:
- How do we give the user a chance to spot there's an issue before they have a load of incorrect files installed in their environment?
- How do they do that in a way that doesn't impact "non-interactive" cases?
- How would the user know the correct constraint to add? (General evidence is that users are not as clear on how requirements work as we'd assume...)
- How do users specify the extra constraint (they might want it as a "one-off", or as "until the next release of some library", or "for everyone in the company", etc, etc)?
¹ For some definition of "easy" 🙂
So I'm wondering if pip could order different solutions and to choose the "best one".
There are prioritisation mechanisms that we can use, but they aren't straightforward (to put it another way, no-one really understands what they'd do - not even the people who wrote the code!) And documenting how pip prioritises solutions would be a nightmare. At the moment, we only document that pip generates a solution that satisfies the constraints, and nothing more. Documenting prioritisation rules means people would start depending on them, and we'd then have to treat those rules as a guarantee (that may stop us changing the resolver in future).
Does the new resolver even know there are multiple solutions? As far as I'm aware, it currently just stops after the first valid solve and uses it. Checking for alternatives could mean doing a complete solve on a potentially huge dependency tree (for example, if there is only one valid answer, confirming that there isn't a second requires checking everything).
Indeed the resolver does not currently know there’s a second solution (or more). It’s not technically difficult to make it check for it (just let it not return eagerly), but I don’t really think it’s a worthy thing to do since the additional resolution can be costly, and that second solution (even found) is not meaningful most of the time anyway.
FWIW, you can restrict the versions that the resolver uses, with constraint files (-c). So, if you don't want gast 0.4.0, you can have gast!=0.4.0 to tell the resolver to not pick that version.
@pradyunsg is this something we need to address for 20.3?
I don't think so, no.
I’m going to close this since there isn’t really much we can do here. Only the user knows pythran 0.9.5 isn’t actually compaible with gast 0.4.0, albeit claiming to say so, the user needs to provide that information manually to the resolver.
This particular scenario is actually no longer an issue, since the new resolver now always resolves user-specified requirements first. So it will always prefer the latest pythran and backtrack on gast if possible, instead of backtracking on pythran to satisfy gast.