Resolving pre-releases when there are no non pre-releases

Question

Resolving pre-releases when there are no non pre-releases

tevansuk opened this issue 5 years ago · 2 comments

Issue description

When trying to work out a way to use a library (flake8-black) that has a dependency (black) that has no release that is not pre-. The latest flake8-black specifies that it wants black>=19.3b0; the project Pipfile specifies that it wants black==19.3b0. This causes pipenv to look for the latest released version of black, there is no released version and so kabloom.

Expected result

Pipenv should accept the pre-release. According to PEP-440 (and this comment by ncoghlan: #1760 (comment) ), pipenv should accept a matching pre-release when there are no non pre-releases.

Actual result

> pipenv lock --verbose
Creating a virtualenv for this project…
Pipfile: /home/tom/devel/flake8-black-pipenv/Pipfile
Using /usr/bin/python3.6m (3.6.8) to create virtualenv…
⠙ Creating virtual environment...Already using interpreter /usr/bin/python3.6m
Using base prefix '/usr'
New python executable in /home/tom/.local/share/virtualenvs/flake8-black-pipenv-zj-rpos7/bin/python3.6m
Also creating executable in /home/tom/.local/share/virtualenvs/flake8-black-pipenv-zj-rpos7/bin/python
Installing setuptools, pip, wheel...
done.
Running virtualenv with interpreter /usr/bin/python3.6m

✔ Successfully created virtual environment!
Virtualenv location: /home/tom/.local/share/virtualenvs/flake8-black-pipenv-zj-rpos7
Locking [dev-packages] dependencies…
Locking [packages] dependencies…
✘ Locking Failed!
Using pip: -i https://pypi.org/simple
Using pip: -i https://pypi.org/simple
Using pip: -i https://pypi.org/simple

                          ROUND 1
Current constraints:
  black==19.3b0
  flake8-black

Finding the best candidates:
  found candidate black==19.3b0 (constraint was ==19.3b0)
  found candidate flake8-black==0.1.1 (constraint was <any>)

Finding secondary dependencies:
  flake8-black==0.1.1       requires black>=19.3b0, flake8>=3.0.0
  black==19.3b0             requires appdirs, attrs>=18.1.0, click>=6.5, toml>=0.9.4

New dependencies found in this round:
  adding ['appdirs', '', '[]']
  adding ['attrs', '>=18.1.0', '[]']
  adding ['black', '>=19.3b0', '[]']
  adding ['click', '>=6.5', '[]']
  adding ['flake8', '>=3.0.0', '[]']
  adding ['toml', '>=0.9.4', '[]']
Removed dependencies in this round:
Unsafe dependencies in this round:
------------------------------------------------------------
Result of round 1: not stable

                          ROUND 2
Current constraints:
  appdirs
  attrs>=18.1.0
  black==19.3b0,>=19.3b0
  click>=6.5
  flake8>=3.0.0
  flake8-black
  toml>=0.9.4

Finding the best candidates:
  found candidate appdirs==1.4.3 (constraint was <any>)
  found candidate attrs==19.1.0 (constraint was >=18.1.0)
Traceback (most recent call last):
  File "/home/tom/.local/pipx/venvs/pipenv/lib/python3.6/site-packages/pipenv/utils.py", line 385, in resolve
    results = self.resolver.resolve(max_rounds=environments.PIPENV_MAX_ROUNDS)
  File "/home/tom/.local/pipx/venvs/pipenv/lib/python3.6/site-packages/pipenv/patched/piptools/resolver.py", line 102, in resolve
    has_changed, best_matches = self._resolve_one_round()
  File "/home/tom/.local/pipx/venvs/pipenv/lib/python3.6/site-packages/pipenv/patched/piptools/resolver.py", line 198, in _resolve_one
_round
    best_matches = {self.get_best_match(ireq) for ireq in constraints}
  File "/home/tom/.local/pipx/venvs/pipenv/lib/python3.6/site-packages/pipenv/patched/piptools/resolver.py", line 198, in <setcomp>
    best_matches = {self.get_best_match(ireq) for ireq in constraints}
  File "/home/tom/.local/pipx/venvs/pipenv/lib/python3.6/site-packages/pipenv/patched/piptools/resolver.py", line 263, in get_best_mat
ch
    best_match = self.repository.find_best_match(ireq, prereleases=self.prereleases)
  File "/home/tom/.local/pipx/venvs/pipenv/lib/python3.6/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 175, in fin
d_best_match
    raise NoCandidateFound(ireq, all_candidates, self.finder)
pipenv.patched.piptools.exceptions.NoCandidateFound: Could not find a version that matches black==19.3b0,>=19.3b0
Skipped pre-versions: 18.3a0, 18.3a0, 18.3a1, 18.3a1, 18.3a2, 18.3a2, 18.3a3, 18.3a3, 18.3a4, 18.3a4, 18.4a0, 18.4a0, 18.4a1, 18.4a1,
18.4a2, 18.4a2, 18.4a3, 18.4a3, 18.4a4, 18.4a4, 18.5b0, 18.5b0, 18.5b1, 18.5b1, 18.6b0, 18.6b0, 18.6b1, 18.6b1, 18.6b2, 18.6b2, 18.6b3
, 18.6b3, 18.6b4, 18.6b4, 18.9b0, 18.9b0, 19.3b0, 19.3b0
There are incompatible versions in the resolved dependencies.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/tom/.local/pipx/venvs/pipenv/lib/python3.6/site-packages/pipenv/resolver.py", line 126, in <module>
    main()
  File "/home/tom/.local/pipx/venvs/pipenv/lib/python3.6/site-packages/pipenv/resolver.py", line 119, in main
    parsed.requirements_dir, parsed.packages)
  File "/home/tom/.local/pipx/venvs/pipenv/lib/python3.6/site-packages/pipenv/resolver.py", line 85, in _main
    requirements_dir=requirements_dir,
  File "/home/tom/.local/pipx/venvs/pipenv/lib/python3.6/site-packages/pipenv/resolver.py", line 69, in resolve
    req_dir=requirements_dir
  File "/home/tom/.local/pipx/venvs/pipenv/lib/python3.6/site-packages/pipenv/utils.py", line 726, in resolve_deps
    req_dir=req_dir,
  File "/home/tom/.local/pipx/venvs/pipenv/lib/python3.6/site-packages/pipenv/utils.py", line 480, in actually_resolve_deps
    resolved_tree = resolver.resolve()
  File "/home/tom/.local/pipx/venvs/pipenv/lib/python3.6/site-packages/pipenv/utils.py", line 395, in resolve
    raise ResolutionFailure(message=str(e))
pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version that matches black==19.3b0,>=19.3b0
Skipped pre-versions: 18.3a0, 18.3a0, 18.3a1, 18.3a1, 18.3a2, 18.3a2, 18.3a3, 18.3a3, 18.3a4, 18.3a4, 18.4a0, 18.4a0, 18.4a1, 18.4a1,
18.4a2, 18.4a2, 18.4a3, 18.4a3, 18.4a4, 18.4a4, 18.5b0, 18.5b0, 18.5b1, 18.5b1, 18.6b0, 18.6b0, 18.6b1, 18.6b1, 18.6b2, 18.6b2, 18.6b3, 18.6b3, 18.6b4, 18.6b4, 18.9b0, 18.9b0, 19.3b0, 19.3b0
There are incompatible versions in the resolved dependencies.
Using pip: -i https://pypi.org/simple
Using pip: -i https://pypi.org/simple

                          ROUND 1
Current constraints:
  black==19.3b0
  flake8-black

Finding the best candidates:
  found candidate black==19.3b0 (constraint was ==19.3b0)
  found candidate flake8-black==0.1.1 (constraint was <any>)

Finding secondary dependencies:
  flake8-black==0.1.1       requires black>=19.3b0, flake8>=3.0.0
  black==19.3b0             requires appdirs, attrs>=18.1.0, click>=6.5, toml>=0.9.4

New dependencies found in this round:
  adding ['appdirs', '', '[]']
  adding ['attrs', '>=18.1.0', '[]']
  adding ['black', '>=19.3b0', '[]']
  adding ['click', '>=6.5', '[]']
  adding ['flake8', '>=3.0.0', '[]']
  adding ['toml', '>=0.9.4', '[]']
Removed dependencies in this round:
Unsafe dependencies in this round:
------------------------------------------------------------
Result of round 1: not stable

                          ROUND 2
Current constraints:
  appdirs
  attrs>=18.1.0
  black==19.3b0,>=19.3b0
  click>=6.5
  flake8>=3.0.0
  flake8-black
  toml>=0.9.4

Finding the best candidates:
  found candidate appdirs==1.4.3 (constraint was <any>)
  found candidate attrs==19.1.0 (constraint was >=18.1.0)
Traceback (most recent call last):
  File "/home/tom/.local/pipx/venvs/pipenv/lib/python3.6/site-packages/pipenv/utils.py", line 385, in resolve
    results = self.resolver.resolve(max_rounds=environments.PIPENV_MAX_ROUNDS)
  File "/home/tom/.local/pipx/venvs/pipenv/lib/python3.6/site-packages/pipenv/patched/piptools/resolver.py", line 102, in resolve
    has_changed, best_matches = self._resolve_one_round()
  File "/home/tom/.local/pipx/venvs/pipenv/lib/python3.6/site-packages/pipenv/patched/piptools/resolver.py", line 198, in _resolve_one_round
    best_matches = {self.get_best_match(ireq) for ireq in constraints}
  File "/home/tom/.local/pipx/venvs/pipenv/lib/python3.6/site-packages/pipenv/patched/piptools/resolver.py", line 198, in <setcomp>
    best_matches = {self.get_best_match(ireq) for ireq in constraints}
  File "/home/tom/.local/pipx/venvs/pipenv/lib/python3.6/site-packages/pipenv/patched/piptools/resolver.py", line 263, in get_best_match
    best_match = self.repository.find_best_match(ireq, prereleases=self.prereleases)
  File "/home/tom/.local/pipx/venvs/pipenv/lib/python3.6/site-packages/pipenv/patched/piptools/repositories/pypi.py", line 175, in find_best_match
    raise NoCandidateFound(ireq, all_candidates, self.finder)
pipenv.patched.piptools.exceptions.NoCandidateFound: Could not find a version that matches black==19.3b0,>=19.3b0
Skipped pre-versions: 18.3a0, 18.3a0, 18.3a1, 18.3a1, 18.3a2, 18.3a2, 18.3a3, 18.3a3, 18.3a4, 18.3a4, 18.4a0, 18.4a0, 18.4a1, 18.4a1,
18.4a2, 18.4a2, 18.4a3, 18.4a3, 18.4a4, 18.4a4, 18.5b0, 18.5b0, 18.5b1, 18.5b1, 18.6b0, 18.6b0, 18.6b1, 18.6b1, 18.6b2, 18.6b2, 18.6b3, 18.6b3, 18.6b4, 18.6b4, 18.9b0, 18.9b0, 19.3b0, 19.3b0
There are incompatible versions in the resolved dependencies.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/tom/.local/pipx/venvs/pipenv/lib/python3.6/site-packages/pipenv/resolver.py", line 126, in <module>
    main()
  File "/home/tom/.local/pipx/venvs/pipenv/lib/python3.6/site-packages/pipenv/resolver.py", line 119, in main
    parsed.requirements_dir, parsed.packages)
  File "/home/tom/.local/pipx/venvs/pipenv/lib/python3.6/site-packages/pipenv/resolver.py", line 85, in _main
    requirements_dir=requirements_dir,
  File "/home/tom/.local/pipx/venvs/pipenv/lib/python3.6/site-packages/pipenv/resolver.py", line 69, in resolve
    req_dir=requirements_dir
  File "/home/tom/.local/pipx/venvs/pipenv/lib/python3.6/site-packages/pipenv/utils.py", line 726, in resolve_deps
    req_dir=req_dir,
  File "/home/tom/.local/pipx/venvs/pipenv/lib/python3.6/site-packages/pipenv/utils.py", line 480, in actually_resolve_deps
    resolved_tree = resolver.resolve()
  File "/home/tom/.local/pipx/venvs/pipenv/lib/python3.6/site-packages/pipenv/utils.py", line 395, in resolve
    raise ResolutionFailure(message=str(e))
pipenv.exceptions.ResolutionFailure: ERROR: ERROR: Could not find a version that matches black==19.3b0,>=19.3b0
Skipped pre-versions: 18.3a0, 18.3a0, 18.3a1, 18.3a1, 18.3a2, 18.3a2, 18.3a3, 18.3a3, 18.3a4, 18.3a4, 18.4a0, 18.4a0, 18.4a1, 18.4a1,
18.4a2, 18.4a2, 18.4a3, 18.4a3, 18.4a4, 18.4a4, 18.5b0, 18.5b0, 18.5b1, 18.5b1, 18.6b0, 18.6b0, 18.6b1, 18.6b1, 18.6b2, 18.6b2, 18.6b3, 18.6b3, 18.6b4, 18.6b4, 18.9b0, 18.9b0, 19.3b0, 19.3b0
There are incompatible versions in the resolved dependencies.

Steps to replicate

[requires]
python_version = "3.6"

[packages]
flake8-black = "*"
black = "==19.3b0"

$ pipenv --support

[28/395]

Pipenv version: '2018.11.26'

Pipenv location: '/home/tom/.local/pipx/venvs/pipenv/lib/python3.6/site-packages/pipenv'

Python location: '/home/tom/.local/pipx/venvs/pipenv/bin/python'

Python installations found:

3.6.8: /usr/bin/python3.6m
3.6.8: /usr/bin/python3
2.7.15: /usr/bin/python

PEP 508 Information:

{'implementation_name': 'cpython',
 'implementation_version': '3.6.8',
 'os_name': 'posix',
 'platform_machine': 'x86_64',
 'platform_python_implementation': 'CPython',
 'platform_release': '4.15.0-55-generic',
 'platform_system': 'Linux',
 'platform_version': '#60-Ubuntu SMP Tue Jul 2 18:22:20 UTC 2019',
 'python_full_version': '3.6.8',
 'python_version': '3.6',
 'sys_platform': 'linux'}

System environment variables:

CLUTTER_IM_MODULE
LS_COLORS
LANG
GDM_LANG
LESS
DISPLAY
OLDPWD
EDITOR
COMPIZ_CONFIG_PROFILE
GTK_OVERLAY_SCROLLING
COLORTERM
XDG_VTNR
SSH_AUTH_SOCK
MANDATORY_PATH
XDG_SESSION_ID
XDG_GREETER_DATA_DIR
USER
PAGER
DESKTOP_SESSION
QT4_IM_MODULE
TEXTDOMAINDIR
DEFAULTS_PATH
QT_QPA_PLATFORMTHEME
PWD
HOME
TEXTDOMAIN
SSH_AGENT_PID
QT_ACCESSIBILITY
TMUX
XDG_SESSION_TYPE
XDG_DATA_DIRS
MATE_DESKTOP_SESSION_ID
XDG_SESSION_DESKTOP
QT_SCALE_FACTOR
LOCALDOMAIN
GTK_MODULES
GLOBIGNORE
UBUNTU_MENUPROXY
VTE_VERSION
TERM
SHELL
MYSQL_HISTFILE
XDG_SEAT_PATH
QT_IM_MODULE
XMODIFIERS
IM_CONFIG_PHASE
XDG_CURRENT_DESKTOP
GPG_AGENT_INFO
PYTHONSTARTUP
TMUX_PANE
XDG_SEAT
SHLVL
LANGUAGE
PROMPT_COMMAND
WINDOWID
GDMSESSION
LOGNAME
DBUS_SESSION_BUS_ADDRESS
XDG_RUNTIME_DIR
XAUTHORITY
XDG_SESSION_PATH
QT_AUTO_SCREEN_SCALE_FACTOR
XDG_CONFIG_DIRS
PATH
PS1
PS2
SVNROOT
SESSION_MANAGER
GTK_IM_MODULE
_
PIP_DISABLE_PIP_VERSION_CHECK
PYTHONDONTWRITEBYTECODE
PIP_SHIMS_BASE_MODULE
PIP_PYTHON_PATH
PYTHONFINDER_IGNORE_UNSUPPORTED

Pipenv–specific environment variables:

Debug–specific environment variables:

PATH: /home/tom/bin:/home/tom/go/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/games:/usr/local/sbin:/usr/local/bin:/usr/X11R6/bin:/home/te/bin:/home/tom/.local/bin:/home/tom/.local/bin
SHELL: /bin/bash
EDITOR: vim
LANG: en_GB.UTF-8
PWD: /home/tom/devel/flake8-black-pipenv

Contents of Pipfile ('/home/tom/devel/flake8-black-pipenv/Pipfile'):

[requires]
python_version = "3.6"

[packages]
flake8-black = "*"
black = "==19.3b0"

Related: peterjc/flake8-black#18

Answer 1 · 2019-09-03T12:32:12.000Z

Also, I find the first of these lines misleading:

pipenv.patched.piptools.exceptions.NoCandidateFound: Could not find a version that matches black==19.3b0,>=19.3b0
Skipped pre-versions: 18.3a0, 18.3a0, 18.3a1, 18.3a1, 18.3a2, 18.3a2, 18.3a3, 18.3a3, 18.3a4, 18.3a4, 18.4a0, 18.4a0, 18.4a1, 18.4a1,
18.4a2, 18.4a2, 18.4a3, 18.4a3, 18.4a4, 18.4a4, 18.5b0, 18.5b0, 18.5b1, 18.5b1, 18.6b0, 18.6b0, 18.6b1, 18.6b1, 18.6b2, 18.6b2, 18.6b3
, 18.6b3, 18.6b4, 18.6b4, 18.9b0, 18.9b0, 19.3b0, 19.3b0
There are incompatible versions in the resolved dependencies.

Ideally Could not find a version that matches black==19.3b0,>=19.3b0 could be simplified to Could not find a version that matches black==19.3b0 since black==19.3b0 is compatible with and narrower than black>=19.3b0.

Answer 2 · 2022-09-12T02:21:04.000Z

I believe even pip will install the pre-release if you specify that exact version, without the --pre flag. With the latest versions of pipenv it would not resolve unless you had specified that exact version, or provided the --pre flag.