Unbreak the build when pyenv is present
dimpase opened this issue · 29 comments
The build fails when pyenv is present, as reported in report and also in #32749/#33103.
We fix it by rejecting the pyenv shims for sage-bootstrap-python.
Tickets:
- #29461 sage --gdb not working
- #32531:
SAGE_ROOT/tox.ini: Add variants that provision a system python usingpyenv
CC: @embray @mkoeppe @slel @tobiasdiez @jhpalmieri
Component: build: configure
Author: Matthias Koeppe
Branch/Commit: 376932f
Reviewer: John Palmieri
Issue created by migration from https://trac.sagemath.org/ticket/29285
Who's doing something with pyenv?
I don't think we need to provide work-arounds for every conceivable user environment misconfiguration.
Replying to @embray:
Who's doing something with pyenv?
https://dblp.org/pers/k/Kruppa:Alexander.html
I don't think we need to provide work-arounds for every conceivable user environment misconfiguration.
Pyenv creates major issues with packages sometimes, I can imagine what Sagemath might be going about. My easiest fix would be symlinking the two paths together which allows communication, between your bin and other paths, it did work for me when I was trying simple builds, of course now I use Sagemath as an application separately.
Pyenv is popular with MacOs users, so this will definitely need a fix
In fact, the breaking of the path isn't something restricted to Sage, it existed with pip too, where pip was not able to define the paths correctly for the packages to go in
Description changed:
---
+++
@@ -1,3 +1,6 @@
we had seen at least one [report](https://groups.google.com/d/msg/sage-devel/-cGmntUHXqE/Pw02KLksCgAJ) where [pyenv](https://github.com/pyenv/pyenv) broke the build, as pyenv'd Python prepends stuff like `/usr/bin` into `PATH`.
It might be a good idea to check this at configure time.
+
+Tickets:
+- #29461 sage --gdb not workingReplying to @embray:
Who's doing something with pyenv?
I do. Because I tried installing a standalone cygdb on Arch Linux for 3 days and only succeeded when I used pyenv, because I wanted two different python versions on my machine: A normal one and one compiled with --with-pydebug. Achieving this without pyenv is very hard (on Arch Linux).
I don't think we need to provide work-arounds for every conceivable user environment misconfiguration.
If it is too hard or not possible to make sagemath work with pyenv that is ok. But there should be a good error message, because no one looks at the message:
ImportError: /home/volker/.pyenv/versions/3.8.1-debug/lib/python3.8/lib-dynload/math.cpython-38d-x86_64-linux-gnu.so: undefined symbol: PyFloat_Type
/tmp/tmppfahgne0:19: Error in sourced command file:
Error while executing Python code.
/home/volker/Sync/git/sage/src/bin/sage-gdb-commands:1: Error in sourced command file:
And thinks: "Ok, it looks like I need to uninstall pyenv." An error message similar to this:
"It looks like you installed pyenv. Sagemath is incompatible with pyenv. Please uninstall pyenv."
can save someone hours of work. If mkoeppe hadn't added me to the Cc, I would be going through the source code of sagemath right now to find the error. That would have costed me multiple days.
In General, I think it is really important that misconfigurations result in nice error messages.
Note: My issue can be found here: #29461
I'm checking this with the up to date pyenv, and it seems that the issue with the PATH is no longer. Currently building latest beta with python 3.9-dev from pyenv on Debian stable, will report the results.
I was able to successfully install Sage using python 3.9 from pyenv (installed following
https://github.com/pyenv/pyenv#basic-github-checkout) on Debian 10.
Perhaps one should check this on Homebrew, but it seems it's OK.
Can be fixed as worksforme, that is.
Let us close this if nobody objects.
Reviewer: Samuel Lelièvre
Description changed:
---
+++
@@ -4,3 +4,4 @@
Tickets:
- #29461 sage --gdb not working
+- #32531: `SAGE_ROOT/tox.ini`: Add variants that provision a system python using `pyenv`Description changed:
---
+++
@@ -1,4 +1,8 @@
we had seen at least one [report](https://groups.google.com/d/msg/sage-devel/-cGmntUHXqE/Pw02KLksCgAJ) where [pyenv](https://github.com/pyenv/pyenv) broke the build, as pyenv'd Python prepends stuff like `/usr/bin` into `PATH`.
+
+It is documented in
+https://github.com/pyenv/pyenv#managing-virtual-environments
+that `pyenv` breaks everything regarding `venv`s.
It might be a good idea to check this at configure time.
Test on macOS with pyenv from homebrew:
$ brew install pyenv
$ pyenv install 3.7.8
$ eval "$(pyenv init --path)"
$ echo $PATH
/Users/mkoeppe/.pyenv/shims:/Users/mkoeppe/miniconda3/condabin:/Users/mkoeppe/bin:/usr/local/bin:/Users/mkoeppe/google-cloud-sdk/bin:/Users/mkoeppe/perl5/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/opt/X11/bin
$ pyenv global 3.7.8
$ ./configure --enable-download-from-upstream-url --with-python=python3
(--with-python=python3 is needed here because otherwise we reject it as misconfigured - see #32531 comment:3)
pip's self-installation gets tricked into installing in the wrong place:
[pip-21.3.1] Using pip 21.3.1 from /Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/var/lib/sage/venv-python3.7/var/tmp/sage/build/pip-21.3.1/src/src/pip (python 3.7)
[pip-21.3.1] Looking in links: /Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/var/lib/sage/venv-python3.7/var/lib/sage/wheels
[pip-21.3.1] Processing /Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/var/lib/sage/venv-python3.7/var/tmp/sage/build/pip-21.3.1/inst/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/var/lib/sage/venv-python3.7/var/lib/sage/wheels/pip-21.3.1-py3-none-any.whl
[pip-21.3.1] Installing collected packages: pip
[pip-21.3.1] changing mode of /Users/mkoeppe/.pyenv/versions/3.7.8/bin/pip to 755
[pip-21.3.1] changing mode of /Users/mkoeppe/.pyenv/versions/3.7.8/bin/pip3 to 755
[pip-21.3.1] changing mode of /Users/mkoeppe/.pyenv/versions/3.7.8/bin/pip3.7 to 755
[pip-21.3.1] Successfully installed pip-21.3.1
[pip-21.3.1]
sage-pip-install -> sage-flock -> sage-bootstrap-python -> $HOME/.pyenv/shims -> $HOME/.pyenv/versions/3.7.8/bin/python3; the python3 shim manipulates PATH.
(sage-buildsh) mkoeppe@egret:pip-21.3.1$ python3 -c 'import os; print(os.environ["PATH"])'
/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/src/bin:/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/libexec/ccache:/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/build/bin:/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/var/lib/sage/venv-python3.7/bin:/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/bin:/Users/mkoeppe/.pyenv/shims:/Users/mkoeppe/miniconda3/condabin:/Users/mkoeppe/bin:/usr/local/bin:/Users/mkoeppe/google-cloud-sdk/bin:/Users/mkoeppe/perl5/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/opt/X11/bin
(sage-buildsh) mkoeppe@egret:pip-21.3.1$ sage-bootstrap-python -c 'import os; print(os.environ["PATH"])'
/Users/mkoeppe/.pyenv/versions/3.7.8/bin:/usr/local/Cellar/pyenv/2.2.3/libexec:/usr/local/Cellar/pyenv/2.2.3/plugins/python-build/bin:/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/src/bin:/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/libexec/ccache:/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/build/bin:/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/var/lib/sage/venv-python3.7/bin:/Users/mkoeppe/s/sage/sage-rebasing/worktree-gcc11/local/bin:/Users/mkoeppe/.pyenv/shims:/Users/mkoeppe/miniconda3/condabin:/Users/mkoeppe/bin:/usr/local/bin:/Users/mkoeppe/google-cloud-sdk/bin:/Users/mkoeppe/perl5/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/opt/X11/bin
Description changed:
---
+++
@@ -1,10 +1,6 @@
-we had seen at least one [report](https://groups.google.com/d/msg/sage-devel/-cGmntUHXqE/Pw02KLksCgAJ) where [pyenv](https://github.com/pyenv/pyenv) broke the build, as pyenv'd Python prepends stuff like `/usr/bin` into `PATH`.
+The build fails when [pyenv](https://github.com/pyenv/pyenv) is present, as reported in [report](https://groups.google.com/d/msg/sage-devel/-cGmntUHXqE/Pw02KLksCgAJ) and also in #32749/#33103.
-It is documented in
-https://github.com/pyenv/pyenv#managing-virtual-environments
-that `pyenv` breaks everything regarding `venv`s.
-
-It might be a good idea to check this at configure time.
+We fix it by rejecting the pyenv shims for `sage-bootstrap-python`.
Tickets:
- #29461 sage --gdb not workingAuthor: Matthias Koeppe
Changed reviewer from Samuel Lelièvre to none
Let's get this into 9.5 please
Changed reviewer from https://github.com/mkoeppe/sage/runs/4682289891?check_suite_focus=true to John Palmieri
Works for me on OS X with homebrew's pyenv: without this branch, sage-bootstrap runs the "wrong" python3, while with this branch, it runs the right python3.
Thank you!
Changed branch from u/mkoeppe/unbreak_the_build_when_pyenv_is_present to 376932f