spkg-configure.m4 for python3
Closed this issue · 137 comments
Do not install python3 if a sufficiently new version is available in $PATH. This could be a system python3, or the python3 from within a venv.
As suggested in #29032, we make $SAGE_LOCAL a venv (https://docs.python.org/3/library/venv.html) over the system python3 -- if a suitable system python3 is found -- by using the venv.EnvBuilder API.
This is done by the new script build/bin/sage-venv, which we use both during configure (to make sure that we select a system python3 for which venvs work correctly) and during make.
The venv does not include the system site-packages: We continue to install all Python packages into our venv using the existing build infrastructure. We keep the task of using system site-packages for the follow-up ticket #29023.
(Note, we use venv (new since Python 3.3), not virtualenv. So this change is limited to Python 3 builds of Sage.)
CC: @jdemeyer @embray @vbraun @kiwifb @jhpalmieri @videlec
Component: build: configure
Author: Matthias Koeppe, Erik Bray
Branch: 1c845a4
Reviewer: Dima Pasechnik
Issue created by migration from https://trac.sagemath.org/ticket/27824
Description changed:
---
+++
@@ -2,4 +2,15 @@
At the moment I don't know how feasible it is - one small obstacle might be that we really do not want to mess with two pythons at the same time.
-
+while it should be possible/desirable to use a system python, any additional Python packages should optionally be installable into a virtualenv instead of the system site-packages, including sagelib itself unless the system *happens* to meet all of sagelib's Python dependencies already. Listing some relevant Python packages below:
+* cysignals
+* cython
+* ipython
+* jinja2
+* jupyter_core
+* numpy
+* scipy
+* sympy
+* pip
+* setuptools
+* sixDescription changed:
---
+++
@@ -13,4 +13,5 @@
* sympy
* pip
* setuptools
+* pkgconfig
* sixAs the Sage-8.8 release milestone is pending, we should delete the sage-8.8 milestone for tickets that are not actively being worked on or that still require significant work to move forward. If you feel that this ticket should be included in the next Sage release at the soonest please set its milestone to the next release milestone (sage-8.9).
For Python packages when installing using the system Python we definitely need a well thought out approach, likely with different options for handling questions such as where to install additional Python packages.
$SAGE_LOCAL already acts as a sort-of virtualenv, but we might still consider using virtualenv/venv in $SAGE_LOCAL (with the --system-site-packages option) as a way of extending sys.path while still using as many system Python packages as possible.
See #29013 - "In a python3 build, install all Python packages into a venv"
Description changed:
---
+++
@@ -1,17 +1,5 @@
-do not install pythonX if we are in the virtenv of pythonX.
+Do not install python3 if a sufficiently new version is available in `$PATH`. This could be a system python3, or the python3 from within a venv.
-At the moment I don't know how feasible it is - one small obstacle might be that we really do not want to mess with two pythons at the same time.
+Via #29013, we install all of our own Python packages into a separate venv that includes the system site-packages.
-while it should be possible/desirable to use a system python, any additional Python packages should optionally be installable into a virtualenv instead of the system site-packages, including sagelib itself unless the system *happens* to meet all of sagelib's Python dependencies already. Listing some relevant Python packages below:
-* cysignals
-* cython
-* ipython
-* jinja2
-* jupyter_core
-* numpy
-* scipy
-* sympy
-* pip
-* setuptools
-* pkgconfig
-* six
+Narrowed the ticket to py3.
Here's a version that works. The venv is created in build/make/Makefile target (in #29013, I created the venv in the spkg-postinst script of the python3 spkg.)
New commits:
1363425 | In a python3 build, install all Python packages into a venv |
a394268 | sage-pip-install: Use PYTHON=sage-python23 |
a84815b | spkg-configure.m4 for python3 |
Author: Matthias Koeppe
how about checking for Py3 deps: sqlite libpng bzip2 xz libffi ?
(I skipped in this list ones that are 2nd order, so no need to check them)
On Gentoo Python 3 does not have sqlite module by default. I had to do
$ pip install pysqlite3 --user, and still elliptic_curves spkg installation errors out.
Branch pushed to git repo; I updated commit sha1. New commits:
0d0495f | build/pkgs/python3/spkg-configure.m4: Add SAGE_SPKG_DEPCHECK for sqlite libpng bzip2 xz libffi |
Branch pushed to git repo; I updated commit sha1. New commits:
98dff9d | Remove use of --system-site-packages |
I've reduced the scope of the ticket, excluding the use of system site packages.
Description changed:
---
+++
@@ -1,5 +1,7 @@
Do not install python3 if a sufficiently new version is available in `$PATH`. This could be a system python3, or the python3 from within a venv.
-Via #29013, we install all of our own Python packages into a separate venv that includes the system site-packages.
+Via #29013, we install all of our own Python packages into a separate venv.
+
+The venv does not include the system site-packages; we keep that task for a follow-up ticket.
Branch pushed to git repo; I updated commit sha1. New commits:
6180132 | build/pkgs/python3/spkg-configure.m4: Find @PYTHON_FOR_VENV@ that has sqlite3 module |
I tried this on gentoo (after a usual struggle to get sqlite3 module built in Python)
and it broke at docbuild, with
RuntimeError: libSingular not found--a working Singular install in $SAGE_LOCAL is required for Sage to work
even though libSingular has built just fine (it's some magic with from sage.env import SINGULAR_SO that is broken, so SINGULAR_SO is None)
Thanks for testing!
Replying to @mkoeppe:
I've reduced the scope of the ticket, excluding the use of system site packages.
do you mean pip/etc-installed site-packages of system's Python?
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
61d7822 | Trac #29002: lower the version bound on the system sqlite check. |
3ab4515 | Merge branch 'u/mjo/ticket/29002' of git://trac.sagemath.org/sage into t/27824/public/27824-python3-spkg-configure |
c7b9ae4 | Add sqlite depcheck |
961f850 | Trac #29002: lower the version bound on the system sqlite check (again). |
d321948 | Trac #29002: drop the readline dependency check for system sqlite. |
14d15fb | Merge branch 'u/mjo/ticket/29002' of git://trac.sagemath.org/sage into t/27824/public/27824-python3-spkg-configure |
258d112 | build/pkgs/pillow/patches/setup.py.patch: Don't fail if Py_MACOS_SYSROOT sysconfig variable does not exist |
43673c0 | Merge branch 't/29019/make_patch_to_pillow_more_robust__do_not_depend_on__py_macos_sysroot__' into t/27824/public/27824-python3-spkg-configure |
56b1618 | build/pkgs/pillow/spkg-install: Help setup.py find zlib.h on macOS; show more build output |
5930b4e | Merge branch 't/29019/make_patch_to_pillow_more_robust__do_not_depend_on__py_macos_sysroot__' into t/27824/public/27824-python3-spkg-configure |
Now this is giving
[pillow-5.3.0.p0] gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/usr/include -I/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.15.sdk/System/Library/Frameworks/Tk.framework/Versions/8.5/Headers -g -DHAVE_OPENJPEG -DHAVE_LIBZ -DHAVE_LIBTIFF -DPILLOW_VERSION="5.3.0" -I/usr/local/Cellar/openjpeg/2.3.1/include/openjpeg-2.3 -I/Users/mkoeppe/s/sage/sage-rebasing/worktree-venv/local/var/tmp/sage/build/pillow-5.3.0.p0/src/src/libImaging -I/usr/local/Cellar/jpeg/9c/include -I/usr/local/Cellar/libtiff/4.1.0/include -I/usr/local/Cellar/freetype/2.10.1/include/freetype2 -I/usr/local/Cellar/little-cms2/2.9/include -I/Users/mkoeppe/s/sage/sage-rebasing/worktree-venv/local/include -I/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include -I/Users/mkoeppe/s/sage/sage-rebasing/worktree-venv/local/lib/sage/venv/sage/include -I/usr/local/include -I/usr/local/Cellar/freetype/2.10.1/include -I/usr/local/include -I/usr/local/opt/openssl@1.1/include -I/usr/local/opt/sqlite/include -I/Users/mkoeppe/s/sage/sage-rebasing/worktree-venv/local/lib/sage/venv/sage/include -I/usr/local/Cellar/python/3.7.6_1/Frameworks/Python.framework/Versions/3.7/include/python3.7m -c src/_imaging.c -o build/temp.macosx-10.15-x86_64-3.7/src/_imaging.o
[pillow-5.3.0.p0] error: $MACOSX_DEPLOYMENT_TARGET mismatch: now "10.9" but "10.15" during configure
[pillow-5.3.0.p0] ********************************************************************************
This is coming from the fake MACOSX_DEPLOYMENT_TARGET set in sage-env. See #18272, #7095, #18254, #16312.
Replying to @dimpase:
I tried this on gentoo (after a usual struggle to get sqlite3 module built in Python)
and it broke at docbuild, withRuntimeError: libSingular not found--a working Singular install in $SAGE_LOCAL is required for Sage to workeven though
libSingularhas built just fine (it's some magic withfrom sage.env import SINGULAR_SOthat is broken, soSINGULAR_SOisNone)
Fixed now.
Description changed:
---
+++
@@ -2,6 +2,6 @@
Via #29013, we install all of our own Python packages into a separate venv.
-The venv does not include the system site-packages; we keep that task for a follow-up ticket.
+The venv does not include the system site-packages; we keep that task for the follow-up ticket #29023.
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
d4fbe40 | _get_shared_lib_filename: Do not assume Python sysconfig paths are in SAGE_LOCAL |
bba08ec | Create module src/sage/env_config.py from src/sage/env_config.py.in, defining SAGE_LOCAL |
d1779cc | Merge branch 't/29022/create_module_src_sage_env_config_py_from_src_sage_env_config_py_in__defining_variables_for_use_in_sage_env' into t/29013/public/29013-use-py3-venv |
cd864e8 | Merge branch 't/29013/public/29013-use-py3-venv' into t/27824/public/27824-python3-spkg-configure |
Compiles OK on macOS with homebrew. make ptestlong mostly good except for
- GLPK output (spkg-configure should really reject a libglpk that talks too much) - it makes it hard to see actual failures
- and numerical failures in
src/sage/matrix/matrix2.pyx,src/sage/finance/time_series.pyx, andsrc/sage/schemes/affine/affine_homset.pyas reported in #29013
sage -t --long src/sage/matrix/matrix2.pyx # 2 doctests failed
sage -t --long src/sage/finance/time_series.pyx # 5 doctests failed
sage -t --long src/sage/schemes/affine/affine_homset.py # 1 doctest failed
Branch pushed to git repo; I updated commit sha1. New commits:
85147f9 | build/pkgs/numpy/lapack_conf.py: Add a [DEFAULT] section to site.cfg |
93bae6f | Merge branch 't/29025/numpy__site_cfg_needs_a__default__section' into t/29013/public/29013-use-py3-venv |
669b1df | Merge branch 't/29013/public/29013-use-py3-venv' into t/27824/public/27824-python3-spkg-configure |
Replying to @mkoeppe:
Replying to @dimpase:
I tried this on gentoo (after a usual struggle to get sqlite3 module built in Python)
and it broke at docbuild, withRuntimeError: libSingular not found--a working Singular install in $SAGE_LOCAL is required for Sage to workeven though
libSingularhas built just fine (it's some magic withfrom sage.env import SINGULAR_SOthat is broken, soSINGULAR_SOisNone)Fixed now.
after this fix, it builds and passes most tests on Gentoo (some errors related to cryptominisat that I had installed, and it probably got broken in some way)
I see a problem with cryptominisat spkg on this branch (which otherwise builds and passes most tests (not related to this packages) on Gentoo)
sage -t --warn-long 88.1 src/sage/sat/solvers/cryptominisat.py
**********************************************************************
File "src/sage/sat/solvers/cryptominisat.py", line 49, in sage.sat.solvers.cryptominisat.CryptoMiniSat
Failed example:
solver = CryptoMiniSat() # optional - cryptominisat
Exception raised:
Traceback (most recent call last):
File "/mnt/opt/Sage/sage-dev/local/lib/sage/venv/sage/lib/python3.7/site-packages/sage/sat/solvers/cryptominisat.py", line 69, in __init__
from pycryptosat import Solver
ModuleNotFoundError: No module named 'pycryptosat'
I wonder whether it uses a convoluted way to create python extensions, which got broken.
I see (on a system where cryptominisat package works) that it is one of few packages that get installed without a sub-directory named the same as package in site-packages:
sage: import pycryptosat
sage: print(pycryptosat.__file__)
/home/scratch2/dimpase/sage/sage/local/lib/python3.7/site-packages/pycryptosat.cpython-37m-x86_64-linux-gnu.so
sage: import numpy
sage: print(numpy.__file__)
/home/scratch2/dimpase/sage/sage/local/lib/python3.7/site-packages/numpy/__init__.py
Not sure whether this is a bug on the package, or a bug in this branch.
On this branch I see (with local=SAGE_LOCAL)
$ ls local/lib/python3.7/site-packages/*
local/lib/python3.7/site-packages/pycryptosat-0.2.0-py3.7.egg-info local/lib/python3.7/site-packages/pycryptosat.cpython-37m-x86_64-linux-gnu.so
So it seems that this branch does not pick up anything from that location.
What's the deal with config_venv? Why is any of that necessary? I looked at the commit that added it but it wasn't clear. It seems to me another reason to avoid all the venv stuff...
So, all that $VIRTUAL_ENV/bin/activate does, essentially, is to set 3 environment variables: PATH (to which it inserts $VIRTUAL_ENV/bin in the front), PS1 (which Sage also already sets when activating the sage environment), and the $VIRTUAL_ENV environment variable itself.
Other than that, the only thing that really makes a directory a virtualenv (with Python 3 + venv, I'm not counting the old virtualenv) is the presence of pyvenv.cfg in the install prefix. The format of this file is more-or-less documented (albeit not very clearly) in https://docs.python.org/3/library/venv.html. So all we really need to do to turn $SAGE_LOCAL into a virtualenv of a system Python 3 is to write the appropriate pyvenv.cfg into $SAGE_LOCAL, and not mess around with the full venv command at all. That, and make $SAGE_LOCAL/bin/python3 a symlink to the real python3.
Although not strictly necessary, it is also possibly helpful to third-party tools to set the $VIRTUAL_ENV environment variable.
Replying to @embray:
What's the deal with
config_venv? Why is any of that necessary? I looked at the commit that added it but it wasn't clear.
It is used in a configure test, build/pkgs/python3/spkg-configure.m4 to make sure that the system python has all optional modules that we need.
Replying to @embray:
So all we really need to do to turn
$SAGE_LOCALinto a virtualenv of a system Python 3 is to write the appropriatepyvenv.cfginto$SAGE_LOCAL, and not mess around with the fullvenvcommand at all.
This amounts to trying to upgrade python3's venv to something like a conda environment. This is what I avoid in #29013 on purpose - the venv is only used in a clean, documented way, using the documented tools.
writing pyvenv.cfg by hand is bound to be broken at some point, as it's an implementation detail of venv.
The correct interface to venv is to use venv, I think.
(By the way, the idea of using venv/virtualenv is probably mine, IIRC I put it in the original version of this ticket :-))
Replying to @dimpase:
(By the way, the idea of using venv/virtualenv is probably mine, IIRC I put it in the original version of this ticket :-))
Glad we have this on record now for the software historians/archaeologists to discover :)
Replying to @dimpase:
writing
pyvenv.cfgby hand is bound to be broken at some point, as it's an implementation detail of venv.
The correct interface tovenvis to usevenv, I think.
That's not true. It's documented both in the documentation for the venv module and the site module. If you really want to avoid writing it by hand the venv module includes code for that as well. This (with the system Python 3) is effectively the same (albeit more opaque). From within SAGE_ROOT, but without the sage environment activated:
$ python3 -c 'import venv; b = venv.EnvBuilder(); c = b.ensure_directories("local"); b.create_configuration(c)'
$ cat local/pyvenv.cfg
home = /usr/bin
include-system-site-packages = false
version = 3.6.8
but I think even that's overkill for now.
Replying to @mkoeppe:
Replying to @embray:
What's the deal with
config_venv? Why is any of that necessary? I looked at the commit that added it but it wasn't clear.It is used in a configure test,
build/pkgs/python3/spkg-configure.m4to make sure that the system python has all optional modules that we need.
But why do you need to make a virtualenv for that? Why not run the system python directly?
Replying to @embray:
But why do you need to make a virtualenv for that? Why not run the system python directly?
On this ticket, the system python is not used directly but rather in a venv without site-packages.
Therefore the correct test is to test whether the module will be available in a venv.
I'll add this documentation to the branch.
Branch pushed to git repo; I updated commit sha1. New commits:
348cf22 | build/pkgs/python3/spkg-configure.m4: Add comments |
By the way, should I be checking for the ssl module too?
Replying to @mkoeppe:
Replying to @embray:
But why do you need to make a virtualenv for that? Why not run the system python directly?
On this ticket, the system python is not used directly but rather in a venv without site-packages.
Therefore the correct test is to test whether the module will be available in a venv.
But why?
Replying to @embray:
Replying to @mkoeppe:
Replying to @embray:
But why do you need to make a virtualenv for that? Why not run the system python directly?
On this ticket, the system python is not used directly but rather in a venv without site-packages.
Therefore the correct test is to test whether the module will be available in a venv.But why?
Wait, I misunderstood what you wrote at first. I know why it uses a virtualenv. What I don't get is why you need to make a venv to test features of the system python when you can just call the system python directly here?
Replying to @embray:
What I don't get is why you need to make a venv to test features of the system python when you can just call the system python directly here?
Because system python could have a module named sqlite3 in its site-packages; but then when we try to use it in a venv, it won't be there.
Replying to @mkoeppe:
Replying to @embray:
What I don't get is why you need to make a venv to test features of the system python when you can just call the system python directly here?
Because system python could have a module named
sqlite3in itssite-packages; but then when we try to use it in a venv, it won't be there.
So use python3 -S if you really want to be sure. That disables site-packages.
An interesting discovery while playing around with this: By default Ubuntu's Python does not have distutils. You have to explicitly apt-get install python3-distutils. So this is something we also need to check for at configure time or else it wont be possible to install Python packages.
FWIW I'm experimenting with an alternative solution based on my comments here (albeit with significant pieces taken from this ticket!) I'll update when I have something that I think works...
Branch pushed to git repo; I updated commit sha1. New commits:
cb399a1 | Set PYTHON_FOR_VENV correctly, use it in build/make/deps |
Replying to @embray:
By default Ubuntu's Python does not have distutils. You have to explicitly
apt-get install python3-distutils. So this is something we also need to check for at configure time or else it wont be possible to install Python packages.
Thanks! I'll add this test.
Branch pushed to git repo; I updated commit sha1. New commits:
71cc491 | build/pkgs/python3/spkg-configure.m4: Check for more modules, including distutils.core |
Also, of course, one needs the Python headers, on Ubuntu from libpython3.7-dev.
So I'll add a check that the distutils can actually build an extension.
Branch pushed to git repo; I updated commit sha1. New commits:
c7c1671 | build/pkgs/python3/spkg-configure.m4: Check that distutils can build an extension |
Attachment: Dockerfile-ubuntu-minimal.gz
Testing with the attached Dockerfile on ubuntu bionic.
module and distutils checking seems to work correctly.
Got stuck at scipy install.
[scipy-1.2.0] numpy.distutils.system_info.NotFoundError: No lapack/blas resources found.
Looks like scipy would need a sitecfg as well so that it finds BLAS. It's strange that we build numpy with providing site.cfg (which we just had to correct in #29025), but scipy without site.cfg.
Replying to @mkoeppe:
Testing with the attached Dockerfile on ubuntu bionic.
module and distutils checking seems to work correctly.
Got stuck at
scipyinstall.[scipy-1.2.0] numpy.distutils.system_info.NotFoundError: No lapack/blas resources found.Looks like
scipywould need a sitecfg as well so that it finds BLAS. It's strange that we build numpy with providingsite.cfg(which we just had to correct in #29025), butscipywithoutsite.cfg.
@kiwifb, based on your work on scipy configuration in #20157, would you be able to help with this?
You could provide an identical site.cfg for scipy. The reason you shouldn't technically have to do it, is that the one from numpy is actually shipped. You should have a file numpy/distutils/site.cfg in your numpy install and it should be the one you used to configure numpy. If it is missing you'll run into the trouble above I am sure.
Replying to @mkoeppe:
Replying to @embray:
What I don't get is why you need to make a venv to test features of the system python when you can just call the system python directly here?
Because system python could have a module named
sqlite3in itssite-packages; but then when we try to use it in a venv, it won't be there.
I wonder about rationale for disabling system's site-packages.
Is this the only way to install custom versions of packages in venv (which can conflict with ones is system's site-packages)?
I wonder about rationale for disabling system's
site-packages.
Is this the only way to install custom versions of packages in venv (which can conflict with ones is system'ssite-packages)?
It’s strictly for making this ticket easier because I don’t have to worry about getting the correct versions of all Python packages.
See #29023 for the next step - let’s discuss any issues regarding use of system site packages there.
Replying to @mkoeppe:
I wonder about rationale for disabling system's
site-packages.
Is this the only way to install custom versions of packages in venv (which can conflict with ones is system'ssite-packages)?It’s strictly for making this ticket easier because I don’t have to worry about getting the correct versions of all Python packages.
See #29023 for the next step - let’s discuss any issues regarding use of system site packages there.
I agree here, it's much simpler for now to focus on using a "bare" Python as much as possible. I think detecting and using system Python packages is where we want to go, but it can get a bit hairier...
For the heck of it, I'm seeing how far I can get with a system Python 3.6. I think anything less than 3.6 is almost definitely too old, but I have a feeling 3.6 might still be usable as a minimum. The only major hassle I've had so far is ensuring that it uses UTF-8 encoding by default.
Replying to @kiwifb:
You could provide an identical
site.cfgfor scipy. The reason you shouldn't technically have to do it, is that the one from numpy is actually shipped. You should have a filenumpy/distutils/site.cfgin your numpy install and it should be the one you used to configure numpy. If it is missing you'll run into the trouble above I am sure.
Thank you! The file is installed. I'll investigate why it is not enough.
Replying to @embray:
FWIW I'm experimenting with an alternative solution based on my comments here (albeit with significant pieces taken from this ticket!) I'll update when I have something that I think works...
Here's a prototype of my alternative approach: #29032
I admit it does not yet address as many issues as this ticket does, like all the site.cfg fixes with Numpy. Personally I didn't need it, but if those fixes are necessary they can easily be applied to my ticket as well.
Branch pushed to git repo; I updated commit sha1. New commits:
356b6d7 | build/pkgs/python3/spkg-configure.m4: Rename config_setup.py and config_build to conftest.py and conftest.dir respectively since autoconf will automatically clean up files and directories beginning with "conftest" |
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
92953ce | build/pkgs/python3/spkg-configure.m4: Rename config_setup.py and config_build to conftest.py and conftest.dir respectively since autoconf will automatically clean up files and directories beginning with "conftest" |
Changed author from Matthias Koeppe to Matthias Koeppe, Erik Bray
Branch pushed to git repo; I updated commit sha1. New commits:
08ec551 | build/make/deps: Make python3_venv a .PHONY target |
b2ab6f3 | src/bin/sage-env: Simplify the venv path to $SAGE_LOCAL/lib/sage/venv |
7a43544 | src/sage/env.py, envconfig.py.in: Add MAXIMA; src/sage/interfaces/maxima.py: Use it |
f2af478 | Merge branch 't/29022/create_module_src_sage_env_config_py_from_src_sage_env_config_py_in__defining_variables_for_use_in_sage_env' into t/29013/public/29013-use-py3-venv |
8959c19 | Merge branch 't/29013/public/29013-use-py3-venv' into t/27824/public/27824-python3-spkg-configure |
4c78bdf | src/bin/sage-env: Make inst_python3_venv = $(SAGE_VENV)/pyvenv.cfg. Adapted from #29032 |
Replying to @mkoeppe:
Replying to @kiwifb:
You could provide an identical
site.cfgfor scipy. The reason you shouldn't technically have to do it, is that the one from numpy is actually shipped. You should have a filenumpy/distutils/site.cfgin your numpy install and it should be the one you used to configure numpy. If it is missing you'll run into the trouble above I am sure.Thank you! The file is installed. I'll investigate why it is not enough.
I've created ticket #29051 for this (Paths configured in installed numpy site.cfg [DEFAULT/ALL] do not affect scipy)
Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
56e3daa | src/bin/sage-env: Simplify the venv path to $SAGE_LOCAL/lib/sage/venv |
208c7e6 | src/sage/env.py: Rework _get_shared_lib_filename, fix for Cygwin: it would privilege system DLLs over DLLs in SAGE_LOCAL |
1bd8aef | spkg-configure.m4 for python3 |
7237aa4 | build/pkgs/python3/spkg-configure.m4: Add SAGE_SPKG_DEPCHECK for sqlite libpng bzip2 xz libffi |
d044744 | build/pkgs/python3/spkg-configure.m4: Find @PYTHON_FOR_VENV@ that has sqlite3 module |
c6fec13 | src/bin/sage-env: Don't set MACOSX_DEPLOYMENT_TARGET if we use system python3 |
b69ac5b | Set PYTHON_FOR_VENV correctly, use it in build/make/deps |
c143043 | build/pkgs/python3/spkg-configure.m4: Check for more modules, including distutils.core |
973c36f | build/pkgs/python3/spkg-configure.m4: Rename config_setup.py and config_build to conftest.py and conftest.dir respectively since autoconf will automatically clean up files and directories beginning with "conftest" |
3d90535 | src/bin/sage-env: Make inst_python3_venv = $(SAGE_VENV)/pyvenv.cfg. Adapted from #29032 |
Branch pushed to git repo; I updated commit sha1. New commits:
73e730c | build/bin/sage-system-python: Prefer python3, check a larger list of executable names, check for urllib; exit with a proper error message if no suitable Python is found |
b7c5335 | configure.ac: Stop early if no suitable Python is found |