./configure --enable-editable
mkoeppe opened this issue · 152 comments
This configure switch will install sagelib in "develop" ("editable", "in-place") mode instead of using sagelib's custom incremental build system.
This will clutter the source directory with build artifacts (which are .gitignore'd, of course) but it has the benefit that for changes to Python files, one does not need to run ./sage -b; restarting Sage is enough.
It may also have benefits in certain develop environments that get confused by sagelib's nonstandard build system.
This ticket is based on a subset of the changes in #30371, which developed a version of setup.py suitable for the in-place build. This version is src/setup.py and distinct from build/pkgs/sagelib/src/setup.py.
The configure switch switches to this version of the build system.
To test:
./bootstrap
./configure --enable-editable
make build
Then use and test as normal. Verify that local/lib/...site-packages/ no longer contains a copy of sage - instead there is an "egg-link" back to the source directory.
Switching to a standard build system (getting rid of the sage-specific "installation cleaner") will also simplify the next step of the modularization effort (#29705).
Depends on #30770
Depends on #30912
Depends on #31357
Depends on #31365
Depends on #31389
Depends on #31390
CC: @tobiasdiez @jhpalmieri @fchapoton @videlec @isuruf @kliem @tscrim
Component: build
Author: Tobias Diez, Matthias Koeppe
Branch: 8b3f390
Reviewer: Matthias Koeppe, Tobias Diez, Jonathan Kliem
Issue created by migration from https://trac.sagemath.org/ticket/31377
Commit: 3dcfbe9
Branch pushed to git repo; I updated commit sha1. New commits:
3dcfbe9 | build/pkgs/sagelib: Implement --enable-editable |
Branch pushed to git repo; I updated commit sha1. New commits:
f8ad239 | src/setup.py: Do not look for namespace packages in bin, doc |
This almost works for me, except for some errors building modules:
[sagelib-9.3.beta7] gcc -Wno-unused-result -Wsign-compare -Wunreachable-code -fno-common -dynamic -DNDEBUG -g -fwrapv -O3 -Wall -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk -O2 -g -march=native -Isage/libs/ntl -I/Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/local/lib/python3.9/site-packages/cysignals -Isage/cpython -I/Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/src -I/usr/local/Cellar/python@3.9/3.9.1_8/Frameworks/Python.framework/Versions/3.9/include/python3.9 -I/Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/local/lib/python3.9/site-packages/numpy/core/include -I/Users/mkoeppe/s/sage/sage-rebasing/worktree-algebraic-2018-spring/local/include -I/usr/local/Cellar/python@3.9/3.9.1_8/Frameworks/Python.framework/Versions/3.9/include/python3.9 -c sage/libs/ntl/ntl_ZZ.cpp -o build/temp.macosx-10.15-x86_64-3.9/sage/libs/ntl/ntl_ZZ.o
[sagelib-9.3.beta7] In file included from sage/libs/ntl/error.cpp:1283:
[sagelib-9.3.beta7] ../local/include/NTL/tools.h:1064:1: error: unknown type name 'constexpr'
[sagelib-9.3.beta7] constexpr bool Relocate_aux_has_trivial_copy(T*)
[sagelib-9.3.beta7] ^
This is probably coming from the fact that the C++ std options that sage_build_cython.create_extension uses are not used by the new build system.
Branch pushed to git repo; I updated commit sha1. Last 10 new commits:
804ebd7 | sage_setup.dpcbuild.AllBuilder: Restrict workaround to macOS |
b4ceee5 | sage_setup.docbuild: In the workaround, do not go through build_many to build serially |
7369a3e | Merge branch 't/31344/homebrew__docbuild_crashes__libtcl_atforkprepare' into t/31365/add_ntl_to_cython_aliases |
ce03e6b | src/sage/schemes/hyperelliptic_curves/hypellfrob.pyx: Merge distutils directives |
32576b4 | sage.misc.cython: Add NTL aliases, cache result of cython_aliases |
6e30e3a | Use variables NTL_INCDIR, NTL_LIBDIR in sage_conf, separate from NTL_PREFIX used in sage-build-env-config; set -std=c++11 in NTL_CFLAGS |
7b1e27b | Merge distutils directives for Cython files using ntl |
4d2828d | Switch Cython files that use NTL to language = c++ |
904a494 | Merge branch 't/31365/add_ntl_to_cython_aliases' into t/31377/__configure___enable_editable |
731ff53 | Switch some Cython files to c++, add some -std=c++11 |
I have merged #31365, which adds directives to files using NTL; and added a few more distutils directives so that things don't depend so much on the code in sage_build_cython.create_extension.
Description changed:
---
+++
@@ -1,4 +1,4 @@
-This configure switch will install sagelib in "develop" ("editable", "in-place") mode instead of using the traditional incremental build system.
+This configure switch will install sagelib in "develop" ("editable", "in-place") mode instead of using sagelib's custom incremental build system.
This will clutter the source directory with build artifacts (which are `.gitignore`'d, of course) but it has the benefit that for changes to Python files, one does not need to run `./sage -b`; restarting Sage is enough.
Description changed:
---
+++
@@ -8,3 +8,5 @@
The configure switch switches to this version of the build system.
+Switching to a standard build system (getting rid of the sage-specific "installation cleaner") will also simplify the next step of the modularization effort (#29705).
+Description changed:
---
+++
@@ -4,7 +4,7 @@
It may also have benefits in certain develop environments that get confused by sagelib's nonstandard build system.
-This ticket is based on #30371, which developed a version of `setup.py` suitable for the in-place build. This version is `src/setup.py` and distinct from `build/pkgs/sagelib/src/setup.py`.
+This ticket is based on a subset of the changes in #30371, which developed a version of `setup.py` suitable for the in-place build. This version is `src/setup.py` and distinct from `build/pkgs/sagelib/src/setup.py`.
The configure switch switches to this version of the build system.
This seems to reasonably work well here on macOS. There are a small number of doctest failures.
sage -t --random-seed=0 src/sage/misc/sagedoc.py # 2 doctests failed
sage -t --random-seed=0 src/sage/misc/sageinspect.py # 13 doctests failed
sage -t --random-seed=0 src/sage/repl/interpreter.py # 1 doctest failed
sage -t --random-seed=0 src/sage/categories/primer.py # 1 doctest failed
sage -t --random-seed=0 src/sage/repl/ipython_tests.py # 1 doctest failed
sage -t --random-seed=0 src/sage/sets/set_from_iterator.py # 2 doctests failed
sage -t --random-seed=0 src/sage/interacts/debugger.py # 2 doctests failed
sage -t --random-seed=0 src/sage/misc/lazy_attribute.pyx # 1 doctest failed
sage -t --random-seed=0 src/sage/misc/nested_class.pyx # 1 doctest failed
sage -t --random-seed=0 src/sage/structure/dynamic_class.py # 1 doctest failed
sage -t --random-seed=0 src/sage/all.py # 1 doctest failed
sage -t --random-seed=0 src/sage/misc/edit_module.py # 1 doctest failed
sage -t --random-seed=0 src/sage/cpython/dict_del_by_value.pyx # 2 doctests failed
sage -t --random-seed=0 src/sage/graphs/graph_decompositions/tree_decomposition.pyx # 35 doctests failed
The last one is due to the fact that in the editable install all .pyx files, even when the modules are not compiled, are available and used for doctesting.
This will need a new solution to suppress the doctests of such modules - see #30778 (sage.doctest.control: Exclude doctests in files from non-installed distributions).
Branch pushed to git repo; I updated commit sha1. New commits:
1cb260a | build/pkgs/sagelib/spkg-install: Remove --ignore-installed |
36bbd6c | build/make/Makefile.in (sagelib-clean): Remove .so files in editable installs |
993c35c | build/pkgs/ntl/spkg-configure.m4: Fix up |
18df2dc | Merge branch 't/31365/add_ntl_to_cython_aliases' into t/31377/__configure___enable_editable |
please stop adding me in cc
So how does one test this?
Description changed:
---
+++
@@ -7,6 +7,14 @@
This ticket is based on a subset of the changes in #30371, which developed a version of `setup.py` suitable for the in-place build. This version is `src/setup.py` and distinct from `build/pkgs/sagelib/src/setup.py`.
The configure switch switches to this version of the build system.
+To test:
+
+```
+./configure --enable-editable
+make build
+```
+Then use and test as normal. Verify that `local/lib/...site-packages/` no longer contains a copy of `sage` - instead there is an "egg-link" back to the source directory.
+
Switching to a standard build system (getting rid of the sage-specific "installation cleaner") will also simplify the next step of the modularization effort (#29705).
I've added quick instructions to the ticket description
Reviewer: Matthias Koeppe, ...
The installation of the Jupyter kernel is missing - done by sage_setup.command.sage_install
Replying to @mkoeppe:
This seems to reasonably work well here on macOS. There are a small number of doctest failures.
... sage -t --random-seed=0 src/sage/graphs/graph_decompositions/tree_decomposition.pyx # 35 doctests failedThe last one is due to the fact that in the editable install all
.pyxfiles, even when the modules are not compiled, are available and used for doctesting.
Wrong guess, this is actually #31389.
Branch pushed to git repo; I updated commit sha1. New commits:
19ca17a | src/sage/graphs/graph_decompositions/tdlib/sage_tdlib.cpp: Move one directory level higher |
f02963b | Merge branch 't/31389/remove_clash_of_tdlib__with_tdlib_pyx_in__src_sage_graphs_graph_decompositions' into t/31377/__configure___enable_editable |
Branch pushed to git repo; I updated commit sha1. New commits:
2b293f8 | src/setup.py: Install the jupyter kernel using sage_install |
sage.misc.sageinspect is broken in an editable build, which causes the docbuild to fail.
Branch pushed to git repo; I updated commit sha1. New commits:
f6b2fc4 | src/sage/repl/interpreter.py: Fix a doctest for editable installs of sagelib |
Now the documentation builds OK and also make ptest is fine.
Can you please include the most recent version of setup.py from #30371. Thanks!
The changes there conflict with the changes done and tested here already.
(Let's please get this in - unless there are problems on this ticket that need solving here. Otherwise please merge this branch into #30371, resolving the conflicts, and let me know)
Replying to @mkoeppe:
The changes there conflict with the changes done and tested here already.
What conflicts do you mean?
I was referring to the improvements to this code:
+# Exclude a few files if the corresponding distribution is not loaded
+optional_packages = ['mcqd', 'bliss', 'tdlib', 'primecount',
+ 'coxeter3', 'fes', 'sirocco', 'meataxe']
+not_installed_packages = [package for package in optional_packages
+ if not is_package_installed_and_updated(package)]
+
+distributions_to_exclude = [f"sage-{pkg}"
+ for pkg in not_installed_packages]
+files_to_exclude = filter_cython_sources(SAGE_SRC, distributions_to_exclude)
+
+log.debug(f"files_to_exclude = {files_to_exclude}")
+
+python_packages = find_namespace_packages(where=SAGE_SRC, exclude=['bin', 'doc'])
+log.debug(f"python_packages = {python_packages}")
+
+log.info(f"Discovered Python/Cython sources, time: {(time.time() - t):.2f} seconds.")
+
+# from sage_build_cython:
+import Cython.Compiler.Options
+Cython.Compiler.Options.embed_pos_in_docstring = True
+
+try:
+ log.info("Generating auto-generated sources")
+ from sage_setup.autogen import autogen_all
+ autogen_all()
+
+ from Cython.Build import cythonize
+ from sage.env import cython_aliases, sage_include_directories
+ extensions = cythonize(
+ ["**/*.pyx"],
+ exclude=files_to_exclude,
+ include_path=sage_include_directories(use_sources=True) + ['.'],
+ compile_time_env=compile_time_env_variables(),
+ compiler_directives=compiler_directives(False),
+ aliases=cython_aliases(),
+ create_extension=create_extension,
+ nthreads=4)
+except Exception as exception:
+ log.warn(f"Exception while generating and cythonizing source files: {exception}")
+ extensions = None
This has a few problems/issues:
- The first few lines still raise an exception if Cython (and maybe other packages) are not installed.
- You want to auto-generate code before filtering it / excluding parts of it.
- The exception catch is to general and was meant to only capture module import errors.
All these things are fixed in #30371.
OK, fine, I'll do the merging.
Concerning this change:
# distutils: language = c++
+# distutils: extra_compile_args = -std=c++11
Woudn't it make sense to specify -std=c++11 automatically once language = c++ is set? In fact there is code that looks like it is doing this job in sage_build_cython.
I think also the cython files in libs/singular need similar changes (at least in #30371 errors are thrown otherwise).
Replying to @tobiasdiez:
Concerning this change:
# distutils: language = c++ +# distutils: extra_compile_args = -std=c++11Woudn't it make sense to specify
-std=c++11automatically oncelanguage = c++is set? In fact there is code that looks like it is doing this job in sage_build_cython.
Of course, but you decided not to use the existing code. So I have made the changes that make it work.
Important to keep in mind that I have created this ticket so that we can ship something in Sage 9.3 that works. So let's please keep focused on what is described here on the ticket description.
Replying to @tobiasdiez:
- The first few lines still raise an exception if Cython (and maybe other packages) are not installed.
Not important for this ticket; we do have Cython.
- You want to auto-generate code before filtering it / excluding parts of it.
Why? Even after modularization, there will be exactly 1 module that will contain the autogenerated files.
Replying to @mkoeppe:
Replying to @tobiasdiez:
Concerning this change:
# distutils: language = c++ +# distutils: extra_compile_args = -std=c++11Woudn't it make sense to specify
-std=c++11automatically oncelanguage = c++is set? In fact there is code that looks like it is doing this job in sage_build_cython.Of course, but you decided not to use the existing code. So I have made the changes that make it work.
I couldn't make it work with the sage_build_cython command, yes. So changing create_extension in sage_setup/extensions.py would be a better solution then?
Your approach of starting the build system from scratch has certainly been valuable, so that we can see what parts of the code are needed.
In terms of complexity of code, I think adding the small of number of explicit directives to the files may actually be preferable to the more complex code regarding the -std flags in sage_build_cython.
Let's try it out as is.
In some follow-up ticket, a more general solution may be needed at some point. Especially when use of c++14/17 becomes more widespread, then fixing things to std=c++11 may no longer be the right thing.
Let's do this when it becomes an issue; not now.
Does this now instead implement ./configure --enable-build-as-root? It sounds like that's related to sudo make; is that what's intended?
Thanks for catching the typo in the help string.
Branch pushed to git repo; I updated commit sha1. New commits:
50f5010 | build/pkgs/sagelib/spkg-configure.m4: Fix help string of --enable-editable |
I like it. Sometimes, I wanted to look into a C/C++ file that cython generated and I just couldn't locate it recently. No it's sitting all there for you.
What vanished though is the progress report, how many files yet need to be compiled. I like this feature, because it tells you whether you should wait on it or whether you should carry on and do something else.
Replying to @kliem:
What vanished though is the progress report, how many files yet need to be compiled. I like this feature, because it tells you whether you should wait on it or whether you should carry on and do something else.
Tobias wrote this as a minimal version using standard setuptools + cython. Let's keep adding additional features of this kind in mind for possible follow-up tickets.
Ok. Just mentioned that this is missing and I like it back, if possible. I think I would even switch to this, without the counter. Just because it is nice to have the files where you think the should be.
You introduced a trailing whitespace in src/sage_setup/find.py.
Why is it python3 and not $PYTHON?
+if [ "$SAGE_EDITABLE" = yes ]; then
+ time python3 -m pip install --verbose --no-deps --no-index --no-build-isolation --isolated --editable . || exit 1
+else
+ time "$PYTHON" -u setup.py --no-user-cfg build install || exit 1
+fi
+You freshly introduced from distutils.command.build_ext import build_ext in /src/sage_setup/command/sage_build_ext_minimal.py.
I think this should be replaced by setuptools somehow or at least import setuptools before as to get rid of setuptools.
Replying to @kliem:
Why is it
python3and not$PYTHON?+if [ "$SAGE_EDITABLE" = yes ]; then + time python3 -m pip install --verbose --no-deps --no-index --no-build-isolation --isolated --editable . || exit 1 +else + time "$PYTHON" -u setup.py --no-user-cfg build install || exit 1 +fi +
Thanks for spotting this inconsistency. Actually, both should be python3 (since #30731, which replaced sage-python23).
Branch pushed to git repo; I updated commit sha1. New commits:
8c5e9c3 | build/pkgs/sagelib/spkg-install: python3, not $PYTHON |
Branch pushed to git repo; I updated commit sha1. New commits:
0df54f7 | src/sage_setup/find.py: Fix whitespace |
Replying to @kliem:
What vanished though is the progress report, how many files yet need to be compiled. I like this feature, because it tells you whether you should wait on it or whether you should carry on and do something else.
I think a progress bar will not be easy. The new version directly uses the cythonize method, which doesn't seem to have any configuration in this direction, see https://cython.readthedocs.io/en/latest/src/userguide/source_files_and_compilation.html?highlight=cythonize#cythonize-arguments.
Replying to @mkoeppe:
In terms of complexity of code, I think adding the small of number of explicit directives to the files may actually be preferable to the more complex code regarding the
-stdflags insage_build_cython.
Let's try it out as is.
Ok, sounds good to me. But notice that explicitly specifying the std changes the behavior on cygwin (for a "normal" build) since the following code in sage_build_cython is no longer invoked:
if sys.platform == 'cygwin':
# Cygwin (particularly newlib, Cygwin's libc) has some bugs
# with strict ANSI C/C++ in some headers; using the GNU
# extensions typically fares better:
# https://github.com/sagemath/sage-prod/issues/24192
if cplusplus:
cflags.append("-std=gnu++11")
else:
cflags.append("-std=gnu99")
I don't know if this may have negative side effects.
Thanks for pointing this out; we'll see when we add automated tests for --enable-editable to the Cygwin GH Actions. (in a ticket after #31064 is merged)
A few more observations:
- There is currently no documentation for how to use the inplace install (and why).
- Running the configure command yields the following warning at the end:
....
config.status: creating directory local/var/lib/sage/installed
configure: WARNING: unrecognized options: --enable-editable
configure:
notice: the following SPKGs did not find equivalent system packages
...
- My only means to test this on mac is with the github action added in #30371. Admittedly, this is using a lightly different setup, but since the sage inplace install fails, I wanted to point this out. First of all, the files in
sage/libs/singulardon't specify that they should be compiled withlanguage=c++andstd=c++11. Secondly, there are many errors of the form
/Library/Developer/CommandLineTools/SDKs/MacOSX10.15.sdk/usr/include/nl_types.h:97:1: error: unknown type name 'nl_catd'
nl_catd catopen(const char *, int);
^
See e.g. https://github.com/tobiasdiez/sage/runs/1893534204?check_suite_focus=true
- On Linux (with wsl), the make build fails with
[pplpy-0.8.6] Copying package files from temporary location /mnt/d/Programming/sage/local/var/tmp/sage/build/pplpy-0.8.6/inst to /mnt/d/Programming/sage/local
[pplpy-0.8.6] Running post-install script for pplpy-0.8.6.
[pplpy-0.8.6] Traceback (most recent call last):
[pplpy-0.8.6] File "/mnt/d/Programming/sage/local/bin/sphinx-build", line 5, in <module>
[pplpy-0.8.6] from sphinx.cmd.build import main
[pplpy-0.8.6] File "/mnt/d/Programming/sage/local/lib/python3.8/site-packages/sphinx/cmd/build.py", line 25, in <module>
[pplpy-0.8.6] from sphinx.application import Sphinx
[pplpy-0.8.6] File "/mnt/d/Programming/sage/local/lib/python3.8/site-packages/sphinx/application.py", line 31, in <module>
[pplpy-0.8.6] from sphinx.config import Config
[pplpy-0.8.6] File "/mnt/d/Programming/sage/local/lib/python3.8/site-packages/sphinx/config.py", line 28, in <module>
[pplpy-0.8.6] from sphinx.util.tags import Tags
[pplpy-0.8.6] File "/mnt/d/Programming/sage/local/lib/python3.8/site-packages/sphinx/util/tags.py", line 11, in <module>
[pplpy-0.8.6] from jinja2 import nodes
[pplpy-0.8.6] File "/mnt/d/Programming/sage/local/lib/python3.8/site-packages/jinja2/__init__.py", line 6, in <module>
[pplpy-0.8.6] from markupsafe import escape
[pplpy-0.8.6] ValueError: source code string cannot contain null bytes
[pplpy-0.8.6] make[5]: *** [Makefile:21: html] Error 1
[pplpy-0.8.6] cp: cannot stat 'build/html/*': No such file or directory
That's probably not related to this ticket, but prevents me from testing it.
Replying to @mkoeppe:
Thanks for pointing this out; we'll see when we add automated tests for
--enable-editableto the Cygwin GH Actions. (in a ticket after #31064 is merged)
The point is that explicitly specifying std in the cython file also overrides it for the sage-distribution compilation, i.e. the one using the setup.py in build.
Replying to @tobiasdiez:
- Running the configure command yields the following warning at the end:
.... config.status: creating directory local/var/lib/sage/installed configure: WARNING: unrecognized options: --enable-editable
Forgot to run ./bootstrap?
Replying to @tobiasdiez:
- My only means to test this on mac is with the github action added in #30371.
Let's please not pull a discussion of these experimental GH actions from #30371 into this ticket.
If we add automatic testing, we will do it on the basis of the existing GH actions.
Replying to @tobiasdiez:
Replying to @mkoeppe:
Thanks for pointing this out; we'll see when we add automated tests for
--enable-editableto the Cygwin GH Actions. (in a ticket after #31064 is merged)The point is that explicitly specifying
stdin the cython file also overrides it for the sage-distribution compilation, i.e. the one using the setup.py in build.
Yes, it is a change and we are using portability testing to make sure that it does not break anything on the supported platforms.
Replying to @tobiasdiez:
A few more observations:
- There is currently no documentation for how to use the inplace install (and why).
Well, I hope we can merge this ticket into the next beta and then ask developers to try it. We can document it in a follow-up ticket.
Replying to @tobiasdiez:
- On Linux (with wsl), the make build fails with
[pplpy-0.8.6] Copying package files from temporary location /mnt/d/Programming/sage/local/var/tmp/sage/build/pplpy-0.8.6/inst to /mnt/d/Programming/sage/local [pplpy-0.8.6] Running post-install script for pplpy-0.8.6. [pplpy-0.8.6] from markupsafe import escape [pplpy-0.8.6] ValueError: source code string cannot contain null bytes [pplpy-0.8.6] make[5]: *** [Makefile:21: html] Error 1 [pplpy-0.8.6] cp: cannot stat 'build/html/*': No such file or directoryThat's probably not related to this ticket, but prevents me from testing it.
I have not seen this before and I don't think it has anything to with the present ticket.
Please check if this also occurs with 9.3.beta7 and open a separate ticket if this problem is reproducible.
I'm getting a few doctest failures:
sage -t --long --warn-long 62.2 --random-seed=0 src/sage/repl/interpreter.py
**********************************************************************
File "src/sage/repl/interpreter.py", line 77, in sage.repl.interpreter
Failed example:
print("dummy line"); shell.run_cell('1/0') # see #25320 for the reason of the `...` and the dummy line in this test
Expected:
dummy line
...
ZeroDivisionError...Traceback (most recent call last)
<ipython-input-...> in <module>...
----> 1 Integer(1)/Integer(0)
.../sage/rings/integer.pyx in sage.rings.integer.Integer...div... (.../sage/rings/integer.c:...)()
...
-> ... raise ZeroDivisionError("rational division by zero")
... x = <Rational> Rational.__new__(Rational)
... mpq_div_zz(x.value, ....value, (<Integer>right).value)
<BLANKLINE>
ZeroDivisionError: rational division by zero
Got:
dummy line
---------------------------------------------------------------------------
ZeroDivisionError Traceback (most recent call last)
<ipython-input-1-72ac74c5f414> in <module>
----> 1 Integer(1)/Integer(0)
<BLANKLINE>
/srv/public/kliem/sage/src/sage/rings/integer.pyx in sage.rings.integer.Integer.__truediv__()
2038 if type(left) is type(right):
2039 if mpz_sgn((<Integer>right).value) == 0:
-> 2040 raise ZeroDivisionError("rational division by zero")
2041 x = <Rational> Rational.__new__(Rational)
2042 mpq_div_zz(x.value, (<Integer>left).value, (<Integer>right).value)
<BLANKLINE>
ZeroDivisionError: rational division by zero
**********************************************************************
1 item had failures:
1 of 20 in sage.repl.interpreter
[141 tests, 1 failure, 1.97 s]
----------------------------------------------------------------------
sage -t --long --warn-long 62.2 --random-seed=0 src/sage/repl/interpreter.py # 1 doctest failed
----------------------------------------------------------------------
Total time for all tests: 7.4 seconds
cpu time: 0.6 seconds
cumulative wall time: 2.0 seconds
kliem@lech:~/localhome/sage$ sage -t --long --random-seed=0 src/sage/stats/hmm/hmm.pyx # 1 doctest failed
Running doctests with ID 2021-02-17-08-07-40-53dddc3c.
Git branch: test_31377
Using --optional=build,debian,dochtml,memlimit,pip,sage,sage_spkg
Doctesting 1 file.
sage -t --long --warn-long 62.2 --random-seed=0 src/sage/stats/hmm/hmm.pyx
**********************************************************************
File "src/sage/stats/hmm/hmm.pyx", line 1234, in sage.stats.hmm.hmm.DiscreteHiddenMarkovModel.baum_welch
Failed example:
m.baum_welch(v,fix_emissions=True)
Expected:
(-66.98630856918774, 100)
Got:
(-66.98630856918776, 100)
**********************************************************************
1 item had failures:
1 of 14 in sage.stats.hmm.hmm.DiscreteHiddenMarkovModel.baum_welch
[121 tests, 1 failure, 0.32 s]
----------------------------------------------------------------------
sage -t --long --warn-long 62.2 --random-seed=0 src/sage/stats/hmm/hmm.pyx # 1 doctest failed
----------------------------------------------------------------------
Total time for all tests: 0.4 seconds
cpu time: 0.3 seconds
cumulative wall time: 0.3 seconds
kliem@lech:~/localhome/sage$ sage -t --long --random-seed=0 src/sage_setup/find.py # 6 doctests failed
Running doctests with ID 2021-02-17-08-07-42-447cd574.
Git branch: test_31377
Using --optional=build,debian,dochtml,memlimit,pip,sage,sage_spkg
Doctesting 1 file.
sage -t --long --warn-long 62.2 --random-seed=0 src/sage_setup/find.py
**********************************************************************
File "src/sage_setup/find.py", line 199, in sage_setup.find.filter_cython_sources
Failed example:
'sage/graphs/graph_decompositions/tdlib.pyx' in cython_modules
Expected:
True
Got:
False
**********************************************************************
File "src/sage_setup/find.py", line 264, in sage_setup.find.find_extra_files
Failed example:
extras["sage/ext/interpreters"]
Expected:
['.../src/sage/ext/interpreters/wrapper_cdf.pxd', ...wrapper_cdf.h...]
Got:
['/srv/public/kliem/sage/src/sage/ext/interpreters/wrapper_py.pxd',
'/srv/public/kliem/sage/src/sage/ext/interpreters/wrapper_cdf.pxd',
'/srv/public/kliem/sage/src/sage/ext/interpreters/wrapper_el.pyx',
'/srv/public/kliem/sage/src/sage/ext/interpreters/wrapper_rdf.pyx',
'/srv/public/kliem/sage/src/sage/ext/interpreters/wrapper_cc.pyx',
'/srv/public/kliem/sage/src/sage/ext/interpreters/wrapper_rr.pxd',
'/srv/public/kliem/sage/src/sage/ext/interpreters/wrapper_py.pyx',
'/srv/public/kliem/sage/src/sage/ext/interpreters/wrapper_cdf.pyx',
'/srv/public/kliem/sage/src/sage/ext/interpreters/wrapper_rr.pyx',
'/srv/public/kliem/sage/src/sage/ext/interpreters/wrapper_rdf.pxd',
'/srv/public/kliem/sage/src/sage/ext/interpreters/wrapper_cc.pxd',
'/srv/public/kliem/sage/src/sage/ext/interpreters/wrapper_el.pxd']
**********************************************************************
File "src/sage_setup/find.py", line 324, in sage_setup.find.installed_files_by_module
Failed example:
(f,) = files_by_module['sage.structure.sage_object']; f
Exception raised:
Traceback (most recent call last):
File "/srv/public/kliem/sage/src/sage/doctest/forker.py", line 714, in _run
self.compile_and_execute(example, compiler, test.globs)
File "/srv/public/kliem/sage/src/sage/doctest/forker.py", line 1133, in compile_and_execute
exec(compiled, globs)
File "<doctest sage_setup.find.installed_files_by_module[5]>", line 1, in <module>
(f,) = files_by_module['sage.structure.sage_object']; f
ValueError: not enough values to unpack (expected 1, got 0)
**********************************************************************
File "src/sage_setup/find.py", line 326, in sage_setup.find.installed_files_by_module
Failed example:
(f1, f2) = sorted(files_by_module['sage.structure'])
Exception raised:
Traceback (most recent call last):
File "/srv/public/kliem/sage/src/sage/doctest/forker.py", line 714, in _run
self.compile_and_execute(example, compiler, test.globs)
File "/srv/public/kliem/sage/src/sage/doctest/forker.py", line 1133, in compile_and_execute
exec(compiled, globs)
File "<doctest sage_setup.find.installed_files_by_module[6]>", line 1, in <module>
(f1, f2) = sorted(files_by_module['sage.structure'])
ValueError: not enough values to unpack (expected 2, got 0)
**********************************************************************
File "src/sage_setup/find.py", line 327, in sage_setup.find.installed_files_by_module
Failed example:
f1
Exception raised:
Traceback (most recent call last):
File "/srv/public/kliem/sage/src/sage/doctest/forker.py", line 714, in _run
self.compile_and_execute(example, compiler, test.globs)
File "/srv/public/kliem/sage/src/sage/doctest/forker.py", line 1133, in compile_and_execute
exec(compiled, globs)
File "<doctest sage_setup.find.installed_files_by_module[7]>", line 1, in <module>
f1
NameError: name 'f1' is not defined
**********************************************************************
File "src/sage_setup/find.py", line 329, in sage_setup.find.installed_files_by_module
Failed example:
f2
Exception raised:
Traceback (most recent call last):
File "/srv/public/kliem/sage/src/sage/doctest/forker.py", line 714, in _run
self.compile_and_execute(example, compiler, test.globs)
File "/srv/public/kliem/sage/src/sage/doctest/forker.py", line 1133, in compile_and_execute
exec(compiled, globs)
File "<doctest sage_setup.find.installed_files_by_module[8]>", line 1, in <module>
f2
NameError: name 'f2' is not defined
**********************************************************************
3 items had failures:
1 of 7 in sage_setup.find.filter_cython_sources
1 of 7 in sage_setup.find.find_extra_files
4 of 11 in sage_setup.find.installed_files_by_module
[42 tests, 6 failures, 0.40 s]
----------------------------------------------------------------------
sage -t --long --warn-long 62.2 --random-seed=0 src/sage_setup/find.py # 6 doctests failed
----------------------------------------------------------------------
Total time for all tests: 0.4 seconds
cpu time: 0.4 seconds
cumulative wall time: 0.4 seconds
kliem@lech:~/localhome/sage$ sage -t --long --random-seed=0 src/sage/tests/cmdline.py # 3 doctests failed
Running doctests with ID 2021-02-17-08-07-44-7353eba5.
Git branch: test_31377
Using --optional=build,debian,dochtml,memlimit,pip,sage,sage_spkg
Doctesting 1 file.
sage -t --long --warn-long 62.2 --random-seed=0 src/sage/tests/cmdline.py
**********************************************************************
File "src/sage/tests/cmdline.py", line 586, in sage.tests.cmdline.test_executable
Failed example:
out.startswith("3.")
Expected:
True
Got:
False
**********************************************************************
File "src/sage/tests/cmdline.py", line 588, in sage.tests.cmdline.test_executable
Failed example:
err
Expected:
''
Got:
'/srv/public/kliem/sage/src/bin/sage: line 588: exec: sqlite3: not found\n'
**********************************************************************
File "src/sage/tests/cmdline.py", line 590, in sage.tests.cmdline.test_executable
Failed example:
ret
Expected:
0
Got:
127
**********************************************************************
1 item had failures:
3 of 231 in sage.tests.cmdline.test_executable
[230 tests, 3 failures, 21.87 s]
----------------------------------------------------------------------
sage -t --long --warn-long 62.2 --random-seed=0 src/sage/tests/cmdline.py # 3 doctests failed
----------------------------------------------------------------------
The test in src/sage/tests/cmdline.py is just there for completeness and can be ignored.
I got failures in repl/interpreter.py and sage_setup/find.py.
Branch pushed to git repo; I updated commit sha1. New commits:
78a1587 | src/setup.py: Add . to sys.path so that pyproject.toml from #30913 does not break the build |
Branch pushed to git repo; I updated commit sha1. New commits:
ccce09e | build/pkgs/sagelib/spkg-install [SAGE_EDITABLE=yes]: Uninstall distutils-installed sagelib |
Branch pushed to git repo; I updated commit sha1. New commits:
5a91877 | sage_setup.find: Fix filter_cython_sources doctests |
The remaining failures in sage_setup/find.py are now all about installed_files_by_module; I'll have to see what to do about them (this function is only needed for the old build system).
The failure in sage/repl/interpreter.py seems to indicate that the C source information (sage/rings/integer.c) is missing.
Branch pushed to git repo; I updated commit sha1. New commits:
67072f3 | src/setup.py: Use include, not exclude in find_namespace_packages to get top_level info right |
Branch pushed to git repo; I updated commit sha1. New commits:
0bcf8a4 | build/pkgs/sagelib/spkg-install: When switching from editable install to traditional install, remove the egg-link |
Branch pushed to git repo; I updated commit sha1. New commits:
981124c | sage_setup.find.installed_files_by_module: Make doctest work for editable installs too |
This fixes the remaining doctests for me.
Not for me:
I still get src/sage/tests/cmdline.py, which can be ignored.
On top:
kliem@lech:~/localhome/sage$ sage -t --long --warn-long 62.2 --random-seed=0 src/sage/repl/interpreter.py
Running doctests with ID 2021-02-18-07-50-51-5ae2b230.
Git branch: test_31377
Using --optional=build,debian,dochtml,memlimit,pip,sage,sage_spkg
Doctesting 1 file.
sage -t --long --warn-long 62.2 --random-seed=0 src/sage/repl/interpreter.py
**********************************************************************
File "src/sage/repl/interpreter.py", line 77, in sage.repl.interpreter
Failed example:
print("dummy line"); shell.run_cell('1/0') # see #25320 for the reason of the `...` and the dummy line in this test
Expected:
dummy line
...
ZeroDivisionError...Traceback (most recent call last)
<ipython-input-...> in <module>...
----> 1 Integer(1)/Integer(0)
.../sage/rings/integer.pyx in sage.rings.integer.Integer...div... (.../sage/rings/integer.c:...)()
...
-> ... raise ZeroDivisionError("rational division by zero")
... x = <Rational> Rational.__new__(Rational)
... mpq_div_zz(x.value, ....value, (<Integer>right).value)
<BLANKLINE>
ZeroDivisionError: rational division by zero
Got:
dummy line
---------------------------------------------------------------------------
ZeroDivisionError Traceback (most recent call last)
<ipython-input-1-72ac74c5f414> in <module>
----> 1 Integer(1)/Integer(0)
<BLANKLINE>
/srv/public/kliem/sage/src/sage/rings/integer.pyx in sage.rings.integer.Integer.__truediv__()
2038 if type(left) is type(right):
2039 if mpz_sgn((<Integer>right).value) == 0:
-> 2040 raise ZeroDivisionError("rational division by zero")
2041 x = <Rational> Rational.__new__(Rational)
2042 mpq_div_zz(x.value, (<Integer>left).value, (<Integer>right).value)
<BLANKLINE>
ZeroDivisionError: rational division by zero
**********************************************************************
1 item had failures:
1 of 20 in sage.repl.interpreter
[141 tests, 1 failure, 1.94 s]
----------------------------------------------------------------------
sage -t --long --warn-long 62.2 --random-seed=0 src/sage/repl/interpreter.py # 1 doctest failed
----------------------------------------------------------------------
Total time for all tests: 7.3 seconds
cpu time: 0.6 seconds
cumulative wall time: 1.9 seconds
kliem@lech:~/localhome/sage$ sage -t --long --warn-long 62.2 --random-seed=0 src/sage/stats/hmm/hmm.pyx
Running doctests with ID 2021-02-18-07-51-08-57678760.
Git branch: test_31377
Using --optional=build,debian,dochtml,memlimit,pip,sage,sage_spkg
Doctesting 1 file.
sage -t --long --warn-long 62.2 --random-seed=0 src/sage/stats/hmm/hmm.pyx
**********************************************************************
File "src/sage/stats/hmm/hmm.pyx", line 1234, in sage.stats.hmm.hmm.DiscreteHiddenMarkovModel.baum_welch
Failed example:
m.baum_welch(v,fix_emissions=True)
Expected:
(-66.98630856918774, 100)
Got:
(-66.98630856918776, 100)
**********************************************************************
1 item had failures:
1 of 14 in sage.stats.hmm.hmm.DiscreteHiddenMarkovModel.baum_welch
[121 tests, 1 failure, 0.33 s]
----------------------------------------------------------------------
sage -t --long --warn-long 62.2 --random-seed=0 src/sage/stats/hmm/hmm.pyx # 1 doctest failed
----------------------------------------------------------------------
Total time for all tests: 0.4 seconds
cpu time: 0.3 seconds
cumulative wall time: 0.3 seconds
kliem@lech:~/localhome/sage$ sage -t --long --warn-long 62.2 --random-seed=0 src/sage_setup/find.py
Running doctests with ID 2021-02-18-07-51-21-d3281c6c.
Git branch: test_31377
Using --optional=build,debian,dochtml,memlimit,pip,sage,sage_spkg
Doctesting 1 file.
sage -t --long --warn-long 62.2 --random-seed=0 src/sage_setup/find.py
**********************************************************************
File "src/sage_setup/find.py", line 264, in sage_setup.find.find_extra_files
Failed example:
extras["sage/ext/interpreters"]
Expected:
['.../src/sage/ext/interpreters/wrapper_cdf.pxd', ...wrapper_cdf.h...]
Got:
['/srv/public/kliem/sage/src/sage/ext/interpreters/wrapper_py.pxd',
'/srv/public/kliem/sage/src/sage/ext/interpreters/wrapper_cdf.pxd',
'/srv/public/kliem/sage/src/sage/ext/interpreters/wrapper_el.pyx',
'/srv/public/kliem/sage/src/sage/ext/interpreters/wrapper_rdf.pyx',
'/srv/public/kliem/sage/src/sage/ext/interpreters/wrapper_cc.pyx',
'/srv/public/kliem/sage/src/sage/ext/interpreters/wrapper_rr.pxd',
'/srv/public/kliem/sage/src/sage/ext/interpreters/wrapper_py.pyx',
'/srv/public/kliem/sage/src/sage/ext/interpreters/wrapper_cdf.pyx',
'/srv/public/kliem/sage/src/sage/ext/interpreters/wrapper_rr.pyx',
'/srv/public/kliem/sage/src/sage/ext/interpreters/wrapper_rdf.pxd',
'/srv/public/kliem/sage/src/sage/ext/interpreters/wrapper_cc.pxd',
'/srv/public/kliem/sage/src/sage/ext/interpreters/wrapper_el.pxd']
**********************************************************************
1 item had failures:
1 of 7 in sage_setup.find.find_extra_files
[41 tests, 1 failure, 0.47 s]
----------------------------------------------------------------------
sage -t --long --warn-long 62.2 --random-seed=0 src/sage_setup/find.py # 1 doctest failed
----------------------------------------------------------------------
Total time for all tests: 0.5 seconds
cpu time: 0.5 seconds
cumulative wall time: 0.5 seconds
find_extra_files does exactly what the documentation claims it does, it lists all pyx, pxd, pxi files.