Remove __init__.py files for subpackages designated to be namespace packages
Closed this issue · 103 comments
This ticket changes the following packages to namespace packages (by removing __init__.py):
sage.matrix(this allows the new distributionsage-meataxeintroduced in #30151 to add the modulesage.matrix.matrix_gfpn_dense) to the package)sage.libs(for.meataxe- #30151,.sirocco,.coexter3,.fes)sage.interfaces(for.primecount)sage.graphs(for.bliss,.mcqd)sage.graphs.graph_decompositions(for.tdlib->sage-tdlib, #29864.)sage.numerical,sage.numerical.backends. (This is preparation for later work, in #30152.)
A complication lies in the behavior of the Python import machinery:
setup.py puts the source path in front of setup.py because sage_setup uses sage.env and is_package_or_sage_namespace_package_dir (#33033).
But when an old version of sage that is an ordinary package is installed already, the source will not shadow it.
To avoid this complication, in this ticket we do not yet remove src/sage/__init__.py. That is done in #34187.
CC: @tobiasdiez @kwankyu
Component: refactoring
Author: Matthias Koeppe
Branch/Commit: 39aa2f1
Reviewer: Kwankyu Lee
Issue created by migration from https://trac.sagemath.org/ticket/33011
Last 10 new commits:
2a4a323 | Merge tag '9.5.beta7' into t/29039/pip_installable_package_sage_bootstrap |
9ef0567 | Merge #29039 |
e0399a1 | pkgs/sagemath-objects: Install a fixed set of scripts, not all of src/bin |
7519048 | src/bin/sage[-env]: Put SAGE_ROOT/src/bin in front of path only if run out of this directory |
00024e6 | Merge #32933 |
bfc953b | src/sage/doctest/reporting.py: Indicate --environment if not default |
36d6b09 | pkgs/sagemath-categories/setup.cfg.m4: Add scripts |
8648741 | Merge #29865 |
d91e23b | Merge #28925 |
137f5ee | Remove __init__.py files |
Description changed:
---
+++
@@ -1 +1,8 @@
+This ticket changes the following packages to namespace packages (by removing `__init__.py`):
+- `sage.matrix` (this allows the new distribution `sage-meataxe` introduced in #30151 to add the module `sage.matrix.matrix_gfpn_dense`) to the package)
+- `sage.libs` (for `.meataxe` - #30151, `.sirocco`, `.coexter3`, `.fes`)
+- `sage.interfaces` (for `.primecount`)
+- `sage.graphs` (for `.bliss`, `.mcqd`)
+- `sage.graphs.graph_decompositions` (for `.tdlib` -> `sage-tdlib`, #29864.)
+- `sage.numerical.backends`. (This is preparation for later work, in #30152.)
Description changed:
---
+++
@@ -1,4 +1,5 @@
This ticket changes the following packages to namespace packages (by removing `__init__.py`):
+- `sage`
- `sage.matrix` (this allows the new distribution `sage-meataxe` introduced in #30151 to add the module `sage.matrix.matrix_gfpn_dense`) to the package)
- `sage.libs` (for `.meataxe` - #30151, `.sirocco`, `.coexter3`, `.fes`)
- `sage.interfaces` (for `.primecount`)Description changed:
---
+++
@@ -7,3 +7,7 @@
- `sage.graphs.graph_decompositions` (for `.tdlib` -> `sage-tdlib`, #29864.)
- `sage.numerical.backends`. (This is preparation for later work, in #30152.)
+A complication lies in the behavior of the Python import machinery:
+`setup.py` puts the source path in front of `setup.py` because `sage_setup` uses `sage.env` and `is_package_or_sage_namespace_package_dir` (#33033).
+But when an old version of `sage` that is an ordinary package is installed already, the source will not shadow it.
+Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
2e995dd | src/sage/{arith,ext,libs,rings,sets}/all__sagemath_objects.py |
f244b57 | pkgs/sagemath-objects/MANIFEST.in: Exclude generated files |
2f6d473 | pkgs/sagemath-categories/MANIFEST.in.m4: Exclude generated files |
1a3afe6 | Merge #28925 |
7f9c9b0 | Remove __init__.py files |
cf7371f | src/sage_setup/find.py: Update import of is_package_or_namespace_package_dir |
5808f3d | src/sage/misc/package_dir.py: New file for is_package_or_sage_namespace_package_dir |
f5addb3 | Merge #33033 |
464b0a5 | Merge #28925 |
699ca47 | Remove __init__.py files, add some all.py files |
Branch pushed to git repo; I updated commit sha1. New commits:
2062384 | src/sage/__init__.py: Restore |
Description changed:
---
+++
@@ -1,5 +1,4 @@
This ticket changes the following packages to namespace packages (by removing `__init__.py`):
-- `sage`
- `sage.matrix` (this allows the new distribution `sage-meataxe` introduced in #30151 to add the module `sage.matrix.matrix_gfpn_dense`) to the package)
- `sage.libs` (for `.meataxe` - #30151, `.sirocco`, `.coexter3`, `.fes`)
- `sage.interfaces` (for `.primecount`)
@@ -11,3 +10,5 @@
`setup.py` puts the source path in front of `setup.py` because `sage_setup` uses `sage.env` and `is_package_or_sage_namespace_package_dir` (#33033).
But when an old version of `sage` that is an ordinary package is installed already, the source will not shadow it.
+To avoid this complication, in this ticket we do not yet remove `src/sage/__init__.py`.
+Author: Matthias Koeppe
Branch pushed to git repo; I updated commit sha1. New commits:
5c873e0 | Merge tag '9.6.beta1' into t/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_ |
37b3299 | pkgs/sagemath-{objects,categories}/setup.cfg.m4: Update python_requires |
5546c63 | pkgs/sagemath-objects/MANIFEST.in: Add sage.misc.package_dir |
be39670 | Merge #28925 |
Work Issues: broken with "configure --enable-editable" (gitpod)
Changed work issues from broken with "configure --enable-editable" (gitpod) to none
Is this supposed to work even in incremental build?
Doing ./configure; make with the branch, I got
[sagelib-9.6.beta5] ************************************************************************
[sagelib-9.6.beta5] Traceback (most recent call last):
[sagelib-9.6.beta5] File "/Users/kwankyu/GitHub/sage-dev/pkgs/sagemath-standard/setup.py", line 89, in <module>
[sagelib-9.6.beta5] from sage_setup.find import find_python_sources, is_package_or_namespace_package_dir
[sagelib-9.6.beta5] ImportError: cannot import name 'is_package_or_namespace_package_dir' from 'sage_setup.find' (/Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage_setup/find.py)
Building from scratch (after make distclean) works. Sage starts with no difference. I am running tests. No problem up to now.
I checked removed __init__.py files. There is still src/sage/numerical/__init__.py file. Is this correct?
What else should I check as a user/reviewer?
A comment perhaps naive and not relevant here:
If a file like all__sagemath_objects.py is only to serve
# The presence of this file ensures that sage_setup for sagemath-objects
# considers this directory as a namespace package
then a simpler name like __namespace_package__.py would not work?
I get these persistent doctest failures
sage -t --warn-long 74.5 --random-seed=209187923013064120554478148791206188460 src/sage/tests/cmdline.py # 4 doctests failed
sage -t --warn-long 74.5 --random-seed=209187923013064120554478148791206188460 src/sage/structure/element.pyx # 35 doctests failed
sage -t --warn-long 74.5 --random-seed=209187923013064120554478148791206188460 src/sage/misc/cython.py # 3 doctests failed
sage -t --warn-long 74.5 --random-seed=209187923013064120554478148791206188460 src/sage/graphs/connectivity.pyx # 2 doctests failed
sage -t --warn-long 74.5 --random-seed=209187923013064120554478148791206188460 src/sage/misc/cachefunc.pyx # 23 doctests failed
sage -t --warn-long 74.5 --random-seed=209187923013064120554478148791206188460 src/sage/rings/tate_algebra_ideal.pyx # 6 doctests failed
sage -t --warn-long 74.5 --random-seed=209187923013064120554478148791206188460 src/sage/graphs/graph_decompositions/fast_digraph.pyx # 5 doctests failed
sage -t --warn-long 74.5 --random-seed=209187923013064120554478148791206188460 src/sage/rings/polynomial/ore_polynomial_element.pyx # 6 doctests failed
sage -t --warn-long 74.5 --random-seed=209187923013064120554478148791206188460 src/sage/arith/long.pxd # 14 doctests failed
sage -t --warn-long 74.5 --random-seed=209187923013064120554478148791206188460 src/sage/ext/memory_allocator.pxd # 6 doctests failed
sage -t --warn-long 74.5 --random-seed=209187923013064120554478148791206188460 src/sage/ext/memory_allocator.pyx # 9 doctests failed
sage -t --warn-long 74.5 --random-seed=209187923013064120554478148791206188460 src/sage/misc/lazy_attribute.pyx # 3 doctests failed
sage -t --warn-long 74.5 --random-seed=209187923013064120554478148791206188460 src/sage/misc/inherit_comparison.pyx # 5 doctests failed
sage -t --warn-long 74.5 --random-seed=209187923013064120554478148791206188460 src/sage/rings/integer_fake.pxd # 1 doctest failed
sage -t --warn-long 74.5 --random-seed=209187923013064120554478148791206188460 src/sage_setup/find.py # 1 doctest failed
Replying to @kwankyu:
If a file like
all__sagemath_objects.pyis only to serve# The presence of this file ensures that sage_setup for sagemath-objects # considers this directory as a namespace packagethen a simpler name like
__namespace_package__.pywould not work?
It would work, but based on discussion in #28925, I decided to use a file pattern that is already in use in the Sage sources
Description changed:
---
+++
@@ -4,7 +4,7 @@
- `sage.interfaces` (for `.primecount`)
- `sage.graphs` (for `.bliss`, `.mcqd`)
- `sage.graphs.graph_decompositions` (for `.tdlib` -> `sage-tdlib`, #29864.)
-- `sage.numerical.backends`. (This is preparation for later work, in #30152.)
+- `sage.numerical`, `sage.numerical.backends`. (This is preparation for later work, in #30152.)
A complication lies in the behavior of the Python import machinery:
`setup.py` puts the source path in front of `setup.py` because `sage_setup` uses `sage.env` and `is_package_or_sage_namespace_package_dir` (#33033).Replying to @kwankyu:
I checked removed
__init__.pyfiles. There is stillsrc/sage/numerical/__init__.pyfile. Is this correct?
Thanks for catching this! I have removed it now.
Replying to @kwankyu:
I get these persistent doctest failures
Thanks for testing! I cannot reproduce these failures. Maybe they have been resolved by merging the latest beta into the branch already.
Replying to @mkoeppe:
Replying to @kwankyu:
If a file like
all__sagemath_objects.pyis only to serve# The presence of this file ensures that sage_setup for sagemath-objects # considers this directory as a namespace packagethen a simpler name like
__namespace_package__.pywould not work?It would work, but based on discussion in #28925, I decided to use a file pattern that is already in use in the Sage sources
I don't find any discussion about naming there... Anyway,
(1) Is it true that the prefix all__ has nothing to do with the purpose of the file all.py?
(2) Should the name depend on the distribution package name (here sagemath_objects)? Then is it subject to change depending on how sage library is organized as distribution packages?
Replying to @kwankyu:
(1) Is it true that the prefix
all__has nothing to do with the purpose of the fileall.py?
Not true. See for example https://github.com/sagemath/sage-prod/blob/develop/src/sage/categories/all__sagemath_objects.py, which is imported in https://github.com/sagemath/sage-prod/blob/develop/src/sage/categories/all.py
Just the all__sagemath_objects.py files added in the present ticket happen to be trivial because the distribution sagemath-objects does not put anything from the modules that it contains in the global namespace.
Replying to @mkoeppe:
Replying to @kwankyu:
(1) Is it true that the prefix
all__has nothing to do with the purpose of the fileall.py?Not true. See for example https://github.com/sagemath/sage-prod/blob/develop/src/sage/categories/all__sagemath_objects.py, which is imported in https://github.com/sagemath/sage-prod/blob/develop/src/sage/categories/all.py
Okay. I see the idea.
With sage built with Volker's branch and #28925, I get these failures only with this branch:
sage -t --warn-long 63.0 --random-seed=176571558312394579781618281386824830468 src/sage/tests/cmdline.py # 4 doctests failed
sage -t --warn-long 63.0 --random-seed=176571558312394579781618281386824830468 src/sage/misc/sagedoc.py # 3 doctests failed
sage -t --warn-long 63.0 --random-seed=176571558312394579781618281386824830468 src/sage/structure/element.pyx # 35 doctests failed
sage -t --warn-long 63.0 --random-seed=176571558312394579781618281386824830468 src/sage/misc/cachefunc.pyx # 23 doctests failed
sage -t --warn-long 63.0 --random-seed=176571558312394579781618281386824830468 src/sage/misc/cython.py # 3 doctests failed
sage -t --warn-long 63.0 --random-seed=176571558312394579781618281386824830468 src/sage/graphs/connectivity.pyx # 2 doctests failed
sage -t --warn-long 63.0 --random-seed=176571558312394579781618281386824830468 src/sage/rings/tate_algebra_ideal.pyx # 6 doctests failed
sage -t --warn-long 63.0 --random-seed=176571558312394579781618281386824830468 src/sage/graphs/graph_decompositions/fast_digraph.pyx # 5 doctests failed
sage -t --warn-long 63.0 --random-seed=176571558312394579781618281386824830468 src/sage/rings/polynomial/ore_polynomial_element.pyx # 6 doctests failed
sage -t --warn-long 63.0 --random-seed=176571558312394579781618281386824830468 src/sage/arith/long.pxd # 14 doctests failed
sage -t --warn-long 63.0 --random-seed=176571558312394579781618281386824830468 src/sage/ext/memory_allocator.pxd # 6 doctests failed
sage -t --warn-long 63.0 --random-seed=176571558312394579781618281386824830468 src/sage/ext/memory_allocator.pyx # 9 doctests failed
sage -t --warn-long 63.0 --random-seed=176571558312394579781618281386824830468 src/sage/misc/lazy_attribute.pyx # 3 doctests failed
sage -t --warn-long 63.0 --random-seed=176571558312394579781618281386824830468 src/sage/misc/inherit_comparison.pyx # 5 doctests failed
sage -t --warn-long 63.0 --random-seed=176571558312394579781618281386824830468 src/sage/rings/integer_fake.pxd # 1 doctest failed
sage -t --warn-long 63.0 --random-seed=176571558312394579781618281386824830468 src/sage_setup/find.py # 1 doctest failed
sage -t --warn-long 63.0 --random-seed=176571558312394579781618281386824830468 src/sage/docs/conf.py # 1 doctest failed
Branch pushed to git repo; I updated commit sha1. New commits:
6258b79 | Merge tag '9.6.rc2' into t/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_ |
4e95de9 | Merge #33803 |
a92db46 | Merge tag '9.7.beta1' into t/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_ |
1e35435 | Merge #28925 |
With the current branch, make distclean; make -j8; sage -tp --all results in
----------------------------------------------------------------------
sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/tests/cmdline.py # 4 doctests failed
sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/misc/sagedoc.py # 3 doctests failed
sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/structure/element.pyx # 35 doctests failed
sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/misc/cython.py # 3 doctests failed
sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/misc/cachefunc.pyx # 23 doctests failed
sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/graphs/connectivity.pyx # 2 doctests failed
sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/graphs/graph_decompositions/fast_digraph.pyx # 5 doctests failed
sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/rings/tate_algebra_ideal.pyx # 6 doctests failed
sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/rings/polynomial/ore_polynomial_element.pyx # 6 doctests failed
sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/ext/memory_allocator.pxd # 6 doctests failed
sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/ext/memory_allocator.pyx # 9 doctests failed
sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/arith/long.pxd # 14 doctests failed
sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/misc/inherit_comparison.pyx # 5 doctests failed
sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/misc/lazy_attribute.pyx # 3 doctests failed
sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/rings/integer_fake.pxd # 1 doctest failed
sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/game_theory/normal_form_game.py # 2 doctests failed
sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage_setup/find.py # 1 doctest failed
sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/game_theory/catalog_normal_form_games.py # 15 doctests failed
sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage_docbuild/conf.py # 1 doctest failed
sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/docs/conf.py # 1 doctest failed
----------------------------------------------------------------------
on mac.
This comment
# To build and test in the tox environment:
#
# ./sage -sh -c '(cd pkgs/sagemath-categories/src && tox -v -v -v)'
in pkgs/sagemath-categories/tox.ini seems wrong.
This ./sage -sh -c '(cd pkgs/sagemath-categories && tox -v -v -v)' works though.
Branch pushed to git repo; I updated commit sha1. New commits:
ed3086d | pkgs/sagemath-categories/tox.ini: Fix instructions in comment |
Replying to @kwankyu:
With the current branch,
make distclean; make -j8; sage -tp --allresults in---------------------------------------------------------------------- sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/tests/cmdline.py # 4 doctests failed sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/misc/sagedoc.py # 3 doctests failed sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/structure/element.pyx # 35 doctests failed sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/misc/cython.py # 3 doctests failed sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/misc/cachefunc.pyx # 23 doctests failed sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/graphs/connectivity.pyx # 2 doctests failed sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/graphs/graph_decompositions/fast_digraph.pyx # 5 doctests failed sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/rings/tate_algebra_ideal.pyx # 6 doctests failed sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/rings/polynomial/ore_polynomial_element.pyx # 6 doctests failed sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/ext/memory_allocator.pxd # 6 doctests failed sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/ext/memory_allocator.pyx # 9 doctests failed sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/arith/long.pxd # 14 doctests failed sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/misc/inherit_comparison.pyx # 5 doctests failed sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/misc/lazy_attribute.pyx # 3 doctests failed sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/rings/integer_fake.pxd # 1 doctest failed sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/game_theory/normal_form_game.py # 2 doctests failed sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage_setup/find.py # 1 doctest failed sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/game_theory/catalog_normal_form_games.py # 15 doctests failed sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage_docbuild/conf.py # 1 doctest failed sage -t --warn-long 59.6 --random-seed=58674280218506165844967914886745987426 src/sage/docs/conf.py # 1 doctest failed ----------------------------------------------------------------------on mac.
Could you share the details of these failures please? I cannot reproduce them (also on mac)
OK, now I do see some failures that come from the runtime use of Cython.
Some doctest failures are caused by the failure of documentation building. Indeed I have
[sagemath_doc_html-none] [graphs ] The inventory files are in local/share/doc/sage/inventory/en/reference/graphs.
[sagemath_doc_html-none] Build finished. The built documents can be found in /Users/kwankyu/GitHub/sage-temp/local/share/doc/sage/inventory/en/reference/graphs
[sagemath_doc_html-none] Error building the documentation.
[sagemath_doc_html-none] Traceback (most recent call last):
[sagemath_doc_html-none] File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 197, in _run_module_as_main
[sagemath_doc_html-none] return _run_code(code, main_globals, None,
[sagemath_doc_html-none] File "/usr/local/Cellar/python@3.9/3.9.12/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 87, in _run_code
[sagemath_doc_html-none] exec(code, run_globals)
[sagemath_doc_html-none] File "/Users/kwankyu/GitHub/sage-temp/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage_docbuild/__main__.py", line 500, in <module>
[sagemath_doc_html-none] sys.exit(main())
[sagemath_doc_html-none] File "/Users/kwankyu/GitHub/sage-temp/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage_docbuild/__main__.py", line 497, in main
[sagemath_doc_html-none] builder()
[sagemath_doc_html-none] File "/Users/kwankyu/GitHub/sage-temp/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage_docbuild/builders.py", line 814, in _wrapper
[sagemath_doc_html-none] for module_name in self.get_new_and_updated_modules():
[sagemath_doc_html-none] File "/Users/kwankyu/GitHub/sage-temp/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage_docbuild/builders.py", line 1014, in get_new_and_updated_modules
[sagemath_doc_html-none] if (module_filename.endswith('.pyc') or module_filename.endswith('.pyo')):
[sagemath_doc_html-none] AttributeError: 'NoneType' object has no attribute 'endswith'
[sagemath_doc_html-none]
[sagemath_doc_html-none] Note: incremental documentation builds sometimes cause spurious
[sagemath_doc_html-none] error messages. To be certain that these are real errors, run
[sagemath_doc_html-none] "make doc-clean doc-uninstall" first and try again.
[sagemath_doc_html-none] cd /Users/kwankyu/GitHub/sage-temp && ./sage --docbuild --no-pdf-links reference/cryptography inventory --no-prune-empty-dirs
[sagemath_doc_html-none] make[6]: *** [doc-inventory--reference-categories] Error 1
[sagemath_doc_html-none] make[6]: *** Waiting for unfinished jobs....
[sagemath_doc_html-none] [asymptoti] no targets are out of date.
Do you have no problem in building documentation?
Now I see that too, sorry. Previously it seems I didn't clean fully
Branch pushed to git repo; I updated commit sha1. New commits:
7a43f7a | src/sage_docbuild/builders.py: Handle namespace packages |
Here's an attempt to fix the docbuild problem
Branch pushed to git repo; I updated commit sha1. New commits:
e254695 | src/setup.py: Fix up use of cython_namespace_package_support |
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
a15f173 | pkgs/sagemath-categories/tox.ini: Fix instructions in comment |
8587d47 | src/sage/misc/package_dir.py: New context manager cython_namespace_package_support |
bc0992d | src/sage/misc/cython.py: Activate cython_namespace_package_support |
7c98c36 | src/setup.py: Fix up use of cython_namespace_package_support |
79822e4 | pkgs/sagemath-objects/setup.py: Monkey-patching cython is now done by the sage_build_cython command |
895abe0 | Remove __init__.py files |
1f45f8a | Remove __init__.py files, add some all.py files |
f9038ed | src/sage/__init__.py: Restore |
1326e17 | src/sage/numerical/__init__.py: Remove |
5aebcf3 | src/sage_docbuild/builders.py: Handle namespace packages |
Only this one remains:
sage -t --random-seed=189805995761835901272169845153382768673 src/sage_setup/find.py
Running doctests with ID 2022-06-08-18-44-23-6fbbcef6.
Using --optional=homebrew,pip,sage,sage_spkg
Features to be detected: 4ti2,benzene,bliss,buckygen,conway_polynomials,csdp,database_cremona_ellcurve,database_cremona_mini_ellcurve,database_jones_numfield,database_knotinfo,dvipng,gfan,graphviz,imagemagick,jupymake,kenzo,latte_int,lrslib,mcqd,meataxe,msolve,nauty,palp,pandoc,pdf2svg,pdftocairo,phitigra,plantri,polytopes_db,polytopes_db_4d,pynormaliz,python_igraph,rubiks,sage.combinat,sage.geometry.polyhedron,sage.graphs,sage.groups,sage.plot,sage.rings.number_field,sage.rings.padics,sage.rings.real_double,sage.symbolic,sage_numerical_backends_coin,sagemath_doc_html,sphinx,tdlib
Doctesting 1 file.
sage -t --warn-long 64.2 --random-seed=189805995761835901272169845153382768673 src/sage_setup/find.py
**********************************************************************
File "src/sage_setup/find.py", line 401, in sage_setup.find.installed_files_by_module
Failed example:
files_by_module['sage.graphs.graph_decompositions']
Expected:
{...'sage/graphs/graph_decompositions/__init__.py'...}
Got:
set()
**********************************************************************
1 item had failures:
1 of 10 in sage_setup.find.installed_files_by_module
[49 tests, 1 failure, 1.47 s]
Branch pushed to git repo; I updated commit sha1. New commits:
bba5284 | src/sage_setup/find.py: Update doctest output |
Thanks for testing! Fixed now
All tests passed! Also I ran tox tests on sagemath-categories. All passed.
And then I did pkgs/sagemath-categories/.tox/python/bin/python, and then
from sage.categories.all import *
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/kwankyu/GitHub/sage-temp/pkgs/sagemath-categories/sage/categories/all.py", line 32, in <module>
from sage.misc.lazy_import import lazy_import
ModuleNotFoundError: No module named 'sage.misc.lazy_import'
Perhaps I did something that I should not do. What are reasonable things to do, to test sagemath-categories?
This works just fine here:
$ ./sage -sh -c '(cd pkgs/sagemath-categories && SAGE_NUM_THREADS=16 tox -r -v -v -v -e py39)'
[...]
$ pkgs/sagemath-categories/.tox/py39/bin/python
Python 3.9.10 (v3.9.10:f2f3f53782, Jan 13 2022, 17:02:14)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from sage.categories.all import *
>>>
Did you by any chance run this from the directory /Users/kwankyu/GitHub/sage-temp/pkgs/sagemath-categories/?
This won't work because by default . is in the front of PYTHONPATH. And then the directory in the source tree (sage), which does not contain the compiled Cython modules, shadows the installed package sage.
(That's what the comment # Beware of the treacherous non-src layout in tox.ini refers to.)
Replying to @mkoeppe:
Did you by any chance run this from the directory
/Users/kwankyu/GitHub/sage-temp/pkgs/sagemath-categories/?
Correct.
This won't work because by default
.is in the front ofPYTHONPATH. And then the directory in the source tree (sage), which does not contain the compiled Cython modules, shadows the installed packagesage.
I see.
(That's what the comment
# Beware of the treacherous non-src layoutin tox.ini refers to.)
So this would be a common gotcha. Then why not src layout? Perhaps this is explained somewhere?
Replying to @kwankyu:
(That's what the comment
# Beware of the treacherous non-src layoutin tox.ini refers to.)So this would be a common gotcha. Then why not
srclayout? Perhaps this is explained somewhere?
As far as I can see, there is no consensus in the Python community regarding which of the two layouts should be preferred. To me, the extra src/ is additional clutter
Replying to @mkoeppe:
So this would be a common gotcha. Then why not
srclayout? Perhaps this is explained somewhere?As far as I can see, there is no consensus in the Python community regarding which of the two layouts should be preferred. To me, the extra
src/is additional clutter
Okay. Not a big deal.
I am working on a ticket on sage merged with the present ticket. When I rebuild sage by sage -b, I observe much bigger log, including
...
...
***********************************************
[sagelib-9.7.beta1] Building interpreters for fast_callable
[sagelib-9.7.beta1] Discovering Python/Cython source code....
[sagelib-9.7.beta1] distributions = ['']
[sagelib-9.7.beta1] Discovered Python/Cython sources, time: 5.48 seconds.
[sagelib-9.7.beta1] running build
[sagelib-9.7.beta1] running build_py
[sagelib-9.7.beta1] package init file 'sage/misc/__init__.py' not found (or not a regular file)
[sagelib-9.7.beta1] package init file 'sage/numerical/__init__.py' not found (or not a regular file)
[sagelib-9.7.beta1] package init file 'sage/numerical/backends/__init__.py' not found (or not a regular file)
[sagelib-9.7.beta1] package init file 'sage/ext/__init__.py' not found (or not a regular file)
[sagelib-9.7.beta1] package init file 'sage/arith/__init__.py' not found (or not a regular file)
[sagelib-9.7.beta1] package init file 'sage/libs/__init__.py' not found (or not a regular file)
[sagelib-9.7.beta1] package init file 'sage/libs/mpfi/__init__.py' not found (or not a regular file)
[sagelib-9.7.beta1] package init file 'sage/libs/mpfr/__init__.py' not found (or not a regular file)
[sagelib-9.7.beta1] package init file 'sage/libs/polybori/__init__.py' not found (or not a regular file)
[sagelib-9.7.beta1] package init file 'sage/libs/mpc/__init__.py' not found (or not a regular file)
[sagelib-9.7.beta1] package init file 'sage/sets/__init__.py' not found (or not a regular file)
[sagelib-9.7.beta1] package init file 'sage/rings/__init__.py' not found (or not a regular file)
[sagelib-9.7.beta1] package init file 'sage/graphs/__init__.py' not found (or not a regular file)
[sagelib-9.7.beta1] package init file 'sage/graphs/graph_decompositions/__init__.py' not found (or not a regular file)
[sagelib-9.7.beta1] copying sage/modules/free_module.py -> build/lib.macosx-12-x86_64-3.9/sage/modules
[sagelib-9.7.beta1] package init file 'sage/categories/__init__.py' not found (or not a regular file)
[sagelib-9.7.beta1] package init file 'sage/matrix/__init__.py' not found (or not a regular file)
[sagelib-9.7.beta1] package init file 'sage/interfaces/__init__.py' not found (or not a regular file)
[sagelib-9.7.beta1] running build_ext
[sagelib-9.7.beta1] running build_cython
[sagelib-9.7.beta1] Enabling Cython debugging support
[sagelib-9.7.beta1] Updating Cython code....
[sagelib-9.7.beta1] Compiling sage/resolutions/free.pyx because it changed.
[sagelib-9.7.beta1] [1/1] Cythonizing sage/resolutions/free.pyx
[sagelib-9.7.beta1] Finished Cythonizing, time: 6.33 seconds.
[sagelib-9.7.beta1] cythonized_files = [('sage/misc', ['./sage/misc/parser.pyx', './sage/misc/classcall_metaclass.pyx', './sage/misc/weak_dict.pyx', './sage/misc/sage_timeit_class.pyx', './sage/misc/lazy_string.pyx', './sage/misc/search.pyx', './sage/misc/c3_controlled.pyx', './sage/misc/instancedoc.pyx', './sage/misc/binary_tree.pxd', './sage/misc/reset.pyx', './sage/misc/sage_ostools.pyx', './sage/misc/function_mangling.pyx', './sage/misc/lazy_list.pyx', './sage/misc/citation.pyx', './sage/misc/nested_class.pxd', './sage/misc/fast_methods.pyx', './sage/misc/misc_c.pxd', './sage/misc/cachefunc.pyx', './sage/misc/session.pyx', './sage/misc/pickle_old.pyx', './sage/misc/allocator.pyx', './sage/misc/inherit_comparison.pxd', './sage/misc/randstate.pxd', './sage/misc/randstate.pyx', './sage/misc/inherit_comparison.pyx', './sage/misc/allocator.pxd', './sage/misc/cachefunc.pxd', './sage/misc/callable_dict.pyx', './sage/misc/fast_methods.pxd', './sage/misc/nested_class.pyx', './sage/misc/derivative.pyx', './sage/misc/misc_c.pyx', './sage/misc/lazy_attribute.pyx', './sage/misc/lazy_list.pxd', './sage/misc/function_mangling.pxd', './sage/misc/lazy_import.pyx', './sage/misc/binary_tree.pyx', './sage/misc/search.pxd', './sage/misc/c3_controlled.pxd', './sage/misc/fpickle.pyx', './sage/misc/lazy_string.pxd', './sage/misc/stopgap.pyx', './sage/misc/persist.pyx', './sage/misc/weak_dict.pxd', './sage/misc/c3.pyx', './sage/misc/constant_function.pyx', './sage/misc/classcall_metaclass.pxd']), ('sage/schemes/hyperelliptic_curves', ['./sage/schemes/hyperelliptic_curves/hypellfrob.pyx']), ('sage/schemes/elliptic_curves', ['./sage/schemes/elliptic_curves/period_lattice_region.pyx', './sage/schemes/elliptic_curves/mod_sym_num.pyx', './sage/schemes/elliptic_curves/descent_two_isogeny.pyx']), ('sage/schemes/toric', ['./sage/schemes/toric/divisor_class.pyx']), ('sage/crypto', ['./sage/crypto/boolean_function.pyx', './sage/crypto/sbox.pyx', './sage/crypto/boolean_function.pxd']), ('sage/data_structures', ['./sage/data_structures/bitset.pyx', './sage/data_structures/bounded_integer_sequences.pyx', './sage/data_structures/bitset_base.pxd', './sage/data_structures/binary_search.pyx', './sage/data_structures/blas_dict.pxd', './sage/data_structures/blas_dict.pyx', './sage/data_structures/binary_matrix.pxd', './sage/data_structures/binary_search.pxd', './sage/data_structures/sparse_bitset.pxd', './sage/data_structures/bitset_base.pyx', './sage/data_structures/bitset.pxd', './sage/data_structures/bounded_integer_sequences.pxd', 'build/cythonized/sage/data_structures/bitset_intrinsics.h']), ('sage/dynamics/complex_dynamics', ['./sage/dynamics/complex_dynamics/mandel_julia_helper.pyx']), ('sage/dynamics/arithmetic_dynamics', ['./sage/dynamics/arithmetic_dynamics/projective_ds_helper.pyx']), ('sage/probability', ['./sage/probability/probability_distribution.pyx']), ('sage/plot', ['./sage/plot/complex_plot.pyx']), ('sage/plot/plot3d', ['./sage/plot/plot3d/point_c.pxi', './sage/plot/plot3d/parametric_surface.pxd', './sage/plot/plot3d/shapes.pyx', './sage/plot/plot3d/index_face_set.pyx', './sage/plot/plot3d/base.pxd', './sage/plot/plot3d/transform.pyx', './sage/plot/plot3d/transform.pxd', './sage/plot/plot3d/base.pyx', './sage/plot/plot3d/implicit_surface.pyx', './sage/plot/plot3d/shapes.pxd', './sage/plot/plot3d/index_face_set.pxd', './sage/plot/plot3d/parametric_surface.pyx']), ('sage/combinat', ['./sage/combinat/permutation_cython.pyx
...
...
long long list of cythonized files!
...
...
[sagelib-9.7.beta1] running install_scripts
[sagelib-9.7.beta1] running egg_info
[sagelib-9.7.beta1] writing sagemath_standard.egg-info/PKG-INFO
[sagelib-9.7.beta1] writing dependency_links to sagemath_standard.egg-info/dependency_links.txt
[sagelib-9.7.beta1] writing requirements to sagemath_standard.egg-info/requires.txt
[sagelib-9.7.beta1] writing top-level names to sagemath_standard.egg-info/top_level.txt
[sagelib-9.7.beta1] reading manifest template 'MANIFEST.in'
[sagelib-9.7.beta1] warning: no files found matching '*.hh' anywhere in distribution
[sagelib-9.7.beta1] warning: no files found matching '*.inc' anywhere in distribution
[sagelib-9.7.beta1] no previously-included directories found matching '.tox'
[sagelib-9.7.beta1] no previously-included directories found matching 'sage_setup'
[sagelib-9.7.beta1] adding license file 'LICENSE.txt'
[sagelib-9.7.beta1] writing manifest file 'sagemath_standard.egg-info/SOURCES.txt'
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-startuptime.py to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-ipynb2rst to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-notebook to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-inline-fortran to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-valgrind to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-coverage to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-run to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-ipython to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-run-cython to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-grep to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-venv-config to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-rebaseall.sh to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-update-version to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-rebase.sh to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-rebaseall.bat to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/math-readline to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-env to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-list-packages to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-callgrind to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-runtests to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-rebase.bat to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-fixdoctests to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-preparse to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-python to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-cleaner to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-omega to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-massif to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-cachegrind to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-grepdoc to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-num-threads.py to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-cython to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-version.sh to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-gdb-commands to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-eval to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage to 755
[sagelib-9.7.beta1] running install_egg_info
[sagelib-9.7.beta1] removing '/Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sagemath_standard-9.7b1-py3.9.egg-info' (and everything under it)
[sagelib-9.7.beta1] Copying sagemath_standard.egg-info to /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sagemath_standard-9.7b1-py3.9.egg-info
[sagelib-9.7.beta1] - cleaning build/lib.macosx-12-x86_64-3.9
[sagelib-9.7.beta1] - cleaning /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages
[sagelib-9.7.beta1] Cleaning up stale file: /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage/ext_data/nbconvert/__pycache__/postprocess.cpython-39.pyc
[sagelib-9.7.beta1] Finished cleaning, time: 0.40 seconds.
[sagelib-9.7.beta1]
[sagelib-9.7.beta1] real 0m22.506s
[sagelib-9.7.beta1] user 0m17.144s
[sagelib-9.7.beta1] sys 0m5.558s
real 0m22.669s
user 0m17.244s
sys 0m5.624s
Sage build/upgrade complete!
Without this ticket, the log is comparatively short:
...
...
***********************************************
[sagelib-9.7.beta1] Building interpreters for fast_callable
[sagelib-9.7.beta1] Discovering Python/Cython source code....
[sagelib-9.7.beta1] distributions = ['']
[sagelib-9.7.beta1] Discovered Python/Cython sources, time: 5.70 seconds.
[sagelib-9.7.beta1] running build
[sagelib-9.7.beta1] running build_py
[sagelib-9.7.beta1] copying sage/schemes/product_projective/space.py -> build/lib.macosx-12-x86_64-cpython-39/sage/schemes/product_projective
[sagelib-9.7.beta1] copying sage/schemes/elliptic_curves/ell_curve_isogeny.py -> build/lib.macosx-12-x86_64-cpython-39/sage/schemes/elliptic_curves
[sagelib-9.7.beta1] copying sage/schemes/plane_conics/con_field.py -> build/lib.macosx-12-x86_64-cpython-39/sage/schemes/plane_conics
[sagelib-9.7.beta1] copying sage/schemes/projective/projective_morphism.py -> build/lib.macosx-12-x86_64-cpython-39/sage/schemes/projective
[sagelib-9.7.beta1] copying sage/modules/free_module.py -> build/lib.macosx-12-x86_64-cpython-39/sage/modules
[sagelib-9.7.beta1] running build_ext
[sagelib-9.7.beta1] running build_cython
[sagelib-9.7.beta1] Enabling Cython debugging support
[sagelib-9.7.beta1] INFO: Disabling color, you really want to install colorlog.
[sagelib-9.7.beta1] Disabling color, you really want to install colorlog.
[sagelib-9.7.beta1] Compiling sage/resolutions/free.pyx because it changed.
[sagelib-9.7.beta1] [1/1] Cythonizing sage/resolutions/free.pyx
[sagelib-9.7.beta1] Executing 1 command (using 1 thread)
[sagelib-9.7.beta1] [1/1] build/cythonized/sage/resolutions/free.cpp:8891:18: warning: unused function '__pyx_pw_4sage_9structure_7element_1parent' [-Wunused-function]
[sagelib-9.7.beta1] static PyObject *__pyx_pw_4sage_9structure_7element_1parent(PyObject *__pyx_self, PyObject *__pyx_v_x) {
[sagelib-9.7.beta1] ^
[sagelib-9.7.beta1] build/cythonized/sage/resolutions/free.cpp:8890:13: warning: unused variable '__pyx_doc_4sage_9structure_7element_parent' [-Wunused-variable]
[sagelib-9.7.beta1] static char __pyx_doc_4sage_9structure_7element_parent[] = "parent(x)\nFile: sage/structure/element.pxd (starting at line 6)\n\n Return the parent of the element ``x``.\n\n Usually, this means the mathematical object of which ``x`` is an\n element.\n\n INPUT:\n\n - ``x`` -- an element\n\n OUTPUT:\n\n - If ``x`` is a Sage :class:`Element`, return ``x.parent()``.\n\n - Otherwise, return ``type(x)``.\n\n .. SEEALSO::\n\n `Parents, Conversion and Coercion <http://doc.sagemath.org/html/en/tutorial/tour_coercion.html>`_\n Section in the Sage Tutorial\n\n EXAMPLES::\n\n sage: a = 42\n sage: parent(a)\n Integer Ring\n sage: b = 42/1\n sage: parent(b)\n Rational Field\n sage: c = 42.0\n sage: parent(c)\n Real Field with 53 bits of precision\n\n Some more complicated examples::\n\n sage: x = Partition([3,2,1,1,1])\n sage: parent(x)\n Partitions\n sage: v = vector(RDF, [1,2,3])\n sage: parent(v)\n Vector space of dimension 3 over Real Double Field\n\n The following are not considered to be elements, so the type is\n returned::\n\n sage: d = int(42) # Python int\n sage: parent(d)\n <... 'int'>\n sage: L = list(range(10))\n sage: parent(L)\n <... 'list'>\n ";
[sagelib-9.7.beta1] ^
[sagelib-9.7.beta1] build/cythonized/sage/resolutions/free.cpp:9236:18: warning: unused function '__pyx_pw_4sage_9structure_7element_3have_same_parent' [-Wunused-function]
[sagelib-9.7.beta1] static PyObject *__pyx_pw_4sage_9structure_7element_3have_same_parent(PyObject *__pyx_self, PyObject *__pyx_args, PyObject *__pyx_kwds) {
[sagelib-9.7.beta1] ^
[sagelib-9.7.beta1] build/cythonized/sage/resolutions/free.cpp:9235:13: warning: unused variable '__pyx_doc_4sage_9structure_7element_2have_same_parent' [-Wunused-variable]
[sagelib-9.7.beta1] static char __pyx_doc_4sage_9structure_7element_2have_same_parent[] = "have_same_parent(left, right) -> bool\nFile: sage/structure/element.pxd (starting at line 111)\n\n Return ``True`` if and only if ``left`` and ``right`` have the\n same parent.\n\n .. WARNING::\n\n This function assumes that at least one of the arguments is a\n Sage :class:`Element`. When in doubt, use the slower\n ``parent(left) is parent(right)`` instead.\n\n EXAMPLES::\n\n sage: from sage.structure.element import have_same_parent\n sage: have_same_parent(1, 3)\n True\n sage: have_same_parent(1, 1/2)\n False\n sage: have_same_parent(gap(1), gap(1/2))\n True\n\n These have different types but the same parent::\n\n sage: a = RLF(2)\n sage: b = exp(a)\n sage: type(a)\n <... 'sage.rings.real_lazy.LazyWrapper'>\n sage: type(b)\n <... 'sage.rings.real_lazy.LazyNamedUnop'>\n sage: have_same_parent(a, b)\n True\n ";
[sagelib-9.7.beta1] ^
[sagelib-9.7.beta1] 4 warnings generated.
[sagelib-9.7.beta1] Time to execute 1 command: 4.20 seconds.
[sagelib-9.7.beta1] Total time spent compiling C/C++ extensions: 4.31 seconds.
[sagelib-9.7.beta1] warning: no files found matching '*.inc' anywhere in distribution
[sagelib-9.7.beta1] no previously-included directories found matching 'sage_setup'
[sagelib-9.7.beta1] Cleaning up stale file: /Users/kwankyu/GitHub/sage-dev/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage/ext_data/nbconvert/__pycache__/postprocess.cpython-39.pyc
[sagelib-9.7.beta1]
[sagelib-9.7.beta1] real 0m26.868s
[sagelib-9.7.beta1] user 0m19.950s
[sagelib-9.7.beta1] sys 0m6.686s
real 0m27.035s
user 0m20.050s
sys 0m6.750s
Sage build/upgrade complete!
I think this is a regression.
It is the same if I repeat rebuilding.
Replying to @kwankyu:
When I rebuild sage by
sage -b, I observe much bigger log, including... [sagelib-9.7.beta1] package init file 'sage/misc/__init__.py' not found (or not a regular file) [sagelib-9.7.beta1] package init file 'sage/numerical/__init__.py' not found (or not a regular file) [...]
These messages are normal output when namespace packages are in use.
[sagelib-9.7.beta1] cythonized_files = [('sage/misc', ['./sage/misc/parser.pyx', './sage/misc/classcall_metaclass.pyx', './sage/misc/weak_dict.pyx', './sage/misc/sage_timeit_class.pyx', './sage/misc/lazy_string.pyx', './sage/misc/search.pyx', './sage/misc/c3_controlled.pyx', > ... long long list of cythonized files! ...I think this is a regression.
I agree, this is too much verbosity
Replying to @mkoeppe:
Replying to @kwankyu:
When I rebuild sage by
sage -b, I observe much bigger log, including... [sagelib-9.7.beta1] package init file 'sage/misc/__init__.py' not found (or not a regular file) [sagelib-9.7.beta1] package init file 'sage/numerical/__init__.py' not found (or not a regular file) [...]These messages are normal output when namespace packages are in use.
They are not very informative and look like error messages.
These long list of changing mode of ...:
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-startuptime.py to 755
[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-ipynb2rst to 755
...
is also not informative. Moreover they look redundant jobs.
Replying to @kwankyu:
Replying to @mkoeppe:
Replying to @kwankyu:
When I rebuild sage by
sage -b, I observe much bigger log, including... [sagelib-9.7.beta1] package init file 'sage/misc/__init__.py' not found (or not a regular file) [sagelib-9.7.beta1] package init file 'sage/numerical/__init__.py' not found (or not a regular file) [...]These messages are normal output when namespace packages are in use.
They are not very informative and look like error messages.
Is this (googled):
Because .. doesn’t contain an __init__.py, setuptools.find_packages() won’t find the sub-package. You must use setuptools.find_namespace_packages() instead or explicitly list all packages in your setup.py.
the cause of those warnings?
Replying to @kwankyu:
Replying to @mkoeppe:
Replying to @kwankyu:
When I rebuild sage by
sage -b, I observe much bigger log, including... [sagelib-9.7.beta1] package init file 'sage/misc/__init__.py' not found (or not a regular file) [sagelib-9.7.beta1] package init file 'sage/numerical/__init__.py' not found (or not a regular file) [...]These messages are normal output when namespace packages are in use.
They are not very informative and look like error messages.
I agree, but this is an upstream setuptools concern.
These long list of
changing mode of ...:[sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-startuptime.py to 755 [sagelib-9.7.beta1] changing mode of /Users/kwankyu/GitHub/sage/local/var/lib/sage/venv-python3.9/bin/sage-ipynb2rst to 755 ...is also not informative. Moreover they look redundant jobs.
I think this is not new from this ticket. And it's also an upstream setuptools concern.
Replying to @kwankyu:
Replying to @kwankyu:
Replying to @mkoeppe:
Replying to @kwankyu:
When I rebuild sage by
sage -b, I observe much bigger log, including... [sagelib-9.7.beta1] package init file 'sage/misc/__init__.py' not found (or not a regular file) [sagelib-9.7.beta1] package init file 'sage/numerical/__init__.py' not found (or not a regular file) [...]These messages are normal output when namespace packages are in use.
They are not very informative and look like error messages.
Is this (googled):
Because .. doesn’t contain an
__init__.py,setuptools.find_packages()won’t find the sub-package. You must usesetuptools.find_namespace_packages()instead or explicitly list all packages in your setup.py.the cause of those warnings?
No. find_namespace_packages lists all directories instead of only directories that have an init.py file. It does not change what setuptools complains about.
Branch pushed to git repo; I updated commit sha1. New commits:
81237e1 | Merge tag '9.7.beta2' into t/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_ |
dd2c525 | Merge tag '9.7.beta3' into t/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_ |
2cc9002 | pkgs/sagemath-objects/setup.py: Remove unused import |
cd4d8e9 | pkgs/sagemath-standard/setup.py: Monkey-patching cython is now done by the sage_build_cython command |
af9ac78 | src/sage_setup/find.py (find_extra_files): Document new arg 'distributions' |
13e66a3 | src/sage_setup/find.py (find_extra_files): Fix indentation |
9ab3200 | src/sage_setup/clean.py (clean_install_dir): Document new arg 'distributions' |
6a76779 | Merge #28925 |
Replying to @kwankyu:
Replying to @mkoeppe:
Replying to @kwankyu:
When I rebuild sage by
sage -b, I observe much bigger log, including... [sagelib-9.7.beta1] package init file 'sage/misc/__init__.py' not found (or not a regular file) [sagelib-9.7.beta1] package init file 'sage/numerical/__init__.py' not found (or not a regular file) [...]These messages are normal output when namespace packages are in use.
They are not very informative and look like error messages.
setuptools 63.1.0 will eliminate them - https://setuptools.pypa.io/en/latest/history.html#v63-1-0
Replying to @mkoeppe:
Replying to @kwankyu:
Replying to @mkoeppe:
Replying to @kwankyu:
When I rebuild sage by
sage -b, I observe much bigger log, including... [sagelib-9.7.beta1] package init file 'sage/misc/__init__.py' not found (or not a regular file) [sagelib-9.7.beta1] package init file 'sage/numerical/__init__.py' not found (or not a regular file) [...]These messages are normal output when namespace packages are in use.
They are not very informative and look like error messages.
setuptools 63.1.0 will eliminate them - https://setuptools.pypa.io/en/latest/history.html#v63-1-0
Nice!
Those changing mode of ... lines are not seen any more already.
Branch pushed to git repo; I updated commit sha1. New commits:
bd185b5 | Merge tag '9.7.beta5' into t/33011/remove___init___py_files_for_packages_designated_to_be_namespace_packages |
I get
sage -t --warn-long 67.8 --random-seed=272636909256794490188818216866457263419 src/sage_setup/find.py
**********************************************************************
File "src/sage_setup/find.py", line 406, in sage_setup.find.installed_files_by_module
Failed example:
files_by_module['sage.graphs.graph_decompositions']
Expected:
set()
Got:
{'sage/graphs/graph_decompositions/__init__.py',
'sage/graphs/graph_decompositions/__pycache__/__init__.cpython-39.pyc'}
**********************************************************************
1 item had failures:
1 of 10 in sage_setup.find.installed_files_by_module
[49 tests, 1 failure, 1.77 s]
----------------------------------------------------------------------
sage -t --warn-long 67.8 --random-seed=272636909256794490188818216866457263419 src/sage_setup/find.py # 1 doctest failed
----------------------------------------------------------------------
There is no __init__.py in src/sage/graphs/graph_decompositions, but there is one in local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage/graphs/graph_decompositions. Even if I delete it, it gets created when I rebuild sage by sage -b.
Do I need to make distclean first?
Hm, I guess the __init__.py file survives in pkgs/sagemath-standard/build/lib.*
make distclean (or just rm -rf pkgs/sagemath-standard/build) will solve it. Not sure if there's a good way to make it work with incremental builds
Replying to @mkoeppe:
Hm, I guess the
__init__.pyfile survives inpkgs/sagemath-standard/build/lib.*
Yes. it's there.
Replying to @mkoeppe:
make distclean(or justrm -rf pkgs/sagemath-standard/build) will solve it. Not sure if there's a good way to make it work with incremental builds
I deleted the directory pkgs/sagemath-standard/build and did make -j8. This will save time, but I don't know if this is a valid way to build sage.
Replying to @kwankyu:
I get
sage -t --warn-long 67.8 --random-seed=272636909256794490188818216866457263419 src/sage_setup/find.py ********************************************************************** File "src/sage_setup/find.py", line 406, in sage_setup.find.installed_files_by_module Failed example: files_by_module['sage.graphs.graph_decompositions'] Expected: set() Got: {'sage/graphs/graph_decompositions/__init__.py', 'sage/graphs/graph_decompositions/__pycache__/__init__.cpython-39.pyc'}
How about adding a comment like
files_by_module['sage.graphs.graph_decompositions'] # make distclean if failed
to guide people what to do to resolve the failure?
Branch pushed to git repo; I updated commit sha1. New commits:
39aa2f1 | src/sage_setup/command/sage_install.py (sage_clean): Clean __init__.py when installed package was ordinary, source package is namespace |
I think I found a proper solution for this
Replying to @mkoeppe:
I think I found a proper solution for this
Yes. It looks good.
I have some questions. The file sage/__init__.py is not deleted yet here. Then can a subpackage of sage be a namespace package? I guess the answer is yes, but no two distribution packages can install into the namespace package. Am I right?
Is the complication of removing sage/__init__.py still valid?
Replying to @kwankyu:
Is the complication of removing
sage/__init__.pystill valid?
I am asking this even though I do not understand the complication (I tried to).
Replying to @kwankyu:
The file
sage/__init__.pyis not deleted yet here. Then can a subpackage ofsagebe a namespace package? I guess the answer is yes, but no two distribution packages can install into the namespace package. Am I right?
That's right.
Is the complication of removing
sage/__init__.pystill valid?
Yes, I've opened #34187 for it
Description changed:
---
+++
@@ -10,5 +10,5 @@
`setup.py` puts the source path in front of `setup.py` because `sage_setup` uses `sage.env` and `is_package_or_sage_namespace_package_dir` (#33033).
But when an old version of `sage` that is an ordinary package is installed already, the source will not shadow it.
-To avoid this complication, in this ticket we do not yet remove `src/sage/__init__.py`.
+To avoid this complication, in this ticket we do not yet remove `src/sage/__init__.py`. That is done in #34187.
I think sage is now ready to get this in.