sagemath/sage

Meta-ticket: Modularize sagelib into separate distributions (pip-installable packages) sagemath-... for Sage 10.x

Opened this issue · 264 comments

We modularize sagelib into separate distributions (pip-installable packages), so that these can be built and run with a much smaller set of dependencies. The packages are made available individually on PyPI in source and wheel form. The modularization is done without changing the overall structure of the source tree by means of PEP 420 implicit namespace packages.

Mega-PR #35095 (not for merging; changes are being cherry-picked from here for integration in Sage) adds the new distributions. Instead of defining a distribution package for every possible community or subfield of mathematics that Sage supports, the modularization project introduces three types of distribution packages:

Distributions named after a basic mathematical structure: The packages may also cover a wide range of generalizations/applications of the structure after which they are named. Users who work in a specialized research area will, of course, recognize what structures they need. The down-to-earth naming also creates discoverability by a broader audience. Not many more distribution packages than these 7 are needed:

Distributions named after a third-party non-Python dependency: This makes technical sense because the dependencies will be localized to this distribution package, but it also helps give attribution and visibility to these libraries and projects that Sage depends on.

Standard packages Optional Packages
sagemath-flint sagemath-bliss
sagemath-gap sagemath-brial
sagemath-glpk sagemath-coxeter3
sagemath-giac sagemath-mcqd
sagemath-homfly sagemath-meataxe
sagemath-lcalc sagemath-sirocco
sagemath-libbraiding sagemath-tdlib
sagemath-libecm
sagemath-linbox
sagemath-ntl
sagemath-pari
sagemath-singular

Distributions named after a technical functionality:

  • sagemath-objects: Sage extends Python's object system by dynamic mix-in classes that are driven by categories and axioms. It is loosely modeled on concepts of category theory and inspired by Scratchpad/Axiom/FriCAS, Magma, and MuPAD. This distribution package makes Sage objects, the element/parent framework, basic categories and functors, the coercion system and the related metaclasses available.
  • sagemath-categories: This distribution package contains the full set of categories defined by Sage, as well as basic mathematical objects such as integers and rational numbers, a basic implementation of polynomials, and affine spaces. None of this brings in additional dependencies.
  • sagemath-environment
  • sagemath-plot: Plotting facilities, depending on matplotlib.
  • sagemath-repl: The top-level interactive environment with the preparser that defines the surface language of Sage. This distribution also includes the doctesting facilities, as the doctests are written in the surface language.
  • sagemath-standard: Everything as provided by a standard installation of the Sage distribution. This is reduced to an empty meta-package.
  • sagemath-standard-no-symbolics

Older posts and presentations:


Initial steps: (2020)

  1. In particular, create a package sagemath-objects that makes Sage objects, the element/parent framework, categories, the coercion system and the related metaclasses available. This package would, for example, not provide or know about integers, polynomials, or matrices (thus avoiding all the dependencies on the various libraries implementing these). Ideally it would only have a build dependency on Cython and no runtime dependencies.

    (Doctesting sagemath-objects would still need more of Sage, but some it could also be done by mocking instead of with the real classes.)

    a) These facilities could see a wider use in the mathematical Python community, and in this way we could hope to benefit from a larger developer base. For example, we would hope that sagemath-objects could be ported for use with PyPy instead of CPython.

    b) Building and deploying a user package would be easier if it could depend on a small package such as sagemath-objects only instead of the whole multigigabyteness of Sage.

    This has been completed in Sage 9.6 (#29865).

  2. Remove the mechanism of OptionalExtensions from sagelib.

    Currently, a user would install, for example, the optional package tdlib and then rebuild sagelib so that the OptionalExtension sage.graphs.graph_decompositions.tdlib is built and installed.

    With this ticket, the user would instead install a new optional package sagemath-tdlib (which has tdlib as a dependency); this installs the Extension sage.graphs.graph_decompositions.tdlib (as a namespace package).

    This has been completed in Sage 9.2 (#29701).

Not within the scope of this ticket:

  • We will not change the overall structure of the source tree (SAGE_ROOT/src/sage). We will achieve this by using the mechanism of native namespace packages introduced in Python 3.3 (PEP 420, see #28925).

  • It is not a prerequisite to fight against all circular imports. In particular, a parent class and its corresponding element class would certainly not be separated into separate distribution packages.

Initially noted constraints and challenges: (2020)

  • Inconsistent handling of symlinks by pip/setuptools. We cannot simply create a distribution of a subset of the files in src by creating a flock of symlinks (see #29850) and expect everything to work; notably, pip fails because of longstanding issues with installs from local directories (see discussion in #29854).

    Solution (implemented in Sage 9.2, #29500): Do not use pip for installations from local source directories. Use setup.py bdist_wheel, followed by pip install of the wheel. pip 21.1's --use-feature=in-tree-build makes this workaround unnecessary (#32046, Sage 9.5).

  • The category framework can not really be separated from the Parent/Element/CategoryObject (e.g. coercion is involved in comparisons and binary operations).

    Solution: Therefore they are in the same distribution sagemath-objects.

  • The category framework cannot live without the integers (e.g. the output of .cardinality() expects an integer).

    Near term solution: #32432 makes sagemath-categories larger, to include integers and some other basic rings.

    Possible future solution: The coercion system already has the notion of a global coercion model - which is determined at runtime, not compile time (sage.structure.element.get_coercion_model). When methods in sagemath-objects need to create an integer, they should obtain the appropriate class of integers in the same way. An existing function provides this interface already (but has a static implementation):

    sage: py_scalar_parent(int)
    Integer Ring
    

    #29875 will make this implementation dynamic.

  • Imports in sage.categories:

    bimodules imports QQ, RR
    classical_crystals imports SR
    finite_coxeter_groups imports QQbar
    

    Such imports can be removed using the techniques described in section Module-level runtime dependencies in the Sage developer's guide.

Tickets for Sage 9.2: (2020)

see also Modularization changes in Sage 9.2 (release tour)

Tickets for Sage 9.3: (2021)

see also Modularization changes in Sage 9.3 (release tour)

Tickets for Sage 9.4: (2021)

see also Modularization changes in Sage 9.4 (release tour)

Tickets for Sage 9.5: (2022)

see also Modularization changes in Sage 9.5 (release tour)

Tickets for Sage 9.6: (2022)

see also Modularization changes in Sage 9.6 (release tour)

Tickets for Sage 9.7: (2022)

see also Modularization changes in Sage 9.7 (release tour)

Tickets for Sage 9.8: (2023)

see also Modularization changes in Sage 9.8 (release tour)

Tickets for Sage 10.0 (2023)

Tickets for Sage 10.1 (2023)

Tickets for Sage 10.2 (2023)

Tickets for Sage 10.3 (2023–2024)

Tickets for Sage 10.4 (2024)

See also: Modularization changes in Sage 10.4 (release notes)

Tickets for Sage 10.5 (2024)

See also: Modularization changes in Sage 10.5

Tickets for Sage 10.x (2024)

Wishlist tickets, loose ends

Related:

CC: @videlec @orlitzky @kiwifb @isuruf @tscrim @kliem @jhpalmieri @tobiasdiez @defeo @thierry-FreeBSD @anneschilling

Component: refactoring

Keywords: sd109, sd110, sd111

Issue created by migration from https://trac.sagemath.org/ticket/29705

Description changed:

--- 
+++ 
@@ -1,6 +1,6 @@
 (From https://groups.google.com/d/msg/sage-devel/M9QTWtln6zU/UHwkrmTKBQAJ)
 
-Modularize sagelib into separate distutils packages, so that these can be built and run with a much smaller set of libraries. The packages can be made available individually on !PyPI. The goal would be to do this **without changing the overall structure of the source tree**.
+Modularize sagelib into separate distutils packages, so that these can be built and run with a much smaller set of libraries. The packages can be made available individually on PyPI. The goal would be to do this **without changing the overall structure of the source tree**.
 
 1. In particular, create a package `sage_objects` that makes Sage objects, the element/parent framework, categories, the coercion system and the related metaclasses available. This package would, for example, not provide or know about integers, polynomials, or matrices (thus avoiding all the dependencies on the various libraries implementing these). Ideally it would only have a build dependency on Cython and no runtime dependencies. 
 

Description changed:

--- 
+++ 
@@ -6,7 +6,7 @@
 
    (Doctesting `sage_objects` would still need more of Sage, but some it could also be done by mocking instead of with the real classes.)
 
-   a) These facilities could see a wider use in the mathematical Python community, and in this way we could hope to benefit from a larger developer base. For example, we would hope that `sage_objects` could be ported for use with PyPy instead of !CPython.
+   a) These facilities could see a wider use in the mathematical Python community, and in this way we could hope to benefit from a larger developer base. For example, we would hope that `sage_objects` could be ported for use with PyPy instead of CPython.
 
    b) Building and deploying a user package would be easier if it could depend on a small package such as `sage_objects` only instead of the whole multigigabyteness of Sage.
 

Description changed:

--- 
+++ 
@@ -17,7 +17,7 @@
 
 **Not** within the scope of this ticket:
 
-- We will not change the overall structure of the source tree (`SAGE_ROOT/src/sage`). We will achieve this by using the mechanism of native namespace packages introduced in Python 3.3 (see https://github.com/sagemath/sage-prod/issues/28925).
+- We will not change the overall structure of the source tree (`SAGE_ROOT/src/sage`). We will achieve this by using the mechanism of native namespace packages introduced in Python 3.3 (PEP 420, see https://github.com/sagemath/sage-prod/issues/28925).
 
 - It is not a prerequisite to fight against all circular imports. In particular, a parent class and its corresponding element class would certainly not be separated into separate distutils packages. 
 

Description changed:

--- 
+++ 
@@ -45,6 +45,7 @@
 
 - #29411: make sagelib a script package
 - #29702: Move all code from `src/setup.py` to `sage_setup`
+- #28197: upgrade to ipython 7 (old ipython completer crashes on namespace packages)
 - #21516: Fix sagelib sdist (src/setup.py sdist)
 - #29701: Meta-ticket: Replace use of `OptionalExtension` by namespace packages
 

Description changed:

--- 
+++ 
@@ -45,6 +45,7 @@
 
 - #29411: make sagelib a script package
 - #29702: Move all code from `src/setup.py` to `sage_setup`
+- #29706: Move `Extension` options from `src/module_list.py` to `distutils:` directives in the individual files
 - #28197: upgrade to ipython 7 (old ipython completer crashes on namespace packages)
 - #21516: Fix sagelib sdist (src/setup.py sdist)
 - #29701: Meta-ticket: Replace use of `OptionalExtension` by namespace packages

Description changed:

--- 
+++ 
@@ -52,3 +52,4 @@
 
 **Related:**
 - #21508 Meta-ticket: Clean up `src/setup.py` to bring it to standard distutils behavior
+- https://wiki.sagemath.org/CodeSharingWorkflow

Changed keywords from none to sd109

Last 10 new commits:

47cb578is_package_or_namespace_package_dir: Recognize namespace packages by file 'namespace'
381a507src/module_list.py: Remove explicit list of sage.numerical.backends extensions
4c51133sage_setup/command/sage_install.py: WIP: Disable cleaning
c0a2ed7src/module_list.py: Remove explicit listing of sage.graphs.graph_decompositions extensions
acaa409Merge branch 't/28925/modify_find_python_sources__clean_stale_files_to_support_modularization_of_sagelib_by_native_namespace_packages__pep_420_' into HEAD
6909e23build/pkgs/coxeter3/distros/fedora.txt: New
2b060aesrc/module_list.py: Move OptionalExtension options for sage.libs.coxeter3.coxeter to distutils directive
4f0d07dsrc/sage/libs/coxeter3/decl.pxd: Add coxeter/ prefix to all header includes; src/module_list.py: remove include_dirs option
1497f3fbuild/pkgs/coxeter3/spkg-configure.m4: New
06a5911Merge branch 't/29721/spkg_configure_m4_for_coxeter3' into HEAD

Commit: 06a5911

Description changed:

--- 
+++ 
@@ -45,10 +45,10 @@
 
 - #29411: make sagelib a script package
 - #29702: Move all code from `src/setup.py` to `sage_setup`
-- #29706: Move `Extension` options from `src/module_list.py` to `distutils:` directives in the individual files
+- #29706, #29720, #29721, #29785, #29786: Move `Extension` options from `src/module_list.py` to `distutils:` directives in the individual files
 - #28197: upgrade to ipython 7 (old ipython completer crashes on namespace packages)
 - #21516: Fix sagelib sdist (src/setup.py sdist)
-- #29701: Meta-ticket: Replace use of `OptionalExtension` by namespace packages
+- #29701 (#28925): Meta-ticket: Replace use of `OptionalExtension` by namespace packages
 
 **Related:**
 - #21508 Meta-ticket: Clean up `src/setup.py` to bring it to standard distutils behavior

Description changed:

--- 
+++ 
@@ -45,7 +45,7 @@
 
 - #29411: make sagelib a script package
 - #29702: Move all code from `src/setup.py` to `sage_setup`
-- #29706, #29720, #29721, #29785, #29786: Move `Extension` options from `src/module_list.py` to `distutils:` directives in the individual files
+- #29706, #29720, #29721, #29785, #29786, #29790: Move `Extension` options from `src/module_list.py` to `distutils:` directives in the individual files
 - #28197: upgrade to ipython 7 (old ipython completer crashes on namespace packages)
 - #21516: Fix sagelib sdist (src/setup.py sdist)
 - #29701 (#28925): Meta-ticket: Replace use of `OptionalExtension` by namespace packages

Changed commit from 06a5911 to 4365e5d

Branch pushed to git repo; I updated commit sha1. New commits:

55bd7basrc/module_list.py: Move options for Extensions in sage.graphs to distutils directives
4365e5dMerge branch 't/29790/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files__part_5__sage_graphs_' into t/29705/META-modularize-sagelib

Branch pushed to git repo; I updated commit sha1. New commits:

7d94648src/module_list.py: Move options for Extensions in sage.libs to distutils directives

Changed commit from 4365e5d to 7d94648

Description changed:

--- 
+++ 
@@ -45,7 +45,7 @@
 
 - #29411: make sagelib a script package
 - #29702: Move all code from `src/setup.py` to `sage_setup`
-- #29706, #29720, #29721, #29785, #29786, #29790: Move `Extension` options from `src/module_list.py` to `distutils:` directives in the individual files
+- #29706, #29720, #29721, #29785, #29786, #29790, #29791: Move `Extension` options from `src/module_list.py` to `distutils:` directives in the individual files
 - #28197: upgrade to ipython 7 (old ipython completer crashes on namespace packages)
 - #21516: Fix sagelib sdist (src/setup.py sdist)
 - #29701 (#28925): Meta-ticket: Replace use of `OptionalExtension` by namespace packages

Changed commit from 7d94648 to b3d3d2f

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

9dc7022Merge branch 't/29706/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files' into t/29791/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files__part_6__last_
f78b06dsrc/module_list.py: Move options for Extensions in sage.libs to distutils directives
1b0e29dsrc/module_list.py: Move options for Extensions in sage.matrix to distutils directives
6421e2csrc/module_list.py: Move remaining options for Extensions in sage.libs, sage.rings to distutils directives
b3d3d2fMerge branch 't/29791/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files__part_6__last_' into t/29705/META-modularize-sagelib

Changed commit from b3d3d2f to 891d12a

Branch pushed to git repo; I updated commit sha1. New commits:

2821934Fix sage_setup directives: Use distribution, not package
9052db4Merge branch 't/29720/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files__part_2___optionalextensions_' into t/29705/META-modularize-sagelib
ff710eesrc/sage_setup/optional_extension.py (is_package_installed_and_updated): Factor out from OptionalExtension
891d12asrc/setup.py: Remove use of module_list.py; filter by distributions

Description changed:

--- 
+++ 
@@ -11,6 +11,11 @@
    b) Building and deploying a user package would be easier if it could depend on a small package such as `sage_objects` only instead of the whole multigigabyteness of Sage.
 
 2. Remove the mechanism of `OptionalExtension`s from `sagelib`.
+
+   Currently, a user would install, for example, the optional package `tdlib` and then rebuild `sagelib` so that the `OptionalExtension` `sage.graphs.graph_decompositions.tdlib` is built and installed.
+
+   With this ticket, the user would instead install a new optional package `sage-tdlib` (which has `tdlib` as a dependency); this installs the `Extension` `sage.graphs.graph_decompositions.tdlib` (as a namespace package).
+
 
    This is #29701 Meta-ticket: Replace use of `OptionalExtension` by namespace packages
 
@@ -46,9 +51,9 @@
 - #29411: make sagelib a script package
 - #29702: Move all code from `src/setup.py` to `sage_setup`
 - #29706, #29720, #29721, #29785, #29786, #29790, #29791: Move `Extension` options from `src/module_list.py` to `distutils:` directives in the individual files
+- #29701 (#28925): Replace use of `module_list` and `OptionalExtension` by extending `sage_setup.find_python_sources`
 - #28197: upgrade to ipython 7 (old ipython completer crashes on namespace packages)
 - #21516: Fix sagelib sdist (src/setup.py sdist)
-- #29701 (#28925): Meta-ticket: Replace use of `OptionalExtension` by namespace packages
 
 **Related:**
 - #21508 Meta-ticket: Clean up `src/setup.py` to bring it to standard distutils behavior

Description changed:

--- 
+++ 
@@ -58,3 +58,10 @@
 **Related:**
 - #21508 Meta-ticket: Clean up `src/setup.py` to bring it to standard distutils behavior
 - https://wiki.sagemath.org/CodeSharingWorkflow
+
+
+
+The branch on this meta-ticket is not intended for merging.
+It is an integration branch to facilitate testing. See https://github.com/mkoeppe/sage/pull/35 for automated tests
+
+

Description changed:

--- 
+++ 
@@ -15,9 +15,6 @@
    Currently, a user would install, for example, the optional package `tdlib` and then rebuild `sagelib` so that the `OptionalExtension` `sage.graphs.graph_decompositions.tdlib` is built and installed.
 
    With this ticket, the user would instead install a new optional package `sage-tdlib` (which has `tdlib` as a dependency); this installs the `Extension` `sage.graphs.graph_decompositions.tdlib` (as a namespace package).
-
-
-   This is #29701 Meta-ticket: Replace use of `OptionalExtension` by namespace packages
 
 
 **Not** within the scope of this ticket:

Description changed:

--- 
+++ 
@@ -48,8 +48,9 @@
 - #29411: make sagelib a script package
 - #29702: Move all code from `src/setup.py` to `sage_setup`
 - #29706, #29720, #29721, #29785, #29786, #29790, #29791: Move `Extension` options from `src/module_list.py` to `distutils:` directives in the individual files
-- #29701 (#28925): Replace use of `module_list` and `OptionalExtension` by extending `sage_setup.find_python_sources`
+- #29701 (depends on all of the above): Replace use of `module_list` and `OptionalExtension` by extending `sage_setup.find_python_sources`
 - #28197: upgrade to ipython 7 (old ipython completer crashes on namespace packages)
+- #28925 Modify find_python_sources, clean_stale_files to support modularization of sagelib by native namespace packages (PEP 420)
 - #21516: Fix sagelib sdist (src/setup.py sdist)
 
 **Related:**

Branch pushed to git repo; I updated commit sha1. New commits:

0295c8fsrc/module_list.py: Move options for Extensions in sage.graphs.graph_decompositions to distutils directives
b582789src/sage_setup/find.py: Filter by directive 'sage_setup: distribution = PKG', find Cython modules
63c64d5Merge branches 't/29720/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files__part_2___optionalextensions_', 't/29721/spkg_configure_m4_for_coxeter3', 't/29785/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files__part_3__get_rid_of_uname_specific_', 't/29786/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files__part_4__sage_rings_', 't/29790/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files__part_5__sage_graphs_' and 't/29791/move_extension_options_from_src_module_list_py_to__distutils___directives_in_the_individual_files__part_6__last_' into t/29701/replace_use_of_module_list_optionalextension
ae70c81src/sage_setup/optional_extension.py (is_package_installed_and_updated): Factor out from OptionalExtension
ec7e9c5src/setup.py: Remove use of module_list.py; filter by distributions
bd26ddfMerge branch 't/29701/replace_use_of_module_list_optionalextension' into t/29705/META-modularize-sagelib

Changed commit from 891d12a to bd26ddf

Description changed:

--- 
+++ 
@@ -51,6 +51,7 @@
 - #29701 (depends on all of the above): Replace use of `module_list` and `OptionalExtension` by extending `sage_setup.find_python_sources`
 - #28197: upgrade to ipython 7 (old ipython completer crashes on namespace packages)
 - #28925 Modify find_python_sources, clean_stale_files to support modularization of sagelib by native namespace packages (PEP 420)
+- Install native namespace packages. Do not use eggs (does not seem to be compatible with native namespace packages).  Either install `sagelib` and, e.g., `sage_tdlib` using pip -- this writes into one shared directory `site-packages/sage/`; or use our `sage_install` command, which installs in the same format.
 - #21516: Fix sagelib sdist (src/setup.py sdist)
 
 **Related:**

Changed commit from bd26ddf to 26b5575

Branch pushed to git repo; I updated commit sha1. New commits:

38b6bcfMerge tag '9.2.beta0' into t/29411/make_sagelib_a_script_package
f9a30f6build/pkgs/sagelib/spkg-install: Fix up error exits
00a1d57Merge branch 't/29411/make_sagelib_a_script_package' into t/29702/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup
25a2340Merge branch 't/29702/move_all_code_from_src_setup_py__src_fpickle_setup_py_to_sage_setup' into t/29701/replace_use_of_module_list_optionalextension
d7bd69eMerge branch 't/29701/replace_use_of_module_list_optionalextension' into t/29705/META-modularize-sagelib
2d866a3src/sage/graphs/graph_decompositions/tdlib.pyx: Fixup
14f488fMerge branch 't/29701/replace_use_of_module_list_optionalextension' into t/29705/META-modularize-sagelib
0d9f653WIP: Add sage_tdlib package
26b5575Use sage_install

Description changed:

--- 
+++ 
@@ -52,6 +52,7 @@
 - #28197: upgrade to ipython 7 (old ipython completer crashes on namespace packages)
 - #28925 Modify find_python_sources, clean_stale_files to support modularization of sagelib by native namespace packages (PEP 420)
 - Install native namespace packages. Do not use eggs (does not seem to be compatible with native namespace packages).  Either install `sagelib` and, e.g., `sage_tdlib` using pip -- this writes into one shared directory `site-packages/sage/`; or use our `sage_install` command, which installs in the same format.
+- Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #21516: Fix sagelib sdist (src/setup.py sdist)
 
 **Related:**

Description changed:

--- 
+++ 
@@ -50,7 +50,9 @@
 - #29706, #29720, #29721, #29785, #29786, #29790, #29791: Move `Extension` options from `src/module_list.py` to `distutils:` directives in the individual files
 - #29701 (depends on all of the above): Replace use of `module_list` and `OptionalExtension` by extending `sage_setup.find_python_sources`
 - #28197: upgrade to ipython 7 (old ipython completer crashes on namespace packages)
-- #28925 Modify find_python_sources, clean_stale_files to support modularization of sagelib by native namespace packages (PEP 420)
+- #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420)
+- #29803: Upgrade `setuptools`, `setuptools_scm`, `pip` (2020-06), add package `wheel`; remove `zope_interface`
+- #13190: make `sagelib` use `setuptools` instead of `distutils`
 - Install native namespace packages. Do not use eggs (does not seem to be compatible with native namespace packages).  Either install `sagelib` and, e.g., `sage_tdlib` using pip -- this writes into one shared directory `site-packages/sage/`; or use our `sage_install` command, which installs in the same format.
 - Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #21516: Fix sagelib sdist (src/setup.py sdist)

Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:

29327d3build/pkgs/entrypoints: Do not provide setup.py, not needed for entrypoints 0.3
bcc79a5build/pkgs/entrypoints/spkg-install.in: Remove pyproject.toml so that setup.py is used
361b98bbuild/pkgs/terminado/spkg-install.in: Remove pyproject.toml so that setup.py is used
5040d7ebuild/pkgs/testpath/spkg-install.in: Remove pyproject.toml so that setup.py is used
371ede6build/pkgs/zope_interface: Change to a dummy script package
b7c63dfMerge branch 't/29803/upgrade_setuptools__pip__2020_06_' into t/29705/META-modularize-sagelib
5f33436src/setup.py: from setuptools import setup
93bbbfeAdd src/setup.cfg
a7de6c0Merge branch 't/13190/make_sagelib_use_setuptools_instead_of_distutils' into t/29705/META-modularize-sagelib
7dd469fWIP: Add sage_tdlib package, use symlinks for sagelib package

Changed commit from 26b5575 to 7dd469f

Description changed:

--- 
+++ 
@@ -53,6 +53,7 @@
 - #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420)
 - #29803: Upgrade `setuptools`, `setuptools_scm`, `pip` (2020-06), add package `wheel`; remove `zope_interface`
 - #13190: make `sagelib` use `setuptools` instead of `distutils`
+- #29845: PEP 517 `buildapi` for `sage_setup`
 - Install native namespace packages. Do not use eggs (does not seem to be compatible with native namespace packages).  Either install `sagelib` and, e.g., `sage_tdlib` using pip -- this writes into one shared directory `site-packages/sage/`; or use our `sage_install` command, which installs in the same format.
 - Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #21516: Fix sagelib sdist (src/setup.py sdist)

Changed commit from 7dd469f to 24e9b30

Branch pushed to git repo; I updated commit sha1. New commits:

24e9b30build/pkgs/sage_tdlib/src/MANIFEST.in: New

Changed commit from 24e9b30 to 9b59435

Branch pushed to git repo; I updated commit sha1. New commits:

9b59435fixup symlinks

Changed commit from 9b59435 to 19e061d

Branch pushed to git repo; I updated commit sha1. New commits:

60ed86csrc/setup.py: WIP: Do not use sage_install
19e061dbuild/pkgs/sage_tdlib/src/sage_setup: Remove, we use the installed one

Changed commit from 19e061d to c3593b2

Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:

c3593b2build/pkgs/sage_tdlib/src/sage_setup: Remove, we use the installed one

Description changed:

--- 
+++ 
@@ -24,6 +24,8 @@
 - It is not a prerequisite to fight against all circular imports. In particular, a parent class and its corresponding element class would certainly not be separated into separate distutils packages. 
 
 **Constraints and challenges:**
+
+- Inconsistent handling of symlinks by `pip`/`setuptools`. We cannot simply create a distribution of a subset of the files in `src` by creating a flock of symlinks (see #29850)
 
 - The category framework can not really be separated from the `Parent`/`Element`/`CategoryObject` (e.g. coercion is involved in comparisons and binary operations).
 

Description changed:

--- 
+++ 
@@ -25,7 +25,7 @@
 
 **Constraints and challenges:**
 
-- Inconsistent handling of symlinks by `pip`/`setuptools`. We cannot simply create a distribution of a subset of the files in `src` by creating a flock of symlinks (see #29850)
+- Inconsistent handling of symlinks by `pip`/`setuptools`. We cannot simply create a distribution of a subset of the files in `src` by creating a flock of symlinks (see #29850) and expect everything to work; notably, pip fails because of longstanding issues with installs from local directories (see discussion in #29854).
 
 - The category framework can not really be separated from the `Parent`/`Element`/`CategoryObject` (e.g. coercion is involved in comparisons and binary operations).
 

Description changed:

--- 
+++ 
@@ -26,6 +26,8 @@
 **Constraints and challenges:**
 
 - Inconsistent handling of symlinks by `pip`/`setuptools`. We cannot simply create a distribution of a subset of the files in `src` by creating a flock of symlinks (see #29850) and expect everything to work; notably, pip fails because of longstanding issues with installs from local directories (see discussion in #29854).
+
+  Solution: Do not use pip for installations from local source directories.  Use `setup.py install` (distutils) directly; or use `setup.py sdist`, followed by `pip install` of the sdist; or use a custom PEP 517 backend (#29845) for making an sdist, followed by `pip install` of the sdist.
 
 - The category framework can not really be separated from the `Parent`/`Element`/`CategoryObject` (e.g. coercion is involved in comparisons and binary operations).
 

Changed commit from c3593b2 to 458d851

Branch pushed to git repo; I updated commit sha1. New commits:

2612091Use setup.py install --single-version-externally-managed
458d851build/pkgs/sage_tdlib/spkg-src: New

Changed commit from 458d851 to 7042564

Branch pushed to git repo; I updated commit sha1. New commits:

5d4e8e7build/pkgs/sage_tdlib/dependencies: Add sagelib, python toolchain
7042564build/pkgs/sagelib/spkg-src: New

Branch pushed to git repo; I updated commit sha1. New commits:

90b33d8Install sagelib, sage_tdlib via sdist defined by MANIFEST.in

Changed commit from 7042564 to 90b33d8

Branch pushed to git repo; I updated commit sha1. New commits:

2ccf6b7Merge tag '9.2.beta1' into t/29705/META-modularize-sagelib

Changed commit from 90b33d8 to 2ccf6b7

Description changed:

--- 
+++ 
@@ -56,9 +56,10 @@
 - #28197: upgrade to ipython 7 (old ipython completer crashes on namespace packages)
 - #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420)
 - #29803: Upgrade `setuptools`, `setuptools_scm`, `pip` (2020-06), add package `wheel`; remove `zope_interface`
+- Install native namespace packages. Do not use eggs (does not seem to be compatible with native namespace packages).  Either install `sagelib` and, e.g., `sage_tdlib` using pip -- this writes into one shared directory `site-packages/sage/`, or use distutils, or disable egg building with setuptools using --single-version-externally-managed.
+   - #29864: Modularization of sagelib: Break out a separate package sage-tdlib
 - #13190: make `sagelib` use `setuptools` instead of `distutils`
 - #29845: PEP 517 `buildapi` for `sage_setup`
-- Install native namespace packages. Do not use eggs (does not seem to be compatible with native namespace packages).  Either install `sagelib` and, e.g., `sage_tdlib` using pip -- this writes into one shared directory `site-packages/sage/`; or use our `sage_install` command, which installs in the same format.
 - Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #21516: Fix sagelib sdist (src/setup.py sdist)
 

Description changed:

--- 
+++ 
@@ -2,13 +2,13 @@
 
 Modularize sagelib into separate distutils packages, so that these can be built and run with a much smaller set of libraries. The packages can be made available individually on PyPI. The goal would be to do this **without changing the overall structure of the source tree**.
 
-1. In particular, create a package `sage_objects` that makes Sage objects, the element/parent framework, categories, the coercion system and the related metaclasses available. This package would, for example, not provide or know about integers, polynomials, or matrices (thus avoiding all the dependencies on the various libraries implementing these). Ideally it would only have a build dependency on Cython and no runtime dependencies. 
+1. In particular, create a package `sage-objects` that makes Sage objects, the element/parent framework, categories, the coercion system and the related metaclasses available. This package would, for example, not provide or know about integers, polynomials, or matrices (thus avoiding all the dependencies on the various libraries implementing these). Ideally it would only have a build dependency on Cython and no runtime dependencies. 
 
-   (Doctesting `sage_objects` would still need more of Sage, but some it could also be done by mocking instead of with the real classes.)
+   (Doctesting `sage-objects` would still need more of Sage, but some it could also be done by mocking instead of with the real classes.)
 
-   a) These facilities could see a wider use in the mathematical Python community, and in this way we could hope to benefit from a larger developer base. For example, we would hope that `sage_objects` could be ported for use with PyPy instead of CPython.
+   a) These facilities could see a wider use in the mathematical Python community, and in this way we could hope to benefit from a larger developer base. For example, we would hope that `sage-objects` could be ported for use with PyPy instead of CPython.
 
-   b) Building and deploying a user package would be easier if it could depend on a small package such as `sage_objects` only instead of the whole multigigabyteness of Sage.
+   b) Building and deploying a user package would be easier if it could depend on a small package such as `sage-objects` only instead of the whole multigigabyteness of Sage.
 
 2. Remove the mechanism of `OptionalExtension`s from `sagelib`.
 
@@ -57,7 +57,8 @@
 - #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420)
 - #29803: Upgrade `setuptools`, `setuptools_scm`, `pip` (2020-06), add package `wheel`; remove `zope_interface`
 - Install native namespace packages. Do not use eggs (does not seem to be compatible with native namespace packages).  Either install `sagelib` and, e.g., `sage_tdlib` using pip -- this writes into one shared directory `site-packages/sage/`, or use distutils, or disable egg building with setuptools using --single-version-externally-managed.
-   - #29864: Modularization of sagelib: Break out a separate package sage-tdlib
+   - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib`
+   - #29865: Modularization of sagelib: Break out a separate package `sage-objects`
 - #13190: make `sagelib` use `setuptools` instead of `distutils`
 - #29845: PEP 517 `buildapi` for `sage_setup`
 - Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.

Description changed:

--- 
+++ 
@@ -35,7 +35,13 @@
 
 - The category framework cannot live without the integers (e.g. the output of `.cardinality()` expects an integer). 
 
-  Possible solution: The coercion system already has the notion of a global coercion model - which is determined at runtime, not compile time (`sage.structure.element.get_coercion_model`). When methods in `sage_objects` need to create an integer, they should obtain the appropriate class of integers in the same way.
+  Possible solution: The coercion system already has the notion of a global coercion model - which is determined at runtime, not compile time (`sage.structure.element.get_coercion_model`). When methods in `sage_objects` need to create an integer, they should obtain the appropriate class of integers in the same way. An existing function provides this interface already (but has a static implementation):
+
+   ```
+   sage: py_scalar_parent(int)
+   Integer Ring
+   ```
+  #29875 will make this implementation dynamic.
 
 - Imports in `sage.categories`:
 
@@ -63,6 +69,8 @@
 - #29845: PEP 517 `buildapi` for `sage_setup`
 - Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #21516: Fix sagelib sdist (src/setup.py sdist)
+- #29874: Mock integer class
+- #29875: `sage.structure.coerce.py_scalar_parent`, `py_scalar_to_element`: Make dynamic
 
 **Related:**
 - #21508 Meta-ticket: Clean up `src/setup.py` to bring it to standard distutils behavior

Description changed:

--- 
+++ 
@@ -71,6 +71,7 @@
 - #21516: Fix sagelib sdist (src/setup.py sdist)
 - #29874: Mock integer class
 - #29875: `sage.structure.coerce.py_scalar_parent`, `py_scalar_to_element`: Make dynamic
+- #29877: `sage.categories`: Replace import of `ZZ` by `py_scalar_parent(int)`
 
 **Related:**
 - #21508 Meta-ticket: Clean up `src/setup.py` to bring it to standard distutils behavior

Description changed:

--- 
+++ 
@@ -65,6 +65,7 @@
 - Install native namespace packages. Do not use eggs (does not seem to be compatible with native namespace packages).  Either install `sagelib` and, e.g., `sage_tdlib` using pip -- this writes into one shared directory `site-packages/sage/`, or use distutils, or disable egg building with setuptools using --single-version-externally-managed.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib`
    - #29865: Modularization of sagelib: Break out a separate package `sage-objects`
+   - #29912: Modularization of sagelib: Break out a separate package `sage-ntl`
 - #13190: make `sagelib` use `setuptools` instead of `distutils`
 - #29845: PEP 517 `buildapi` for `sage_setup`
 - Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.

Description changed:

--- 
+++ 
@@ -64,7 +64,7 @@
 - #29803: Upgrade `setuptools`, `setuptools_scm`, `pip` (2020-06), add package `wheel`; remove `zope_interface`
 - Install native namespace packages. Do not use eggs (does not seem to be compatible with native namespace packages).  Either install `sagelib` and, e.g., `sage_tdlib` using pip -- this writes into one shared directory `site-packages/sage/`, or use distutils, or disable egg building with setuptools using --single-version-externally-managed.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib`
-   - #29865: Modularization of sagelib: Break out a separate package `sage-objects`
+   - #29865: Modularization of sagelib: Break out separate packages `sage-objects`, `sage-categories`
    - #29912: Modularization of sagelib: Break out a separate package `sage-ntl`
 - #13190: make `sagelib` use `setuptools` instead of `distutils`
 - #29845: PEP 517 `buildapi` for `sage_setup`

Description changed:

--- 
+++ 
@@ -66,6 +66,7 @@
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib`
    - #29865: Modularization of sagelib: Break out separate packages `sage-objects`, `sage-categories`
    - #29912: Modularization of sagelib: Break out a separate package `sage-ntl`
+   - #29941: Modularization of sagelib: Break out a separate package `sage-repl`
 - #13190: make `sagelib` use `setuptools` instead of `distutils`
 - #29845: PEP 517 `buildapi` for `sage_setup`
 - Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.

Description changed:

--- 
+++ 
@@ -53,24 +53,31 @@
 
   One can try to not populate the whole category system.
 
-**Tickets and steps:**
+**Tickets for Sage 9.2:**
 
 - #29411: make sagelib a script package
 - #29702: Move all code from `src/setup.py` to `sage_setup`
 - #29706, #29720, #29721, #29785, #29786, #29790, #29791: Move `Extension` options from `src/module_list.py` to `distutils:` directives in the individual files
 - #29701 (depends on all of the above): Replace use of `module_list` and `OptionalExtension` by extending `sage_setup.find_python_sources`
 - #28197: upgrade to ipython 7 (old ipython completer crashes on namespace packages)
+- #29803: Upgrade `setuptools`, `setuptools_scm`, `pip` (2020-06), add package `wheel`; remove `zope_interface`
+- #21559: Install `src/bin` scripts by sagelib's `setup.py`, not make
+- #29850: Install `sage-env-config` as part of `sage_conf`
+- Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
+- #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts"
+- #29950: Build sagelib using the installed `sage_setup`, add `spkg-src` (supersedes: #21516: Fix sagelib sdist (src/setup.py sdist))
+- #29865: Modularization of sagelib: Break out separate experimental packages `sage-objects`, `sage-categories`, depending on Cython, pkgconfig, cysignals, gmpy2.  Use them as a testsuite for testing the filtered structure of sagelib.
+- #29869, #29873, #29892, #29883, #16351, #29881, #29880, #29916, #29922: Improve/repair filtered structure of sagelib.
+- #13190: make `sagelib` use `setuptools` instead of `distutils`
+
+**Tickets for Sage 9.3:**
+
 - #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420)
-- #29803: Upgrade `setuptools`, `setuptools_scm`, `pip` (2020-06), add package `wheel`; remove `zope_interface`
 - Install native namespace packages. Do not use eggs (does not seem to be compatible with native namespace packages).  Either install `sagelib` and, e.g., `sage_tdlib` using pip -- this writes into one shared directory `site-packages/sage/`, or use distutils, or disable egg building with setuptools using --single-version-externally-managed.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib`
-   - #29865: Modularization of sagelib: Break out separate packages `sage-objects`, `sage-categories`
    - #29912: Modularization of sagelib: Break out a separate package `sage-ntl`
    - #29941: Modularization of sagelib: Break out a separate package `sage-repl`
-- #13190: make `sagelib` use `setuptools` instead of `distutils`
 - #29845: PEP 517 `buildapi` for `sage_setup`
-- Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
-- #21516: Fix sagelib sdist (src/setup.py sdist)
 - #29874: Mock integer class
 - #29875: `sage.structure.coerce.py_scalar_parent`, `py_scalar_to_element`: Make dynamic
 - #29877: `sage.categories`: Replace import of `ZZ` by `py_scalar_parent(int)`

Description changed:

--- 
+++ 
@@ -74,6 +74,7 @@
 
 - #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420)
 - Install native namespace packages. Do not use eggs (does not seem to be compatible with native namespace packages).  Either install `sagelib` and, e.g., `sage_tdlib` using pip -- this writes into one shared directory `site-packages/sage/`, or use distutils, or disable egg building with setuptools using --single-version-externally-managed.
+   - Rework `sage-objects`, `sage-categories`, and `sagelib` as namespace packages.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib`
    - #29912: Modularization of sagelib: Break out a separate package `sage-ntl`
    - #29941: Modularization of sagelib: Break out a separate package `sage-repl`

Description changed:

--- 
+++ 
@@ -77,7 +77,9 @@
    - Rework `sage-objects`, `sage-categories`, and `sagelib` as namespace packages.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib`
    - #29912: Modularization of sagelib: Break out a separate package `sage-ntl`
+   - Modularization of sagelib: Break out a separate package `sage-flint-arb-e_antic`
    - #29941: Modularization of sagelib: Break out a separate package `sage-repl`
+   - Modularization of sagelib: Break out a separate package `sage-maxima` (depends on ecl and maxima, packages `sage.interfaces.maxima*`, `sage.libs.ecl`)
 - #29845: PEP 517 `buildapi` for `sage_setup`
 - #29874: Mock integer class
 - #29875: `sage.structure.coerce.py_scalar_parent`, `py_scalar_to_element`: Make dynamic

Description changed:

--- 
+++ 
@@ -65,7 +65,7 @@
 - #29850: Install `sage-env-config` as part of `sage_conf`
 - Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts"
-- #29950: Build sagelib using the installed `sage_setup`, add `spkg-src` (supersedes: #21516: Fix sagelib sdist (src/setup.py sdist))
+- #29950: Build sagelib from build/pkgs/sagelib/src, fix `setup.py sdist`, add `spkg-src` and `tox.ini` (supersedes: #21516: Fix sagelib sdist (src/setup.py sdist))
 - #29865: Modularization of sagelib: Break out separate experimental packages `sage-objects`, `sage-categories`, depending on Cython, pkgconfig, cysignals, gmpy2.  Use them as a testsuite for testing the filtered structure of sagelib.
 - #29869, #29873, #29892, #29883, #16351, #29881, #29880, #29916, #29922: Improve/repair filtered structure of sagelib.
 - #13190: make `sagelib` use `setuptools` instead of `distutils`

Description changed:

--- 
+++ 
@@ -76,10 +76,11 @@
 - Install native namespace packages. Do not use eggs (does not seem to be compatible with native namespace packages).  Either install `sagelib` and, e.g., `sage_tdlib` using pip -- this writes into one shared directory `site-packages/sage/`, or use distutils, or disable egg building with setuptools using --single-version-externally-managed.
    - Rework `sage-objects`, `sage-categories`, and `sagelib` as namespace packages.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib`
+   - #30151: Modularization of sagelib: Break out a separate package `sage-meataxe`
    - #29912: Modularization of sagelib: Break out a separate package `sage-ntl`
    - Modularization of sagelib: Break out a separate package `sage-flint-arb-e_antic`
    - #29941: Modularization of sagelib: Break out a separate package `sage-repl`
-   - Modularization of sagelib: Break out a separate package `sage-maxima` (depends on ecl and maxima, packages `sage.interfaces.maxima*`, `sage.libs.ecl`)
+   - Modularization of sagelib: Break out a separate package `sage-maxima` (depends on ecl and maxima, packages `sage.interfaces.maxima*`, `sage.libs.ecl`)... or `sage-calculus`(?)
 - #29845: PEP 517 `buildapi` for `sage_setup`
 - #29874: Mock integer class
 - #29875: `sage.structure.coerce.py_scalar_parent`, `py_scalar_to_element`: Make dynamic

Description changed:

--- 
+++ 
@@ -64,11 +64,12 @@
 - #21559: Install `src/bin` scripts by sagelib's `setup.py`, not make
 - #29850: Install `sage-env-config` as part of `sage_conf`
 - Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
+- #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
 - #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts"
-- #29950: Build sagelib from build/pkgs/sagelib/src, fix `setup.py sdist`, add `spkg-src` and `tox.ini` (supersedes: #21516: Fix sagelib sdist (src/setup.py sdist))
+- #29950: Build sagelib from build/pkgs/sagelib/src, fix `setup.py sdist`, add `spkg-src` and `tox.ini` (supersedes: #21516: Fix sagelib sdist (src/setup.py sdist), #13190: make `sagelib` use `setuptools` instead of `distutils`)
 - #29865: Modularization of sagelib: Break out separate experimental packages `sage-objects`, `sage-categories`, depending on Cython, pkgconfig, cysignals, gmpy2.  Use them as a testsuite for testing the filtered structure of sagelib.
 - #29869, #29873, #29892, #29883, #16351, #29881, #29880, #29916, #29922: Improve/repair filtered structure of sagelib.
-- #13190: make `sagelib` use `setuptools` instead of `distutils`
+ 
 
 **Tickets for Sage 9.3:**
 

Description changed:

--- 
+++ 
@@ -65,27 +65,37 @@
 - #29850: Install `sage-env-config` as part of `sage_conf`
 - Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
-- #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts"
 - #29950: Build sagelib from build/pkgs/sagelib/src, fix `setup.py sdist`, add `spkg-src` and `tox.ini` (supersedes: #21516: Fix sagelib sdist (src/setup.py sdist), #13190: make `sagelib` use `setuptools` instead of `distutils`)
-- #29865: Modularization of sagelib: Break out separate experimental packages `sage-objects`, `sage-categories`, depending on Cython, pkgconfig, cysignals, gmpy2.  Use them as a testsuite for testing the filtered structure of sagelib.
+- #29865: Modularization of sagelib: Prepare experimental subset distributions `sage-objects`, `sage-categories`, depending on Cython, pkgconfig, cysignals, gmpy2.  Use them as a testsuite for testing the filtered structure of sagelib.
 - #29869, #29873, #29892, #29883, #16351, #29881, #29880, #29916, #29922: Improve/repair filtered structure of sagelib.
  
 
 **Tickets for Sage 9.3:**
 
-- #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420)
-- Install native namespace packages. Do not use eggs (does not seem to be compatible with native namespace packages).  Either install `sagelib` and, e.g., `sage_tdlib` using pip -- this writes into one shared directory `site-packages/sage/`, or use distutils, or disable egg building with setuptools using --single-version-externally-managed.
-   - Rework `sage-objects`, `sage-categories`, and `sagelib` as namespace packages.
+- #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420). Keep monolithic package `sagelib` for use in sage-the-distribution. 
+- Deploy mildly modularized distributions `sage-core`, `sage-tdlib`, ... to PyPI.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib`
    - #30151: Modularization of sagelib: Break out a separate package `sage-meataxe`
+- #29039: Deploy `sage_bootstrap` to PyPI
+- #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts". Deploy `sage_setup` to PyPI.
+- #29875: `sage.structure.coerce.py_scalar_parent`, `py_scalar_to_element`: Make dynamic
+- #29877: `sage.categories`: Replace import of `ZZ` by `py_scalar_parent(int)`
+
+
+**Tickets for Sage 9.4:**
+
+- #30152: `sage_setup`: Modify `clean_stale_files` to support out-of-tree namespace packages
+- Further modularization:
+   - Rework `sage-objects`, `sage-categories` as namespace packages.
    - #29912: Modularization of sagelib: Break out a separate package `sage-ntl`
    - Modularization of sagelib: Break out a separate package `sage-flint-arb-e_antic`
    - #29941: Modularization of sagelib: Break out a separate package `sage-repl`
    - Modularization of sagelib: Break out a separate package `sage-maxima` (depends on ecl and maxima, packages `sage.interfaces.maxima*`, `sage.libs.ecl`)... or `sage-calculus`(?)
+   - `sage-singular`
+- #29874: Mock integer class
 - #29845: PEP 517 `buildapi` for `sage_setup`
-- #29874: Mock integer class
-- #29875: `sage.structure.coerce.py_scalar_parent`, `py_scalar_to_element`: Make dynamic
-- #29877: `sage.categories`: Replace import of `ZZ` by `py_scalar_parent(int)`
+
+
 
 **Related:**
 - #21508 Meta-ticket: Clean up `src/setup.py` to bring it to standard distutils behavior

Description changed:

--- 
+++ 
@@ -74,7 +74,7 @@
 
 - #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420). Keep monolithic package `sagelib` for use in sage-the-distribution. 
 - Deploy mildly modularized distributions `sage-core`, `sage-tdlib`, ... to PyPI.
-   - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib`
+   - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib` (and create `sage-core`)
    - #30151: Modularization of sagelib: Break out a separate package `sage-meataxe`
 - #29039: Deploy `sage_bootstrap` to PyPI
 - #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts". Deploy `sage_setup` to PyPI.

Description changed:

--- 
+++ 
@@ -76,6 +76,7 @@
 - Deploy mildly modularized distributions `sage-core`, `sage-tdlib`, ... to PyPI.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib` (and create `sage-core`)
    - #30151: Modularization of sagelib: Break out a separate package `sage-meataxe`
+   - #29171: `sage-giac` (ex `giacpy_sage` + `sage.libs.giac` + dependency on `libgiac`)
 - #29039: Deploy `sage_bootstrap` to PyPI
 - #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts". Deploy `sage_setup` to PyPI.
 - #29875: `sage.structure.coerce.py_scalar_parent`, `py_scalar_to_element`: Make dynamic

Description changed:

--- 
+++ 
@@ -65,6 +65,7 @@
 - #29850: Install `sage-env-config` as part of `sage_conf`
 - Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
+- #29868: pip-installable packages `sage-doc-html`, `sage-doc-pdf`
 - #29950: Build sagelib from build/pkgs/sagelib/src, fix `setup.py sdist`, add `spkg-src` and `tox.ini` (supersedes: #21516: Fix sagelib sdist (src/setup.py sdist), #13190: make `sagelib` use `setuptools` instead of `distutils`)
 - #29865: Modularization of sagelib: Prepare experimental subset distributions `sage-objects`, `sage-categories`, depending on Cython, pkgconfig, cysignals, gmpy2.  Use them as a testsuite for testing the filtered structure of sagelib.
 - #29869, #29873, #29892, #29883, #16351, #29881, #29880, #29916, #29922: Improve/repair filtered structure of sagelib.

Description changed:

--- 
+++ 
@@ -78,6 +78,7 @@
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib` (and create `sage-core`)
    - #30151: Modularization of sagelib: Break out a separate package `sage-meataxe`
    - #29171: `sage-giac` (ex `giacpy_sage` + `sage.libs.giac` + dependency on `libgiac`)
+   - #30332: `sage-brial` (`sage.rings.polynomial.pbori`, `sage.libs.polybori`, `sage.crypto.boolean_function`)
 - #29039: Deploy `sage_bootstrap` to PyPI
 - #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts". Deploy `sage_setup` to PyPI.
 - #29875: `sage.structure.coerce.py_scalar_parent`, `py_scalar_to_element`: Make dynamic

Description changed:

--- 
+++ 
@@ -64,8 +64,6 @@
 - #21559: Install `src/bin` scripts by sagelib's `setup.py`, not make
 - #29850: Install `sage-env-config` as part of `sage_conf`
 - Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
-- #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
-- #29868: pip-installable packages `sage-doc-html`, `sage-doc-pdf`
 - #29950: Build sagelib from build/pkgs/sagelib/src, fix `setup.py sdist`, add `spkg-src` and `tox.ini` (supersedes: #21516: Fix sagelib sdist (src/setup.py sdist), #13190: make `sagelib` use `setuptools` instead of `distutils`)
 - #29865: Modularization of sagelib: Prepare experimental subset distributions `sage-objects`, `sage-categories`, depending on Cython, pkgconfig, cysignals, gmpy2.  Use them as a testsuite for testing the filtered structure of sagelib.
 - #29869, #29873, #29892, #29883, #16351, #29881, #29880, #29916, #29922: Improve/repair filtered structure of sagelib.
@@ -73,6 +71,8 @@
 
 **Tickets for Sage 9.3:**
 
+- #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
+- #29868: pip-installable packages `sage-doc-html`, `sage-doc-pdf`
 - #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420). Keep monolithic package `sagelib` for use in sage-the-distribution. 
 - Deploy mildly modularized distributions `sage-core`, `sage-tdlib`, ... to PyPI.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib` (and create `sage-core`)

Changed commit from 2ccf6b7 to none

Description changed:

--- 
+++ 
@@ -106,7 +106,3 @@
 
 
 
-The branch on this meta-ticket is not intended for merging.
-It is an integration branch to facilitate testing. See https://github.com/mkoeppe/sage/pull/35 for automated tests
-
-

Changed branch from u/mkoeppe/META-modularize-sagelib to none

Description changed:

--- 
+++ 
@@ -62,17 +62,16 @@
 - #28197: upgrade to ipython 7 (old ipython completer crashes on namespace packages)
 - #29803: Upgrade `setuptools`, `setuptools_scm`, `pip` (2020-06), add package `wheel`; remove `zope_interface`
 - #21559: Install `src/bin` scripts by sagelib's `setup.py`, not make
-- #29850: Install `sage-env-config` as part of `sage_conf`
-- Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #29950: Build sagelib from build/pkgs/sagelib/src, fix `setup.py sdist`, add `spkg-src` and `tox.ini` (supersedes: #21516: Fix sagelib sdist (src/setup.py sdist), #13190: make `sagelib` use `setuptools` instead of `distutils`)
-- #29865: Modularization of sagelib: Prepare experimental subset distributions `sage-objects`, `sage-categories`, depending on Cython, pkgconfig, cysignals, gmpy2.  Use them as a testsuite for testing the filtered structure of sagelib.
 - #29869, #29873, #29892, #29883, #16351, #29881, #29880, #29916, #29922: Improve/repair filtered structure of sagelib.
- 
+
 
 **Tickets for Sage 9.3:**
 
+- #29500 Install all Python packages via `pip wheel`, create PEP 503 simple repository for wheels
 - #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
-- #29868: pip-installable packages `sage-doc-html`, `sage-doc-pdf`
+- #29868: pip-installable package `sage-doc`
+- #29850: Install `sage-env-config` as part of `sage_conf`; Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420). Keep monolithic package `sagelib` for use in sage-the-distribution. 
 - Deploy mildly modularized distributions `sage-core`, `sage-tdlib`, ... to PyPI.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib` (and create `sage-core`)
@@ -81,6 +80,7 @@
    - #30332: `sage-brial` (`sage.rings.polynomial.pbori`, `sage.libs.polybori`, `sage.crypto.boolean_function`)
 - #29039: Deploy `sage_bootstrap` to PyPI
 - #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts". Deploy `sage_setup` to PyPI.
+- #29865: Modularization of sagelib: Prepare experimental subset distributions `sage-objects`, `sage-categories`, depending on Cython, pkgconfig, cysignals, gmpy2.  Use them as a testsuite for testing the filtered structure of sagelib.
 - #29875: `sage.structure.coerce.py_scalar_parent`, `py_scalar_to_element`: Make dynamic
 - #29877: `sage.categories`: Replace import of `ZZ` by `py_scalar_parent(int)`
 

Description changed:

--- 
+++ 
@@ -64,7 +64,8 @@
 - #21559: Install `src/bin` scripts by sagelib's `setup.py`, not make
 - #29950: Build sagelib from build/pkgs/sagelib/src, fix `setup.py sdist`, add `spkg-src` and `tox.ini` (supersedes: #21516: Fix sagelib sdist (src/setup.py sdist), #13190: make `sagelib` use `setuptools` instead of `distutils`)
 - #29869, #29873, #29892, #29883, #16351, #29881, #29880, #29916, #29922: Improve/repair filtered structure of sagelib.
-
+- #29171: Move `giacpy_sage` into sage source code
+- #30332: Merge `sage_brial` into sagelib
 
 **Tickets for Sage 9.3:**
 
@@ -76,8 +77,8 @@
 - Deploy mildly modularized distributions `sage-core`, `sage-tdlib`, ... to PyPI.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib` (and create `sage-core`)
    - #30151: Modularization of sagelib: Break out a separate package `sage-meataxe`
-   - #29171: `sage-giac` (ex `giacpy_sage` + `sage.libs.giac` + dependency on `libgiac`)
-   - #30332: `sage-brial` (`sage.rings.polynomial.pbori`, `sage.libs.polybori`, `sage.crypto.boolean_function`)
+   - `sage-giac` (ex `giacpy_sage` in `sage.libs.giac` + dependency on `libgiac`)
+   - `sage-brial` (ex `sage-brial` in `sage.rings.polynomial.pbori` +  `sage.libs.polybori`, `sage.crypto.boolean_function`)
 - #29039: Deploy `sage_bootstrap` to PyPI
 - #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts". Deploy `sage_setup` to PyPI.
 - #29865: Modularization of sagelib: Prepare experimental subset distributions `sage-objects`, `sage-categories`, depending on Cython, pkgconfig, cysignals, gmpy2.  Use them as a testsuite for testing the filtered structure of sagelib.

Description changed:

--- 
+++ 
@@ -27,7 +27,7 @@
 
 - Inconsistent handling of symlinks by `pip`/`setuptools`. We cannot simply create a distribution of a subset of the files in `src` by creating a flock of symlinks (see #29850) and expect everything to work; notably, pip fails because of longstanding issues with installs from local directories (see discussion in #29854).
 
-  Solution: Do not use pip for installations from local source directories.  Use `setup.py install` (distutils) directly; or use `setup.py sdist`, followed by `pip install` of the sdist; or use a custom PEP 517 backend (#29845) for making an sdist, followed by `pip install` of the sdist.
+  Solution: Do not use pip for installations from local source directories.  Use `setup.py install` (distutils) directly; or use `setup.py sdist`, followed by `pip install` of the sdist; or use `setup.py bdist_wheel`, followed by `pip install` of the wheel; or use a custom PEP 517 backend (#29845) for making an sdist, followed by `pip install` of the sdist.
 
 - The category framework can not really be separated from the `Parent`/`Element`/`CategoryObject` (e.g. coercion is involved in comparisons and binary operations).
 

Description changed:

--- 
+++ 
@@ -69,7 +69,8 @@
 
 **Tickets for Sage 9.3:**
 
-- #29500 Install all Python packages via `pip wheel`, create PEP 503 simple repository for wheels
+- #29500 Install all Python packages via `pip wheel`, store wheels in $SAGE_LOCAL/var/lib/sage/wheels
+- #30527 Create PEP 503 simple repository for wheels built during installation
 - #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
 - #29868: pip-installable package `sage-doc`
 - #29850: Install `sage-env-config` as part of `sage_conf`; Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.

Description changed:

--- 
+++ 
@@ -82,6 +82,7 @@
    - `sage-brial` (ex `sage-brial` in `sage.rings.polynomial.pbori` +  `sage.libs.polybori`, `sage.crypto.boolean_function`)
 - #29039: Deploy `sage_bootstrap` to PyPI
 - #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts". Deploy `sage_setup` to PyPI.
+- #30036: pip-installable sage
 - #29865: Modularization of sagelib: Prepare experimental subset distributions `sage-objects`, `sage-categories`, depending on Cython, pkgconfig, cysignals, gmpy2.  Use them as a testsuite for testing the filtered structure of sagelib.
 - #29875: `sage.structure.coerce.py_scalar_parent`, `py_scalar_to_element`: Make dynamic
 - #29877: `sage.categories`: Replace import of `ZZ` by `py_scalar_parent(int)`

Description changed:

--- 
+++ 
@@ -98,6 +98,7 @@
    - #29941: Modularization of sagelib: Break out a separate package `sage-repl`
    - Modularization of sagelib: Break out a separate package `sage-maxima` (depends on ecl and maxima, packages `sage.interfaces.maxima*`, `sage.libs.ecl`)... or `sage-calculus`(?)
    - `sage-singular`
+- #30534 Repackage `pynac` as a pip-installable package
 - #29874: Mock integer class
 - #29845: PEP 517 `buildapi` for `sage_setup`
 

Description changed:

--- 
+++ 
@@ -72,6 +72,7 @@
 - #29500 Install all Python packages via `pip wheel`, store wheels in $SAGE_LOCAL/var/lib/sage/wheels
 - #30527 Create PEP 503 simple repository for wheels built during installation
 - #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
+- #22731: Replace `$SAGE_LOCAL/bin` by more specific variables to make Sage easier to package, use in venvs
 - #29868: pip-installable package `sage-doc`
 - #29850: Install `sage-env-config` as part of `sage_conf`; Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420). Keep monolithic package `sagelib` for use in sage-the-distribution. 

Description changed:

--- 
+++ 
@@ -84,6 +84,7 @@
 - #29039: Deploy `sage_bootstrap` to PyPI
 - #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts". Deploy `sage_setup` to PyPI.
 - #30036: pip-installable sage
+- #30587: Remove import of `ppl` at startup
 - #29865: Modularization of sagelib: Prepare experimental subset distributions `sage-objects`, `sage-categories`, depending on Cython, pkgconfig, cysignals, gmpy2.  Use them as a testsuite for testing the filtered structure of sagelib.
 - #29875: `sage.structure.coerce.py_scalar_parent`, `py_scalar_to_element`: Make dynamic
 - #29877: `sage.categories`: Replace import of `ZZ` by `py_scalar_parent(int)`

Description changed:

--- 
+++ 
@@ -72,6 +72,7 @@
 - #29500 Install all Python packages via `pip wheel`, store wheels in $SAGE_LOCAL/var/lib/sage/wheels
 - #30527 Create PEP 503 simple repository for wheels built during installation
 - #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
+- #30606: `sage.features.Feature.resolution`: If `SAGE_ROOT` is available, recommend system packages
 - #22731: Replace `$SAGE_LOCAL/bin` by more specific variables to make Sage easier to package, use in venvs
 - #29868: pip-installable package `sage-doc`
 - #29850: Install `sage-env-config` as part of `sage_conf`; Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.

Description changed:

--- 
+++ 
@@ -77,6 +77,7 @@
 - #29868: pip-installable package `sage-doc`
 - #29850: Install `sage-env-config` as part of `sage_conf`; Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420). Keep monolithic package `sagelib` for use in sage-the-distribution. 
+- #30666 Add "sage_setup: distribution" directives to all Cython modules needing external libraries
 - Deploy mildly modularized distributions `sage-core`, `sage-tdlib`, ... to PyPI.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib` (and create `sage-core`)
    - #30151: Modularization of sagelib: Break out a separate package `sage-meataxe`

Description changed:

--- 
+++ 
@@ -73,6 +73,8 @@
 - #30527 Create PEP 503 simple repository for wheels built during installation
 - #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
 - #30606: `sage.features.Feature.resolution`: If `SAGE_ROOT` is available, recommend system packages
+- #30647: Make `lazy_import` more friendly to pyflakes and other static checkers
+- #30616: Replace use of `sage.misc.package.PackageNotFoundError` (2); lazy_import: Add keyword argument 'feature'
 - #22731: Replace `$SAGE_LOCAL/bin` by more specific variables to make Sage easier to package, use in venvs
 - #29868: pip-installable package `sage-doc`
 - #29850: Install `sage-env-config` as part of `sage_conf`; Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.

Description changed:

--- 
+++ 
@@ -66,10 +66,10 @@
 - #29869, #29873, #29892, #29883, #16351, #29881, #29880, #29916, #29922: Improve/repair filtered structure of sagelib.
 - #29171: Move `giacpy_sage` into sage source code
 - #30332: Merge `sage_brial` into sagelib
+- #29500 Install all Python packages via `pip wheel`, store wheels in $SAGE_LOCAL/var/lib/sage/wheels
 
 **Tickets for Sage 9.3:**
 
-- #29500 Install all Python packages via `pip wheel`, store wheels in $SAGE_LOCAL/var/lib/sage/wheels
 - #30527 Create PEP 503 simple repository for wheels built during installation
 - #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
 - #30606: `sage.features.Feature.resolution`: If `SAGE_ROOT` is available, recommend system packages

Description changed:

--- 
+++ 
@@ -71,6 +71,7 @@
 **Tickets for Sage 9.3:**
 
 - #30527 Create PEP 503 simple repository for wheels built during installation
+- #30715 Remove a few `.all` imports
 - #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
 - #30606: `sage.features.Feature.resolution`: If `SAGE_ROOT` is available, recommend system packages
 - #30647: Make `lazy_import` more friendly to pyflakes and other static checkers

Description changed:

--- 
+++ 
@@ -86,6 +86,7 @@
    - #30151: Modularization of sagelib: Break out a separate package `sage-meataxe`
    - `sage-giac` (ex `giacpy_sage` in `sage.libs.giac` + dependency on `libgiac`)
    - `sage-brial` (ex `sage-brial` in `sage.rings.polynomial.pbori` +  `sage.libs.polybori`, `sage.crypto.boolean_function`)
+   - #30716 Towards making Arb library optional
 - #29039: Deploy `sage_bootstrap` to PyPI
 - #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts". Deploy `sage_setup` to PyPI.
 - #30036: pip-installable sage

Description changed:

--- 
+++ 
@@ -67,15 +67,15 @@
 - #29171: Move `giacpy_sage` into sage source code
 - #30332: Merge `sage_brial` into sagelib
 - #29500 Install all Python packages via `pip wheel`, store wheels in $SAGE_LOCAL/var/lib/sage/wheels
+- #30606: `sage.features.Feature.resolution`: If `SAGE_ROOT` is available, recommend system packages
+- #30616: Replace use of `sage.misc.package.PackageNotFoundError` (2); lazy_import: Add keyword argument 'feature'
 
 **Tickets for Sage 9.3:**
 
 - #30527 Create PEP 503 simple repository for wheels built during installation
 - #30715 Remove a few `.all` imports
 - #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
-- #30606: `sage.features.Feature.resolution`: If `SAGE_ROOT` is available, recommend system packages
 - #30647: Make `lazy_import` more friendly to pyflakes and other static checkers
-- #30616: Replace use of `sage.misc.package.PackageNotFoundError` (2); lazy_import: Add keyword argument 'feature'
 - #22731: Replace `$SAGE_LOCAL/bin` by more specific variables to make Sage easier to package, use in venvs
 - #29868: pip-installable package `sage-doc`
 - #29850: Install `sage-env-config` as part of `sage_conf`; Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.

Description changed:

--- 
+++ 
@@ -74,16 +74,19 @@
 
 - #30527 Create PEP 503 simple repository for wheels built during installation
 - #30715 Remove a few `.all` imports
+- #30739 Remove a few more `.all` imports
+- #30779 Duplicate `src/setup.py`
+- #30780 `sage_setup`: Separate installing and cleaning
+- #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420)
+- #30151: Modularization of sagelib: Break out a separate package `sage-meataxe`
 - #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
 - #30647: Make `lazy_import` more friendly to pyflakes and other static checkers
 - #22731: Replace `$SAGE_LOCAL/bin` by more specific variables to make Sage easier to package, use in venvs
 - #29868: pip-installable package `sage-doc`
 - #29850: Install `sage-env-config` as part of `sage_conf`; Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
-- #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420). Keep monolithic package `sagelib` for use in sage-the-distribution. 
 - #30666 Add "sage_setup: distribution" directives to all Cython modules needing external libraries
 - Deploy mildly modularized distributions `sage-core`, `sage-tdlib`, ... to PyPI.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib` (and create `sage-core`)
-   - #30151: Modularization of sagelib: Break out a separate package `sage-meataxe`
    - `sage-giac` (ex `giacpy_sage` in `sage.libs.giac` + dependency on `libgiac`)
    - `sage-brial` (ex `sage-brial` in `sage.rings.polynomial.pbori` +  `sage.libs.polybori`, `sage.crypto.boolean_function`)
    - #30716 Towards making Arb library optional

Description changed:

--- 
+++ 
@@ -75,6 +75,7 @@
 - #30527 Create PEP 503 simple repository for wheels built during installation
 - #30715 Remove a few `.all` imports
 - #30739 Remove a few more `.all` imports
+- #30784 `sage.matrix`: Resolve circular imports without using `__init__.py`
 - #30779 Duplicate `src/setup.py`
 - #30780 `sage_setup`: Separate installing and cleaning
 - #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420)

Changed keywords from sd109 to sd109, sd110

Description changed:

--- 
+++ 
@@ -16,6 +16,7 @@
 
    With this ticket, the user would instead install a new optional package `sage-tdlib` (which has `tdlib` as a dependency); this installs the `Extension` `sage.graphs.graph_decompositions.tdlib` (as a namespace package).
 
+   This has been completed in Sage 9.2 (#29701).
 
 **Not** within the scope of this ticket:
 
@@ -27,7 +28,7 @@
 
 - Inconsistent handling of symlinks by `pip`/`setuptools`. We cannot simply create a distribution of a subset of the files in `src` by creating a flock of symlinks (see #29850) and expect everything to work; notably, pip fails because of longstanding issues with installs from local directories (see discussion in #29854).
 
-  Solution: Do not use pip for installations from local source directories.  Use `setup.py install` (distutils) directly; or use `setup.py sdist`, followed by `pip install` of the sdist; or use `setup.py bdist_wheel`, followed by `pip install` of the wheel; or use a custom PEP 517 backend (#29845) for making an sdist, followed by `pip install` of the sdist.
+  Solution (implemented in Sage 9.2, #29500): Do not use pip for installations from local source directories.  Use `setup.py bdist_wheel`, followed by `pip install` of the wheel.
 
 - The category framework can not really be separated from the `Parent`/`Element`/`CategoryObject` (e.g. coercion is involved in comparisons and binary operations).
 

Description changed:

--- 
+++ 
@@ -1,6 +1,9 @@
 (From https://groups.google.com/d/msg/sage-devel/M9QTWtln6zU/UHwkrmTKBQAJ)
 
 Modularize sagelib into separate distutils packages, so that these can be built and run with a much smaller set of libraries. The packages can be made available individually on PyPI. The goal would be to do this **without changing the overall structure of the source tree**.
+
+Presentation video: https://researchseminars.org/talk/SageDays110/11/
+
 
 1. In particular, create a package `sage-objects` that makes Sage objects, the element/parent framework, categories, the coercion system and the related metaclasses available. This package would, for example, not provide or know about integers, polynomials, or matrices (thus avoiding all the dependencies on the various libraries implementing these). Ideally it would only have a build dependency on Cython and no runtime dependencies. 
 

Description changed:

--- 
+++ 
@@ -76,35 +76,42 @@
 
 **Tickets for Sage 9.3:**
 
-- #30527 Create PEP 503 simple repository for wheels built during installation
 - #30715 Remove a few `.all` imports
 - #30739 Remove a few more `.all` imports
 - #30784 `sage.matrix`: Resolve circular imports without using `__init__.py`
 - #30779 Duplicate `src/setup.py`
 - #30780 `sage_setup`: Separate installing and cleaning
+- #22731: Replace `$SAGE_LOCAL/bin` by more specific variables to make Sage easier to package, use in venvs
+- #30013: src/bin/sage-env: Make sure $SAGE_VENV/bin is at the beginning of the PATH		
+- #29850: Install `sage-env-config` as part of `sage_conf`; Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
+- #30527: Create PEP 503 simple repository for wheels built during installation
+- #30578: Add src/requirements.txt and src/setup.cfg [install_requires] for installation of sagelib in a venv
+- #30371: In-place (editable) installs of sagelib in a venv
+- #30896: `./configure --without-python`
+- #29039: Deploy `sage_bootstrap` to PyPI
+- #30036: pip-installable sage
+
+**Tickets for Sage 9.4:**
+
+- #30666 Add "sage_setup: distribution" directives to all Cython modules needing external libraries
+- #30647: Make `lazy_import` more friendly to pyflakes and other static checkers
+- #29868: pip-installable package `sage-doc`
 - #28925 Modify `find_python_sources`, `clean_stale_files` to support modularization of `sagelib` by native namespace packages (PEP 420)
 - #30151: Modularization of sagelib: Break out a separate package `sage-meataxe`
 - #30010: Split sage_setup.docbuild out to a separate distribution (distutils package)
-- #30647: Make `lazy_import` more friendly to pyflakes and other static checkers
-- #22731: Replace `$SAGE_LOCAL/bin` by more specific variables to make Sage easier to package, use in venvs
-- #29868: pip-installable package `sage-doc`
-- #29850: Install `sage-env-config` as part of `sage_conf`; Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
-- #30666 Add "sage_setup: distribution" directives to all Cython modules needing external libraries
 - Deploy mildly modularized distributions `sage-core`, `sage-tdlib`, ... to PyPI.
    - #29864: Modularization of sagelib: Break out a separate package `sage-tdlib` (and create `sage-core`)
    - `sage-giac` (ex `giacpy_sage` in `sage.libs.giac` + dependency on `libgiac`)
    - `sage-brial` (ex `sage-brial` in `sage.rings.polynomial.pbori` +  `sage.libs.polybori`, `sage.crypto.boolean_function`)
    - #30716 Towards making Arb library optional
-- #29039: Deploy `sage_bootstrap` to PyPI
 - #29847: Install `sage_setup` with `setuptools` as a prerequisite for building/installing sagelib, install sage-the-distribution scripts as "scripts". Deploy `sage_setup` to PyPI.
-- #30036: pip-installable sage
 - #30587: Remove import of `ppl` at startup
 - #29865: Modularization of sagelib: Prepare experimental subset distributions `sage-objects`, `sage-categories`, depending on Cython, pkgconfig, cysignals, gmpy2.  Use them as a testsuite for testing the filtered structure of sagelib.
 - #29875: `sage.structure.coerce.py_scalar_parent`, `py_scalar_to_element`: Make dynamic
 - #29877: `sage.categories`: Replace import of `ZZ` by `py_scalar_parent(int)`
 
 
-**Tickets for Sage 9.4:**
+**Tickets for Sage 9.5:**
 
 - #30152: `sage_setup`: Modify `clean_stale_files` to support out-of-tree namespace packages
 - Further modularization:

Description changed:

--- 
+++ 
@@ -82,14 +82,14 @@
 - #30779 Duplicate `src/setup.py`
 - #30780 `sage_setup`: Separate installing and cleaning
 - #22731: Replace `$SAGE_LOCAL/bin` by more specific variables to make Sage easier to package, use in venvs
-- #30013: src/bin/sage-env: Make sure $SAGE_VENV/bin is at the beginning of the PATH		
+- #30013: `src/bin/sage-env`: Make sure `$SAGE_VENV/bin` is at the beginning of the `PATH`
 - #29850: Install `sage-env-config` as part of `sage_conf`; Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #30527: Create PEP 503 simple repository for wheels built during installation
-- #30578: Add src/requirements.txt and src/setup.cfg [install_requires] for installation of sagelib in a venv
+- #30578: Add `src/requirements.txt` and `src/setup.cfg` [install_requires] for installation of sagelib in a venv
 - #30371: In-place (editable) installs of sagelib in a venv
 - #30896: `./configure --without-python`
 - #29039: Deploy `sage_bootstrap` to PyPI
-- #30036: pip-installable sage
+- #30036: Deploy `sagelib` to PyPI
 
 **Tickets for Sage 9.4:**
 

Description changed:

--- 
+++ 
@@ -86,6 +86,7 @@
 - #29850: Install `sage-env-config` as part of `sage_conf`; Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #30527: Create PEP 503 simple repository for wheels built during installation
 - #30578: Add `src/requirements.txt` and `src/setup.cfg` [install_requires] for installation of sagelib in a venv
+- #30912: sagelib: Update metadata for PyPI deployment
 - #30371: In-place (editable) installs of sagelib in a venv
 - #30896: `./configure --without-python`
 - #29039: Deploy `sage_bootstrap` to PyPI

Description changed:

--- 
+++ 
@@ -85,7 +85,7 @@
 - #30013: `src/bin/sage-env`: Make sure `$SAGE_VENV/bin` is at the beginning of the `PATH`
 - #29850: Install `sage-env-config` as part of `sage_conf`; Move `build/pkgs/sage_conf/src/sage_conf.py[.in]` into `src/`, mark it (and also `src/bin/sage-env-config[.in]`) `# sage_setup: distribution = sage_conf`.
 - #30527: Create PEP 503 simple repository for wheels built during installation
-- #30578: Add `src/requirements.txt` and `src/setup.cfg` [install_requires] for installation of sagelib in a venv
+- #30578: Add `src/requirements.txt` for installation of sagelib in a venv
 - #30912: sagelib: Update metadata for PyPI deployment
 - #30371: In-place (editable) installs of sagelib in a venv
 - #30896: `./configure --without-python`