Add "configure --disable-notebook"; show descriptions of optional packages in "configure --help"
mkoeppe opened this issue · 62 comments
We refactor the code in sage_spkg_enable.m4 and sage_spkg_collect.m4 so that it becomes possible to add configure --disable-SPKG options to disable standard packages. This also simplifies build/make/Makefile.in slightly because it no longer has to make a distinction between standard and optional packages.
We demonstrate this by adding one such option, configure --disable-notebook, which disables building the Jupyter notebook package ... and all of its exclusive dependencies. This is useful for people who want to use the system Jupyter notebook -- the Jupyter kernel is still built and can be installed there.
The new option appears before the --enable... options for optional packages.
We also make the configure help a bit more informative (by including the 1-line descriptions of the optional packages) and prettier.
$ ./configure --help
...
Optional Features:
--disable-option-checking ignore unrecognized --enable/--with options
--disable-FEATURE do not include FEATURE (same as --enable-FEATURE=no)
--enable-FEATURE[=ARG] include FEATURE [ARG=yes]
...
--enable-experimental-packages
allow installing experimental packages (default: no
= ask for user confirmation for each package)
--enable-download-from-upstream-url
allow downloading packages from their upstream URL
if they cannot be found on the Sage mirrors
--disable-notebook disable build of the Jupyter notebook and related
packages
--enable-4ti2={no|if_installed (default)|yes}
enable build and use of the optional package 4ti2: Algebraic, geometric
and combinatorial problems on linear spaces
* package info: ./sage -info 4ti2
--disable-4ti2 disable build and uninstall if previously installed by Sage in PREFIX;
same as --enable-4ti2=no
--enable-atlas={no|if_installed (default)|yes}
enable build and use of the optional package atlas: Automatically Tuned
Linear Algebra Software (BLAS implementation)
* package info: ./sage -info atlas
--disable-atlas disable build and uninstall if previously installed by Sage in PREFIX;
same as --enable-atlas=no
--enable-awali={no|if_installed (default)|yes}
enable build and use of the experimental package awali: Computation of/with
finite state machines
* package info: ./sage -info awali
--disable-awali disable build and uninstall if previously installed by Sage in PREFIX;
same as --enable-awali=no
This is also preparation for #30556 (packages that will not work without openssl), and for testing modularized installs (#30778, #29864).
CC: @jhpalmieri @orlitzky @slel @seblabbe @dimpase @embray @kliem
Component: build
Keywords: sd111
Author: Matthias Koeppe
Branch/Commit: 4916415
Reviewer: John Palmieri, Dima Pasechnik
Issue created by migration from https://trac.sagemath.org/ticket/30383
Description changed:
---
+++
@@ -5,3 +5,5 @@
Users can use `configure --disable-SPKG` to disable them.
+
+Needed for #30556Suggestions for a better name for this new package type are welcome
Alternatively, we could also make it possible to disable any standard package.
Description changed:
---
+++
@@ -1,9 +1,4 @@
-These would be like "optional" packages, but enabled by default.
-
-Like "standard" packages, they would be included in the distribution built by `make dist`.
-
-Users can use `configure --disable-SPKG` to disable them.
-
+We add `configure --disable-SPKG` options to disable standard packages. (They will still be installed if they appear as dependencies.)
Needed for #30556New commits:
c73460c | Add options 'configure --disable-SPKG' for standard packages |
22d7ae4 | m4/sage_spkg_*.m4: At the end of configure, indicate which optional/experimental packages are configured to be installed |
bd6c109 | Merge branch 't/29363/at_the_end_of_configure__indicate_which_optional_experimental_packages_are_configured_to_be_installed' into t/30383/new_package_type__optional_enabled_by_default |
3c98700 | Replace SAGE_STANDARD_PACKAGES by use of SAGE_OPTIONAL_INSTALLED_PACKAGES |
182c1fd | m4/sage_spkg_collect.m4: Reduce verbosity by removing 'does not support check for system package' |
Author: Matthias Koeppe
Description changed:
---
+++
@@ -1,4 +1,44 @@
We add `configure --disable-SPKG` options to disable standard packages. (They will still be installed if they appear as dependencies.)
+`./configure --help` shows these options before the `--enable` options for optional/experimental packages; and the help string is deliberately brief.
-Needed for #30556
+```
+$ ./configure --help
+...
+Optional Features:
+ --disable-option-checking ignore unrecognized --enable/--with options
+ --disable-FEATURE do not include FEATURE (same as --enable-FEATURE=no)
+ --enable-FEATURE[=ARG] include FEATURE [ARG=yes]
+ --enable-experimental-packages
+ allow installing experimental packages (default: no
+ = ask for user confirmation for each package)
+ --enable-download-from-upstream-url
+ allow downloading packages from their upstream URL
+ if they cannot be found on the Sage mirrors
+ --disable-alabaster disable standard package alabaster
+ --disable-appnope disable standard package appnope
+ --disable-arb disable standard package arb
+ --disable-attrs disable standard package attrs
+ --disable-babel disable standard package babel
+ --disable-backcall disable standard package backcall
+ ...
+ --disable-zeromq disable standard package zeromq
+ --disable-zipp disable standard package zipp
+ --disable-zlib disable standard package zlib
+ --disable-zn_poly disable standard package zn_poly
+ --disable-zope_interface
+ disable standard package zope_interface
+ --enable-4ti2={no|if_installed|yes}
+ enable build and use of the optional package 4ti2
+ (default: "if_installed")
+ package information: ./sage -info 4ti2
+ --disable-4ti2 disable build and uninstall if previously installed
+ by Sage in PREFIX; same as --enable-4ti2=no
+ --enable-atlas={no|if_installed|yes}
+ enable build and use of the optional package atlas
+ (default: "if_installed")
+ package information: ./sage -info atlas
+```
+
+
+Needed for #30556 (packages that will not work without openssl), and probably will be helpful for testing modularized installs (#29864)Description changed:
---
+++
@@ -9,6 +9,7 @@
--disable-option-checking ignore unrecognized --enable/--with options
--disable-FEATURE do not include FEATURE (same as --enable-FEATURE=no)
--enable-FEATURE[=ARG] include FEATURE [ARG=yes]
+...
--enable-experimental-packages
allow installing experimental packages (default: no
= ask for user confirmation for each package)
@@ -21,7 +22,7 @@
--disable-attrs disable standard package attrs
--disable-babel disable standard package babel
--disable-backcall disable standard package backcall
- ...
+...
--disable-zeromq disable standard package zeromq
--disable-zipp disable standard package zipp
--disable-zlib disable standard package zlibDoesn't this create a bajillion ./configure options that basically do nothing (because the packages are dependencies), and another bajillion that cause sage to function improperly (test suite failures, etc.)?
I think the original approach was more on-track, although I wouldn't have created a separate package type for it (we don't need to proliferate a new package type throughout the build system every time we tweak a bit of metadata). Instead I would have added an "enabled by default" (default: no) feature somewhere to the optional packages. Long-term, a way to control (and depend on) optional features is needed, but that's a much larger undertaking akin to USE flags in Gentoo.
Replying to @orlitzky:
Doesn't this create a bajillion ./configure options that basically do nothing (because the packages are dependencies), and another bajillion that cause sage to function improperly (test suite failures, etc.)?
Yes, a gazillion of "advanced options"
Most "standard" packages are not really required in any strict sense other than passing doctests that test particular features. Making it possible to disable them gives users a remedy when the build fails on a system
Changed keywords from none to sd111
Hoping we can make progress on this ticket this week - https://wiki.sagemath.org/days111
Branch pushed to git repo; I updated commit sha1. New commits:
e64ef1a | Merge tag '9.3.beta4' into t/30383/new_package_type__optional_enabled_by_default |
Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
ea182d7 | Copy changes from build/pkgs/sagelib/src to src |
a1a10b9 | src/VERSION.txt: New |
5697335 | src/setup.cfg: Add license_file=LICENSE.txt |
deb9eb3 | Merge tag '9.3.beta3' into t/30912/sagelib__update_metadata_for_pypi_deployment |
7ad4c0e | Merge tag '9.3.beta4' into t/30912/sagelib__update_metadata_for_pypi_deployment |
27b589b | Merge branch 't/30912/sagelib__update_metadata_for_pypi_deployment' into t/31362/make_all_sage_local__make_all_sage_venv |
8796008 | Put pynac into SAGE_LOCAL, sagelib into SAGE_VENV |
d6831b1 | build/make/Makefile.in: Add all-build-local, all-build-venv, which include dependency on toolchain |
61f6ba6 | Makefile: Add top-level targets build-local, build-venv |
af2e1fb | Merge #31362 |
Branch pushed to git repo; I updated commit sha1. New commits:
340fbb7 | bootstrap: Emit SAGE_SPKG_ENABLE calls for standard packages |
Branch pushed to git repo; I updated commit sha1. New commits:
47e83d8 | Move computation of SAGE_OPTIONAL_{INSTALLED,CLEANED}_PACKAGES to sage_spkg_collect.m4 |
affcad3 | bootstrap: Emit initialization of SAGE_ENABLE_... variables for all packages; do not call SAGE_SPKG_ENABLE for standard packages |
f89c506 | build/make/Makefile.in: Restore variable OPTIONAL_INSTALLED_PACKAGES |
50262a4 | Include short package description in help string for configure --enable-SPKG, improve formatting |
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
0a83740 | Initialize SAGE_ENABLE_... properly, add 'configure --disable-notebook' |
Description changed:
---
+++
@@ -1,6 +1,11 @@
-We add `configure --disable-SPKG` options to disable standard packages. (They will still be installed if they appear as dependencies.)
+We refactor the code in `sage_spkg_enable.m4` and `sage_spkg_collect.m4` so that it becomes possible to add `configure --disable-SPKG` options to disable standard packages. This also simplifies `build/make/Makefile.in` slightly because it no longer has to make a distinction between standard and optional packages.
-`./configure --help` shows these options before the `--enable` options for optional/experimental packages; and the help string is deliberately brief.
+We demonstrate this by adding one such option, `configure --disable-notebook`, which disables building the Jupyter `notebook` package ... and all of its exclusive dependencies. This is useful for people who want to use the system Jupyter notebook -- the Jupyter kernel is still built and can be installed there.
+
+The new option appears before the `--enable...` options for optional packages.
+
+We also make the configure help a bit more informative and prettier.
+
```
$ ./configure --help
@@ -16,30 +21,28 @@
--enable-download-from-upstream-url
allow downloading packages from their upstream URL
if they cannot be found on the Sage mirrors
- --disable-alabaster disable standard package alabaster
- --disable-appnope disable standard package appnope
- --disable-arb disable standard package arb
- --disable-attrs disable standard package attrs
- --disable-babel disable standard package babel
- --disable-backcall disable standard package backcall
-...
- --disable-zeromq disable standard package zeromq
- --disable-zipp disable standard package zipp
- --disable-zlib disable standard package zlib
- --disable-zn_poly disable standard package zn_poly
- --disable-zope_interface
- disable standard package zope_interface
- --enable-4ti2={no|if_installed|yes}
- enable build and use of the optional package 4ti2
- (default: "if_installed")
- package information: ./sage -info 4ti2
- --disable-4ti2 disable build and uninstall if previously installed
- by Sage in PREFIX; same as --enable-4ti2=no
- --enable-atlas={no|if_installed|yes}
- enable build and use of the optional package atlas
- (default: "if_installed")
- package information: ./sage -info atlas
+ --disable-notebook disable build of the Jupyter notebook and related
+ packages
+ --enable-4ti2={no|if_installed⁽ᵈᵉᶠᵃᵘˡᵗ⁾|yes}
+ enable build and use of the optional package 4ti2: Algebraic, geometric
+ and combinatorial problems on linear spaces
+ * package info: ./sage -info 4ti2
+ --disable-4ti2 disable build and uninstall if previously installed by Sage in PREFIX;
+ same as --enable-4ti2=no
+ --enable-atlas={no|if_installed⁽ᵈᵉᶠᵃᵘˡᵗ⁾|yes}
+ enable build and use of the optional package atlas: Automatically Tuned
+ Linear Algebra Software (BLAS implementation)
+ * package info: ./sage -info atlas
+ --disable-atlas disable build and uninstall if previously installed by Sage in PREFIX;
+ same as --enable-atlas=no
+ --enable-awali={no|if_installed⁽ᵈᵉᶠᵃᵘˡᵗ⁾|yes}
+ enable build and use of the experimental package awali: Computation of/with
+ finite state machines
+ * package info: ./sage -info awali
+ --disable-awali disable build and uninstall if previously installed by Sage in PREFIX;
+ same as --enable-awali=no
```
-Needed for #30556 (packages that will not work without openssl), and probably will be helpful for testing modularized installs (#29864)
+This is also preparation for #30556 (packages that will not work without openssl), and for testing modularized installs (#30778, #29864).
+Branch pushed to git repo; I updated commit sha1. New commits:
4146d88 | configure --disable-notebook: Do not disable widgetsnbextension |
a8035aa | build/pkgs/ipywidgets: Upgrade to 7.6.3 |
5141bc3 | build/pkgs/widgetsnbextension: Patch out dependency on notebook (backport from widgetsnbextension-4) |
fee8379 | Merge #31278 |
82d8030 | m4/sage_spkg_enable.m4: Cosmetic change to help strings |
Branch pushed to git repo; I updated commit sha1. New commits:
3e92998 | configure --disable-notebook: Fix up what packages are disabled |
This version of the ticket no longer offers --disable... options for all standard packages, which was criticized previously.
Description changed:
---
+++
@@ -4,7 +4,7 @@
The new option appears before the `--enable...` options for optional packages.
-We also make the configure help a bit more informative and prettier.
+We also make the configure help a bit more informative (by including the 1-line descriptions of the optional packages) and prettier.
```Why the change to matplotlib dependencies? Also the change in line 66 of m4/sage_spkg_configure.m4 renders very strangely (strange font for "default") when I click on the branch here for example or view the diff on my computer. Any ideas what that's about?
Branch pushed to git repo; I updated commit sha1. New commits:
74a83fc | build/pkgs/matplotlib/dependencies: Undo removal of tornado |
Replying to @jhpalmieri:
Why the change to
matplotlibdependencies?
Thanks for catching this - I had thought I could get rid of tornado at some point but jupyterlab_widgets also needs it as a dependency, so that wouldn't help.
Replying to @jhpalmieri:
the change in line 66 of
m4/sage_spkg_configure.m4renders very strangely (strange font for "default") when I click on the branch here for example or view the diff on my computer. Any ideas what that's about?
Sorry, I was playing with Unicode superscripts. If it does not render nice, I can just go back to using ascii
Branch pushed to git repo; I updated commit sha1. New commits:
93d31b9 | Remove use of unicode superscripts in configure --help |
Description changed:
---
+++
@@ -23,19 +23,19 @@
if they cannot be found on the Sage mirrors
--disable-notebook disable build of the Jupyter notebook and related
packages
- --enable-4ti2={no|if_installed⁽ᵈᵉᶠᵃᵘˡᵗ⁾|yes}
+ --enable-4ti2={no|if_installed (default)|yes}
enable build and use of the optional package 4ti2: Algebraic, geometric
and combinatorial problems on linear spaces
* package info: ./sage -info 4ti2
--disable-4ti2 disable build and uninstall if previously installed by Sage in PREFIX;
same as --enable-4ti2=no
- --enable-atlas={no|if_installed⁽ᵈᵉᶠᵃᵘˡᵗ⁾|yes}
+ --enable-atlas={no|if_installed (default)|yes}
enable build and use of the optional package atlas: Automatically Tuned
Linear Algebra Software (BLAS implementation)
* package info: ./sage -info atlas
--disable-atlas disable build and uninstall if previously installed by Sage in PREFIX;
same as --enable-atlas=no
- --enable-awali={no|if_installed⁽ᵈᵉᶠᵃᵘˡᵗ⁾|yes}
+ --enable-awali={no|if_installed (default)|yes}
enable build and use of the experimental package awali: Computation of/with
finite state machines
* package info: ./sage -info awaliTypo in the comments? In m4/sage_spkg_collect.m4, the descriptions for SAGE_OPTIONAL_INSTALLED_PACKAGES and SAGE_OPTIONAL_CLEANED_PACKAGES are the same. Should installed be changed to uninstalled as the last word in the description for the CLEANED version?
Configuring with --disable-notebook does indeed disable the notebook, as advertised. Sage builds, and so does the documentation. I was a little surprised that all doctests passed, too.
I don't know the syntax well enough to be able to thoroughly review this, unfortunately. Anyone else?
Branch pushed to git repo; I updated commit sha1. New commits:
18fbb85 | m4/sage_spkg_collect.m4: Fix description of SAGE_OPTIONAL_CLEANED_PACKAGES |
Replying to @jhpalmieri:
Configuring with
--disable-notebookdoes indeed disable the notebook, as advertised. Sage builds, and so does the documentation. I was a little surprised that all doctests passed, too.
This is, in fact, a weakness in our doctest coverage. We have nothing at all that tests whether the Jupyter notebook works.
Branch pushed to git repo; I updated commit sha1. New commits:
c3e4093 | Rename SAGE_OPTIONAL_CLEANED_PACKAGES to SAGE_OPTIONAL_UNINSTALLED_PACKAGES |
Branch pushed to git repo; I updated commit sha1. New commits:
4916415 | Merge tag '9.3.beta9' into t/30383/new_package_type__optional_enabled_by_default |
Replying to @mkoeppe:
Replying to @jhpalmieri:
Configuring with
--disable-notebookdoes indeed disable the notebook, as advertised. Sage builds, and so does the documentation. I was a little surprised that all doctests passed, too.This is, in fact, a weakness in our doctest coverage. We have nothing at all that tests whether the Jupyter notebook works.
Maybe related to the fact that you can't start the notebook once you are running Sage from the command line? With SageNB you could run notebook() from within Sage, but this is not possible any more, or at least not in an obvious way, or maybe it just hasn't been implemented by us.
I think it's just that nobody has worked on writing such tests. I guess there must be technologies for automatically testing web apps but I don't know about them or if Jupyter already uses them.
Replying to @jhpalmieri:
With SageNB you could run
notebook()from within Sage, but this is not possible any more, or at least not in an obvious way
This is certainly a feature that I certainly could have used if it was available. Transforming an existing Python process to a Jupyter kernel just amounts to running sage.repl.ipython_kernel.__main__, I think. But to my understanding no feature in the Jupyter notebook allows to connect to an existing running kernel. I found some discussions in this direction, for example jupyter/help#298, ipython/ipython#4066
Reviewer: Dima Pasechnik
this looks good to go.
Changed reviewer from Dima Pasechnik to John Palmieri, Dima Pasechnik
Thanks!
Changed branch from u/mkoeppe/new_package_type__optional_enabled_by_default to 4916415