Restructure docbuild as script packages sagemath_doc_html, sagemath_doc_pdf
Closed this issue · 74 comments
The specific code in Makefile.in for the documentation and its dependencies is moved to the new script packages.
In preparation for #32759:
- We introduce another installation tree (in addition to
SAGE_LOCALandSAGE_VENV), calledSAGE_DOCS. When the installation tree variable is empty, we suppress building the package. - We introduce a mechanism to select an installation tree, the file
trees.txt. Using this mechanism, we associate the new script packages with the installation treeSAGE_DOCS.
Next step:
- #32759
./configure --disable-doc - #29868 pip-installable packages
sagemath-doc-src,sagemath-doc-inventory,sagemath-doc-html,sagemath-doc-pdf - rename
src/doc/bootstraptobuild/pkgs/sagemath_doc_html/bootstrap.
Depends on #30383
CC: @jhpalmieri
Component: documentation
Author: Matthias Koeppe
Branch/Commit: fd298c4
Reviewer: John Palmieri
Issue created by migration from https://trac.sagemath.org/ticket/31356
Setting new milestone based on a cursory review of ticket status, priority, and last modification date.
Branch pushed to git repo; I updated commit sha1. This was a forced push. Last 10 new commits:
c262e44 | build/pkgs/sage_docbuild/src: Update metadata |
52778c3 | build/pkgs/sage_docbuild/spkg-install: Use sdh_setup_bdist_wheel |
b6abe5e | src/sage/docs/conf.py: Use absolute module names for sage_docbuild extensions |
b42920b | trac 30010: (typo) change sage_doctest -> sage_docbuild |
0a472f2 | build/pkgs/sage_docbuild/src/setup.py: Restrict find_namespace_packages to sage_docbuild |
1dfeb84 | build/make/Makefile.in (sage_docbuild-clean): New, run it from build-clean |
881c4af | src/sage_setup/__init__.py: Restore |
4eca937 | build/pkgs/sage_docbuild/src/setup.{cfg,py}: Use an explicit list of packages, add install_requires |
aaab1d3 | build/pkgs/sage_docbuild/dependencies: Add dependencies on .py files |
f68feab | build/pkgs/sagemath_doc_{html,pdf}: New, delegate 'make doc-{html,pdf}' to here |
Branch pushed to git repo; I updated commit sha1. New commits:
0e05512 | build/pkgs/sagemath_doc_html/dependencies: Add sage_docbuild |
Description changed:
---
+++
@@ -1,2 +1,6 @@
In particular, we make building the HTML documentation optional.
+Next step:
+- #29868 pip-installable packages `sagemath-doc-src`, `sagemath-doc-inventory`, `sagemath-doc-html`, `sagemath-doc-pdf`
+
+Description changed:
---
+++
@@ -1,4 +1,4 @@
-In particular, we make building the HTML documentation optional.
+In particular, we make building the HTML documentation disablable using `./configure --disable-doc` - in which case also various sphinx packages are deselected.
Next step:
- #29868 pip-installable packages `sagemath-doc-src`, `sagemath-doc-inventory`, `sagemath-doc-html`, `sagemath-doc-pdf`Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
6dd12bd | build/pkgs/sagemath_doc_{html,pdf}: New, delegate 'make doc-{html,pdf}' to here |
ec43b4c | build/pkgs/sagemath_doc_html/dependencies: Add sage_docbuild |
862c6ac | build/pkgs/sagemath_doc_html: Add SPKG.rst, type |
3cd70ff | build/pkgs/sagemath_doc_html: Add package-version.txt |
Description changed:
---
+++
@@ -1,4 +1,6 @@
In particular, we make building the HTML documentation disablable using `./configure --disable-doc` - in which case also various sphinx packages are deselected.
+
+Also `src/doc/bootstrap` will be renamed `build/pkgs/sagemath_doc_html/bootstrap`.
Next step:
- #29868 pip-installable packages `sagemath-doc-src`, `sagemath-doc-inventory`, `sagemath-doc-html`, `sagemath-doc-pdf`Description changed:
---
+++
@@ -1,4 +1,4 @@
-In particular, we make building the HTML documentation disablable using `./configure --disable-doc` - in which case also various sphinx packages are deselected.
+In particular, we make building the HTML documentation disablable using `./configure --disable-doc` - in which case also various sphinx packages are deselected. (In this case, we also set `SAGE_SPKG_INSTALL_DOCS` explicitly = no, in order to disable the `pplpy` docbuild.)
Also `src/doc/bootstrap` will be renamed `build/pkgs/sagemath_doc_html/bootstrap`.
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
d62fc48 | WIP sagemath_doc_html sagemath_doc_pdf |
Branch pushed to git repo; I updated commit sha1. This was a forced push. New commits:
0968c1c | WIP sagemath_doc_html sagemath_doc_pdf |
Branch pushed to git repo; I updated commit sha1. New commits:
8a6db35 | build/make/Makefile.in: WIP Disablable trees |
Description changed:
---
+++
@@ -1,8 +1,9 @@
In particular, we make building the HTML documentation disablable using `./configure --disable-doc` - in which case also various sphinx packages are deselected. (In this case, we also set `SAGE_SPKG_INSTALL_DOCS` explicitly = no, in order to disable the `pplpy` docbuild.)
-Also `src/doc/bootstrap` will be renamed `build/pkgs/sagemath_doc_html/bootstrap`.
+Also
Next step:
+- #32759 `./configure --disable-doc`
- #29868 pip-installable packages `sagemath-doc-src`, `sagemath-doc-inventory`, `sagemath-doc-html`, `sagemath-doc-pdf`
+- rename `src/doc/bootstrap` to `build/pkgs/sagemath_doc_html/bootstrap`.
-Author: Matthias Koeppe
Branch pushed to git repo; I updated commit sha1. New commits:
c8e19a0 | fixup |
Branch pushed to git repo; I updated commit sha1. New commits:
5a4424b | build/make/Makefile.in (all-sage): Do not include packages installed in SAGE_DOCS |
Description changed:
---
+++
@@ -1,6 +1,8 @@
-In particular, we make building the HTML documentation disablable using `./configure --disable-doc` - in which case also various sphinx packages are deselected. (In this case, we also set `SAGE_SPKG_INSTALL_DOCS` explicitly = no, in order to disable the `pplpy` docbuild.)
+The specific code in `Makefile.in` for the documentation and its dependencies is moved to the new script packages.
-Also
+In preparation for #32579:
+- We introduce another installation tree (in addition to `SAGE_LOCAL` and `SAGE_VENV`), called `SAGE_DOCS`. When the installation tree variable is empty, we suppress building the package.
+- We introduce a mechanism to select an installation tree, the file `trees.txt`. Using this mechanism, we associate the new script packages with the installation tree `SAGE_DOCS`.
Next step:
- #32759 `./configure --disable-doc`Description changed:
---
+++
@@ -1,6 +1,6 @@
The specific code in `Makefile.in` for the documentation and its dependencies is moved to the new script packages.
-In preparation for #32579:
+In preparation for #32759:
- We introduce another installation tree (in addition to `SAGE_LOCAL` and `SAGE_VENV`), called `SAGE_DOCS`. When the installation tree variable is empty, we suppress building the package.
- We introduce a mechanism to select an installation tree, the file `trees.txt`. Using this mechanism, we associate the new script packages with the installation tree `SAGE_DOCS`.
I feel like trees.txt should be tree.txt, unless there is some way to specify more than one tree there. Obviously not a big deal. I would like this file to be documented, though, and maybe a comment in m4/sage_spkg_collect.m4 is the right place.
Yes, actually several trees are allowed to be specified there.
We don't need this quite yet, but a separate environment for the Jupyter notebook could make sense; and then some packages such as jupyter_core would have to be installed in both SAGE_VENV and the new notebook environment.
I'll add a bit to the developer's guide
Branch pushed to git repo; I updated commit sha1. New commits:
cb78741 | src/doc/en/developer/packaging.rst: Exlpain trees.txt |
It's not clear to me what ./configure --enable-sagemath_doc_pdf is supposed to do just by reading ./configure --help. If I run make, will it also run make doc-pdf, or does it just do some prep work for the PDF build? I see this:
--enable-sagemath_doc_pdf={no|if_installed (default)|yes}
enable build and use of the optional package sagemath_doc_pdf
* package info: ./sage -info sagemath_doc_pdf
and then unfortunately ./sage -info sagemath_doc_pdf is not very useful.
I'll add SPKG.rst files to make the info more useful
I think that sage_docbuild should be a dependency of each of the new packages: I just got this error (sagemath_doc_html-none.log):
/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.5.beta5/local/var/lib/sage/venv-python3.9/bin/python3: No module named sage_docbuild
make doc-inventory--
cd /Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.5.beta5 && ./sage --docbuild --no-pdf-links .o inventory
/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.5.beta5/local/var/lib/sage/venv-python3.9/bin/python3: No module named sage_docbuild
make[6]: *** [doc-inventory--.o] Error 1
make[5]: *** [doc-inventory-reference] Error 2
Branch pushed to git repo; I updated commit sha1. New commits:
7ad6cd9 | build/pkgs/sagemath_doc_*/SPKG.rst: New |
Branch pushed to git repo; I updated commit sha1. New commits:
eb65e03 | build/pkgs/sagemath_doc_*/dependencies: Add sage_docbuild |
Branch pushed to git repo; I updated commit sha1. New commits:
7989d87 | src/doc/Makefile: Handle errors from './sage --docbuild --all-documents' |
I'm getting an error with the pdf build, one that I don't get with the develop branch: en/reference/plotting is failing, and the file local/share/doc/sage/latex/en/reference/plotting/plotting.log ends with
Package pdftex.def Info: plot-23.pdf used on input line 864.
(pdftex.def) Requested size: 462.52684pt x 346.89516pt.
<plot-24.pdf, id=486, 462.52798pt x 346.89601pt>
File: plot-24.pdf Graphic file (type pdf)
<use plot-24.pdf>
Package pdftex.def Info: plot-24.pdf used on input line 884.
(pdftex.def) Requested size: 462.52684pt x 346.89516pt.
[23 <./plot-22.pdf>] [24 <./plot-23.pdf>] [25 <./plot-24.pdf
!pdfTeX error: pdflatex (file ./plot-24.pdf): PDF inclusion: type <cmd> cannot
be copied
==> Fatal error occurred, no output PDF file produced!
Any ideas?
No idea, I haven't seen this failure
Branch pushed to git repo; I updated commit sha1. New commits:
b2cf40c | build/make/Makefile.in: Undo reintroduction of SAGE_SKIP_PLOT_DIRECTIVE, SAGE_DOC_MATHJAX settings (fixup) |
I'm also getting an error with the html build:
[developer] The HTML pages are in local/share/doc/sage/html/en/developer.
Error building the documentation.
Traceback (most recent call last):
File "/usr/local/Cellar/python@3.9/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 197, in _run_module_as_main
return _run_code(code, main_globals, None,
File "/usr/local/Cellar/python@3.9/3.9.7_1/Frameworks/Python.framework/Versions/3.9/lib/python3.9/runpy.py", line 87, in _run_code
exec(code, run_globals)
File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.5.beta5/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage_docbuild/__main__.py", line 2, in <module>
main()
File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.5.beta5/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage_docbuild/__init__.py", line 1814, in main
builder()
File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.5.beta5/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage_docbuild/__init__.py", line 133, in f
runsphinx()
File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.5.beta5/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage_docbuild/sphinxbuild.py", line 323, in runsphinx
sys.stderr.raise_errors()
File "/Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.5.beta5/local/var/lib/sage/venv-python3.9/lib/python3.9/site-packages/sage_docbuild/sphinxbuild.py", line 258, in raise_errors
raise OSError(self._error)
OSError: /Users/palmieri/Desktop/Sage/sage_builds/TESTING/sage-9.5.beta5/src/doc/en/developer/packaging.rst:721: WARNING: Duplicate explicit target name: "section-dependencies".
This is after make distclean && ./configure && make.
Thanks, I see this one now too, after make distclean
Branch pushed to git repo; I updated commit sha1. New commits:
37a03bd | src/doc/en/developer/packaging.rst: Fix up section label |
Branch pushed to git repo; I updated commit sha1. New commits:
fc89a66 | build/pkgs/sagemath_doc_pdf/spkg-install: Set SAGE_DOC_MATHJAX as a workaround |
PDF builds for me without error
I think that sagemath_doc_html should be a dependency for sagemath_doc_pdf. Otherwise both may try to build at the same time, and they may both try to build the inventory files at the same time, which could easily lead to race conditions.
Yes, I agree
Branch pushed to git repo; I updated commit sha1. New commits:
fd298c4 | build/pkgs/sagemath_doc_pdf/dependencies: Depend on sagemath_doc_html |
Everything now builds for me.
Thanks for testing!
I think this is ready to go. Merge now or wait until 9.6? I can imagine it breaking something for unforeseen reasons.
Reviewer: John Palmieri
I think we can merge it now
Thank you!