add intersphinx mapping for SciPy
mwageringel opened this issue · 28 comments
This ticket adds an intersphinx mapping for Sphinx to resolve hyperlinks to the Scipy documentation.
With this change, Sphinx fetches the Scipy inventory file objects.inv from https://docs.scipy.org/doc/scipy/reference/ during the docbuild. It links against the latest version of the Scipy documentation to keep it simple.
If the download fails, for example when building Sage without internet access, the docbuild will still succeed, but the Scipy cross-links will not be resolved.
CC: @embray @jhpalmieri
Component: documentation
Author: Markus Wageringel
Branch/Commit: u/gh-mwageringel/29231 @ d19232d
Issue created by migration from https://trac.sagemath.org/ticket/29231
Commit: 3bd5293
Author: Markus Wageringel
Branch: u/gh-mwageringel/29231
Description changed:
---
+++
@@ -1,4 +1,3 @@
This ticket adds an intersphinx mapping for Sphinx to resolve hyperlinks to the Scipy documentation.
-With this change, Sphinx fetches the Scipy inventory file `objects.inv` from https://docs.scipy.org/doc/scipy/reference/ during the docbuild. (If this fails for some reason, the docbuild will also fail.)
-
+With this change, Sphinx fetches the Scipy inventory file `objects.inv` from https://docs.scipy.org/doc/scipy/reference/ during the docbuild. (If this fails for some reason, the docbuild will also fail.) It links against the latest version of the Scipy documentation to keep it simple.Sage should in principle build without an internet connection, so I am not in favor of this approach. (“If this fails for some reason, the docbuild will also fail.”)
Branch pushed to git repo; I updated commit sha1. New commits:
2bd8a8a | 29231: fallback for scipy inventory file when not connected to internet |
You are right, that is a problem. It is possible to set a local inventory file as a fallback*. For Python, this was done by adding src/doc/common/python3.inv to the repository. However, I do not really like the idea of adding another binary file to the repository that would need to be updated manually.
Instead, I have configured intersphinx to use the python3.inv as fallback for Scipy. This means that cross-links for Scipy are not resolved when the fallback is used, as they are not present in the Python inventory file, but the documentation build still succeeds.
With these changes applied, I have successfully built the documentation without internet access.
*For reference, see Multiple target for the inventory.
Description changed:
---
+++
@@ -1,3 +1,5 @@
This ticket adds an intersphinx mapping for Sphinx to resolve hyperlinks to the Scipy documentation.
-With this change, Sphinx fetches the Scipy inventory file `objects.inv` from https://docs.scipy.org/doc/scipy/reference/ during the docbuild. (If this fails for some reason, the docbuild will also fail.) It links against the latest version of the Scipy documentation to keep it simple.
+With this change, Sphinx fetches the Scipy inventory file `objects.inv` from https://docs.scipy.org/doc/scipy/reference/ during the docbuild. It links against the latest version of the Scipy documentation to keep it simple.
+
+If the download fails, for example when building Sage without internet access, the docbuild will still succeed, but the Scipy cross-links will not be resolved.Replying to @jhpalmieri:
Sage should in principle build without an internet connection, so I am not in favor of this approach. (“If this fails for some reason, the docbuild will also fail.”)
+1.
Though, it would be great that something like :obj:`scipy:scipy.integrate.ode` in Sage documentation resolves correctly (either pointing to the local documentation or online). pplpy intersphinx relies on the fact that its documentation is built and installed. This is implemented in build/pkgs/pplpy/spkg-postinst
$ cat build/pkgs/pplpy/spkg-postinst
if [[ "$SAGE_SPKG_INSTALL_DOCS" != no ]] ; then
# Compile pplpy's documentation
cd src
cd docs
$MAKE html
# install pplpy's documentation
PPLPY_DOCS=$SAGE_SHARE/doc/pplpy
mkdir -p $PPLPY_DOCS
cp -r build/html/* $PPLPY_DOCS
fi
Though, if we start allowing for pplpy to come from the system the intersphinx will be broken and that also needs a solution.
Replying to @videlec:
Though, if we start allowing for
pplpyto come from the system theintersphinxwill be broken and that also needs a solution.
I would suggest to change the pplpy intersphinx mapping to point to the online version of the pplpy documentation instead of the local version. The local inventory file can be used as fallback (and the dummy file as secondary fallback). This ensures that cross-links (to the online version) can still be resolved correctly if either an internet connection or the local pplpy documentation exists during the docbuild.
Admittedly, it might be better if the cross-links resolved to the local documentation in the case when the local objects.inv is used, and to the online documentation if the online version is used, but there does not seem to be an easy way to configure this, as far as I understand.
Branch pushed to git repo; I updated commit sha1. New commits:
ad16967 | 29231: use online pplpy docs for intersphinx with local fallback |
I have just made the change for pplpy as well, so Intersphinx gets the online version of the objects.inv file and uses the local version as fallback. I have successfully built the documentation without internet access. There do not seem to be any pplpy cross-links, so far. The dochtml.log shows that Intersphinx correctly finds the fallback, though.
Branch pushed to git repo; I updated commit sha1. New commits:
d19232d | 29231: fix pyflakes warnings |
This proposal is not acceptable. Building Sage should not trigger any internet connection beyond downloading packages (that can be done prior to the build with sage -d).
One option would actually be to package the various inventories. But that look tedious to maintain.
Do you know what other projects do?
Other projects like numpy, scipy and matplotlib all use the online versions of inventory files. None of them even provides a fallback for the case that the download fails. This also means we cannot follow the pplpy approach of compiling the documentation for those packages ourselves, as it would connect to the internet.
I am not sure why triggering an internet connection is a problem, as long as download failures are handled gracefully. In the worst case, one ends up without cross-references, which is exactly what we have now.
I would like to avoid extra maintenance burdens that would be caused by packaging the inventory files. How about making the download of inventory files optional, similar to the --no-plot docbuild option? It would be nice then if this was used for the documentation on our website, at least, which is probably where most people read the Sage documentation.
Replying to @mwageringel:
Other projects like numpy, scipy and matplotlib all use the online versions of inventory files. None of them even provides a fallback for the case that the download fails. This also means we cannot follow the
pplpyapproach of compiling the documentation for those packages ourselves, as it would connect to the internet.I am not sure why triggering an internet connection is a problem, as long as download failures are handled gracefully. In the worst case, one ends up without cross-references, which is exactly what we have now.
Because a user do not expect make build to trigger an internet connection. I do not quite agree with the term "handled gracefully" since you end up with a different documentation.
I would like to avoid extra maintenance burdens that would be caused by packaging the inventory files.
+1. It makes no sense to have these files as packages. It will be outdated very soon.
How about making the download of inventory files optional, similar to the
--no-plotdocbuild option? It would be nice then if this was used for the documentation on our website, at least, which is probably where most people read the Sage documentation.
Could you make a detailed proposal on sage-devel? This kind of changes in the build system have to go through a common agreement by devs.
Here is the thread on sage-devel. There is also a very similar ticket #27164.
There is also #27495, which is somewhat related.
What's the status of this? There has been discussion of this recently again, and it would be good to do at least something about it, as I need this for #31404 as well.
I think currently this looks good--one thing I might change is for the dummy fallback see if the docs are installed locally first (e.g. if pplpy was installed with SAGE_SPKG_INSTALL_DOCS, then we can find its docs in SAGE_SHARE/doc/pplpy).
I think this is better than nothing for now.
I think the main concern voiced on the mailing list discussion linked in comment:14 is that intersphinx eagerly tries to connect to the internet, which may be undesirable for some people in some situations. To avoid this, Sage could by default ask for permission to connect to the internet to download missing files, but that would be quite a complex change to the build system.
We should just make this optional, enabled by another SAGE_DOC_.... option.
When building the version for the website (see also #31415), we would enable the option.
Sage development has entered the release candidate phase for 9.3. Setting a new milestone for this ticket based on a cursory review of ticket status, priority, and last modification date.
Setting a new milestone for this ticket based on a cursory review.
Stalled in needs_review or needs_info; likely won't make it into Sage 9.5.