Older doc versions appear to reference new package version

[riedgar-ms]

When building older documentation versions of our package, it appears that the builds of the older documentation version are using the current version of the actual package.

To be concrete, our documentation contains a sphinx-gallery section. The code here is matched to the current version (indeed, changes here are our major reason for wanting versioned documentation). We are trying to build two versions - master and the tag v0.4.6. The former builds fine, but on the latter sphinx-multiversion errors out:

/tmp/tmpks949rp2/66894d7d7cdf663a2caa51bf4d31f080e242b2af/examples/plot_binary_classification_COMPAS.py failed leaving traceback:
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/sphinx_gallery/gen_gallery.py", line 159, in call_memory
    return 0., func()
  File "/usr/local/lib/python3.7/site-packages/sphinx_gallery/gen_rst.py", line 466, in __call__
    exec(self.code, self.fake_main.__dict__)
  File "/tmp/tmpks949rp2/66894d7d7cdf663a2caa51bf4d31f080e242b2af/examples/plot_binary_classification_COMPAS.py", line 325, in <module>
    from fairlearn.postprocessing._roc_curve_utilities import _calculate_roc_points
ModuleNotFoundError: No module named 'fairlearn.postprocessing._roc_curve_utilities'

This can be seen in a recent CircleCI build. We can compare the postprocessing version between master and v0.4.6. The latter contains _roc_curve_utilities.py but the former does not - so this suggests that the examples from v0.4.6 are being built against the code in master.

Although there is no obvious failure, I believe that this is affecting the autodoc documentation too. For example, our DemographicParity Moment in master shows that it is derived from UtilityParity, and the documentation in v0.4.6 shows the same thing. However, note that in the v0.4.6 code the DemographicParity class is derived from ConditionalSelectionRate.

There is also an issue about the links to the source code in the documentation, but that's a problem for some code in conf.py.

The pull request trying to set up sphinx-multiversion:
fairlearn/fairlearn#551

[Holzhaus]

You have this in your conf.py:

sys.path.insert(0, os.path.abspath('../'))

As we're always building with master's conf.py, it will always insert the master branch path here. If you want to load the appropriate version of the fairlearn module, use something like:

rootdir = os.path.join(os.getenv("SPHINX_MULTIVERSION_SOURCEDIR", default="."), "..")
sys.path.insert(0, rootdir)

The same goes for relative paths, e.g. for sphinx galleries.

[riedgar-ms]

That was the issue, thanks. The printout from the start of the conf.py should have reminded me about that extra line (necessary for autodoc etc. to work as I recall).

[zhongjiajie]

You have this in your conf.py:

sys.path.insert(0, os.path.abspath('../'))

As we're always building with master's conf.py, it will always insert the master branch path here. If you want to load the appropriate version of the fairlearn module, use something like:

rootdir = os.path.join(os.getenv("SPHINX_MULTIVERSION_SOURCEDIR", default="."), "..")
sys.path.insert(0, rootdir)

The same goes for relative paths, e.g. for sphinx galleries.

up on Aug 10, 2022
For someone who faces the same issue, the related patch #41 is still not released, but you can still install the package from Github, in my case, I use

doc = [
    "sphinx>=4.3",
    "sphinx_rtd_theme>=1.0",
    "sphinx-click>=3.0",
    "sphinx-inline-tabs",
    "sphinx-copybutton>=0.4.0",
    "sphinx-multiversion @ git+https://github.com/Holzhaus/sphinx-multiversion#egg=sphinx-multiversion",
]

extras_require={
    "doc": doc,
},

in my setup.py file and it work well