docs.python.org pages have stale version pickers -> make cache expire faster
ned-deily opened this issue · 7 comments
The Sphinx-generated Python doc pages on docs.python.orgs each have an embedded JS version picker that is included in the daily docs rebuild. The version numbers are updated with new releases. But it seems that the CDN cache max age is currently set to 604800 secs (a week). It would be better if the cache expired more frequently, say daily, so that the docs were more up-to-date. There's a lot of confusion at the moment due to the Python 3.5.0 release doc updates (for example, http://bugs.python.org/issue25113).
On #python-infra, @dstufft suggested changing "Surrogate-control: max-age=604800" here:
https://github.com/python/psf-salt/blob/master/salt/docs/config/nginx.docs-backend.conf#L15
and possibly also setting stale-while-revalidate
"like Surrogate-Control: max-age=NNN, stale-while-revalidate=NNN
that'll cause Fastly to serve a "stale" (e.g. older than the max-age document) to someone while it goes in the background and refetches the latest version of that page
so people still get almost all docs served from Fastly's cache, but fastly's cache stays more current"
The documentation is invalidated as needed when it is automatically rebuilt. Unfortunately, the automatic rebuilding rather frequently breaks due to manual changes on the server. I've now repaired it again.
@ewdurbin can we take a look at this, maybe tomorrow, to check reproducibility and see whether it needs fixing? This came up in the context of closing out the Python 2 sunsetting communications tasks.
You should involve @JulienPalard in this as he is the cpython documentation expert responsible for maintaining the docsbuild system.
In theory, each rebuild of the docs should invalidate the cache for each changed page but there may be an issue with some of the JS files pulled into a page like the version picker. There are also issues of browsers just not updating their caches unexpectedly; I've seen this with Safari, for example, where I purged the version picker cached on the CDN, verified that the version picker file was in fact updated by downloading via curl, yet repeated refreshes in the browser continued to use the out-of-date version picker until the browser was restarted.
Yes, rebuilds are automatically invalidating cache, so there is no need to lower the cache retention time. I think this issue can be closed. (I don't have a safari to test though).
I think the only surefire fix is to add a form of cache busting to the url for switcher.js itself.
It is served with appropriate Last Modified and etag headers, but browsers may helpfully ignore those and pull from local cache regardless.
I recommend closing this, and opening an issue on https://github.com/python/docsbuild-scripts to discuss cache-busting for CSS and JS assets.
/switchers.js is already automatically invalidated by docsbuild-scripts. I can confirm it by grepping it in the invalidation logs:
docsbuild@docs:~$ zgrep -c 'PURGE.*switchers.js' /var/log/docsbuild/docsbuild.log.*.gz
...
/var/log/docsbuild/docsbuild.log.12.gz:7
...
I agree: if this happen again, it should be opened in docsbuild-scripts. I recommend closing this.