bug: 404 page is indexed by search

[celestehorgan]

What's wrong?

When using the search box on Aiven's devportal, the 404 Page not Found page is indexed and returns as a result:

Steps to remediation

The general remediation for these is to add a noindex tag to the 404 page.

The 404 page in Sphinx is generated via a python package (commit 4384416), so we don't have direct access to the page as we would in other static site generators.

According to sphinx-notfound-page's configuration docs, the 404 by default inherits its template from page.html. Since we can't add noindex to page.html (as that would un-index all our pages) we need to create a template specific to the notfound page to un-index.

The other half of this might be convincing the website's OpenSearch implementation to not index a specific page? I don't know nearly enough about OpenSearch to make that call though.