Validate URLs in ES JSON spec
Closed this issue · 7 comments
Problem
Elasticsearch engineers are required to create a JSON spec for any new ES REST APIs (e.g., https://github.com/elastic/elasticsearch/blob/master/rest-api-spec/src/main/resources/rest-api-spec/api/get.json)
This spec requires a documentation.url property, which links to an ES docs page.
However, engineers often create and merge the spec before creating the docs page. This means
the link in the documentation.url property is broken.
Some downstream Elasticsearch clients, such as the Elasticsearch-JS client, use these
documentation.url links to generate their own docs.
A broken link in the JSON spec ends up breaking the docs build for the Elasticsearch-JS client and the rest of our docs.
Proposed solution
If we could extract the links in the documentation.url property and check them before passing the docs-CI test, we could prevent ES engineers from merging JSON specs with broken links. This would prevent broken docs builds later.
Just so I understand this completely, we just need to grab all the documentation.url fields out of all the JSON files in the rest-api-spec/src/main/resources/rest-api-spec/api/ directory, and verify that they all point to existing documentation pages?
Is this on all (active) branches, or just master?
Does it need to do anything if there's not a documentation.url property?
Just so I understand this completely, we just need to grab all the documentation.url fields out of all the JSON files in the rest-api-spec/src/main/resources/rest-api-spec/api/ directory, and verify that they all point to existing documentation pages?
Yep. That's correct.
Is this on all (active) branches, or just master?
I would do this on all active branches. Merging to master and backporting to 7.x is a common pattern for new development in ES.
Does it need to do anything if there's not a documentation.url property?
No. That should be fine. The primary concern is broken links.
@mark-vieira and @jakelandis did some work (elastic/elasticsearch#55117) for validating the json spec, so they might have some suggestions about how to solve this issue :)
Thanks @delvedor .
On the Elasticsearch side, all that's really needed (I think) is code to extract all the links (and maybe a hint about what line/file they're found in, to make it easier to track down any broken ones). The code to validate the links would probably best be this repo (we already do something similar for cross-book links and documentation links in the Kibana interface).
I've been thinking about ways to generalize link-checking across products. The code to actually check links is the same regardless, but I've been thinking about having a way for projects to "opt-in" to additional checking using a standard interface to generate the list of links to check. For now, it's probably easiest to just do it all in this repo.
This would have prevented elastic/elasticsearch#60582
There was another case of builds failing which would have been caught earlier by this.
Related: elastic/elasticsearch#40596