ClusterStatistics Specification Introduced in 8.11 Breaks Compatibility with Earlier 8.x Versions

[cadandcookies]

Java API client version

8.12.0

Java version

17

Elasticsearch Version

8.5.2

Problem description

The current version of the Elasticsearch API Specification requires nonnullable values for fields that are not present in earlier versions of Elastic:
https://github.com/elastic/elasticsearch-specification/blob/74186647f9360ae41292509bf08b36add7f0672c/specification/_types/Stats.ts#L27
https://javadoc.io/doc/co.elastic.clients/elasticsearch-java/latest/co/elastic/clients/elasticsearch/_types/ClusterStatistics.html

With an ES 8.5.2, calling a cross cluster search results in JSON with the following:
{ ... "_clusters": { "total": 1, "successful": 0, "skipped": 0 } ... }

Which does not meet the current definitions.

These seem to have been added in an incompatible manner in these PRs:
elastic/elasticsearch-specification#2301
and this commit on 8.12 branch: 1be77af

I would have expected that these new counters would have been made optional to preserve backwards compatibility with earlier 8.x versions (or even further back.

This does not appear in any of the listed breaking between 8.5.x and 8.12.x, so it seems possibly unintentional. The following is the change page for 8.11, which does not list this as a breaking change: https://www.elastic.co/guide/en/elasticsearch/reference/current/migrating-8.11.html

A minimal reproducible example of this would be the following in Dev Console:

POST /<cluster>:<index>/_search

or as far as I can tell any cross cluster search against a pre-8.11 version.

Recommended remediation would be to make these values nullable to preserve compatibility.