kubectl version priority results in get using older version of resources than it should

[matthchr]

The versioning scheme we use for APIs doesn't comply with well with version priority, which states:

Version priority
Regardless of the order in which versions are defined in a CustomResourceDefinition, the version with the highest priority is used by kubectl as the default version to access objects. The priority is determined by parsing the name field to determine the version number, the stability (GA, Beta, or Alpha), and the sequence within that stability level.

The algorithm used for sorting the versions is designed to sort versions in the same way that the Kubernetes project sorts Kubernetes versions. Versions start with a v followed by a number, an optional beta or alpha designation, and optional additional numeric versioning information. Broadly, a version string might look like v2 or v2beta1. Versions are sorted using the following algorithm:

Entries that follow Kubernetes version patterns are sorted before those that do not.
For entries that follow Kubernetes version patterns, the numeric portions of the version string is sorted largest to smallest.
If the strings beta or alpha follow the first numeric portion, they sorted in that order, after the equivalent string without the beta or alpha suffix (which is presumed to be the GA version).

If another number follows the beta, or alpha, those numbers are also sorted from largest to smallest.
Strings that don't fit the above format are sorted alphabetically and the numeric portions are not treated specially. Notice that in the example below, foo1 is sorted above foo10. This is different from the sorting of the numeric portion of entries that do follow the Kubernetes version patterns.

Our APIs are versioned like so:
v1api20210201 - GA ASO, GA ARM API
v1api20240201 - GA ASO, GA ARM API (newer)
v1api20210201preview - GA ASO, Preview ARM API
v1beta20210201 - Beta ASO, GA ARM API
v1beta20210201preview - Beta ASO, Preview ARM API

According to the Kubernetes version priority ordering, these get ordered:

• v1beta20210201 - Kubernetes treats this as iteration 20210201 of v1beta, so it complies the the Kubernetes versioning scheme and is ordered first.
• v1api20210201 - Doesn't comply with Kuberentes versioning structure and so is ordered lexically.
• v1api20210201preview - Doesn't comply with Kuberentes versioning structure and so is ordered lexically.
• v1api20240201 - Doesn't comply with Kuberentes versioning structure and so is ordered lexically.
• v1beta20210201preview - Doesn't comply with Kuberentes versioning structure and so is ordered lexically.

This ordering results in the following problems:

• Beta versions are preferred over GA versions (because the beta versions do comply with the Kubernetes versioning scheme)
• Older GA versions are preferred over newer GA versions (because newer versions sort lexically later)

Workaround
users can work around this problem and specify the specific API version they'd like to use when getting the objects, with kubectl get virtualmachines.v1api20220301.compute.azure.com (specifying the APIVersion as part of the get command), but this is annoying.