"Version" docker tags broken for many images

[easimon]

Details

A recent refactoring on the docker build introduced a typo in most of the build workflows (missing colon).

The push step pushes e.g. docker.io/raspbernetes/kustomizev3.8.2, and since missing tags are replaced by :latest,
this results in docker images named e.g. docker.io/raspbernetes/kustomizev3.8.2:latest.

But there seems to be another issue for the latest tag, since docker.io/raspbernetes/kustomize:latest does not seem to contain the latest version (neither docker.io/raspbernetes/kustomizev3.8.2:latest nor docker.io/raspbernetes/kustomize:v3.8.2, but something else).

What steps did you take and what happened:

Take a look at the following kustomize version outputs:

$ docker run docker.io/raspbernetes/kustomize:v3.8.2 kustomize version
{Version:kustomize/v3.8.2 GitCommit:e2973f6ecc9be6187cfd5ecf5e180f842249b3c6 BuildDate:2020-09-14T09:28:56UTC GoOs:linux GoArch:amd64}

$ docker run docker.io/raspbernetes/kustomizev3.8.2:latest kustomize version
{Version:kustomize/v3.8.2 GitCommit:e2973f6ecc9be6187cfd5ecf5e180f842249b3c6 BuildDate:2020-09-14T09:28:56UTC GoOs:linux GoArch:amd64}

$ docker run docker.io/raspbernetes/kustomize:latest kustomize version
Version: {Version:unknown GitCommit:$Format:%H$ BuildDate:1970-01-01T00:00:00Z GoOs:linux GoArch:amd64}

What did you expect to happen:

• Version tags should be created correctly
• kustomize:latest should correspond to the same binary as v3.8.2

Anything else you would like to add:

MR for the :version tag is incoming, but no idea what's wrong with the latest tag.

[issue-label-bot]

Issue-Label Bot is automatically applying the label bug to this issue, with a confidence of 0.98. Please mark this comment with 👍 or 👎 to give our bot feedback!

Links: app homepage, dashboard and code for this bot.

[easimon]

The latest bug might be something else, and maybe limited to kustomize.
Some auto-updating mechanism (I assume this one: https://github.com/raspbernetes/multi-arch-images/blob/master/.github/workflows/schedule.yml)

reverts the .version file for kustomize regularly, like here:

https://github.com/raspbernetes/multi-arch-images/commits/master/build/kustomize/.version

Most probably because of the changed tagging semantics in kustomize, the version detection goes wrong here.

[xunholy]

Yeah, we might need to update the schedule to only pull the latest tag with the correct syntax to avoid the other releases.

[easimon]

Unfortunately there seems to be no easy way to achieve this...

I hacked together the following, but that's a lot more (and ugly) code than the simple single curl...
And while testing this, I ran into the rate limit on the Github REST API as well.

#!/usr/bin/env bash

set -euo pipefail
# Get the first tag for a given github repo matching a given prefix.
# Assumption: Github tags are returned in reverse alphabetical order,
# so the first match will be semantically "latest"
# 
# @param REPO: repository name in $ORG/$REPO format
# @param PREFIX: a prefix to filter for. Items not matching the prefix will be ignored.
# @return tag name or empty string if none matched
function get_latest_tag() {
  REPO="$1"
  PREFIX="${2-}"
  PER_PAGE=100
  HEADER_FILE="$(mktemp)"
  trap "rm -f ${HEADER_FILE}" RETURN

  NEXT_URL="https://api.github.com/repos/${REPO}/tags?per_page=${PER_PAGE}"

  while [[ "${TAG-}" == "" && "${NEXT_URL}" != "" ]]; do 

    TAG="$(
      curl -sSLD "${HEADER_FILE}" "${NEXT_URL}" \
      | jq -r "[.[].name | select (test(\"${PREFIX}.*\"))] | first // empty" \
    )"

    NEXT_URL="$(
      grep -e '^link:.*' "${HEADER_FILE}" \
      | tr ',' '\n' \
      | grep 'rel="next"' \
      | cut -f2 -d'<' \
      | cut -f1 -d'>'
    )"

  done

  echo "${TAG}"
}

get_latest_tag kubernetes-sigs/kustomize kustomize/

[xunholy]

I'm also happy for us to disable the schedule for this particular instance and potentially bump the version ourselves until we have a better solution to avoid it breaking, or creating a schedule in the same workflow and keep it out of the existing schedule.

[easimon]

Disabling it for kustomize would be fine for me as well -- better to have no automation than one that consistently breaks things :)

[xunholy]

Feel free to raise a PR to disable, we can comment it out and leave a description as to why it's being no longer managed via the workflow 👍

Thanks for investigating this issue further @easimon

[easimon]

Might have found a simple way that does not require to disable the auto-upgrade. Not bullet-proof, but the current approach is not either, so this might still qualify.