Manifest for docker:latest not found: manifest unknown

[pykeras]

I encountered an issue when attempting to pull the docker image using the following command:

docker pull docker

This command outputs:

Using default tag: latest
latest: Pulling from library/docker
manifest for docker:latest not found: manifest unknown: manifest unknown

It appears that the docker is not available (other tags raise same error).

[max-sternitzke]

I'm glad, that i'm not the only one. See #520

[benz0li]

Pulling the linux/amd64 image by digest (i.e. docker pull docker@sha256:30564d91cacde37f23816c72f6f86a2d02cbe19d681af326204dce73aee3117) results in the same error.

[muocod]

The same here. Waiting for updates

[nfacha]

Same here, also affecting the dind tag

[ivanjx]

many of my CI builds are failing because of this. i need a proper fix as soon as possible.

[thaJeztah]

👋 I work at docker, but not on the official images; I'm currently trying our internal slack chatting with folks on call to see if someone working on official images is available to look.

It looks like the "manifest-index" for the image exists, but the image that's referenced from it is either missing, or resulting in the "not found".

As a workaround, it looks like the previous versions of the image are functional, so if you pin to the previous version, you can use that as a workaround, e.g.;

docker pull docker:27.3.1-dind / docker pull docker:27.3.1-cli both work

[thaJeztah]

I can give an intermediate status update from the Slack thread.

Team is still looking. They managed to narrow down the cause of the issue to be related to cross-repo links. The image was pushed successfully (data / blobs are there), but some links were not created after the push, causing it to be producing a 404 for those references. A fix was deployed that should prevent this situation from happening, but won't automatically apply to content that's already there, and those parts are really locked down, so no direct access to (manually) correct, so they're currently checking logs to identify images that are affected (beyond the ones mentioned in this ticket), and to see if there's ways to correct it. Re-pushing the images (same image, but trigger a push again) may work, but may also be locked down (currently being looked into).

[roysbike]

+1

[thaJeztah]

OK, team was able to get the list of images affected, and may have found a solution; some manual work is involved, so fixes may roll out over time. I'll post updates once available.

[rickkln]

Thanks for the great updates! Our builds seem to be working again so some of the fixes have succeeded.

[thaJeztah]

I think they managed to go through most of the list by now, but some may still be pending. Currently verifying all tags and all variants of those images (which takes a while to run).

There may be some images in the platform-specific orgs that are used as intermediate image that may not yet be addressed, and (given much lower / corner-case use), e.g. images in the platrform-specific namespace (e.g. https://hub.docker.com/u/arm64v8/, and other similar namespaces listed in; https://github.com/docker-library/official-images#architectures-other-than-amd64. Those namespaces are effectively intermediate stages for the multi-platform images.

[thaJeztah]

OK; colleague confirmed that the list of official images should be all processed. The verification script will still be running for some hours, but we're updating the statuspage to "monitoring"

There may be some images in the platform-specific orgs that are used as intermediate image that may not yet be addressed, and (given much lower / corner-case use), e.g. images in the platrform-specific namespace (e.g. https://hub.docker.com/u/arm64v8/, and other similar namespaces listed in; https://github.com/docker-library/official-images#architectures-other-than-amd64. Those namespaces are effectively intermediate stages for the multi-platform images.

These images will likely be processed later, which may be done after the weekend.