Docker multi-arch example

This repository demonstrates how to create a multi-arch Docker image supporting multiple platforms, e.g. both x86_64 and ARM, but building all images on an x86_64 platform. It is primarily for personal use for reference on how to build manifests and how they work.

This example also only relies on Docker Hub to build all images, including the ARM variants, and does not rely on separate build servers or environments to build non-amd64 images.

It's a TL;DR version of the references linked in the README.

Steps

To create a multi-arch image, follow the next 4 steps

Build and push the image for the amd64 platform
Build and push the image for the arm* platforms
Create the manifest for the multi-arch image
Push the maniest for the multi-arch image

Automated builds via Docker Hub

The Docker Hub Custom build phase hooks allow in combination with QEMU an entirely automated build of the Docker images via Docker Hub for all major platforms - as it is used in this repository.

To see how it's done in this repository, see

Prepare QEMU in a custom build phase hook, hooks/pre_build
Dockerfile to build an arm32v7 image on x86_64 (Docker Hub), arm32v7.dockerfile
Dockerfile to build an arm64v8 image on x86_64 (Docker Hub), arm64v8.dockerfile

Use the following build rules in Docker Hub for each multi-arch image:

One build rule per Dockerfile/platform, e.g. amd64.dockerfile
The Docker Tag indicates the platform, e.g. amd64
No build rule for the (multi-arch) image tag in the manifest, e.g. latest

Note that the builds may fail at the beginning until each build rule ran once. This can be because the manifest references images that don't exist yet. Hence, after running all of them, each referenced image exists and the manifest can be pushed succesfully.

Push multi-arch manifest automatically (Docker Hub)

Once Docker Hub has published the amd64 and arm* images, Docker Hub executes the hooks/post_push script.

The script downloads the manifest-tool and pushes the multi-arch manifest multi-arch-manifest.yaml, which - simply put - makes ckulka/multi-arch-example:latest a list containing references to the various platform variants.

To see how it's done in this repository, see

Multi-arch manifest file, multi-arch-manifest.yaml
Push the multi-arch manifest file in a custom build phase hook, hooks/post_push

Push the multi-arch manifest manually (manifest-tool)

If you want to push the multi-arch manifest yourself using the manifest-tool, here are the steps:

Create the mulit-arch manifest file, e.g. multi-arch-manifest.yaml
Download the manifest-tool
Push the manifest using manifest-tool

# Download the manifest-tool
curl -Lo manifest-tool https://github.com/estesp/manifest-tool/releases/download/v0.9.0/manifest-tool-linux-amd64
chmod +x manifest-tool

# Push the multi-arch manifest
./manifest-tool push from-spec multi-arch-manifest.yaml

# On Docker for Mac, see https://github.com/estesp/manifest-tool#sample-usage
./manifest-tool --username ada --password lovelace push from-spec multi-arch-manifest.yaml

For more details, see manifest-tool: Create/Push.

Push the multi-arch manifest manually (Docker CLI)

If you want to push the multi-arch manifest yourself using the Docker CLI, here's how.

As long as the docker manifest commands are experimental, enable the experimental mode for the Docker CLI in ~/.docker/config.json first:

{
  "experimental": "enabled"
}

Next up, create and publish multi-arch the manifest.

# Create and push the manifest
docker manifest create ckulka/multi-arch-example:latest ckulka/multi-arch-example:amd64 ckulka/multi-arch-example:arm32v7
docker manifest push --purge ckulka/multi-arch-example:latest

The created manifest acts as a reference for the linked images. The Docker client, when pulling ckulka/multi-arch-example:latest, looks up a "fitting" image and then uses that one.

The --amend parameters allows adding additional platforms:

docker manifest create --amend ckulka/multi-arch-example:latest ckulka/multi-arch-example:arm64v7

The --purge parameter deletes the local manifest, which allows recreating and subsequently replacing the list:

# Release version 1.0 as latest image variant
docker manifest create ckulka/multi-arch-example:latest ckulka/multi-arch-example:1.0-amd64 ckulka/multi-arch-example:1.0-arm32v7
docker manifest push --purge ckulka/multi-arch-example:latest

# Release version 2.0 as latest image variant
docker manifest create ckulka/multi-arch-example:latest ckulka/multi-arch-example:2.0-amd64 ckulka/multi-arch-example:2.0-arm32v7
docker manifest push --purge ckulka/multi-arch-example:latest

Inspecting the result

The docker manifest inspect command shows the image manifest details - for the multi-arch image, it's the list of images it references and their respective platforms:

docker manifest inspect ckulka/multi-arch-example:latest

{
  "schemaVersion": 2,
  "mediaType": "application/vnd.docker.distribution.manifest.list.v2+json",
  "manifests": [
    {
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "size": 1728,
      "digest": "sha256:3a859e0c2bdfb60f52b2c805e2cb55260998b3c343d9e2ea04a742d946be1b1e",
      "platform": {
        "architecture": "amd64",
        "os": "linux"
      }
    },
    {
      "mediaType": "application/vnd.docker.distribution.manifest.v2+json",
      "size": 1728,
      "digest": "sha256:3ef9264d6e5ad96ad7bac675d40edf265ae838ae6ca60865abed159c8c5124c8",
      "platform": {
        "architecture": "arm",
        "os": "linux"
      }
    }
  ]
}

Known limitations

Building non-x86_64 images on Docker Hub results in images with incorrectly labelled architectures.

For example, the arm32v7 image runs on arm32v7, but will labelled as amd64, because Docker Hub builds the images on x86_64. There is a workaround for it, but I've not added it here to keep things simple.

This issue is currently tracked in moby/moby#36552.

nimmis/docker-multi-arch-example