Local Kubernetes registry for Knative

A local registry lets you use the same image URLs in all clusters, in-container (Kaniko builds, Knative tag-to-digest resolver etc.) as well as on nodes (dockerd image pull). Local registry with temporary storage eliminates a security concern and bandwidth cost of playing around with Knative. Local registry with S3/GCS persistence lets multiple clusters share images. If desired, any one of them can be your production registry, exposed with your choice of security to outside the cluster.

The question is: How do we chose the hostname for our image URLs? Running registry in Kubernetes is trivial, but:

We can't use a real SSL certificate issuer.
Docker on k8s nodes typically fails to resolve cluster DNS names.

Knative doesn't (yet?) abstract out the registry host from Source-to-URL workflows, so we need the same user-facing FQDNs to work both with Knative and nodes' Docker.

In this example setup we've chosen knative.registry.svc.cluster.local as the registry host. Using .local we by docker conventions avoid SSL, and we communicate the scope of the configuration.

Set up name resolution

You must chose how to make a registry DNS name resolvable to nodes. There are basically three options:

Docker daemons somehow resolve Kubernetes services *.svc.cluster.local.
You add the chosen registry FQDN to /etc/hosts on all nodes, including those provisioned in future.
A local DNS points knative.registry.svc.cluster.local to the clusterIP of the service.

Set up the registry

Apply the templates folder.

There's also a script ./minikube-start.sh that sets up a Knative-compatible Minikube with local registry.

If docker pull fails, which is likely (you can use our test jobs to check), you need to fix name resolution. There's a suggestion in the sysadmin folder.

Persistence

Note that this setup uses transient registry storage. See Registry docs for options on how to make it persistent, and modify config and/or volume(s) accordingly.

It's worth pointing out that S3/GCS persistence is particularily useful for local registry, as any other cluster or registry setup that can access the same bucket is automatically a mirror. Furthermore it makes horizontal scaling as easy as kubectl scale on the registry replicaset.

Authentication

Beware that this setup grants anyone access. See the Registry docs for how to enable authentication.

To keep using Knative locally without a service account but provide outside access with authentication, start a separate Registry deployment backed by the same bucket or expose a proxy. The nginx:alpine image can use htpasswd files created by registry.

Support

We would love your feedback on this project so don't hesitate to let us know what is wrong and how we could improve it, just file an issue

Code of Conduct

This work is by no means part of CNCF but we abide by its code of conduct