support Open container initiative compatible package registries as Preston remote (or content registries)

[jhpoelen]

from @cboettig cboettig/contentid#94

@jeroen pointed out to me that (docker or OCI) container registries like GitHub Package Registry are really just a bunch of sha256 addressed blobs. Moreover, that existing open-source tools like oras already make it pretty easy to push arbitrary content there.

Notably, the docker/OCI registry is already a widely adopted standard, easy to self-host and readily found on any major cloud provider. GitHub package registry is 'free for public packages' -- Jeroen notes that large projects like brew use it as their back-end storage and distribution medium.

For instance, here's a command to grab my favorite example from the GitHub content registry, you can merely request it by it's sha256 hash:

curl -fL --header "Authorization: Bearer QQ=="  "https://ghcr.io/v2/cboettig/content-store/blobs/sha256:9412325831dab22aeebdd674b6eb53ba6b7bdd04bb99a4dbb21ddff646287e37" -o test.txt

and verify this is indeed the Vostok ice core file. To push to the content store I've used the oras client tool:

oras login ghcr.io
oras push ghcr.io/cboettig/content-store ./vostok.co2

Container registeries actually have a nice manifest system for associating different files with a single manifest, and adding metadata, including file names, tags (i.e. version tags), and (I think) MIME types, as well as generic extensible metadata in labels, which all can facilitate much more discovery than the approach above where I just use opaque blobs. These features could be particularly interesting but maybe beyond the scope/generality of contentid?

Anyway, does seem like an intriguing way to deploy (public) data to a system that acts both as a content store and registry addressable by SHA256 hash, using a mechanism that is easily deployed locally (self-hosted container registry) and already in use by so many major providers. @jhpoelen @mbjones thoughts?

[jhpoelen]

support for OCI repositories as Preston remotes included in v0.7.2 .