opencontainers/distribution-spec

Should reference manifest also contains `Subject` field?

zyyw opened this issue · 3 comments

zyyw commented

After running the "test03ContentDiscovery()" test case against Harbor registry, the "Teardown > References teardown" case failed.
The request sent out is:

DELETE  /v2/conformance/testrepo/manifests/sha256:be25eb22cf43e837439915194b9b29baee1927d89dba1a04ac318b3dda49bc40  HTTP/1.1

cat the content of sha256:be25:

ubuntu@dev-sc1:/data/registry/docker/registry/v2/blobs/sha256$ cat be/be25eb22cf43e837439915194b9b29baee1927d89dba1a04ac318b3dda49bc40/data
{
	"schemaVersion": 2,
	"mediaType": "application/vnd.oci.image.index.v1+json",
	"artifactType": "application/vnd.food.stand",
	"manifests": [
		{
			"mediaType": "application/vnd.oci.image.manifest.v1+json",
			"digest": "sha256:a77afb05eb321cd20528e7be46927a3f4da704e9a29629faf07aa7a7ebf47f6e",
			"size": 711,
			"newUnspecifiedField": null
		},
		{
			"mediaType": "application/vnd.oci.image.manifest.v1+json",
			"digest": "sha256:f2a0351729ea7f6aa9cc88d0d4d9f534b5a2ec24be6ff489492a6f750ed905e0",
			"size": 771,
			"newUnspecifiedField": null
		}
	],
	"subject": {
		"mediaType": "application/vnd.oci.image.manifest.v1+json",
		"digest": "sha256:ab434234add0d7537c0c2a92955ceb483290231c4110eca15f2a97503aeb6cf9",
		"size": 714,
		"newUnspecifiedField": null
	}
}

cat the content of sha256:a77a:

ubuntu@dev-sc1:/data/registry/docker/registry/v2/blobs/sha256$ cat a7/a77afb05eb321cd20528e7be46927a3f4da704e9a29629faf07aa7a7ebf47f6e/data
{
	"schemaVersion": 2,
	"mediaType": "application/vnd.oci.image.manifest.v1+json",
	"config": {
		"mediaType": "application/vnd.nhl.peanut.butter.bagel",
		"digest": "sha256:ee29d2e91da0e5dbf6536f5b369148a83ef59b0ce96e49da65dd6c25eb1fa44f",
		"size": 33,
		"newUnspecifiedField": null
	},
	"layers": [
		{
			"mediaType": "application/vnd.oci.empty.v1+json",
			"digest": "sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a",
			"size": 2,
			"newUnspecifiedField": null
		}
	],
	"subject": {
		"mediaType": "application/vnd.oci.image.manifest.v1+json",
		"digest": "sha256:ab434234add0d7537c0c2a92955ceb483290231c4110eca15f2a97503aeb6cf9",
		"size": 714,
		"newUnspecifiedField": null
	}
}

cat the content of sha256:f2a0:

ubuntu@dev-sc1:/data/registry/docker/registry/v2/blobs/sha256$ cat f2/f2a0351729ea7f6aa9cc88d0d4d9f534b5a2ec24be6ff489492a6f750ed905e0/data
{
	"schemaVersion": 2,
	"mediaType": "application/vnd.oci.image.manifest.v1+json",
	"artifactType": "application/vnd.nhl.peanut.butter.bagel",
	"config": {
		"mediaType": "application/vnd.oci.empty.v1+json",
		"digest": "sha256:44136fa355b3678a1146ad16f7e8649e94fb4fc21fe77e8310c060f61caaff8a",
		"size": 2,
		"newUnspecifiedField": null
	},
	"layers": [
		{
			"mediaType": "application/vnd.nhl.peanut.butter.bagel",
			"digest": "sha256:ee29d2e91da0e5dbf6536f5b369148a83ef59b0ce96e49da65dd6c25eb1fa44f",
			"size": 33,
			"newUnspecifiedField": null
		}
	],
	"subject": {
		"mediaType": "application/vnd.oci.image.manifest.v1+json",
		"digest": "sha256:ab434234add0d7537c0c2a92955ceb483290231c4110eca15f2a97503aeb6cf9",
		"size": 714,
		"newUnspecifiedField": null
	}
}

We noticed that the index manifest sha256:be25, and reference manifest sha256:a77a & sha256:f2a0 all contain a Subject field. My questions are:

  1. Should the reference manifest sha256:a77a & sha256:f2a0 also contain a Subject field? Or there should not be a Subject field within the reference manifest sha256:a77a & sha256:f2a0.
  2. We noticed that the index manifest sha256:be25, and reference manifest sha256:a77a & sha256:f2a0 all contain the same Subject identified by hash digest sha256:ab43. This seems violates the DAG (directed acyclic graph) rule.

Thanks for your help!

sudo-bmitch commented

We noticed that the index manifest sha256:be25, and reference manifest sha256:a77a & sha256:f2a0 all contain a Subject field. My questions are:

1. Should the reference manifest `sha256:a77a` & `sha256:f2a0` also contain a `Subject` field? Or there should not be a `Subject` field within the reference manifest `sha256:a77a` & `sha256:f2a0`.

It's not a great example, but we don't have any good examples for adding a subject to the Index manifest to work from, so this seems good enough to verify registries support a subject on both the Index manifest and Image manifests.

2. We noticed that the index manifest `sha256:be25`, and reference manifest `sha256:a77a` & `sha256:f2a0` all contain the same `Subject` identified by hash digest `sha256:ab43`. This seems violates the DAG (directed acyclic graph) rule.

This example doesn't violate the DAG since it's still directed and acyclic. sha256:ab43 is referenced by multiple manifests, but it never has references back to those referring images.

However, if you treat referrers to an image as child objects of that image, then the addition of the subject to the index introduces the possibility of a logical loop. E.g. you can have in Index with:

manifests: [
  { digest: "sha256:12345..." }
],
subject: {
  digest: "sha256:12345..."
}

And traversing from the index to 12345 as the child manifest, you would query the referrers to 12345 and return back to the index. For an example of that, see ghcr.io/sudo-bmitch/oci-sandbox:loop. That's a simplified example, an index can have a nested index, or there may be multiple layers of referrers between the two references.

wy65701436 commented

Actually, we are currently seeking clarification on the meaning of the "subject" field in the image index. From my understanding, the image index serves as a collection of multiple image manifests rather than an image manifest itself, so it may not have a specific meaning in this context. Consequently, it raises the question of whether the subject of an image index should imply that all the references within the image index point to the same subject manifest. If so, it would not be necessary to define the subject for each reference within the image index.

From the perspective of registry rendering, the structure of the image index(with subject)is not strictly hierarchical like a tree; it resembles more of a diagram. When making a call to the Referrers API to list the referrers, both the image index and its references will be displayed, meanwhile all the references are pointing the same image index.

sudo-bmitch commented

From my understanding, the image index serves as a collection of multiple image manifests rather than an image manifest itself, so it may not have a specific meaning in this context.

The index is a manifest of it's own, pushed to the manifest API with its own digest and media type. It happens to contain a list of manifest descriptors instead of a list of blob descriptors.

Consequently, it raises the question of whether the subject of an image index should imply that all the references within the image index point to the same subject manifest. If so, it would not be necessary to define the subject for each reference within the image index.

An image can be listed in multiple index manifests, and single platform image manifests may be copied to a different repository without copying the index. I worry this would create more problems for both clients and servers.

From the perspective of registry rendering, the structure of the image index(with subject)is not strictly hierarchical like a tree; it resembles more of a diagram. When making a call to the Referrers API to list the referrers, both the image index and its references will be displayed, meanwhile all the references are pointing the same image index.

Different people have suggested different interpretations of the DAG structure. Some suggest that the index with a subject field would be the top level of the DAG, and the subject image is a child. I tend to agree with you on the path users take to read it, following the Referrers API, which means the index gets added as a child of the subject manifest. The complication with the index having a subject is that the index's own children could be parents to the subject manifest, resulting in a loop in that concept of the DAG. So depending on how you build the DAG structure in your own tooling, that may require a change to the existing logic.