oras-project/artifacts-spec

Referrers API pagination mechanism

Closed this issue · 10 comments

michaelb990 commented

Was the pagination mechanism described in the current spec intentionally different than the one that already exists for tags?

I'd prefer to not introduce a new way of paginating, and I wasn't aware of tag pagination until now. Thoughts?

If people are in agreement, I'll make a PR.

jonjohnsonjr commented

To pull my opinion out of the linked PR, it would be nice if whatever pagination mechanism is used would set a Link header appropriately such that existing clients can use the same mechanism as tags and catalog: https://docs.docker.com/registry/spec/api/#pagination

sajayantony commented

Moving the pagination discussion here from -
#3 (comment)

Would like link header can push the nextToken downstream to providers for continuation.
For ordering - Lexical order like the tags listing doesn't make sense to me and it's worth calling out here what the order woudl be. I would however lean towards reverse chronological since typically clients want the newest of the reference artifact first.

michaelb990 commented

I would however lean towards reverse chronological since typically clients want the newest of the reference artifact first.

I don't see a reason to define the order of the list items in the spec. I'd prefer to leave it up to registry operators unless we have a strong use case. Typically, I'd imagine that an image won't have enough references for this to make a difference, especially if clients are able to filter by the type of reference artifact they want to see.

I'd especially like to avoid making the ordering dependent on a piece of metadata that doesn't come from the manifest itself, as that would have implications in the additional metadata that a registry had to include in the reverse index used to power this API.

michaelb990 commented

To pull my opinion out of the linked PR, it would be nice if whatever pagination mechanism is used would set a Link header appropriately such that existing clients can use the same mechanism as tags and catalog:

@jonjohnsonjr Do you mean that registries should return a nextToken or nextLink in the response body AND a fully constructed Link header for clients to use? Or only one or the other?

jonjohnsonjr commented

Do you mean that registries should return a nextToken or nextLink in the response body AND a fully constructed Link header for clients to use? Or only one or the other?

It doesn't bother me either way as long as I can use the Link header. I would personally avoid having both, as it feels redundant, but I can just ignore that field and be happy.

sajayantony commented

I don't see a reason to define the order of the list items in the spec. I'd prefer to leave it up to registry operators unless we have a strong use case. Typically, I'd imagine that an image won't have enough references for this to make a difference, especially if clients are able to filter by the type of reference artifact they want to see.

If there is no ordering - then it would recommend we call call it out explicitly out since the distribution tag listing api does require lexical ordering.

michaelb990 commented

I'd propose leaving the token opaque to allow us to abstract the implementation of pagination away from clients. I believe this is compatible with the Link approach. For when we (ECR) implements this, we would probably prefer not including a cleartext representation of the last or next object to use like the tag/catalog pagination is doing.

Linking some pagination guidelines from other registry providers that provide similar guidance:

Next steps for this would be a PR to change to providing the next link in a Link header in the response unless anyone has any other concerns.

mnltejaswini commented

@SteveLasker I will submit a PR for the pagination

  • 👍1
  • 🚀1
mnltejaswini commented

Do you mean that registries should return a nextToken or nextLink in the response body AND a fully constructed Link header for clients to use? Or only one or the other?

It doesn't bother me either way as long as I can use the Link header. I would personally avoid having both, as it feels redundant, but I can just ignore that field and be happy.

I was looking at the Docker Registry spec for catalog listing and it calls out that the next URL is part of both Link header and the response @jonjohnsonjr @michaelb990

https://docs.docker.com/registry/spec/api/#catalog

Content-Length: <length>
Link: <<url>?n=<last n value>&last=<last entry from response>>; rel="next"
Content-Type: application/json; charset=utf-8

{
	"repositories": [
		<name>,
		...
	]
	"next": "<url>?last=<name>&n=<last value of n>"
}
michaelb990 commented

Fixed by #16.