Address Collection Resource Misnomer

[fosrias]

There is no such thing as a collection resource. Rails bad conventions. Thus, should not have a separate media-type and resolve this as a resource and look into using IANA item or other options.

[lhazlewood]

There is if there is a media type for it ;)

Tongue-in-cheek aside, the impetus for this isn't related to Rails at all (I didn't know Rails has such a thing). It could be a separate resource because the properties (not metadata) would be codified as required:

• items
• size
• offset
• limit
• first/next/previous/last links

Also, pagination logic (how IRIs represent different pages, etc) would be codified in the media type.

That said, if not a separate media type, how does one distinguish what I'm currently calling a Collection Resource as different from an Instance Resource (with all the pagination logic codified as necessary)?

[fosrias]

That said, if not a separate media type, how does one distinguish what I'm currently calling a Collection Resource as different from an Instance Resource (with all the pagination logic codified as necessary)?

So, IMO, having spent a lot of time on both sides of the fence, I have finally convinced myself that REST is about resources and there is no such thing as an instance or collection resource. Those ideas have ties to type marshalling and OOP, but not to REST.

In REST there is a resource specified by a URI. Thats it. It has data and hypermedia and you have to introspect that message at runtime to decide what to do with it. Spent a long time de-programming myself from this type of thinking, personally.

That being said, with first/next/previous/last as IANA defined relation types, their presence indicates a resource that is part of a series and you can rely on using them and what they mean with respect to a resource. So, if I have a resource with those relations, it indicates that resource is part of a series.

I look at size, offset and limit as metadata about the resource. If a client understands them it can use them. I don't think that codifying it into a media-type is necessary, nor universal. What is universal is what first/next/previous/last mean and you should just really on resolving them for navigation. There are some ideas I have seen around about modifying those values in a hypermedia friendly fashion, but I will have to dig around a bit to refresh my memory on that. At the very least, with support for defining query parameters and templated URIs, this could be resolved that way as one option, which then completely removes the idea of pagination logic in the media-type.

Further, there are IANA relation types item and collection that can be used as well, I think to solve this.

So, long and short, I think we should not think about instance or collection resources, but just resources and rely on other specs and IANA registries to define what is going on in resources and whether they are part of a series or not.

Hopefully that all translates ;).

[smizell]

If forms are introduced and IANA link relations supported, you can handle most of this in the same media type. Going off of the example by @fosrias in another issue:

{
  "meta": {
    "href": "..."
  },
  "search": {
    "meta": {
      "href": "https://api.example.com/users",
      "queryParams": {
        "size": {
          "some": "input description", 
          "value": 218
        },
        "offset": {
          "some": "input description",
          "value": 0
        },
        "limit": {
          "some": "input description",
          "value": 25
        }
      }
    }
  },
  "first": { "meta": { "href": "https://api.example.com/users?offset=0" } },
  "next": { "meta": { "href": "https://api.example.com/users?offset=25" } },
  "last": { "meta": { "href": "https://api.example.com/users?offset=200" } },
  "item": [
    {
      "meta": {
        "href": "https://api.example.com/users/1"
      },
      "firstName": "Bob",
      "lastName": "Smith",
      "birthDate": "1977-04-18"
    },
    {
      "meta": {
        "href": "https://api.example.com/users/2"
      },
      "firstName": "John",
      "lastName": "Smith",
      "birthDate": "1977-04-18"
    }
  ]
}

This only shows two items, but uses proper link relations (such as search and item) along with providing query parameters to build the query.

[lhazlewood]

Per RFC 6573, neither item nor collection link relations satisfy the current structure.

item is intended to represent a single member of a collection. The items field in contrast is an array that contains multiple members in the collection. The collection relation is intended to point to the referencing collection (i.e. the collection itself, https://api.example.com/users).

RFC 6573 largely reflects the capabilities of an XML grammar, which is why it doesn't really directly apply for JSON. I presume this is why Mike Amundsen proposed a separate generic media type just for JSON collections: vnd.collection+json

As you can see, this is its own media type just for the purpose of a collection. He even labels the items array as the string literal items. This effectively reflects the same goal I had when I thought of application/ionc+json, because item and collection - as defined - aren't sufficient for JSON use cases. Link relations are good for referencing one resource from another. The items name/value pair is not a resource and can't be referenced as a resource (hence the need for a new media type).

I have three nit-picks with vnd.collection+json:

1. It is part of the vnd namespace, which doesn't reflect Ion.
2. Ion collections, to remain congruent with 'instance' resources, should support meta semantics, which is why application/ion+json makes more sense to me. This means Ion couldn't use vnd.collection+json because the resource format is just different (slightly so, but still different such that a different media type is warranted).
3. vnd.collection+json, while IANA approved, I've never seen used in practice. To me, this means it doesn't have enough 'de facto' weight behind it such that we are compelled to use it.

Having an Ion-specific collection concept, that allows us to mold it specifically for the use cases we have in mind, sounds safer.

[smizell]

@lhazlewood Maybe this should be a separate issue, but I believe the spec will have to tell how to handle multiple resources that have the same link relation (such as in the case of item). HAL handles them by allowing you to use an array, though you still use the singular form of the link relation. That link relation then applies to each item in the array. As an example, start with HTML.

<a href="/item/1" rel="item">Item 1</a>
<a href="/item/2" rel="item">Item 2</a>
<a href="/item/3" rel="item">Item 3</a>

Here, we have multiple item link relations to different resources. The same could be possible in Ion, just with using an array like HAL.

{
  "item": [
    { "meta": { "href": "/item/1" } },
    { "meta": { "href": "/item/2" } },
    { "meta": { "href": "/item/3" } }
  ]
}

The spec would then state that if there are meta properties in an array's item, you should consider the property name to be the link relation of each if it's an IANA link relation. Additionally, per issue #1, if an rel property was specified, you could do this:

{
  "items": [
    { "meta": { "href": "/item/1", "rel": ["item"] } },
    { "meta": { "href": "/item/2", "rel": ["item"] } },
    { "meta": { "href": "/item/3", "rel": ["item"] } }
  ]
}

This allows you to specify a rel per item, though the top-level property may simply be a name.

[fosrias]

Having an Ion-specific collection concept, that allows us to mold it specifically for the use cases we have in mind, sounds safer

Agreed. I am all for an ion flavored way to handle resources contain lists of other resources. However, I think there are ways to use item depending on how we define a resource as one or many.

That being said, as I have been thinking about this, I think introducing a reserved word items into Ion is worth considering. However, I think if we define a relation to tie to one or many can allow more flexible use of existing relation types.

I am not really a fan of collection JSON for a number of reasons myself. People do use collection+json, but as is true of a lot of hypermedia types on the frontier, exact adoption is murky. HAL gets most exposure, but is frankly the weakest IMO compared to others other than its ability to inject into existing JSON.

[fosrias]

@smizell You posted just before me.

[lhazlewood]

I think we're in agreement here :)

In my mind, an application/ionc+json spec should dictate that an element in the items array defaults to the item link relation unless specifically indicated otherwise by a rel attribute in the element's meta object. For example, these two Ion collection resources would be considered semantically equivalent:

{
  "meta": { "href": "..." },
  "first": { "meta": { "href": "..." } },
  "next": { "meta": { "href": "..." } },
  "prev": { "meta": { "href": "..." } },
  "last": { "meta": { "href": "..." } },
  "items": [
    { "meta": { "href": "/users/1" } },
    { "meta": { "href": "/users/2" } },
    { "meta": { "href": "/users/3" } }
  ]
}

{
  "meta": { "href": "..." },
  "first": { "meta": { "href": "..." } },
  "next": { "meta": { "href": "..." } },
  "prev": { "meta": { "href": "..." } },
  "last": { "meta": { "href": "..." } },
  "items": [
    { "meta": { "href": "/users/1", "rel": ["item"] } },
    { "meta": { "href": "/users/2", "rel": ["item"] } },
    { "meta": { "href": "/users/3", "rel": ["item"] } }
  ]
}

[smizell]

Ah, I see. I think what I was meaning to get at was that the main spec itself will probably have to specify how to have multiple resources with the same link relation in one message. If it does this, you can use the item rel for each of them without introducing a new property. For instance, maybe you're linking to multiple orders.

{
  "order": [
    { "meta": { "href": "/order/1" } },
    { "meta": { "href": "/order/2" } },
    { "meta": { "href": "/order/3" } }
  ]
}

Of course, for this to work, you'd have to specify what an order is in something like ALPS, and have in the media type spec that each array item gets the link relation.

So then if that's possible, using item would be possible.

Thanks for the clarification :)

[fosrias]

Even without ALPS, couldn't you do this if we define support for arrays under a relation name like:

{
  "order": [
    { "meta": { "href": "/order/1", "rel": ["item", "http://example.com/rels/order"] } },
    { "meta": { "href": "/order/2", "rel": ["item", "http://example.com/rels/order"] } },
    { "meta": { "href": "/order/3", "rel": ["item", "http://example.com/rels/order"] } }
  ]
}

And be conformant and explicit?

[smizell]

@fosrias Yes, if rel was supported in this way.

[lhazlewood]

With reference expansion, the following structure would be probably be more "Ion-y".

For example, assume a GET to https://api.example.com/users/1?expand=orders:

{
  "meta": { "href": "https://api.example.com/users/1" },
  "username": "jsmith",
  "orders":  {
    "meta": { "href": "https://api.example.com/users/1/orders", "mediaType": "application/ionc+json"},
    // ... omit first, next, prev, last for brevity
    "items": [
      { "meta": { "href": "/orders/1", "rel": ["http://api.example.com/rels/order"] } },
      { "meta": { "href": "/orders/2", "rel": ["http://api.example.com/rels/order"] } },
      { "meta": { "href": "/orders/3", "rel": ["http://api.example.com/rels/order"] } }
    ]
  }
}

The gist of this is that orders is a collection resource (not an array). For collection resources, the array containing the collection's (or page's) elements is always named items.

[fosrias]

@lhazlewood So, looking at this example, the mediaType is used to indicate the resource is ionc+json. Strikes me that instead of having a separate media-type, that we could introduce an attribute into an ion+json meta object that indicates the list/collection nature of the the resource. That flag per se, would tell a client to implement the rules associated with ionc+json (which I think would be pretty minimal, like defining items if we leverage iana link relations).

I like the idea of being able to recursively handle these ideas, that is we can expand any type of resource in another resource.

In any case, not trying to grind an axe, but my gut feeling is that we can accommodate lists in ion+json, or at least should try and let the failure trying make the case for another media-type. I think that is the main point of my starting this particular issue.

[lhazlewood]

@fosrias I don't feel like you're trying to grind an axe :) This is good/necessary discussion for something so critical to Ion.

I'm on the fence here, but I do see your logic. What are the pros/cons with either approach in your eyes?

If we don't introduce an application/ionc+json media type, is it really just as simple as defining a rule that says "If a JSON object has a meta property and an items array, it is a collection"?

That would be my hope :) But are there any 'gotchas' that might surface with that approach?

I think the rule to identify Ion links/relations/references are equally simple/elegant:

An unmaterialized link/relation/reference is simply a JSON object that has only a meta property, and that meta object value has, at a minimum, an href property. Simple, sweet, immediately recognizable when parsing...

[lhazlewood]

An addendum to my above comment for brainstorming: why did Mike not do something similar and instead create a new application/vnd.collection+json media type?

[fosrias]

Simple, sweet, immediately recognizable when parsing...

Yeah. This is going to be the secret sauce of ion. It can be very minimalistic and, I hope, it can also be very expressive semantically.

Good question about collection+json. Never asked Mike about its origins, however, I think it was an early hypermedia media-type experiment. Mike basically thinks in xml, so when he tries to solve things in JSON they sometimes feel forced (or as in the case of Uber+json, he concedes not so nice yet).

Given that, I don't think Mike has ever approached JSON hypermedia ideas by minimalistically injecting into common JSON as a value. I will have to ask him, but that would be my guess why he create the media-type. I think in his mind it is a simplified JSON version of ATOM.

[fosrias]

Per Pros and Cons of having two media-types:

Cons:

• A client that understands ion+json must also understand ionc+json in order to consume resources containing lists. Would be nice for a client to be able to accommodate one spec for full functionality with ion.
• Introducing different media-types into the meta object to do this, complicates the ion+json spec in having to specify rules for including another media-type (albeit, this might be an interesting idea to think about regardless of the collection functionality for other reasons to allow embedding other json based media-types, but it makes the ion spec more complicated).
• Pushing through multiple specs and getting adoption in the community/participation and ietf.
• IMO, for it to be useful, it would need to be an extension of ion so that you could add other relations and lists at which point the value of the separation is not very clear.
• Continues to propagate non-resourceful thinking IMO as I discussed before. Think we should exorcise thinking like instance and collection and just think resource and understanding what is in a message.

Pros (going to have to work for this one 😄):

• I can see some advantages to your original ideas around navigation focus (but I think this media-type would end up being too opinionated and constricted if it just targeted a list of one thing).
• Possibly reduces the complexity of the ion spec.

@lhazlewood Would be interested in your pro and con list as well. My pros are a little forced.

[smizell]

@lhazlewood A thought on this:

If we don't introduce an application/ionc+json media type, is it really just as simple as defining a rule that says "If a JSON object has a meta property and an items array, it is a collection"?

I'd like to walk through an example here. First, a collection is only an IANA collection if it is linked using the collection rel. For example, here is a representation of a user with a link to an orders collection.

{
  "meta": {
    "href": "/users/1"
  },
  "first_name": "John",
  "last_name": "Doe",
  "orders": { 
    "meta": { 
      "href": "/user/1/orders",
      "rel": ["collection", "http://example.com/rels/orders"]
    }
  }
}

Now let's say we invoke this orders transition and receive the following representation.

{
  "meta": {
    "href": "/users/1/orders"
  },
  "item": [
    {
      "meta": {
        "href": "/users/1/orders/1"
      }
    },
    {
      "meta": {
        "href": "/users/1/orders/2"
      }
    },
    {
      "meta": {
        "href": "/users/1/orders/3"
      }
    }
  ]
}

Here I have three resources that have the item rel. Going back to my first example, I could have embedded (or expanded) this.

{
  "meta": {
    "href": "/users/1"
  },
  "first_name": "John",
  "last_name": "Doe",
  "orders": { 
    "meta": { 
      "href": "/users/1/orders",
      "rel": ["collection", "http://example.com/rels/orders"]
    },
    "item": [
      {
        "meta": {
          "href": "/users/1/orders/1"
        }
      },
      {
        "meta": {
          "href": "/users/1/orders/2"
        }
      },
      {
        "meta": {
          "href": "/users/1/orders/3"
        }
      }
    ]
  }
}

With this, you can have a collection with items and only use defined link relations. To do this, you would need to define rules similar to these:

1. If the meta property is found in the root object, it refers to the received representation
2. If the meta property is found in an object that is a value for a property (like for orders in the user representation above), the link relation name is the property name (e.g. orders)
3. If the meta property is found in an object that is within an array (like for item above), the link relation is the name of the property for the array for each item.
4. If the value for any of the above scenarios is not an established link relation, it should be specified in the rel property (or in a profile doc like ALPS).

Just thinking out loud here :) Hope it makes sense.

[lhazlewood]

@fosrias I agree with your list. My biggest Pro that I was thinking of was 'reduce complexity', which is what I really care about.

In retrospect, I think I was too tightly correlating a media type to a particular feature in the specification. I was thinking along the lines of JWT's specifications: instead of one big one, it is broken up into separate RFCs, one that addresses a particular subset of functionality (JWS (signatures), JWE (encryption), etc).

I kind of liked how they (and now, HTTP) did this: it maybe makes them more digestible, where you only have to be concerned about a particular subset if you care about it, rather than being overwhelmed by one massive spec.

For example, the OAuth 2 RFC is potentially one such uber spec. I've seen so much confusion and frustration around this particular spec (by readers/implementors) that I really wanted to avoid the same thing with Ion.

That said, I think (maybe) the difficulty with OAuth 2 is really its nomenclature and its desire to be an uber spec (all things to all people) - most of its concepts, names, etc never really 'clicked' for most people. I want to avoid that like the plague.

[lhazlewood]

@smizell Nice example! I agree with everything you wrote except number 3:

If the meta property is found in an object that is within an array (like for item above), the link relation is the name of the property for the array for each item.

I wouldn't want to name of the array property to be in the singular (item) because, as a foundational concept, it would be unintuitive for anyone that doesn't immediately recognize IANA link relation names. I'd like Ion's representations to be intuitive for people who've never even seen the spec or don't know about IANA relations (for example, developers manually integrating with REST APIs).

In my opinion, an Ion resource in an items array would always assume the item relation unless explicitly specified otherwise by the element's meta rel property, or potentially, in the collection's meta. That is:

1. Is the array element an Ion resource, and if so, does it have a rel meta property? If so, that is the relation name for that element.
2. If the array element is an Ion resource and it does not have its own rel meta property, check the parent collection resource's meta. Does it define an itemRel property? If so, that is the element's relation name.
3. If numbers 1 and 2 don't apply, the link relation name for that element will be assumed to be item.

Thoughts?

[fosrias]

I want to avoid that like the plague.

Ditto. I think the question is what is requisite functionality and what is optional, e.g. JWE is optional in the JWT world. In this world, handling lists inside resources is not optional IMO. It is just part of a complete resource-friendly media-type, much like HTML includes

.

@lhazlewood @smizell So, at this point, shall we agree to accommodate lists in ion+json spec? If so, we can close this issue with a future PR that updates the spec to address lists.

[lhazlewood]

@fosrias I definitely agree to a single application/ion+json spec at this point.

[lhazlewood]

Closing the loop on this - I'll update the current ion-doc to reflect a single media type declaration. Any objections to closing this issue?

[fosrias]

Fine with me.

[lhazlewood]

Closed. New discussion around this can be opened as a new issue.