Separate data and metadata in records.

[almet]

I'm opening this for discussion in Kinto, but it's entierly possible that it should live in Cliquet instead.

Currently, records have some metadata attributes directly returned alongside the business attributes. These are "id" and "last_modified".

{
  'data': {'id': '1234', 'last_modified': 123456, 'foo': 'bar'}
}

It could make sense to separate these metadata attributes from the data. e.g.

{
  'id': '1234',
  'last_modified': 123456,
  'data': {'foo': 'bar'}
}

One of the challenges with this approach is that record lists should also contain the associated metadata. I'll let @leplatrem expend on this as I don't see clearly where the problem lies.

Other than that, this change would break the API in an incompatible way, and thus the clients would need to be updated accordingly. Validation features (Schema) and permissions would also be impacted by this approach.

Related to #158

[leplatrem]

Very related to #174

[Natim]

Actually what I understand of your proposal, taking care of @leplatrem concerns, is to rather do:

{
  "data": {
      "id": "1234",
      "last_modified": 123456,
      "data": {"foo": "bar"}
    }
}

So adding a new level to put all records data in it.

Then the collection_get will look like :

{
    "data": [{
      "id": "1234",
      "last_modified": 123456,
      "data": {"foo": "bar"}
    }]
}

The question is, will permission be part of this data or not?

{
  "data": {
      "id": "1234",
      "last_modified": 123456,
      "data": {"foo": "bar"},
      "permissions": {}
    }
}

or

{
  "data": {
      "id": "1234",
      "last_modified": 123456,
      "data": {"foo": "bar"}
    },
    "permissions": {}
}

[Natim]

If we go furthermore in the proposal and look at collections, will it look like that:

{
    "data": {
        "id": "tasks",
        "last_modified": 1446646942107,
        "data": {"fingerprint": "1a4089078fda74bc20441f62e621b057a92cffb6ace35ccde0c32b6394949801"},
        "schema": {"JSONSchema": "in it"}
    }
}

or like that?

{
    "data": {
        "id": "tasks",
        "last_modified": 1446646942107,
        "data": {
            "fingerprint": "1a4089078fda74bc20441f62e621b057a92cffb6ace35ccde0c32b6394949801",
            "schema": {"JSONSchema": "in it"}
        }
    }
}

Depending of if we consider the schema being part of the collection attributes (that we want to take into account when signing the collection object).

[Natim]

If we want to get closer to what jsonapi is doing, it should probably look like this:

{
  "data": {
      "id": "1234",
      "last_modified": 123456,
      "attributes": {"foo": "bar"}
    }
}

[leplatrem]

The question is, will permission be part of this data or not?

From what I understand I guess it should. Like schema and other metadata.

One of the challenges with this approach is that record lists should also contain the associated metadata.

In terms of implementation, this would probably mean to get rid of the multiple backends and put everything into one. Otherwise, joining data and permissions during the list construction might not be efficient at all. Especially if we want stuff like #123

I'll let @leplatrem expend on this as I don't see clearly where the problem lies.

As I said privately, this could be quite a change for the whole ecosystem, and I see no emergency in doing this. I personally would rather keep metadata mixed up with data and put the efforts into other challenges that bring more value to the end users (docs, install, demos, integrity, ...)

[almet]

@Natim I don't understand what we get by having two different levels of "data" (or data/attributes). Having data alongside the other metadata (last modified etc) seems cleaner to me.

Permissions, schema etc should go alongside the rest. Eg.

{
  'id': '1234',
  'last_modified': 123456,
  'data': {'foo': 'bar'},
  'permissions': {},
  'schema': 'blah'
}

[smuriu]

I'm very much in favour of the proposal by @almet. I only just recently came across kinto but already thinking of things I could do with it. One question I have, related to the kinto-attachment plugin, would this proposal if implemented add the attachment attribute as metadata alongside id, permissions, ...? Seems a lot cleaner that way IMO.

On a side note, has anyone proposed a related records plugin? In my mind, it would add the key relations which is an array of a relation objects:

'relations': [
  {
    'relation': {
      'collection': 'collection_x',
      'record': 'record_x'
    }
  },
  {
    'relation': {
      'collection': 'collection_x',
      'record': 'record_y'
    }
  }
]

Again, this would feel nice if added as metadata alongside id, ...

[leplatrem]

Let's be realistic. This won't happen