Allow adding a small amount of (optional) metadata for modfiles

[leper]

We internally use some JSON data to handle mods and their compatibility. Parts of it are also provided in the mod/modfile data returned currently, however a few custom fields (best encapsulated in some "custom" object) that can be set when uploading a mod (or at a later point using some tool) would help a lot.

Mainly we wouldn't allow the user to download mods that are not compatible with the current version of the game. We could also easily show that there is a compatible update of the mod available, or if that requires a newer version of the game indicate that.

I guess limiting the size of the custom data would be important, but given the current mods the biggest of these files is 314 bytes. And most of that is the description, which wouldn't be included in the data I'd like to see stored with the mod files. So I guess limiting the custom data to something like 512 bytes should be plenty.

[intenscia]

There are 3 ways to include metadata with your modfile so we have you covered.

1. First is via tags. You can add both public and private tags, that you or the mod creator selects when uploading their mod. Then you and your users can filter on these tags.
2. Second and probably more suitable is the metadatakvp system. When submitting a mod, you can include hidden metadata key value pairs, which can include any information you want (version, type of mod, compatibility etc) which you can also use for filtering, but this information is always hidden from your users.
3. Finally, we also have a metadatablob field, which you can place any text in (like previews etc). This isn't searchable so it might not quite do what you want, but it is available.

Does that do what you require? We have a very comprehensive metadata system to hopefully cover any needs your game has.

[leper]

1. Those are not really anything the user should care about (apart from the compatible/not compatible thing, but that's deduced from the data). And since that is just some string I could force JSON in there, but that would get somewhat ugly and 3 seems nicer for that.

2. Ah, that's only available when doing a specific query for the mod. That seems slightly unfortunate since currently I only need a single query (after getting the id) for everything, and I'd mostly like to keep it that way.

3. Being able to search that isn't really required, so metadatablob sounds like it could work. Ah that's actually just some (probably) escaped string, hm that would complicate the client code a bit.

I guess I can work with 3, the best thing would be some custommetadata field with a better name that one can store actual JSON in. That would be simplest for the parsing code.

[intenscia]

1. Tags are important, because they allow your mod creators and users to understand what it is mods do. They might be purely for show, but you could add tags like "theme: desert, city, western etc" or tags like "type: AI, total conversion, unit etc".

2. After you get a mods ID, you can request all of the metadatakvp that has been allocated for that mod. We could include the metadatakvp in the mods object if that helps save a request.

3. You can store JSON in the metadatablob field. It is your field to put whatever information you please in it.

[leper]

1. Yes, but that's orthogonal to the issue of having some metadata that the game needs for mods.

2. Yet the documentation lists the type of metadatablob as string, so there'd be a need for some decoding and given that this should store JSON unescaping of " characters.

[intenscia]

I will update the blob to be a blob. So you can store JSON in there (you will need to decode it of coursE), but you should be able to use that field to store anything.

[leper]

I just noticed something: All those are for the mod, not a specific modfile. I need the latter, not the former.

[intenscia]

You would like to include metadata with a specific modfile, not in the mod profile?

[leper]

Yes.

[intenscia]

Understood, at the moment there is no functionality to do this. A few questions:

• How critical is this feature to you?
• Could you use the mod metadata?
• Would it need to be searchable
• Would it be stored as a blob or key/value pairs

This isn't on our roadmap and we are trying to do complete API v1 lockdown soon, so trying to minimize changes unless they are very beneficial to API consumers.

[leper]

• Not very critical. Unless the integration should be more than a simple mod downloader without any dependency resolution or anything.
• That would be a bad workaround. Dependencies (which is most of what I'd store there) can change based on versions.
• No. It would be nice if it would be in the modfile object so all queries that currently return a modfile object also return that tiny amount of optional metadata.
• Blob. Having that defined as some valid JSON would be best, but if it needs to be encoded in some way I can also handle that.

As mentioned above I don't really need this, it would however make nicer integration a lot easier (or rather possible).

[intenscia]

Mod files now also contain a metadata blob field. See documentation for details.

[leper]

I can see the field in responses, however setting the value in the webinterface does not change the result (null). Changing the mod (not modfile) metadata shows up in the responses however. Is this not yet live?

Thanks, that will allow nicer integration.

[leper]

I noticed that the metadata_blob is always null in /games/{game-id}/mods queries, not in /games/{game-id}/mods/{mod-id}/files queries however.

For both simplicity and not doing tons of pointless API calls it would be great if this would be returned in /games/{game-id}/mods queries.

[intenscia]

These are two different blobs:

metadata_blob in the /games/{game-id}/mods query is metadata relating to the mod object.

metadata_blob in the /games/{game-id}/mods/{mod-id}/files query is metadata relating to an individual file object.

It doesn't make sense to return individual file metadata in the mod object. How would you know what metadata belongs to what file if the mod has many files and we merged all of that metadata together?

[leper]

/games/{game-id}/mods returns two metadata_blobs in different places.

The first one is part of the mod object. The second (and currently always null) one is contained in the modfile object that is the primary modfile of the given mod.
This is about always returning the full modfile and not the modfile but with metadata_blob set to null.

See below an example with the response reduced to just show some of the returned data:

{
  "data":[{
    "id":332,"game_id":57,
    "name_id":"terra-magna",
    "metadata_blob":null, // Mod metadata blob
    "modfile": {
      "id":838,
      "mod_id":332,
      "filehash": {"md5":"5a60b31418aab823a717f0f86003efdc"},
      "filename":"tm.zip",
      "version":"0.0.22",
      "changelog":null,
      "metadata_blob":null, // Mod file metadata blob XXX
      "download_url":"...",
      ...
    },
    ...
  }],
  "result_count":1,
  "result_limit":100,
  "result_offset":0
}

[intenscia]

It appears you are right and this was a bug. We have fixed it now.

[leper]

Works as advertised now! Thanks for the quick fix.