Revisit Verifiable Credential media types
brentzundel opened this issue · 21 comments
After the mediaman meeting we need to revisit our media types and likely change them
Summary of the meeting at IETF 119 for those that were not in attendance:
This all started about a decade ago with JSON-LD. We wanted to register "application/json-ld" for the media type. IETF's Media Types group feedback was effectively: "Well, what you have is a media type encoded in JSON, so you should choose something with a "+json" at the end of it. The JSON-LD WG at the time said: "Ok, that's weird, but we'll do that, and selected application/ld+json".
Later, when the W3C DID WG was under way, the group went: "Ok, based on IETF's guidance on suffixes, one serialization of this application/did thing is a JSON-LD document, so, obviously, the media type is application/did+ld+json, and when we went to register that, the IETF Designated Experts went... "Hmm, that's syntactically valid, but we don't have any guidance on how to interpret multiple plus signs in a media type, so we can't approve the registration."
At that point, the DID WG was asked to justify the usage of multiple structured suffixes, which were allowed, but not thought through cleanly when media type suffixes were introduced at IETF -- that was 5 years ago, it's taken 3 of those years to write up the rules on how to interpret multiple suffixes such that no one would object to what was written up.
We have now done this, reducing the open issue count on the spec down to zero, but new ones keep being raised (at the last minute during the IETF meetings) because a few people continue to feel "uneasy" about suffixes... and it seems like this is because there is a contingent of people at IETF that feel that suffixes were a bad idea to begin with, except that there are now multiple media types that use suffixes at IETF.
The latest set of suggestions are to:
- List the use cases for suffixes, AND multiple suffixes, and then specify what constitutes "good" suffixes and "bad" suffixes.
- Grandfather "application/vc+ld+json" and "application/vc+ld+json+sd-jwt" in and then, potentially point to them in the future as examples of what NOT to do.
- Mark existing suffixes like
+gzip, and+zipas "bad". - Deprecate all suffixes, assert that it was a terrible idea, and deprecate the suffixes registry.
At this point, the time it will take IETF to come to a conclusion on this has an unbounded time horizon. It might take one more meeting, or it might take many more years for IETF to figure out if they want to keep suffixes or not, and the VCWG cannot be coupled to a timeline like that.
Gory details (video recording of the meeting): https://youtu.be/xEGSm6oN7gw?si=DBpVIhmbVXK78j6j&t=600
There are a few paths forward that we could take:
Option A: Take the Exception
IETF Media Types registration process would treat us as an "exceptional case" and grandfather application/vc+ld+json and all the other media types for DIDs and VCs in while they sort out whether or not to deprecate suffixes, or multiple suffixes.
Pros:
- We are decoupled from IETF's "suffixes decision" timeline.
Cons:
- IETF might point to our registrations in the future as an example of what not to do, which could cause reputational damage to the specifications.
Option B: Use No Suffixes
We could just register application/vc, assert the same thing that we do in the specification today (that it's a JSON-LD document), and move on.
Pros:
- We are decoupled from IETF's "suffixes decision" timeline.
Cons:
- Suffixes are the right thing to do here, so we'd be making a technical compromise to just move on.
Option C: Stay the Course
We could take a strong position in favor of multiple structured suffixes at IETF, use this as a use case for the need for multiple structured suffixes, help them figure out the language around suffixes and multiple structured suffixes, and stay the course.
Pros:
- We could help fix what's broken around IETF Mediatypes and all the glory that would come with solving this issue.
Cons:
- Unbounded time horizon to complete the task.
I'd be in favor of Option A + Option C. 😃 As in, we need to be able to move forward and because multiple structured suffixes are a perennial request from many communities across the Web, we need to make sure this work at the IETF heads in an enabling direction and reaches the finish line.
For vc-jose-cose, even if we register applicaiton/vc for use as the "cty" value, I believe we'd still want to use application/vc+jwt, application/vc+sd-jwt, and application/vc+cose for the "typ" values.
It's clear to me that suffixes make sense to express sub-syntaxes (of increasing specificity) of a super syntax. I tend to offer this sub-class analogy for how I think of it:
application/square+rectangle+shape
In this case, a square is a more specific type of rectangle and a rectangle is a more specific type of shape. In this same way, you could define the expression of an Animal in XML -- and add more specific characteristics for a specific Animal, say, a Dog:
application/dog+animal+xml
I think this is the useful way to use suffixes -- and that "The Right Thing To Do" is to say that this is the main use case for multiple suffixes and a good use of the feature. I'd say that we should either not speak to other use cases or discourage them (at least for now). I personally would have probably expressed the sub-classes in reverse order, but that's water under the bridge.
However, it does not seem to me that there's consensus on this understanding of multiple suffixes and I don't know if we can get there with little effort.
There are other people looking to use (or are already using) suffixes as a mechanism to express ("smuggle" in?) media type information about the content of envelopes or compression-style archive formats into a single media type (i.e., application/my-content+gzip). I don't think we should do this because, in that situation, there are essentially two different documents/files/things: one is a container of some sort that contains the other one, the content of some sort. It's usually the case that these containers are designed to carry arbitrary content; there's a strong decoupling. I think it is better for container types, if desired, to include a field where the media type for the content is expressed -- rather than inventing a new media type for every possible combination of these things. This does not seem to be a scalable approach to me and I think it would ideally be discouraged for these reasons.
This problem does not exist when one is just expressing a sub-syntax of another syntax as a single, more specific media type that can then be consumed by broader applications (e.g., editors of Shapes/XML) and increasingly more specific ones (Rectangle/XML Animal consumers) or (Square/XML Dog consumers).
I think there is significant utility in being able to express sub-syntaxes in this way with few drawbacks. However, if we're not able to get to consensus on doing it, it's not the end of the world, it's just a shame that will probably generate more work and might slow innovation in various directions. That's an acceptable outcome, but not ideal. If there is significant push back on this approach, I think choosing Option A or B is the way to go with B being an unfortunate, but acceptable compromise to move things along.
The issue was discussed in a meeting on 2024-03-27
- no resolutions were taken
View the transcript
3.5. Revisit media types (issue vc-data-model#1462)
See github issue vc-data-model#1462.
Manu Sporny: can we highlight the options now please.
… and write your opinions into the issue.
… might not be able to use multiple + signs.
David Chadwick: 1) application/vc+ld+json.
Manu Sporny: 2) application/vc.
… with +json or +jwt added to this.
… 3) do not register anything, wait for IETF group to finish its RFC.
Ivan Herman: a variation of 2 or 3 is to put a statement into the new charter saying that we will do the mediaType after the IETF has finished.
Manu Sporny: I think that's still Option C?
Manu Sporny: (but explicitly stating it in our Charter).
Ivan Herman: yes.
The issue was discussed in a meeting on 2024-03-27
- no resolutions were taken
View the transcript
2.2. Revisit media types (issue vc-data-model#1462)
See github issue vc-data-model#1462.
Manu Sporny: mediaType for DM does not have a consensus yet.
… e.g. multiple suffixes is a bad idea.
… Can the WG members review this issue.
Brent Zundel: multiple suffixes draft means that the authors did not describe single suffix properly.
@mnot @martinthomson please take a look at #1462 (comment) , which goes some way towards providing what could become text regarding "What makes a good suffix? What makes a bad suffix?" in the multiple suffixes draft at IETF. Would text along those lines address some/all of your concerns? Or is there additional guidance we'd need to provide?
Can we pick not use this venue for the discussion? We also have https://github.com/ietf-wg-mediaman/suffixes and mediaman@ietf.org. I think that @dlongley's comment is constructive, but I don't want to fracture the discussion between venues.
(This opinion is mine, with my W3C staff contact's hat put down.)
I agree with @martinthomson: this issue is not the right place to have a discussion on the merits (or not) of multiple suffixes. The problem we have to decide on is: what should the WG do with the VC spec. As (most of) the rec-track specifications are in CR right now, it seems that we should not have a dependency on the IETF discussions; this would delay the timely completion of the WG's work.
Looking at the alternatives in #1462 (comment), outlined by @msporny, option "B" is the most optimal as far as I am concerned. Version 1.1 of VC could be deployed without the introduction of multiple suffixes; we can do the same with Version 2.0. Also, as I said on the call, we can come back to this issue in future releases in case the IETF discussion eventually conclude in favour of multiple suffixes; we may want to add a note on that into the new, proposed charter of the WG to make it clear.
Also... I know it is not popular, it is not handy, etc., but we could register some profiles for the .../vc media type if we feel it would really help implementations. That can be done at any time.
I prefer options A and C since as @dlongley said, there are sensible reasons for having multiple suffixes.
I prefer options A and C since as @dlongley said, there are sensible reasons for having multiple suffixes.
... except that their meaning is not specified by any existing standard (yet?). I do not think we can publish a Rec with it, unless we put our document on hold, hoping that IETF gives a green light to it.
I support option B.
Possibly with a redundant entry for:
application/ld+json with a profile, as is supported in activity pub today.
I prefer A+C mixed (not B).
@dlongley -- I'm still digging through the suffixes threads on the IETF media-types mailing list and the suffixes GitHub issues... If you haven't voiced some version of #1462 (comment) in one or more of those, please do.
The issue was discussed in a meeting on 2024-04-10
- no resolutions were taken
View the transcript
2.1. Revisit Verifiable Credential media types (issue vc-data-model#1462)
See github issue vc-data-model#1462.
Brent Zundel: next topic to discuss is - 1462, revisit VC media type.
Brent Zundel: See https://mailarchive.ietf.org/arch/msg/media-types/iWc8TLcWOyO0jyqeiuF9VCZClIs/.
Brent Zundel: here's a link to the discussion that's continuing among the media type mgmt WG at IETF.
… it's a long discussion, a number of us have taken part in it, have shared opinions.
… manu, if you would be willing to give the group a summary of where the thread has gone?
Manu Sporny: Summary of what has happened to date: #1462 (comment).
Manu Sporny: I won't go into the summary in too much depth, there's a link to it here.
… we need to register a media type for VCs, we suggested using one that uses multiple suffixes, that has led to a long discussion at IETF over the past 5+ years.
… we thought we were done as of last IETF, that was not the case, we don't know when we're gonna be done.
… we're probably at the point that this group can't wait any longer for "the right thing to happen".
… a number of options has been brought up -- from use suffixes, to deprecate suffixes, and every variation in between.
… based on the last 2 weeks, seems like eventually IETF is going to allow people to do whatever, in certain patterns and categories.
… people are definitely using suffixes in some kind of ways, there seem to be different philosophies on it.
… the outcome if we don't try and force the discussion to terminate any time soon is -- everything registered in the past in fine, here's some guidance for the future (for media types w suffixes).
… so what it seems like will happen, they'll punt the explanation of what suffixes mean back to the spec that is registering the suffix.
… like "this is the media type we're registering, this is what suffixes mean, etc".
… we thought this was gonna end 6 months ago, and it hasn't yet, so that's where we are today.
… we can take a couple of paths forward. there was the 'exception path' where we can register whatever.
… option b was - not use any suffixes at all, and just use application/vc and /vp.
… and then for SD-JWT, you can add +jwt to the end of that.
… option c was - wait until discussion ends at IETF.
… as the editor of that spec, I don't know when that will actually happen. might be within 6 months. 3 months unlikely.
… I wouldn't be surprised if it went on for another year. Option C is - we're not gonna register a media type until the WG figures it out.
… there has been some discussion in the issue; my gut tells me that Option B would result in us being able to register it immediately with no major pushback.
… at the same time, it's not necessarily ideal, as some people are pointing out.
… there are likely more technically correct ways to address the issue.
… another option not there in the beginning, coming from Mozilla / Martin Thompson, was - our spec can say, here's the base media type,.
… however you can ALSO use these other media types -- you can serve it as application/ld+json, or something else, and here are the rules for doing it that way.
… that would require a decent bit of discussion.
Brent Zundel: I'm wondering -- my perspective, chair hat off, it seems like one of the reasons we'd want to move forward with a media type registered now, is because a data format that has a media type registered, there's a perception that it's "a real thing",.
… like, we've graduated, we're real, we have a media type, I think that's the perception.
… so from that perspective, it'd make sense to move forward and claim one.
… the pros of option A, and moving forward w that, is that multiple suffixes make sense to us. drawbacks -- registration of multiple suffixes media types will be used for all time as a bad example.
… and I don't want to be painted with that brush.
… I recognize the drawbacks of choosing something simple like application/vc.
… or you can just use ld+json in our description of the media type.
Dave Longley: +1 to registering
application/vc,application/vp, and saying thatapplication/ld+jsonis also permitted -- and keeping open the option to add multiple suffixes in the future.
Ivan Herman: I am pretty worried about all of these things; we've been fighting w this for I don't know how many years now.
… if we're getting to a registration of something that won't be really accepted by IETF, it leads to a lot of discussion, and at the end of the day, it will create problems for the final Rec.
… in the meantime, it's a lot of discussion; I don't remember how many media types we're talking about, and to add jose/cose to it, we're talking about 6 or 7 media types we have to fight for.
… and I don't feel like doing that. I repeat what I said last week, I still believe an option is, we either don't do anything right now, or just try and register /vc and /vp.
… additionally, we have this pending charter proposal that we have to talk about at some point.
… and we should add to the charter proposal -- if IETF takes a final stand, and if it allows us to register ld+json or whatever, we can do it in the next incarnation of the WG, and we don't do it now. I think that's the best option.
Ted Thibodeau Jr.: my sense of the IETF position at this moment is - there are a couple of very loud voices who have been around for a while, and type a lot, who are very much against what is being called "multiple suffixes", which I think of as sub-subtypes.
… it is not my sense that this is an IETF position as a whole, or even just of the decision makers.
… I don't think we should be scared off from doing the right thing, because we think it won't get through.
… most of us have a sense of how media types have worked historically, and that is not what is currently being trumpeted by the loud voices.
… the loud voices are saying, everything we've been doing historically is wrong, and we won't do it anymore. that should be objected to, by as many voices as possible.
… there are some errors in what has been registered in the past, and there are some errors in what's being requested in the jose/cose thread.
… there's other options, and I believe those won't get objections from IETF.
… the timing sucks, no way around that.
… but I don't want to do anything that precludes us doing the right thing as time moves on.
… IETF is giant in scope.
… but it's also a giant cruise ship, difficult to turn quickly. more voices help in that.
… partly because more voices can put it in different words, and can be heard differently. if you haven't participated in the thread, and understand what we're trying to do, weigh in.
Manu Sporny: I agree with a lot of what Ted said, I do think there is a minority opinion driving the discussion at IETF.
… I am still hopeful we can come to something that won't undo 20-30 years of media registrations. But that's not our fight. in this group, we just need a media type.
… I'm a strong +1 to what Brent said, that when you have a media type, something is real.
… for better or for worse. so let's pick a media type that does not limit our future options.
… for example, one thing we can do (I'm willing to write a PR), is to say, we're registering application/vc and /vp, today.
… and for the jose/cose specs, we can add a +jwt to it.
… in theory, that matches everything done previously, won't get pushback.
… and in our specs, we can say, alternatively, you may serve things as application/ld+json,.
… we'll make a very strong case for application/vc.
… but if you want general semantics, you can just do ld+json.
… in the future, if the multiple suffix stuff gets to consensus, in a future spec, we can say, you may serve it as application/vc+ld+json.
… the only downside to that approach is - all of a sudden, we have potentially 3 media types. but it's not that difficult for software to deal with 3.
… we do have one issue regarding file extensions.
… which you can associate with just one media type, if I remember correctly. but that's a smaller problem to deal with.
Ivan Herman: i have the impression that we are widely agreeing. the only thing I'd ask for this PR, which I agree with, can you also make a PR which adds something into the charter proposal for the future WG.
… or, rather, the maintenance WG.
Manu Sporny: happy to do that.
Brent Zundel: sounds like we have a path forward.
… we'll see how the discussion pans out with the PR. seems like we're in a place where that PR can be written.
IANA registrations requested, per Wednesday's working group decision: https://mailarchive.ietf.org/arch/msg/media-types/_pmQrj8nkW25FOqIXmPa1wyzizU/
In this case, a square is a more specific type of rectangle and a rectangle is a more specific type of shape. In this same way, you could define the expression of an Animal in XML -- and add more specific characteristics for a specific Animal, say, a Dog:
application/dog+animal+xml
Looks like we're building an ontology of media types. That's a fair use case for this description with a long history. I like it!
The issue was discussed in a meeting on 2024-06-12
- no resolutions were taken
View the transcript
3.2. Revisit Verifiable Credential media types (issue vc-data-model#1462)
See github issue vc-data-model#1462.
Brent Zundel: next we have 1462 - we have already talked about w.r.t. media types. will be closed soon.