ActivityPub support
strugee opened this issue · 76 comments
ActivityPub is a standards-track protocol for server-to-server federation, as well as client-to-server APIs at the W3C. https://www.w3.org/TR/activitypub/
My understanding is that this is something Mastodon is somewhat interested in already, but I'm filing an issue to have "somewhere" to track this.
Edit: here's a checklist for things to implement. I cut out client-to-server stuff, although most of it applies to both.
Does your application implement URLs for:
- Object
idproperties? (Section 3.2 Retrieving objects) - Actor inboxes?
- Actor outboxes?
- Actor Following collections?
- Actor Followers collections?
- Actor Likes collections? (optional)
- Binary uploads? (Section 6 Uploading Media, optional)
Does your application handle:
- The special Public collection?
- Server-to-server Activity side effects, like Undo side effects?
- Linked Data Notifications RDF representations (very optional)? (Section 8.1 Delivery)
Do you:
- Properly support Actor simplified login naming? (Section 4)
- Deliver activities properly?
- Implement authentication and authorization checks for server-to-server interactions?
- Scrub Activities for cross-site scripting attacks?
- Have a plan for combating spam?
Can you compile a checklist of things that need to be implemented to become AP-compliant from a server-to-server perspective? (Not interested in client-to-server at all, right now)
Something like inbox, outbox, representation of stuff, etc.
My biggest desire wrt. AP is the ability to implement follower-only messages as well as direct messages without leaking them to old GNU social server that don't understand privacy scopes. As far as I understand AP servers must respect the addressee collections, so it should be a thing out of the box, plus there are no old faulty AP servers to worry about leaking stuff to accidentally.
cc @evanminto
I'm already gearing up to look at what still needs to be done for implementing AP integration Friday through this weekend. I've been within the W3C group, so I have a good understanding of it already. Working off of the existing AP mastodon branch, of course. Anybody else want to look at it with me?
FTR that's also my biggest desire wrt what AP integration would provide.
@Gargron sure, I drew up a list for pump.io. pump-io/pump.io#1241 is our tracking issue but I have a generic version of that list that I'll find for you shortly
I am suspecting that the first step is writing JSON representations of ActivityStreams items. Currently we do that for Atom XML only. The logic should be similar enough, except you'd probably write Rabl templates instead of using a class like AtomSerializer and Ox (although from a performance perspective, doing something with Oj directly without invoking Rabl would be a huge boost)
@Gargron yeah I'm not familiar with exactly how the OStatus protocol works but probably for any endpoint that would previously return some ActivityStreams XML you'll want to just content-negotiate and return JSON when needed. Note that ActivityStreams 2.0 also includes an appendix on how to "upgrade" ActivityStreams 1.0 objects to AS2, so you could probably just do that in the request handler and serve the result
I didn't get very far in my AP integration before I got sidetracked by some personal life stuff, but there IS at least a basic example of what Gargron is talking about for representing accounts as Activity Streams 2.0 on master. I have some more unfinished work (rudimentary outboxes) on a private branch.
I was planning on jumping back into it a bit this weekend too, and I'm glad to have some more folks looking into it!
The ActivityPub network page on the w3c wiki has some info on the progress happening in other social networking software projects.
That sounds like a reasonable first step: the ActivityStreams 2 data/vocab model. Get existing endpoints to render content-negotiated JSON (application/activity+json) for posts/feeds. I still need to look to see what the existing work does, but let's get a checklist... with this on top.
We should obviously validate against pump.io's AP implementation and any others that might be at all ahead: https://www.w3.org/wiki/Socialwg/ActivityPub_network (rstatus is on there... should I even? nono) and Soci-el and Pubstrate which are both a kind of reference implementation.
@wilkie note also the pump.io's implementation is in very early stages and is very much a WIP. That being said it's very high priority for us and I'm hoping to at least make good progress by the end of the month
@strugee definitely understandable. hopefully we can work off of one another in terms of testing implementations. pump.io is obviously much more of a real-world federation than the reference libraries.
@wilkie definitely. I should note also that our implementation may be slightly wonky since we have a large existing network to worry about... but Mastodon has the same problem, and in an even worse form (since at least pump's existing stuff is pretty close to AP)
How does discovery work? Links to AP resources from Webfinger?
Authentication/authorization for server-to-server communication - is there anything prescribed for this? Do we just reuse the magic key signing concept from Salmon?
How does discovery work? Links to AP resources from Webfinger?
What do you mean by "discovery"? Discovery of what?
Authentication/authorization for server-to-server communication - is there anything prescribed for this? Do we just reuse the magic key signing concept from Salmon?
No formal prescription, but there are some suggestions for auth schemes that we expect people will probably want to focus on. Eventually these may become standardized as extensions to ActivityPub core. https://www.w3.org/TR/activitypub/#authorization
How does discovery work? Links to AP resources from Webfinger?
ActivityPub does "follow your nose" of links to links to links; a person's identifier is a uri of some type, as opposed to using a "magic" type toplevel tied-to-domain route such as /.well-known (which is known to have problems; eg what if you have multiple applications with usernames on the same domain? Who gets the webfingers?) But! There is a uri scheme draft for webfinger (expired, but could be still followed if you don't care about that), so technically you could do it this way: "acct:foo@website.url" for webfinger style lookups.
When Linked Data Signatures is used in combination with ActivityPub, the server should assign an actor a public and private key pair, and link the public key to the actor's profile object, which may later be used with the Linked Data Signatures verification algorithm to validate the authenticity of messages passed along the network.
Okay, this sounds like the Salmon verification mechanism. We already have public/private keys generated for accounts. Could sign and verify messages this way, though perhaps a HTTP header signature as used in PubSubHubbub would be more applicable than the magic envelope Base64/XML approach of Salmon.
There is a uri scheme draft for webfinger (expired, but could be still followed if you don't care about that), so technically you could do it this way: "acct:foo@website.url" for webfinger style lookups.
The less things we have to change to get AP in, the better. We do user discovery based on WebFinger. So now we just need to assign more links to AP-specific resources in the response. I am guessing there are only two such links, inbox and outbox, am I correct?
The less things we have to change to get AP in, the better. We do user discovery based on WebFinger. So now we just need to assign more links to AP-specific resources in the response. I am guessing there are only two such links, inbox and outbox, am I correct?
Those are the big ones, but you probably also want to add the read-only collections of followers / following (and maybe also likes, if you like); see the Actors section of ActivityPub.
(PS: The way you'd do Webfinger discovery via ActivityPub btw is to do the "acct:foo@example.org" to look up the http(s) url, then do content negotiation to pull out the ActivityStreams actor profile object... which is just some json. You could also just start passing around peoples' handle uris as their main identifiers, but I know that would be a bigger change.)
Sorry, I'm a little confused about the WebFinger discussion. Wouldn't relying on WF for account discovery make Mastodon incompatible with other AP implementations? I figured we'd be building new AP-specific endpoints, in which case we would just use URLs to JSON resources as @cwebber mentioned.
I'm not super familiar with WF and how it's used on Mastodon, so I may just be missing something.
Indeed, it would seem unnecessary to go through the process of implementing a new protocol without having compatibility to other AP implementations. Using webfinger would essentially kill that.
Relating to signing, it would be great if the first implementers could somehow agree on a way, otherwise there is risk that many implementations will be incompatible due to different signing required. It would be great if we could agree on Linked Data Signatures embedded into the payload, like the examples in https://w3c-dvcg.github.io/ld-signatures/#linked-data-signature-overview .
Using HTTP signatures would IMHO not be a good idea, since often signatures need to be checked at a later point. The receiver might delay the signature check (due to processing load or otherwise). Also having the signature in the message itself allows it to be relayed, if needed, which AFAICT is not possible with HTTP signatures. For example, the relay system used with the Diaspora protocol relies on the possibility to forward payloads using a third-party server. Would love to build AP support for that once my federation library has AP support :)
@cwebber what do you think, is there any possibility to still clarify the signature part in AP spec to recommend Linked Data Signatures over anything else? Right now it is kind of confusing, saying "Linked Data Signatures and/or HTTP signatures", which basically might mean some implementers implement one, one or the other or both.
@jaywink agreed it's kind of messy having the spec not pick a method. See w3c/activitypub#77. Probably the way forward is to closely track what folks are implementating, in a very visible way, and see if we can get quick convergence. If we can, we can at least observe that, even if the spec doesn't pick one. For more on this, we should probably move to an AP issue; feel free to create one.
I'll also add different auth profiles to the extensions page. This seems like a great thing to pursue in the Community Group.
Here's a question, how hard would it be to use plain http(s) URIs as the identifiers on the federation protocol level? webfinger identifiers could still be used to look someone up in the UI
By the way, if ActivityPub is a lot to absorb still, I've written a tutorial that might be a good way to grasp the concepts quickly!
@cwebber Is there a typo in the very last example? It sends the message to "alice/following", rather than "alice/followers"
Also, discovery of the actors' URLs is still a mystery to me. The reason I brought up Webfinger before is simple: I am Gargron@mastodon.social, you are cwebber@toot.cat. I know how to address you in a post by typing @cwebber@toot.cat, my server knows that this means it needs to ask toot.cat for the user cwebber. This is discovery based on handles. I don't understand how this is supposed to happen for AP actors.
@Gargron Fixed, thanks! That was a typo.
Going to type up more responses re: webfinger in a few.
It's still a WIP, but I might as well post what I've got so we avoid duplication of work and/or me going down an unnecessary rabbit-hole: https://github.com/tootsuite/mastodon/compare/master...evanminto:activitypub-outbox?expand=1
My plan was to add endpoints for outboxes and basic statuses (in the form of create activities, announce activities, and notes) to get us started on a sort of "read-only" version of AP. I was just making all the endpoints public for now and restricting the statuses to show only publicly available ones. Is it acceptable to start with this approach, or should we tackle Auth issues first (definitely not my area of expertise)?
Thanks @wilkie! I think your summary of how to use the tags is particularly useful.
I've been thinking about it, and I think that ActivityPub and WebFinger can work together fairly seamlessly, without creating disconnect by using both the acct: and https: schemes at the same
time.
There are two scenarios, so let me go through each.
Mapping webfingers to https:// addresses during composition
This one's fairly simple, because WebFinger already does most of the work for us; imagine someone's posting something something like the following:
Thanks to @alyssa@social.example for another exciting release of FrobinatorPlus!
(Or maybe @alyssa@social.example is entered into some addressing place in the UI.)
We can do a webfinger lookup on @alyssa@social.example to find her user page, which would be an https:// address anyway, from which we can use content negotiation to pull down her AS2 actor description. We can then use that address in the addressing portion of the constructed AS2 object to be sent around via ActivityPub.
(I think there is some variance, if I remember correctly, as to how that user page may be looked up in webfinger; I think Diaspora has you pull out the user profile via hcard but I think OStatus w/ GNU Social / Mastodon / postActiv may do things differently? Can someone clarify?)
Infering a user's Webfinger address from their Actor profile
Say you had an actor's profile already:
{"@context": "https://www.w3.org/ns/activitystreams",
"type": "Person",
"id": "https://social.example/alyssa/",
"name": "Alyssa P. Hacker",
"preferredUsername": "alyssa",
"summary": "Lisp enthusiast hailing from MIT",
"inbox": "https://social.example/alyssa/inbox/",
"outbox": "https://social.example/alyssa/outbox/",
"followers": "https://social.example/alyssa/followers/",
"following": "https://social.example/alyssa/following/",
"likes": "https://social.example/alyssa/likes/"}How can we extract the webfinger identifier from this?
Well luckily the preferredUsername can help! Simply concatenate the preferredUsername onto the domain name of that user, and whammo, you've got alyssa@social.example!
What do people think? We could maybe add this as an informative section of the ActivityPub document if people think this is the right route, since many existing federated documents have webfinger identifiers. Maybe combine that with the "tagging with webfinger identifiers" bit from Wilkie's writeup?
@evanminto Working on the publicly-viewable outbox is a great place to start! Thanks for working on it :)
I've referenced the issue of WebFinger identities over on the ActivityPub tracker. I'd like to explore including an informative section. @Gargron, could you look at what @wilkie and I wrote above and see if you think it answers questions? I wonder especially if the section I wrote captures things well enough to be usable, and if it matches what you think would fit Mastodon's usage, particularly re: its UI. It would be great if, even without changes to the existing UI, the above worked.
Good suggestions from evanminto and from oshepherd on the ActivityPub issue tracking this. I've also put a stub wiki page for WebFinger and ActivityPub. It seems like there are good options though!
Basic outboxes are in, though there's definitely a lot of work left to be done to get them to federate correctly, etc.
It sounds like @Gargron's top priority is federated private messages, so I'm inclined to aim for that as the first big milestone. I imagine it will end up looking something like this (I deliberately left out the client-to-server bits since Eugen isn't interested in that yet):
- User on Mastodon Instance A sends private DM as usual (sets privacy to Private and tags a user on Mastodon Instance B via WebFinger-style tag). The UX can be identical to the current one, though it will now open up the ability to send DMs to/from non-Mastodon AP implementations.
- A creates a status in Masto-land with privacy set to Private.
- A does WebFinger lookup and gets an
Actorobject from B. - A locates the
Actor's inbox and sends an authenticated POST request to it. Payload is aCreateactivity with aNoteobject, withtoset to theActorobject representing the user on B. - B creates a status in Masto-land representing the received activity.
- B sends a notification to the receiving user if necessary.
This looks like it involves a number of intermediate features, namely:
- WebFinger lookup
- Server-to-server authentication (this is the big one, and I may want somebody with more auth expertise to work on it)
- POST to inbox
- A mechanism of falling back to OStatus-based private messaging for older Mastodon instances.
Please let me know if any of this sounds off to anyone or if I missed anything. I'm still learning about both Mastodon and ActivityPub as I work on this stuff.
It would be cool if there was some public test suite where you could test if your implementation works (kind of like the webfinger one on its official website, or websub.rocks)
WebFinger discovery of Actor's and inboxes seems like a good start, yeah. And we more or less know how that should be done, yes?
Linked data signatures/json web signatures for signing the messages for this? There's no s2s recommendation for private message verification in ActivityPub... but it is hinted that LDS/JWS is the way to go. That seems fine for now until somebody wants to specify an encryption scheme where it is decrypted in the client app so admins cannot see DMs either. That would be an interesting extension... some day.
There is a lot of discussion about how you even recognize a private message in ActivityPub: w3c/activitypub#196 It is annoyingly under-specified. Particularly, if you are meant to use bto or to when you put the message in the outbox and whether or not it determines between a private message and a directed message, respectively, and how that looks from the perspective of the inbox. This might not be resolved until a week from today :/ But I generally like the assumption that a lack of a to field or any addressing is automatically assumed private, where bto is used to deliver (and stripped from the message).
Yes indeed a test suite would be very helpful. I'm not sure the status of that, but I think we've got some spec editors/contributors in this thread (@cwebber?) who can speak to it.
@wilkie Isn't there a convention where you specify "followers" and "public" collections as recipients for various levels of visibility?
Yeah the spec seems relatively clear to me. Default is private to everyone, and then you explicitly add recipients (followers, public, individual users, etc.) to expose it to more people.
@Gargron yeah, your normal posts would be "to" your "followers" collection. a post to everybody is "to" the "https://www.w3.org/ns/activitystreams#Public" collection. You can invent a local instance collection for that option. a reply might be "to" a person and "cc" whatever the "to" would normally be.
The github issue there is also about the differences, if there needs to be any, between a direct message and a private message. It's kind of a silly issue, but it brought up a weird case. I said there, to argue against my better judgment that there is a subtle difference I guess, that you might get a message with you in the to field (direct), and at the same time it is technically possible to get a message with absolutely no to field at all (more-direct... private?, heh, aka sent via bto or bcc). So I wanted to point out that this is kind of weird and you have to handle that (and optionally render it differently, if you're like the people in that issue.) Is there actually a difference?? I don't really know. Not really?
ActivityStream Objects are retrievable, yet ActivityPub says basically nothing about access control. It only talks about delivery and who gets a notification, etc. It only says implementations may decide to support access control for object retrieval (s 3.2). Hard to verify that anyway. That's a strong case for an extension... some day... that does end-to-end crypto for private messages so that weaker implementations that don't support them could only ever leak gibberish. But, yep, default to private (although allow an authenticated account see everything in their own inbox), filter content by to and cc on any GET of an Object/Collection seems like a perfectly reasonable thing to do. Just wanted to point out that, well, it's not very clear how "private" these messages actually are in the wild.
Linked data signatures/json web signatures for signing the messages for this? There's no s2s recommendation for private message verification in ActivityPub... but it is hinted that LDS/JWS is the way to go.
What about simply using Linked Data Signatures to sign and verify the message contents and authorship? If I understand correctly, it should be enough alone without adding JSON web signatures on top? As a future implementer it would be nice to have some kind of consensus, otherwise there will be a lot of incompatible implementations.
I think it is important to use a signature that can be implemented into the object and verified later. This allows for the receiver to process the message at a later time and is a must for relaying payloads forward to third in line receivers (for example A comments on Note by B but C has also received the note but A and C don't know each other -> B should relay the comment to C and C should verify that it is really from A). Linked Data Signatures seem perfect here since the signature would travel with the object itself. JWS feels like a lot of work reading the spec, if it isn't needed to validate integrity and authorship.
@evanminto re message signing using LDS, some additional discussion here that might help.
What about simply using Linked Data Signatures to sign and verify the message contents and authorship? If I understand correctly, it should be enough alone without adding JSON web signatures on top? As a future implementer it would be nice to have some kind of consensus, otherwise there will be a lot of incompatible implementations.
LDS is perfectly fine for origin verification; it has a lot of provenance which isn't at all a bad thing. JWS/JWT are fine, too, and well-supported in various languages. This isn't the place for general consensus... that should be part of SWICG and then as an extension. The main spec isn't the place to recommend how to use any particular signing algorithm or data format. That said, totally fine to just pick one here, implement it, and present it to the CG, I'd think? And simply make it the general consensus via being first.
I'm ok with LDS.
I'm a bit confused about all of this. LDS seems like it handles the case where we want to confirm message authenticity, but what mechanism do we use to ensure that servers acting on behalf of a user have permission to access a restricted resource (like a Note or Create activity representing a DM)? Is that what HTTP Signatures are for?
@evanminto Yes, it should be usable for that.
TBH this is the biggest challenge right now in the spec; I personally think Linked Data Signatures + HTTP Signatures is the way to go, but OAuth 2.0 bearer token direction is better understood by many existing implementors (some of whom are very skeptical of a signature based direction). The spec is wishy-washy here, loosely describes how to do both. Obviously this is going to be important to sort out for federation purposes.
BTW, a slight tangent but related: after the Social Working Group is done, the Social Web Incubator Community Group is going to be taking over its space to continue this stuff (I'm co-chair of the group). I encourage everyone here interested to join the group, it's open to everyone, and we could really use voices from Mastodon folks as well as people coming from OStatus type directions in general who are working on ActivityPub support.
but what mechanism do we use to ensure that servers acting on behalf of a user have permission to access a restricted resource (like a Note or Create activity representing a DM)? Is that what HTTP Signatures are for?
@evanminto wouldn't that be "client to origin or remote server" authentication? AFAICT, @cwebber correct me if wrong, but federation between servers basically happens via POST to remote endpoints. AFAICT, HTTP signatures would not really be useful here. Just want to be sure I understand what is going on, as planning implementation myself.
My general rough idea for server-to-server auth was that
- The user's profile would contain a reference to one or more JSON Web Keys with a usage something like "https://www.w3c.org/ns/activitypub#origin" (maybe via a "jwks" endpoint URI?) (N.B. there is no registered activitypub namespace at w3c at the moment at least. this is just an example)
- The origin server would sign a JWT with the audience containing the destination server hostname and the subject of the origin user using said key. (the destination server hostname MUST be included to avoid impersonation by the destination server!)
So, the profile ends up getting something like this:
{
...
"endpoints": {
"jwks": "https://user.example.com/jwks"
}
}
which points to
{"keys":
[
{"kty":"EC",
"crv":"P-256",
"x":"MKBCTNIcKUSDii11ySs3526iDZ8AiTo7Tu6KPAqv7D4",
"y":"4Etl6SRW2YiLUrN5vfvVHuhp7x8PxltmWWlbbM4IFyM",
"use":"https://www.w3c.org/ns/activitypub#origin",
"key_ops": "sign",
"kid":"c425159456ddf15f"}
]
}
(sample key stolen shamelessly from the JWK spec)
This would result in a JWT with a JWS protected header of
{"alg": "ES256","kid": "c425159456ddf15f","typ":"JWT"}
and a body of
{"sub":"https://user.example.com","aud":"https://social.example.net","iat":"1493472375","exp":"1493472375","jti":"07482eed30a00df8"}
These would then be base64url'd/concatenated/signed per JWT/JWS and stuffed into an authorization header.
POST /~user/inbox
Host: social.example.net
Authorization: JWT-Bearer eyJhbGciOiJFUzI1NiIsImtpZCI6ImM0MjUxNTk0NTZkZGYxNWYiLCJ0eXAiOiJKV1QifQ.eyJzdWIiOiJodHRwczovL3VzZXIuZXhhbXBsZS5jb20iLCJhdWQiOiJodHRwczovL3NvY2lhbC5leGFtcGxlLm5ldCIsImlhdCI6IjE0OTM0NzIzNzUiLCJleHAiOiIxNDkzNDcyMzc1IiwianRpIjoiMDc0ODJlZWQzMGEwMGRmOCJ9.<signature>
(Warning: JWT-Bearer as an authorization method is completely made up!)
The destination server would verify that it is in the audience, look up the subject's keys, make sure the specified key had the correct usage, and verify that the issue and expiration times are not invalid (allowing for some timing slop!)
--
HTTP signatures are interesting but I think I'd be tempted to start from the old OAuth 2 MAC drafts instead as they received quite a bit more scrutiny.
Linked data signatures are also interesting but I also see them as somewhat orthogonal. I see them as useful for authenticating messages are actually from somebody - essentially, I wouldn't want my server to be in possession of that key
(N.B. there is no registered activitypub namespace at w3c at the moment at least. this is just an example)
BTW, this is because activitypub is using the (as:) activitystreams namespace http://www.w3.org/ns/activitystreams. That space is open for extensions.
AFAICT, arent JSON web signatures incompatible with some of the use cases for server to server delivery? I'm talking about this one in particular, involving three people on three different servers:
- User A posts a note to his/her followers User B and User C
- User B and User C do not know each other
- User B replies to the note, delivering the reply to User A
- User A delivers the reply to User C
This is basically covered in https://www.w3.org/TR/activitypub/#inbox-delivery and is the basis of how interactions should be delivered by the author, since he/she is the only one who knows all the recipients in all cases (works the same in Diaspora, Friendica, etc).
As far as I understand it, there is no way User A could sign the reply by User B to deliver it to User C, using JSON web signatures? IMHO Linked data signatures is exactly the case for this where User A just forwards the already signed payload to User B, who is then able to check authenticity of it using the signature done by User C.
The above is about authorization, not authentication. You want the onward delivery to be signed by the user doing the onward delivery so you understand who is sending you this object (User A is delivering me this thing by user C).
LD-Signatures or similar would provide a method of authenticating a message (did user B actually write it?) without fetching it from its' origin, but they add considerable complexity.
The above is about authorization, not authentication. You want the onward delivery to be signed by the user doing the onward delivery so you understand who is sending you this object (User A is delivering me this thing by user C).
This is exactly my point. The server of user C should NOT trust content delivered by user A server, that claims to be authored by user B. If such trust is given, user A can pretty much impersonate and generate fake content about any other user on the network. If such trust is not given and the payload from user A is properly validated (using LD signatures), against the public key of user B, user A would never be able to fake content related to other servers.
This is not even complex, but totally necessary for receiving servers to be sure that the author really did write what the payload claims. It's also how Diaspora and Friendica do things.
Would be interesting to hear (as we're in the Mastodon issue tracker 😆 ) how OStatus does delivering and signing replies so that all participants receive the reply? OStatus specs are a bit lacking or I just can't find good documentation.
In OStatus, all participants don't receive all replies. Only the author of the original post receives replies to it. Other servers may receive them incidentally by virtue of following the authors, but there is no redistribution.
Thanks everybody for the detailed discussion about auth schemes! Again I'm no auth expert, but from where I'm standing, HTTP Signatures + Linked Data Signatures seems like the best long-term bet, since a) they use JSON-LD just like the rest of ActivityPub and b) they're going to be fairly consistent from implementation to implementation, and the priority here is for services to be able to talk to each other without deliberately coordinating their auth implementations. I'm working on adding HTTP Signatures support to Mastodon in a private branch, should have something to show within a week.
I think we can close this? 😀
1.6.0 implements ActivityPub
Awesome. 😁. We've just mentioned it on the w3.org homepage news, too.
Related awkward question: did you pay attention to the client-to-server part of the spec? I understand you've got a good working protocol there and no substantive reason to change. Do you think we should take out that part of AP and switch to documenting what Mastodon does, so the industry can converge better?
Do you think we should take out that part of AP and switch to documenting what Mastodon does, so the industry can converge better?
No no, I don't expect you to do anything like that. Mastodon REST API is very Mastodon-specific, and that's also part of my reasoning for not really wanting a standard client-to-server API... With our REST API app developers have a very clear-cut direction for designing their app, and any Mastodon app acts more or less the same. But would a Mastodon-type mobile be a good fit for MediaGoblin, or vice versa? I don't think so.
C2S basically means the server has to do little more than perform distribution and some persistent storage. The business logic, anything that makes Mastodon different from say, Hubzilla, moves to the client.
Can you give me an example of that "business logic"? What do you expect to be distinctive about Mastodon for years to come? .... And why/how is that distinctiveness helpful to end users?
I've always imagined these systems should all be the same, so users can move between them and connect between them. Curious what's missing from my mental picture.
The 500char limit comes to mind, but that seems pretty trivial...
@sandhawke Right now: When you use Mastodon, you have an expectation of capabilities it has, UX, packaging, branding. App developers just have to implement the REST API, and there's a whole bunch of apps for iOS, Android, Windows Phone, etc - all relatively simple to make. Now, if instead it was up for the app developer to come up with "this works like twitter" or "this works like instagram" or "this works like facebook", you would have Mastodon for Android, and nothing else, because that functionality would be entirely within one app on one platform, that app would be as hard to make as Mastodon right now, and for other platforms you might not have a port of the thing, you might have totally different apps instead.
I agree the current Mastodon brand wouldn't make a whole lot of sense in the world I'm imagining. Is that important to you? Would it be okay if people starting thinking they were using (android UI) Tusky, and talking about Tusky, and not even know or care their server was running your code? If Tusky had account creation that'd be pretty easy. Or if someone made another server speaking the Mastodon client-server API, and called it tusky-server, and it got really popular, mostly just used with Tusky? What harm would that do?
And then what if they upped the character limit...?
Also, aside from the UI, what's the difference between Twitter and Instagram? I'm not a heavy ig user. Mostly they seem to have the same functionality. Ponder, ponder. Oh, yeah, you can't post text to IG, only pictures and video. And that means you can't post links. And you can't share/boost. And replies are not first-class objects. It's funny how two systems can feel so similar when one is so restricted.
So, are we doomed to a future where I'm going to always have to use eight different platforms, each with its own quirks and privacy issues and abuse issues, if I want to stay in touch online with 90% of my friends and family? (To reach 99% of them I'd probably need to be using fifty platforms, including the postal service and voice telephone calls!)
@sandhawke I don't think gargron is primarily talking about the brand. The mastodon server is not a fully-general ActivityPub server, it's built with a specific set of assumptions and limitations. We don't believe these assumptions are going to be the same with everyone we federate with—that's the whole point of federation. But every client that wants to interact with our server does need to be aware of these assumptions and intentional limitations. Otherwise it would just be a very limited and confusing user experience.
I think there are ways to address both of these problems, and I think personally I would be interested in seeing mastodon speak some of the C2S api, but the problem is that it addresses a different use-case and has a fundamentally different scope then mastodon's current API.
So, are we doomed to a future where I'm going to always have to use eight different platforms, each with its own quirks and privacy issues and abuse issues, if I want to stay in touch online with 90% of my friends and family? (To reach 99% of them I'd probably need to be using fifty platforms.)
Isn't this why we have S2S federation? So you can talk between different platforms?
@nightpool From how you frame it, it sounds like it'd be very useful to spell out a Mastodon profile for AP. Explicitly and precisely state every restriction Mastodon makes from full AP, so people know what they are trying to interoperate with. Probably also distinguish which restrictions are expected to remain in place forever and which are subject to change at any point.
Doesn't S2S federation suffer from all these interop problems, just like C2S? I mean, how could you federate from Twitter to IG, given just the differences I named? Or if I started a Mastodon instance which I modified to increase or decrease restrictions, how would that impact users of my system and others' systems?
@Gargron and I have already talked about this plenty over PM, and I know he's unconvinced of AP C2S... and that's fine! It's a lot less important than S2S. I still think it's likely doable and desirable (would be nice to use my AP emacs client with Masto) but I'm not a Mastodon dev, so :)
Note that one (slightly futuristic?) clear use case for using the AP C2S API (aside from giving a nice universal C2S API that you could use for multiple applications) would be if you would want to support end to end encryption as an optional feature, In that case the communicating clients could wrap up the AS2 objects in an encrypted envelope and send them to each other across the servers, without the servers being able to read them. Since the object you unpack is also AS2, assuming your client already knows how to render AS2 because you're using the C2S API rendering it should be quite feasible. (Of course there's a whole swath of other challenges around key management in E2E!)
Would it be okay if people starting thinking they were using (android UI) Tusky, and talking about Tusky, and not even know or care their server was running your code? If Tusky had account creation that'd be pretty easy. Or if someone made another server speaking the Mastodon client-server API, and called it tusky-server, and it got really popular, mostly just used with Tusky? What harm would that do?
Technically I don't have much against a world like this, but this would make educating the masses even harder in my opinion. Instead of talking about "switch to Mastodon", we'd have all this potentially half-finished apps trying to get people's attention, and among the noise the fact that they're all actually using ActivityPub in the background might not get noticed and some apps could slip by that don't use it at all.
I'd also like to note there is no precedent for anything like that. While both XMPP and IMAP/POP3/SMTP have defined client-to-server APIs, their functionality is so narrowly focused that there is significant difference between using one e-mail app over another. ActivityPub allows you to do a lot of things through its vocabulary.
Doesn't S2S federation suffer from all these interop problems, just like C2S?
No, or maybe to some extent, but we can solve these problems at the server level. Much better than having app developers have to individually solve it in each app.
How do you handle federating to a server that's images-only? Or has a 140char limit? Or which doesn't do first-class replies? Or boosts?
My impression has been, instead, that we need to converge on a complete and precise model, and let systems compete on UI and policies and performance and price. I think a world where the model is different in different parts of the federation is never going to go mainstream. But... hopefully I'll be proven wrong.
How do you handle federating to a server that's images-only? Or has a 140char limit? Or which doesn't do first-class replies? Or boosts?
This reminds me - @Gargron how does Mastodon handle or plan to handle if an incoming Note(?) has over 500 characters? It would be tricky for senders to know to shorten notes since they wont know if the recipient is a Mastodon server or some other AP server.
In general, to the thread: Interesting reading, but tbh, I don't think there will be a "one use case fits all". And actually IMHO that would be the worst thing that could happen. We have fantastic apps and platforms because people want different things. Some people want short video clips, some auto-destructing messages, some only text, some hashtags, etc. People have different needs and choose platforms according to their needs. I agree totally that S2S is the only way to bridge, partially, these platforms which have different features.
ActivityPub allows you to do a lot of things through its vocabulary.
Exactly, this as well. I can't see any platforms going to support everything it can represent.
@jaywink "how does Mastodon handle or plan to handle if an incoming Note(?) has over 500 characters" this isn't a new problem. I've seen a few solutions. the one I've implemented back in the day was you would truncate them as you receive them and link to the originating server with a "..." or "read more" or whatever link in your rendered version.
@wilkie yes though this was a very specific question regarding Mastodon :) I had a quick look in the code but couldn't find any place - but my RoR isn't that good.
I'd like to share a thought on this that I've been thinking since before I started work on an ActivityPub side project (and I hope that it's not too off-topic). And that is "a protocol is not an application". It seems to me that historically there has been a tendency for open social developers to think a lot about protocols and not much about applications, assuming that the protocol is, in some sense, the application.
The ActivityPub spec provides the primitives for a lot of different types of applications; a Facebook-like app is not a Twitter-like app is not an Instagram-like app, but you can express any of them with ActivityStreams objects. And a common c2s API is a convenience for anyone implementing a client -- but it does not make much sense for there to be a generic ActivityPub client app.
Yeah, same here.
@nightpool I really doubt that they would remove c2s from the standard from this feedback, so that's quite pointless. Mastodon, on the other hand, does not implement c2s and I see no reason to spend time on that right now.
I don't think c2s should be removed from the standard; I think it's a great convenience for client app developers, especially as client libraries mature. I just think an ActivityPub client that is not designed around the "business logic" of a particular ActivityPub application would not actually be useful for anything but testing.
@sandhawke Re: "And then what if they upped the character limit...?"
Mastodon.host is an example of an instance whose limit is 1024 characters per toot. When I post from there with my https://mastodon.host/@FuzboleroXV account, all (?) other Mastodon instances that are using the default 500 limit are still showing the full posts (all 1024 characters). They just do not allow their own users to write that long toots. Within the Mastodon realm, this works well, and there is no real "issue" with a "fixed" character limit.
It is not and does not need to be an enforced standard. Whenever such longer posts are sent to other (non-Mastodon) systems, they already need to cope with the the fact that they will continue to have to handle different lenghts from different sources/systems (which for example can be done like @wilkie is describing with the "read more" example).