Let's stop treating URLs as secrets: the removal of hub.verify_token in prior draft 0.3

[jcrites]

A precursor protocol to WebSub had a hub.verify_token that a subscriber would pass at subscription time, and that the hub would pass back at challenge time.

This was later removed in draft 0.4 and was opposed by a number of participations, but – this my summary of my email thread that I read discussing the change on the Google Group – Julien Genestoux persisted in his desire to remove it due to the desire to simplify the protocol.

I believe the removal of this field was a mistake. The lack of a positive confirmation that a subscription was requested, which the hub provides when it passes hub.verify_token back in the challenge request, means that subscription URLs must essentially be treated as secrets, or else a URL that is designed to respond to all changes is something that an attacker could manipulate.

The rationale at the time seemed to be that the attacker won't know the URL and that DDOSes like this wouldn't be a problem.

Similarly, the protocol treats URLs as secrets by making hub.secret optional in the Authenticated Content Distribution process. An attacker who discovers a URL that does not authenticate calls may perform all manner of attacks, depending on the exact meaning of the events flowing through the subscription. The attacker can post their own content to the URL which the subscriber may be unable to differentiate from legitimate content.

It's 2021 now. HTTP is dead and all serious systems use HTTPS, which requires TLS certificates. Certificate Transparency is an Internet Standard, and browsers are beginning to expect that certificates are present in a Certificate Transparency log in order for websites to work; as a consequence, many CAs publish the certificate into CT logs when issuing them.

CT logs provide a source of URLs that attackers can mine to look for WebSub subscription URLs that accept publications without authenticating them. Realistically, many more developers are building systems on the Internet than fully understand the nuances of Internet & Web security. I'd hardly say that I do.

For that reason, in our modern environment where were can expect attackers to find any WebSub subscription URL that does not authenticate calls to, a revision to the WebSub standard would do well to make the following changes:

1. Restore the hub.verification_token to enable subscription URLs to authenticate that a challenge attempt is from a source that it expects. Better yet, provide hub.verification_secret which is a key used to compute an HMAC of the challenge that's passed as a header, so that the secret doesn't pass over the wire twice. (An attacker that poisons DNS and intercepts inbound calls still does not learn hub.verification_secret, for example.)
2. Make hub.secret mandatory. All WebSub-compliant subscribers MUST authenticate that calls to them are authenticated using an HMAC from hub.secret
3. Ideally, allow either the hub or the subscriber to provide the secret, according to the policy of the hub – such as in response to the subscribe request. In the modern environment subscribers might be low power IoT devices that don't have access to high quality entropy or can't generate a high quality keys. Allowing the hub to supply hub.secret and provide it in response to the subscription request solves that issue.
4. During the challenge step, provide the X-Hub-Signature HTTP header so that subscribers can simplify their request validation and rejection logic, by simply dropping all requests that don't have a valid X-Hub-Signature.

In my use-case we are requiring hub.verification_token since it seems irresponsible to allow subscription challenges that are not a mutual handshake.