HTTP 401 from tasks to Cloud Run backend after 3.2.0 upgrade
Tarpsvo opened this issue · 6 comments
Heya!
When upgrading from 3.2.0 to 3.5.2 we're receiving HTTP 401 against our Cloud Run backend. All the permissions are correct and even beyond the expected limits. 3.2.0 works perfectly with the permissions provided.
It seems like it started using a wrong service account and an invalid access token.
We're experimenting a bit with the new ->setAudience() feature to see if that is the cause. I will let you know.
From the documentation, the audience should not be a hash. It should be an URL.
I have also experienced this problem with 2 separate projects in different cloud instances and had to hard-lock the package to 3.2.1 on both of them.
@marickvantuil Heya.
I identified the cause of our issue.
If I use ->getHandler() as the audience, everything works as expected. However, hashing the audience causes the requests to receive HTTP 401.
I would propose making this like optional, behind a config flag:
$token->setAudience(hash_hmac('sha256', $this->getHandler(), config('app.key')));
Granted, we also have to check this when validating the token.
@Tarpsvo Thanks a lot for the PR!! 👍
Perhaps not desirable, but currently the 'audience' is used as a security key (hence the hash_hmac) and upon receiving the task payload, the application will check if the 'audience' matches the application key again. Removing the setAudience will make it possible for anyone to send tasks to the application.
I've never gotten these 401 errors but is it possible you are running a setup in which the application sending the task to the queue is not necessarily the application that handles it? If so, that is what also happened in #101 and I haven't been able to fix that yet - partially because it means we have to change the way tokens are validated.
From the documentation, the audience should not be a hash. It should be an URL.
Good point, so this hash has to change either way. But currently it's used to guard against malicious requests so I can't just change it into the URL.
Heya!
The way we ensure security is by using Google Cloud's native authentication. Only Cloud Project's internal IAM users are allowed to connect to the Cloud Run setup and the instance handling the tasks is not publicly accessable.
We're currently running our own branch which works perfectly in our setup and it does indeed match exactly what #101 has shown (I probably couldn't relate my issue to the title, sorry about that).
Anyhow, would you be open to allowing the audience signing to be behind a configurable option, as shown in the PR? It could definitely be true by default, if you think that's best. I would just love to have an option to disable it for our usecase (and for #101).
On a further in-depth note - it's actually not the Cloud Run instance that is rejecting the audience key. The IAM access policy we've set up actually stops the request in its tracks when Cloud Tasks tries to POST to the route when the task has an audience set. So the /handle-task request actually never reaches the Cloud Run instance. It could be an issue within Google Cloud itself, as from what we've seen, it starts using an unknown access token (maybe it is the hmac?) instead of the actual access token of our IAM user.
Just released 3.6.0 with this change!
The way we ensure security is by using Google Cloud's native authentication. Only Cloud Project's internal IAM users are allowed to connect to the Cloud Run setup and the instance handling the tasks is not publicly accessable.
On a further in-depth note - it's actually not the Cloud Run instance that is rejecting the audience key. The IAM access policy we've set up actually stops the request in its tracks when Cloud Tasks tries to POST to the route when the task has an audience set. So the /handle-task request actually never reaches the Cloud Run instance. It could be an issue within Google Cloud itself, as from what we've seen, it starts using an unknown access token (maybe it is the hmac?) instead of the actual access token of our IAM user.
Thanks, TIL, this makes a lot more sense! I did change it so the signing is still turned on by default because last year there was a user who reported receiving malicious requests and I feel like more users have a set up running where disabling the signing leaves the app right open to malicious requests?
Also thanks for the other bugfix! Can you let me know if this issue can be closed?
Hey! Thanks a ton! Everything works as intended for me now. :)