Chain of Trust errors on signing tasks

Question

Chain of Trust errors on signing tasks

ahal opened this issue 2 years ago · 10 comments

RyanVM noticed some chain of trust errors:
https://firefox-ci-tc.services.mozilla.com/tasks/groups/CrIshxNpQPGHuAMsnsOjEA

This is due to a key rotation we did last week and the fact that the cached docker-image task was run on a worker that had the old key. To fix it, we'll need to cause a rebuild.

Unfortunately glean is using a non-standard index route which is causing the add-new-jobs action to fail, and it is also using an older version of Taskgraph that is missing the rebuild-cached-tasks action. Both of these things should be simple to fix.

Answer 1 · 2023-02-27T10:56:34.000Z

@ahal is there anything needed from us, here?

Answer 2 · 2023-02-27T14:57:04.000Z

@Dexterp37 Hi, yes.. It looks like there are some gradle failures that are likely caused by my PR:
https://firefox-ci-tc.services.mozilla.com/tasks/EV1q2hJdSa62PBlo46kbyQ/runs/0/logs/live/public/logs/live.log

I think by using the newer decision task image, it caused gradle to upgrade and fail due to incompatibilities. Unfortunately this newer image is needed to upgrade Taskgraph. Do you think these would be easy to resolve (I'm not familiar with gradle)?

@JohanLorenzo have you encountered gradle errors like this while updating any of the mobile repos to the newer Decision image?

Answer 3 · 2023-02-27T14:57:53.000Z

Also if we just need the signing tasks fixed ASAP, it should be sufficient to land the first commit in that PR which doesn't touch the Decision task image.

Answer 4 · 2023-02-27T15:27:56.000Z

Hm, I think my earlier diagnosis was wrong.

Either that failure is simply an intermittent that is unrelated to my PR, or re-running the build-docker-image-linux cached task somehow caused a newer version of gradle to be used (though afaict, that image doesn't install gradle at all).

Answer 5 · 2023-02-27T15:45:45.000Z

Ok, looks like it was just an unrelated intermittent after all, went green on a re-run. I'll still need help getting the PR reviewed and landed though, as I don't have collaborator permissions here.

Answer 6 · 2023-02-27T19:00:48.000Z

@travis79 You'll need to respin the 52.3.0 release (or create a new 52.3.1 one) for this issue too.

Answer 7 · 2023-02-27T19:04:27.000Z

But not quite yet, I'm fixing the scope error that got hit on the push to main.

Answer 8 · 2023-02-27T20:41:16.000Z

Scopes should be fixed now.

Answer 9 · 2023-02-27T20:45:28.000Z

There was a crash report related to 52.3.0 from Firefox iOS. I'm going to try and get a fix for that and then I'll cut a 52.3.1

Answer 10 · 2023-02-27T21:10:35.000Z

Sounds good, we'll need the build-docker-image-linux task to run on the main branch before the chain of trust error is fixed. But sounds like your crash fix will take care of that. If it doesn't run, please ping me and I'll trigger it manually.