mozilla-releng/balrog

provide an easier way to point nightly at an earlier build

Opened this issue · 3 comments

Today when release management determine a nightly build is "bad", they stop updates (with the emergency_shutoff hammer), get new/fixed nightlies build, then re-enable updates once that's done.
It would be preferable if, instead of stopping all updates, the Firefox-mozilla-central-nightly-latest blob could be pointed at the previous "known good" build, and then the respin would just replace it as it becomes available.
Today that's too cumbersome though (have to find the previous buildid, copy the corresponding blob, add necessary platform aliases, change the name, and upload that as -latest), so it'd be nice if we had some way to do that in the UI.

@jcristau What you describe, seems like a fallbackMapping to me, but in a "emergency level". The Rule will not be evaluated and, even so, a fallbackMapping will be present.
Makes sense ?

I've been trying to understand the issue here though. @bhearsum @gabrielBusta Please help me out here :

So, in my understanding, the emergency shutoff hammer pauses updates for the entire time the nightlies build is being fixed and so therefore users (of Firefox) wouldn't be served any update during this time if they tried to update their browsers.

So @jcristau is suggesting that rather than shutting updates down, we find a way to serve a previous working version of the nightly build whilst the current one is being worked on and this should be done with a button in the UI ?

Please can more light be shed on this?

I might suggest that we don't need code changes to make this better. The original idea behind emergency shutoff was that it was a panic button that a single person can hit to stem the damage from something. After the button is hit, there's no reason we can't update the rules to point at the last known good build, and then re-enable updates.

If we end up doing anything around this, I think the main complication here is how to determine what the "last known good build" is. The obvious thing to do is to find the immediately previous buildid to the one currently in the latest release. That would probably hold true most of the time...but we'd definitely want some way to override it (either as part of the shutoff procedure or a way to update it after the shutoff has happened).