What is the source of this inconsistency?
Closed this issue · 10 comments
For 56.0.1:
When I change the version to 56.0.build6:
Questions:
- why is the balrog shipping a different version?
- When the balrog check works, the archive doesn't exist. How come?
- This seems confusing for a human with no context. Can we explain why the inconsistency happens or address it with some UX?
- Why does the channel switch to "Beta" for the second image?
For the 56.0.1 part, it looks like it may be pulling that from fallbackMapping (which is probably not right, since that's not the "current" shipping Release).
For the Beta, I have no idea what's going on...the Balrog links points at https://aus-api.mozilla.org/api/v1/rules/firefox-beta, which has no references to 56.0build6 in its current state.
Err, I misread the 56.0.1 Balrog rules...the primary mapping is set to 56.0. The reason for this is because we have a special watershed release (56.0.1) that does 32 -> 64-bit windows upgrades for some users. This is expected.
These sorts of things will happen from time to time -- it seems like Delivery Dashboard is doing the right thing here in making it a warning -- the humans that read Delivery Dashboard should know enough to understand that this is OK.
Hmm... I suppose I don't actually know how Balrog works.
Looking at the source of the data:
- How does that map to what's served via https://aus5.mozilla.org?
- What would be a valid AUS5.mozilla.org URL that a real firefox would check?
@mozbhearsum
The balrog API returns Firefox-56.0-build6. This fails the PollBot check because the version is lower.
How does somebody looking at the Dashboard know this is an acceptable/ can ignore warning?
Would they have to have some other info about the current release via a meeting/document/somewhere?
@mostlygeek it is not a bug, it is a patch release for esr and release only for mac, and because there are no changes for other releases they don't update other platforms to not risk an update failure.
56.0build6 is not a valid version number for pollbot did you mean 56.0 ?
After talking with Release Management and specifically with @jcristau and @sylvestre, they told me it was fine to show this inconsistency and that they were able to know to enter the 56.0 version number themselves when this things happen.
@mostlygeek - The dashboard isn't looking at the full Rules, it's shortcutting to the "primary" Firefox release channel rule -- the one whose mapping almost always points at the latest shipping Firefox release. We happen to be in an exceptional situation where it doesn't, currently. Instead, there's a higher priority Rule (https://aus-api.mozilla.org/api/v1/rules/659) that points at 56.0.1.
Another way to go about answering questions like "has release X shipped" could be to query through Releases (eg: https://aus-api.mozilla.org/api/v1/releases?name_prefix=Firefox-56.0-build6), and then look up the Rules to see if the Release is mapped to by a Rule on particular channel (eg: release).
I suppose answering the "has release X shipped" may be too complex to squash down into a yes || no+reason response from PollBot.
they told me it was fine to show this inconsistency and that they were able to know to enter the 56.0 version number themselves when this things happen.
I was also curious what the actual user's of the dashboard would do. If they're happy I'm happy.