Failed builds are displayed as neutral
markuskowa opened this issue · 6 comments
I noticed that a failed darwin build shows up as neutral in the new CI tab.
See https://github.com/NixOS/nixpkgs/pull/51467/checks?check_run_id=37204539 for example.
Is this intended? Is there a possibility to flags this in a way it sticks out more, such marking it in red or giving it a label like "fails-on-<arch>"?
Well, red is bad because we want red to be «definitely do not merge»…
@7c6f434c So a build failure on darwin is then not a merge blocker? What about at least labeling the PR then? Eventually those failures should be fixed somehow, either by really fixing it or by removing the derivation from the respective platform.
I tihnk there are cases where x86_64-linux build failure on ofBorg is not a merge blocker… I mean, builds are complicated and being sure whether the failure is real is hard, and the failure can be a knock-on effect of some other thing which will be fixed independently and even on Hydra there are sometimes failed builds for unclear reasons (even without counting clear but weird and Hydra-specific reasons!)
Just look at ZHF issues (which seem to have become the venue to agree on mass-marking broken packages as broken)
Labelling is probably a good idea, althoug handling of an additional push to the PR branch is something to think about.
On the other hand, a platform being disabled is also displayed as neutral; maybe this should be a success? If there is a genuine evaluation problem, we will get a red cross from another check…
I can see your point. My question is then: can we mark ofborg failures a bit more clearly in terms of visibility? In the old ofborg version such failures would pop up very clearly as comment. If we mark such build failures a bit "bigger", maybe that urges people to look more into it, resulting in a little less failures?
I guess labels could work. Not sure if commenting if there is a build (not disabled-platform, not timeout) failure is a good idea.