w3c/prbuildbot

Post a single comment for all browsers

Opened this issue · 3 comments

Similar to #3.

Ideally we should post a single comment with a single table per file which has columns for each different UA we're running the stability checker on. Once we add Edge and Safari having comments per UA is going to get even worse!

I can get behind this. It would simplify the find/replace logic that makes #3 work (only allow the stability-bot user 1 comment, no matter the content), and remove the multiple POST/PATCH requests.

So there are a couple of reasons not to do this, as discussed on irc. The main one is that there is a huge complexity cost; we would either need to reparse the comments when updating results, or store all the data on the server ~indefinitely (at least until the PR is merged). Adding state to something that could be stateless is a big burden. The other problem is that github has a maximum comment size and some of our tables get pretty big; dumping the results of multiple browsers in a single comment could lead to unnecessary truncation.

I think the win here is disproportionately small compared to the benefit.

After our IRC discussion, I agree.