[Feature Request] Merge queue compatibility

[btrautmann]

Hi there!

Our team is investigating turning on merge queue for our repository and in doing so, ran into an issue (expectedly) with PolicyBot. Merge queue does not operate on PRs but rather allows you to add PRs to a queue once all their required checks are green, after which it creates specialized branches with the naming structure gh-readonly-queue/main/pr-number-<hash>. PolicyBot never "runs" because that branch isn't associated with a pull request. Even if it did try to run, there'd be no way of adding approvals.

So, a general question: Is there any existing functionality that might make this possible? If not, some ideas in no particular order:

1. Since merge queue branches have a special trigger event within Github Actions workflows (merge_group), we could create a workflow that runs Policy Bot whenever a merge_group checks requested event occurs. If the binary allows for us to invoke commands which add approvals, we could automatically provide necessary approvals because we know the code changes have already been approved (as that's a requirement before adding a PR to the queue).
2. If the above won't work, could we somehow add functionality to PolicyBot that would allow it to consider a branch approved to merge based on the approval status of an open PR? In other words, if PR 9243 has been approved and added to the queue and a branch gh-readonly-queue/main/9243* has been created, we could consider that branch "approved to merge". This may be considered safe because you may not push to either branch (the original one or the queue one) while the PR is in the queue. You must remove the PR from the queue, push changes (therefore revoking PolicyBot approvals), and then get approvals and add to the queue again.

Note: These ideas are based on the expectation that we can provide a status check update for a specific commit on a branch (rather than a PR) like Github Actions do!

I'll add more ideas in comments as they come to me, but was hoping to get the discussion started in the meantime. Thank you!

[bluekeyes]

Thanks for the initial thoughts on how to support this in Policy Bot! I don't think there's any existing functionality for this because, as you noted, Policy Bot currently only evaluates pull requests, not branches.

Given some initial research on the merge queue feature, I think the best way to support this is to have Policy Bot listen for the merge_group webhook event. I think that handler will do something like:

1. Check for a policy file on the base_ref from the webhook payload
2. If a policy exists, unconditionally post a success status on the head_sha

I don't think we actually need to evaluate the policy, if GitHub is the only thing that can generate merge_group events and push to the special merge queue branches. I think we can assume that if Policy Bot is a required check, GitHub verified it was passing before adding the PR to the queue, and so anything that exists in the queue must be approved.

[btrautmann]

I don't think we actually need to evaluate the policy, if GitHub is the only thing that can generate merge_group events and push to the special merge queue branches. I think we can assume that if Policy Bot is a required check, GitHub verified it was passing before adding the PR to the queue, and so anything that exists in the queue must be approved.

This is my understanding as well, and was sort of what I was hoping for because if it's supported first party by Policy Bot, consumers have very little (nothing?) to do (beyond maybe some additional optional configuration, like disabling this functionality if desired) to adopt merge queue 😄

[bluekeyes]

To set expectations, I don't think we will implement this until the merge queue feature is available in a GitHub Enterprise Server release and we can test in our primary environment. That said, if you or anyone else would like to submit a PR implementing the merge_group handler before then, I'm happy to review it or provide advice.