Webhook for Bazel Buildkite pipeline
GMNGeoffrey opened this issue · 10 comments
As part of adding an unsupported Bazel configuration, I've set up a Buildkite pipeline that runs the Bazel build on the main branch. Currently it's configured to run every 10 minutes (the shortest cron Buildkite allows), but it would be much nicer to have it instead trigger on pushes to main via webhook. Adding a webhook to a repository requires admin access. I don't think it has any real security implications for a public repository. Is this something someone could help me add?
Note that since this is an unsupported build, I'm not posting statuses on the GitHub commits.
Side note: I wish GitHub had a pubsub model so anyone could subscribe to updates to a public repository. Would also make automatically syncing forks much easier
How are the existing buildkite jobs configured?
I didn't set any of those up, but just poking around the settings:
The one building the release branch https://buildkite.com/llvm-project/llvm-releases appears to have webhooks set up, which you can see by looking attaching "/webhook" to a build: https://buildkite.com/llvm-project/llvm-releases/builds/1104/webhook.
The one that runs on the main branch appears to be on a 4 hour cron (https://buildkite.com/llvm-project/llvm-main/settings/schedules). The ones doing pre-merge checks are pointed at the https://buildkite.com/llvm-project/premerge-checks/settings/repository repository and doing some kind of triggering from phabricator that I know nothing about (but probably https://buildkite.com/docs/integrations/phabricator)
The native Buildkite GitHub integration has one webhook URL per pipeline though, so this would be a separate webhook.
It seems strange that you would need a new webhook for each pipeline, is there any way to configure it so the entire buildkite instance shares the same webhook?
I think that's a result of the way the Buildkite models things. Each pipeline can potentially be for a different repository (or even different source control system) and so the integrations are per-pipeline, not per-organization. Buildkite has the ability for one pipeline to trigger another: https://buildkite.com/docs/pipelines/trigger-step, but if I'm reading the docs right this would require that the author of a commit themselves have permissions to trigger pipelines, so having a single pipeline that redistributes webhooks wouldn't work. I can talk to support to see if there's another way to have one webhook trigger multiple pipelines, but I'm pretty sure there's not a configuration setting or anything like that (because we have quite a few Buildkite pipelines in my project and I was annoyed about having to add the webhook each time). From the documentation:
If you want to use the same repository in multiple pipelines (including pipelines in different Buildkite organizations), you need to configure a separate webhook for each pipeline.
@GMNGeoffrey Ok, what's the webhook URL? If you have the ability to customize the URL it would be great if you could make it self-documenting: so that it has bazel in its name.
The setup instructions including the payload are at https://buildkite.com/llvm-project/upstream-bazel-rbe/settings/setup/github. You should be able to see it since you're a member of the Buildkite org. I'm guessing there's a reason Buildkite hides it behind auth, so not posting it directly here.
Unfortunately, there doesn't seem to be any customization available. I'll email Buildkite support with this set of questions and see if we can work out something nicer longer term. I agree that one UUID webhook URL per pipeline is pretty annoying...
@GMNGeoffrey I've added the webhook. Let me know if it is working.
It worked: https://buildkite.com/llvm-project/upstream-bazel-rbe/builds/977/webhook. Thanks!
I'm still in conversations with support about how to have pipelines trigger each other, but one option that might reduce the manual maintenance here would be the one for auto-creating webhooks that I just discovered:
https://buildkite.com/changelog/108-webhooks-can-be-auto-created-for-github-repositories