Brigade Github Gateway

This is a work-in-progress Brigade 2 compatible gateway that can be used to receive events (webhooks) from one or more GitHub Apps and propagate them into Brigade 2's event bus.

Installation

The installation for this gateway is multi-part, and not particularly easy, at least in part because of a potential "chicken and egg" problem. Setting up this gateway requires a value obtained during the creation of a GitHub App. Setting up the GitHub App may require the gateway's public IP (if you're not using a domain or subdomain name instead). We will use an approach of setting up the GitHub App first, with a placeholder value for the gateway's address, if necessary, in which case the GitHub App configuration will be revisited after the gateway is configured and deployed.

Prerequisites:

A GitHub account
A Kubernetes cluster:
- For which you have the admin cluster role
- That is already running Brigade 2
- Capable of provisioning a public IP address for a service of type LoadBalancer. (This means you won't have much luck running the gateway locally in the likes of kind or minikube unless you're able and willing to mess with port forwarding settings on your router, which we won't be covering here.)
kubectl, helm (commands below assume Helm 3.7.0+), and brig (the Brigade 2 CLI)

If you want to avoid the aforementioned "chicken and egg" problem, reserving a domain or subdomain name (for which you control DNS) also helps. If you don't want to do this or are unable, we'll cover that scenario as well.

1. Create a GitHub App

A GitHub App is a special kind of trusted entity that is "installable" into GitHub repositories to enable integrations.

This gateway can support multiple GitHub Apps, but these instructions walk you through the steps for setting up just one.

Visit https://github.com/settings/apps/new.
Choose a globally unique GitHub App name.
- When you submit the form, you'll be informed if the name you selected is unavailable.
Set the Homepage URL to https://github.com/brigadecore/brigade-github-gateway. Really, any URL will work, but this is the URL to which users will be directed if they wish to know more information about the App, so something informative is best. The URL above links to what is presently the best source of information about this gateway.
Set the Webhook URL to https://<your gateway domain or subdomain name>/events.
- If you're not using a domain or subdomain name and want to use a public IP here instead, put a placeholder such as http://example.com/events here for now and revisit this section later after a public IP has been established for your gateway.
Set the Webhook Secret to a complex string. It is, fundamentally, a password, so make it strong. If you're personally in the habit of using a password manager and it can generate strong passwords for you, consider using that. Make a note of this shared secret. You will be using this value again in another step.
Under the Subscribe to events section, select any events you wish to propagate to Brigade.
- Note: Selecting additional permissions in the Repository permissions section adds additional options to the menu of subscribable events.
- For the examples that follow, you would require the Watching permission and a subscription to the Watch event.
Unless you want anyone on GitHub to be able to send events to your gateway, toward the bottom of the page, select Only This account to constrain your GitHub App to being installed only by repositories in your own account or organization. If you select Any account instead, be sure you know what you're doing!
Submit the form.

After submitting the form, find the App ID field and take note. You will be using this value again in another step.

Under the Private keys section of this page, click Generate a private key. After generating, immediately download the new key. It is your only opportunity to do so, as GitHub will only save the public half of the key. You will be using this key in another step.

2. Create a Service Account for the Gateway

Note: To proceed beyond this point, you'll need to be logged into Brigade 2 as the "root" user (not recommended) or (preferably) as a user with the ADMIN role. Further discussion of this is beyond the scope of this documentation. Please refer to Brigade's own documentation.

Using Brigade 2's brig CLI, create a service account for the gateway to use:

$ brig service-account create \
    --id brigade-github-gateway \
    --description brigade-github-gateway

Make note of the token returned. This value will be used in another step. It is your only opportunity to access this value, as Brigade does not save it.

Authorize this service account to read all events and to create new ones:

$ brig role grant READER \
    --service-account brigade-github-gateway

$ brig role grant EVENT_CREATOR \
    --service-account brigade-github-gateway \
    --source brigade.sh/github

Note: The --source brigade.sh/github option specifies that this service account can be used only to create events having a value of brigade.sh/github in the event's source field. This is a security measure that prevents the gateway from using this token for impersonating other gateways.

3. Install the GitHub Gateway

For now, we're using the GitHub Container Registry (which is an OCI registry) to host our Helm chart. Helm 3.7 has experimental support for OCI registries. In the event that the Helm 3.7 dependency proves troublesome for Brigade users, or in the event that this experimental feature goes away, or isn't working like we'd hope, we will revisit this choice before going GA.

First, be sure you are using Helm 3.7.0 or greater and enable experimental OCI support:

$ export HELM_EXPERIMENTAL_OCI=1

As this chart requires custom configuration as described above to function properly, we'll need to create a chart values file with said config.

Use the following command to extract the full set of configuration options into a file you can modify:

$ helm inspect values oci://ghcr.io/brigadecore/brigade-github-gateway \
    --version v0.3.0 > ~/brigade-github-gateway-values.yaml

Edit ~/brigade-github-gateway-values.yaml, making the following changes:

brigade.apiAddress: Address of the Brigade API server, beginning with https://
brigade.apiToken: Service account token from step 2
github.apps: Specify the details of your GitHub App(s), including:
- appID: App ID from step 1
- apiKey: The private key downloaded in step 1, beginning with -----BEGIN RSA PRIVATE KEY----- and ending with -----END RSA PRIVATE KEY-----. All line breaks should be preserved and the beginning of each line should be indented exactly four spaces.
- sharedSecret: Shared secret from step 1
receiver.host: Set this to the host name where you'd like the gateway to be accessible.

Save your changes to ~/brigade-github-gateway-values.yaml and use the following command to install the gateway using the above customizations:

$ helm install brigade-github-gateway \
    oci://ghcr.io/brigadecore/brigade-github-gateway \
    --version v0.3.0 \
    --create-namespace \
    --namespace brigade-github-gateway \
    --values ~/brigade-github-gateway-values.yaml

4. (RECOMMENDED) Create a DNS Entry

In the prerequisites section, we suggested that you reserve a domain or subdomain name as the address of your gateway. At this point, you should be able to associate that name with the gateway's public IP address.

If you installed the gateway without enabling support for an ingress controller, this command should help you find the gateway's public IP address:

$ kubectl get svc brigade-github-gateway-receiver \
    --namespace brigade-github-gateway \
    --output jsonpath='{.status.loadBalancer.ingress[0].ip}'

If you overrode defaults and enabled support for an ingress controller, you probably know what you're doing well enough to track down the correct IP without our help. 😉

With this public IP in hand, edit your name servers and add an A record pointing your domain to the public IP.

Note: If you do not want to use a domain or subdomain name, or are unable to, and elected to use a placeholder URL when initially setting up your GitHub App, return to GitHub (your App can be found on GitHub using a URL of the form https://github.com/settings/apps/<app name>) and edit your App's configuration to send webhooks (events) to https://<public ip>/events.

5. Confirm Connectivity

Your App can be found on GitHub using a URL of the form https://github.com/settings/apps/<app name>.

Go to the Advanced tab and check out the Recent Deliveries section. Here you can view events that your GitHub App has recently attempted to deliver to your new gateway. There shouldn't be many events displayed yet, but there should be at least one ping event that the App attempted to deliver to the gateway when the App was created. This should have failed since we set up the App on GitHub's end prior to installing the gateway on our cluster. Click Redeliver. If re-delivery succeeds, you're all set!

If re-delivery failed, you can examine request and response headers and payload to attempt to make some determination as to what has gone wrong.

Some likely problems include:

Your A record in DNS is incorrect.
DNS changes have not propagated.
Your gateway is not listening on a public IP.
The Webhook URL you entered when configuring the GitHub App is incorrect.
The gateway was not configured correctly using the GitHub App's App ID and shared secret.

6. Install the App

Your App can be found on GitHub using a URL of the form https://github.com/settings/apps/<app name>.

Under the Install App tab you can see all accounts and organizations into whose repositories you can install your App. Click the gear icon next to the desired account or organization and, under Repository access choose All repositories OR Only select repositories then specify which ones, and click Save.

7. Add a Brigade Project

You can create any number of Brigade projects (or modify an existing one) to listen for events sent from your GitHub App to your gateway and, in turn, emitted into Brigade's event bus. You can subscribe to all event types emitted by the gateway, or just specific ones.

In the example project definition below, we subscribe to all events emitted by the gateway, provided they've originated from the example-org/example-repo repository (see the repo qualifier). You should adjust this value to match a repository into which you have installed your new GitHub App.

apiVersion: brigade.sh/v2-beta
kind: Project
metadata:
  id: github-demo
description: A project that demonstrates integration with GitHub
spec:
  eventSubscriptions:
  - source: brigade.sh/github
    types:
    - *
    qualifiers:
      repo: example-org/example-repo
  workerTemplate:
    defaultConfigFiles:
      brigade.js: |-
        const { events } = require("@brigadecore/brigadier");

        events.on("brigade.sh/github", "watch:started", () => {
          console.log("Someone starred the example-org/example-repo repository!");
        });

        events.process();

In the alternative example below, we subscribe only to watch:started events. (Note that, counterintuitively, this event occurs when someone stars a repository; not when they start watching it. This is a peculiarity of GitHub and not a peculiarity of this gateway.)

apiVersion: brigade.sh/v2-beta
kind: Project
metadata:
  id: github-demo
description: A project that demonstrates integration with GitHub
spec:
  eventSubscriptions:
  - source: brigade.sh/github
    types:
    - watch:started
    qualifiers:
      repo: example-org/example-repo
  workerTemplate:
    defaultConfigFiles:
      brigade.js: |-
        const { events } = require("@brigadecore/brigadier");

        events.on("brigade.sh/github", "watch:started", () => {
          console.log("Someone starred the example-org/example-repo repository!");
        });

        events.process();

Assuming this file were named project.yaml, you can create the project like so:

$ brig project create --file project.yaml

Adding a star to the repo into which you installed your new GitHub App should now send an event (webhook) from GitHub to your gateway. The gateway, in turn, will emit the event into Brigade's event bus. Brigade should initialize a worker (containerized event handler) for every project that has subscribed to the event, and the worker should execute the brigade.js script that was embedded in the project definition.

List the events for the github-demo project to confirm this:

$ brig event list --project github-demo

Full coverage of brig commands is beyond the scope of this documentation, but at this point, additional brig commands can be applied to monitor the event's status and view logs produced in the course of handling the event.

Events Received and Emitted by this Gateway

Events received by this gateway from GitHub are, in turn, emitted into Brigade's event bus.

Event of certain types received from GitHub are further qualified by the value of an action field. In all such cases, the event emitted into Brigade's event bus will have a type of the form <original event type>:<action>. For instance, if this gateway receives an event of type pull_request from GitHub with the value opened in the action field, the event emitted into Brigade's event bus will be of type pull_request:opened.

Events received from GitHub vary in scope of specificity. All events handled by this gateway are at least indicative of activity involving some specific repository in some way -- for instance, a GitHub user having starred or forked a repository. Some events, however, are more specific than this, being indicative of activity involving not only a specific repository, but also some specific branch, tag, or commit -- for instance, a new pull request has been opened or a new tag has been pushed. In such cases (and only in such cases), this gateway includes git reference or commit information in the event that is emitted into Brigade's event bus. By doing so, Brigade (which has built in git support; not GitHub support) is enabled to locate specific code affected by the event.

If the gateway is able to infer a human-friendly title for any event, the event emitted into Brigade's event bus is augmented with this information.

The following table summarizes all GitHub event types that can be received by this gateway and the corresponding event types that are emitted into Brigade's event bus.

GitHub Event Type	Scope	Possible Action Values	Event Type(s) Emitted
`check_run`	specific commit	`created` `completed` `rerequested` `rerequested_action`	`check_run:created` `check_run:completed` `check_run:rerequested` `check_run:rerequested_action`
`check_suite`	specific commit	`completed` `requested` `rerequested`	`check_suite:completed` `check_suite:requested` `check_suite:rerequested`
`create`	specific branch or tag		`create`
`deleted`	specific branch or tag		`deleted`
`fork`	specific repository		`fork`
`gollum`	specific repository		`gollum`
`installation`	multiple specific repositories; the gateway will split this into multiple repository-specific events	`created` `deleted` `suspend` `unsuspend` `new_permissions_accepted`	`installation:created` `installation:deleted` `installation:suspend` `installation:unsuspend` `installation:new_permissions_accepted`
`installation_repositories`	multiple specific repositories; the gateway will split this into multiple repository-specific events	`added` `removed`	`installation_repositories:added` `installation_repositories:removed`
`issue_comment`	specific repository	`created` `edited` `deleted`	`issue_comment:created` `issue_comment:edited` `issue_comment:deleted`
`issues`	specific repository	`opened` `edited` `deleted` `pinned` `unpinned` `closed` `reopened` `assigned` `labeled` `unlabeled` `locked` `unlocked` `transferred` `milestoned` `demilestoned`	`issues:opened` `issues:edited` `issues:deleted` `issues:pinned` `issues:unpinned` `issues:closed` `issues:reopened` `issues:assigned` `issues:labeled` `issues:unlabeled` `issues:locked` `issues:unlocked` `issues:transferred` `issues:milestoned` `issues:demilestoned`
`label`	specific repository	`created` `edited` `deleted`	`label:created` `label:edited` `label:deleted`
`member`	specific repository	`added` `removed` `edited`	`member:added` `member:removed` `member:edited`
`milestone`	specific repository	`created` `closed` `opened` `edited` `deleted`	`milestone:created` `milestone:closed` `milestone:opened` `milestone:edited` `milestone:deleted`
`page_build`	specific repository		`page_build`
`project_card`	specific repository	`created` `edited` `moved` `converted` `deleted`	`project_card:created` `project_card:edited` `project_card:moved` `project_card:converted` `project_card:deleted`
`project_column`	specific repository	`created` `edited` `moved` `deleted`	`project_column:created` `project_column:edited` `project_column:moved` `project_column:deleted`
`project`	specific repository	`created` `edited` `closed` `reopened` `deleted`	`project:created` `project:edited` `project:closed` `project:reopened` `project:deleted`
`public`	specific repository		`public`
`pull_request`	specific commit	`opened` `edited` `closed` `assigned` `unassigned` `review_requested` `review_request_removed` `ready_for_review` `converted_to_draft` `labeled` `unlabeled` `synchronize` `auto_merge_enabled` `auto_merge_disabled` `locked` `unlocked` `reopened`	`pull_request:opened` `pull_request:edited` `pull_request:closed` `pull_request:assigned` `pull_request:unassigned` `pull_request:review_requested` `pull_request:review_request_removed` `pull_request:ready_for_review` `pull_request:converted_to_draft` `pull_request:labeled` `pull_request:unlabeled` `pull_request:synchronize` `pull_request:auto_merge_enabled` `pull_request:auto_merge_disabled` `pull_request:locked` `pull_request:unlocked` `pull_request:reopened`
`pull_request_review`	specific commit	`submitted` `edited` `dismissed`	`pull_request_review:submitted` `pull_request_review:edited` `pull_request_review:dismissed`
`pull_request_review_comment`	specific commit	`created` `edited` `deleted`	`pull_request_review_comment:created` `pull_request_review_comment:edited` `pull_request_review_comment:deleted`
`push`	specific commit		`push`
`release`	specific repository	`published` `unpublished` `created` `edited` `deleted` `prereleased` `released`	`release:published` `release:unpublished` `release:created` `release:edited` `release:deleted` `release:prereleased` `release:released`
`repository`	specific repository	`created` `deleted` `archived` `unarchived` `anonymous_access_enabled` `edited` `renamed` `transferred` `publicized` `privatized`	`repository:created` `repository:deleted` `repository:archived` `repository:unarchived` `repository:anonymous_access_enabled` `repository:edited` `repository:renamed` `repository:transferred` `repository:publicized` `repository:privatized`
`status`	specific commit		`status`
`team_add`	specific repository		`team_add`
`watch`	specific repository	`started`	`watch:started`

Examples Projects

See examples/ for complete Brigade projects that demonstrate various scenarios.

Contributing

The Brigade project accepts contributions via GitHub pull requests. The Contributing document outlines the process to help get your contribution accepted.

Support & Feedback

We have a slack channel! Kubernetes/#brigade Feel free to join for any support questions or feedback, we are happy to help. To report an issue or to request a feature open an issue here

Code of Conduct

Participation in the Brigade project is governed by the CNCF Code of Conduct.

czunker/brigade-github-gateway