Labbs/github-actions-exporter

Exporter is not using exponential backoff/maxRetries on GitHub API-Calls

Opened this issue · 0 comments

As the github api exerts a rate limit, and the exporter does not respect that which can cause delays in metric delivery.

Expected Behavior

The Exporter should respect a configurable ratelimit to not exhaust all requests at once.

Current Behavior

On an recurring error, such as wrongly configured creds (wrong permissions) the exporter just depletes all of its requests in a matter of seconds.
For what ever reason this infinite loop does not stop when it hits an error
https://github.com/Spendesk/github-actions-exporter/blob/ec79f05a21c132a0c27f546c4955a159a3721baa/pkg/metrics/get_runners_organization_from_github.go#L29-L46

Possible Solution

Add a ratelimiting RoundTripper to the github client, like this one https://gist.github.com/zdebra/10f0e284c4672e99f0cb767298f20c11
This would also eleviate the need for the caching layer

Steps to Reproduce

  1. Create a GH token with wrong permissons (or anything that reliably produces an error with the gh client)
  2. See it flood the GH API

Context (Environment)

Just me being stoopid and not configureing a proper token, but this could have happend on any error with the github client.
Lets say you alert based on metrics provided my this exporter, any error that would cause the rate limit to be reached would cause alerts.