Exporter is not using exponential backoff/maxRetries on GitHub API-Calls
Opened this issue · 0 comments
As the github api exerts a rate limit, and the exporter does not respect that which can cause delays in metric delivery.
Expected Behavior
The Exporter should respect a configurable ratelimit to not exhaust all requests at once.
Current Behavior
On an recurring error, such as wrongly configured creds (wrong permissions) the exporter just depletes all of its requests in a matter of seconds.
For what ever reason this infinite loop does not stop when it hits an error
https://github.com/Spendesk/github-actions-exporter/blob/ec79f05a21c132a0c27f546c4955a159a3721baa/pkg/metrics/get_runners_organization_from_github.go#L29-L46
Possible Solution
Add a ratelimiting RoundTripper to the github client, like this one https://gist.github.com/zdebra/10f0e284c4672e99f0cb767298f20c11
This would also eleviate the need for the caching layer
Steps to Reproduce
- Create a GH token with wrong permissons (or anything that reliably produces an error with the gh client)
- See it flood the GH API
Context (Environment)
Just me being stoopid and not configureing a proper token, but this could have happend on any error with the github client.
Lets say you alert based on metrics provided my this exporter, any error that would cause the rate limit to be reached would cause alerts.