RobustPerception/azure_metrics_exporter

Expose remaining rate limit

Opened this issue · 5 comments

carlpett commented

The header x-ms-ratelimit-remaining-subscription-reads contains the remaining allowed read request quota, and would be use very useful. See https://docs.microsoft.com/en-us/azure/azure-resource-manager/resource-manager-request-limits.

I'm happy to implement this, but would like some feedback on this approach:

  • In AzureClient.getMetricValue read and parse the header if present
  • Return the value as a third value (this is the major question mark, could also bundle it into AzureMetricValueResponse)
  • In Collector.Collect, check the min value returned from all get:s, and finally create a metric from this min.

Sounds reasonable?

There is a write quota header too, but it is only returned on writes, so to collect it we'd need to make some form of write and the service principal would need to be allowed to do so. I'd consider this out of scope initially. Main concern for us at least is reads.

brian-brazil commented

Another option would be storing the most recent value inside AzureClient.

This sounds useful.

carlpett commented

Yes, that is an even better idea!
When going at this, should we additionally start respecting the Retry-After if we get a HTTP 429? And/or even proactively failing scrapes if the rate limit is getting close to depleted? That might be hard to do well, though, and would probably require a lot of flags for user tweakability.

brian-brazil commented

Let's get the data in first.

lukasmrtvy commented

@carlpett any progress on this? Or is it passé for You?

carlpett commented

@lukasmrtvy Sorry, I left the client where we used Azure before it could be finished. Hopefully someone with an Azure account could pick up the PR.