pulp/pulpcore

Add metrics support for downloaded file counts by type (per domain)

Closed this issue · 3 comments

Is your feature request related to a problem? Please describe.

Currently the only consumption metrics emitted by the content app is that of bytes requested by clients (by domain). This is helpful for cost purposes, but isn't super helpful for determining usage.

For the RPM side, it would be more helpful to be able to know the count of RPM files that are downloaded. For this we would need to differentiate the files by type. I imagine just using the file extension should be sufficient for this case, identifying it by the 'content type' is not needed.

Describe the solution you'd like

currently you can query the size via:

sum by(domain_name) (rate(pulp_artifacts_size_counter_Bytes_total[2m]) * 60)

I would imagine being able to query the counts with something like:

sum by(domain_name) (rate(pulp_artifacts_counter_total{filetype="rpm"}[2m]))
sum by(domain_name) (rate(pulp_artifacts_counter_total{filetype="gz"}[2m]))
sum by(domain_name) (rate(pulp_artifacts_counter_total{filetype="xml"}[2m]))

Describe alternatives you've considered

Additional context

We're gonna need a new metric for this

Maybe, just a maybe, we could use a histogram here.
It generates buckets, counts and sums automatically.

We have begun visualizing this in our dashboard using access logs, based on the assumption that this data type does not require constant monitoring as a dedicated metric. Additionally, the large variety of file types presents a risk of causing a cardinality explosion.

image