API for creating metrics with specific buckets or quantiles

[ktoso]

Some more users often need to pre-configure their metrics with pre-determined bucket sizes (and numbers) or quantiles they want to report. This matters most for metrics which do some form of aggregation client side, like prometheus (e.g. https://github.com/MrLotU/SwiftPrometheus/blob/master/Sources/Prometheus/Prometheus.swift#L210 ).

We should make it possible to use the SwiftMetrics API without dropping to raw APIs and still be able to configure bucket sizes etc. These are a common and not specific-backend bound configuration parameter.

It could look something like:

Timer(
    label: "hello", 
    dimensions: [("a": "aaa")], 
    buckets: [.005, .01, .025, .05, .075, .1, .25, .5, .75, 1, 2.5, 5, 7.5, 10]
)
Timer(
    label: "hello",
    dimensions: [("a": "aaa")], 
    buckets: .linear([.100, .100, 100)]
)
Timer(
    label: "hello",
    dimensions: [("a": "aaa")], 
    buckets: .exponential([1, 2, 64)]
)

Timer(
    label: "hello",
    dimensions: [("a": "aaa")], 
    quantiles: [0.5, 0.05, 0.9, 0.01]
)

Use case discussed with @avolokhov, we'll iterate on the exact needs and shapes of this soon

[ktoso]

FYI @MrLotU as well :-)

[MrLotU]

Big +1! This would be useful for both Timer as well as Recorder (Summary and Histogram in Prometheus land)

[ktoso]

Right, sounds good Recorder as well 👍

[tomerd]

thanks for bringing this up. what would backends that do not support such feature do with this information?

[avolokhov]

I'm not sure if it sounds crazy, but I have this thought in mind:
Imagine you want to instrument you, say, http client library with a request payload size histogram.
With the old library, you create a bunch of metrics and feed the data to it.
With a new library you see an api that expects quantiles and histograms. Being a library creator you have 0 knowledge about how this library is going to be used (and it may vary a lot - average payload may be as low as a few bytes or as high as a few gigabytes). So you have to either guess, or drop the idea in a fear that misconfigured metrics may hurt.

[ktoso]

I think there's really only two ways out, one of them being as Anton suggests.

Option 1) backends may ignore this information silently

So given the example:

let timer = Timer(
    label: "hello", 
    dimensions: [("a": "aaa")], 
    buckets: [.005, .01, .025, .05, .075, .1, .25, .5, .75, 1, 2.5, 5, 7.5, 10]
)

Say, statsd client would simply ignore it. It directly emits values without aggregating, so the setting has no effect.
Silently ignoring config is a bit meh, but not the end of the world here actually... c'est la vie, backends to the best they can with the passed in information. 🤔

Option 2) optional initializers

We could make those new initializers failable, then they'd return nil if unable to support this feature, usage becomes more verbose then:

guard let timer = Timer(
    label: "hello", 
    dimensions: [("a": "aaa")], 
    buckets: [.005, .01, .025, .05, .075, .1, .25, .5, .75, 1, 2.5, 5, 7.5, 10]
) else { fatalError("OH NO, I DEFINITELY NEED BUCKETS") }

or, falling back to non buckets if not necessary

let timer = Timer(
    label: "hello", 
    dimensions: [("a": "aaa")], 
    buckets: [.005, .01, .025, .05, .075, .1, .25, .5, .75, 1, 2.5, 5, 7.5, 10]
) ?? Timer(label: "hello", dimensions: [("a": "aaa")]) // "ok, i don't actually care/need buckets"

---

The option 2 is technically more correct but seems like a large burden and could accidentally make some libraries not usable with some backends, only because they took the "oh well, lets crash" approach.

We could try to make option 2 spell somewhat nicer, if we really believe in it?

Personally I think Option 1 is fine... I'll check what similar APIs do in the Java ecosystem though

[ktoso]

• OpenTelemetry Metrics do not define ways to configure such details.
  • otel discussions I've found are pretty inconclusive and ignoring such details to be honest.
• APIs like dropwizard metrics don't expose this configuration detail in the metric types.

Alternatively, we could double down and design some form of Metric(..., configured: ...) which would allow specific instrumentations to "add" and "pass through" the APIs but that looks pretty nasty to be honest.

[tomerd]

@ktoso did you want to do anything here for 2.1.0, or should we release 2.1.0 sooner?

[ktoso]

We can cut at any time as far as I'm concerned - this does not have to be in 2.1.0 :) you can cut or I'll do tomorrow

[blindspotbounty]

I believe this feature would be nice to have.
Though, if some backends (like swift prometheus) which actually supports that, can be used by providing implementation handlers to Recorder/Timer instead of having it in swift-metrics frontend api.