Allow for multiple backends to emit metrics

Expected behavior

let amazingMetricsSystem = AmazingMetricsSystem(backends: [
  ConsoleBackend(),
  OSLog(Backend),
  FileBackend(),
  StatsdBackend(),
  PrometheusBackend(),
  HTTPSPostBackend()
])

Maybe this is too complex, but would be helpful if I want to see console, File, and maybe a remote host of some sort.

This could be done through a protocol conformance

protocol Backend {
  func emit(metric: Metric)
}

working example

import ClientKit
import MyMetricsKit

class HttpBackend: Backend {
  private let client: Client

  public var url: String
  public var configuration: Client.Configuration
  
  public init(url: String, configuration: Client.Configuration) {
    self.url = url
    self.configuration = configuration
    self.client = Client(eventLoopGroupProvider: .createNew, configuration: configuration)
  }
  
  public func emit(_ metric: Metric) {
    _ = client.post(url: self.url, item: metric, headers: ["Content-Type":"application/json"])
  }
}


  public func recordNanoseconds(_ duration: Int64) {
    self.lock.withLock {
      values.append((Date(), duration))
    }
    //print("recoding \(duration) \(self.label)")
    self.backends.forEach { $0.emit( Metric(name: self.label, value: duration, type: .timer) )}
  }

Potentially could add metadata to each metric as well for OS and other things ( that can be reset for privacy).

thanks for the idea @thecb4, would MultiplexMetricsHandler work in this use case?

For reference:

swift-metrics/Sources/CoreMetrics/Metrics.swift

Line 519 in f1514a4

public final class MultiplexMetricsHandler: MetricsFactory {

I think that addresses the requirement?

To some degree. Maybe I'm missing the thought process. The "Factory" manages how each metric type is created. The "BackEnd" is meant to bring consistency to communication to whoever is observing the Metric. Regardless of it being a Counter, a Gauge, etc. I may only print all of those metrics vs. ship them off to a server backend. I may do both.I may have various factories all shipping to the same backend.

hey @thecb4 the "Factory" manages how each backend handler is created, a backend handler is just a closure per metric type (CounterHandler, RecorderHandler, etc) that can do something with the metric data, such handlers can print to console, send to http server, etc. pretty much what you describe. the multiplexer then allows you to send the metrics to several handlers in parallel.

see the mux test: https://github.com/apple/swift-metrics/blob/master/Tests/MetricsTests/CoreMetricsTests.swift#L272

let me know if this still does not cover the need

I may be trying to solve for a different problem.
I would only want to implement a Single set of Metrics but send those same metrics to different backends. The above requires me to create a different set of Metrics for each backend. If the backend is only transforming the output of the Metrics, it feels like overkill to create a whole new set of metrics for printing vs. sending to file vs shipping off device (http, etc).

I would only want to implement a Single set of Metrics but send those same metrics to different backends.

That's though exactly what the link above and the Mux enables?Example

let factories = [PrintBackend(), FileBackend()]
MetricsSystem.bootstrapInternal(MultiplexMetricsHandler(factories: factories))

Counter() // goes to both backends

We seem to be talking past each other, can you point out how this is different of what you are aiming for?

Sure. From the way it’s written today in the example, TestMetrics implements MetricsFactory. Every backend has to implement MetricsFactory (every backend is a different MetricsFactory).

What I am asking for is a “Backend” that is agnostic of MetricsFactory. I should only implement a single MetricsFactory and the MetricsFactory offload the final step of capturing metrics (console, file, etc) to as many backend as it needs to. This backend takes a standard Metric structure and transforms it as necessary for the given backend.

I'm sorry but I'm really not sure what specific gain there would be from this or why you cannot do it yourself today: implement one metrics factory which accepts many "whatever api you want" and make that one delegate to those -- that addresses your requirement, but I'm still not sure why do it that way.

I guess my “why” wasn’t clear. No worries. I did implement this for myself. I may write a blog post about it. I think it’s fairly useful. I can always share that after I post.

@thecb4 / @ktoso can we close this one?

I think so, It would be nice to see the blog post by @thecb4 - that should explain what specifically he meant there as well.
Are you still up for writing that up @thecb4 ?

Appreciate the ask. Yep. I am still up for writing it. will try to have something up by the end of the week.

awesome, thank you!

Article is posted:
https://thecb4.io/posts/swift-metrics-multibackend/

Hello again, sorry for being slow to revisit this.

Thanks for the post, now I understand what you mean. So you suggest having a single "emit" function to be implemented by a backend rather than all metrics one by one. I understand the reasoning behind this but I don't think we'll adopt this architectural change. In the "multibackend" architecture you'd loose information and would need to recover it somehow if you needed to record a counter differently than a different metric. I believe the current design indeed has more functions to be implemented, but it gives implementations a lot more flexibility, if that's really necessary or not is to be tested by time and new implementations I suppose though.

Thanks for the post again, though I think we should close the ticket, sounds okey?

Sure. Thanks for the feedback.