Add support for dynamic dimension configuration?

[PaulFurtado]

We run the signalfx metricproxy on each node in our kubernetes cluster, and apps on the node send their custom metrics through the local metricproxy. This works, but requires apps to be responsible for implementing our standard dimensions correctly in various programming languages and the infrastructure team can't trivially start applying new dimensions. What we really want is to be able to apply different dimensions based on the source IP of the client.

As it currently stands, we can hack this out in the metricproxy by setting up a listening port per-client IP and then using iptables to direct connections to different ports based on the IP, but that's rather complex. Additionally, that requires restarting the metricproxy because I was unable to find a way to dynamically reconfigure it at runtime.

It'd be awesome if I could provide a json file (or hit an http endpoint) mapping IPs (or some more generic rules) to dimensions and if there was some way to signal to the metricproxy that it should reload that. It'd also be awesome if this could be applied generically for all listeners somehow.

In the absence of this feature, I'm just going to make things use the collectd protocol since it puts the dimensions in URL parameters and use nginx instead of metricproxy to modify them, but that forces everything to use the collectd protocol so it'd be great if the metricproxy had a built-in feature for this.

What do you think?

Thanks,
Paul

[codesmith14]

Hi @PaulFurtado,
The metric proxy is intended to be used as a light-weight mechanism for forwarding metrics from multiple sources to SignalFx. At the moment we do not plan to add this functionality to it.

Metric filtering and global dimensions are relatively simple tasks for the proxy because these operations apply to all metrics that pass through it. Implementing anything that is dimension or source specific starts to deviate away from this use case, and it could pose a significant impact to the proxy when it's under a sizeable amount of traffic.

Your solution with nginx sounds like it might be a good for your usecase. If you can, it is best to implement these dimensions at their source.

Thank you for your interest in the Metric Proxy and the suggested enhancement.

Thanks,
Charlie