Send gauges instead of counts to Statsd?

[dschiavu]

Hi,

I see that the current implementation sends Storm's metrics to Statsd as Statsd counters, not gauges.

Counters are aggregated by Statsd in the specified time window. Gauges are not aggregated by Statsd.

This rappresents a problem for my topology. I record gauge values in my Storm metrics stream. It seems Storm's metric emit timer isn't very precise -- in my example, I set Storm's metric time window to 10 seconds, but often the Storm's metrics are emitted to the metrics consumer in 3 or even 2-second intervals, not the specified 10 second intervals, which, of course, is a problem, as Statsd will sum those two values to a single counter and emit it to the backend (Graphite), giving us the wrong value.

Since Storm's Metric emit interval is so imprecise, I think it would be better to change the implementation to emit the metrics to Statsd as gauges, not counters. Or possibly, support both types of Statsd metric types, and add a configuration option to make it user-configurable.

What do you think? Do you have similar issues with the Storm's metric emit interval varying around the specified interval? My Storm version in use is 0.9.0.1 (latest).

Thanks,
Danijel

[staslev]

In addition, counters work with integers which poses a problem with metrics such as the execute-latency, which can very well be a fraction (0.012), in which case ((Number) p.value).intValue()) needs to become ((Number) p.value).doubleValue()), along with the rest of the places where the value is assumed to be an integer.

This somewhat corresponds with the general question of counters vs. gauges in this case. It would seem that gauges might be the way to go since Storm already does the counting, though this requires the logic to not overwrite values as neatly mentioned in the discussion concerning pull request #1.

Bottom line is, latency under 1 ms is currently floored to 0, which might not be very intuitive.