yaorg/node-measured

Support for Value and Rate in Histogram

krishanfrog opened this issue · 5 comments

krishanfrog commented

I have a requirement where I would like to know the last value recorded for a metric and rate of a particular metric

To get the Latest value I can use a gauge but it is synchronous and requires developers to maintain the values they require to measure

Rate of a metric is not possible right now. If I want to understand rate of a metric I have to use counters and calculate rate of change manually.

Let me know if the 2 requirements make sense and I can raise a Pull request for the same.

tlisonbee commented

For rates you might consider a Meter or a Timer. Meters give you rates. Timers are a Histogram plus a Meter.

Or if you are using a vendor like SignalFx, Counters are displayed as rates in their UI by default.

fieldju commented

@krishanfrog in reply to

Rate of a metric is not possible right now. If I want to understand rate of a metric I have to use counters and calculate rate of change manually.

Like what @tlisonbee said Meters are for measuring rates of events.

See the output of toJSON() for the Meter

Here is an example of a Meter tracking the rate of the someEvent event being emitted by an instance of an Object aptly named myEventEmitter that extends event emitter.

const { Meter } = require('measured-core');
const myEventMeter = new Meter();

someEventEmitter.on('someEvent', () => {
    // https://yaorg.github.io/node-measured/Meter.html#mark
    myEventMeter.mark();
})

As for getting the last value of some variable you care about, you are correct about Gauges, they are used to track a specific value of some thing you care about.

There are currently 3 types of Gauges.

  1. The default Gauge, a Gauge that gets its value synchronously from a supplied call back.
const Gauge = new Gauge(() => {
    return someValue;
});
  1. A SettableGauge, a Gauge that has a method called setValue(value) that allows you to set the value externally. This gauge always returns the set value and does not take callback.
const settableGauge = new SettableGauge();
// Update the settable gauge ever 10'ish seconds
setInterval(() => {
    calculateSomethingAsync().then((value) => {
        settableGauge.setValue(value);
    });
}, 10000);
  1. A CachedGauge, a Gauge that takes a promise producing callback that will resolve the value that should be cached in this Gauge, and an interval that the callback should be executed on to update the cached value. This Gauge will always return the cached value from the latest value resolved from the promise producing callback. This gauge lets you do asynchronous calculations.
const cpuAverageCachedGauge = new CachedGauge(() => {
    return new Promise(resolve => {
      //Grab first CPU Measure
      const startMeasure = cpuAverage();
      setTimeout(() => {
        //Grab second Measure
        const endMeasure = cpuAverage();
        const percentageCPU = calculateCpuUsagePercent(startMeasure, endMeasure);
        resolve(percentageCPU);
      }, sampleTimeInSeconds);
    });
  }, updateIntervalInSeconds);

@krishanfrog please let me know if, you are able to accomplish what you need with a Gauge and a Meter.

If you are not able to, please give me more details into what you are trying to do so that I may better understand the problem.

fieldju commented

Id imaging if you have some function that we can define as your event you can tie the SettableGauge and Meter together to accomplish tracking the rate and last value.

const { Meter, SettableGauge } = require('measured-core')
const { SelfReportingMetricsRegistry, Reporter } = require('measured-reporting')

// Create anonymous console logging instance as an example
const reporter = new class extends Reporter {
    _reportMetrics(metrics) {
        metrics.forEach(metric => console.log(JSON.stringify(metric.toJSON())))
    }
};
const registry = new SelfReportingMetricsRegistry(reporter);
const myEventMeter = new Meter();
const myLatestValue = new SettableGauge();
registry.register('myApp.someEvent.rates', myEventMeter);
registry.register('myApp.someEvent.latestValue', myLatestValue);

const myEvent = () => {
    return new Promise((resolve, reject) => {
        doMyEvent().then((myAsyncValue) => {
            myEventMeter.mark();
            myLatestValue.setValue(myAsyncValue)
            resolve(myAsyncValue);
        });
    });
}

Please note that the Meter creates an interval and when you are done with it you must caller myEventMeter.end(), this occurs automatically if you use a registry and call registry.shutdown(). If you do not do this than your Node process will not shutdown gracefully.

EDIT

I just saw the that the title says Histogram, so you would just use a Timer which is a histogram and meter combined. or manually use all three (Histogram, SettableGauge and a Meter) together.

const { Timer, SettableGauge } = require('measured-core')
const { SelfReportingMetricsRegistry, Reporter } = require('measured-reporting')

// Create anonymous console logging instance as an example
const reporter = new class extends Reporter {
    _reportMetrics(metrics) {
        metrics.forEach(metric => console.log(JSON.stringify(metric.toJSON())))
    }
};
const registry = new SelfReportingMetricsRegistry(reporter);
const myEventTimer = new Timer();
const myLatestValue = new SettableGauge();
registry.register('myApp.someEvent.rates', myEventMeter);
registry.register('myApp.someEvent.latestValue', myLatestValue);

const myEvent = () => {
    return new Promise((resolve, reject) => {
        doMyEvent().then((myAsyncValue) => {
            myEventTimer.update(myAsyncValue);
            myLatestValue.setValue(myAsyncValue)
            resolve(myAsyncValue);
        });
    });
}
myEventTimer.toJSON()
{
    meter: {} // value of the meter.toJSON()
    histogram: {} // value of the histogram.toJSON()
}

See https://yaorg.github.io/node-measured/global.html#MeterData__anchor and https://yaorg.github.io/node-measured/global.html#HistogramData__anchor

krishanfrog commented

Thanks for the update.

Will try out Meter.

Does it make sense to add SettableGauge functionalities to Histogram by default so that a single api can be used to send the details?

fieldju commented

Does it make sense to add SettableGauge functionalities to Histogram by default so that a single api can be used to send the details?

@krishanfrog, my initial reaction to this would be to say no, and here is why.

A histogram measures the statistical distribution of values in a stream of data. In addition to minimum, maximum, mean, etc., it also measures median, 75th, 90th, 95th, 98th, 99th, and 99.9th percentiles.

I would hypothesis that most use cases do not have a need for knowing the last added value as they mostly care about the trends in the stream of values.

Adding a new datapoint (last added value) to the outputted JSON while seemingly not a big deal, has consequences that you may not be thinking about. Lets say that you are running 1000 node based microservices using this lib and they each have 10 histograms that report every 10 seconds. Adding a new datapoint adds 60,000 (1000 instances * 10 histogram * 6 reports per minute) more Data Points per Minute for a value that they likely don't care about. Some vendors such as SignalFx charge customers by DPM, so this would have a cost consequence to people who upgraded their library.

Aside from the cost concerns, there are also concerns about avoiding over complicating the metrics types and thus also over complicating what it takes to implement reporting. Lets say to avoid adding extra DPM to the Histogram as outlined above, you made a new Metric type called LastValueCachingMeteredHistogram that was a Settable Gauge, Meter, Histogram hybrid. What would its MetricType be? All reporters and registries would need to be updated to support it.

I think an acceptable middle ground may be to implement some sort of white / black list of values for either the Metric#toJSON() methods or in the Registry / Reporters. Then you could just add the extra value to the base Histogram with out worrying about increasing Datapoints.

The whitelist / black list has been a thing that is on my TODO list for a while. Take for example the SignalFx Reporter getValuesToProcessForHistogram() Method currently a subset of the Histograms available data is hardcoded, being able to define what values you want or don't want would be a nice mechanism for users to have complete control of what gets reported.

If you wanted to make a PR that modified the Base Reporter and its Options and the method signature of Abstract _reportMetrics method to take a whitelist or blacklist and update the Signal Fx reporter to honor the filtering. Then adding more data points to existing metrics would not be a concern as long as there isn't performance issues.