Measure on streams

[msiebuhr]

Create a handler of sort, that makes it easy to measure what's going on in streams;

var measureStream = sdc.getStreamMetric('key.to.report'); // Return readable+writable stream
input.pipe(measureStream).pipe(output)

This would then report how much data goes from input to output via the counter key.to.report.

An advanced version would allow one to do custom stuff on stream events; eg data, end, error and so on:

var mS = sdc.getStreamMetric('key.to.report', {
    data: function (sdc, data) { sdc.increment('data', data.length); },
    end: function (sdc) {sdc.increment('end'); }
});
...

Which would count all passing data in key.to.report.data and how many connections that end in key.to.report.end.

[msiebuhr]

So. In 1a9aeed, I simply add listeners on streams, which is fairly effective (I only add listeners), giving an API that behave thusly:

var sdc = new StatsDClient({...});

var source = fs.createReadStream('/usr/share/dict/english'),
    dest = fs.createWriteStream('/dev/null');
sdc.measureStreamSize('size', source);
sdc.measureStreamLatency('latency', source);
sdc.measureBandwidth('bytes_ms', source);

source.pipe(dest);

Alternately, I'm experimenting with using ... streams (see 0ffa8921):

var sdc = new StatsDClient({...});

var source = fs.createReadStream('/usr/share/dict/english'),
    dest = fs.createWriteStream('/dev/null');

source
    .pipe(sdc.measureStreamSize('size'))
    .pipe(sdc.measureStreamLatency('latency'))
    .pipe(sdc.measureStreamBandwidth('bytes_ms'))
    .pipe(dest);

(Also, bandwidth probably should be measured in bytes/s instead of bytes/ms...)

[msiebuhr]

Historical note: It turned out to be quite simple to support both at the same time. Hope it doesn't confuse people too much.