shumway

A micro library for sending metrics to a FFWD agent.

Requirements

Python 3.6. Tests pass on Python 3.6, 3.7, and 3.8, and on the latest PyPy3 at build-time.
Support for Linux & OS X

To Use

(env) $ pip install shumway

Counters

Create a default counter and send to FFWD:

import shumway

mr = shumway.MetricRelay(SERVICE_NAME)
mr.incr(METRIC_NAME)
mr.flush()

Initialize a counter with a value

import shumway

mr = shumway.MetricRelay(SERVICE_NAME)
counter = shumway.Counter(metric_name, SERVICE_NAME, value=10)
mr.set_counter(metric_name, counter)
mr.incr(metric_name)
mr.flush()

Different increment values

Create a named counter and increment by a value different than 1:

import shumway

mr = shumway.MetricRelay(SERVICE_NAME)
mr.incr(METRIC_NAME, 2)
mr.flush()

Custom Counter Attributes

Set custom attributes for metrics:

import shumway

mr = shumway.MetricRelay(SERVICE_NAME)
counter = shumway.counter(metric_name, SERVICE_NAME,
                          {attr_1: value_1,
                           attr_2: value_2})

mr.set_counter(metric_name, counter)
mr.incr(metric_name)
mr.flush()

NB: If you use duplicate names when calling set_counter it will overwrite the counter. You will likely want to use a unique metric name for each set of attributes you are setting.

Timers

import shumway

mr = shumway.MetricRelay(SERVICE_NAME)
timer = mr.timer('timing-this-thing')

with timer:
    ...task you want to time

mr.flush()

Custom Timer Attributes

Timers can also be created independently in order to set custom attributes:

import shumway

mr = shumway.MetricRelay(SERVICE_NAME)
timer = shumway.Timer('timing-this-thing', SERVICE_NAME,
                      {'attr_1': value_1, 'attr_2': value_2})

with timer:
    # ...task you want to time

mr.set_timer('timing-this-thing', timer)
mr.flush()

Interacting with metrics objects

Metric objects (like a timer) themselves have a flush function as well as a as_dict function

import shumway

timer = shumway.Timer('timing-this-thing', SERVICE_NAME,
                      {'attr_1': value_1, 'attr_2': value_2})
timer_as_dict = timer.as_dict()
timer.flush(lambda dict: do_smth())

Default attributes for non-custom metrics

MetricRelay can create metrics with a common set of attributes as well:

import shumway

attributes = dict(foo='bar')
mr = shumway.MetricRelay(SERVICE_NAME, default_attributes=attributes)

Resource Identifiers

MetricsRelay and send resource identifiers as well:

import shumway

resources = dict(podname='my_ephemeral_podname')
mr = shumway.MetricRelay(SERVICE_NAME, default_resources=resources)

For more on resource identifiers see Heroic Documentation

Sending Metrics

There are two ways to send metrics to the ffwd agent:

Emit one metric

You can emit a one-off, event-type metric immediately:

import shumway

mr = shumway.MetricRelay('my-service')

# some event happened
mr.emit('a-successful-event', 1)

# some event happened with attributes
mr.emit('a-successful-event', 1, {'attr_1': value_1, 'attr_2': value_2})

# an event with a multiple value happened
mr.emit('a-successful-event', 5)

Flushing all metrics

For batch-like metrics, you can flush metrics once you're ready:

import shumway

mr = shumway.MetricRelay('my-service')

# measure all the things
# time all the things

if not dry_run:
    mr.flush()

Existing Metrics

Check for existence of metrics in the MetricRelay with in:

>>> import shumway
>>> mr = shumway.MetricRelay('my-service')
>>> counter = shumway.Counter('thing-to-count', 'my-service', value=5)
>>> mr.set_counter('thing-to-count', counter)
>>> 'thing-to-count' in mr
True
>>> 'not-a-counter' in mr
False

Custom FFWD agents

By default, shumway will send metrics to a local ffwd agent at 127.0.0.1:19000.

If your ffwd agent is elsewhere, then pass that information through when initializing the MetricRelay:

import shumway

mr = shumway.MetricRelay(SERVICE_NAME, ffwd_ip='10.99.0.1', ffwd_port=19001)

# do the thing

Sending Metrics via HTTP to FFWD

Instead of via UDP it is also possible to send metrics via HTTP by setting the use_http flag:

import shumway

mr = shumway.MetricRelay(SERVICE_NAME,
                         ffwd_host="http://my-metrics-api.com",
                         ffwd_port=8080,
                         ffwd_path="/v1/metrics",
                         use_http=True)

The ffwd_host parameter should be the HTTP endpoint and optionally ffwd_path can be set to specify the path.

Changes

Unreleased

3.0.0 - 3.0.3

Major version bump due to dependency updates and change to supported Python versions.

2.0.0

Positional arguments for Meter(), Counter(), Timer(), and MetricRelay(...).emit() were changed to add resources. If using only named arguments this should not be a problem.

Developer Setup

For development and running tests, your system must have all supported versions of Python installed. We suggest using pyenv.

Setup

$ git clone git@github.com:spotify/shumway.git && cd shumway
# make a virtualenv
(env) $ pip install -r dev-requirements.txt

Running tests

To run the entire test suite:

# outside of the virtualenv
# if tox is not yet installed
$ pip install tox
$ tox

If you want to run the test suite for a specific version of Python:

# outside of the virtualenv
$ tox -e py36

To run an individual test, call nosetests directly:

# inside virtualenv
(env) $ nosetests test/metrics_test.py

Code of Conduct

This project adheres to the Open Code of Conduct. By participating, you are expected to honor this code.

ConnorGraham/shumway