Metric systems like prometheus aggregate metric at "processing time", whenever the scraper is
able to scrape the metric. event_metrics capture and record metrics at event time, when the
event happens.
Comparing to other metric system, the event_metrics library:
- Write data to sqlite3 database to keep low-memory footprint
- Minimal dependency (only requires numpy for numeric aggregation)
- Aggregate with the full data by default (no reservior sampling)
- Allow select from past duration with timedelta windowing
- Timestamp all observation by default
- Small API footprint:
observeandqueryis all you need to know - Compute raw timeseries with different aggregation strategies:
- Scalers: last, min, max, mean, count, sum
- Buckets for histogram
- Percentiles for summary
- Array and timestamps for native python wrangling
- Metrics can be labeled with arbitrary key value pair and querying supports multidimensional label matching.
- Install from source:
pip install -e . pip install event_metrics
from event_metrics import MetricConnection
conn = MetricConnection("/tmp/event_metrics_demo")
conn.observe("latency", 1.2)
conn.increment("counter", -1)
# labeling
conn.observe("latency", 2.0, labels={"service": "myapp", "other": "label"})
# querying
(conn.query("latency", labels={"service":"myapp"})
# select from past duration using one of the following
.from_beginning()
.from_timestamp(...)
.from_timedelta(...)
# perform aggregation using one of the following
.to_scaler(agg="last/min/max/mean/count/sum") # counter, guage
.to_buckets(buckets=[1, 5, 10], cumulative=False) # histogram
.to_percentiles(percnetiles=[50, 90, 99]) # summary
.to_array() # -> value array
.to_timestamps() # -> timestamp array
.to_timestamps_array() # -> 2 array, (timestamp, value array)
)The library is fast enough. It can ingest about 34,000 data point per seconds:
You can run pytest tests -k bench to generate benchmark on local hardware:
-------------------------------------------------- benchmark: 1 tests --------------------------------------------------
Name (time in us) Min Max Mean StdDev Median IQR Outliers OPS (Kops/s) Rounds Iterations
------------------------------------------------------------------------------------------------------------------------
test_bench_ingestion 25.3340 297.9320 28.8541 9.1582 26.8090 0.8650 521;814 34.6571 6496 1
------------------------------------------------------------------------------------------------------------------------