Metrics

[rightfold]

Icepeak should publish metrics so that Prometheus can read them.

It should do this by listening for HTTP requests on a port separate from the port used by user requests. Prometheus will send requests to retrieve the metrics whenever it wants to; Prometheus is pull-based.

This feature should be optional: it may be enabled only if starting with a --metrics <host>:<port> option or similar environment variable.

It should publish at least the following:

• Total size of the data stored, so that one can be alerted when it increases beyond normal size. We can just stat the data file for this (near constant time); no need to traverse the JSON data (linear time). This should be a Prometheus "Gauge".
• Total number of PUTs and DELETEs since starting Icepeak. Prometheus can itself split this into units of time. This should be a Prometheus "Counter".
• Might as well add GHC metrics, it's one line of code.

You can use the following packages:

• prometheus-client for keeping the metrics in memory.
• prometheus-metrics-ghc for gathering the GHC metrics.
• wai-middleware-prometheus (metricsApp application) for publishing the metrics.