A pretty generic Storm metrics reporter, currently used to adapt Storm metrics to Yammer metrics API and report them to Graphite, inspired by storm-metrics-statsd.
Build the jar:
git clone https://github.com/staslev/storm-metrics-reporter.git
cd storm-metrics-reporter
mvn package install
Add a pom dependency:
<dependency>
<groupId>com.github.staslev</groupId>
<artifactId>storm-metrics-reporter</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>
Configure your topology:
- Specify
MetricReporter
as the metric consumer. YOUR_GRAPHITE_SERVER
andYOUR_GRAPHITE_PORT
values.- Specify the metrics you want to report by providing an appropriate regular expression, in the example below we're reporting any metric that has the words "execute", "latency" or "capacity".
- Specify your
StormMetricGauge
implementation, in the example we're using the providedSimpleGraphiteStormMetricGauge
class.
final Config config = new Config();
config.put(MetricReporter.METRICS_HOST, YOUR_GRAPHITE_SERVER);
config.put(MetricReporter.METRICS_PORT, YOUR_GRAPHITE_PORT);
config.registerMetricsConsumer(MetricReporter.class,
new MetricReporterConfig("(.*execute.*|.*latency.*|.*capacity.*)",
SimpleGraphiteStormMetricGauge.class.getCanonicalName()),
1);
- This project was inspired by storm-metrics-statsd, thanks jt6211!
- The screenshots above were generated using a custom
StormMetricGauge
, not the one included in the sources. It depicts a particular Graphite naming convention (the host for instance, is not seen in the metric hierarchy chart as it's located higher in the hierarchy). Metrics naming styles are subject to change in the other environemnt. - storm-metrics-reporter currently supports reporting metrics to Graphite only, but it should be pretty straight forward to extend it to support other metrics reporting mechanisms.