java-influxdb-metrics-agent

Java agent for collecting and reporting JMX metrics to InfluxDB

Installation

Build from source

Clone this repository and mvn clean package to build the artifact.
Put target/java-influxdb-metrics-agent-{version}.jar somewhere on your server.

Download from Maven Central

curl -O 'http://central.maven.org/maven2/net/thisptr/java-influxdb-metrics-agent/0.0.5/java-influxdb-metrics-agent-0.0.5.jar'

Usage

Add -javaagent option to JVM arguments.

-javaagent:<PATH_TO_AGENT_JAR>=<CONF1>=<VALUE1>,...

Example

At least, servers and database must be specified.

-javaagent:/opt/java-influxdb-metrics-agent-0.0.5.jar=servers=influxdb.example.com,database=test

Configuration

Global options

Key	Default	Description
servers	-	Comma-separated list of `hostname:port` of InfluxDB servers.
database	-	The name of the database to store metrics to. The database is automatically created if it does not exist. Currently, only accepts unquoted identifier described in the spec (i.e. no hyphens, etc.).
interval	30	Time, in seconds, between consecutive reporting to InfluxDB servers.
user	root	The user name to use when connecting to InfluxDB servers.
password	root	The password to use when connecting to InfluxDB servers.
tags.<key>	-	Additional tags to set on each measurements. For example, to add a `host` tag, configuration should be: `tags.host = foo.example.com`
retention	-	The name of the RetentionPolicy to write metrics to. If not specified, `DEFAULT` policy is used.
log.level	INFO	One of OFF, ERROR, WARN, INFO, DEBUG, TRACE, ALL.
log.path	(empty)	A path to agent log file. Set '' (empty) to write to STDERR or set - (hyphen) to STDOUT.

Metric-specific options

Metric-specific options can be specified in /MBEAN_REGEX/ { <KEY1> = <VALUE1>, ... } form.

Key	Default	Description
namekeys	<empty>	Comma-separated list of MBean key properties to append to measurement names. For example, `/java.lang/{namekeys=type}` will generate measurements such as `java.lang:type=MemoryPool` and `java.lang:type=GarbageCollector` instead of a single `java.lang` measurement containing mulitple series with different `type` tags.
exclude	false	Exclude metrics from being sent to InfluxDB.

Using configuration files

To read configurations from agent.conf, add @agent.conf somewhere in -javaagent option.

-javaagent:/opt/java-influxdb-metrics-agent-0.0.5.jar=tags.host=`hostname`,@agent.conf

Examples

Apache Flume

# FILE: flume-env.sh
export JAVA_OPTS="-javaagent:/opt/flume/java-influxdb-metrics-agent-0.0.5.jar=tags.host=`hostname`,@/opt/flume/agent.conf"

// FILE: agent.conf
servers = influxdb.example.com
database = myapp
interval = 10
namekeys = type

// We don't need JMImplementation recorded in InfluxDB.
/JMImplementation/ {
	exclude = true
}

/org.apache.flume.*/ {
	// It's convenient to have these channel metrics in a single measurement,
	// rather than separated into many unrelated measurements.
	//  - org.apache.flume.channel:type=foo-ch
	//  - org.apache.flume.channel:type=bar-ch
	//  - org.apache.flume.channel:type=baz-ch
	namekeys =
}

If you prefer, an equivalent configuration can be set without agent.conf:

export JAVA_OPTS="-javaagent:/opt/flume/java-influxdb-metrics-agent-0.0.5.jar=tags.host=`hostname`,servers=influxdb.example.com,database=myapp,interval=10,namekeys=type,/JMImplementation/{exclude=true},/org.apache.flume.*/{namekeys=}"

Apache Tomcat 8

# FILE: setenv.sh
export CATALINA_OPTS="-javaagent:/opt/java-influxdb-metrics-agent-0.0.5.jar=tags.host=`hostname`,@/opt/tomcat/agent.conf"

// FILE: agent.conf
servers = influxdb.example.com
database = myapp
interval = 10
namekeys = type
tags.env = production
tags.role = web

/JMImplementation/ { exclude = true }
/Users/ { exclude = true }

Dropwizard Metrics

Dropwizard Metrics is a Java library for measuring and publishing application metrics. Because MBeans exposed by Metrics does not have a type= key property even through the MBeans sometimes differ in its attribute types, we have to separate measurements using a name= key property when storing into InfluxDB.

/metrics/ { namekeys = name }

Apache HBase

# FILE: hbase-env.sh
export HBASE_JMX_BASE="-javaagent:/opt/java-influxdb-metrics-agent-0.0.5.jar=servers=localhost,tags.host=`hostname`,tags.active='\${jmx|Hadoop:service=HBase,name=Master,sub=Server:tag.isActiveMaster}',database=hbase,interval=10,namekeys=type,/JMImplementation/{exclude=true},/Hadoop/{namekeys='service,name,sub'},/Hadoop::tag\\\\..+/{exclude=true} -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false"
export HBASE_MASTER_OPTS="$HBASE_MASTER_OPTS $HBASE_JMX_BASE -Dcom.sun.management.jmxremote.port=10101"
export HBASE_REGIONSERVER_OPTS="$HBASE_REGIONSERVER_OPTS $HBASE_JMX_BASE -Dcom.sun.management.jmxremote.port=10102"

// FILE: agent.conf
servers = localhost
database = hbase
interval = 10
namekeys = type

tags.host = "${sh|hostname}"
// Set 'active' tag based on HMaster is active or not.
tags.active = "${jmx|Hadoop:service=HBase,name=Master,sub=Server:tag.isActiveMaster}"

/JMImplementation/ {
	// Remove JMImplementation.
	exclude = true
}

/Hadoop/ {
	// Value contains commas, so we need quotes.
	namekeys = 'service,name,sub'
}
/Hadoop::tag\\..+/ {
	// Exclude MBean attributes with "tag." prefix in "Hadoop" domain.
	exclude = true
}

References

Java Management Extensions (JMX) - Best Practices

License

The MIT License.

eiiches/java-influxdb-metrics-agent