collectd-iostat-python is an iostat plugin for Collectd that allows you to
graph Linux iostat metrics in Graphite or other output formats that are
supported by Collectd.
This plugin (and mostly this README) is rewrite of the
collectd-iostat from Ruby to Python
using the
collectd-python
plugin.
Disk performance is quite crucial for most of modern server applications, especially databases (e.g. MySQL - check out this slides from Percona Live conference).
Although Collectd provides disk statistics out of the
box, graphing the metrics as
shown by iostat was found to be more useful and graphic, because iostat
reports usage of block devices, partitions, multipath devices and LVM volumes.
Also this plugin was rewritten in Python, because it's a preferable language for siteops' tools on my current job, and choice of using collectd-python instead of collectd-exec was made for performance and stability reasons.
collectd-iostat-python functions by calling iostat with some predefined
intervals and push that data to Collectd using Collectd python plugin.
Collectd can be then configured to write the Collected data into many output
formats that are supported by it's write plugins, such as graphite, which was
the primary use case for this plugin.
Deploy the Collectd Python plugin into a suitable plugin directory for your Collectd instance.
Configure Collectd's python plugin to execute the iostat plugin using a
stanza similar to the following:
<LoadPlugin python>
Globals true
</LoadPlugin>
<Plugin python>
ModulePath "/usr/lib/collectd/plugins/python"
Import "collectd_iostat_python"
<Module collectd_iostat_python>
Path "/usr/bin/iostat"
Interval 2
Count 2
Verbose false
NiceNames false
PluginName collectd_iostat_python
</Module>
</Plugin>
If you would like to use more legible metric names (e.g.
requests_merged_per_second-read instead of rrqm_s), you have to set
NiceNames to true and add load the custom types database (see the
iostat_types.db file) by adding the following into the Collectd config file:
# The default Collectd types database
TypesDB "/usr/share/collectd/types.db"
# The custom types database
TypesDB "/usr/share/collectd/iostat_types.db"
Once functioning, the iostat data should then be visible via your various
output plugins.
In the case of Graphite, Collectd should be writing data in the
hostname_domain_tld.collectd_iostat_python.DEVICE.column-name style namespaces.
Symbols like /, - and % in metric names (but not in device names) are
automatically replaced by underscores (i.e. _).
Please note that this plugin will take only last line of iostat output, so big
Count numbers also have no sense, but Count needs to be more than 1 to get
actual and not historical data. And please make Interval * Count << Collectd.INTERVAL (4 seconds by default). The deafult value of Count=2 and
Interval=2 works quite well for me.
For parsing iostat output, I'm using
jakamkon's
python-iostat Python module, but
as an internal part of the script instead of a separate module because of couple
of fixes I have made (using Kbytes instead of blocks, adding -N to iostat for
LVM endpoint resolving, migration to subprocess module as replacement of
deprecated popen3, objectification etc).
Plugin was tested on Ubuntu 12.04/14.04 (Collectd 5.2/5.3/5.4, Python 2.7) and
CentOS (Collectd 5.4 / Python 2.6). Please note that if running Python 2.6 or
older (i.e. on CentOS and its derivatives) we trying to restore SIGCHLD signal
handler to mitigate a known bug which
according to the Collectd's
documentation
breaks the exec plugin, unfortunately.
- Maybe some data aggregation needed (e.g. we can use some max / avg aggregation
of data across intervals instead of picking last line of
iostatoutput). - The
Disksparameter could support regexp.