This is and simple way to monitor your elasticsearch cluster and is a fork and enhancement on the original by abronner. It assumes you already have Elasticsearch and Kibana installed.
The idea is simple and straightforward:
- Get statistics via the nodes stats API and then index them back in elasticsearch,
- Repeat it periodically (e.g. every N minutes),
- Use a daily index as per ELK standards,
- Visualize cluster statistics with kibana.
Execute scripts/elasticsearch-nodesstats-template.sh on one of your cluster nodes. This will create a template for all indexes that are created.
Copy scripts/elasticsearch-nodesstats.py to one of your cluster nodes. Start executing it periodically, e.g. using cron (below).
Load dashboards/Elasticsearch Monitoring.json into kibana.
Please note, these are out of date but should give you an idea on what's possible
Two production nodes, sampling nodes stats every 5 minutes, keeping history for 60 days:
- All statistics for a single node are indexed under a single type, identified by the node's name. Use fixed (predefined) node names otherwise any restart of elasticsearch will generate new node names and statistics for a single node will be indexed under a different type.
- scripts/elasticsearch-nodes-stats.py is a python script. It uses the http library requests (if not installed see here).
- Sampling period is defined by the executor of scripts/elasticsearch-nodes-stats.sh. For example, a simple crontab setup for execution every 5 minutes:
*/5 * * * * <path>/elasticsearch-nodes-stats.py
- There are many ways to visualize the data and the provided dashboard is only one of them. Consider creating multiple dashboards according to your needs (e.g. os monitor, jvm monitor, indices monitor, etc).
- Split the graphs, maybe a high level page and then subpages for OS, Memory/Heap, ES specifics etc.
- Should this be a plugin instead?