A highly scalable framework for the performance and energy monitoring of HPC servers
This setup will install all server-side components of the ExaMon framework:
- MQTT broker and Db connector
- Grafana
- KairosDB
- Cassandra
Since Cassandra is the component that requires the majority of resources, you can find more details about the suggested hardware configuration of the system that will host the services here:
To install all the services needed by ExaMon we will use Docker and Docker Compose:
Install Docker and Docker Compose.
First you will need to clone the Git repository:
git clone https://github.com/ExamonHPC/examon.gitOnce you have the above setup, you need to create the Docker services:
docker-compose up -dThis will build the Docker images and fetch some prebuilt images and then start the services. You can refer to the docker-compose.yml file to see the full configuration.
Log in to the Grafana server using your browser and the default credentials:
Follow the normal procedure for adding a new data source:
Fill out the form with the following settings:
- Type:
KairosDB - Name:
kairosdb - Url: http://kairosdb:8083
- Access:
Server
Once all Docker services are running (can be started either by docker-compose up -d or docker-compose start), the MQTT broker is available at TEST_SERVER port 1883 where TEST_SERVER is the address of the server where the services run.
To test the installation we can use the examon_pub plugin available in the publishers/examon_pub folder of this project.
It is highly recommended to follow the tutorial described in the Jupyter notebook README-notebook.ipynb to understand how an Examon plugin works.
After having installed and configured it on one or more test nodes we can start the data collection running for example:
[root@testnode00]$ ./examon_pub -b TEST_SERVER -p 1883 -t org/myorg -s 1 runIf everything went well, the data are available both through the Grafana interface and using the examon-client.