monitoring-tool

A powerful and easy-to-use monitoring tool for server hardware and validator nodes with alerts via telegram bot and grafana dashboards

Includes

Containers

Dashboards

Node Exporter Full dashboard
Node Exporter for Prometheus Dashboard EN
Cosmos-based Chain Validator Dashboard (my own dashboard)

Alerts

Server alerts

Server down
Out of memory (<10%)
Out of disk space (<10%)
Out of disk space within 24h
High CPU load (>85%)

Cosmos-based validator node alerts

Missing blocks
Degraded syncing (sync less than 40 blocks in last 5 min)
Low peers count (<5)

How to run

Automatic installation

bash <(curl -s https://raw.githubusercontent.com/nodejumper-org/monitoring-tool/main/utils/install.sh)

Manual installation

Install docker

bash <(curl -s https://raw.githubusercontent.com/nodejumper-org/monitoring-tool/main/utils/install_docker.sh)

Clone the repo

cd ~
git clone https://github.com/nodejumper-org/monitoring-tool.git

Create configuration files from examples

cd monitoring-tool
cp docker-compose.yml.example docker-compose.yml
cp prometheus/prometheus.yml.example prometheus/prometheus.yml
cp alertmanager/config.yml.example alertmanager/config.yml
cp Caddyfile.example Caddyfile

Start containers

sudo docker compose up -d

Open in browser http://<your_server_ip>:3000
default credentials: admin/admin

How to configure

Servers to monitor

Add your servers with installed node_exporter or installed cosmos-based node with enabled prometheus port to file prometheus/prometheus.yml

  # example for servers with node_exporter installed
  - job_name: "my-servers"
    static_configs:
    - targets: ["172.0.0.1:9100"]
      labels:
        instance: "server1"
    - targets: ["172.0.0.2:9100"]
      labels:
        instance: "server2"
    
  # example for servers with node_exporter and cosmos-based node installed
  - job_name: "cosmos-validator-nodes"
    static_configs:
    - targets: ["192.0.0.1:9100","192.0.0.1:26660"]
      labels:
        instance: "sentry1"
    - targets: ["192.0.0.2:9100","192.0.0.2:26660"]
      labels:
        instance: "sentry2"
    - targets: ["192.0.0.3:9100","192.0.0.3:26660"]
      labels:
        instance: "validator"

Telegram notifications

In order to enable telegram notifications, create your own bot and fill in the following fields in the file alertmanager/config.yml

chat_id=1111111                 # your telegram user id
bot_token=11111111:AAG_XXXXXXX  # your telegram bot token

How to install node_exporter

Just run next command

bash <(curl https://raw.githubusercontent.com/nodejumper-org/monitoring-tool/main/utils/install_node_exporter.sh)

SSL

If you would like to set up SSL then use the following instruction
Open docker-compose file in text editor and uncomment a part with Caddy service

  reverse-proxy:
    container_name: caddy
    image: caddy/caddy:2-alpine
    restart: unless-stopped
    volumes:
      - caddy_data:/data
      - caddy_config:/config
      - ./Caddyfile:/etc/caddy/Caddyfile
    ports:
      - 80:80
      - 443:443
    networks:
      - monitoring

Second step - replace example.domain.com with your domain name in Caddyfile:

example.domain.com {
	reverse_proxy grafana:3000
}

How to update

Just run next commands

cd monitoring-tool
sudo docker-compose down
git pull
sudo docker-compose pull
sudo docker-compose up -d

deaabie/monitoring-tool