creates several containers with different icinga2 characteristics:
- icinga2 as master with a certificats service
- icinga2 as satellite
I will not continue to develop this project in the foreseeable future and will therefore archive it.
In my opinion a monitoring tool has no right to exist in a Docker Container or a Kubernetes Cluster.
After a long time with alpine as the base I had to go back to a debian based distribution. I couldn't run Icinga stable with the musl lib. :(
The current Dockerfiles are structured so that you can use both debian (9-slim) and/or ubuntu (18.10).
You can use the included Makefile.
- To build the Containers:
make
bundle following calls:make build_base(builds an base container with all needed components)make build_master(builds an master container with the icinga certificate service)make build_satellite(build also an satellite)
- To build an valid
docker-compose.ymlfile:make compose-file - To remove the builded Docker Images:
make clean - use a container with login shell:
make base-shellmake master-shellmake satellite-shell
- run tests
make testbundle following calls:make lintermake integration_test
You can specify an image version by using the ICINGA2_VERSION environment variable (This defaults to the "latest" tag).
To change this export an other value for ICINGA2_VERSION (e.g. export ICINGA_VERSION=2.8.4)
The test checks whether all container instances were started successfully. It also checks the accessibility of the individual containers and whether the certificate exchange was successful.
inspect needed containers
/nginx has pid 25414 - healthy
/icingaweb2 has pid 25089 - healthy
/icinga2-satellite-1 has pid 24228 - healthy
/icinga2-satellite-2 has pid 23941 - healthy
/icinga2-satellite-3 has pid 24095 - starting
/icinga2-master has pid 30569 - healthy
/database has pid 23390 - healthy
wait for the certificate service
wait for the icinga2 instances
the master instance must run 100 seconds without interruption
the icinga2 instance icinga2-master is 197 seconds up and alive
the icinga2 instance icinga2-satellite-1 is 223 seconds up and alive
the icinga2 instance icinga2-satellite-2 is 225 seconds up and alive
the icinga2 instance icinga2-satellite-3 is 157 seconds up and alive
service icinga2-master (fqdn: icinga2-master.matrix.lan / ip: 172.27.0.2) | version: r2.10.3-1
service icinga2-satellite-1 (fqdn: icinga2-satellite-1.matrix.lan / ip: 172.27.0.5) | version: r2.10.3-1
service icinga2-satellite-2 (fqdn: icinga2-satellite-2.matrix.lan / ip: 172.27.0.3) | version: r2.10.3-1
service icinga2-satellite-3 (fqdn: icinga2-satellite-3.matrix.lan / ip: 172.27.0.4) | version: r2.10.3-1
api request are successfull
endpoints summary:
totaly: '3' / connected: '3' / not connected: '0'
connected endpoints:
[
"icinga2-satellite-2.matrix.lan",
"icinga2-satellite-3.matrix.lan",
"icinga2-satellite-1.matrix.lan"
]
not connected endpoints:
[]
API zones:
{
"icinga2-master.matrix.lan": {
"client_log_lag": 0,
"connected": true,
"endpoints": [
"icinga2-master.matrix.lan"
],
"parent_zone": ""
},
"icinga2-satellite-1.matrix.lan": {
"client_log_lag": 0,
"connected": true,
"endpoints": [
"icinga2-satellite-1.matrix.lan"
],
"parent_zone": "icinga2-master.matrix.lan"
},
"icinga2-satellite-2.matrix.lan": {
"client_log_lag": 0,
"connected": true,
"endpoints": [
"icinga2-satellite-2.matrix.lan"
],
"parent_zone": "icinga2-master.matrix.lan"
},
"icinga2-satellite-3.matrix.lan": {
"client_log_lag": 0,
"connected": true,
"endpoints": [
"icinga2-satellite-3.matrix.lan"
],
"parent_zone": "icinga2-master.matrix.lan"
}
}Please read Contribution
The master Branch is my Working Horse includes the "latest, hot shit" and can be complete broken!
If you want to use something stable, please use a Tagged Version or an Branch like 2.9.1
if use need some enhancements, you can add some (bash) scripts and add them via volume to the container:
--volume=/${PWD}/tmp/test.sh:/init/custom.d/test.shThis scripts will be started before everything else!
YOU SHOULD KNOW WHAT YOU'RE DOING.
THIS CAN BREAK THE COMPLETE ICINGA2 CONFIGURATION!
I use the official Icinga2 packages from Icinga.
I remove branches as soon as they are disfunctional (e. g. if a package is no longer available at Alpine). Not immediately, but certainly after 2 months.
You can find the Container also at DockerHub
The actuall Container Supports a stable MySQL Backend to store all needed Datas into it.
- api
- command
- checker
- mainlog
- notification
- graphite (only available if the environment variables are set)
To connect a satellite to a master, you need a certificate issued by the master and signed by its CA.
I have tried to automate the exchange of certificates so that you can launch the satellites unattended, and they then take care of a certificate themselves.
The Icinga2 documentation provides more information about Distributed Monitoring and Certificates.
I strongly recommend a study of the documentation!
My solution was to create my own ReST service which runs on the Master Container.
This allows me to e.g. create a ticket for issuing a certificate. But also the direct creation of a certificate is possible.
Within a docker environment this is a bit more difficult, so an external service is used to simplify this. This service is constantly being developed further, but is integrated into the docker container in a stable version.
The certificate service is only available at an Icinga2 Master!
You need a valid and configured API User in Icinga2.
The certificate service requires the following environment variables:
ICINGA2_MASTER(default: ``)BASIC_AUTH_USER(default:admin)BASIC_AUTH_PASS(default:admin)ICINGA2_API_PORT(default:5665)ICINGA2_API_USER(default:root)ICINGA2_API_PASSWORD(default:icinga)
Certificate exchange is automated within the containers. If you want to issue your own certificate, you can use the following API calls.
Since Version 2.8 you can use icinga2 node wizard on a satellite or agent to create an certificate request.
To create a certificate:
curl \
--request GET \
--user ${ICINGA2_CERT_SERVICE_BA_USER}:${ICINGA2_CERT_SERVICE_BA_PASSWORD} \
--silent \
--header "X-API-USER: ${ICINGA2_CERT_SERVICE_API_USER}" \
--header "X-API-PASSWORD: ${ICINGA2_CERT_SERVICE_API_PASSWORD}" \
--output /tmp/request_${HOSTNAME}.json \
http://${ICINGA2_CERT_SERVICE_SERVER}:${ICINGA2_CERT_SERVICE_PORT}/v2/request/${HOSTNAME}Extract the session checksum from the request above.
checksum=$(jq --raw-output .checksum /tmp/request_${HOSTNAME}.json)Download the created certificate:
curl \
--request GET \
--user ${ICINGA2_CERT_SERVICE_BA_USER}:${ICINGA2_CERT_SERVICE_BA_PASSWORD} \
--silent \
--header "X-API-USER: ${ICINGA2_CERT_SERVICE_API_USER}" \
--header "X-API-PASSWORD: ${ICINGA2_CERT_SERVICE_API_PASSWORD}" \
--header "X-CHECKSUM: ${checksum}" \
--output /tmp/${HOSTNAME}/${HOSTNAME}.tgz \
http://${ICINGA2_CERT_SERVICE_SERVER}:${ICINGA2_CERT_SERVICE_PORT}/v2/cert/${HOSTNAME}The generated certificate has an timeout from 10 minutes between beginning of creation and download.
You can also look into rootfs/init/examples/use_cert-service.sh
For Examples to create a certificate with commandline tools look into rootfs/init/examples/cert-manager.sh
You can use expect on a satellite or agent to create an certificate request with the icinga2 node wizard:
expect /init/examples/node-wizard.expectAfter this, you can use the cert-service to sign this request:
curl \
--user ${ICINGA2_CERT_SERVICE_BA_USER}:${ICINGA2_CERT_SERVICE_BA_PASSWORD} \
--silent \
--request GET \
--header "X-API-USER: ${ICINGA2_CERT_SERVICE_API_USER}" \
--header "X-API-PASSWORD: ${ICINGA2_CERT_SERVICE_API_PASSWORD}" \
--write-out "%{http_code}\n" \
--output /tmp/sign_${HOSTNAME}.json \
http://${ICINGA2_CERT_SERVICE_SERVER}:${ICINGA2_CERT_SERVICE_PORT}/v2/sign/${HOSTNAME}After a restart of the Icinga2 Master the certificate is active and a secure connection can be established.
But... (unfortunately there is always one but)
If there is a CNAME on a satellite which is resolved to an external system this does not work. Therefore I had to modify the certificate exchange.
From now on I generate the certificate again with board resources. For this I need a ticket from the master, which I request via the certificate service.
All around to create the certificate you can see here
make sure you only use the environment variable you need!
| Environmental Variable | Default Value | Description |
|---|---|---|
ICINGA2_LOGLEVEL |
warning |
The minimum severity for the main-log. For more information, see into the icinga doku |
| Environmental Variable | Default Value | Description |
|---|---|---|
MYSQL_HOST |
- | MySQL Host |
MYSQL_PORT |
3306 |
MySQL Port |
MYSQL_ROOT_USER |
root |
MySQL root User |
MYSQL_ROOT_PASS |
randomly generated | MySQL root password |
IDO_DATABASE_NAME |
icinga2core |
Schema Name for IDO |
IDO_PASSWORD |
randomly generated | MySQL password for IDO |
| Environmental Variable | Default Value | Description |
|---|---|---|
ICINGA2_API_USERS |
- | comma separated List to create API Users. The Format are username:password(e.g. admin:admin,dashing:dashing and so on) |
| Environmental Variable | Default Value | Description |
|---|---|---|
CARBON_HOST |
- | hostname or IP address where Carbon/Graphite daemon is running |
CARBON_PORT |
2003 |
Carbon port for graphite |
| Environmental Variable | Default Value | Description |
|---|---|---|
ICINGA2_MASTER |
- | The Icinga2-Master FQDN for a Satellite Node |
ICINGA2_PARENT |
- | The Parent Node for an Cluster Setup (not yet implemented) |
BASIC_AUTH_USER |
admin |
both BASIC_AUTH_* and the ICINGA2_MASTER are importand, if you use and modify the authentication of the icinga-cert-service |
BASIC_AUTH_PASS |
admin |
|
CERT_SERVICE_BA_USER |
admin |
The Basic Auth User for the certicate Service |
CERT_SERVICE_BA_PASSWORD |
admin |
The Basic Auth Password for the certicate Service |
CERT_SERVICE_API_USER |
- | The Certificate Service needs also an API Users |
CERT_SERVICE_API_PASSWORD |
- | |
CERT_SERVICE_SERVER |
localhost |
Certificate Service Host |
CERT_SERVICE_PORT |
80 |
Certificate Service Port |
CERT_SERVICE_PATH |
/ |
Certificate Service Path (needful, when they run behind a Proxy |
| Environmental Variable | Default Value | Description |
|---|---|---|
ICINGA2_MSMTP_RELAY_SERVER |
- | SMTP Service to send Notifications |
ICINGA2_MSMTP_REWRITE_DOMAIN |
- | |
ICINGA2_MSMTP_RELAY_USE_STARTTLS |
- | |
ICINGA2_MSMTP_SENDER_EMAIL |
- | |
ICINGA2_MSMTP_SMTPAUTH_USER |
- | |
ICINGA2_MSMTP_SMTPAUTH_PASS |
- | |
ICINGA2_MSMTP_ALIASES |
- | |
ICINGA2_MSMTP_RECV_ROOT |
- | |
ICINGA2_MSMTP_ACC_NAME |
- |
| Environmental Variable | Default Value | Description |
|---|---|---|
DEMO_DATA |
false |
copy demo data from /init/demo-data into /etc/icinga2 config path |
To connect a satellite to a master, the master must have activated the Cert service and the satellite must know how to reach it.
A docker-compose file can be created with make compose-file and look like this::
networks:
backend: {}
database: {}
frontend: {}
satellite: {}
services:
database:
container_name: database
environment:
MYSQL_ROOT_PASS: vYUQ14SGVrJRi69PsujC
MYSQL_SYSTEM_USER: root
hostname: database
image: bodsch/docker-mysql:latest
networks:
backend: null
database: null
volumes:
- /etc/localtime:/etc/localtime:ro
icinga2-master:
build:
args:
BUILD_DATE: '2018-08-25'
BUILD_VERSION: '1808'
CERT_SERVICE_TYPE: stable
CERT_SERVICE_VERSION: 0.18.3
ICINGA2_VERSION: 2.9.1
context: /src/docker/docker-icinga2
dockerfile: Dockerfile.master
container_name: icinga2-master
environment:
BASIC_AUTH_PASS: admin
BASIC_AUTH_USER: admin
CARBON_HOST: ''
CARBON_PORT: '2003'
CERT_SERVICE_API_PASSWORD: icinga
CERT_SERVICE_API_USER: root
CERT_SERVICE_BA_PASSWORD: admin
CERT_SERVICE_BA_USER: admin
CERT_SERVICE_PATH: /cert-service/
CERT_SERVICE_PORT: '443'
CERT_SERVICE_SERVER: nginx
DEBUG: '0'
DEMO_DATA: "false"
ICINGA2_API_USERS: root:icinga,dashing:dashing,cert:foo-bar
ICINGA2_MASTER: icinga2-master.matrix.lan
IDO_PASSWORD: qUVuLTk9oEDUV0A
LOG_LEVEL: INFO
MYSQL_HOST: database
MYSQL_ROOT_PASS: vYUQ14SGVrJRi69PsujC
MYSQL_ROOT_USER: root
hostname: icinga2-master.matrix.lan
links:
- database
networks:
backend: null
database: null
ports:
- published: 5665
target: 5665
- published: 8080
target: 8080
privileged: false
restart: always
volumes:
- /etc/localtime:/etc/localtime:ro
icinga2-satellite-1:
build:
args:
BUILD_DATE: '2018-08-25'
BUILD_VERSION: '1808'
ICINGA2_VERSION: 2.9.1
context: /src/docker/docker-icinga2
dockerfile: Dockerfile.satellite
container_name: icinga2-satellite-1
environment:
CERT_SERVICE_API_PASSWORD: icinga
CERT_SERVICE_API_USER: root
CERT_SERVICE_BA_PASSWORD: admin
CERT_SERVICE_BA_USER: admin
CERT_SERVICE_PATH: /cert-service/
CERT_SERVICE_PORT: '443'
CERT_SERVICE_SERVER: nginx
DEBUG: '0'
ICINGA2_MASTER: icinga2-master.matrix.lan
ICINGA2_PARENT: icinga2-master.matrix.lan
hostname: icinga2-satellite-1.matrix.lan
links:
- icinga2-master:icinga2-master.matrix.lan
networks:
backend: null
satellite: null
privileged: true
restart: always
volumes:
- /dev:/dev:ro
- /proc:/host/proc:ro
- /sys:/host/sys:ro
- /sys:/sys:ro
icinga2-satellite-2:
build:
args:
BUILD_DATE: '2018-08-25'
BUILD_VERSION: '1808'
ICINGA2_VERSION: 2.9.1
context: /src/docker/docker-icinga2
dockerfile: Dockerfile.satellite
container_name: icinga2-satellite-2
environment:
CERT_SERVICE_API_PASSWORD: icinga
CERT_SERVICE_API_USER: root
CERT_SERVICE_BA_PASSWORD: admin
CERT_SERVICE_BA_USER: admin
CERT_SERVICE_PATH: /cert-service/
CERT_SERVICE_PORT: '443'
CERT_SERVICE_SERVER: nginx
DEBUG: '0'
ICINGA2_MASTER: icinga2-master.matrix.lan
ICINGA2_PARENT: icinga2-master.matrix.lan
hostname: icinga2-satellite-2.matrix.lan
links:
- icinga2-master:icinga2-master.matrix.lan
networks:
backend: null
satellite: null
privileged: true
restart: always
volumes:
- /dev:/dev:ro
- /proc:/host/proc:ro
- /sys:/host/sys:ro
- /src/docker/docker-icinga2/import:/import:ro
- /sys:/sys:ro
icingaweb2:
container_name: icingaweb2
environment:
ICINGA2_CMD_API_PASS: icinga
ICINGA2_CMD_API_USER: root
ICINGA2_MASTER: icinga2-master.matrix.lan
ICINGAWEB2_USERS: icinga:icinga,foo:bar
ICINGAWEB_DIRECTOR: "false"
IDO_DATABASE_NAME: icinga2core
IDO_PASSWORD: qUVuLTk9oEDUV0A
MYSQL_HOST: database
MYSQL_ROOT_PASS: vYUQ14SGVrJRi69PsujC
MYSQL_ROOT_USER: root
hostname: icingaweb2.matrix.lan
image: bodsch/docker-icingaweb2:2.6.1
links:
- database
- icinga2-master:icinga2-master.matrix.lan
networks:
backend: null
database: null
frontend: null
ports:
- target: 80
nginx:
container_name: nginx
depends_on:
- icinga2-master
- icingaweb2
hostname: nginx
image: bodsch/docker-nginx:1.14.0
links:
- icinga2-master
- icingaweb2:icingaweb2.matrix.lan
networks:
backend: null
frontend: null
ports:
- published: 80
target: 80
- published: 443
target: 443
restart: always
volumes:
- /src/docker/docker-icinga2/compose/config/nginx.conf:/etc/nginx/nginx.conf:ro
- /src/docker/docker-icinga2/compose/ssl/cert.pem:/etc/nginx/secure/localhost/cert.pem:ro
- /src/docker/docker-icinga2/compose/ssl/dh.pem:/etc/nginx/secure/localhost/dh.pem:ro
- /src/docker/docker-icinga2/compose/ssl/key.pem:/etc/nginx/secure/localhost/key.pem:ro
version: '3.3'In the above example the nginx will start with SSL support.
You musst create the required certificate locally as follows or you use your own.
At the following prompt, the most important line is the one requesting the common name.
Here you have to enter the domain name which is assigned to the respective computer (hostname -f or localhost).
$ openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout ssl/key.pem -out ssl/cert.pem
Generating a 2048 bit RSA private key
.....+++
..........................................+++
writing new private key to 'ssl/key.pem'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [AU]:DE
State or Province Name (full name) [Some-State]:Hamburg
Locality Name (eg, city) []:Hamburg
Organization Name (eg, company) [Internet Widgits Pty Ltd]:
Organizational Unit Name (eg, section) []:
Common Name (e.g. server FQDN or YOUR name) []:localhost
Email Address []:Then we create a Diffie-Hellman group to activate Perfect Forward Secrecy:
$ openssl dhparam -out ssl/dh.pem 2048
Generating DH parameters, 2048 bit long safe prime, generator 2
This is going to take a long time
....................................The 3 files created then belong in the directory compose/ssl.
In this example I use my own docker containers:
Please check for deviating tags at Docker Hub!
This example can be used as follows: docker-compose up --build
Afterwards you can see Icinga Web2 in your local browser at http://localhost.
