it-novum/openITCOCKPIT

Recurring Graphite errors in syslog

Closed this issue · 1 comments

Describe the bug
Recurring error in syslog

To Reproduce
tail -f /var/log/syslog

Expected behavior
No log entries ;-)

Syslog

Aug 31 14:29:37 monitoring systemd[1]: openitcockpit-graphing.service: Scheduled restart job, restart counter is at 59.
Aug 31 14:29:37 monitoring systemd[1]: Stopped Graphing Backend for openITCOCKPIT.
Aug 31 14:29:37 monitoring systemd[1]: Starting Graphing Backend for openITCOCKPIT...
Aug 31 14:29:37 monitoring systemd[1]: Started Graphing Backend for openITCOCKPIT.
Aug 31 14:29:38 monitoring StatusengineWorker.php[6939]: statusengine-worker[6939]: Graphite error: Could not connect: [] Success
Aug 31 14:29:38 monitoring StatusengineWorker.php[6939]: statusengine-worker[6939]: Graphite error: Could not connect: [] Success
Aug 31 14:29:38 monitoring statusengine-worker[6939]: Graphite error: Could not connect: [] Success
Aug 31 14:29:38 monitoring statusengine-worker[6939]: Graphite error: Could not connect: [] Success
Aug 31 14:29:39 monitoring StatusengineWorker.php[6939]: statusengine-worker[6939]: Graphite error: Could not connect: [] Success
Aug 31 14:29:39 monitoring StatusengineWorker.php[6939]: statusengine-worker[6939]: Graphite error: Could not connect: [] Success
Aug 31 14:29:39 monitoring statusengine-worker[6939]: Graphite error: Could not connect: [] Success
Aug 31 14:29:39 monitoring statusengine-worker[6939]: Graphite error: Could not connect: [] Success
...
Aug 31 14:36:01 monitoring statusengine-worker[6939]: Graphite error: Could not connect: [] Success
Aug 31 14:36:01 monitoring statusengine-worker[6939]: message repeated 50 times: [ Graphite error: Could not connect: [] Success]

Versions

  • openITCOCKPIT Server Version: 4.5.1
  • Ubuntu 22.04.1 LTS

For some reason, the required docker images where missing on the system.
To resolve the issue, the docker images needed to be loaded again.

To resolve this issue, run as root:

systemctl stop openitcockpit-graphing

docker load < /opt/openitc/docker/images/docker-graphing.tar.bz2
docker load < /opt/openitc/docker/images/docker-puppeteer.tar.bz2

systemctl startopenitcockpit-graphing

openitcockpit-update

The system had only 6GB free disk space, maybe this was causing the issue in the first place. 🤷‍♂️