tinkerbell/osie

Send the worker logs to the boots syslog server?

rgl opened this issue · 6 comments

rgl commented

#51 dropped support for forwarding the logs to a central place but left the worker without any way to troubleshoot the actions (and since the containers are deleted, we cannot use docker logs).

We should have a way to see the actions/worker logs somewhere, maybe forward them to the boots syslog server?

This is somewhat related to #48, but saving the logs to a local file is not enough because when the worker reboots we loose the logs.

completed the POC using rsyslog on provisioner and docker daemon conf on both provisioner and worker node. Below is the HLD for the same. Logs from both, the tinkerbell services and worker nodes are reflecting correctly in the provisioner. Will moe forward on the development.

Screenshot from 2020-07-29 16-53-13

rgl commented

Looking good! Happy to see this progress :-)

Will this use --log-opt syslog-format=rfc5424micro? Are/Have you considered other options?

Will it use TLS to transport the logs from worker to provisioner?

@rgl Sure,

I have added an RFD https://github.com/tinkerbell/proposals/pull/7/files for the same. Please have a look.

Was this ever implemented? I know that boots has a syslog server currently, but wasn't sure if workers were sending data to it.

rgl commented

@tstromberg, I was able to do something about this. Have a look at tinkerbell/hook#86 (comment).

Archiving repo, closing issue per GitHub's recommendation.