vpenso/prometheus-slurm-exporter

Question: Where to install?

Closed this issue · 2 comments

jeevadotnet commented

Can someone please point me on to where exactly the exporter must be installed? Login Node? Controller Node? Worker Node? or on all Worker nodes?

Regards

mtds commented

You can place this exporter wherever you want: the controller node can be an option but basically any node in your cluster which is able to issue the sinfo, squeue and sdiag commands can be
used. I would probably avoid the worker nodes, since by definition they can come and go pretty quickly.

The important thing is only one: the exporter internally will try to execute the commands which I have
listed above and use them to query your Slurm controller, wherever it's installed with whatever
configuration you have.

It is necessary to install only one instance but nobody stops you to install multiple ones. It will be
basically be idle unless it's contacted by a client: in most cases the Prometheus server but you
can also decide to consume the metrics via cmd line using (for example) curl.

As a basic recommendation, follow the advice under the "Prometheus Configuration for the SLURM exporter" section in the README: it would be better not to call the exporter too often otherwise
you may risk to overload your controller while continually requesting new data.

Regards,
Matteo

  • 👍1
jeevadotnet commented

Hi Matteo

Thanks for the feedback.

I see our login node can run all the commands as you stipulated so I will install it on there.

Regards