job().get() not working

[carlos-encs]

Details

slurm-22.05.6
Python 3.9.1
Cython 0.29.33
pyslurm 22.5.1
Linux Scientific Linux 7

Issue

Hi Guys,

pyslurm.job().get() shows no results

import pyslurm
jobs = pyslurm.job()
jobs.get()
{}

The issue was already reported, but the thread was closed:

Thanks

[tazend]

Hi @carlos-encs

In Slurm, there are two different APIs to get Job information - live in-memory job data from the slurm controller and historical job data from the database. In Pyslurm, this is currently implemented via the pyslurm.job and pyslurm.slurmdb_jobs class, respectively.

Doing pyslurm.jobs().get() will only retrieve data for "recent" Jobs from the slurm controller.
In other words: This function cannot retrieve Jobs which have already completed for more than 180 seconds (by default). After a Job is completed, the slurm controller will still keep the data 180 Seconds in memory.

After this time period, the Job is purged from memory, and you can only query it again from the Slurm Job database - implemented in Pyslurm via pyslurm.slurmdb_jobs (which is what the issue you have linked is mostly talking about - a way to query historical job data)

---

At the moment you execute pyslurm.job().get(), are there any jobs showing up when you execute squeue?. If not, then the behaviour of pyslurm.job().get() is correct in this case and you'll need to use the pyslurm.slurmdb_jobs().get() function to retrieve historical job data.

[carlos-encs]

Hi @tazend

Thank you for the explanation, I now can understand better how PySlurm works.
I'll send some jobs to the scheduler and execute pyslurm.job().get() and see what happens.

My goal is to retrieve data from recent jobs and send it to influxdb and then create a dashboard on Grafana.

[tazend]

Hi @carlos-encs

alright - let me know if it works properly for you.