DaniilBoiko/slurm-cheatsheet

slurm-cheatsheet

Helpful resources

• https://slurm.schedmd.com — official docs
• https://www.sherlock.stanford.edu/docs/user-guide/running-jobs/ — Stenford's Sherlock docs
• https://www.psc.edu/resources/bridges-2/user-guide-2-2/ — PSC Bridges-2 docs

Sructure of a file with a slurm job

#!/bin/bash
#SBATCH -n 8
#SBATCH -t 48:00:00
#SBATCH --gpus 1

source .bashrc

<your code>

List your tasks

squeue --me

Save current queue as JSON

squeue --json > jobs_2023-03-08.json

Listing available resources

sinfo -o "%20N  %10c  %20m  %30G %30P"

What are the job limits?

sacctmgr show qos

How to check GPU utilization on a specific machine?

You can SSH directly to any machine that is running any of your tasks.

Dumb questions section

Can I move job file after running sbatch <file>?

Yes, it will not affect submitted jobs. Scripts for submitted jobs are copied to another directory.