/SlurmCluster

:flashlight: Ansible playbooks to setup a multinode cluster with Slurm + MPI benchmarks + HPC containers

Primary LanguageShell

Slurm Cluster

Ansible playbooks to setup a multinode cluster with Slurm.

3 playbooks for the cluster initialization:

  • controllers (slurmctdl)
  • workers (slurmd)
  • NAS (for network storage)

+ 1 extra playbook for benchmarks initialization (http://mvapich.cse.ohio-state.edu/benchmarks/, mpiBench)

+ 1 extra playbook for HPC containers initialization (Singularity, CharlieCloud, Sarus, + DockerCE (for convenience))


The playbooks (controllers, workers, nas) will setup:

  • hostname
  • ssh connection
  • ntp synch
  • nfs server and client
  • Munge and key
  • Slurm (controller and worker)
  • helpers

It can be used on Azure with the CentOS-based HPC images (Rogue Wave Software / OpenLogic) to quickly deploy a cluster with MPI and pre-installed drivers for Infiniband (e.g. on HB60).

Setup Local

  • Install Ansible and deps:
virtualenv env
source env/bin/activate
pip install -r requirements.txt
  • Edit inventory as needed
  • Edit conf/hosts as needed the init controllers, nas and workers

Setup Single Components

Setup Controllers

  • Edit conf/slurm.conf as needed (info)
  • Edit conf/cgroup.conf as needed (info)
ansible-playbook -i inventory init_controllers.yml

Setup NAS

A folder called /home/user/workspace is created and exported. It is automounted on workers

If the NAS and the controller are on the same node, then run

ansible-playbook -i inventory init_nas.yml

otherwise run

ansible-playbook -i inventory init_nas.yml --extra-vars "external_nas=Y"

Setup Workers

ansible-playbook -i inventory init_workers.yml

Setup Container

The playbook will install Docker, Singularity, Charliecloud and Sarus

  • Edit init_containers.yml vars to set the containers release versions
ansible-playbook -i inventory init_containers.yml

Setup Benchmarks

  • Edit init_containers.yml vars to set the containers release versions
ansible-playbook -i inventory init_benchmarks.yml

Setup All

Execute all the playbooks with:

ansible-playbook -i inventory init_all.yml