Tools for use in cluster environments.
build_mpirun_configfile- builds a
hostfileandconfigfilefor use with MPICH3 mpirun - used as a workaround to set the hosts and CUDA_VISIBLE_DEVICES environment variable properly on the cbio-cluster (both hal and lila), and on the the Merck GPU cluster (SLURM based)
- builds a
monq.py- Provides an interface to the XML returned by the command "qstat -f"
- Allows custom XPath queries
- Currently outdated
build_mpirun_configfile is installed as a command line command and has a very simple usage syntax:
build_mpirun_configfile [--configfilepath {CONFIGFILE}] [--hostfilepath {HOSTFILE}] [--nocheckmpich] COMMANDS--configfilepathsets the name of the Multi-Process Multi-Data (MPMD) settings (i.e. the config of each MPI task). Default:configfile--hostfilepathsets the name of the host file which houses the list of hosts, 1 line per process, repeating as required and determined automatically from the scheduler. Default:hostfile--nocheckmpichis a flag that, if set, ignores determining the MPICH version on disk. Right nowclusterutilscan only build against MPICH3 versions. The configuration and host files will still be built if this check fails, but a warning is raised.COMMANDSis the complete command that will be executed by each MPI task. For instance, if you are using a Pythons script, your command may look likepython myscript.py.
The output from build_mpirun_configfile will be a host file, and a configuration file. The way to execute your MPI
job from these files is the following:
mpiexec.hydra -f hostfile -configfile configfileThis assumes default options and the hydra MPI variant. Replace parts of this command with what is appropriate to your
environment.