ClusterPoolSH
Author
Forrest Koch (forrest.c.koch@gmail.com)
Description:
This script allows the user to initialize multiple pools of workers drawing from a shared queue. It is intended to be used alongside schedulers like SGE or PBS when the user needs to submit more jobs than the queuing system will allow.
Pools will continue running jobs until receiving a SIGTERM or SIGINT which is normally sent after a job exceeds it's walltime. Each running job will then be terminated and rescheduled.
Usage:
./cluster-pool.sh [Options]
Options:
-h|--help Show this help message
Working Directory Options:
-i|--init Initialize working directory
-w|--workdir [PATH] Set working directory path (Default: pwd)
-b|--batches [N] Set number of batches (Default: 1)
Pool Options:
-p|--init-pool [NAME] Initialize new pool with name [NAME]
-n|--max-workers [N] Set poolsize (Default: 4)
-t|--max-time [N] Don't schedule jobs exceeding this time (m)
-d|--degrade Degrade priority after reschedule
Job Options:
-a|--add \"COMMAND\" [NAME] Add command to queue *NEEDS \"s
-B|--set-batch Set the batch for command (Default: 1)
-P|--priority [1-9] Set priority of job (Default 4)
Example:
# Initialize current directory as a working directory
# with 8 batch folders
./cluster-pool.sh -i -b 8
# Schedule some jobs
# Note that the command is enclosed in quotes
for i in $(seq 1 1000); do
batch=$(( i%8 + 1 ))
./cluster-pool.sh -a "bash script.sh arg1 arg2 ..." [Job Name] -B $batch
done
# and Start a pool with 8 workers
./cluster-pool.sh -p Pool-1 -n 8Notes on usage:
- Jobs should be divided into batches to prevent directories with too many files.
- Jobs and pool should be given unique identifiers -- behaviour is undefined otherwise.
- When adding a command to be run, remember to wrap it in quotes to pass it as a single argument.
- The degrade option will degrade the priority of a job each time it is rescheduled. If priority is 9 before rescheduling, the job will not be rescheduled.