/para

Script to control running parallel jobs in terminal

Primary LanguageShell

This single scipt without dependencies can distribute terminal jobs over cores and manage the maximum number of jobs that are running in parallel.

How to run

  • Jobs are read from a text file (one job per line).
  • The maximum number of jobs to be run simultaneously needs to be specified.

Flags

  • -c Text file with commands (one per line)
  • -d Log file directory
  • -p Number of processors (cores) to be used
  • -k Kill switch to terminate all running jobs

Features

  • Jobs will be terminated on first error via 'bash -e'.
  • Job standard output and error are recorded to a log file, one per job, that is stored in a user defined folder. Proces ID, exit status and run time are also recoreded.
  • After all jobs have finished an overview report will be created in the log folder that shows the exit status and the runtime per job.
  • Jobs that failed to run are printed to the terminal after para has finished to run.
  • Jobs that are invoked by para can be killed all at once by quiting para and running para again with the kill flag (-k) and specyfing the log folder (-d).

Platforms This script has been tested on OSX and Debian.

Example

para -c /Volumes/Data/joblist.txt -d /Volumes/Data/Project -p 10

Screenshot

Screenshot