runtime and parallelization

[antonkratz]

Going through the example from the readme. When I run the second snakemake all command, it initiates a process which seems to be extremely long-running (3 of 7256 steps (0.04%) done ). At the same time sigma only seems to use one core (?). Is there a way of speeding up the process, enabling multi-core, of even even multi-CPU on a cluster?

[mdml]

@antonkratz: yes, you can use snakemake to run many jobs in parallel or on a cluster.

To run with multiple cores
You can specify the number of available cores with the -j flag. So, to run with 32 cores, you would execute:

snakemake all -j 32

To run on a cluster
You can run snakemake on a cluster (or in the cloud) using standard cluster engines. I refer you to the documentation for additional details.