ScalableCytometryImageProcessing/SCIP

Add pipeline and run configuration

Closed this issue · 0 comments

There is an important distinction between pipeline configuration and run configuration. The pipeline configuration concerns things like what channels to load, which segmentation algorithm to use on what channel, which features to compute... The run configuration concerns things like how many workers to use, how much memory each worker receives, what logging level to use...

Pipeline configuration should be implemented with a YAML config file that can be passed when launching the command line interface (CLI) on the command line. Example:

data_loading:
  file_format: multiframe_tiff
  channels:
    - 0
    - 1
    - 2
    - 4
    - 5
segmentation:
  noisy_channels:
    - 0
    - 5

The run configuration will be passed to the pogram through arguments of the CLI. Example

sip --n-workers 4 --worker-mem 20G

Argument handling is done with the click Python library.