Setting environment variables in workflow engine container
Opened this issue · 1 comments
miniwdl allows any of its configuration options to be overridden by environment variables (details). This is often a more-convenient way to tune them compared to editing a configuration file (also supported), especially in a cloud setting where the engine process will be running remotely. The existing engineOptions in the workflow manifest doesn't suffice because some of the options we'll be interested in tuning (e.g. AWS request rate limiters) are a little too obscure to be exposed through dedicated command-line options.
A cursory search of the Nextflow and Snakemake docs also suggest it may be useful to set environment variables for them.
Practically, the project YAML engine definition could have a sub-section with a map of environment variables to set in the engine container. For the engines using the head process model, these perhaps could be further overridden by some engineEnvironment map alongside engineOptions in the workflow manifest.
Greetings! Sorry to say but this is a very old issue that is probably not getting as much attention as it deserves. We encourage you to check if this is still an issue in the latest release and if you find that this is still a problem, please feel free to open a new one.