Enable the use of `requirements.txt` files to specify stage dependencies
AlexIoannides opened this issue · 0 comments
AlexIoannides commented
"As a ML Engineer, I would like to be able to specify the requirement for a pipeline stage by referencing a requirements.txt file, so that I do not have to specify requirements in YAML that can be very clunky and lead to bloated bodywork.yaml files."
Tasks
- Extend the YAML schema defined in
bodywork.config
so that a path to arequirements.txt
files can be specified for each stage's dependencies, that overrides any other dependency. - Extend the validation of
bodywork.yaml
so that it can check for the existence of anyrequirements.txt
files. - Extend
bodywork.stage_execution
so that it can install requirements from therequirements.txt
files provided in the config. - Add unit tests to cover this new functionality.