Handle Global configuration properties
potiuk opened this issue · 0 comments
potiuk commented
It is related to #39 but it's about making sure that global configuration properties are passed to all tasks inside generated dags.
Most actions in a workflow need several of the same settings (e.g., the JobTracker URI, NameNode URI, Hadoop scheduler queue, etc.). Users can avoid repeating all these settings and job configuration values in every action of the workflow by defining them once in the workflow’s section. The properties defined in the section are available to all actions of the workflow.
Seems that might be probably somehow mapped to default_args in most cases, but then we have to make sure we map all the configuration properties from #39 to some arg: