Align cluster and job implementations
jafreck opened this issue · 0 comments
jafreck commented
Job and cluster implementations should be aligned to ease transition between the two modes for users. This will also decrease code complexity/duplication and facilitate support.
Currently a cluster is a: Batch pool and job.
A job is a: Batch job schedule with a job specification that has an autopool and predefined Tasks for each application.
Going forward, we should convert clusters to job specifications with autopool. So a cluster will be a job with an empty set of Batch tasks and keep_alive
set to True on the autopool specification.
API Changes:
- job configuration has a cluster configuration and a list of applications
Refactors:
- cluster is implemented as job with
keep_alive
set to false and no applications defined - add plugin support to jobs