Yelp/mrjob

max_clusters_in_pool option

Closed this issue · 0 comments

We should add an option to limit the number of clusters in a pool. If a pooled job can't find a cluster to join and the pool already has max_clusters_in_pool clusters active, the job will wait rather than launching its own cluster.