WIPACrepo/pyglidein

Allow client to "claim" jobs

Opened this issue · 3 comments

from slack:

e.g. if the client reports that it has just submitted 10 jobs with 1 CPU and 16 GB RAM each, then the server could deduct that from the state it distributes to other clients

that of course assumes that the jobs will actually run at some point. might need to apply a degree-of-belief scaling to prevent one site from over-promising and starving others

Probably also need some timeout, before a job goes back to being "available"

You'd be surprised. It happens quite a lot that we're in at least the 4 GB/core regime, and I've seem batches of jobs in the 20 GB/core regime.