Rate limiting the number of processed jobs concurrently

[stefan-hc]

I.e. for a single-queue app that serves multiple tenants. It would be beneficial if Que would provide capabilities to limit/throttle the amount of jobs executed concurrently per given attribute (tenant). This would help in avoiding situations when one tenant generates occasionally a lot of Jobs that would otherwise starve other tenants from their Jobs being processed.

Something like i.e. https://github.com/sensortower/sidekiq-throttled adds for Sidekiq

[ZimbiX]

I wonder if there would be another approach better suited to this use case - a way of dividing up the available CPU time, effectively, between tenants. So there's no impact on throughput while only one tenant is producing jobs

[stefan-hc]

In my particular case: unlikely. We're running our app on Heroku stack and our Que workers are running on separate dynos (autoscaled up and down according to daily traffic). I don't see how in such environment divide the CPU time among dynamic number of workers, but I guess in single-queue-single-worker-machine envs it could work...

[ZimbiX]

Sorry, I didn't mean CPU time exactly. Theoretically, which would be preferable (for tenants that have jobs): spending the same amount of time on each tenant, or processing the same number of jobs for each tenant, or something else? I suspect this is a solved problem and algorithms for it would be around.

[jasoncodes]

Something akin to what kernels do to try and fairly manage CPU and disk I/O sounds like an ideal way to go. I’m guessing this would invoke tracking how much worker time each tenant has used recently and weighting the priority of their jobs based on this.

This seems like a middleware could be used to record usage and a periodic job could adjust the priorities of waiting jobs. One would not want to lose the original job priority but this could be backed up elsewhere if there’s no built-in support for this.

[ZimbiX]

Thanks, @jasoncodes; interesting idea!

[stefan-hc]

which would be preferable (for tenants that have jobs): spending the same amount of time on each tenant, or processing the same number of jobs for each tenant, or something else?

Honestly, I have no idea 🤷 . I'm inclined to say the "amount of time for each tenant" as it seems more natural, but am not sure if in our particular app this would be the way to go. Perhaps "amount of jobs for each tenant" would suit our needs more: some of our jobs are reliant on remote services (REST calls, etc) and so the job processing time for a single tenant could vary outside of our "jurisdiction". Time spent alone might be not the most just criteria for our tenants then. Hard to predict for me.

Thanks for your ideas, I appreciate you taking time to share different perspective(s).