Feature request: allow throttled workers to have different cost

[RKushnir]

Let's say an API has a limit of 100 calls per minute. And there's 2 kinds of jobs — one makes 2 calls, the other makes 3.

To calculate the guaranteed limit I need to use the higher value, so the threshold will be 33 jobs/min. But this is obviously inefficient when most of the jobs are of the first kind(2 calls), in the worst case reducing the effective limit to 66 calls/min instead of 100.

So, it would be nice to be able to specify how much each job costs. Ideally, with an option to dynamically calculate the value.

[ixti]

Oh. That sounds pretty interesting! I'll need to think about it. In fact I'm already slowly working on changing the throttling code (making it separated of sidekiq sot that sidekiq-throttled will use it eventually) - but that's a slowly going process as here in SensorTower we don't need this ATM.

But the idea you described seems pretty interesting to me. At least at this very moment. :D

[ixti]

Can you describe in pseudo code how do you see it?

[RKushnir]

I haven't ever written a sidekiq middleware, so I have no idea what I'm talking about :)

Anyway, as I understand the current implementation, it keeps a list of the timestamps when previous jobs where started. And when there's a new job, it counts the timestamps to check if the limit has not been exceeded, in pseudo-SQL:

if (select count(*) from jobs where jobs.timestamp > 1.minute.ago) < 100 then execute

With my suggestion, it would store a pair (timestamp, job cost), and the calculation would be:

if (select sum(jobs.cost) from jobs where jobs.timestamp > 1.minute.ago) + new_job.cost <= 100 then execute