Tinkoff/Horarium

Add ability to throttle job

PDmatrix opened this issue · 1 comments

PDmatrix commented

Currently, the only way to decrease the load on the database is to increase IntervalStartJob, but there might be problems with it
I suggest that we add another setting JobThrottleSettings that will dynamically increase IntervalStartJob if there are no jobs available for a certain period

Algorithm:

  1. Try to get jobs with IntervalStartJob
  2. If there is no jobs after JobThrottleSettings.JobRetrievalAttempts we increase IntervalStartJobs by JobThrottleSettings.IntervalMultiplier
  3. To prevent massive IntervalStartJob, there will be JobThrottleSettings.MaxJobThrottleInterval so we won't wait too long
  4. After we successfully retrieve a job, IntervalStartJob will reset to its original value
Bobreshovr commented

it's a good idea, add one comment to PR.
What about the load test of this throttle?