Resque Director¶ ↑

Resque Director is a plugin for the Resque queueing system (github.com/defunkt/resque) that “directs” workers on a queue by automatically adding or removing workers from a queue based on how backed up a queue becomes, or how long it takes for a job to get pulled off the queue.

About¶ ↑

Resque-director is useful for when you are managing a large number of workers and don’t want to waste resources keeping all of them waiting when they are not being used. For jobs that are high priority or time sensitive you can have it autoscale workers based on how long it takes to go through the queue, or how big the queue becomes. In queues where the influx of jobs can change dramatically from time to time, resque director automatically scales more workers during the times when the queue is filling up more quickly, and less in the opposite scenario. Different queues can be given different directions as well.

Usage¶ ↑

When creating your jobs you should extend Resque::Plugins::Director and add direction options.

For Example:

class Job
  extend Resque::Plugins::Director
  direct :min_workers => 2, :max_workers => 4, :max_time => 60, :max_queue => 10, :wait_time => 30
  @queue = :test

  #rest of your Job class here
end

Configuration Options¶ ↑

min_workers: specifies the minimum number of workers running at any point in time. If there are no workers running or less than the minimum running it will start as many workers necessary to get it to the minimum. The default is 1.
max_workers: specifies the maximum number of workers running at any point in time. It will never start more than the maximum number of workers. If anything less than or equal to zero is specified as the maximum it will be treated as if there is no maximum, and theoretically an infinite number of workers could be started. The default is 0.
max_time: the maximum time in seconds that a job takes to go through the queue, if it takes longer than this time then a worker is started. If anything less than or equal to zero is specified as the maximum time, this field will be ignored. The default is 0.
max_queue: the maximum jobs that can build up in a queue, if more than this number of jobs build up then a worker is started. If anything less than or equal to zero is specified as the maximum queue, this field will be ignored. The default is 0.
wait_time: the time in seconds that it will wait after adding or removing a worker before being allowed to add or remove workers again. The default is 60 seconds.

Conditions For Starting Workers¶ ↑

A worker will be started if the queue length is greater than max_queue or if the time it takes a job to go through the queue is greater than max_time. Also a worker will only be started if the time since the last scaling is greater than wait_time. Workers will not be started if there are already the max_workers number of workers. By default resque-director allows you to have zero workers running, when a job is enqueued then workers will be scaled within the max/min requirements you set. If there is not a single worker running, then the minimum number of workers will be scaled up (one worker will be scaled up if the minimum is zero).

Conditions For Removing Workers¶ ↑

A worker will be removed if the jobs in the queue fall below half of the max_queue, and if the time it takes for a job to be pulled off of a queue falls below half of the max_time. Workers will be scaled down to the minimum if there are no jobs on the queue. Workers will not be stopped if there are only min_workers number of workers. Also a worker will only be stopped if the time since the last scaling is greater than wait_time.

Special Cases¶ ↑

If a max_worker is less than min_worker then the default for max_worker will be used (there will be no maximum).
If a min_workers is set to anything less than 1 then it will be treated as 0.

Advanced Options¶ ↑

The above options are plenty to handle the basic scaling of workers. The options below allow the user much more control over the way in which workers are scaled, the information tracked, and the times when a worker is scaled.

Start/Stop Options¶ ↑

start_override: You can set this option if you want to override the way to start a worker. This option takes a lambda closure that accepts the queue as an argument, the block you pass in will be responsible for starting a SINGLE worker allowing you to fully customize the starting of a worker. If your block returns false then that signifies that a worker was not started and the last scaled time will not be set. The default way a worker is started is with the system command: “QUEUE=queue_name rake resque:work &” where “queue_name” is the queue on which the job is running.
stop_override: You can set this option if you want to override the way to stop a worker. This option takes a lambda closure that accepts the queue as an argument, the block you pass in will be responsible for stopping a SINGLE worker allowing you to fully customize the stopping of a worker. If your block returns false then that signifies that a worker was not stopped and the last scaled time will not be set. The default way a worker is stopped is that a QUIT signal is sent to a worker process.

Customized Starting/Stopping Workers Example¶ ↑

class Job
  extend Resque::Plugins::Director

  start_block = lambda{|queue| ... }
  stop_block = lambda{|queue| ... }
  direct :start_override => start_block, :stop_override => stop_block
  @queue = :test

  #rest of your Job class here
end

Logging¶ ↑

To add director logging you can pass a logger as an option. It will prepend all log messages with “DIRECTORS LOG:” and will log when scaling up or down a worker. It will also log when a worker wants to scale up or down but is unable to do so due to the fact that the maximum or minimum number of workers has been reached for that queue.

Logging Example¶ ↑

class Job
  logger = Logger.new('logfile.log')
  extend Resque::Plugins::Director

  direct :logger => logger, :log_level => :info
  @queue = :test

  #rest of your Job class here
end

Logger Options ¶ ↑

logger: This will set the logger that will be used. It takes a Logger from the Ruby Standard Library. If this is not set the director will not write to any logs.
log_level: This sets the level to log at as a symbol. The default level is :debug.

Scale Options¶ ↑

no_enqueue_scale: When a job is enqueued workers will be scaled within the max/min requirements you set, if there is not a single worker running then the minimum number of workers will be scaled up (one worker will be scaled up if the minimum is zero). The worker itself handles the rest of the scaling. If you do not want scaling on enqueue and want to have the workers perform all the scaling then you can set this option to true, however if you do this you must be responsible for making sure that at least one worker is running on the queue. The default is false.

Multiqueue Options¶ ↑

queue: The queue name(s) that you will be scaling and using to calculate the number of workers working. The default is the queue name of the Job.

Multiqueue Example¶ ↑

class Job
  direct :queue = [:test, :test2]
  @queue = :test

  #rest of your Job class here
end

Multiqueue¶ ↑

resque-director will still look at the job’s queue to determine when to scale. However if you set the queue option it will scale a worker working on that queue, or a worker working on multiple queues if you pass it an array of queues.

Requirements¶ ↑

resque-director requires resque ~> 1.10.
resque-director may be incompatible with gems that modify resque’s push/pop functionality.

Contributing to resque-director¶ ↑

Check out the latest master to make sure the feature hasn’t been implemented or the bug hasn’t been fixed yet
Check out the issue tracker to make sure someone already hasn’t requested it and/or contributed it
Fork the project
Start a feature/bugfix branch
Commit and push until you are happy with your contribution
Make sure to add tests for it. This is important so I don’t break it in a future version unintentionally.
Please try not to mess with the Rakefile, version, or history. If you want to have your own version, or is otherwise necessary, that is fine, but please isolate to its own commit so I can cherry-pick around it.

frausto/resque-director