Scale your dyno workers on Heroku as needed, pay only for what you use!
Add this line to your application's Gemfile:
gem 'dyno_scaler'
And then execute:
$ bundle
Or install it yourself as:
$ gem install dyno_scaler
Just include this module in your Resque job and you're good to go:
class MyJob
include DynoScaler::Workers::Resque
...
end
If you are using Sidekiq, then just configure the middlewares:
# Client middleware
Sidekiq.configure_client do |config|
config.client_middleware do |chain|
chain.add DynoScaler::Workers::Sidekiq::ClientMiddleware
end
end
# Server middleware
Sidekiq.configure_server do |config|
# Configure client middleware here if your jobs may enqueue another jobs
config.client_middleware do |chain|
chain.add DynoScaler::Workers::Sidekiq::ClientMiddleware
end
config.server_middleware do |chain|
chain.add DynoScaler::Workers::Sidekiq::ServerMiddleware
end
end
You can access the configuration with (for example):
DynoScaler.configuration.max_workers = 3
In Rails, you can access it easily in your application.rb:
config.dyno_scaler.max_workers = 3
If you want to scale up or down your workers manually, you can use the manager directly:
DynoScaler.manager.scale_up(options)
DynoScaler.manager.scale_down(options)
You must pass an options hash with the number of workers, the number of pending jobs, and the number of running jobs, like so:
{
workers: 10,
working: 3,
pending: 5
}
Resque.info
returns a hash with these keys, so you may just pass it instead:
DynoScaler.manager.scale_up(Resque.info)
You can also use the DynoScaler::Manager#scale_with
method, passing the Resque.info
:
DynoScaler.manager.scale_with(Resque.info)
It will check whether to scale up or down based on the number of workers running, pending jobs, and working jobs.
For Sidekiq you can use DynoScaler::Workers::Sidekiq.info
method instead of Resque.info
.
When deploying to heroku you'll want to add these two configuration keys:
HEROKU_OAUTH_TOKEN=<YOUR-OAUTH-TOKEN-HERE>
HEROKU_APP=<THE-NAME-OF-YOUR-APP-ON-HEROKU-HERE>
They are used by the platform-api gem to scale dynos of your application.
Whenever DynoScaler needs to scale workers up it will perform a request to the
Heroku API. This request may sometimes take longer to return than one would want.
Because of this we have a async option that uses
GirlFriday to handle this call
asynchronously. To enable it, just set it to true
:
config.dyno_scaler.async = true
You can also give it a block to better customize it. It will receive an options
hash that can be passed to the DynoScaler::Manager#scale_with
method, like so:
MY_QUEUE = GirlFriday::WorkQueue.new(:my_queue, size: 5) do |options|
DynoScaler.manager.scale_with(options)
end
config.dyno_scaler.async do |options|
MY_QUEUE << options
end
- Fork it
- Create your feature branch (
git checkout -b my-new-feature
) - Commit your changes (
git commit -am 'Add some feature'
) - Push to the branch (
git push origin my-new-feature
) - Create new Pull Request