/activejob-uniqueness

Unique jobs for ActiveJob. Ensure the uniqueness of jobs in the queue.

Primary LanguageRubyMIT LicenseMIT

Job uniqueness for ActiveJob

Build Status Gem Version

The gem allows to protect job uniqueness with next strategies:

Strategy The job is locked The job is unlocked
until_executing when pushed to the queue when processing starts
until_executed when pushed to the queue when the job is processed successfully
until_expired when pushed to the queue when the lock is expired
until_and_while_executing when pushed to the queue when processing starts
a runtime lock is acquired to prevent simultaneous jobs
while_executing when processing starts when the job is processed
with any result including an error

Inspired by SidekiqUniqueJobs, uses Redlock under the hood.

Installation

Add the activejob-uniqueness gem to your Gemfile.

gem 'activejob-uniqueness'

And run bundle install command.

Configuration

ActiveJob::Uniqueness is ready to work without any configuration. It will use REDIS_URL to connect to Redis instance. To override the defaults, create an initializer config/initializers/active_job_uniqueness.rb using the following command:

rails generate active_job:uniqueness:install

Usage

Define uniqueness strategy for your job via unique class method:

class MyJob < ActiveJob::Base
  unique :until_executed

  # Custom expiration:
  # unique :until_executed, lock_ttl: 3.hours

  # Do not raise error on non unique jobs enqueuing:
  # unique :until_executed, on_conflict: :log

  # Handle conflict by custom Proc:
  # unique :until_executed, on_conflict: ->(job) { job.logger.info 'Oops' }

  # The :until_and_while_executing strategy supports extra attributes for a runtime lock:
  # unique :until_and_while_executing runtime_lock_ttl: 10.minutes, on_runtime_conflict: :log
end

ActiveJob::Uniqueness allows to manually unlock jobs:

# Remove the lock for particular arguments:
MyJob.unlock!(foo: 'bar')
# or
ActiveJob::Uniqueness.unlock!(job_class_name: 'MyJob', arguments: [{foo: 'bar'}])

# Remove all locks of MyJob
MyJob.unlock!
# or
ActiveJob::Uniqueness.unlock!(job_class_name: 'MyJob')

# Remove all locks
ActiveJob::Uniqueness.unlock!

Test mode

Most probably you don't want jobs to be locked in tests. Add this line to your test suite (rails_helper.rb):

ActiveJob::Uniqueness.test_mode!

Logging

ActiveJob::Uniqueness instruments ActiveSupport::Notifications with next events:

  • lock.active_job_uniqueness
  • runtime_lock.active_job_uniqueness
  • unlock.active_job_uniqueness
  • runtime_unlock.active_job_uniqueness
  • conflict.active_job_uniqueness
  • runtime_conflict.active_job_uniqueness

And then writes to ActiveJob::Base.logger.

ActiveJob prior to version 6.1 will always log Enqueued MyJob (Job ID) ... even if the callback chain was halted. Details

Testing

Run redis server (in separate console):

docker run --rm -p 6379:6379 redis

Run tests with:

bundle
rake

Use wwtd to run test matrix:

gem install wwtd
wwtd

Sidekiq adapter support

ActiveJob::Uniqueness patches Sidekiq API to unset locks on jobs cleanup. Starting Sidekiq 5.1 job death also triggers locks cleanup.

Contributing

Bug reports and pull requests are welcome on GitHub at https://github.com/veeqo/activejob-uniqueness.

License

The gem is available as open source under the terms of the MIT License.

About Veeqo

At Veeqo, our team of Engineers is on a mission to create a world-class Inventory and Shipping platform, built to the highest standards in best coding practices. We are a growing team, looking for other passionate developers to join us on our journey. If you're looking for a career working for one of the most exciting tech companies in ecommerce, we want to hear from you.

Veeqo developers blog