One consumer only for specific queue

[TarasBardiuk]

Hi, guys.

I need to fetch and process messages from one specific queue in a strict stream, messages should be processed one by one, I mean NOT in parallel. We need this to avoid parallel creation of records, only one by one.

I was trying to achieve that by local worker config. As a result, thread: 1 doesn't help because I still have 4 consumers (workers: 4 specified in a config yml file). Then I was trying to use local workers option in a specific worker class, but it doesn't work, I guess it is not supported at all.

Then I found Handling different workloads Wiki-page, but it didn't help as well.

1. I configured ./config/sneaker_worker_groups.yml file:

notparallelgroup:
  classes: MyWorkerClass
  workers: 1

2. Then I executed bundle exec rake sneakers:run task
   After this step, I assumed that now all my queues should be handled by 4 consumers (as by default), but one specific queue (managed by MyWorkerClass) by a single consumer only. But no, all queues where listened by 4 consumers. Okay, maybe I need to run spawner manually?
3. So I executed bundle exec ruby -e "require 'sneakers/spawner';Sneakers::Spawner.spawn" as mentioned in a Wiki page and... nothing changed. Every queue was still listened by 4 consumers.

Then I tried to create my own rake task to run it instead of default sneakers:run:

require 'sneakers'
require 'sneakers/runner'

task :environment

namespace :sneakers do
  desc 'Custom sneakers process start'
  task start: :environment do
    ::Sneakers.server = true
    ::Rake::Task['environment'].invoke
    ::Rails.application.eager_load!

    default, specific = ::Sneakers::Worker::Classes.partition do |classy|
      !classy.queue_opts.key?(:workers) || classy.queue_opts[:workers] ==
        ::Sneakers.const_get(:CONFIG)[:workers]
    end

    ::Sneakers::Runner.new(default, {}).run

    specific = specific.group_by { |classy| classy.queue_opts[:workers] }
    specific.each do |workers, classes|
      ::Sneakers::Runner.new(classes, workers: workers).run
    end
  end
end

But I found it impossible to be a single task only, because after the first #run method call, the rest of the rake task is not executed at all. So in such case I will have to define separate rake tasks for each local workers config option value.

It may be worth to say that on a server I need to start sneakers in a daemonize: false mode, but locally it is always daemonize: true.

Please help me get one specific queue to be processed by a single consumer only.

P.S. Why don't worker groups work? Or am I wrong and they are useless for my needs?

Thank you in advance.