A way to schedule many instances of a recurring task with different args

[zavan]

I've been using the whenever gem to do something like for a lot of my recurring jobs:

every 3.hours do
  runner "UpdateCustomerStatsJob.enqueue_all"
end

class UpdateCustomerStatsJob < ApplicationJob
  def self.enqueue_all
    Customer.in_batches do |batch|
      jobs = batch.map { |customer| new(customer) }
      ActiveJob.perform_all_later(jobs)
    end
  end

  def perform(customer)
    customer.update_stats
  end
end

I wanted to use solid_queue recurring tasks to do this. However since it only allows passing a job class on which it always calls perform_later, if I want to do this I think I have to create a separate job responsible solely for enqueuing the individual jobs:

class EnqueueAllUpdateCustomerStatsJob < ApplicationJon
  def perform
    Customer.in_batches do |batch|
      jobs = batch.map { |customer| UpdateCustomerStatsJob.new(customer) }
      ActiveJob.perform_all_later(jobs)
    end
  end
end

Do you think there could be a simpler way to do this with solid_queue? I guess the difficult part would be to avoid duplicate jobs in case of multiple dispatchers (for 'whenever' I only run it on one host right now and I use 'activejob-uniqueness'). Not sure how it could handle doing things in batches also (I'm thinking I don't want to create hundreds of thousands of job instances in memory before enqueuing them), maybe some yielding mechanism.

I don't know if it's just me doing things this way or if it could be useful for more people, but I guess that's the only way to know :)

[rosa]

Hey @zavan! Yes, I think that'd be the way 🤔 Perhaps if I had more cases like UpdateCustomerStatsJob, I'd do something more generic like:

class RunnerJob < ApplicationJob
  def perform(command)
    eval(command)
  end
end

      recurring_tasks:
        update_customer_stats:
          class: "RunnerJob"
          schedule: "every 3 hours"
          args: "UpdateCustomerStatsJob.enqueue_all"

but if it's only one case, I'd go with your solution, I think.

[idrista]

Hello, I agree with @rosa, take a look at the configuration of simple_scheduler is very simple and clear. https://github.com/simplymadeapps/simple_scheduler

[zavan]

I will go that route. Thanks!