breamware/sidekiq-batch

Negative pending on some batches

Closed this issue · 2 comments

We used this gem to handle batches.

For some batches the success callback does not run, I checked those batches and the pending is negative. Any idea why the pending is negative?

This is what we get from redis

{"description"=>"Description", "created_at"=>"1714276864.5737576", "pending"=>"-9913", "children"=>"0", "total"=>"9913"}

I checked the code and the success/complete callback runs when pending is 0.

After some debugging found the reason for negative pending.

If a job is scheduled inside the batch. job, it will trigger a process_successful_job

So you have

batch.do 
  FirstJob.perfom_async
end


class FirstJob
  include Sidekiq::Worker

  def perform
    SecondJob.perform_async 
  end
end

process_successful_job will be called after FirstJob and after SecondJob, while only the FirstJob is added to pending.

Stale issue message