lykmapipo/kue-scheduler

Lock expiry can potentially cause job to run twice

Roam-Cooper opened this issue · 1 comments

In kue-scheduler 0.6.3 (and higher versions as far as I know), it seems as though the test that the key expiry listener used to determine whether it should run "_onJobKeyExpiry" for the expired key is also satisfied by lock key expiries.

The test is:
queue._jobExpiryKeyValidator = new RegExp('^' + queue.options.prefix + ':scheduler:');

Which a lock key of q:scheduler:locks:unique_task_name would meet.

I am working on a pull request to resolve this issue, I just have to look into our deployment a bit further as the root of the issue is a lock not being unlocked, and the expiry of the lock passes as a scheduler expiry because the regex allows it.

Seems in our deployment it was causing the task to run twice, because when the lock key expires and passes for a scheduler expiry, it gets a lock on the expired lock key, and that lock will expire after 1000ms and also pass for a scheduler expiry. After all is said and done the "scheduler key" (really the lock's key) is reset with the tasks interval time and the process repeats.