josegonzalez/cakephp-queuesadilla

Add ability to disable concurrent job workers

suhaboncukcu opened this issue · 2 comments

I'm using Queuesadilla for CakePhp to manage several heavy duty tasks as data extraction from excel and so. I run the shell command with a cron job with an interval. However, if there are more than one job listed in the queue; shell command starts the worker on the next job without waiting the previous one's ending or failure. Since I want to run shell command with a very small interval as 5seconds and some of the jobs take 3 mins to run completely; I want worker to wait for the previous job to complete and not start till then.

Is there a way to do it that I missed? If there is; I would gladly send a PR for the documentation after figuring out.

AFAIR, the worker doens't pick up another job if a current one is already being procesed, based on php's serial execution.
Do you mean you run sth like bin/cake queuesedialla ... with a cron every x hours/minutes?
If so, you should consider to just spawn one process and manage it with sth like supervisor.

The worker itself runs in an infinit loop, and just picks up new jobs if it doesn't process one job atm.

I would just use a process supervisor like:

  • init
  • upstart
  • systemd
  • circus
  • supervisord

Closing as this isn't something we should handle in queuesadilla.