Support dependencies among jobs
greefea opened this issue · 2 comments
What is the best way to do job dependencies with saq, like in rq? In rq, users can specify a depends_on argument when creating a job, and only enqueue the job if the dependencies are finished. I had thought I can hack the before_enqueue and/or before_process hook for this, but it looks like the job will be enqueued/processed no matter what happens in the hook. If anyone can help point me to a good way to support it with saq, I can write the code and do a PR.
BTW, thanks for a fast, light-weighted, yet powerful package!
depends_on / DAG is something i've explicitly not added due to the complexity involved. i'd be open to a very light weight solution for it though. you probably need to do modify def enqueue to check for dependent jobs,
if dependent jobs are active, or queued, they cannot be enqueued, otherwise they can. you'll also need to make sure they don't start processing.
happy to chat about it on slack https://tobikodata.com/slack