sequential job execution
tbrand opened this issue · 3 comments
Currently all jobs at depends_on are executed concurrently.
But sometimes we want to do it sequentially.
(The idea is needed at least here)
You may know you can do it like
some_job:
depends_on:
- sub_job1
sub_job1:
depends_on:
- sub_job2
But when we remove the sub_job1 from it, the pipeline will be broken.
So it's not good at the view point of flexibility and stability.
How about adding this function to neph.yaml?
some_job:
depends_on:
- sub_job1
- sub_job2
seq: true
When the seq config is true, the series of jobs on depends_on will be executed sequentially.
The default is false.
What do you guys think?
I had this idea also, and I agree.
I am now working on the new config format (which is not pushed yet). It will be included in it. Unfortunately, the progress is slow, because it requires very careful designing and coding if we want to get very good quality software at the end (I am rewriting it from scratch, because it was written with wrong programming practices). I don't want to spend time on the current version, because it would delay the new branch more.
If you can't wait for the new version, you can implement this feature in the current.
Awesome.
Yup, I can wait for it. Thanks.