geeklearningio/gl-vsts-tasks-inception

queue and wait feature

mscaldas2012 opened this issue · 5 comments

Hi,
this is a great plugin to be able to pipeline jobs on TFS.
Sometimes, I need to wait for the queued job to finish before continuing with the build that called it.

will it be possible to implement such functionality?

thanks,

This is definitely achievable, and I had it in mind, but this present a high risk of deadlock because of pipeline licenses. In such cases I find that chain 3 build is less risky.

Any thoughts about how to mitigate this issue if we are to proceed with this?

cc @asiffermann

I am wondering if we couldn't do even better!
Maybe we could implement a sort of build scheduler, that would be defined outside of build processes, and would listen to triggers (just as the services hooks) to queue the next builds?

I think I got lost... @sandorfr what's the problem with pipeline licenses? what do you mean chain 3 build is less risky?

thanks and sorry for being slow !

@mscaldas2012 the point is, if you have only one Build and Release concurrent pipeline in your VSTS account, you would create a deadlock:

  • Build1 is executed (the pipeline is considered as "In progress")
  • Build1 queues Build2 with our task, and waits for it being completed
  • Build2 cannot be executed, because the only pipeline is being busy with Build1, so it is queued
  • Build1 never finishes, waiting for Build2
  • Deadlock 😞

And we cannot "predict" how many concurrent pipelines you would need to achieve a particular build process...

About the workaround of the 3 builds, I suppose @sandorfr meant that instead of doing:

  • Build1 does some work, then queues Build2, and waits for it
  • Build2 does some work, and finishes
  • Build1 goes on with final work

You got:

  • Build1 does some work, then queues Build2, and finishes
  • Build2 does some work, then queues Build3, and finishes
  • Build3 does the final work

It is not you being slow, it is the subject that is a little bit tricky! 😃

ah! of course... i see what you mean by pipeline licenses now :-) Unfortunately the 3 build scenarios does not help me, because my intent is to reuse tasks. Implementing 3 builds will not allow me to reuse tasks. I need builds A, B and C to all execute Build X as a first step. so If I make Build X call the other builds, it can only call one of them, which will force me to have Build Xa to call Build A, Xb to call Build B, etc. ... purpose defeated :-(