amphp/parallel

The nested use of pools

hitriyvalenok opened this issue · 1 comments

Hello there! I am actively using your implementation and it works great! But I ran into an issue with the nested use of ContextWorkerPool. I have two tasks: Main and Sub. I create a pool with Main, Main creates a pool with Sub. The issue is its work looks like an absolute mess.

An example. I've created a pool with workers limited to one and submitted three Main tasks that create one Sub task each. It should be running sequentially, one after another. But the real behavior is: the first thread executed well but then the two remaining start simultaneously! Log of the example:

[2023-05-12 16-22-17.548300] ====== Start ======
[2023-05-12 16-22-17.614900] Main task 'main-1' started
[2023-05-12 16-22-17.669900] Sub task 'main-1 -> sub-1' started
[2023-05-12 16-22-20.670400] Sub task 'main-1 -> sub-1' end
[2023-05-12 16-22-20.671800] Sub task 'main-1 -> sub-1' done
[2023-05-12 16-22-20.671900] Main task 'main-1' end
[2023-05-12 16-22-20.673400] Main task 'main-2' started
[2023-05-12 16-22-20.674500] Main task 'main-3' started
[2023-05-12 16-22-20.717600] Sub task 'main-2 -> sub-1' started
[2023-05-12 16-22-20.717600] Sub task 'main-3 -> sub-1' started
[2023-05-12 16-22-23.718900] Sub task 'main-3 -> sub-1' end
[2023-05-12 16-22-23.718900] Sub task 'main-2 -> sub-1' end
[2023-05-12 16-22-23.719900] Sub task 'main-2 -> sub-1' done
[2023-05-12 16-22-23.720000] Main task 'main-2' end
[2023-05-12 16-22-23.720500] Sub task 'main-3 -> sub-1' done
[2023-05-12 16-22-23.720600] Main task 'main-3' end

As seen from the logs, at 16-22-20, just after main-1 had been done, main-2 and main-3 starts simultaneously! But if I run only one pool (Main or Sub - no matter), it runs ok.

The demo project with the implementation of the example above: project.zip

Thank you!

There was an issue with pushing workers both into the idle list and fulfilling a deferred, so potentially two tasks would execute on the same worker simultaneously. The issue has now been fixed. I'll tag a new release soon.