remove auto calls to co_shutdown(); expose an explicit shutdown() method on schedulers
Closed this issue · 1 comments
Before we introduced nested schedulers, the logic of co_shutdown()
was that (Pure)Scheduler.co_shutdown() would primarily send co_shutdown()
on all component jobs.
In a flat world, the intention was that to notify all job instances at the end of the main scheduler.
With nested schedulers OTOH, there is a need to alter this protocol, because it would now mean that jobs receive the co_shutdown
method when the nested-most scheduler ends its work, i.e. way before the toplevel scheduler is done.
In the context of apssh
where co_shutdown()
is used to close ssh connections to the nodes, this has devastating effects; users currently need to set keep_connection=True
on their SshJob
instance to work around this problem.
This being said: maybe this automatic housecleaning business is more harmful than helpful.
One option is to simply remove the automatic calling to scheduler.co_shutdown() at the end of the scheduler, and to expose this as a separate method that must be explicitly called by the library user.
As the title reflects, we're going to go for the latter option; this means
- removing automatic invokations of (co_)shutdown in (co_)run
- adding a proper shutdown() method on schedulers - a synchronous wrapper just like run() - returns nothing
- tweaking the documentation in apssh accordingly;
- adding a note on that in the R2lab tuto ?