natcap/taskgraph

Allow `TaskGraph` and `Task` to be pickled

richpsharp opened this issue · 2 comments

richpsharp commented

This will require implementing the __reduce__ method for these objects: https://docs.python.org/3/library/pickle.html#object.__reduce__

Note this feasible since TaskGraph can share the same database.

The tricky issue, if addressed, will be how to lock or detect duplicate submitted tasks in separate TaskGraph objects.

Currently TaskGraph stores some global state, it may be worth offloading that state exclusively to its database.

phargogh commented

Interesting! Out of curiosity, what's the use case for this? Allowing TaskGraph objects to be returned from a Task?

richpsharp commented

Two things:

  1. allow Tasks to be passed to add_task as arguments so the Task can call get at runtime and,
  2. allow TaskGraphs to be passed to Tasks so add_task could be called at runtime as well.
  • 👍1