IDSIA/sacred

Serialise the run.Run

GoingMyWay opened this issue · 4 comments

GoingMyWay commented

Hi, it seems the run.Run obj is not serializable, will you plan to make it serializable?

thequilo commented

This is related to #874. Why do you want to serialize run.Run objects? Do you want to store them after the experiment finished (output of ex.run) or before it started?

GoingMyWay commented

This is related to #874. Why do you want to serialize run.Run objects? Do you want to store them after the experiment finished (output of ex.run) or before it started?

@thequilo, hi, I am using ray (ray.io) and I want to create a copy of the run.Run so that other processes can use it and save results.

thequilo commented

So, you use it after the run finished?

There are currently three things that prevent it from being pickled (that I can see). First is the captured function (see #874), second is the MetricsLogger that internally uses a queue which is not pickleable and third is the heartbeat timer. I don't have a good solution for the captured function yet (It loses its config when pickled and loaded in another process). This wouldn't matter when the experiment is already finished. The MetricsLogger and heartbeat timer are also only required during the run of the experiment. If you only need the final run object, you can set these to None (run._metrics = run._heartbeat = run.main_function = None) as a workaround to make it pickleable.

The question is then, what do you have from the run object that an observer doesn't give you?

GoingMyWay commented

So, you use it after the run finished?

@thequilo, Hi, I am using it while training. I want to use it just to record the data.