georgemilosh/Climate-Learning

It is a bad idea to write to home directory commit #d1ba925cbddd69d00bd297c4cdfc90efc8a5f1f5

Closed this issue · 0 comments

I had an error because the trainer/scheduler were not able to delete a non existing .current_run while the run was finishing. I bet what happened was that another parallel run, executed on a different machine with the same home I was performing, crashed it by deleting the file earlier.

I know that the file is small but I prefer not to write to home directory period, especially since it may cause issues when two parallel machines share home