Add manager._start_time to state_dict
linshokaku opened this issue · 1 comments
linshokaku commented
pytorch-pfn-extras/pytorch_pfn_extras/training/manager.py
Lines 642 to 644 in 2dcd7d7
pytorch-pfn-extras/pytorch_pfn_extras/training/manager.py
Lines 233 to 239 in 2dcd7d7
When restarting from a snapshot, _start_time
is initialized and elapsedtime
cannot be obtained correctly, so _start_time
should be treated as state.
kmaehashi commented
If we are to fix this, I think it's better to record elapsed_time
in state dict and accumulate the "real" time consumed (i.e., excluding the time process is stopped)