Use symbolic link for saving best/last checkpoints
ZeroRin opened this issue ยท 0 comments
ZeroRin commented
๐ Feature Request
When preparing checkpoint_best.pt
and checkpoint_last.pt
, create symbolic link instead of making exact copy of the checkpoint
Motivation
I'm running on a machine that somehow has bad io performance, writing the same file 3 times seems extremely inefficient.
Seems that there is plan to implement a asynchronous copying but i believe a symlink is much more efficient
fairseq/fairseq/checkpoint_utils.py
Lines 128 to 141 in bedb259