Resuming NFSP Training

[i-Madsen]

I'd like to save NFSP agents during the training process and be able to restore them for further training later. However, the nfsp.py file states that the save/restore functions "don't save the experience replay buffers and should only be used to restore the agent's policy, not resume training."

I've tried modifying it to save/restore the replay buffer and step counter agents, but haven't been successful in continuing training (it still appears to restart).

What is the proper way to do this?

[lanctot]

Hi @i-Madsen, yes that's right -- the NFSP code does not support full checkpointing, so you'll have to implement it yourself.

I'm not sure why your custom solution wouldn't work; it should be straight-forward to write everything in the replay buffer and reload it?

If you get it working, it would great to contribute it as I'm sure others would find this useful.