How to generate interactive conditional samples after retraining on custom dataset?

Question

How to generate interactive conditional samples after retraining on custom dataset?

nikilp opened this issue 5 years ago · 4 comments

When I retrain GPT-2 345M on custom dataset I get good samples during the training process itself, but after I stop the training and I run interactive_conditional_samples.py the samples are NOT from the same retrained model. How do I obtain samples from the new model?
Thanks

Answer 1 · 2020-06-12T22:49:37.000Z

Did you make sure to use the --model_name parameter?

python interactive_conditional_samples.py --model_name YOURMODELNAME

Answer 2 · 2020-06-17T15:05:24.000Z

I use the same model which I used during training 345M:
python3 src/interactive_conditional_samples.py --top_k 40 --model_name "345M"
but it is not loading the checkpoint, just the basic model. Anyway, i pull out some of the relevant code from the training script which got the trick for me.

Answer 3 · 2020-08-19T03:09:58.000Z

@nikilp could you share that code?

Answer 4 · 2021-04-12T22:46:09.000Z

After training, you have to copy the latest checkpoint files from the checkpoint folder into your previous model folder.
See https://youtu.be/oEpLMb5D_G0?t=312