GPT2 Generator Demo in GluonNLP
First, you need to convert the pretrained models in openai/gpt-2 to GluonNLP
Step-1: Download the pretrained models provided by OpenAI.
git clone https://github.com/openai/gpt-2.git
cd gpt-2
python download_model.py 117M
python download_model.py 345M
cd ..
Step-2: Convert the model to GluonNLP (Make sure the current directory is gluonnlp-gpt2)
python gpt2_tf_to_gluonnlp.py --src_dir gpt-2/models --dst_dir models --model 117M
python gpt2_tf_to_gluonnlp.py --src_dir gpt-2/models --dst_dir models --model 345M
Step-3: Test the correctness of the pretrained models:
nosetests3 test_transform.py
python test_model.py
Step-4: Try the generation demo
For the conditional generator, use the following command
python sampling_demo.py --model 117M
For the unconditional generator, use the following command
python sampling_demo.py --model 117M --unconditional
You can also try the large model with 345M parameters
python sampling_demo.py --model 345M
python sampling_demo.py --model 345M --unconditional
Dependencies
mxnet
gluonnlp
regex
tensorflow