Ancient History LM
Rik Koncel-Kedziorski and Noah A. Smith https://arxiv.org/abs/2104.08742
Download selector model trained on Wikitext-2 at https://drive.google.com/file/d/1NVXrnrztxiS-hT128AhD5bVAgKfSgIN_/view?usp=sharing
Developed with:
pytorch 1.6 pytorch-lightning 1.1.2 transformers 4.2.2
Inference:
Place saved model in models/wiki_gpt_ranker.ckpt
Run 03-Inference.py
Training on other data:
01-CreatePPLTrainingData.py -- modify to create training data for the selector model. Currently uses Wikipedia and GPT2-large.
02-TrainSelector.py -- trains model on data from step 01