The paper is accepted to Findings of EMNLP 2022.
transformers==4.16.2
To run the code, change the parameters as you need and use
bash run.sh
We have offered a checkpoint for DORE + negative sampling_digonal + distant
, since it takes a long time (about 7 days) to pre-train on the distantly supervised dataset. Please download it from here.