GPT-J used in "Domain-Specific Text Generation for Machine Translation"

[ymoslem]

If you use this repository or any of the pretrained weights to do something cool, we would love to hear about it. Feel free to open a github issue...

Many thanks for your efforts! We used your GPT-J in our paper "Domain-Specific Text Generation for Machine Translation". The related scripts can be found here.

More specifically, we used GPT-J (Wang and Komatsuzaki, 2021) to generate texts from in-domain sentences. We investigated the feasibility of this domain-specific text generation technique when either no or limited bilingual in-domain dataset is available to fine-tune a generic machine translation model. Combining this approach with back-translation, we could generate huge amounts of synthetic bilingual in-domain data for both use cases. After fine-tuning our baseline generic machine translation model on the purely synthetic dataset, we observed significant improvements of the translation quality of the in-domain test set.