Low Bleu Score

[eujhwang]

Hello, I ran your code with the same hyper-parameter you provided in the paper and code and I'm not able to get the same bleu score as you put in the paper. The model I ran is KGMixtureOfExpertCho.sh with eg dataset. The output score is "topk_bleu_4": 0.1459059145584853, "topk_rouge_l": 0.3882581646863371, which are much lower than the numbers in the paper. (The metrics (distinct_2, self_bleu,..etc) look similar to the one with your paper )

Could you guide me on what things I'm possibly missing?

Thank you,

[wyu97]

Hi,

The data pre-processing code is added after paper submission because previously I directly used their code (https://github.com/cdjhz/multigen).

I did not realize a small error when I added the code to my repo. But now it has been fixed up.

Best regards,
Wenhao