rpryzant/delete_retrieve_generate

why the deleteonly and the delete_retrieve mode get the same inference result

zhunipingan opened this issue · 6 comments

zhunipingan commented

Hi,
Thanks for your sharing.
In my view, according to the config.json, the model can only do transfer style A to style B. If I want to transfer style B to style A, I should modify the src and tgt setting. If my understanding is right?
If I want to change the model mode, I should modify the "model_mode" from "delete" to "delete_retrieve".
I do in the above way, but I find the result of delete mode is same as the result of delete retrieve mode(I means the golds file. I think the golds file contains the generated result of the model). I can not find the reason, can you give me some help, thanks.

zhunipingan commented

And what is the differences between the vocab.attribute and ngram.15.attribute. In the yelp_config.json, only ngram.15.attribute is used.

naveen-kinnal commented

@zhunipingan may I know what was your BLEU score for both ? Was it YELP dataset?

zhunipingan commented

@zhunipingan may I know what was your BLEU score for both ? Was it YELP dataset?

I don't know how to calculate the BLEU. During the training, the value of 'METRIC' is about 14. But when I turn to my own dataset, the metric is near to 0. Is there something wrong?

naveen-kinnal commented

@zhunipingan Could you please write me on iamkinnal@gmail.com. I would love to see your data and setup.

rpryzant commented

  • I'd recommend looking at those two attribute files. The vocab.attribute file was generated using make_attribute_vocab.py, which only considers a unigram vocabulary. The ngram file was generated using make_ngram_attribute_vocab.py which uses ngram vocabularies (up to 4-grams).

  • Is your metric still near 0? What do the outputs look like? I would recommend adjusting your hyperparameters to try and achieve better model fit.

rpryzant commented

Closing, ~1 week since last activity.