Extremely slow when running without prompts under zero-shot setting
Closed this issue · 1 comments
pluiez commented
I tested neurologic decoding with given prompts and without prompts on a V100 GPU device. The former took ~62 seconds to decode the top-10 examples from dev set, while the latter took 24 minutes. Thus on average one input example requires 2.4 minutes to generate a text. It this normal as expected?
GloriaXimingLu commented
Hi, thanks for the question. Yes it's as expected, since neurologic decoding without prompt generate sentences for all possible prompts in a prompt set and return the one with the highest likelihood.