princeton-nlp/QuRating

Question about Gumbel trick

yuzc19 opened this issue · 0 comments

Hi! Thanks for your solid and insightful work.

I am checking the implementation of data selection but didn't find the place corresponding to this equation. The implementation didn't use the exp operation and normalize by expsum, which I think is necessary for the Gumbel trick. Could you explain a little bit? Thank you very much!

metrics = metrics / args.temperature
if args.sample and args.temperature != 0.0:
metrics += np.random.gumbel(size=len(metrics)) # Use topk-gumbel trick

Screenshot 2024-02-21 at 14 35 23