project-baize/baize-chatbot

Does prob_sum in greedy search always add up to 1?

qrdlgit opened this issue · 1 comments

qrdlgit commented

https://github.com/project-baize/baize/blob/ccf0bb8485657b7c16a57456bbb835503bac2456/demo/app_modules/utils.py#L278

Was just looking through the code. Doesn't softmax ensure that all the elements add up to 1?

JetRunner commented

This is for top-p sampling. We mask out tokens outside top-p (by default 95%).