rwitten/HighPerfLLMs2024

The formula for softmax seems wrong in s1p20

Opened this issue · 0 comments

In S1P20, the softmax formula is given:
p( guesses, j) = exp(guesses[j]) / sum(guesses(l))

But shouldn't it be:
p( guesses, j) = exp(guesses[j]) / sum(exp(guesses(l)))