NielsOerbaek/cBLSTM-RNN

Underflow issue in perplexity calculations.

Closed this issue · 1 comments

NielsOerbaek commented

cBLSTM-RNN/utils.py

Line 67 in eb2d343

def perplexity(review):

When we calculate the perplexity, we run into an underflow issue, making a lot of the perplexities return (infinity). Possibly this is because our LMs are weak.
A solution could be to split the numbers in to exponentials and add them along the way. Or see if we can calculate perplexities of smaller parts and then merge them together.

NielsOerbaek commented

We just add the exponents and compare those instead :-)