graykode/nlp-tutorial

different Embedding way

lhbrichard opened this issue · 1 comments

In the code 'Seq2seq-torch.py', i saw u use np.eye,the one-hot representation, to represent embedding, so i change in a normal way ,using nn.Embedding(dict_length,embedding_dim),it can work out. but the loss i got is very high.
i wanna ask the differences between this two ways. here are my code and the result.

image
image

I think nn.Embedding is not trainale Parameter because it's not in Model init(), also nn.Embedding was initialized as randomly...