graykode/nlp-tutorial

Why is first parameter `src_vocab_size`?

yangdechuan opened this issue · 1 comments

self.pos_emb = nn.Embedding.from_pretrained(get_sinusoid_encoding_table(src_vocab_size, d_model),freeze=True)

position encoding table should be (src_len, d_model). Why (src_vocab_size, d_model) here?

Oops.. Thanks for right catching
In Encoder src_len+1 is right, and as same tgt_len+1 in Decoder.
I will fixed as soon as possible Thanks