hkproj/pytorch-transformer

Clarification regarding decoder_input and label

Closed this issue · 1 comments

They way I see it now, we feed the decoder input entirely into the decoder. But the decoder input is formed form the target text.

Shouldn't the decoder only get a start-of-sentence token plus padding? And then get it's own output in each iteration?

Oh of course, we use a mask. Handing over the entire text in one go just speeds up training. Closing issue.