hirofumi0810/neural_sp

relative positional encoding

zyzpower opened this issue · 1 comments

zyzpower commented

Hi, which dataset/configuration is relative positional encoding implemented? Is the transformer-XL implemented in encoder?

hirofumi0810 commented

@zyzpower Yes. relative and relative_xl options correspond to relative positional encoding.
But I didn't implement recurrence like Transformer-XL because it breaks the parallel computation.
Transformer-XL LM is already implemented.