RotaryEmbedding
xieanheng opened this issue · 1 comments
xieanheng commented
I have a question: is the "RotaryEmbedding" added in the latest version or used from the beginning
yiyixuxu commented
it seems like they used a learnable positional embedding -
this implementation used RotaryEmbedding https://github.com/lucidrains/TimeSformer-pytorch