VachanVY/Rotary-Embeddings

Simple Implementation of Rotary Positional Embeddings (RoPE) and Sinusoidal Positional Embeddings in JAX

Rotary Positional Embedding: [paper]

• 

Also See

• Reference-1
• Reference-2