/heinsen_position_embeddings

Reference implementation of "Encoding Position by Decaying and Updating Different Exponentiated States Differently," as proposed by Franz A. Heinsen in March 2024

Primary LanguagePythonMIT LicenseMIT

Watchers