/Mixture-of-Depths

Implementation of the paper: "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"

Primary LanguagePythonMIT LicenseMIT

Watchers