/SwiGlu-manual-implementation

i implemented the SwiGLu used in the feedforward method of Llama2 , and i apply its gradient manually

Primary LanguageJupyter Notebook

This repository is not active