kyegomez/Griffin

Implementation of Griffin from the paper: "Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models"

PythonMIT

Issues

[BUG] Dimension errors in RGLRU class
#3 opened 4 months ago by MisakiWho
2
[BUG] [RuntimeError: Expected all tensors to be on the same device, but found at least two devices, cpu and cuda:0! (when checking argument for argument mat1 in method wrapper_CUDA_addmm)]
#2 opened 4 months ago by kyegomez
3
[BUG] Calculation of alpha and initialization of lambda incorrect
#6 opened 3 months ago by ozppupbg
4
[BUG] implementation of Griffin
#4 opened 4 months ago by ruoyxue
2