fkodom/grouped-query-attention-pytorch
(Unofficial) PyTorch implementation of grouped-query attention (GQA) from "GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints" (https://arxiv.org/pdf/2305.13245.pdf)
PythonMIT
Stargazers
- Alan55555
- apoorv2904Meta FAIR
- Athe-kunalUnited States
- atuxhe
- brandnewchoppa
- d9249Kyonggi University, Computer Science
- devkadeKyung Hee University
- ell-holFater AI
- fkodomPlainsight
- fly51flyPRIS
- hand0ngmumei
- HarshaSatyavardhanindia
- HongbosherlockBaidu Search
- Innary
- iqianchengLLMs, AIGC, DeepRec, AI Fintech
- JeffCarpenterCanada
- jiachenwei
- jim4399266
- JizhongpengShanghai
- jjonhwaGlorang
- m0saan1337 FIL
- MHarris021Pachyderm
- Nanayali
- notnilPlainsight
- shubhampachori12110095Somewhere in India
- snoop2headKAIST AI
- sNug0
- stereoplegic🧪Science🖌️Art🪄Magic
- switiz
- t-thirupathi
- tcxia
- TGLTommy
- tsingcooNanjing University
- ultranity
- vgoklaniNew York, NY
- XYPBYale University