zhixuan-lin's Stars
karpathy/llm.c
LLM training in simple, raw C/CUDA
EleutherAI/gpt-neox
An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries
EleutherAI/pythia
The hub for EleutherAI's work on interpretability and learning dynamics
sustcsonglin/flash-linear-attention
Efficient implementations of state-of-the-art linear attention models in Pytorch and Triton
alxndrTL/mamba.py
A simple and efficient Mamba implementation in pure PyTorch and MLX.
HazyResearch/safari
Convolutions for Sequence Modeling
google-deepmind/recurrentgemma
Open weights language model from Google DeepMind, based on Griffin.
radarFudan/Awesome-state-space-models
Collection of papers on state-space models
google/paxml
Pax is a Jax-based machine learning framework for training large scale models. Pax allows for advanced and fully configurable experimentation and parallelization, and has demonstrated industry leading model flop utilization rates.
apple/ml-sigma-reparam
AvivBick/awesome-ssm-ml
Reading list for research topics in state-space models
Sea-Snell/JAXSeq
Train very large language models in Jax.
MichaelTMatthews/Craftax
(Crafter + NetHack) in JAX. ICML 2024 Spotlight.
proger/accelerated-scan
Accelerated First Order Parallel Associative Scan
ShacharHarshuv/open-ear
berlino/gated_linear_attention
proger/hippogriff
Griffin MQA + Hawk Linear RNN Hybrid
BlinkDL/LinearAttentionArena
Here we will test various linear attention designs.
kyegomez/Griffin
Implementation of Griffin from the paper: "Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models"
OpenNLPLab/HGRN2
HGRN2: Gated Linear RNNs with State Expansion
sustcsonglin/mamba-triton
johnryan465/pscan
EleutherAI/rnngineering
Engineering the state of RNN language models (Mamba, RWKV, etc.)
dangxingyu/rnn-icrag
Official repository of paper "RNNs Are Not Transformers (Yet): The Key Bottleneck on In-context Retrieval"
tobiaskatsch/GatedLinearRNN
glassroom/heinsen_attention
Reference implementation of "Softmax Attention with Constant Cost per Token" (Heinsen, 2024)
buttercutter/Mamba_SSM
A simple implementation of [Mamba: Linear-Time Sequence Modeling with Selective State Spaces](https://arxiv.org/abs/2312.00752)
srush/mamba-scans
Blog post
simudt/Griffin-Jax
Jax implementation of "Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models"
radarFudan/Mamba_State_Space_Model_Paper_List
Paper list for State-Space-Model and it's Applications