Pinned Repositories
RWKV-LM
RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
megablocks
lm-evaluation-harness
A framework for few-shot evaluation of language models.
transformers
🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
Megatron-LM
Ongoing research training transformer models at scale
nccl
Optimized primitives for collective multi-GPU communication
composer
Supercharge Your Model Training
megablocks
open_lm_fork
A repository for research on medium sized language models.
transformers
🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.
sedrick-keh-tri's Repositories
sedrick-keh-tri/composer
Supercharge Your Model Training
sedrick-keh-tri/megablocks
sedrick-keh-tri/open_lm_fork
A repository for research on medium sized language models.
sedrick-keh-tri/transformers
🤗 Transformers: State-of-the-art Machine Learning for Pytorch, TensorFlow, and JAX.