Pinned Repositories
cookbook
Deep learning for dummies. All the practical details and useful utilities that go into working with real models.
gpt-neo
An implementation of model parallel GPT-2 and GPT-3-style models using the mesh-tensorflow library.
gpt-neox
An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries
lm-evaluation-harness
A framework for few-shot evaluation of language models.
math-lm
minetest
Minetest is an open source voxel game engine with easy modding and game creation
polyglot
Polyglot: Large Language Models of Well-balanced Competence in Multi-languages
pythia
The hub for EleutherAI's work on interpretability and learning dynamics
sparsify
Sparsify transformers with SAEs and transcoders
the-pile
EleutherAI's Repositories
EleutherAI/concept-erasure
Erasing concepts from neural representations with provable guarantees
EleutherAI/github-downloader
Script for downloading GitHub.
EleutherAI/semantic-memorization
EleutherAI/features-across-time
Understanding how features learned by neural networks evolve throughout training
EleutherAI/rnngineering
Engineering the state of RNN language models (Mamba, RWKV, etc.)
EleutherAI/steering-llama3
EleutherAI/tokengrams
Efficiently computing & storing token n-grams from large corpora
EleutherAI/training-jacobian
EleutherAI/w2s
EleutherAI/mdl
Minimum Description Length probing for neural network representations
EleutherAI/polyapprox
Closed-form polynomial approximations to neural networks
EleutherAI/transformer-reasoning
Experiments in transformer knowledge and reasoning
EleutherAI/equinox-llama
Equinox implementation of llama3 and llama3.1
EleutherAI/ccs
EleutherAI/cupbearer
A library for mechanistic anomaly detection
EleutherAI/clearnets
EleutherAI/optax-galore
Adds GaLore style projection wrappers to optax optimizers
EleutherAI/monkfish
EleutherAI/ngrams-across-time
EleutherAI/scalable-elicitation
The code used in "Balancing Label Quantity and Quality for Scalable Elicitation"
EleutherAI/sae_overlap
Acompanying code for our research on SAE feature overlap when trained on different seeds.
EleutherAI/auto-circuit
A library for efficient patching and automatic circuit discovery.
EleutherAI/fmri
Analogue of fMRI on artificial neural networks
EleutherAI/huggingface.js
Utilities to use the Hugging Face Hub API
EleutherAI/CAA
Steering Llama 2 with Contrastive Activation Addition
EleutherAI/llm-score-behavior
EleutherAI/Megatron-LM
Ongoing research training transformer models at scale
EleutherAI/mixture-of-depths
EleutherAI/rllm
Democratizing Reinforcement Learning for LLMs
EleutherAI/TransformerEngine
A library for accelerating Transformer models on NVIDIA GPUs, including using 8-bit floating point (FP8) precision on Hopper and Ada GPUs, to provide better performance with lower memory utilization in both training and inference.