Quentin-Anthony
High-Performance Deep Learning PhD student at OSU NOWLAB
The Ohio State UniversityColumbus, OH
Pinned Repositories
cookbook
Deep learning for dummies. All the practical details and useful utilities that go into working with real models.
gpt-neox
An implementation of model parallel autoregressive transformers on GPUs, based on the Megatron and DeepSpeed libraries
nanoGPT-mup
The simplest, fastest repository for training/finetuning medium-sized GPTs.
DeepSpeedExamples
Example models using DeepSpeed
magma
MAGMA - a GPT-style multimodal model that can understand any combination of images and language. NOTE: The freely available model from this repo is only a demo. For the latest multimodal and multilingual models from Aleph Alpha check out our website https://app.aleph-alpha.com
ml-engineering
Machine Learning Engineering Open Book
xminGPT
BlackMamba
Code repository for Black Mamba
zcookbook
Training hybrid models for dummies.
Quentin-Anthony's Repositories
Quentin-Anthony/magma
MAGMA - a GPT-style multimodal model that can understand any combination of images and language. NOTE: The freely available model from this repo is only a demo. For the latest multimodal and multilingual models from Aleph Alpha check out our website https://app.aleph-alpha.com
Quentin-Anthony/DeepSpeedExamples
Example models using DeepSpeed
Quentin-Anthony/ml-engineering
Machine Learning Engineering Open Book
Quentin-Anthony/xminGPT
Quentin-Anthony/apex
A PyTorch Extension: Tools for easy mixed precision and distributed training in Pytorch
Quentin-Anthony/DeepSpeed
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
Quentin-Anthony/horovod
Distributed training framework for TensorFlow, Keras, PyTorch, and MXNet.
Quentin-Anthony/Megatron-DeepSpeed
Ongoing research training transformer language models at scale, including: BERT & GPT-2
Quentin-Anthony/Awesome-LLMs-on-device
Awesome LLMs on Device: A Comprehensive Survey
Quentin-Anthony/awesome-open-source-lms
Friends of OLMo and their links.
Quentin-Anthony/aws-neuron-reference-for-megatron-lm
Quentin-Anthony/BERT-PyTorch
BERT for Distributed PyTorch + AMP Training
Quentin-Anthony/DL-SR
Tensorflow/keras implementation for image transformation from low-resolution (LR) image to super-resolved one, including single wide-field (WF) image super-resolution prediction and SIM reconstruction.
Quentin-Anthony/examples
A set of examples around pytorch in Vision, Text, Reinforcement Learning, etc.
Quentin-Anthony/flash-attention
Fast and memory-efficient exact attention
Quentin-Anthony/gpu-questions
Quentin-Anthony/grace
GRACE - GRAdient ComprEssion for distributed deep learning
Quentin-Anthony/HierarchicalKV
HierarchicalKV is a part of NVIDIA Merlin and provides hierarchical key-value storage to meet RecSys requirements. The key capability of Merlin-KV is to store key-value feature-embeddings on high-bandwidth memory (HBM) of GPUs and in host memory. It also can be used as a generic key-value storage.
Quentin-Anthony/ior
IOR and mdtest
Quentin-Anthony/Megatron-DeepSpeed-MS
Ongoing research training transformer language models at scale, including: BERT & GPT-2
Quentin-Anthony/minGPT
minGPT in JAX
Quentin-Anthony/nanoGPT
The simplest, fastest repository for training/finetuning medium-sized GPTs.
Quentin-Anthony/NeMo
NeMo: a toolkit for conversational AI
Quentin-Anthony/Ok-Topk
Ok-Topk is a scheme for distributed training with sparse gradients. Ok-Topk integrates a novel sparse allreduce algorithm (less than 6k communication volume which is asymptotically optimal) with the decentralized parallel Stochastic Gradient Descent (SGD) optimizer, and its convergence is proved theoretically and empirically.
Quentin-Anthony/pythia
Quentin-Anthony/pytorch
Tensors and Dynamic neural networks in Python with strong GPU acceleration
Quentin-Anthony/Quentin-Anthony.github.io
A beautiful, simple, clean, and responsive Jekyll theme for academics
Quentin-Anthony/RWKV-LM
RWKV is an RNN with transformer-level LLM performance. It can be directly trained like a GPT (parallelizable). So it's combining the best of RNN and transformer - great performance, fast inference, saves VRAM, fast training, "infinite" ctx_len, and free sentence embedding.
Quentin-Anthony/SwinIR
SwinIR: Image Restoration Using Swin Transformer
Quentin-Anthony/transformers
🤗 Transformers: State-of-the-art Natural Language Processing for Pytorch, TensorFlow, and JAX.