Pinned Repositories
pytorch
Tensors and Dynamic neural networks in Python with strong GPU acceleration
ao
torchao: PyTorch Architecture Optimization (AO). A repository to host AO techniques and performant kernels that work with PyTorch.
attention-gym
Helpful tools and examples for working with flex-attention
float8_experimental
This repository contains the experimental PyTorch native float8 training UX
lit-gpt
Hackable implementation of state-of-the-art open-source LLMs based on nanoGPT. Supports flash attention, 4-bit and 8-bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.
mem-and-run-estimator
pytorch
Tensors and Dynamic neural networks in Python with strong GPU acceleration
scyther
An optimizer for Auto-wrapping FSDP
torchtitan
A native PyTorch Library for large model training
torchtune
A Native-PyTorch Library for LLM Fine-tuning
weifengpy's Repositories
weifengpy/ao
torchao: PyTorch Architecture Optimization (AO). A repository to host AO techniques and performant kernels that work with PyTorch.
weifengpy/attention-gym
Helpful tools and examples for working with flex-attention
weifengpy/float8_experimental
This repository contains the experimental PyTorch native float8 training UX
weifengpy/lit-gpt
Hackable implementation of state-of-the-art open-source LLMs based on nanoGPT. Supports flash attention, 4-bit and 8-bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.
weifengpy/mem-and-run-estimator
weifengpy/pytorch
Tensors and Dynamic neural networks in Python with strong GPU acceleration
weifengpy/scyther
An optimizer for Auto-wrapping FSDP
weifengpy/torchchat
Run PyTorch LLMs locally on servers, desktop and mobile
weifengpy/torchtitan
A native PyTorch Library for large model training
weifengpy/torchtune
A Native-PyTorch Library for LLM Fine-tuning
weifengpy/transformer_nuggets
A place to store reusable transformer components of my own creation or found on the interwebs
weifengpy/torchft
PyTorch per step fault tolerance (actively under development)
weifengpy/torchrec
Pytorch domain library for recommendation systems