lightmatmul

axolotl

Go ahead and axolotl questions

Data-Generation-with-OpenAI

Generating synthetic data for model training using OpenAI API.

flash-attention

Fast and memory-efficient exact attention

gptq

Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".

llm.c

LLM training in simple, raw C/CUDA

LoRA

Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"

LoRA-from-scratch

Low-Rank Adaptation of LLMs implemented using PyTorch

minbpe

Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.

RLHF-trainer

Supervised Finetune and Align LLMs using RLHF (RM and PPO).

Transformer-from-scratch

Transformer from scratch using Pytorch [https://arxiv.org/pdf/1706.03762]

lightmatmul/Transformer-from-scratch

Transformer from scratch using Pytorch [https://arxiv.org/pdf/1706.03762]

lightmatmul/axolotl

Go ahead and axolotl questions

lightmatmul/Data-Generation-with-OpenAI

Generating synthetic data for model training using OpenAI API.

lightmatmul/flash-attention

Fast and memory-efficient exact attention

lightmatmul/gptq

Code for the ICLR 2023 paper "GPTQ: Accurate Post-training Quantization of Generative Pretrained Transformers".

lightmatmul/llm.c

LLM training in simple, raw C/CUDA

lightmatmul/LoRA

Code for loralib, an implementation of "LoRA: Low-Rank Adaptation of Large Language Models"

lightmatmul/LoRA-from-scratch

Low-Rank Adaptation of LLMs implemented using PyTorch

lightmatmul/minbpe

Minimal, clean code for the Byte Pair Encoding (BPE) algorithm commonly used in LLM tokenization.

lightmatmul/RLHF-trainer

Supervised Finetune and Align LLMs using RLHF (RM and PPO).

lightmatmul/sft-toolkit

lightmatmul/xformers

Hackable and optimized Transformers building blocks, supporting a composable construction.