fkodom

clip-text-decoder

Generate text captions for images from their embeddings.

dilated-attention-pytorch

(Unofficial) Implementation of dilated attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens" (https://arxiv.org/abs/2307.02486)

fft-conv-pytorch

Implementation of 1D, 2D, and 3D FFT convolutions in PyTorch. Much faster than direct convolutions for large kernel sizes.

grouped-query-attention-pytorch

(Unofficial) PyTorch implementation of grouped-query attention (GQA) from "GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints" (https://arxiv.org/pdf/2305.13245.pdf)

lora-pytorch

Simple but robust implementation of LoRA for PyTorch. Compatible with NLP, CV, and other model types. Strongly typed and tested.

semantle

Fastest 'Semantle' solver this side of the Mississippi.

soft-mixture-of-experts

PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)

transformer-from-scratch

Code implementation from my blog post: https://fkodom.substack.com/p/transformers-from-scratch-in-pytorch

wnet-unsupervised-image-segmentation

yet-another-retnet

A simple but robust PyTorch implementation of RetNet from "Retentive Network: A Successor to Transformer for Large Language Models" (https://arxiv.org/pdf/2307.08621.pdf)

fkodom/fft-conv-pytorch

Implementation of 1D, 2D, and 3D FFT convolutions in PyTorch. Much faster than direct convolutions for large kernel sizes.

fkodom/yet-another-retnet

A simple but robust PyTorch implementation of RetNet from "Retentive Network: A Successor to Transformer for Large Language Models" (https://arxiv.org/pdf/2307.08621.pdf)

fkodom/clip-text-decoder

Generate text captions for images from their embeddings.

fkodom/grouped-query-attention-pytorch

(Unofficial) PyTorch implementation of grouped-query attention (GQA) from "GQA: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints" (https://arxiv.org/pdf/2305.13245.pdf)

fkodom/transformer-from-scratch

Code implementation from my blog post: https://fkodom.substack.com/p/transformers-from-scratch-in-pytorch

fkodom/soft-mixture-of-experts

PyTorch implementation of Soft MoE by Google Brain in "From Sparse to Soft Mixtures of Experts" (https://arxiv.org/pdf/2308.00951.pdf)

fkodom/dilated-attention-pytorch

(Unofficial) Implementation of dilated attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens" (https://arxiv.org/abs/2307.02486)

fkodom/wnet-unsupervised-image-segmentation

fkodom/semantle

Fastest 'Semantle' solver this side of the Mississippi.

fkodom/lora-pytorch

Simple but robust implementation of LoRA for PyTorch. Compatible with NLP, CV, and other model types. Strongly typed and tested.

fkodom/python-repo-template

Template repo for Python projects, especially those focusing on machine learning and/or deep learning.

fkodom/byol

PyTorch implementation of BYOL: a fantastically simple method for self-supervised image representation learning with SOTA performance.

fkodom/document-rag

RAG application to answer questions about PDF documents using LLMs.

fkodom/blip-inference

Pretrained BLIP with a similar API to CLIP.

fkodom/metis

Minimalist library for training RL agents in PyTorch. Implements many common training algorithms, with a focus on actor-critic methods. Includes SAC, TD3, PPO, A2C, VPG.

fkodom/octconv-pytorch

fkodom/gpt-fast

Simple and efficient pytorch-native transformer text generation in <1000 LOC of python.

fkodom/simple-cocotools

A simple, modern alternative to 'pycocotools'.

fkodom/unipipe

Build batch pipelines in Python that run anywhere.

fkodom/ncsn

Personal repo recreating Noise Contrastive Score Networks (https://arxiv.org/abs/1907.05600v2) for learning purposes.

fkodom/wordle

Fastest Wordle solver in the West.

fkodom/ddpg-her-pytorch

Implementation of the Hindsight Experience Replay (HER) algorithm using PyTorch. Utilizes Deep Deterministic Policy Gradients for off-policy optimization of RL agents in continuous action spaces.

fkodom/fkodom

fkodom/intro-to-deep-learning

Code samples to accompany my "Intro to Deep Learning" article series on Medium.

fkodom/visual-chatgpt

Personal fork of microsoft/visual-chatgpt