Pinned Repositories
lm-evaluation-harness
A framework for few-shot evaluation of language models.
fast_sequential_scan
A fast sequential scan on GPU
Flax-ResNets
CIFAR10 ResNets implemented in JAX+Flax
LeagueMatchScraper
Code to scrape League of Legends matches using the Riot Games API.
Little-GPT
GPT* - Training faster small transformers using ALiBi, Parallel Residual Connections and more!
RepVGG-CIFAR10
RepVGG models specifically for CIFAR10 and CIFAR 100. Based on RepVGG: Making VGG-style ConvNets Great Again (Ding et. al)
StochasticDepthNets
PyTorch implementation of ResNet110 as described in Deep Networks with Stochastic Depth (Huang et al.)
transformer_shmap
Tensor Parallelism with JAX + Shard Map
tritonformer
Differentiable transformer in Triton, matching the performance of PyTorch + cuDNN/cuBLAS
ZeRO-transformer
Two implementations of ZeRO-1 optimizer sharding in JAX
fattorib's Repositories
fattorib/Little-GPT
GPT* - Training faster small transformers using ALiBi, Parallel Residual Connections and more!
fattorib/transformer_shmap
Tensor Parallelism with JAX + Shard Map
fattorib/ZeRO-transformer
Two implementations of ZeRO-1 optimizer sharding in JAX
fattorib/Flax-ResNets
CIFAR10 ResNets implemented in JAX+Flax
fattorib/LeagueMatchScraper
Code to scrape League of Legends matches using the Riot Games API.
fattorib/RepVGG-CIFAR10
RepVGG models specifically for CIFAR10 and CIFAR 100. Based on RepVGG: Making VGG-style ConvNets Great Again (Ding et. al)
fattorib/StochasticDepthNets
PyTorch implementation of ResNet110 as described in Deep Networks with Stochastic Depth (Huang et al.)
fattorib/wtf-wikipedia-python
raw wikipedia XML to LM_Dataformat in under 4 hours
fattorib/Monte-Carlo-Fractal-Dimensionality
Efficient algorithm using random sampling to calculate the dimension of many basic fractals. Implemented algorithm in Python.
fattorib/picograd
picograd - Fully connected neural networks in Python
fattorib/fast_sequential_scan
A fast sequential scan on GPU
fattorib/GeometricDeepLearning
Introductory Geometric Deep Learning Presentation from September 2021
fattorib/Python-Unigram
Unigram tokenization algorithm in Python
fattorib/tritonformer
Differentiable transformer in Triton, matching the performance of PyTorch + cuDNN/cuBLAS
fattorib/CudaSoftmax
Softmax CUDA kernel :)
fattorib/fattorib.github.io
Website
fattorib/flashy_linear_attention
Flash linear attention kernels in Triton
fattorib/Fundamental-Domain
Code to generate a section of the fundamental domain for the action of the special linear group on the space of (integral) binary cubic forms. As it stands, the code is quite inefficient. In the future I hope to optimize it.
fattorib/fusedswiglu
Fused SwiGLU Triton kernels
fattorib/InfoGAN-Jax
InfoGAN in Jax with small Gradio app
fattorib/jaxvae
Variational Autoencoder in JAX
fattorib/lm-evaluation-harness
Fork of lm-evaluation-harness for evaluating my custom models
fattorib/Python-BPE
I wrote Byte-Pair encoding but its 600x slower than 🤗
fattorib/ResNets-CIFAR10
PyTorch implementation of the CIFAR10 ResNets, based on Deep Residual Learning for Image Recognition (He et al.)