Pinned Repositories
GradientAscent-Jax
Custom gradient ascent solver (optimizer) for JAX/Flax models
Griffin-Jax
Jax implementation of "Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models"
kmeansops
PyKeops Powered K-Means Clustering Algorithms Module both on CPU & GPU
lmppl-cli-csv-wrapper
A tiny CLI wrapper around lmppl for Pre-Trained Language Models Perplexity Calculation for CSV files
LongConv-Jax
Jax/Flax/Linen implementation of "Simple Hardware-Efficient Long Convolutions for Sequence Modeling"
miniF2F-code
Dataset of formal Olympiad-level mathematics problems solved with Python code instructions.
Mixture-of-Depths-Jax
Jax module for the paper: "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
mpi-ds
MPI Operator DeepSpeed Base Configuration for CIFAR-10
Tri-RMSNorm
Efficient kernel for RMS normalization with fused operations, includes both forward and backward passes, compatibility with PyTorch.
triton-activations
Collection of neural network activation function kernels for Triton Language Compiler by OpenAI
simudt's Repositories
simudt/Griffin-Jax
Jax implementation of "Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models"
simudt/mpi-ds
MPI Operator DeepSpeed Base Configuration for CIFAR-10
simudt/miniF2F-code
Dataset of formal Olympiad-level mathematics problems solved with Python code instructions.
simudt/Tri-RMSNorm
Efficient kernel for RMS normalization with fused operations, includes both forward and backward passes, compatibility with PyTorch.
simudt/LongConv-Jax
Jax/Flax/Linen implementation of "Simple Hardware-Efficient Long Convolutions for Sequence Modeling"
simudt/triton-activations
Collection of neural network activation function kernels for Triton Language Compiler by OpenAI
simudt/GradientAscent-Jax
Custom gradient ascent solver (optimizer) for JAX/Flax models
simudt/kmeansops
PyKeops Powered K-Means Clustering Algorithms Module both on CPU & GPU
simudt/lmppl-cli-csv-wrapper
A tiny CLI wrapper around lmppl for Pre-Trained Language Models Perplexity Calculation for CSV files
simudt/Mixture-of-Depths-Jax
Jax module for the paper: "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
simudt/Ring-Attention-Jax
Packaged Ring Attention with Blockwise Transformers for Near-Infinite Context implemented in Jax + Flax.
simudt/Python-Template
Python Package Template is all you need
simudt/simudt.github.io
blog for the AI era
simudt/Composable-Datasets
Transform JSONL Q&A datasets to instruct format with ease
simudt/jax-triton
jax-triton contains integrations between JAX and OpenAI Triton
simudt/MEGABYTE-pytorch-DS
Modificated DeepSpeed training setup fork of MEGABYTE - PyTorch by lucidrains, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch
simudt/PaLM-rlhf-pytorch-DS
Modificated DeepSpeed training setup fork of RLHF (Reinforcement Learning with Human Feedback) by lucidrains on top of the PaLM architecture. Basically ChatGPT but with PaLM
simudt/Simba
A simpler Pytorch + Zeta Implementation of the paper: "SiMBA: Simplified Mamba-based Architecture for Vision and Multivariate Time series"
simudt/zeta
Build high-performance AI models with modular building blocks