dtunai
Building AI products, whispering AIs to imaginr things ⚡ AI infrastructure, video gen, her, diffusion inference 🍊
Istanbul, Turkey
Pinned Repositories
Griffin-Jax
Jax implementation of "Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models"
LongConv-Jax
Jax/Flax/Linen implementation of "Simple Hardware-Efficient Long Convolutions for Sequence Modeling"
miniF2F-code
Dataset of formal Olympiad-level mathematics problems solved with Python code instructions.
Mixture-of-Depths-Jax
Jax module for the paper: "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
mpi-ds
MPI Operator DeepSpeed Base Configuration for CIFAR-10
SynthToT
SynthToT: Generate synthetic dataset for your training dataset through deliberate problem-solving et al S Yao, 2023.
Tri-RMSNorm
Efficient kernel for RMS normalization with fused operations, includes both forward and backward passes, compatibility with PyTorch.
triton-activations
Collection of neural network activation function kernels for Triton Language Compiler by OpenAI
xLSTM-Jax
Jax implementation of x-LSTM: Extended Long Short-Term Memory by Beck et al. (2024)
jax-triton
jax-triton contains integrations between JAX and OpenAI Triton
dtunai's Repositories
dtunai/xLSTM-Jax
Jax implementation of x-LSTM: Extended Long Short-Term Memory by Beck et al. (2024)
dtunai/Griffin-Jax
Jax implementation of "Griffin: Mixing Gated Linear Recurrences with Local Attention for Efficient Language Models"
dtunai/SynthToT
SynthToT: Generate synthetic dataset for your training dataset through deliberate problem-solving et al S Yao, 2023.
dtunai/Tri-RMSNorm
Efficient kernel for RMS normalization with fused operations, includes both forward and backward passes, compatibility with PyTorch.
dtunai/triton-activations
Collection of neural network activation function kernels for Triton Language Compiler by OpenAI
dtunai/mpi-ds
MPI Operator DeepSpeed Base Configuration for CIFAR-10
dtunai/LongConv-Jax
Jax/Flax/Linen implementation of "Simple Hardware-Efficient Long Convolutions for Sequence Modeling"
dtunai/miniF2F-code
Dataset of formal Olympiad-level mathematics problems solved with Python code instructions.
dtunai/Mixture-of-Depths-Jax
Jax module for the paper: "Mixture-of-Depths: Dynamically allocating compute in transformer-based language models"
dtunai/cpp-langchain
Tool for executing C/C++ code snippets with Langchain Agents.
dtunai/GradientAscent-Jax
Custom gradient ascent solver (optimizer) for JAX/Flax models
dtunai/kmeansops
PyKeops Powered K-Means Clustering Algorithms Module both on CPU & GPU
dtunai/Ring-Attention-Jax
Packaged Ring Attention with Blockwise Transformers for Near-Infinite Context implemented in Jax + Flax.
dtunai/smooth-activations
Smooth ReLU activations in CUDA. Shamir, G., I. et al.
dtunai/jax-triton
jax-triton contains integrations between JAX and OpenAI Triton
dtunai/MEGABYTE-pytorch-DS
Modificated DeepSpeed training setup fork of MEGABYTE - PyTorch by lucidrains, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch
dtunai/PaLM-rlhf-pytorch-DS
Modificated DeepSpeed training setup fork of RLHF (Reinforcement Learning with Human Feedback) by lucidrains on top of the PaLM architecture. Basically ChatGPT but with PaLM
dtunai/Python-Template
Python Package Template is all you need