shmap
There are 2 repositories under shmap topic.
fattorib/transformer_shmap
Tensor Parallelism with JAX + Shard Map
azzeddineCH/flash-nanoGPT
Jax/Flax re-write of @karpathy 🐐 NanoGPT using some of the common Jax libraries/features (shmap, pallas, jmp, optax, orbax)