Pinned Repositories
m2
Repo for "Monarch Mixer: A Simple Sub-Quadratic GEMM-Based Architecture"
TinyLlama
The TinyLlama project is an open endeavor to pretrain a 1.1B Llama model on 3 trillion tokens.
LongNet
Implementation of plug in and play Attention from "LongNet: Scaling Transformers to 1,000,000,000 Tokens"
MAP-NEO
yangsp5.github.io
mixinglaws