An rust reimplementatin of https://github.com/karpathy/llama2.c.
This repo contains a tinystories15M in the testdata/ folder, you can run it with the following command:
cargo run --release
./target/release/llama2-rs "rust is a crab"Screen.Recording.2023-09-10.at.21.30.10.mov
| model | tokens/s |
|---|---|
| tinystories15M | 120~197 |