This is a simple port of Andrej Karpathy’s llama2.c:
https://github.com/karpathy/llama2.c
The idea is to keep the no dependency idea alive. However, I use
cligen for argument parsing and because it’s available its
memfile interface (the latter could be replace by std/memfiles, but
initially I thought about using more features)
Example compile: (see README of the original for more details)
nim c -d:danger llama2nim.nimIt needs a checkpoint file, see the README of the original.
Just run the program with
./llama2nim -c <path to checkpoint>and you can adjust the number of steps as well as the temperature.
Because I thought it would be a fun thing to do during a train ride & waiting at the airport. :)
I wrap the raw pointers in a Buf[T] object for slightly saner
handling of the data incl. freeing of the memory.