Simple Implementation for Transformer with C and Python
Implement Tranformer with C and Python for educational purpose. Code is written for readability.
Build
cd lib/sml
make
cd ..
cd src/c
make
Run
C version
cd src/c
./transformer
Python version
cd src/python
python ./experiments/classify.py --random-seed=1234 --num-epochs=1 --tiny
TODO:
- main Makefile for build library and c executable
- add config load
- add trained weight load
- add python code to generate test vector, and use that to test the C code
Reference:
- SML: small math library, http://www.bios.unc.edu/distrib/bios235/sml/
- Transformer tutorial: http://jalammar.github.io/illustrated-transformer/
- Python Transformer implementation: http://peterbloem.nl/blog/transformers
- tranformer implementation: https://github.com/tunz/transformer-pytorch