made a internlm model by python & c++ & cuda from scratch.
- made a internlm2 model by python and pytorch 101. and tell it step by step
- using c++ and python made a model reference gguf
- using pybind bind the CUDA kernel
- add quant module
- RoPE https://www.yuque.com/yyfan-qapeo/bphtwh/cko188656pqyed4n?singleDoc# 《聊聊并写写RoPE(Rotary Position Encoding)》
- GQA https://www.yuque.com/yyfan-qapeo/bphtwh/py1dkgnrlfnpxcna?singleDoc# 《聊聊并写写GQA(Group Query Attention)》