rotary-positional-embedding

There are 5 repositories under rotary-positional-embedding topic.

Llama-2.0-architecture-and-inference-from-scratch-with-PyTorch
Language:Python
Decoder-only-transformer_Time_Series_Prediction
使用Decoder-only的Transformer进行时序预测，包含SwiGLU和RoPE(Rotary Positional Embedding)，Time series prediction using Decoder-only Transformer, Including SwiGLU and RoPE(Rotary Positional Embedding)
Language:Python16
pytorch_video_classification
Transformer-Based Video Classification in PyTorch using RoPE
Language:Jupyter Notebook
Language-Modelling
Implementations and Experiments: Transformers, RoPE, KV cache, SAEs, Tokenisers
Language:Python
llama3
A single-file implementation of LLaMA 3, with support for jitting, KV caching and prompting
Language:Python