/RWKV-Infer

A large-scale RWKV v6 inference wrapper using the Cuda backend. Easy to deploy on docker. Supports multi-batch generation and dynamic State switching. Let's spread RWKV, which combines RNN technology with impressively low inference costs!

Primary LanguagePythonApache License 2.0Apache-2.0

Stargazers

No one’s star this repository yet.