Cthulhu is a high-performance service framework designed for large-scale language model reasoning deployment. It mainly implements Token Batching to meet service requirements.
Apache License 2.0Apache-2.0
No issues in this repository yet.