Cthulhu is a high-performance service framework designed for large-scale language model reasoning deployment. It mainly implements Token Batching to meet service requirements.
Apache-2.0
Cthulhu
Cthulhu is a high-performance service framework designed for large-scale language model reasoning deployment. It mainly implements Token Batching to meet service requirements.