the-crypt-keeper/can-ai-code

Upgrade to exllamav2 dynamic generator to support batching

Closed this issue 3 months ago · 0 comments

the-crypt-keeper commented 3 months ago

https://github.com/turboderp/exllamav2?tab=readme-ov-file#dynamic-generator