Issues
- 0
awq quantization is not fully optimized yet. The speed can be slower than non-quantized models.
#57 opened by jackNhat - 1
CUDA out of memory
#42 opened by Chasapas - 1
- 4
Upgrading to VLLM 0.4.1 - TypeError
#38 opened by ProVega - 1
- 1
Using Bentoml v 1.2.10, cant connect to host network and fails to download the debian
#30 opened by OriAlpha - 6
No Swagger UI on localhost:3000
#22 opened by FBR65 - 2
BentoVLLM Service Fails to Start on Linux Server Due to Pydantic Related Errors
#14 opened by Juhong-Namgung