Issues
- 0
Running FastChat on GPTQ (and quantized) models
#3530 opened by NamburiSrinath - 2
Does the FastChat framework support multiple NPU reasoning? I changed the value of num_gpus to 4, but after he loads the model, it will not distribute it equally to each graphics card.
#3459 opened by xunmenglt - 0
Inquire about adding a new language to the category section on the Arena
#3526 opened by alielfilali01 - 0
How to use best-of-N in MT-Bench
#3524 opened by Yuancheng-Xu - 1
lmarena.ai is down with 503 Service Unavailable
#3522 opened by youself64github - 8
不更新了吗?
#3473 opened by a1015498799 - 3
Some models on Chatbot Arena causes network error
#3503 opened by youself64github - 0
GPTQ models - are they deprecated?
#3520 opened by surak - 1
【bug】KeyError: 'factor' when use Triplex
#3470 opened by ForgetThatNight - 1
- 0
- 1
[BUG] vllm_worker: RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method
#3507 opened by quanshr - 1
Is the logprobs of fschat==0.2.36 supported now?
#3506 opened by wupaopao123 - 0
- 0
sglang_work not working on sglang v0.2.13
#3498 opened by exceedzhang - 0
is_blocking is bool
#3497 opened by tp-nan - 3
After multiple model workers start working concurrently for the first time, requests will only be received by one of the workers.
#3484 opened by PaulX1029 - 4
- 0
插件使用process.env读取不到环境变量
#3485 opened by blvyoucan - 1
[Bug]: Garbled Tokens appears in vllm generation result every time change to new LLM model
#3430 opened by Jason-csc - 3
AttributeError: 'GenerationConfig' object has no attribute '_eos_token_tensor'
#3479 opened by Demainlip - 0
Support BaiLing model
#3487 opened by cuauty - 0
Request to add [HODACHI/EZO-Humanities-9B-gemma-2-it] and [HODACHI/EZO-Common-9B-gemma-2-it] to Chatbot Arena
#3481 opened by kazuya-hodatsu-336-1 - 0
`httpx.RemoteProtocolError: peer closed connection without sending complete message body (incomplete chunked read)`
#3480 opened by DHaru85 - 0
ERROR | stderr | ERROR: [Errno 99] error while attempting to bind on address ('::1', 21002, 0, 0): cannot assign requested address
#3475 opened by LIUKAI0815 - 0
Llama3 Local Model
#3474 opened by sven-123 - 0
⚔️ Arena (side-by-side) sometimes show one model as "rate-exceeded" while the other model keeps generating things.
#3472 opened by yuxi-liu-wired - 0
How to modify URLs when using third-party API
#3471 opened by ChangPingfirst - 3
When using sglang as the inference framework, if a word starting with "\n" appears in the stop parameter, the sglang will Missing '\n' during inference
#3467 opened by nstl-zyb - 1
Does the current project provide an interface for calculating the number of tokens for testing inference performance?
#3464 opened by tensorflowt - 0
Slower throughput with openai_server
#3468 opened by tacacs1101-debug - 0
Duplicate GeminiAdapter class definition found
#3462 opened by KangmoonSeo - 0
npu 910B run fastchat + baichuan-13B: DefaultCPUAllocator: can't allocate memory
#3458 opened by GuIIWen - 0
No permission to push to the branch
#3455 opened by caizhi-mt - 1
[Suggest] When to support Codestral
#3438 opened by lance-maxwell - 3
Error in Gemma 2 using model_worker (probably an error in conversation.py)
#3448 opened by vikrantrathore - 0
Does the FastChat model support gemma2-27b it?
#3454 opened by zhouyuustc - 0
Does MT-Bench gen_model_answer.py accept adapter weights (adapter_config.json) file as model path?
#3453 opened by RitikParkar - 0
- 4
Support Mistral-7B-Instruct-v0.3
#3450 opened by WxxShirley - 0
How to finetune mistral-7b with fastchat?
#3449 opened by Jerry-hyl - 1
- 0
- 0
Model worker with Nvidia NIM?
#3443 opened by surak - 1
Error reported when deploying DeepSeek-Coder-V2-Lite-Chat with latest Fastchat
#3429 opened by xierbut - 0
Dealing with filtering user queries where they ask models to give their names
#3441 opened by fjiangAI - 1
Must provide either the `api_version` argument or the `OPENAI_API_VERSION` environment variable
#3440 opened by Bob199511 - 0
- 0
Error reported when deploying DeepSeek-Coder-V2-Lite-Chat with latest Fastchat
#3428 opened by xierbut - 0