evalscope perf wandb
zll0000 opened this issue · 3 comments
evalscope perf --url 'http://127.0.0.1:65000/v1/chat/completions' --parallel 128 --model 'Qwen2-72B-Instruct' --log-every-n-query 10 --read-timeout=120 --dataset-path './data/open_qa.jsonl' -n 1 --max-prompt-length 128000 --api openai --stream --stop '<|im_end|>' --dataset openqa --debug --wandb-api-key 'my_api_key'
运训完后卡住
s":[{"index":0,"delta":{"content":"风格和"},"logprobs":null,"finish_reason":null}],"usage":null}
2024-08-07 08:22:30,493 - perf - http_client.py - send_requests_worker - 570 - DEBUG - {"id":"cmpl-8e3d5cb891284cde9bc925b092b3f1af","object":"chat.completion.chunk","created":1723018910,"model":"Qwen2-72B-Instruct","choices":[{"index":0,"delta":{"content":"风格和"},"logprobs":null,"finish_reason":null}],"usage":null}
2024-08-07 08:22:30,493 - perf - http_client.py - _handle_stream - 119 - DEBUG -
2024-08-07 08:22:30,548 - perf - http_client.py - _handle_stream - 119 - DEBUG - data: {"id":"cmpl-8e3d5cb891284cde9bc925b092b3f1af","object":"chat.completion.chunk","created":1723018910,"model":"Qwen2-72B-Instruct","choices":[{"index":0,"delta":{"content":"你的团"},"logprobs":null,"finish_reason":null}],"usage":null}
2024-08-07 08:22:30,548 - perf - http_client.py - send_requests_worker - 570 - DEBUG - {"id":"cmpl-8e3d5cb891284cde9bc925b092b3f1af","object":"chat.completion.chunk","created":1723018910,"model":"Qwen2-72B-Instruct","choices":[{"index":0,"delta":{"content":"你的团"},"logprobs":null,"finish_reason":null}],"usage":null}
2024-08-07 08:22:30,548 - perf - http_client.py - _handle_stream - 119 - DEBUG -
2024-08-07 08:22:30,603 - perf - http_client.py - _handle_stream - 119 - DEBUG - data: {"id":"cmpl-8e3d5cb891284cde9bc925b092b3f1af","object":"chat.completion.chunk","created":1723018910,"model":"Qwen2-72B-Instruct","choices":[{"index":0,"delta":{"content":"队"},"logprobs":null,"finish_reason":null}],"usage":null}
2024-08-07 08:22:30,603 - perf - http_client.py - send_requests_worker - 570 - DEBUG - {"id":"cmpl-8e3d5cb891284cde9bc925b092b3f1af","object":"chat.completion.chunk","created":1723018910,"model":"Qwen2-72B-Instruct","choices":[{"index":0,"delta":{"content":"队"},"logprobs":null,"finish_reason":null}],"usage":null}
2024-08-07 08:22:30,603 - perf - http_client.py - _handle_stream - 119 - DEBUG -
2024-08-07 08:22:30,659 - perf - http_client.py - _handle_stream - 119 - DEBUG - data: {"id":"cmpl-8e3d5cb891284cde9bc925b092b3f1af","object":"chat.completion.chunk","created":1723018910,"model":"Qwen2-72B-Instruct","choices":[{"index":0,"delta":{"content":"需求是非常重要的。"},"logprobs":null,"finish_reason":"stop","stop_reason":null}],"usage":null}
2024-08-07 08:22:30,659 - perf - http_client.py - send_requests_worker - 570 - DEBUG - {"id":"cmpl-8e3d5cb891284cde9bc925b092b3f1af","object":"chat.completion.chunk","created":1723018910,"model":"Qwen2-72B-Instruct","choices":[{"index":0,"delta":{"content":"需求是非常重要的。"},"logprobs":null,"finish_reason":"stop","stop_reason":null}],"usage":null}
2024-08-07 08:22:30,659 - perf - http_client.py - _handle_stream - 119 - DEBUG -
2024-08-07 08:22:30,660 - perf - http_client.py - _handle_stream - 119 - DEBUG - data: {"id":"cmpl-8e3d5cb891284cde9bc925b092b3f1af","object":"chat.completion.chunk","created":1723018910,"model":"Qwen2-72B-Instruct","choices":[],"usage":{"prompt_tokens":34,"total_tokens":763,"completion_tokens":729}}
2024-08-07 08:22:30,660 - perf - http_client.py - send_requests_worker - 570 - DEBUG - {"id":"cmpl-8e3d5cb891284cde9bc925b092b3f1af","object":"chat.completion.chunk","created":1723018910,"model":"Qwen2-72B-Instruct","choices":[],"usage":{"prompt_tokens":34,"total_tokens":763,"completion_tokens":729}}
2024-08-07 08:22:30,660 - perf - http_client.py - _handle_stream - 119 - DEBUG -
2024-08-07 08:22:30,660 - perf - http_client.py - _handle_stream - 119 - DEBUG - data: [DONE]
这里卡住不报错也不运行 不加 --wandb_api_key 正常
那可能是网络问题，您确认是否可以脸上wandb服务器。
那可能是网络问题，您确认是否可以脸上wandb服务器。
wandb 网页上测试代码跑是正常的