perf 只发送一条请求
shell-nlp opened this issue · 1 comments
shell-nlp commented
evalscope perf --url 'http://localhost:8082/v1/chat/completions' --parallel 128 --model 'qwen' --log-every-n-query 10 --read-timeout=120 -n 1 --max-prompt-length 128000 --api openai --stream --stop '<|im_end|>' --dataset openqa --dataset-path './datasets/open_qa.jsonl' --debug
我运行以上命令的时候 发现只请求了第一条数据就停止了:
Benchmarking summary:
Time taken for tests: 10.011 seconds
Expected number of requests: 1
Number of concurrency: 128
Total requests: 1
Succeed requests: 1
Failed requests: 0
Average QPS: 0.100
Average latency: 9.822
Throughput(average output tokens per second): 41.556
Average time to first token: 0.319
Average input tokens per request: 23.000
Average output tokens per request: 416.000
Average time per output token: 0.02406
Average package per request: 405.000
Average package latency: 0.024
Too little data to calculate quantiles!
这个是什么问题
jackqdldd commented
-n NUMBER, --number NUMBER
How many requests to be made,