modelscope/evalscope

perf 只发送一条请求

shell-nlp opened this issue · 1 comments

evalscope perf --url 'http://localhost:8082/v1/chat/completions' --parallel 128 --model 'qwen' --log-every-n-query 10 --read-timeout=120 -n 1 --max-prompt-length 128000 --api openai --stream --stop '<|im_end|>' --dataset openqa --dataset-path './datasets/open_qa.jsonl' --debug

我运行以上命令的时候 发现只请求了第一条数据就停止了:

Benchmarking summary: 
 Time taken for tests: 10.011 seconds
 Expected number of requests: 1
 Number of concurrency: 128
 Total requests: 1
 Succeed requests: 1
 Failed requests: 0
 Average QPS: 0.100
 Average latency: 9.822
 Throughput(average output tokens per second): 41.556
 Average time to first token: 0.319
 Average input tokens per request: 23.000
 Average output tokens per request: 416.000
 Average time per output token: 0.02406
 Average package per request: 405.000
 Average package latency: 0.024
 Too little data to calculate quantiles!

这个是什么问题

-n NUMBER, --number NUMBER
How many requests to be made,