Concurrent Clients mismatch
emil916 opened this issue · 6 comments
After updating loadtest, I am seeing a different value for CC in the end results. See below:
loadtest -t 5 -c 25 --rps 1000 "http://localhost:30000/fib?30"
Target URL: http://localhost:30000/fib?30
Max time (s): 5
Target rps: 1000
Concurrent clients: 23
Running on cores: 4
Agent: none
Completed requests: 4999
Total errors: 2
Total time: 5.182 s
Mean latency: 3.7 ms
Effective rps: 965
Percentage of requests served within a certain time
50% 3 ms
90% 5 ms
95% 7 ms
99% 12 ms
100% 30 ms (longest request)
-1: 2 errors
Keep in mind that the latest version is creating concurrent clients on demand; if it cannot create 25 clients in the five assigned seconds it will report less, as in your case. Can you try with a longer time to see if it reaches 25? Thanks!
I see. Honestly, I was actually thinking of how the cc (concurrent clients) option becomes useless when one specifies the rps. I feel like when the rps is specified loadtest should be the one to decide the most appropriate number of cc, based on the parameters (load, cpus, input size (file option) etc.).
I will test it out again and report the outcome here. If it matches my mental model I will close the issue, if that is okay with you. If not, feel free to close it.
Thank you!
Emil
PhD student from GW
Oh, you are right! I did not notice the --rps above. Actually it's explicitly mentioned in the documentation:
Note: the `--concurrency` option will be ignored if `--requestsPerSecond` is specified;
clients will be created on demand.
Oh, then my bad. Is it a new thing or was it like this since old times? (I dont have the most update version yet).
I think this Note should be shown as a warning when one uses both options anyways.
It's a new thing for versions 8+ I think. Instead of showing the warning I opted for displaying the actual number of concurrent clients, perhaps it could be made more evident 🤔
Alright, it all make sense now. Thanks! Closing the issue.