TechEmpower Benchmark
cgarvis opened this issue · 5 comments
cgarvis commented
Think they are ramping up to run the benchmarks at TechEmpower. Can’t wait to see how bandit does.
mtrudel commented
Great to see! Thanks for the PR over there.
Where do we see the results of this?
mtrudel commented
Closing as there's nothing actionable here, other than to peep the benchmarks when they're released. Thanks for the note!
cgarvis commented
@mtrudel official benchmarks could come out end of March but updated elixir ones to phoenix 1.7 and the latest bandit. Running locally on my M1 Mac I'm seeing ~20% performance increase using Bandit.
$ tail -n 19 20230309162025/phoenix/json/raw.txt
---------------------------------------------------------
Concurrency: 512 for json
wrk -H 'Host: tfb-server' -H 'Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 15 -c 512 --timeout 8 -t 4 "http://tfb-server:8080/json"
---------------------------------------------------------
Running 15s test @ http://tfb-server:8080/json
4 threads and 512 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 8.82ms 7.50ms 329.23ms 94.83%
Req/Sec 15.07k 1.20k 24.71k 81.17%
Latency Distribution
50% 8.24ms
75% 10.15ms
90% 12.70ms
99% 19.08ms
900413 requests in 15.05s, 188.06MB read
Requests/sec: 59810.16
Transfer/sec: 12.49MB
STARTTIME 1678378997
ENDTIME 1678379012
$ tail -n 19 20230309161531/phoenix-bandit/json/raw.txt
---------------------------------------------------------
Concurrency: 512 for json
wrk -H 'Host: tfb-server' -H 'Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 15 -c 512 --timeout 8 -t 4 "http://tfb-server:8080/json"
---------------------------------------------------------
Running 15s test @ http://tfb-server:8080/json
4 threads and 512 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 7.47ms 5.03ms 68.64ms 77.41%
Req/Sec 18.13k 2.36k 31.04k 73.67%
Latency Distribution
50% 5.77ms
75% 9.70ms
90% 14.34ms
99% 23.60ms
1085340 requests in 15.10s, 210.12MB read
Requests/sec: 71884.97
Transfer/sec: 13.92MB
STARTTIME 1678378713
ENDTIME 1678378728
mtrudel commented
I'm more impressed with those latency numbers! Thanks for sharing!