mtrudel/bandit

TechEmpower Benchmark

cgarvis opened this issue · 5 comments

cgarvis commented

Think they are ramping up to run the benchmarks at TechEmpower. Can’t wait to see how bandit does.

TechEmpower/FrameworkBenchmarks#7768

mtrudel commented

Great to see! Thanks for the PR over there.

Where do we see the results of this?

mtrudel commented

Closing as there's nothing actionable here, other than to peep the benchmarks when they're released. Thanks for the note!

cgarvis commented

@mtrudel official benchmarks could come out end of March but updated elixir ones to phoenix 1.7 and the latest bandit. Running locally on my M1 Mac I'm seeing ~20% performance increase using Bandit.

$ tail -n 19 20230309162025/phoenix/json/raw.txt
 ---------------------------------------------------------
 Concurrency: 512 for json
 wrk -H 'Host: tfb-server' -H 'Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 15 -c 512 --timeout 8 -t 4 "http://tfb-server:8080/json"
---------------------------------------------------------
Running 15s test @ http://tfb-server:8080/json
  4 threads and 512 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     8.82ms    7.50ms 329.23ms   94.83%
    Req/Sec    15.07k     1.20k   24.71k    81.17%
  Latency Distribution
     50%    8.24ms
     75%   10.15ms
     90%   12.70ms
     99%   19.08ms
  900413 requests in 15.05s, 188.06MB read
Requests/sec:  59810.16
Transfer/sec:     12.49MB
STARTTIME 1678378997
ENDTIME 1678379012

$  tail -n 19 20230309161531/phoenix-bandit/json/raw.txt
  ---------------------------------------------------------
 Concurrency: 512 for json
 wrk -H 'Host: tfb-server' -H 'Accept: application/json,text/html;q=0.9,application/xhtml+xml;q=0.9,application/xml;q=0.8,*/*;q=0.7' -H 'Connection: keep-alive' --latency -d 15 -c 512 --timeout 8 -t 4 "http://tfb-server:8080/json"
---------------------------------------------------------
Running 15s test @ http://tfb-server:8080/json
  4 threads and 512 connections
  Thread Stats   Avg      Stdev     Max   +/- Stdev
    Latency     7.47ms    5.03ms  68.64ms   77.41%
    Req/Sec    18.13k     2.36k   31.04k    73.67%
  Latency Distribution
     50%    5.77ms
     75%    9.70ms
     90%   14.34ms
     99%   23.60ms
  1085340 requests in 15.10s, 210.12MB read
Requests/sec:  71884.97
Transfer/sec:     13.92MB
STARTTIME 1678378713
ENDTIME 1678378728
mtrudel commented

I'm more impressed with those latency numbers! Thanks for sharing!