bench some web microframeworks (Ruby)
Because we don't have enough drama, let's bench some hello world apps!
Why?
As part of some research, collected some benchmark around Ruby web microframeworks.
The idea is bench the performance from Rack, throught the stack of the framework to the minimum representation of an application (Hello World apps).
It is not the goal to bechmark hits to the database or external services, nor to do a silly fibonacci-as-a-service, ok?
What?
Rack 2 compatible frameworks
- Cuba - cuba.ru
- Flame - flame.ru
- Grape - grape.ru
- Hanami (Router) - hanami-router.ru
- Hobbit - hobbit.ru
- Hobby - hobby.ru
- Newark - newark.ru
- Plezi - plezi.ru
- Rack - rack.ru + rack-response.ru
- Rack-App - rack-app.ru
- RackStep - rackstep.ru
- Rails (API + Metal) - rails-api.ru + rails-metal.ru
- Rambutan - rambutan.ru
- Roda - roda.ru
- Sinatra - sinatra.ru
- Syro - syro
- Watts - watts
Please note that while Rails has been added to the list, it is just a minimalistic representation with ActionPack gem (using API and Metal, no full middleware stack, etc). You shouldn't take the performance numbers mentioned here about Rails (or any other) as scientific and decision-taking references.
How?
Used wrk to benchmark, locally, a burst of requests (in 2 threads) over 10 seconds. The command line used was:
$ wrk -t 2 http://localhost:9292/
All the frameworks using Puma on Ruby 2.4, in production mode with 16 threads:
$ puma -e production -t 16:16 apps/<framework.ru>
Run benchmark for all frameworks
# use `bundle exec`, if needed
$ bench/summary-memory
$ bench/summary-speed
Have some numbers around?
Yup, I do:
Requests/sec
Framework Requests/sec % from best
----------------------------------------------
rack 3912.20 100.00%
syro 3626.70 92.70%
watts 3560.82 91.02%
cuba 3509.21 89.70%
rack-response 3351.42 85.67%
rambutan 3285.18 83.97%
roda 3203.41 81.88%
hanami-router 3115.99 79.65%
hobbit 3080.47 78.74%
hobby 3022.36 77.25%
rack-app 2950.24 75.41%
newark 2930.81 74.91%
plezi 2822.49 72.15%
rackstep 2748.13 70.25%
rails-metal 2202.20 56.29%
sinatra 2157.43 55.15%
grape 1813.13 46.35%
flame 1787.63 45.69%
rails-api 616.35 15.75%
Memory Allocation/Request
Framework Allocs/Req Memsize/Req
--------------------------------------
rack 40 2880
roda 43 3488
syro 44 3568
cuba 45 3568
watts 47 3160
hobbit 48 3888
hobby 52 4232
rack-response 55 4096
rails-metal 62 5848
plezi 65 5000
hanami-router 66 4800
newark 71 5888
rackstep 79 4864
rambutan 81 5864
rack-app 82 7336
flame 123 8456
sinatra 188 12160
grape 253 20008
rails-api 370 27050
Hardware
These numbers were collected on:
- Ubuntu 16.10 64-bit (kernel: 4.8.0-56-generic)
- Dell XPS 13 (9343, QHD, Developer Edition)
- Intel® Core™ i7-5500U CPU @ 2.40GHz × 4
- 8GB RAM
- 256GB SSD
- ruby 2.4.1p111 (2017-03-22 revision 58053) [x86_64-linux]