Which is the fastest?
This project aims to be a load benchmarking suite, no more, no less
Measuring response times (routing times) for each framework (middleware).
Results are not production-ready yet
Additional purposes :
- Helping decide between languages, depending on use case
- Learning languages, best practices, devops culture ...
- Having fun ❤️
Requirements
- Crystal as
built-intools are made in this language - Docker as frameworks are
isolatedinto containers - wrk as benchmarking tool,
>= 4.1.0 - postgresql to store data,
>= 10
ℹ️ℹ️ℹ️ℹ️ℹ️
OSX you need docker-machine to use docker containerization
brew install docker-machine
docker-machine create default
eval $(docker-machine env default)
ℹ️ℹ️ℹ️ℹ️ℹ️
Usage
- Install all dependencies
shards install- Build internal tools
shards build- Create and initialize the database
createdb -U postgres benchmark
psql -U postgres -d benchmark < .ci/dump.sqlDocker can be used to set up the database:
docker run -it --rm -d \
-p 5432:5432 \
-e POSTGRES_DB=benchmark \
-e POSTGRES_HOST_AUTH_METHOD=trust \
-v /tmp/pg-data:/var/lib/postgresql/data \
--name pg postgres:12-alpineWait several seconds for the container to start, then inject the dump:
docker exec pg sh -c "echo \"$(cat .ci/dump.sql)\" | psql -U postgres -d benchmark"After creating the database, export its URL:
export DATABASE_URL="postgresql://postgres@localhost/benchmark"- Make configuration
bin/make config- Build containers
jobs are either languages (example : crystal) or frameworks (example : router.cr)
bin/neph [job1] [job2] [job3] ...- Export all results readme
bin/db to_readmeResults
ℹ️ Updated on 2020-06-09 ℹ️
Benchmarking with wrk
- Threads : 8
- Timeout : 8
- Duration : 15s (seconds)
ℹ️ Sorted by max req/s on concurrency 64 ℹ️
| Language | Framework | Speed (64) | Speed (256) | Speed (512) | |
|---|---|---|---|---|---|
| 1 | nim (1.2) | httpbeast (0.2) | 187 194 | 199 770 | 200 472 |
| 2 | javascript (13.14) | nanoexpress-pro (1.11) | 179 666 | 195 557 | 189 730 |
| 3 | php (7.4) | simps (1.0) | 175 799 | 191 078 | 192 432 |
| 4 | javascript (13.14) | sifrr (0.0) | 170 130 | 185 206 | 189 772 |
| 5 | go (1.14) | fasthttp (1.14) | 162 949 | 172 822 | 175 894 |
| 6 | java (8) | jooby (2.8) | 160 456 | 181 791 | 183 543 |
| 7 | java (8) | rapidoid (5.5) | 159 637 | 174 917 | 175 031 |
| 8 | javascript (13.14) | nanoexpress (2.0) | 158 207 | 171 961 | 172 158 |
| 9 | kotlin (1.3) | kooby (2.8) | 158 154 | 186 320 | 188 450 |
| 10 | php (7.4) | workerman (4.0) | 157 628 | 170 695 | 171 630 |
| 11 | go (1.14) | fiber (1.10) | 157 534 | 164 509 | 162 250 |
| 12 | crystal (0.34) | toro (0.4) | 157 376 | 163 717 | 160 174 |
| 13 | go (1.14) | router (1.1) | 156 849 | 166 378 | 169 696 |
| 14 | go (1.14) | fasthttprouter (0.1) | 156 611 | 166 402 | 169 311 |
| 15 | crystal (0.34) | router.cr (0.2) | 156 319 | 163 551 | 159 593 |
| 16 | go (1.14) | atreugo (11.3) | 155 325 | 166 331 | 170 705 |
| 17 | go (1.14) | gorouter-fasthttp (4.4) | 154 573 | 164 118 | 167 454 |
| 18 | crystal (0.34) | spider-gazelle (3.0) | 152 780 | 159 921 | 156 463 |
| 19 | crystal (0.34) | grip (0.28) | 147 173 | 152 873 | 148 687 |
| 20 | crystal (0.34) | kemal (0.26) | 144 775 | 151 860 | 148 220 |
| 21 | nim (1.2) | jester (0.4) | 142 221 | 152 088 | 153 070 |
| 22 | crystal (0.34) | amber (0.34) | 137 534 | 141 684 | 139 078 |
| 23 | rust (1.44) | actix (2.0) | 135 049 | 137 762 | 139 488 |
| 24 | crystal (0.34) | orion (2.3) | 132 241 | 135 477 | 130 884 |
| 25 | crystal (0.34) | athena (0.8) | 124 858 | 126 515 | 119 728 |
| 26 | java (8) | act (1.8) | 120 779 | 132 402 | 132 376 |
| 27 | go (1.14) | rte (0.0) | 109 826 | 108 975 | 112 698 |
| 28 | go (1.14) | echo (4.1) | 108 926 | 108 743 | 112 032 |
| 29 | go (1.14) | httprouter (1.3) | 108 702 | 107 486 | 110 916 |
| 30 | go (1.14) | gin (1.6) | 105 999 | 108 778 | 111 994 |
| 31 | go (1.14) | chi (4.1) | 104 365 | 102 134 | 105 641 |
| 32 | go (1.14) | gorouter (4.4) | 104 189 | 106 959 | 109 760 |
| 33 | fsharp (4.7) | frank (6.0) | 102 691 | 109 611 | 111 141 |
| 34 | go (1.14) | aero (1.3) | 102 269 | 102 221 | 105 211 |
| 35 | csharp (8.0) | aspnetcore (3.1) | 101 209 | 107 750 | 105 295 |
| 36 | go (1.14) | violetear (7.0) | 101 123 | 99 962 | 103 792 |
| 37 | go (1.14) | goroute (0.0) | 99 261 | 97 129 | 100 102 |
| 38 | go (1.14) | gorilla-mux (1.7) | 98 135 | 94 847 | 98 054 |
| 39 | go (1.14) | kami (2.2) | 97 858 | 102 054 | 103 869 |
| 40 | ruby (2.7) | agoo (2.12) | 96 424 | 119 233 | 123 228 |
| 41 | go (1.14) | beego (1.12) | 95 973 | 98 990 | 101 788 |
| 42 | cpp (14/17) | drogon (1.0) | 89 209 | 93 729 | 97 267 |
| 43 | fsharp (4.7) | falco (1.1) | 88 585 | 94 545 | 95 288 |
| 44 | fsharp (4.7) | giraffe (4.1) | 86 737 | 89 969 | 88 602 |
| 45 | javascript (13.14) | 0http (2.5) | 85 021 | 92 963 | 89 895 |
| 46 | go (1.14) | air (0.17) | 83 407 | 84 394 | 86 397 |
| 47 | javascript (13.14) | polka (0.5) | 81 667 | 86 654 | 82 928 |
| 48 | elixir (1.1) | cowboy_stream (2.8) | 79 967 | 79 836 | 78 165 |
| 49 | c (99) | kore (3.3) | 79 515 | 135 948 | 130 080 |
| 50 | javascript (13.14) | polkadot (1.0) | 78 811 | 91 647 | 87 981 |
| 51 | javascript (13.14) | restana (4.6) | 75 986 | 83 157 | 84 910 |
| 52 | javascript (13.14) | rayo (1.3) | 75 496 | 80 301 | 81 948 |
| 53 | swift (5.2) | perfect (3.1) | 75 278 | 83 980 | 88 688 |
| 54 | go (1.14) | gf (1.13) | 75 196 | 80 603 | 81 818 |
| 55 | scala (2.13) | akkahttp (10.1) | 74 268 | 80 208 | 77 118 |
| 56 | java (8) | javalin (3.8) | 73 984 | 78 403 | 78 700 |
| 57 | php (7.4) | siler-swoole (1.7) | 70 558 | 77 476 | 77 463 |
| 58 | javascript (13.14) | muneem (2.4) | 69 494 | 75 670 | 74 133 |
| 59 | c (11) | agoo-c (0.7) | 68 812 | 182 189 | 184 122 |
| 60 | python (3.8) | falcon (2.0) | 67 169 | 72 134 | 72 286 |
| 61 | kotlin (1.3) | ktor (1.2) | 66 758 | 79 332 | 79 860 |
| 62 | java (8) | spring-boot (2.2) | 66 040 | 72 664 | 71 609 |
| 63 | php (7.4) | hyperf (1.1) | 65 297 | 69 088 | 69 442 |
| 64 | go (1.14) | mars (1.0) | 64 596 | 63 453 | 66 231 |
| 65 | java (8) | micronaut (1.2) | 63 986 | 70 963 | 70 979 |
| 66 | javascript (13.14) | foxify (0.1) | 63 624 | 67 467 | 65 508 |
| 67 | php (7.4) | one (2.0) | 62 643 | 67 503 | 67 627 |
| 68 | haskell (8.8) | scotty (0.12) | 62 605 | 64 842 | 69 197 |
| 69 | javascript (13.14) | nestjs-fastify (7.1) | 60 967 | 65 313 | 63 730 |
| 70 | javascript (13.14) | koa (2.12) | 60 730 | 62 195 | 60 120 |
| 71 | javascript (13.14) | fastify (2.14) | 60 551 | 65 254 | 62 459 |
| 72 | python (3.8) | bottle (0.12) | 59 075 | 61 568 | 61 765 |
| 73 | rust (1.44) | nickel (0.11) | 56 105 | 55 716 | 55 766 |
| 74 | clojure (1.1) | coast (1.0) | 55 273 | 56 858 | 56 541 |
| 75 | java (8) | spring-framework (5.2) | 55 157 | 62 586 | 62 345 |
| 76 | python (3.8) | apidaora (0.26) | 54 920 | 59 749 | 59 429 |
| 77 | swift (5.2) | kitura-nio (2.9) | 54 895 | 55 742 | 55 517 |
| 78 | swift (5.2) | kitura (2.9) | 54 053 | 53 173 | 53 422 |
| 79 | python (3.8) | asgineer (0.7) | 53 201 | 57 291 | 57 204 |
| 80 | javascript (13.14) | express (4.17) | 53 062 | 54 505 | 54 698 |
| 81 | javascript (13.14) | feathersjs (4.5) | 52 593 | 53 880 | 52 546 |
| 82 | rust (1.44) | gotham (0.4) | 52 113 | 58 921 | 59 836 |
| 83 | javascript (13.14) | iotjs-express (0.0) | 50 925 | 55 034 | 54 292 |
| 84 | scala (2.13) | http4s (0.21) | 50 905 | 53 598 | 51 967 |
| 85 | swift (5.2) | vapor (4.8) | 48 620 | 50 708 | 50 168 |
| 86 | elixir (1.1) | cowboy (2.8) | 48 448 | 48 422 | 49 335 |
| 87 | python (3.8) | blacksheep (0.2) | 48 027 | 53 757 | 53 650 |
| 88 | python (3.8) | pyramid (1.1) | 46 810 | 48 020 | 48 014 |
| 89 | php (7.4) | imi (1.2) | 46 752 | 48 820 | 48 672 |
| 90 | javascript (13.14) | moleculer (0.14) | 46 358 | 47 295 | 47 908 |
| 91 | python (3.8) | hug (2.6) | 45 592 | 47 438 | 47 449 |
| 92 | python (3.8) | sanic (20.3) | 45 577 | 49 761 | 48 768 |
| 93 | python (3.8) | starlette (0.13) | 44 811 | 48 026 | 47 558 |
| 94 | cpp (11) | evhtp (1.2) | 44 136 | 44 542 | 44 713 |
| 95 | javascript (13.14) | hapi (19.1) | 41 677 | 43 273 | 42 825 |
| 96 | elixir (1.1) | plug (1.10) | 40 131 | 41 216 | 41 488 |
| 97 | javascript (13.14) | restify (8.5) | 38 995 | 41 387 | 40 114 |
| 98 | php (7.4) | sw-fw-less (preview) | 38 707 | 41 050 | 41 004 |
| 99 | scala (2.13) | play (2.8) | 38 611 | 39 934 | 39 589 |
| 100 | ruby (2.7) | hanami-api (0.1) | 38 134 | 38 530 | 35 550 |
| 101 | python (3.8) | emmett (2.0) | 37 735 | 40 690 | 40 252 |
| 102 | javascript (13.14) | nestjs-express (7.1) | 37 147 | 39 593 | 40 277 |
| 103 | ruby (2.7) | syro (3.2) | 36 042 | 38 628 | 36 325 |
| 104 | php (7.4) | swoft (2.0) | 35 933 | 37 219 | 36 905 |
| 105 | elixir (1.1) | phoenix (1.5) | 35 811 | 36 277 | 36 357 |
| 106 | ruby (2.7) | roda (3.32) | 34 931 | 36 276 | 34 682 |
| 107 | php (7.4) | yii-swoole (2.0) | 34 733 | 35 817 | 35 623 |
| 108 | clojure (1.1) | luminus (1.0) | 33 448 | 34 443 | 34 214 |
| 109 | fsharp (4.7) | suave (2.5) | 32 724 | 32 353 | 34 207 |
| 110 | dart (2.8) | aqueduct (3.3) | 32 062 | 32 020 | 31 212 |
| 111 | ruby (2.7) | cuba (3.9) | 32 048 | 32 223 | 31 090 |
| 112 | haskell (8.8) | servant (0.17) | 30 474 | 29 529 | 28 708 |
| 113 | crystal (0.34) | shivneri (0.15) | 29 452 | 27 529 | 25 838 |
| 114 | python (3.8) | responder (2.0) | 29 113 | 30 911 | 30 510 |
| 115 | python (3.8) | fastapi (0.55) | 28 387 | 30 706 | 30 445 |
| 116 | ruby (2.7) | rack_app (7.6) | 26 348 | 26 327 | 24 834 |
| 117 | python (3.8) | molten (1.0) | 26 196 | 26 845 | 26 840 |
| 118 | ruby (2.7) | camping (2.1) | 25 929 | 25 537 | 25 047 |
| 119 | python (3.8) | clastic (19.9) | 25 435 | 25 927 | 25 661 |
| 120 | python (3.8) | aiohttp (3.6) | 25 298 | 27 167 | 27 287 |
| 121 | go (1.14) | macaron (1.3) | 25 090 | 27 803 | 28 001 |
| 122 | ruby (2.7) | rack-routing (0.0) | 24 628 | 25 062 | 25 459 |
| 123 | rust (1.44) | iron (0.6) | 24 033 | 24 214 | 24 129 |
| 124 | python (3.8) | masonite (2.3) | 23 457 | 24 034 | 23 728 |
| 125 | python (3.8) | flask (1.1) | 23 411 | 24 014 | 23 822 |
| 126 | javascript (13.14) | turbo_polka (0.3) | 22 294 | 21 465 | 20 314 |
| 127 | php (7.4) | spiral (2.4) | 17 766 | 18 013 | 18 207 |
| 128 | java (8) | blade (2.0) | 17 462 | 20 522 | 19 621 |
| 129 | go (1.14) | tango (0.6) | 17 336 | 17 642 | 17 666 |
| 130 | ruby (2.7) | sinatra (2.0) | 15 276 | 14 672 | 14 687 |
| 131 | go (1.14) | gramework (1.7) | 14 555 | 14 934 | 14 825 |
| 132 | ruby (2.7) | grape (1.3) | 14 147 | 13 995 | 14 592 |
| 133 | dart (2.8) | start (0.4) | 14 067 | 13 758 | 12 878 |
| 134 | python (3.8) | quart (0.12) | 11 918 | 11 769 | 11 037 |
| 135 | php (7.4) | fastsitephp (1.2) | 11 414 | 11 265 | 11 152 |
| 136 | javascript (13.14) | sails (1.2) | 11 402 | 11 830 | 13 171 |
| 137 | ruby (2.7) | flame (4.18) | 11 206 | 10 623 | 10 858 |
| 138 | swift (5.2) | swifter (1.4) | 11 011 | 11 049 | 11 018 |
| 139 | pony (0.35) | jennet (0.1) | 10 714 | 17 373 | 16 072 |
| 140 | ruby (2.7) | hanami (1.3) | 10 485 | 10 488 | 10 496 |
| 141 | python (3.8) | django (3.0) | 9 890 | 9 757 | 9 573 |
| 142 | python (3.8) | tornado (6.0) | 9 876 | 10 323 | 10 085 |
| 143 | php (7.4) | ubiquity (2.3) | 9 301 | 9 295 | 9 281 |
| 144 | python (3.8) | cherrypy (18.6) | 9 181 | 9 092 | 8 988 |
| 145 | php (7.4) | one-fpm (2.0) | 8 032 | 8 081 | 7 986 |
| 146 | php (7.4) | siler (1.7) | 7 765 | 7 821 | 7 763 |
| 147 | php (7.4) | phalcon (4.0) | 7 626 | 7 700 | 7 680 |
| 148 | php (7.4) | hamlet (3.2) | 7 576 | 7 660 | 7 555 |
| 149 | php (7.4) | ice (1.5) | 6 265 | 6 285 | 6 425 |
| 150 | crystal (0.34) | onyx (0.5) | 5 851 | 6 038 | 6 019 |
| 151 | php (7.4) | chubbyphp (2.8) | 5 752 | 5 799 | 5 917 |
| 152 | php (7.4) | slim (4.5) | 4 688 | 4 784 | 4 842 |
| 153 | php (7.4) | nette (3.0) | 4 358 | 4 465 | 4 527 |
| 154 | php (7.4) | lumen (7.1) | 4 285 | 4 342 | 4 430 |
| 155 | php (7.4) | yii (2.0) | 4 164 | 4 234 | 4 348 |
| 156 | ruby (2.7) | rails (6.0) | 3 910 | 3 707 | 3 508 |
| 157 | julia (1.5) | merly (0.2) | 3 218 | 7 827 | 6 185 |
| 158 | php (7.4) | symfony (5.1) | 3 177 | 3 227 | 3 290 |
| 159 | php (7.4) | mezzio (3.2) | 2 856 | 2 921 | 2 984 |
| 160 | python (3.8) | cyclone (1.3) | 2 224 | 2 235 | 2 207 |
| 161 | perl (5.3) | dancer2 (2.0) | 1 848 | 1 664 | 805 |
| 162 | python (3.8) | klein (19.6) | 1 589 | 1 622 | 1 602 |
| 163 | python (3.8) | nameko (2.12) | 1 552 | 1 514 | 1 489 |
| 164 | crystal (0.34) | lucky (0.21) | 1 469 | 1 506 | 1 489 |
| 165 | php (7.4) | laminas (3.1) | 1 440 | 1 467 | 1 465 |
| 166 | php (7.4) | basicphp (0.9) | 534 | 442 | 1 838 |
| 167 | php (7.4) | laravel (7.14) | 208 | 175 | 1 938 |
| 168 | go (1.14) | webgo (4.0) | 0 | 0 | 0 |
How to contribute ?
In any way you want ...
- Request a framework addition
- Report a bug (on any implementation)
- Suggest an idea
- ...
Any kind of idea is ❤️
Contributors
- Taichiro Suzuki - Author | Maintainer
- OvermindDL1 - Maintainer
- Marwan Rabbâa - Maintainer