binakot/Abstract-Rest-Service-Benchmark

A set of template services in different languages and frameworks. An abstract performance comparison based on the endpoint throughput. There are no objective benefits, just an elementary comparison of primitive RESTful API services. :shipit:

Abstract Rest Service Benchmark

Requirements

• PORT: 8080

• API request: /api/test

• API response: Hello, World! (text/plain, UTF-8, 13 bytes)

Benchmark

Command for WRK:

$ wrk -t8 -c512 -d2m --timeout 10s --latency http://localhost:8080/api/test

---

Results

#	Service	Language	Framework	RPS
	Java Light 4J	Java	oracle jdk 9.0.4 + ligth4j	401511.94
	.Net Core	C#	dotnet 2.1.4	249758.46
	Haskell Warp	Haskell	ghc 7.10.3 + warp	191020.62
	Rust Iron	Rust	rust 1.24.1 + iron	186726.17
	Go Gorilla/Mux	Golang	go sdk 1.9.4 + gorilla/mux	131499.79
	.Net Core ASP.NET MVC	C#	dotnet 2.1.4	99091.85
	NodeJS	JavaScript	nodejs 8.9.4	94132.92
	Python Sanic	Python	python 3.5.2 + sanic	74854.41
	NodeJS Express	JavaScript	nodejs 8.9.4 + express	58769.29
	Java Spring Boot	Java	oracle jdk 9.0.4 + spring boot	56253.12
	.Net Core Freya	F#	dotnet 2.1.4 + freya-hopac	49736.08
	Python Aiohttp	Python	python 3.5.2 + aiohttp	4603.01
	Python Flask	Python	python 3.5.2 + flask	200.44
	.Net Core Websocket	C#	dotnet 2.1.4 + websocket	TBD
	.Net Core Giraffe	F#	dotnet 2.1.4 + giraffe	TBD
	Python API Star	Python	python 3.5.2 + apistar	TBD
	Rust Rocket	Rust	rust 1.24.1 + rocket	TBD

---

Services

.Net Core by @vanbukin

Language: C#

Framework: .Net Core 2

Run:

$ dotnet restore
$ dotnet publish -c Release
$ export ASPNETCORE_ENVIRONMENT=Production
$ export ASPNETCORE_URLS=http://0.0.0.0:8080
$ cd ./dotNetCoreRestService/bin/Release/netcoreapp2.1/publish
$ dotnet dotNetCoreRestService.dll

Result:

TBD

---

.Net Core ASP.NET MVC by @binakot

Language: C#

Framework: .Net Core 2, ASP.NET MVC

Main tutorial: https://docs.microsoft.com/en-us/aspnet/core/tutorials/first-web-api

Run:

$ dotnet restore
$ dotnet publish -c Release
$ export ASPNETCORE_ENVIRONMENT=Production
$ export ASPNETCORE_URLS=http://0.0.0.0:8080
$ cd ./dotNetCoreRestService/bin/Release/netcoreapp2.0/publish
$ dotnet dotNetCoreRestService.dll

Result:

TBD

---

.Net Core Websocket by @petriashev

Language: C#

Framework: .Net Core 2, Websocket

Run:

$ dotnet restore
$ dotnet publish -c Release
$ export ASPNETCORE_ENVIRONMENT=Production
$ export ASPNETCORE_URLS=http://0.0.0.0:8080
$ cd ./bin/Release/netcoreapp2.0/publish
$ dotnet dot-net-core-websocket.dll

Result:

TBD

---

.Net Core Freya by @Szer

Language: F#

Framework: .Net Core 2, Freya

Run:

$ dotnet restore
$ dotnet publish -c Release
$ export ASPNETCORE_ENVIRONMENT=Production
$ export ASPNETCORE_URLS=http://0.0.0.0:8080
$ cd ./bin/Release/netcoreapp2.0/publish
$ dotnet dot-net-core-hopac-freya.dll

Result:

TBD

---

.Net Core Giraffe by @Szer

Language: F#

Framework: .Net Core 2, Giraffe

Run:

$ dotnet restore
$ dotnet publish -c Release
$ export ASPNETCORE_ENVIRONMENT=Production
$ export ASPNETCORE_URLS=http://0.0.0.0:8080
$ cd ./bin/Release/netcoreapp2.0/publish
$ dotnet dot-net-core-giraffe.dll

Result:

TBD

---

Go Gorilla/Mux by @binakot

Language: Go

Framework: Go SDK, Gorilla/Mux

Main tutorial: https://www.codementor.io/codehakase/building-a-restful-api-with-golang-a6yivzqdo

Run:

$ go build
$ ./go

Result:

TBD

---

Haskell Warp by @kchugalinskiy

Language: Haskell

Framework: GHC, Warp

Main tutorial: http://taylor.fausak.me/2014/10/21/building-a-json-rest-api-in-haskell

Requirements:

• Install Haskell stack: https://docs.haskellstack.org/en/stable/README
• Install Cabal package: cabal install Cabal-2.0.1.1

Run:

$ stack setup
$ stack build
$ stack exec .stack-work/dist/**/build/test-exe/test-exe

Result:

TBD

---

Java

Language: Java

Framework: Oracle JDK 8 or above

Run:

$ gradle clean run

Result:

TBD

---

Java Light 4J by @0ffer

Language: Java

Framework: Oracle JDK 9, Light4J

Main tutorial: https://github.com/networknt/light-example-4j/tree/master/demo

Run:

$ mvn clean install
$ java -jar target/service-example-0.1.0.jar

Result:

TBD

---

Java Spring Boot by @binakot

Language: Java

Framework: Oracle JDK 9, Spring Boot

Main tutorial: http://spring.io/guides/gs/rest-service

Run:

$ gradle clean build
$ java -jar build/libs/java-spring-boot-rest-service-1.0-SNAPSHOT.jar

Result:

TBD

---

NodeJS by @WilixLead

Language: JavaScript

Framework: Node.js

Main tutorial: https://nodejs.org/api/http.html#http_class_http_server

Run:

$ node ./index.js

Result:

TBD

---

NodeJS Express by @WilixLead

Language: JavaScript

Framework: Node.js, Express

Main tutorial: http://expressjs.com/en/starter/hello-world.html

Run:

$ npm i
$ node ./index.js

Result:

TBD

---

Python Aiohttp by @TyVik

Language: Python

Framework: Python 3, Aiohttp

Main tutorial: https://aiohttp.readthedocs.io/en/stable/

Run:

$ pip install -r requirements.txt
$ python app.py

Result:

TBD

---

Python API Star by @sergeibershadsky

Language: Python

Framework: Python 3, API Star

Main tutorial: https://github.com/encode/apistar

Run:

pip install -r requirements.txt
uvicorn app:app --workers=8 --bind=0.0.0.0:8080 --pid=pid

Results:

TBD

---

Python Flask by @TyVik

Language: Python

Framework: Python 3, Flask

Main tutorial: http://flask.pocoo.org/

Run:

$ pip install -r requirements.txt
$ python app.py

Result:

TBD

---

Python Sanic by @TyVik

Language: Python

Framework: Python 3, Sanic

Main tutorial: http://sanic.readthedocs.io/en/latest/

Run:

$ pip install -r requirements.txt
$ python app.py

Results:

TBD

---

Rust Iron by @kchugalinskiy

Language: Rust

Framework: Rust SDK, Iron

Main tutorial: https://github.com/iron/router/blob/master/examples/simple.rs

Run:

$ cargo run --release

Result:

TBD

---

Rust Rocket by @qezz

Language: Rust

Framework: Rust SDK, Rocket

Main tutorial: https://rocket.rs/guide/getting-started/#hello-world

Run:

$ ROCKET_ENV=prod cargo run --release

Result:

TBD

---

Hardware

• Server (rest-service) host:

TBD

• Client (wrk) host:

TBD