Tradias/asio-grpc

[Question] About bench mark

Closed this issue · 2 comments

rogerwrld commented

Hello Tradias,

I have a question regarding the bench mark provided in the read me page.
I tried to make a multi threaded server using your great library.
I used the streaming example as reference to send a request containing just the word Hello. Using 20 threads.
But I'm a little bit surprised by the performance.
I got 10K 11K req/s.
This so far from the bench marks provided (60k~70k)
My server CPU Intel R Xeon 4114 CPU @ 2.2 GHz
(nproc 40)
I used cpp client and nodejs but I got the same KPIs

Is it possible that the unary call is most efficient than streaming call ?
Could you please help on that ?

Thank you in advance

Tradias commented

I have added a client-streaming benchmark to https://github.com/Tradias/grpc_bench. Currently only for cpp_asio_grpc_callback_bench and cpp_asio_grpc_coroutine_bench. You run it using

./build.sh cpp_asio_grpc_callback_bench cpp_asio_grpc_coroutine_bench

export GRPC_USE_CLIENT_STREAMING=true
export GRPC_REQUEST_SCENARIO=string_100B
# further env vars like GRPC_SERVER_CPUS and GRPC_CLIENT_CPUS
./bench.sh cpp_asio_grpc_callback_bench cpp_asio_grpc_coroutine_bench

When running the benchmarks make sure that:

  • GRPC_CLIENT_CPUS is set high enough. In the result, avg. cpu must be equal to 100% * GRPC_SERVER_CPUS.
  • Your CPU is not running at 100% during the benchmark.

On my machine I only see a minor difference between streaming and unary:

name req/s avg. latency 90 % in 95 % in 99 % in avg. cpu avg. memory
cpp_asio_grpc_callback 27007 36.86 ms 39.64 ms 40.34 ms 41.94 ms 102.74% 6.45 MiB
cpp_asio_grpc_coroutine 25190 39.54 ms 42.16 ms 42.83 ms 44.98 ms 102.09% 6.42 MiB

Benchmark Execution Parameters:

  • GRPC_BENCHMARK_DURATION=20s
  • GRPC_BENCHMARK_WARMUP=5s
  • GRPC_SERVER_CPUS=1
  • GRPC_SERVER_RAM=512m
  • GRPC_CLIENT_CONNECTIONS=50
  • GRPC_CLIENT_CONCURRENCY=1000
  • GRPC_CLIENT_QPS=0
  • GRPC_CLIENT_CPUS=10
  • GRPC_REQUEST_SCENARIO=string_100B
  • GRPC_USE_CLIENT_STREAMING=false
name req/s avg. latency 90 % in 95 % in 99 % in avg. cpu avg. memory
cpp_asio_grpc_callback 25427 39.16 ms 41.62 ms 42.42 ms 45.01 ms 102.45% 6.94 MiB
cpp_asio_grpc_coroutine 21566 46.17 ms 49.24 ms 50.01 ms 51.90 ms 103.17% 6.7 MiB

Benchmark Execution Parameters:

  • GRPC_BENCHMARK_DURATION=20s
  • GRPC_BENCHMARK_WARMUP=5s
  • GRPC_SERVER_CPUS=1
  • GRPC_SERVER_RAM=512m
  • GRPC_CLIENT_CONNECTIONS=50
  • GRPC_CLIENT_CONCURRENCY=1000
  • GRPC_CLIENT_QPS=0
  • GRPC_CLIENT_CPUS=10
  • GRPC_REQUEST_SCENARIO=string_100B
  • GRPC_USE_CLIENT_STREAMING=true
rogerwrld commented

Resolved