Detailed Benchmark Comparison (not just Cowboy, but Go/Java/etc)
michealp-coder opened this issue · 4 comments
(I'm not sure if this is the appropriate place to ask this question, so please direct me to the appropriate channel if not)
Mat, amazing work on Bandit.
Question: would you be willing to do a detailed benchmark comparison of current Bandit/Elixir vs not just Cowboy, but also Go, Java, etc?
Something like the detailed benchmark that StressGrid did a few years ago.
--
I'd do it myself, but admittedly I wouldn't know how to do it properly.
I would though be willing to donate funding to help run the test (and documentation).
E.g. OVH has brand new dedicated AMD EPYC 9654 (96-core/192-thread) at amazingly low price points. I'm willing to donate money toward the time and/or hardware needed for such a test to be performed.
There has been some movement with third party benchmarks in the past: #69. Unsure how timely that remains.
We also have an open PR to improve the visibility of our micro benchmarks: #300
In terms of larger scale benchmarks, I have to say I don't really have the time to dedicate to running any myself. We use the micro benchmarks in CI quite heavily, but mostly to compare ourselves to ourselves for the purposes of avoiding performance regressions. It's certainly interesting to see how we measure up against other web servers, but it's not really actionable for day-to-day Bandit development.
I'll leave this issue open for a bit to see if it picks up anyone interested in moving this forward!
Hi @michealp-coder, I'm potentially interested in picking this up.
I built https://github.com/getsentry/sentry-sdk-benchmark, which may still be viable as a starting point.
Would love to meet you and chat about what you have in mind with the benchmarks. Let's book a call?
Hi @rhcarvalho , I just emailed you.
Closing for hygiene