benchmark JVM varieties (graal, jdk9, ...)
retronym opened this issue · 5 comments
Our benchmarking infrastructure doesn't currently support benchmarking with alternative JVM without "polluting" the main data series. We need update it to add the contents of the system properties "java.vm.name" and "java.runtime.version" as tags that can be used for filtering in Grafana.
We also need to migrate historical data to populate the tags, as grafana makes it impossible (or at least really hard) to to the equivalent of getOrElse in the query to default absent tags.
First result looks great, better-files is 0.72x (!!!). scala appears on track for a similar speedup. Wow.
Cool! That's measuring the warm VM I assume? Can you easily run a cold bench? As far as I remember, Graal took more effort to warm up in the past than HotSpot.
Will do when I get a chance. It "felt" equal or better given the timings of the 1st/2nd iteration of hot benchmark, though.
IIRC the first iteration in our jmh benchmark already includes a number of executions, so there's quite some time for warmup already in there.