First bench is always slower. Need warmup?
skratchdot opened this issue · 3 comments
The first bench
call always seems to be slower than other tests. I've included a simple test case below.
benchmark.js:
require('matcha');
[1, 2, 3, 4].forEach((num) => {
suite(`Benchmark #${num}`, () => {
bench('empty', () => {});
bench('empty', () => {});
});
});
command line:
npm install -g matcha
matcha ./benchmark.js
output:
Benchmark #1
414,441,918 op/s » empty
126,934,245 op/s » empty
Benchmark #2
130,458,368 op/s » empty
132,597,178 op/s » empty
Benchmark #3
132,507,794 op/s » empty
131,705,064 op/s » empty
Benchmark #4
135,100,324 op/s » empty
131,982,266 op/s » empty
Suites: 4
Benches: 8
Elapsed: 10,695.31 ms
I would expect all the benchmarks to execute around the same amount of time.
Thanks for this library btw!
This framework is no good. I suggest people avoid it.
https://www.reddit.com/r/node/comments/54rueh/using_let_significantly_slower_vs_var_v8_disables/
https://gist.github.com/Salakar/6d7b84f7adf1f3bc62a754752a6e5d0e
@EvanCarroll rather than bashing the project without any justification other than this bug (I followed your link, where you also don't give any other explanation), you could also help figure out what's causing this behavior.
As this issue (and other issues and PRs) have been open for years without update, I've created a newer rewrite of this tool. It uses the widely-used Benchmark.js, which includes warmup.