benchmark fixtures used incorrectly
Closed this issue · 5 comments
The benchmarked library checks (here) if the first arg is an array. When it is, it provides the args array as the arguments. Essentially unwrapping your fixture.
It appears you knew that and were dealing with it in here, and here.
It seems that no longer works tho. Those functions don't get called and the target functions are called with extra, unintended, arguments.
I noticed this when the "if-depth" benchmark was way faster than the others. I had it show its arguments the first time it ran and it was receiving a 'b' as the second arg. When I looked at "fixtures/large.js" I found that was the second element in the exported (outermost) array.
If I alter your fixtures files to wrap their exported value an extra time then they are used correctly for the flatten suite, but not the arguments/depth suites.
After looking at the source code for benchmarked I haven't seen a way to provide a similar option which will run those "extra wrapping" functions you have.
So, what I've done, for now, is put the current fixtures into a sub-directory named "fixtures/arguments", similar to "code/arguments". Then, created corresponding files in "fixtures/flatten" and "fixtures/depth" which require the ones in "fixtures/arguments" and wraps them. The "flatten" ones are only wrapped into an array. The "depth" ones are wrapped and a second element of 4 provided; same as what you had in the benchmark main file.
Welcome to accepting a PR. I might have missed the changelog when I merged e45559d.
Edit: Or it could have always existed too!
I'd also be down to replace this with either raw code (e.g. https://github.com/blakeembrey/free-style/blob/master/benchmarks/index.ts) or use benchmark.js directly.
I'll supply a PR with the fix.
I also struggle with the whole do I use "custom testing", "benchmarkjs", "or something else". :)
Let me complete this task
Fixed as part of 8de93a1.