Run benchmark tests on node v15
ColinEberhardt opened this issue · 3 comments
ColinEberhardt commented
Currently the CI tests (that run npm test
) use node v15. However, the benchmarks use the default node version, which happens to be v10.
When running the benchmarks on node v15 they fail as follows:
$ npm run benchmark
> assemblyscript-regex@0.1.0 benchmark
> node benchmark/benchmark.js
baseline x 134,050 ops/sec ±0.79% (88 runs sampled)
character class x 43,542 ops/sec ±2.28% (87 runs sampled)
concatenation x 9,974 ops/sec ±1.11% (86 runs sampled)
/Users/colineberhardt/Projects/as-regex/node_modules/@assemblyscript/loader/umd/index.js:78
throw Error(`abort: ${getString(memory, msg)} at ${getString(memory, file)}:${line}:${colm}`);
^
Error: abort: at ~lib/rt/tlsf.ts:238:14
at abort (/Users/colineberhardt/Projects/as-regex/node_modules/@assemblyscript/loader/umd/index.js:78:13)
at ~lib/rt/tlsf/insertBlock (<anonymous>:wasm-function[2]:0x69f)
at ~lib/rt/tlsf/addMemory (<anonymous>:wasm-function[3]:0x8e7)
at ~lib/rt/tlsf/growMemory (<anonymous>:wasm-function[8]:0xc0c)
at ~lib/rt/tlsf/allocateBlock (<anonymous>:wasm-function[10]:0xce1)
at ~lib/rt/tlsf/__alloc (<anonymous>:wasm-function[11]:0xd46)
at ~lib/rt/pure/__new (<anonymous>:wasm-function[12]:0xd72)
at ~lib/rt/__newArray (<anonymous>:wasm-function[33]:0x1d7f)
at assembly/regexp/recursiveBacktrackingSearch (<anonymous>:wasm-function[181]:0x5025)
at assembly/regexp/recursiveBacktrackingSearch (<anonymous>:wasm-function[181]:0x507a)
Ideally everything should run on the same node version!
MaxGraey commented
I tried 0.18 version from this PR on node v15 and everything works fine:
untouched.wasm:
baseline x 103,084 ops/sec ±1.14% (88 runs sampled)
character class x 28,805 ops/sec ±0.52% (89 runs sampled)
concatenation x 8,267 ops/sec ±0.61% (91 runs sampled)
quantifiers x 15,565 ops/sec ±0.96% (86 runs sampled)
range quantifiers x 5,022 ops/sec ±0.56% (89 runs sampled)
alternation x 16,263 ops/sec ±0.63% (88 runs sampled)
multiple regex matches x 1,723 ops/sec ±0.44% (86 runs sampled)
optimized.wasm:
baseline x 205,100 ops/sec ±0.66% (89 runs sampled)
character class x 58,661 ops/sec ±0.76% (88 runs sampled)
concatenation x 15,636 ops/sec ±0.37% (89 runs sampled)
quantifiers x 30,233 ops/sec ±0.51% (88 runs sampled)
range quantifiers x 9,227 ops/sec ±0.39% (84 runs sampled)
alternation x 29,869 ops/sec ±0.70% (88 runs sampled)
multiple regex matches x 2,309 ops/sec ±1.03% (86 runs sampled)
torch2424 commented
Ah I was just about to try a benchmark, thanks for checking this @MaxGraey ! 😄 🎉
ColinEberhardt commented
Fixed 12c8e55