ColinEberhardt/assemblyscript-regex

Run benchmark tests on node v15

ColinEberhardt opened this issue · 3 comments

Currently the CI tests (that run npm test) use node v15. However, the benchmarks use the default node version, which happens to be v10.

When running the benchmarks on node v15 they fail as follows:

$ npm run benchmark

> assemblyscript-regex@0.1.0 benchmark
> node benchmark/benchmark.js

baseline x 134,050 ops/sec ±0.79% (88 runs sampled)
character class x 43,542 ops/sec ±2.28% (87 runs sampled)
concatenation x 9,974 ops/sec ±1.11% (86 runs sampled)
/Users/colineberhardt/Projects/as-regex/node_modules/@assemblyscript/loader/umd/index.js:78
      throw Error(`abort: ${getString(memory, msg)} at ${getString(memory, file)}:${line}:${colm}`);
      ^

Error: abort:  at ~lib/rt/tlsf.ts:238:14
    at abort (/Users/colineberhardt/Projects/as-regex/node_modules/@assemblyscript/loader/umd/index.js:78:13)
    at ~lib/rt/tlsf/insertBlock (<anonymous>:wasm-function[2]:0x69f)
    at ~lib/rt/tlsf/addMemory (<anonymous>:wasm-function[3]:0x8e7)
    at ~lib/rt/tlsf/growMemory (<anonymous>:wasm-function[8]:0xc0c)
    at ~lib/rt/tlsf/allocateBlock (<anonymous>:wasm-function[10]:0xce1)
    at ~lib/rt/tlsf/__alloc (<anonymous>:wasm-function[11]:0xd46)
    at ~lib/rt/pure/__new (<anonymous>:wasm-function[12]:0xd72)
    at ~lib/rt/__newArray (<anonymous>:wasm-function[33]:0x1d7f)
    at assembly/regexp/recursiveBacktrackingSearch (<anonymous>:wasm-function[181]:0x5025)
    at assembly/regexp/recursiveBacktrackingSearch (<anonymous>:wasm-function[181]:0x507a)

Ideally everything should run on the same node version!

I tried 0.18 version from this PR on node v15 and everything works fine:

untouched.wasm:

baseline x 103,084 ops/sec ±1.14% (88 runs sampled)
character class x 28,805 ops/sec ±0.52% (89 runs sampled)
concatenation x 8,267 ops/sec ±0.61% (91 runs sampled)
quantifiers x 15,565 ops/sec ±0.96% (86 runs sampled)
range quantifiers x 5,022 ops/sec ±0.56% (89 runs sampled)
alternation x 16,263 ops/sec ±0.63% (88 runs sampled)
multiple regex matches x 1,723 ops/sec ±0.44% (86 runs sampled)

optimized.wasm:

baseline x 205,100 ops/sec ±0.66% (89 runs sampled)
character class x 58,661 ops/sec ±0.76% (88 runs sampled)
concatenation x 15,636 ops/sec ±0.37% (89 runs sampled)
quantifiers x 30,233 ops/sec ±0.51% (88 runs sampled)
range quantifiers x 9,227 ops/sec ±0.39% (84 runs sampled)
alternation x 29,869 ops/sec ±0.70% (88 runs sampled)
multiple regex matches x 2,309 ops/sec ±1.03% (86 runs sampled)

Ah I was just about to try a benchmark, thanks for checking this @MaxGraey ! 😄 🎉