Minor performance degradation for v1.0 vs v0.6
Closed this issue · 4 comments
I ran js-reactivity-benchmark your latest v1.0.0 vs v0.6 to see if there was a performance change after your rewrite, & found that that v1.0 is about 1.8% slower.
Not that you (should) care, but as a FYI. I can share the spreadsheet if you want to dig to see which tests were the most different.
Averaged from 6 runs
13th Gen Intel(R) Core(TM) i7-13620H 2.40 GH
Win10
| alien-signals v | Bun v1.1.43 | Node v22.13.0 |
|---|---|---|
| 0.6 | 2041.18 | 2077.95 |
| 1.0 | 2078.73 | 2128.91 |
| diff | 1.84% | 2.45% |
Can you share the results of your execution of npm run bench in this repo between v1.0.0 and v0.6.0?
sure! Sorry I don't have Node v23 installed....
Technique; 2 repo copies in 2 different VS Code at same time, open side-by side, ran in Git-Bash.
Ran 3 times, copy/pasted what seemed to be the middle value.
Seems for your 1.0.0, the complex test never ran?
Note: he does use a wrapper for his tests, which I have a theory may explain the difference?
> alien-signals@1.0.0 build:esm
> esbuild src/index.ts --bundle --format=esm --outfile=esm/index.mjs
esm\index.mjs 17.9kb
Done in 2ms
Warning: disabling flag --expose_wasm due to conflicting flags
clk: ~0.06 GHz
cpu: 13th Gen Intel(R) Core(TM) i7-13620H
runtime: node 22.13.0 (x64-win32)
| benchmark | avg | min | p75 | p99 | max |
| -------------------- | ---------------- | ----------- | ----------- | ----------- | ----------- |
| propagate: 1 * 1 | `715.19 ns/iter` | `685.62 ns` | `725.54 ns` | `794.48 ns` | `881.98 ns` |
| propagate: 1 * 10 | ` 3.24 µs/iter` | ` 3.19 µs` | ` 3.26 µs` | ` 3.36 µs` | ` 3.37 µs` |
| propagate: 1 * 100 | ` 27.88 µs/iter` | ` 27.58 µs` | ` 28.00 µs` | ` 28.12 µs` | ` 28.14 µs` |
| propagate: 10 * 1 | ` 6.01 µs/iter` | ` 5.94 µs` | ` 6.04 µs` | ` 6.09 µs` | ` 6.13 µs` |
| propagate: 10 * 10 | ` 31.02 µs/iter` | ` 30.63 µs` | ` 31.09 µs` | ` 31.42 µs` | ` 31.48 µs` |
| propagate: 10 * 100 | `276.71 µs/iter` | `265.40 µs` | `274.40 µs` | `395.80 µs` | `567.30 µs` |
| propagate: 100 * 1 | ` 59.23 µs/iter` | ` 58.55 µs` | ` 59.52 µs` | ` 59.97 µs` | ` 60.15 µs` |
| propagate: 100 * 10 | `308.86 µs/iter` | `295.30 µs` | `307.10 µs` | `396.40 µs` | `533.30 µs` |
| propagate: 100 * 100 | ` 2.78 ms/iter` | ` 2.70 ms` | ` 2.80 ms` | ` 3.06 ms` | ` 3.13 ms` |
---
> alien-signals@0.6.0 build:esm
> esbuild src/index.ts --bundle --format=esm --outfile=esm/index.mjs
esm\index.mjs 19.7kb
Done in 3ms
Warning: disabling flag --expose_wasm due to conflicting flags
clk: ~0.06 GHz
cpu: 13th Gen Intel(R) Core(TM) i7-13620H
runtime: node 22.13.0 (x64-win32)
| benchmark | avg | min | p75 | p99 | max |
| -------------------- | ---------------- | ----------- | ----------- | ----------- | ----------- |
| propagate: 1 * 1 | `743.58 ns/iter` | `707.69 ns` | `742.70 ns` | ` 1.04 µs` | ` 1.12 µs` |
| propagate: 1 * 10 | ` 3.60 µs/iter` | ` 3.50 µs` | ` 3.62 µs` | ` 4.02 µs` | ` 4.02 µs` |
| propagate: 1 * 100 | ` 32.58 µs/iter` | ` 31.21 µs` | ` 33.42 µs` | ` 34.36 µs` | ` 34.88 µs` |
| propagate: 10 * 1 | ` 6.67 µs/iter` | ` 6.35 µs` | ` 6.70 µs` | ` 7.35 µs` | ` 7.39 µs` |
| propagate: 10 * 10 | ` 35.47 µs/iter` | ` 34.55 µs` | ` 35.78 µs` | ` 35.85 µs` | ` 36.79 µs` |
| propagate: 10 * 100 | `329.03 µs/iter` | `296.90 µs` | `311.70 µs` | `702.40 µs` | `765.60 µs` |
| propagate: 100 * 1 | ` 64.37 µs/iter` | ` 62.07 µs` | ` 64.86 µs` | ` 65.88 µs` | ` 66.27 µs` |
| propagate: 100 * 10 | `352.11 µs/iter` | `325.90 µs` | `342.20 µs` | `701.30 µs` | `846.10 µs` |
| propagate: 100 * 100 | ` 3.17 ms/iter` | ` 3.04 ms` | ` 3.12 ms` | ` 4.34 ms` | ` 5.05 ms` |
Warning: disabling flag --expose_wasm due to conflicting flags
clk: ~0.06 GHz
cpu: 13th Gen Intel(R) Core(TM) i7-13620H
runtime: node 22.13.0 (x64-win32)
| benchmark | avg | min | p75 | p99 | max |
| ------------------ | ---------------- | ----------- | ----------- | ----------- | ----------- |
| complex: 1 * 1 | ` 1.07 µs/iter` | `977.22 ns` | ` 1.04 µs` | ` 2.12 µs` | ` 2.31 µs` |
| complex: 1 * 10 | ` 6.47 µs/iter` | ` 6.19 µs` | ` 6.61 µs` | ` 7.03 µs` | ` 7.12 µs` |
| complex: 1 * 100 | ` 63.34 µs/iter` | ` 61.55 µs` | ` 64.42 µs` | ` 64.99 µs` | ` 65.73 µs` |
| complex: 10 * 1 | ` 9.63 µs/iter` | ` 9.11 µs` | ` 9.93 µs` | ` 10.03 µs` | ` 10.13 µs` |
| complex: 10 * 10 | ` 66.84 µs/iter` | ` 65.31 µs` | ` 67.43 µs` | ` 69.09 µs` | ` 69.24 µs` |
| complex: 10 * 100 | `647.83 µs/iter` | `581.60 µs` | `624.80 µs` | ` 1.24 ms` | ` 1.38 ms` |
| complex: 100 * 1 | ` 94.97 µs/iter` | ` 86.60 µs` | ` 91.70 µs` | `208.30 µs` | `424.40 µs` |
| complex: 100 * 10 | `674.76 µs/iter` | `614.70 µs` | `647.20 µs` | ` 1.18 ms` | ` 1.46 ms` |
| complex: 100 * 100 | ` 8.62 ms/iter` | ` 7.83 ms` | ` 8.54 ms` | ` 12.33 ms` | ` 12.51 ms` |Thanks for the information! I can't draw a conclusion easily, it needs more digging.
I usually modify js-reactivity-benchmark to run only a single test for alien-signals and enable advanced paint instrumentation in Chrome's performance panel to analyze the execution time of each line of code.
I currently have some other assignments that I might investigate in the future.
Seems for your 1.0.0, the complex test never ran?
In the current implementation, it is impossible for complex to produce contradictory results with propagate, so I changed it to only run propagate to save time.
We have improved the performance again in 1.0.7 and it should now be faster than 1.0.0/0.6!