lukeed/kleur

Potential benchmark flaws

LJNeon opened this issue · 0 comments

I've noticed two potential flaws with the benckmarks used. I ran all benchmarks on the LTS version of node (v14.17.0) and used import instead of require. I also included the colorette package from #44.

  1. The load time of all libraries is essentially the same, but the library that's placed first will be significantly slower. An example where kleur was moved to the top is below.
# Load Times
  kleur         x 65μs
  kleur/colors  x 4μs
  chalk         x 4μs
  ansi-colors   x 4μs
  colorette     x 9μs

Also, it appears that console.time() has significant overhead. When I used process.hrtime.bigint() instead, I got much faster results:

# Load Times
  kleur         x 5.4μs
  kleur/colors  x 0.7μs
  chalk         x 0.9μs
  ansi-colors   x 0.8μs
  colorette     x 1μs
  1. When you import the entire module, both kleurs/colors and colorette perform much worse in the first test.
# All Colors
  ansi-colors   x 166,182 ops/sec ±0.96% (90 runs sampled)
  chalk         x 710,172 ops/sec ±2.34% (83 runs sampled)
  kleur         x 857,874 ops/sec ±2.88% (85 runs sampled)
  kleur/colors  x 251,574 ops/sec ±0.77% (85 runs sampled)
  colorette     x 234,007 ops/sec ±0.80% (89 runs sampled)

I believe this is due to the import * as colors from "lib"; syntax, but that's just a guess. I have no time nor expertise to look further into it.