feat: add benchmarking against other implementations
Closed this issue · 0 comments
zmalatrax commented
Add a target which benchmarks the different VM implementations against some benchmark programs
An already existing (& powerful) tool to use is hyperfine.
Currently benchmarking:
- Rust Cairo VM
- TypeScript Cairo VM
- Python Cairo VM
The benchmark programs currently are the ones with no hint taken from the benchmarks/ folder of the cairo-vm, namely big_factorial.cairo, big_fibonacci.cairo, builtin_integration.cairo and pedersen.cairo.
Those programs might change in the future.
The values have been reduced by 1 or 2 order of magnitude for the typescript & python VM to follow up:
- 50000! (factorial)
- 40000-th element of the fibonacci suite
- 100 iterations with multiple operations on range_check, pedersen and bitwise builtins
- 5000 Pedersen hashes