yardstiq/quantum-benchmarks

Flat scaling of qiskit

Closed this issue · 1 comments

Roger-luo commented

@atilag the single gate benchmark is almost flat for qiskit, but for other frameworks it seems normal (at least is scaling with the system size). Could you help me review if there is anything wrong in the benchmark or this is the correct? I'm not sure if the script is correct since the scaling is kinda strange. This benchmark is running with the master branch with the commit SHA given in README.

Roger-luo commented

Hi, @atilag

Just an update , I found this is because we were not benchmarking the actual execution time instead of something else, after I use execute (although this will include the transpiler time, but it is basically a constant time), the scaling looks correct.

Since benchmarks for other softwares uses their user interface, I deleted the original benchmark script to be more fair (which hacks into execute).

Please let me know if I miss anything in the benchmark script.