Sudden hiccups during benchmarking
unsuitable001 opened this issue · 1 comments
unsuitable001 commented
While doing benchmarking, we've observed sudden and sorta random dips in performance.
[0712/103533.128980:ERROR:cert_verify_proc_builtin.cc(559)] No net_fetcher for performing AIA chasing.
Cronet(Throughput): Total Spawned: 8, Returned in time: 8.
[0712/103533.583837:ERROR:cert_verify_proc_builtin.cc(559)] No net_fetcher for performing AIA chasing.
Cronet(Throughput): Total Spawned: 16, Returned in time: 16.
[0712/103534.120373:ERROR:cert_verify_proc_builtin.cc(559)] No net_fetcher for performing AIA chasing.
Cronet(Throughput): Total Spawned: 32, Returned in time: 30.
[0712/103548.869642:ERROR:cert_verify_proc_builtin.cc(559)] No net_fetcher for performing AIA chasing.
Cronet(Throughput): Total Spawned: 64, Returned in time: 64.
[0712/103549.496886:ERROR:cert_verify_proc_builtin.cc(559)] No net_fetcher for performing AIA chasing.
Cronet(Throughput): Total Spawned: 128, Returned in time: 121.
[0712/103557.274973:ERROR:cert_verify_proc_builtin.cc(559)] No net_fetcher for performing AIA chasing.
Cronet(Throughput): Total Spawned: 256, Returned in time: 256.
See dips in 32 and 128 marks. Errors are due to #10 . This issue may affect reports in #3 .
dcharkes commented
It's rather common that the benchmarks results are a statistical distribution in nature.
We might want to add a repeat command line argument and report the min/max/median when repeat is more than 1.