google/cronet.dart

Sudden hiccups during benchmarking

unsuitable001 opened this issue · 1 comments

While doing benchmarking, we've observed sudden and sorta random dips in performance.

[0712/103533.128980:ERROR:cert_verify_proc_builtin.cc(559)] No net_fetcher for performing AIA chasing.
Cronet(Throughput): Total Spawned: 8, Returned in time: 8.
[0712/103533.583837:ERROR:cert_verify_proc_builtin.cc(559)] No net_fetcher for performing AIA chasing.
Cronet(Throughput): Total Spawned: 16, Returned in time: 16.
[0712/103534.120373:ERROR:cert_verify_proc_builtin.cc(559)] No net_fetcher for performing AIA chasing.
Cronet(Throughput): Total Spawned: 32, Returned in time: 30.
[0712/103548.869642:ERROR:cert_verify_proc_builtin.cc(559)] No net_fetcher for performing AIA chasing.
Cronet(Throughput): Total Spawned: 64, Returned in time: 64.
[0712/103549.496886:ERROR:cert_verify_proc_builtin.cc(559)] No net_fetcher for performing AIA chasing.
Cronet(Throughput): Total Spawned: 128, Returned in time: 121.
[0712/103557.274973:ERROR:cert_verify_proc_builtin.cc(559)] No net_fetcher for performing AIA chasing.
Cronet(Throughput): Total Spawned: 256, Returned in time: 256.

See dips in 32 and 128 marks. Errors are due to #10 . This issue may affect reports in #3 .

It's rather common that the benchmarks results are a statistical distribution in nature.

We might want to add a repeat command line argument and report the min/max/median when repeat is more than 1.