fmtlib/format-benchmark

integer overflow in int-benchmark

travisdowns opened this issue · 4 comments

In int-benchmark the following code is used for generating the random distribution:

    std::generate(values.begin(), values.end(), []() {
      int scale = std::rand() / 100 + 1;
      return (std::rand() * std::rand()) / scale;
    });

std::rand() * std::rand() may or may not suffer from integer overflow depending on RAND_MAX. On Linux where RAND_MAX is usually 2^31-1, overflow almost always occurs (> 99.99% of the time) and this results in very different results (50% of values are negative) to windows where RAND_MAX is 2^15 (all positive values and smaller values in general). Not sure about OSX.

I guess this is not intentional. Ideally, once would use the same RNG on every platform. I don't know if the 50% negative values is a "feature" or not.

This code originates from karma generate benchmark (https://www.boost.org/doc/libs/1_52_0/libs/spirit/optimization/karma/int_generator.cpp). I think 50% negative is a feature and a PR to make the benchmark more consistent across platforms would be welcome.

Fixed in 7ccec77. Thanks for reporting!

Thanks @vitaut and sorry on the delay.

I kind of got blocked when thinking about a PR because it seemed like making it deterministic across platforms would also be a good goal and <random> is kind of awkward since IIRC it has no good cross-plat RNGs, so I was like bleh.

deterministic across platforms would also be a good goal

That would be good and PR is still welcome =). However, the current version shouldn't be too bad - the inputs are not identical but at least distribution is similar.