Significant differences in ea_non_iid selftest [Windows10]

[skbhaskarla]

FYI: I ran all the ea_non_iid output files through the compare perl script and wanted to let you know I noticed "Significant difference" for two files:

1. perl compareresults.pl result\biased-random-bytes.res refdata\biased-random-bytes.res
result\biased-random-bytes.res: Significant difference for Literal Lag Prediction Estimate: min entropy (reference: 0.46625826502784795 new: 0.4662582648795216 delta: 1.48326351201433e-10)
result\biased-random-bytes.res: Significant difference for Bitstring MultiMCW Prediction Estimate: P_local (reference: 0.9713579012997926 new: 0.97135790142486211 delta: 1.25069510303888e-10)
result\biased-random-bytes.res: Significant difference for Bitstring MultiMMC Prediction Estimate: P_local (reference: 0.9713578878098017 new: 0.97135788791663613 delta: 1.06834430191327e-10)
result\biased-random-bytes.res: Significant difference for Bitstring MultiMMC Prediction Estimate: min entropy (reference: 0.041925153682126653 new: 0.041925153523452385 delta: 1.58674268169179e-10)
result\biased-random-bytes.res: Significant difference for Bitstring MultiMCW Prediction Estimate: min entropy (reference: 0.041925133646315872 new: 0.04192513346055822 delta: 1.85757652315655e-10)
result\biased-random-bytes.res: Significant difference for Bitstring Lag Prediction Estimate: min entropy (reference: 0.042001643639251546 new: 0.042001643777700305 delta: 1.38448759401388e-10)
result\biased-random-bytes.res: Significant difference for Bitstring LZ78Y Prediction Estimate: min entropy (reference: 0.04192514875535195 new: 0.041925148882084547 delta: 1.26732596639201e-10)
Maximum delta: 1.85757652315655e-10

2. perl compareresults.pl result\ringOsc-nist.res refdata\ringOsc-nist.res
result\ringOsc-nist.res: Significant difference for Literal MultiMCW Prediction Estimate: min entropy (reference: 0.29051922736594488 new: 0.29051922704532573 delta: 3.20619142346601e-10)
result\ringOsc-nist.res: Significant difference for Literal MultiMCW Prediction Estimate: P_local (reference: 0.81760774776600598 new: 0.81760774794770807 delta: 1.8170209781232e-10)
Maximum delta: 3.20619142346601e-10

[celic]

Deltas are all around 1e-10 which isn't that bad. They must be marginally below the threshold the perl script is looking for.

[joshuaehill]

A few points:

1. These delta values are fairly small, and there isn't an obviously correct choice of delta. The current value (1E-10) was chosen as it was lower than the deltas that I encountered when recompiling the codebase using different compilers on the same platform. It may be that the math routines in the new platform experience a bit more variation than I have previous seen, and that the correct response is to change epsilon to a larger value.
2. I'd be uncomfortable with suggesting that folks use this branch / platform until someone roots around the code and find any places where "long ints" were used for protection from overflows in calculations. In this platform "long int" means "int32_t", but in the development environment it meant "int64_t". There surely exist such instances, and in these instances, the "long int" type should be changed to uint64_t and int64_t, depending on the context. Similarly, I'm curious as to the precision of double vs long double on the platform (this can be determined by just printing DBL_EPSILON and LDBL_EPSILON). If some sort of extended precision type isn't available, it may not advisable to perform calculations on this platform at all.

[celic]

Of course. We do not support Windows at the moment. That should be made clear. I know some instances of long int were changed to be int64_t. The code needs to be tested more before we say we support Windows. I think only non-iid was converted as well (not sure on that).