Inconstist and Incorrect Comparisions

[cgbsu]

Hello, I am using your library for my project, I perform 16.16 + 16.16 I get what seems to be the correct result 32.32. When I compare it too 32.32 raw it says the result is less than 32.32. However 32.32 == 32.32 correctly

-------------------
VALUE?: 32.32
COMPARE TO: 32.32
EQUAL: 1
-------------------
left: 32.32
right: 32.32
equal: 0
greater? 0
less? 1

The way I am using this is somewhat complicated (in a compiler, fixed point type for my language), I have an overlaying type for it, but no matter how I write the comparison operators it does not seem to work.

[cgbsu]

As a note, calling to_uint() then comparing seems to work.

[eteran]

I'll take a look, but do keep in mind that as with floats, fixed point values often involve a lot of rounding, especially when the fraction has a small bit count.

So it may be a case where the numbers appear equal but are different but some small amount in the underlying representation

[eteran]

So i think the problem is precision vs expectations here. I'm guessing that maybe you're using a numeric::fixed<16, 16> type?

Here's the deal, the integer portion is easy, it's just 32, so it fits easily in 16 bits. That gets represented as b10'0000
The fractional part, is 0.32, which is not so easily represented!

The main thing to keep in mind is that the fractional part is represented basically, as a value * 32,768

Which in this case, is 10,485.76 That non-zero remainder is problematic... It means that the value not quite exactly 0.32 as we'd like.

It's able to print 32.32 because of heroics done by libc to print "what you probably want" when it comes to values that are almost the expected value".

I can say that if you use a (numeric::fixed<32, 32>) you get a LOT more precision and these comparisons have much more expected results.

[eteran]

To fully illustrate the issue, we can adjust the precision of cout like this:

	using fixed16 = numeric::fixed<16, 16>;
	fixed16 f1 = 16.16;
	fixed16 f2 = 16.16;
	fixed16 f3 = f1 + f2;
	std::cout << std::setprecision(12) << f3 << std::endl;
	std::cout << std::setprecision(12) << 32.32 << std::endl;

Which prints the results which are now much more clear:

32.3199768066
32.32

As you can see, the fixed number is indeed, ever so slightly less than 32.32, but it's the closest value we can get with 16 fractional bits.