bartaz/ieee754-visualization

You could show that the actual value that is represented by the fp expansion is different

yurivict opened this issue · 0 comments

fp expansion that you give for 0.97 actually corresponds to 0.96999999999999997335464740899624302983...

Pari calculator computes the given series to the above number:
2^-1 + 2^-2 + 2^-3 + 2^-4 + 2^-5 + 2^-10 + 2^-12 + 2^-16 + 2^-17 + 2^-18 + 2^-19 + 2^-21 + 2^-23 + 2^-24 + 2^-25 + 2^-30 + 2^-32 + 2^-36 + 2^-37 + 2^-38 + 2^-39 + 2^-41 + 2^-43 + 2^-44 + 2^-45 + 2^-50 + 2^-52

So, there is rounding involved that you could show to the user, because the purpose of the program is supposedly educational, and this could make it more precise and educational. -)