Floating Point Binary Math Errors
Closed this issue · 1 comments
DallasHoff commented
A mitigation is needed for floating point binary math errors causing long trailing decimal amounts in the results for operations involving decimals.
For example, 0.1 + 0.2 currently gives the result: 0.30000000000000004
It should give the result: 0.3
DallasHoff commented
Investigate adding exact-math to enable more precise math on the calculator overall.