jonahsnider/convert

problem with rounding by all measures

Oleg55 opened this issue · 1 comments

Hello, a problem with rounding by all measures was found.
for example, if we take 10 square meters and translate them into square kilometers, we get 0.000009999999999999999, while if we take 11 we get the correct answer 0.000011, and so on for all metrics

This slight loss in precision is because of how JavaScript defines floating point arithmetic (according to the IEEE 754 spec). While it is possible to avoid the loss in precision for operations like '10 square meters to square kilometers', it would require significant refactors to how Convert works internally. So, it doesn't really make sense for me to try changing this behavior.

I would recommend using something like Number.prototype.toFixed() to round the resulting number to a given level of precision:

const result = convert(10, 'sq m').to('sq km');
const rounded = Number(result.toFixed(10)); // 10 digits of accuracy after the decimal point
console.log(rounded);