typelevel/squants

Converting Micrograms to Milligrams for 5x values create non-rounded results

royashcenazi opened this issue · 2 comments

an example worth a 1000 words
Micrograms(30).toMilligrams
results in
0.029999999999999995 instead of 0.03

This does not happen for numbers that are not divided without a remain by 5.

It seems to occur when multiplying the value in the conversion factor Micro = 1e-6

Yes, this is an anomaly with using Double as the underlying type.  It's compounded when you go from one non-Primary unit to another as both conversion factors need to be combined and then applied.

There is solid work in progress on making the underlying value type generic, allowing you to use BigDecimal, which reduces that problem significantly.

Is there a branch or separate release to check out and help on?