rubysolo/dentaku

0.089*105.0 should equal 9.344999999999999, not 9.345

rhelsing opened this issue · 1 comments

rhelsing commented
0.089*105.0 should equal 9.344999999999999, not 9.345
rubysolo commented

This is the rounding error behavior for Ruby Floats, but using BigDecimal is more accurate:

> require 'bigdecimal'
> BigDecimal("0.089") * 105
=> 0.9345e1