ohler55/oj_mimic_json

Decoding floats with trailing zeros results in wrong values

tsujigiri opened this issue · 3 comments

When requiring oj_mimic_json, decoding a float with more than 18 decimal places, including more than 4 trailing zeros, results in wrong values:

[19] pry(main)> Oj.load('{"foo":0.1000000000000000000}')
=> {"foo"=>-0.11838881245526249}
[20] pry(main)> Oj.load('{"foo":0.100000000000000000}')
=> {"foo"=>0.1}
[21] pry(main)> Oj.load('{"foo":0.110000000000000000}')
=> {"foo"=>0.11}
[22] pry(main)> Oj.load('{"foo":0.1100000000000000000}')
=> {"foo"=>-0.13022769370078874}
[23] pry(main)> Oj.load('{"foo":0.1000000000000000000}')
=> {"foo"=>-0.11838881245526249}
[24] pry(main)> Oj.load('{"foo":0.1100000000000000000}')
=> {"foo"=>-0.13022769370078874}
[25] pry(main)> Oj.load('{"foo":0.1110000000000000000}')
=> {"foo"=>-0.13141158182534138}
[26] pry(main)> Oj.load('{"foo":0.1111000000000000000}')
=> {"foo"=>-0.13152997063779662}
[27] pry(main)> Oj.load('{"foo":0.1111100000000000000}')
=> {"foo"=>-0.13154180951904215}
[28] pry(main)> Oj.load('{"foo":0.1111110000000000000}')
=> {"foo"=>-0.13154299340716671}
[29] pry(main)> Oj.load('{"foo":0.1111111000000000000}')
=> {"foo"=>-0.13154311179597916}
[30] pry(main)> Oj.load('{"foo":0.1111111100000000000}')
=> {"foo"=>-0.1315431236348604}
[31] pry(main)> Oj.load('{"foo":0.1111111110000000000}')
=> {"foo"=>-0.13154312481874852}
[32] pry(main)> Oj.load('{"foo":0.1111111111000000000}')
=> {"foo"=>-0.13154312493713735}
[33] pry(main)> Oj.load('{"foo":0.1111111111100000000}')
=> {"foo"=>-0.13154312494897621}
[34] pry(main)> Oj.load('{"foo":0.1111111111110000000}')
=> {"foo"=>-0.1315431249501601}
[35] pry(main)> Oj.load('{"foo":0.1111111111111000000}')
=> {"foo"=>-0.1315431249502785}
[36] pry(main)> Oj.load('{"foo":0.1111111111111100000}')
=> {"foo"=>-0.13154312495029036}
[37] pry(main)> Oj.load('{"foo":0.1111111111111110000}')
=> {"foo"=>#<BigDecimal:1f157b0,'0.1111111111 11111E0',18(36)>}
[38] pry(main)> Oj.load('{"foo":0.1111111111111110000}')['foo'].to_f
=> 0.111111111111111

This does not happen if only oj is being required.

I'll look into it. Strange.

I seemed to have missed this. Oj.mimic_JSON attempts to mimic the json gem. Past experimenting with the gem showed that the gem always uses floats and not big decimals so mimic_JSON puts Oj in the same mode by setting the Oj default values of :bigdecimal_as_decimal to false and :bigdecimal_load to :float. According to IEEE 754 precision on 64 bit floats can be expected to be between 15 and 17 decimal places. Expecting a Ruby float to be precise to 18 places is no a realistic expectation.

This is outdated and actually applies to Oj, not this gem. I believe it has been tested on Oj. If not please enter an issue there.