Incorrectly Loading Binary Data?
cfis opened this issue · 8 comments
Assume you have the following array of bytes representing the 32 bit integer 2_147_483_647.
bytes = [255, 255, 255, 127]
Now:
binary_string = bytes.pack('C*')
value1 = binary_string .unpack('L')
=> [2147483647]
value2 = Numo::Int32.from_binary(binary_string ).to_a
=> [2147483647]
They look the same. But they are NOT - at least according to Ruby (version 2.6.5):
value1 == value2
=> false
value1.first == value2.first
=> false
This is on Ruby 2.6.5, Windows 64 bit.
ruby -v
=> ruby 2.6.5p114 (2019-10-01 revision 67812) [x64-mswin64_140]
I confirmed that it reproduces on Windows 10 + ruby 2.6.3
require 'numo/narray'
p Numo::Int32::MAX # 2147483647
A = 2_147_483_647
value3 = Numo::Int32[A].to_a
p value3 # [2147483647]
p value3.first # 2147483647
p value3.first == A # false
p value3.first + 1 # 1
p value3.first + 2 # 2
p value3.first - 1 # -1
p value3.first + 0.0 # 2147483647.0
p value3.first * 1.0 # 2147483647.0
100.times do |i|
j = (A * (i / 100.0)).floor
p [j, Numo::Int32[j].to_a.first != j]
end
# ↑ all false
# [1030792150, false]
# [1052266987, false]
# [1073741823, false]
# [1095216659, true]
# [1116691496, true]
# [1138166332, true]
# ↓ all true
(1073741823..1095216659).to_a.bsearch do |i|
Numo::Int32[i].to_a.first == i
end
I tried the bsearch
method to find out from which location the strange behavior start, but segmentation fault error occured.
I get the following results, but I don't know if this is the actual boundary.
p Numo::Int32[1073741823].to_a.first == 1073741823 # true
p Numo::Int32[1073741824].to_a.first == 1073741824 # false
Windows 10 + ruby 2.6.3
I wrote that I could reproduce it on Ubuntu, but I couldn't do it now.
It may be a misunderstanding.
I'm sorry, Tanaka-sensei.
So you can reroduce on windows but not Ubuntu? I haven't tried macos or ubuntu yet. I guess this would be a bug in narray though?
That's right. I can reproduce on Windows but not on Ubuntu.
I think this is a bug too, but it may take time for @masa16 to solve it.
Great. Thanks for looking into it. Let me know if you need help with testing a fix.
The commit (851a2f4) fixes a bug in conversion from C-int to Ruby-Integer, and solves this issue. Thank you for reporting.
Awesome, thanks!