The input record separator on method IO.each is changed when nmatrix library in required
seoanezonjic opened this issue · 9 comments
On ruby version ruby 2.2.2p95 (2015-04-13 revision 50295) [x86_64-linux] when nmatrix is required, the following code:
` File.open(file).each do |line|
line.chomp!
puts line + '-'
fields = line.split("\t")
puts fields.inspect
records << fields.map(&:to_f)
end
Generates this output:
c1 c2 c3 c4 c5 c6 c7 c8 c9 c10
b1 1 1 1 1 1 1 1 1 1 0
b2 1 1 1 1 1 1 1 1 1 0
b3 0 0 0 0 0 0 0 0 0 0
b4 0 0 0 0 0 0 0 0 0 0
b5 0 0 0 0 0 0 0 0 0 0
b6 0 0 0 0 0 0 0 0 0 0
b7 0 0 0 0 0 0 0 0 0 0
b8 0 0 0 0 0 0 0 0 0 0
b9 0 0 0 0 0 0 0 0 0 0
b10 0 0 0 0 0 0 0 0 0 0-
["c1", "c2", "c3", "c4", "c5", "c6", "c7", "c8", "c9", "c10\nb1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "0\nb2", "1", "1", "1", "1", "1", "1", "1", "1", "1", "0\nb3", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0\nb4", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0\nb5", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0\nb6", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0\nb7", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0\nb8", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0\nb9", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0\nb10", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0"]`
If I change the each line to:
File.open(file).each(sep=$/) do |line|
The each method has its normal behaviour and I get this ouput:
c1 c2 c3 c4 c5 c6 c7 c8 c9 c10-
["c1", "c2", "c3", "c4", "c5", "c6", "c7", "c8", "c9", "c10"]
b1 1 1 1 1 1 1 1 1 1 0-
["b1", "1", "1", "1", "1", "1", "1", "1", "1", "1", "0"]
b2 1 1 1 1 1 1 1 1 1 0-
["b2", "1", "1", "1", "1", "1", "1", "1", "1", "1", "0"]
b3 0 0 0 0 0 0 0 0 0 0-
["b3", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0"]
b4 0 0 0 0 0 0 0 0 0 0-
["b4", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0"]
b5 0 0 0 0 0 0 0 0 0 0-
["b5", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0"]
b6 0 0 0 0 0 0 0 0 0 0-
["b6", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0"]
b7 0 0 0 0 0 0 0 0 0 0-
["b7", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0"]
b8 0 0 0 0 0 0 0 0 0 0-
["b8", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0"]
b9 0 0 0 0 0 0 0 0 0 0-
["b9", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0"]
b10 0 0 0 0 0 0 0 0 0 0-
["b10", "0", "0", "0", "0", "0", "0", "0", "0", "0", "0"]
[[1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0], [1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 1.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0], [0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]]
Why nmatrix changes the IO.each method?
Thanks in advance
Honestly, I'm not sure. I don't see where it is changing IO.each, but it's been a long time since I messed with any of the IO stuff. I'd suggest starting by looking in lib/nmatrix/io, probably.
@MohawkJohn I was hit by this and investigated a little. It looks like this is result of using Packable gem (https://github.com/marcandre/packable/blob/master/lib/packable/extensions/io.rb). That gem is doing some extensive monkey patching and overwrites number of core methods. Is it possible to strip that gem?
Unfortunately, packable is pretty necessary for the MATLAB-format matrix readers and writers. If someone wanted to rewrite those, it'd be fine to shed packable.
@moloh Please open an issue here: https://github.com/sciruby/packable/issues
Close this? Doesn't seem related to nmatrix.
@v0dro I think the idea is that we probably need to come up with a solution in NMatrix that doesn't require packable. Or perhaps that we should make Matlab IO into a plugin.
I like the plugin idea. Its not a core functionality anyway.
Is there a fix for this already? Can't run my neural network implementation using nmatrix's latest version due to the packable gem. :(
No. We're all volunteers. I'd encourage you to submit whatever workaround you come up with.
Hi all
I'm using nmatrix again and this problem seems that is gone. When I have revised the packable gem repo, there are three commits in Sep 20, 2018 that seems to fix this issue. I did some tests with the initial data and I cannot reproduce the issue.
Maybe it would be nice to close this issue.
Thank by you help