StreamWriter crashes when writing more than a couple of bytes
u3shit opened this issue · 2 comments
u3shit commented
This works fine:
$ ruby -rxz -e 'XZ::StreamWriter.open("foo.xz") {|x| x.write "a" }'
But this crashes:
$ ruby -rxz -e 'XZ::StreamWriter.open("foo.xz") {|x| x.write "a"*10000 }'
corrupted size vs. prev_size
Aborted
So does this:
$ ruby -rxz -e 'XZ::StreamWriter.open("foo.xz") {|x| x.write "abcdefghijkl"*10000 }'
/home/u3/.gem/ruby/3.0.0/gems/ruby-xz-1.0.0/lib/xz/stream_writer.rb:237: [BUG] Segmentation fault at 0x0000000000000018
ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) [x86_64-linux]
-- Control frame information -----------------------------------------------
c:0006 p:0023 s:0029 e:000025 METHOD /home/u3/.gem/ruby/3.0.0/gems/ruby-xz-1.0.0/lib/xz/stream_writer.rb:237
c:0005 p:0008 s:0022 e:000021 METHOD /home/u3/.gem/ruby/3.0.0/gems/ruby-xz-1.0.0/lib/xz/stream.rb:225
c:0004 p:0010 s:0018 e:000017 RESCUE /home/u3/.gem/ruby/3.0.0/gems/ruby-xz-1.0.0/lib/xz/stream_writer.rb:131
c:0003 p:0047 s:0014 e:000013 METHOD /home/u3/.gem/ruby/3.0.0/gems/ruby-xz-1.0.0/lib/xz/stream_writer.rb:131
c:0002 p:0018 s:0006 e:000005 EVAL -e:1 [FINISH]
c:0001 p:0000 s:0003 E:001b80 (none) [FINISH]
-- Ruby level backtrace information ----------------------------------------
-e:1:in `<main>'
/home/u3/.gem/ruby/3.0.0/gems/ruby-xz-1.0.0/lib/xz/stream_writer.rb:131:in `open'
/home/u3/.gem/ruby/3.0.0/gems/ruby-xz-1.0.0/lib/xz/stream_writer.rb:131:in `ensure in open'
/home/u3/.gem/ruby/3.0.0/gems/ruby-xz-1.0.0/lib/xz/stream.rb:225:in `close'
/home/u3/.gem/ruby/3.0.0/gems/ruby-xz-1.0.0/lib/xz/stream_writer.rb:0:in `finish'
-- Machine register context ------------------------------------------------
RIP: 0x00007ff3632a03d0 RBP: 0x00007fffaad080b0 RSP: 0x00007fffaad07ff0
RAX: 0x000056462ff36780 RBX: 0x00007ff3621acec0 RCX: 0x0000000000000000
RDX: 0x000056462fc4a0b0 RDI: 0x00007ff3621acec0 RSI: 0x0000000000000000
R8: 0x000056462ff8fb08 R9: 0x0000564630083500 R10: 0x000056462fc439b8
R11: 0x00007ff3620ad120 R12: 0x000056462fbf47e0 R13: 0x000056462ff36790
R14: 0x000056462ff367a8 R15: 0x00007ff3621acec0 EFL: 0x0000000000010246
-- C level backtrace information -------------------------------------------
malloc(): smallbin double linked list corrupted
Aborted
In the above example malloc aborts because of a corrupted heap, but I've also seen SIGSEGVs. The error I get seems pretty random, if I rerun the above examples I get a different error (and sometimes they do work).
I'm using gentoo linux with the following ruby/liblzma versions:
$ ruby --version
ruby 3.0.2p107 (2021-07-07 revision 0db68f0233) [x86_64-linux]
$ qfile -v /usr/lib64/liblzma.so
app-arch/xz-utils-5.2.5-r1: /usr/lib64/liblzma.so
u3shit commented
Okay, this is the problem:
--- lib/xz/stream.rb~ 2021-11-06 14:47:49.272470250 +0100
+++ lib/xz/stream.rb 2021-11-06 16:12:47.648799304 +0100
@@ -118,7 +118,7 @@
pos = 0
until pos > str.bytesize # Do not use >=, that conflicts with #lzma_finish
substr = str[pos, XZ::CHUNK_SIZE]
- @input_buffer_p[0, str.bytesize] = substr
+ @input_buffer_p[0, substr.bytesize] = substr
pos += XZ::CHUNK_SIZE
@lzma_stream.next_in = @input_buffer_pEven though substr is only XZ::CHUNK_SIZE long, that assignment still copies str.bytesize number of bytes from substr:
#0 __memmove_avx_unaligned_erms () at ../sysdeps/x86_64/multiarch/memmove-vec-unaligned-erms.S:269
#1 0x00007f046994ef46 in memcpy (__len=120000, __src=<optimized out>, __dest=<optimized out>) at /usr/include/bits/string_fortified.h:29
#2 ruby_nonempty_memcpy (n=120000, src=<optimized out>, dest=<optimized out>) at ../.././include/ruby/internal/memory.h:269
#3 rb_fiddle_ptr_aset (argc=3, argv=<optimized out>, self=<optimized out>) at pointer.c:719
Quintus commented
Thanks for the report, but: I am abandoning this project now. I don’t use it myself anymore and simply don’t have time to further maintain it.