Heap-use-after free in Json::iterator::reinterpret_label()
futile opened this issue · 7 comments
Running ./jtc -w '<>F3' Bookmarks reports a heap-use-after-free error when compiled with asan.
To reproduce:
$ clang++ --std=c++14 -Og -g -pthread -lpthread -lrt -fsanitize=address,undefined -o jtc jtc.cpp
$ ./jtc -w '<>F3' Bookmarks
Which results in:
=================================================================
==3636414==ERROR: AddressSanitizer: heap-use-after-free on address 0x613000000a40 at pc 0x558c2551455f bp 0x7fffdba409c0 sp 0x7fffdba409b8
READ of size 4 at 0x613000000a40 thread T0
#0 0x558c2551455e in Json::iterator::reinterpret_label() const ./jtc/./lib/Json.hpp:2568:50
#1 0x558c2548bb12 in Json::iterator::operator*() ./jtc/./lib/Json.hpp:3878:9
#2 0x558c254dfba0 in Jtc::console_output_(Json::iterator&, Json&, Jtc::Grouping) ./jtc/jtc.cpp:2770:47
#3 0x558c254d46f0 in Jtc::output_by_iterator(Json::iterator&, Jtc::Grouping) ./jtc/jtc.cpp:1970:3
#4 0x558c254e9774 in Jtc::process_offsets_(std::deque<std::deque<Json::iterator, std::allocator<Json::iterator> >, std::allocator<std::deque<Json::iterator, std::allocator<Json::iterator> > > >&, std::vector<std::vector<long, std::all
ocator<long> >, std::allocator<std::vector<long, std::allocator<long> > > >&, unsigned long, std::vector<unsigned long, std::allocator<unsigned long> >&) ./jtc/jtc.cpp:2749:2
#5 0x558c254e8033 in Jtc::process_walk_iterators_(std::deque<std::deque<Json::iterator, std::allocator<Json::iterator> >, std::allocator<std::deque<Json::iterator, std::allocator<Json::iterator> > > >&) ./jtc/jtc.cpp:2
673:2
#6 0x558c254cdd00 in Jtc::walk_interleaved_(void (Jtc::*)(Json::iterator&, Jtc::Grouping)) ./jtc/jtc.cpp:2657:3
#7 0x558c254c8d1b in Jtc::walk_json() ./jtc/jtc.cpp:1896:2
#8 0x558c254bf644 in Jtc::demux_opt() ./jtc/jtc.cpp:1517:25
#9 0x558c254bd625 in run_single_optset(CommonResource&, Streamstr::const_iterator&, Json&, Json&) ./jtc/jtc.cpp:903:29
#10 0x558c254bae77 in run_decomposed_optsets(CommonResource&, Streamstr::const_iterator&) ./jtc/jtc.cpp:877:3
#11 0x558c254b5943 in main ./jtc/jtc.cpp:803:6
#12 0x7f6520d61022 in __libc_start_main (/usr/lib/libc.so.6+0x27022)
#13 0x558c25386ced in _start (./jtc/jtc+0x2ecced)
0x613000000a40 is located 320 bytes inside of 384-byte region [0x613000000900,0x613000000a80)
freed by thread T0 here:
#0 0x558c2545fcd9 in operator delete(void*) (./jtc/jtc+0x3c5cd9)
#1 0x558c256c5430 in __gnu_cxx::new_allocator<Json::iterator>::deallocate(Json::iterator*, unsigned long) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/9.3.0/../../../../include/c++/9.3.0/ext/new_allocator.h:128:2
#2 0x558c254cc7db in Jtc::walk_interleaved_(void (Jtc::*)(Json::iterator&, Jtc::Grouping)) ./jtc/jtc.cpp:2622:3
#3 0x558c254c8d1b in Jtc::walk_json() ./jtc/jtc.cpp:1896:2
#4 0x558c254bf644 in Jtc::demux_opt() ./jtc/jtc.cpp:1517:25
#5 0x558c254bd625 in run_single_optset(CommonResource&, Streamstr::const_iterator&, Json&, Json&) ./jtc/jtc.cpp:903:29
#6 0x558c254bae77 in run_decomposed_optsets(CommonResource&, Streamstr::const_iterator&) ./jtc/jtc.cpp:877:3
#7 0x558c254b5943 in main ./jtc/jtc.cpp:803:6
#8 0x7f6520d61022 in __libc_start_main (/usr/lib/libc.so.6+0x27022)
previously allocated by thread T0 here:
#0 0x558c2545f2b9 in operator new(unsigned long) (./jtc/jtc+0x3c52b9)
#1 0x558c256d0358 in __gnu_cxx::new_allocator<Json::iterator>::allocate(unsigned long, void const*) /usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/9.3.0/../../../../include/c++/9.3.0/ext/new_allocator.h:114:27
#2 0x558c256d77ab in std::_Deque_base<Json::iterator, std::allocator<Json::iterator> >::_Deque_base(std::_Deque_base<Json::iterator, std::allocator<Json::iterator> >&&, std::integral_constant<bool, true>) /usr/bin/../lib64/gcc/x86_64-
pc-linux-gnu/9.3.0/../../../../include/c++/9.3.0/bits/stl_deque.h:532:2
SUMMARY: AddressSanitizer: heap-use-after-free ./jtc/./lib/Json.hpp:2568:50 in Json::iterator::reinterpret_label() const
Shadow bytes around the buggy address:
0x0c267fff80f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c267fff8100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c267fff8110: 00 00 00 00 00 00 00 00 fa fa fa fa fa fa fa fa
0x0c267fff8120: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c267fff8130: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
=>0x0c267fff8140: fd fd fd fd fd fd fd fd[fd]fd fd fd fd fd fd fd
0x0c267fff8150: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
0x0c267fff8160: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c267fff8170: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c267fff8180: fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa fa
0x0c267fff8190: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
==3636414==ABORTING
This bug was found using Symbolic Execution techniques developed in the course of the SYMBIOSYS research project at COMSYS, RWTH Aachen University. This research is supported by the European Research Council (ERC) under the EU's Horizon 2020 Research and Innovation Programme grant agreement n. 647295 (SYMBIOSYS).
hmmm, that's interesting one. jtc does not use bare memory pointers as resource holders and every resource allocator is implemented either via guards or through STL.
also, the same crash does not occur once compiled using gcc, or default line.
it'll take me a while looking into this issue.
I expected the bug to be more straight-forward, I'll also take another look. A few days ago I think it wasn't possible to compile with g++, which is why I didn't try this time.
By 'default line' you mean c++ -o jtc -Wall -std=gnu++14 -Ofast -pthread -lpthread jtc.cpp? (with sanitizers enabled I guess)
When I compile with:
$ g++ -o jtc-gcc-sanitized -Wall -std=gnu++14 -Ofast -pthread -lpthread -fsanitize=address,undefined jtc.cpp
I get the same bug (or a very similar one).
As a deque and an iterator seem to be involved, could this be a case of iterator invalidation (e.g., an operation on a deque causes the deque to be reallocated internally, which invalidates existing iterators, one of which is then used somewhere)?
yes, it's my bad, it does produce the same failure (I used a wrong copy of jtc when ran), so, indeed a valid point, I confirm it. I'll take a look, thanks
Thanks! :)
Okay, I see where the bug is, will be fixing soon (probably would need to redesign a code a bit, will see).
HI Felix, I have published the fix for this issue, turned out a trivial one. Thanks again for catching and reporting it.