The breakpad reader doesn't split merged inlinees
mstange opened this issue · 0 comments
mstange commented
The breakpad reader / FunctionBuilder currently does not deal well with a particular representation of inlinees in breakpad .sym files. This causes it to create sibling inlinees with overlapping line ranges.
With the added validation from #718, this problem can be confirmed as follows:
% cargo run --release -p debuginfo_debug -- symbolic-testutils/fixtures/macos/crash.inlines.sym > /dev/null
Finished release [optimized + debuginfo] target(s) in 0.07s
Running `target/release/debuginfo_debug symbolic-testutils/fixtures/macos/crash.inlines.sym`
WARNING: Overlapping line at 0x2799 in inlinee std::__1::vector<unsigned char, std::__1::allocator<unsigned char> >::~vector() in outer function 0x2210 google_breakpad::ReadImageInfo<google_breakpad::MachO64>(google_breakpad::DynamicImages&, unsigned long long): Starts before the end of the previous line (0x277e..0x27d8, from inlinee std::__1::vector<unsigned char, std::__1::allocator<unsigned char> >::~vector())
WARNING: Overlapping line at 0x2d69 in inlinee std::__1::vector<unsigned char, std::__1::allocator<unsigned char> >::~vector() in outer function 0x27e0 google_breakpad::ReadImageInfo<google_breakpad::MachO32>(google_breakpad::DynamicImages&, unsigned long long): Starts before the end of the previous line (0x2d4e..0x2da8, from inlinee std::__1::vector<unsigned char, std::__1::allocator<unsigned char> >::~vector())
WARNING: Overlapping line at 0x3133 in inlinee std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::basic_string() in outer function 0x3010 google_breakpad::ReadTaskString(unsigned int, unsigned long long): Starts before the end of the previous line (0x311a..0x314c, from inlinee std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::basic_string())
WARNING: Overlapping line at 0x352c in inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const in outer function 0x3480 std::__1::__sort<std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&, google_breakpad::DynamicImageRef*>(google_breakpad::DynamicImageRef*, google_breakpad::DynamicImageRef*, std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&): Starts before the end of the previous line (0x3517..0x353c, from inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const)
WARNING: Overlapping line at 0x3839 in inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const in outer function 0x3480 std::__1::__sort<std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&, google_breakpad::DynamicImageRef*>(google_breakpad::DynamicImageRef*, google_breakpad::DynamicImageRef*, std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&): Starts before the end of the previous line (0x3835..0x3852, from inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const)
WARNING: Overlapping line at 0x3841 in inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const in outer function 0x3480 std::__1::__sort<std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&, google_breakpad::DynamicImageRef*>(google_breakpad::DynamicImageRef*, google_breakpad::DynamicImageRef*, std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&): Starts before the end of the previous line (0x383d..0x3848, from inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const)
WARNING: Overlapping line at 0x391a in inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const in outer function 0x3480 std::__1::__sort<std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&, google_breakpad::DynamicImageRef*>(google_breakpad::DynamicImageRef*, google_breakpad::DynamicImageRef*, std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&): Starts before the end of the previous line (0x390e..0x392a, from inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const)
WARNING: Overlapping line at 0x3969 in inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const in outer function 0x3480 std::__1::__sort<std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&, google_breakpad::DynamicImageRef*>(google_breakpad::DynamicImageRef*, google_breakpad::DynamicImageRef*, std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&): Starts before the end of the previous line (0x3965..0x3982, from inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const)
WARNING: Overlapping line at 0x3971 in inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const in outer function 0x3480 std::__1::__sort<std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&, google_breakpad::DynamicImageRef*>(google_breakpad::DynamicImageRef*, google_breakpad::DynamicImageRef*, std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&): Starts before the end of the previous line (0x396d..0x3978, from inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const)
WARNING: Overlapping line at 0x3a48 in inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const in outer function 0x3480 std::__1::__sort<std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&, google_breakpad::DynamicImageRef*>(google_breakpad::DynamicImageRef*, google_breakpad::DynamicImageRef*, std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&): Starts before the end of the previous line (0x3a40..0x3a4c, from inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const)
WARNING: Overlapping line at 0x362e in inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const in outer function 0x3480 std::__1::__sort<std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&, google_breakpad::DynamicImageRef*>(google_breakpad::DynamicImageRef*, google_breakpad::DynamicImageRef*, std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&): Starts before the end of the previous line (0x3627..0x3633, from inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const)
WARNING: Overlapping line at 0x36df in inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const in outer function 0x3480 std::__1::__sort<std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&, google_breakpad::DynamicImageRef*>(google_breakpad::DynamicImageRef*, google_breakpad::DynamicImageRef*, std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&): Starts before the end of the previous line (0x36dc..0x36f7, from inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const)
WARNING: Overlapping line at 0x36ea in inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const in outer function 0x3480 std::__1::__sort<std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&, google_breakpad::DynamicImageRef*>(google_breakpad::DynamicImageRef*, google_breakpad::DynamicImageRef*, std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&): Starts before the end of the previous line (0x36e6..0x36f7, from inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const)
WARNING: Overlapping line at 0x3aad in inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const in outer function 0x3aa0 std::__1::__sort5<std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&, google_breakpad::DynamicImageRef*>(google_breakpad::DynamicImageRef*, google_breakpad::DynamicImageRef*, google_breakpad::DynamicImageRef*, google_breakpad::DynamicImageRef*, google_breakpad::DynamicImageRef*, std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&): Starts before the end of the previous line (0x3aa0..0x3abd, from inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const)
WARNING: Overlapping line at 0x3c40 in inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const in outer function 0x3bf0 std::__1::__insertion_sort_incomplete<std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&, google_breakpad::DynamicImageRef*>(google_breakpad::DynamicImageRef*, google_breakpad::DynamicImageRef*, std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&): Starts before the end of the previous line (0x3c3d..0x3c5d, from inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const)
WARNING: Overlapping line at 0x3c48 in inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const in outer function 0x3bf0 std::__1::__insertion_sort_incomplete<std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&, google_breakpad::DynamicImageRef*>(google_breakpad::DynamicImageRef*, google_breakpad::DynamicImageRef*, std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&): Starts before the end of the previous line (0x3c44..0x3c4f, from inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const)
WARNING: Overlapping line at 0x3c79 in inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const in outer function 0x3bf0 std::__1::__insertion_sort_incomplete<std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&, google_breakpad::DynamicImageRef*>(google_breakpad::DynamicImageRef*, google_breakpad::DynamicImageRef*, std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&): Starts before the end of the previous line (0x3c6e..0x3c8e, from inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const)
WARNING: Overlapping line at 0x3cc3 in inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const in outer function 0x3bf0 std::__1::__insertion_sort_incomplete<std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&, google_breakpad::DynamicImageRef*>(google_breakpad::DynamicImageRef*, google_breakpad::DynamicImageRef*, std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&): Starts before the end of the previous line (0x3cc0..0x3cdc, from inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const)
WARNING: Overlapping line at 0x3ccb in inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const in outer function 0x3bf0 std::__1::__insertion_sort_incomplete<std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&, google_breakpad::DynamicImageRef*>(google_breakpad::DynamicImageRef*, google_breakpad::DynamicImageRef*, std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&): Starts before the end of the previous line (0x3cc7..0x3cd2, from inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const)
WARNING: Overlapping line at 0x3e77 in inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const in outer function 0x3bf0 std::__1::__insertion_sort_incomplete<std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&, google_breakpad::DynamicImageRef*>(google_breakpad::DynamicImageRef*, google_breakpad::DynamicImageRef*, std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>&): Starts before the end of the previous line (0x3e6f..0x3e7b, from inlinee std::__1::__less<google_breakpad::DynamicImageRef, google_breakpad::DynamicImageRef>::operator()(google_breakpad::DynamicImageRef const&, google_breakpad::DynamicImageRef const&) const)
WARNING: Overlapping line at 0xaf52 in inlinee std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::basic_string(char const*) in outer function 0xadc0 google_breakpad::UTF16ToUTF8(std::__1::vector<unsigned short, std::__1::allocator<unsigned short> > const&, bool): Starts before the end of the previous line (0xaf3f..0xaf68, from inlinee std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >::basic_string(char const*))
These overlapping inlinee lines confuse the symcache writer and lead to incorrect results.
Background info
If two adjacent inlinees at the same depth have the same function name and call location, then dump_syms combines them into one inlinee, even if they're nested inside of different caller functions. This may be surprising but it's unambiguous and more compact.
Here's an example from macos/crash.dSYM. The DWARF reader emits the following structure:
5 [ ] [ ] [ ]
4 [ | ][ ] [ | ][ ] [ | ][ ]
3 [ ][ ] [ ][ ] [ ][ ]
2 [ | | | ][ | | | ][ | | | ]
1 [ ~vector() ][ ~vector() ][ ~vector() ]
0 [ ~vector() ][ ~vector() ][ ~vector() ]
o ][ 499 ][ 511 ][ 511 ][ ...
^-----^-----^-----^-----^-----^-----^-----^-----^-----^-----^-----^-----^-----^-----^-----^----
277e 2783 2788 278f 2794 2799 279e 27a3 27aa 27af 27b4 27bc 27c1 27cb 27d3 27d8
Then dump_syms notices that the ~vector() calls are adjacent, and merges them into one. Specifically, it emits the following in the .sym file:
FUNC 2210 5d0 0 google_breakpad::ReadImageInfo<google_breakpad::MachO64>(google_breakpad::DynamicImages&, unsigned long long)
[...]
INLINE 0 511 12 37 2294 3f 2799 3f
INLINE 1 458 8 38 2294 3f 261a 21 277e 5a
INLINE 2 458 8 39 2294 3f 261a 21 277e 5a
[...]
INLINE 0 499 12 37 261a 21 277e 1b
[...]
Specifically, here the 5a is the merged length.
At the moment, the FunctionBuilder turns this .sym file into the following monstrosity:
5 [ ]
4 [ | ][ ]
3 [ ][ ] [ ] [ ]
2 [ | | | ] [ | ][ ] [ | ][ ]
1 [ ][ ] [ ][ ]
0 [ | | | | | | | ]
1 [ ~vector() ]
0 [ ~vector() ]
o ][ 499 ][ 511 ][ ...
^-----^-----^-----^-----^-----^-----^-----^-----^-----^-----^-----^-----^-----^-----^-----^----
277e 2783 2788 278f 2794 2799 2799 27a3 27a3 27af 27b4 27bc 27c1 27cb 27d3 27d8
We have multiple overlapping lines at the same depth (at depths 0 and 1), and part of the "tree" has collapsed down to a lower level.