Some C++ symbols emitted by clang-14 cannot be demangled
gabrielesvelto opened this issue · 11 comments
Environment
Symbolic 10.0.0
Steps to Reproduce
symbolic-demangle seems to be unable to demangle any of the symbols in this list. They are taken from a Firefox build and have been generated with clang 14.0.5
Expected Result
The symbols are demangled.
Actual Result
The symbols aren't demangled. That being said I'm not sure if this is an issue with symbolic-demangle itself or if clang emitted buggy demangled symbols... it happened in the past already.
I was briefly looking at some of the simpler identifiers here.
Essentially it all boils down to https://github.com/gimli-rs/cpp_demangle not supporting them.
For some of these and looking at the llvm-cxxfilt output:
> llvm-cxxfilt '__Z14WasmMemoryFillIPhPDoFPvS1_imEjEiP9JSContextT_mT1_jS7_T0_'
int WasmMemoryFill<unsigned char*, void* (*)(void*, int, unsigned long) noexcept, unsigned int>(JSContext*, unsigned char*, unsigned long, unsigned int, unsigned int, unsigned int, void* (*)(void*, int, unsigned long) noexcept)
A bit unclear, maybe the noexcept
part that is not supported?
> llvm-cxxfilt "__ZL21CreateWasmConstructorIN2js13WasmTagObjectEXadsoKcL_ZL11WasmTagNameEEEEP8JSObjectP9JSContext10JSProtoKey"
JSObject* CreateWasmConstructor<js::WasmTagObject, &(WasmTagName.<char const at offset 0>)>(JSContext*, JSProtoKey)
> llvm-cxxfilt "__ZN3hwy13FunctionCacheIvJEE13ChooseAndCallIXadsoKPFvvEL_ZN3jxlL42BenchmarkFastIDCT32x32HighwayDispatchTableEEEEEEvv"
void hwy::FunctionCache<void>::ChooseAndCall<&(jxl::BenchmarkFastIDCT32x32HighwayDispatchTable.<void (* const)() at offset 0>)>()
For these two, I believe the .<X at offset Y>
is something that is not supported.
Its a bit hard to find out what exactly it is unhappy with. I’m not quite sure what it would take to contribute this upstream, I found the code extremely hard to work with last time I contributed something. Also, the whole name mangling is way too underdocumented / underspecified.
I just cloned https://github.com/gimli-rs/cpp_demangle, built it (using cargo build --release
), and installed a c++filt
clone of it locally (using cargo install cpp_demangle --example cppfilt
). It works with all the non-demangled symbols from https://bugzilla.mozilla.org/show_bug.cgi?id=1795461#c0.
So I don't think this bug can be blamed on cpp_demangle. I expect, rather, that it's being given the wrong input.
By the way, I did my tests with and without the "extra" underscore that all symbols have in binaries' symbol tables. cppfilt
worked both ways. But it did fail if I removed all leading underscores.
But cppfilt
does fail with the examples you gave above, @Swatinem. And c++filt
does work with those examples.
So maybe we have two bugs -- one that (sometimes) gives the wrong input to cpp_demangle, and another that's a problem with cpp_demangle itself.
@gabrielesvelto, do you know if there's a (reasonably simple) way to do a local Firefox build with tools from a different version of LLVM? Say 15.0.3, which is the current version? It'd be interesting to see if that made any difference here.
Edit: I think I've figured this out, and am working on it.
Edit: No, building with the tools from LLVM 15.0.3 doesn't seem to have made any difference.
maybe not relevant for your usecase, but we are using a patched version of cpp_demangle
ourselves which might be a bit behind master.
Either way, there doesn’t seem to be any significant changes between master and the last released version either.
Can you provide a filtered down list of identifiers that work with upstream cpp_demangle
but fail in symbolic? I doubt we are doing anything special, except maybe different size/recursion limits
Can you provide a filtered down list of identifiers that work with upstream
cpp_demangle
but fail in symbolic? I doubt we are doing anything special, except maybe different size/recursion limits
I can do that, but it's proving funny because some of these symbols cause the latest version of cpp_demangle
to panic.
All of the 1472 symbols in the list I attached fail on symbolic. The last version of cpp_demangle
is capable of demangling 1385 of those symbols correctly. It fails on 78 of them with an error and panics on 9.
I filed gimli-rs/cpp_demangle#272 and gimli-rs/cpp_demangle#273 to address the symbols that cpp_demangle
does not process correctly, for the rest updating symbolic
to use version 0.3.5
should provide significant improvements for us.
updating
symbolic
to use version0.3.5
should provide significant improvements for us.
That is a semver compatible range, and we don’t have any lockfiles in symbolic directly. So a simple cargo update
should solve it on your end?
I believe this is completely upstream work to do in https://github.com/gimli-rs/cpp_demangle which I believe is already being tackled, so there is nothing really to do on our end.
I can confirm that this has been fixed upstream and we've been using the fix via symbolic w/o encountering any other issues.