getsentry/symbolic

Symbolic cannot find unwind information

fmoc opened this issue · 5 comments

fmoc commented

We use Sentry to collect crash reports for our desktop client, which is written in C++. Recently, we changed our deployment to upload full debug binaries created following this guide in the Sentry docs, according to which they should have unwind information.

Unfortunately, Sentry disagrees and marks all these files as having no unwind information. By reading the code in this repository, we learned that symbolic expects .eh_frame to be present in the files. This is not the case, as we have double and triple checked: objcopy --only-debug-sections does not copy .eh_frame, .eh_frame_hdr or .debug_frame. Sentry therefore has a point in saying that while debug information is there, unwind information is missing.

The unwind information is still contained in the original binaries and I assume if we uploaded the entire binaries to our symbols server instead of the .debug files, they would work just fine.

My guess is that as Sentry as no access to the original files, the unwind information needs to be added to the .debug files. I think the docs are wrong and mislead people to thinking the files already do while this is not the case.

Do we have to add the section manually to the files, e.g., with objcopy --add-section, to create a working setup?

CC @TheOneRing

The unwind information is still contained in the original binaries and I assume if we uploaded the entire binaries to our symbols server instead of the .debug files, they would work just fine.

My guess is that as Sentry as no access to the original files, the unwind information needs to be added to the .debug files. I think the docs are wrong and mislead people to thinking the files already do while this is not the case.

I agree, the docs are a bit misleading, and indeed for ELF that does not have split-debug files by default, it is best to just upload the full executable. Otherwise, you would have to upload both the stripped executable and the corresponding debug file.

I would love to figure out how we can improve our docs to make this clearer. I have seen a couple of support requests recently related to the same problem: ELF users uploading only the debug-info, without the unwind info in the executable.

So where exactly in the docs is it mentioned that uploading only the debug-info would be sufficient?

fmoc commented

Otherwise, you would have to upload both the stripped executable and the corresponding debug file.

This is super useful information. I guess we would have never thought of that! In fact, I just ran symsorter on both, and the output now totally makes sense to me:

$ symsorter --output output binary binary.debug 
WARNING: No compression used. Consider to pass -zz.

Sorting debug information files
binary (lib, x86_64) -> output/27/e78675fd347a768b25b547fd9e1344d87d7dfc/executable
binary.debug (dbg, x86_64) -> output/27/e78675fd347a768b25b547fd9e1344d87d7dfc/debuginfo

Done.
Sorted 2 debug files

Since binaries tend to be quite large, do you think it would make sense to upload an objcopyd file next to the .debug that just contains the .eh_frame section? I think there is no need to upload the entire binary at this point. (symsorter could do the extraction.)

So where exactly in the docs is it mentioned that uploading only the debug-info would be sufficient?

I think the docs kind of suggest this, but they don't explicitly mention it. That said, I only started working on this project recently, debugging an existing setup. I'll have a closer look to the docs and will open issues on the docs repository accordingly.

Thanks!

Since binaries tend to be quite large, do you think it would make sense to upload an objcopyd file next to the .debug that just contains the .eh_frame section? I think there is no need to upload the entire binary at this point. (symsorter could do the extraction.)

That sounds like a good idea! We do not currently use the actual executable code or data in any way, though that could potentially change in the future.

Are you suggesting symsorter should do that "only keep .eh_frame"? Might be possible, but I would rather not have symsorter edit the files directly.

I will close the issue, as it seems like there was a solution to your problem. Please feel free to file requests for improvement on the docs as you suggested.