Add support for `split-debuginfo`
Opened this issue · 3 comments
We currently emit debug info into the output spv "binary". Rust has an option to split it out into different files(https://doc.rust-lang.org/cargo/reference/profiles.html#split-debuginfo). We should read that option and support putting the debug info in a different file from the artifact.
Possibly related: https://www.samjs.io/blog/rudy
Split debuginfo works because DWARF is already "split" from the code/data of a binary (it goes into .
In fact, it's really only a debugger feature. You could've always split the debuginfo after the fact.
The rustc "split debuginfo" support is an optimization that allows the debuginfo to (mostly?) bypass the linker, it's not really a new technical feature (more of a combination of increasing both convenience and performance).
SPIR-V never had debuginfo designed for it, and the two existing debuginfo specs for it:
They both vaguely copy DWARF, but only at a conceptual level, so really they copy LLVM !DIFoo metadata nodes.
And even worse than LLVM, SPIR-V is just a stream of instructions, so those extended instruction sets have a mix of declarative (OpTypeVoid-typed constants that work more or less like the LLVM metadata nodes) and imperative (setting various kinds of metadata that apply to following instructions, until another instruction to change them again, or often the end of a basic block, when inside a function).
DWARF gets to point to executable code (i.e. byte ranges in it) and reference registers and memory, but a lot of that doesn't make sense for an IR's more abstract nature, and we're also not emitting any of the kinds of debuginfo (e.g. types, fn signatures, etc.) that could even be reused in a setting like SPIR-V (we just have file:LL:CC information, really, plus the "inlined at" stuff, but that doesn't get emitted into the final SPIR-V).
Oh, and, there's baffling decisions, too, like NonSemantic.Shader.DebugInfo.100's DebugInlinedAt having a line number, but not a column number.
So the SPIR-T "inlined at" in-memory debuginfo added in Rust-GPU/spirt#9 (with output compactified in Rust-GPU/spirt#18), cannot be serialized to existing SPIR-V debuginfo (even if we put in the effort of emitting all the auxiliary stuff needed to use any of those annotations, which is why I put it off originally).
So yeah, if we wanted "split debuginfo" we would have to invent a format and either try to get wider adoption of it, and/or have a tool to reincorporate it - but we barely emit any debuginfo, any inefficiencies it adds are due to SPIR-V design decisions that affect everything else as well, and e.g. LZ4/zstd compression should be encouraged if it becomes.
(or a better binary format that is easy to losslessly convert to/from SPIR-V, I think I have some notes somewhere about a "SPIR-🇼" that uses more the wasm conventions but also is more self-describing etc.)
I suppose one easy thing we could do is split out the OpSource contents into a separate SPIR-V file (those are included in the intermediate ..spv files, but not the final shader, as a semi-standard way to be able to account for the contents of files that we want to generate rustc diagnostics into etc.) - but since they were never in the final shader output (AFAIK), we would be adding them in, so it's new information, not splitting out the existing one
EDIT: okay I went and checked, and we do seem to keep OpSource, at least in disassembly tests?
(and maybe only in release mode they get removed - either way, that's something, I suppose, however small)
Yeah, I was thinking of (ab)using dwarf so that visualization tools that deal with dwarf might just work.