cmd/link: compress debug info on windows
dajoo75 opened this issue · 19 comments
What did you do?
Compiled a basic hello world program on 64-bit Windows 10 with go tip (+b459e00), go 1.8.1 and go 1.10.3 and compared the binary sizes.
hello_go1.8.1.exe | hello_go1.10.3.exe | hello_+b459e00.exe
binary size 1 633 280 bytes | 2 057 728 bytes | 2 848 768 bytes
| |
.rdata N/A | N/A | 752 128 bytes
.text 1 068 032 bytes | 1 181 696 bytes | 575 488 bytes
.debug_info 260 608 bytes | 396 800 bytes | 457 216 bytes
.debug_loc N/A | N/A | 404 480 bytes
.symtab 98 304 bytes | 141 312 bytes | 142 336 bytes
.debug_frame 70 144 bytes | 75 264 bytes | 76 288 bytes
.debug_line 68 519 bytes | 72 192 bytes | 139 264 bytes
.debug_pubtypes 30 720 bytes | 42 496 bytes | 44 544 bytes
.debug_pubnames 17 408 bytes | 40 960 bytes | 41 472 bytes
.data 15 360 bytes | 77 824 bytes | 78 336 bytes
.debug_ranges N/A | 25 600 bytes | 133 632 bytes
.idata 1 536 bytes | 1 024 bytes | 1 024 bytes
.debug_aranges 512 bytes | N/A | N/A
.debug_abbrev 512 bytes | 512 bytes | 512 bytes
.debug_gdb_scripts N/A | 512 bytes | 512 bytes
What did you expect to see?
The binary size to be about the same at tip (+b459e00) as the previous release (go1.10.3 at the time of writing this). And if possible, the binary size to have gone down.
Other related issues: #6853, #11799
What did you see instead?
The binary size at tip (+b459e00) has increased by 38 percent since the previous release (go1.10.3 at the time of writing this).
Does this issue reproduce with the latest release (go1.10.3)?
Yes
System details
go version devel +b459e00 Sat Jun 16 13:35:45 2018 +0000 windows/amd64
GOARCH="amd64"
GOBIN=""
GOCACHE="C:\Users\loki\AppData\Local\go-build"
GOEXE=".exe"
GOHOSTARCH="amd64"
GOHOSTOS="windows"
GOOS="windows"
GOPATH="C:\home\projects\go"
GOPROXY=""
GORACE=""
GOROOT="C:\home\installs\go\go.tip"
GOTMPDIR=""
GOTOOLDIR="C:\home\installs\go\go.tip\pkg\tool\windows_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\loki\AppData\Local\Temp\go-build103655438=/tmp/go-build -gno-record-gcc-switches"
VGOMODROOT=""
GOROOT/bin/go version: go version devel +b459e00 Sat Jun 16 13:35:45 2018 +0000 windows/amd64
GOROOT/bin/go tool compile -V: compile version devel +b459e00 Sat Jun 16 13:35:45 2018 +0000
gdb --version: GNU gdb (GDB) 8.0
As you word it, this would technically be a dup of #6853.
What I think it's worth noting is that CL 118276 (cmd/link: compress DWARF sections in ELF binaries) only helped with ELF binaries. So I think it would be better if you renamed this issue to point out this specific issue (i.e., make this issue a TODO reminder to compress debug info on windows, too).
In theory we had #11799 (cmd/link: compress debug info) for the general problem, but it got closed after the ELF work.
@ALTree I've renamed the issue to cmd/link: compress debug info on windows as you suggested.
Others more in the know about Windows, the PE file format and the debug story around Windows (gdb, delve etc.) have to chime in on the correct way forward.
I tried to get this working on Windows (made some small changes to cmd/link and, debug/pe) and hello world.exe went from 2.8MB to 1.9MB. I almost got delve to start but didn't have time to finish it.
There doesn't seem to be an insurmountable amount of work left to do to get compression working on Windows as well (since most of the work seems to be done already in #11799) but I may be wrong (the devil is in the details and all that).
I would feel better if the original authors of #11799 looked at this as well (together with Alex) since they know what they are doing :).
cc @heschik, @alexbrainman, @ianlancetaylor, @aclements
@dajoo75 what approach are you taking? Elf supports compression but PE doesn't (based on a ctrl-f search in the relevant MSDN page), so .zdebug?
Ah, I didn't notice the switch to zdebug. In that case: https://github.com/derekparker/delve/pull/1237
Thank you @dajoo75 for creating the issue.
If I understand correctly, CL 118276 added compression for ELF DWARF sections. I would try and add some code to make .zdebug_... sections appear in Windows executables, and see where it gets us. I doubt I would have time before weekend. So others are welcome to this.
I also wonder if debuggers used by our users will support this. Lucky we have @aarzilli here, so he can fix Delve. Not so easy to fix gdb. We will see.
Alex
Not so easy to fix gdb. We will see.
I think gdb was the original user of the zdebug convention, I imagine that should be fine. The problem is lldb.
The problem is lldb.
Does lldb work on Windows?
Alex
Does lldb work on Windows?
Partially: https://lldb.llvm.org/status.html
@dajoo75 I'm happy to review a CL and help you over some bumps; I'd have to set up a Windows machine to even start on this and it sounds like you're well into it. We are pretty deep into the freeze so it'd be nice to get it done soon.
Can someone confirm that on Windows we're basically ignoring the platform convention for debug info, which I guess would either be COFF or PDB? As long as that's the case and none of the standard tools work anyway, I think it makes sense to go ahead and compress it. I doubt lldb is very popular there, even if it does work.
@heschik That is correct: we only generate DWARF. COFF debug info is entirely inadequate. As far as I know PDB is not publicly documented.
Okay. @alexbrainman, if you don't claim this by tomorrow I'll take a crack at it.
Okay. @alexbrainman, if you don't claim this by tomorrow I'll take a crack at it
Please, do. I will try it myself, but I do not know when.
Alex
Change https://golang.org/cl/119816 mentions this issue: cmd/link: enable DWARF compression on Windows
Change https://golang.org/cl/119815 mentions this issue: debug/elf,macho,pe: support compressed DWARF
@ianlancetaylor
PDB's definition: https://github.com/Microsoft/microsoft-pdb
@be5invis Thanks.