golang/go

cmd/compile: rc1.14rc1 build internal/ssa out of memory

n2vi opened this issue · 13 comments

n2vi commented

What version of Go are you using (go version)?

go version go1.13.7 openbsd/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

See attached file.

What did you do?

git pull
git checkout go1.14rc1
cd src
./all.bash

What did you expect to see?

compile and test results as usual

What did you see instead?

Building go toolchain3 using go_bootstrap and Go toolchain2
# cmd/compile/internal/ssa
fatal error: runtime: out of memory

There was actually plenty of available RAM at the time, according to top.

go1.14rc1.txt

josharian commented

There was actually plenty of available RAM at the time, according to top.

What does "plenty" mean here? I don't know how quickly top refreshes, but building package ssa eats ram (and potentially dies) very quickly.

I suspect this is probably a duplicate of #27739. I have a series of CLs ready for 1.15 that will shrink package ssa by about 25%.

n2vi commented
josharian commented

Yeah, 12gb really ought to be enough. :) Does it reproduce consistently?

cc @mknyszek @aclements just in case

n2vi commented

Yes, reproduces 100%. Happy to re-run with any additional debugging you can teach me.

In any event, if a normal Go install requires tons of memory, it is probably worth measuring this at release candidate build time and adding a comment to the install doc, or even better finding a way to determine at runtime that the process is likely to fail for a known reason.

n2vi commented

Also, just reproduced it on another OpenBSD machine running stable+syspatch, so this is probably not some recent OpenBSD issue.

I'm happy to give Austin a login if constructing such a machine would be burdensome to y'all. I know I'm running on a niche OS and aspire to the skill to give you a better bug report.

josharian commented

To rule out some things, let's try disabling parallel and concurrent compilation:

GOFLAGS="-p=1 -gcflags=cmd/compile/internal/ssa=-c=1" ./all.bash

If that still fails, let's see how much RAM the shell thinks it is using:

GOFLAGS=-toolexec="/usr/bin/time -l" ./all.bash

aclements commented

OpenBSD by default has a very low RLIMIT_DATA. What's the output of ulimit?

Even if RLIMIT_DATA is raised, OpenBSD limits anonymous mappings to a fairly small factor larger than physical memory (even if they're untouched), and Go 1.14 does create some larger sparse mappings than Go 1.13 did. For this limit, the relevant top column to watch would be VSZ, I believe.

aclements commented

What's the output of ulimit?

Sorry, ulimit -a.

n2vi commented

"ulimit -a" on this 16GB physical NUQkit reports stack 8MB, lockedmem 5GB, mem 16GB.

GOFLAGS="-p=1 -gcflags=cmd/compile/internal/ssa=-c=1" ./all.bash
still fails in the same way.

Maybe I'm blind, but
GOFLAGS=-toolexec="/usr/bin/time -l" ./all.bash
seems to do the same with no extra output. Sorry, I don't know your build system enough to do better yet.

josharian commented

Sounds like Austin probably has a better bead on what’s going on here.

I’m curious though whether my planned changes will help sufficiently. To test,

git fetch "https://go.googlesource.com/go" refs/changes/03/213703/6 && git checkout FETCH_HEAD

and then try again.

n2vi commented

Hurray and thank you! With your change, builds and tests successfully.

josharian commented

That’s good to hear. That won’t land until the 1.15 release, though.