cmd/compile: rc1.14rc1 build internal/ssa out of memory

[n2vi]

What version of Go are you using (go version)?

go version go1.13.7 openbsd/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

See attached file.

What did you do?

git pull
git checkout go1.14rc1
cd src
./all.bash

What did you expect to see?

compile and test results as usual

What did you see instead?

Building go toolchain3 using go_bootstrap and Go toolchain2
# cmd/compile/internal/ssa
fatal error: runtime: out of memory

There was actually plenty of available RAM at the time, according to top.

go1.14rc1.txt

[cagedmantis]

/cc @randall77 @josharian @cherrymui

[josharian]

There was actually plenty of available RAM at the time, according to top.

What does "plenty" mean here? I don't know how quickly top refreshes, but building package ssa eats ram (and potentially dies) very quickly.

I suspect this is probably a duplicate of #27739. I have a series of CLs ready for 1.15 that will shrink package ssa by about 25%.

[n2vi]

The machine has 12G RAM. Top showed memory usage varying between 250MB and 500MB most of the time, once getting up to about 1GB, but never anywhere close to running out of physical memory. I didn't do any ulimit or other special action, so could be there was some artificial software limit exceeded.

…

On Fri, Feb 7, 2020 at 12:30 PM Josh Bleecher Snyder < ***@***.***> wrote: There was actually plenty of available RAM at the time, according to top. What does "plenty" mean here? I don't know how quickly top refreshes, but building package ssa eats ram (and potentially dies) very quickly. I suspect this is probably a duplicate of #27739 <#27739>. I have a series of CLs ready for 1.15 that will shrink package ssa by about 25%. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#37122?email_source=notifications&email_token=ACADPOUMWWDAUO4Q46AAAILRBXAFDA5CNFSM4KRT4L52YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOELEP2QI#issuecomment-583597377>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ACADPOWHJ7XWMVXDU3JSO5DRBXAFDANCNFSM4KRT4L5Q> .

[josharian]

Yeah, 12gb really ought to be enough. :) Does it reproduce consistently?

cc @mknyszek @aclements just in case

[n2vi]

Yes, reproduces 100%. Happy to re-run with any additional debugging you can teach me.

In any event, if a normal Go install requires tons of memory, it is probably worth measuring this at release candidate build time and adding a comment to the install doc, or even better finding a way to determine at runtime that the process is likely to fail for a known reason.

[n2vi]

Also, just reproduced it on another OpenBSD machine running stable+syspatch, so this is probably not some recent OpenBSD issue.

I'm happy to give Austin a login if constructing such a machine would be burdensome to y'all. I know I'm running on a niche OS and aspire to the skill to give you a better bug report.

[josharian]

To rule out some things, let's try disabling parallel and concurrent compilation:

GOFLAGS="-p=1 -gcflags=cmd/compile/internal/ssa=-c=1" ./all.bash

If that still fails, let's see how much RAM the shell thinks it is using:

GOFLAGS=-toolexec="/usr/bin/time -l" ./all.bash

[aclements]

OpenBSD by default has a very low RLIMIT_DATA. What's the output of ulimit?

Even if RLIMIT_DATA is raised, OpenBSD limits anonymous mappings to a fairly small factor larger than physical memory (even if they're untouched), and Go 1.14 does create some larger sparse mappings than Go 1.13 did. For this limit, the relevant top column to watch would be VSZ, I believe.

[aclements]

What's the output of ulimit?

Sorry, ulimit -a.

[n2vi]

"ulimit -a" on this 16GB physical NUQkit reports stack 8MB, lockedmem 5GB, mem 16GB.

GOFLAGS="-p=1 -gcflags=cmd/compile/internal/ssa=-c=1" ./all.bash
still fails in the same way.

Maybe I'm blind, but
GOFLAGS=-toolexec="/usr/bin/time -l" ./all.bash
seems to do the same with no extra output. Sorry, I don't know your build system enough to do better yet.

[josharian]

Sounds like Austin probably has a better bead on what’s going on here.

I’m curious though whether my planned changes will help sufficiently. To test,

git fetch "https://go.googlesource.com/go" refs/changes/03/213703/6 && git checkout FETCH_HEAD

and then try again.

[n2vi]

Hurray and thank you! With your change, builds and tests successfully.

[josharian]

That’s good to hear. That won’t land until the 1.15 release, though.