cmd/compile: ssa.a contains the string SetsockoptIPv6Mreq

[josharian]

$ grep -c "SetsockoptIPv6Mreq" $GOROOT/pkg/darwin_amd64/cmd/compile/internal/ssa.a
1

This is a bit surprising. How did it get there?

There are a few places in the ssa package where we have types that refer to *os.File, like map[string]*os.File. When exporting, we re-export *os.File, which is struct { *os.file }.

os.file is (on my machine) struct { os.pfd poll.FD; os.name string; os.dirinfo *os.dirInfo; os.nonblock bool }.

poll.FD is struct { poll.fdmu poll.fdMutex; Sysfd int; poll.pd poll.pollDesc; poll.iovecs *[]syscall.Iovec; IsStream bool; ZeroReadIsEOF bool }.

And so on all the way down to SetsockoptIPv6Mreq, leaving a trail of types in our wake.

I've been trying to think of how the compiler might possibly need these types in a package that imports only cmd/compile/internal/ssa. It doesn't need them directly for typechecking, because os.file is not exported, and thus unreachable via our code. The struct size/shape/alignment doesn't matter, because it is only reference is via a pointer (*os.file); same reasoning for GC maps.

I'd love enlightenment.

If they're not needed, we should figure out how to get rid of them. Maybe part of the problem in #20070 is that our object files have lots of stuff in them that we don't even need in the first place?

@griesemer @mdempsky

[josharian]

To help quantify, CL 41493 contains a quick hack to remove explicit references to *os.File from package SSA. It results in a 10% reduction in export data size for the package.

[griesemer]

This seems like another good example of a situation where using an interface to abstract enormous amounts of detail (which in turn turns out as type information) would help. Having an abstract File notion (which has come up many times before) would probably avoid pulling in these details.

[josharian]

@griesemer agreed, although it'd also be better for the compiler to handle this more gracefully as well.

I spent some time on this today but didn't make much progress. There are at least a few things this apparently inaccessible data is being used for: (1) method sets, (2) hash/eq routines, (3) sanity checks across imported packages, (4) re-typechecking of inlined method bodies. (1) and (2) could be represented more parsimoniously. (3) is nice but substantively dropping the cost of compilation is probably nicer. (4) could possibly be worked around.

I wonder whether the sanity checks could be protected behind a flag, in favor of not constructing the type the second time around, and if so, how much that would save in practice.

[josharian]

I just updated CL 41493 with some numbers: This improved compilation time for a downstream package (cmd/compile/internal/amd64) by > 3%. Over a large compilation graph, like benchjuju has, this could be significant.

Moving to 1.10, though, because I don't see this happening in the next week.

[gopherbot]

CL https://golang.org/cl/41493 mentions this issue.

[cherrymui]

The string "SetsockoptIPv6Mreq" is not there anymore with tip (2f588ff).

$ ../bin/go version
go version devel +2f588ff08f Wed Nov 29 20:40:41 2017 +0000 darwin/amd64
$ grep -c "SetsockoptIPv6Mreq" ../pkg/darwin_amd64/cmd/compile/internal/ssa.a
0

Is there anything else expected to be done in this issue? @josharian

[mdempsky]

This was "fixed" by 4ee934a. The more general issue will be fixed by #20070.