cmd/go: generating C header files for transitive dependencies is awkward

[bcmills]

As discovered in CL 208117 (#30316), cmd/go (as of Go 1.14) lacks a good way to generate C header files for Go libraries in order to use them from C.

go install -buildmode=c-archive and go install -buildmode=c-shared will, as a side-effect, generate header files for the requested package. However, -buildmode=c-archive and -buildmode=c-shared today require the named package to be a package main, and those commands do not generate headers for the (transitive) Go dependencies of the named package.

The -i flag does generate headers for transitive dependencies, but it also has a second, unwanted side effect: it builds (and attempts to install) libraries for the transitive dependencies of the named package, including transitive dependencies in the standard library. That fails if the user cannot write to GOROOT/pkg, such as when a non-root user is working with a go tool installed as root.

As a workaround, one can run go tool cgo -exportheader, but go tool cgo accepts a list of files rather than a list of packages, and generates a number of additional outputs (normally in an _obj subdirectory) that the C caller does not directly need. So we're still doing extra work, just in a different location.

We should see what we can do to make it easier to generate C headers without unnecessary or unwanted side-effects.

CC @ianlancetaylor @jayconrod

[bcmills]

One possibility might be to add another -buildmode, such as -buildmode=c-header, that does whatever work is needed (and possibly writes extra artifacts into GOCACHE) but only emits (or installs) header files rather than entire archives.

That could be used as a more direct substitute for the go tool cgo call with specific package arguments.

[bcmills]

Another option might be to make the -i flag suppress errors for standard-library targets, so that the go install -i command would work even if GOROOT is not writable.

[gopherbot]

Change https://golang.org/cl/208117 mentions this issue: misc/cgo/testcarchive: avoid writing to GOROOT in tests

[gopherbot]

Change https://golang.org/cl/208119 mentions this issue: misc/cgo/testcshared: avoid writing to GOROOT in tests

[ianlancetaylor]

We could have go install install all headers for imported packages, even without -i. That would be a change in behavior, but only for packages that are not main but have exported functions, which is an uncommon case.

[bcmills]

Maybe, but the import itself seems like a bit of a non-sequitur: if the C code is using identifiers from package a, then it seems like the build process should mention package a rather than some other package b that happens to import a.

On the other hand, I suppose that the C library must link against some (shared or archive) file generated from a main package, so having go install generate all of the headers for identifiers linked into that main package doesn't seem terrible.

[crockeea]

Any movement on this? I'm trying the workaround suggested by bcmills@ of using go tool cgo -exportheader, so my build script includes:

go tool cgo -exportheader ${CGO_BUILD_ROOT}/foo.h foo.go
&& go tool cgo -exportheader ${CGO_BUILD_ROOT}/bar.h bar.go
&& go build -buildmode=c-shared -o ${LIB_FULL_PATH}
&& rm -rf _obj

where both foo.go and bar.go are in the same package. This was working fine until I tried to use a type alias.

I'd like to define a type alias type T1 = T2 and use that alias in both foo.go and bar.go.

If I define the alias in foo.go, go build works fine, as expected (T1 is in scope in bar.go). However, exporting headers for bar.go fails with unrecognized Go type T1.

If I define the same alias in foo.go and bar.go, exporting headers succeeds for both files, but I get a redeclaration error with go build (not surprising).