misc/cgo/stdio: test persistently broken on linux-arm-arm5spacemonkey builder

[bcmills]

(See previously #29177 and #28041.)

The current failure on the arm5spacemonkey builder is (usually) a permission denied error in misc/cgo/stdio. I'm not sure whether it's a bug (or unrealistic expectation) in the test, or a misconfiguration in the builder.

##### ../misc/cgo/stdio
go test misc/cgo/stdio.test: open /home/builder/stage0scratch/workdir/go/misc/cgo/stdio/test.test: permission denied
2019/06/02 22:52:55 Failed: exit status 1

CC @ianlancetaylor @dmitshur

[bcmills]

I'm not sure whether it's a bug (or unrealistic expectation) in the test, or a misconfiguration in the builder.

Perhaps it's neither: it may instead be a bug in cmd/dist.registerHostTest, which (possibly erroneously?) assumes that it can write executables in $GOROOT/misc/cgo/....

[esnolte]

Is there anything I can help with? The /home/builder/stage0scratch/workdir/go/misc/cgo/stdio directory is owned by builder but doesn't have write permissions on the directory.

builder@go-builder-1:~$ ls -al /home/builder/stage0scratch/workdir/go/misc/cgo/stdio
total 20
dr-xr-xr-x 3 builder builder 4096 Oct 21 12:29 .
dr-xr-xr-x 20 builder builder 4096 Oct 21 12:29 ..
-r--r--r-- 1 builder builder 1977 Oct 21 11:41 overlaydir_test.go
-r--r--r-- 1 builder builder 1504 Oct 21 11:41 stdio_test.go
dr-xr-xr-x 3 builder builder 4096 Oct 21 12:29 testdata

The missing write permission causes a permission denied error from the shell, e.g.
builder@go-builder-1:~$ touch /home/builder/stage0scratch/workdir/go/misc/cgo/stdio/foo
touch: cannot touch ‘/home/builder/stage0scratch/workdir/go/misc/cgo/stdio/foo’: Permission denied

[bcmills]

@esnolte, I suspect that the fix belongs here:

go/src/cmd/dist/test.go

Lines 1009 to 1017 in a38a917

	func (t *tester) runHostTest(dir, pkg string) error {
	defer os.Remove(filepath.Join(goroot, dir, "test.test"))
	cmd := t.dirCmd(dir, t.goTest(), "-c", "-o", "test.test", pkg)
	cmd.Env = append(os.Environ(), "GOARCH="+gohostarch, "GOOS="+gohostos)
	if err := cmd.Run(); err != nil {
	return err
	}
	return t.dirCmd(dir, "./test.test", "-test.short").Run()
	}

Currently runHostTest writes the test executable to the same directory as the package under test, then runs it. I don't know why that is split into two separate phases.

That behavior seems to have been present from the start, in CL 21102. Perhaps @eliasnaur or @ianlancetaylor remembers why it was done that way — perhaps it was something do to with caching?

[bcmills]

Ah, that's the difference: the GOARCH and GOOS values used when compiling the test binary need to match the host arch and OS, whereas the GOARCH and GOOS values used when running the test should match the target platform.

So perhaps we just need to pick a more appropriate location to write the resulting binary.

[bcmills]

From #33058 (comment), this seems to be an intentional consequence of #30316:

go/src/cmd/dist/test.go

Lines 194 to 197 in a38a917

	// On a few builders, make GOROOT unwritable to catch tests writing to it.
	if strings.HasPrefix(os.Getenv("GO_BUILDER_NAME"), "linux-") {
	t.makeGOROOTUnwritable()
	}

[bcmills]

This turns out to be easy to reproduce locally by setting GO_BUILDER_NAME=linux-foo and running go tool dist test cstdio_test.

So the real question is: why doesn't it reproduce on the other linux- builders‽

[gopherbot]

Change https://golang.org/cl/206458 mentions this issue: misc: ensure that test overlay directories are writable

[gopherbot]

Change https://golang.org/cl/206459 mentions this issue: cmd/dist: write binaries to to GOTMPDIR instead of GOROOT in runHostTest

[gopherbot]

Change https://golang.org/cl/206477 mentions this issue: cmd/go: in 'go build -o', allow the destination file to exist if it is empty

[bcmills]

misc/cgo/stdio is no longer failing on this builder. On to the next failure! 😤