cmd/go: make rebuild logic content-based instead of time-based

[rsc]

This is a tracking issue for work toward making cmd/go rebuild logic use a content hash of all inputs to the build instead of modification times.

[josharian]

Possibly related: #15752

[gopherbot]

Change https://golang.org/cl/73212 mentions this issue: cmd/go: switch to entirely content-based staleness determination

[ALTree]

I haven't investigated this yet, but just FYI after CL 73212 (cmd/go: switch to entirely content-based staleness determination) I see a 10x slowdown in the compiler-fuzzing driver I use to fuzz gc.

The driver is a go program that generates random go programs using gosmith and then uses exec.Cmd to invoke go build on them. This is the typical output when using 8 workers on a 8-cores machine:

$ go run driver.go -p 8 -checkers amd64
2017/10/31 15:05:51 testing with 8 workers
2017/10/31 15:05:51 0 tests, 0 known, 0 build, 0 ssadump, 0 gofmt, 0 exec
2017/10/31 15:05:54 39 tests, 0 known, 0 build, 0 ssadump, 0 gofmt, 0 exec
2017/10/31 15:05:57 82 tests, 0 known, 0 build, 0 ssadump, 0 gofmt, 0 exec

Notice how in the 6 seconds before I stopped it, 82 multi-package go programs were generated and compiled successfully. After git revision 7dea509 (the CL 73212 I mentioned above), the same command gives this output

$ go run driver.go -p 8 -checkers amd64
2017/10/31 15:07:38 testing with 8 workers
2017/10/31 15:07:38 0 tests, 0 known, 0 build, 0 ssadump, 0 gofmt, 0 exec
2017/10/31 15:07:41 0 tests, 0 known, 0 build, 0 ssadump, 0 gofmt, 0 exec
2017/10/31 15:07:44 8 tests, 0 known, 0 build, 0 ssadump, 0 gofmt, 0 exec

The same 6 seconds of fuzzing, and a ~10x reduction in the number of compiled programs.