cmd/go: go test in a directory without any test files uses network access to try to fetch the source for the code running locally

[davecheney]

What version of Go are you using (go version)?

% go version
go version devel +c485e8b559 Sun Jun 30 05:48:31 2019 +0000 darwin/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output

% go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/dfc/Library/Caches/go-build"
GOENV="/Users/dfc/Library/Application Support/go/env"
GOEXE=""
GOFLAGS="-ldflags=-w"
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/dfc"
GOPRIVATE=""
GOPROXY="https://gocenter.io"
GOROOT="/Users/dfc/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/Users/dfc/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/dfc/devel/mod/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/by/3gf34_z95zg05cyj744_vhx40000gn/T/go-bui
ld247327765=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

(~/devel) % mkdir mod
(~/devel) % cd mod
/Users/dfc/devel/mod
(~/devel/mod) % go mod init github.com/foo/bar
go: creating new go.mod: module github.com/foo/bar
(~/devel/mod) % go test -x

What did you expect to see?

A message about no test files for the package.

What did you see instead?

The go tool is accessing a network service to try to fetch the source code for the package I am currently in.

# get https://gocenter.io/github.com/@v/list
# get https://gocenter.io/github.com/foo/@v/list
# get https://gocenter.io/github.com/@v/list: 404 Not Found (1.182s)
# get https://gocenter.io/github.com/foo/@v/list: 404 Not Found (1.184s)
can't load package: package github.com/foo/bar: unknown import path "github.com/foo/bar": cannot find module providing package github.com/foo/bar

[bcmills]

The underlying cause is likely the same as #28459, but I'm going to leave this one open since the failure mode is somewhat different.

[bcmills]

This certainly needs to be fixed, but a proper fix is fairly invasive (and probably best left for 1.14).

I think it is important that we at least eliminate the network access for 1.13, but the error message will probably be abysmal.

(CC @jayconrod in case he disagrees or can spot a simpler robust fix.)

[jayconrod]

Can we actually avoid going out to the network without breaking existing behavior?

Suppose the main module is example.com/a, and there's another module, example.com/a/b in a different repository (not in the directory tree). There is a package example.com/a/b/c in example.com/a, not in example.com/a/b. That means b must be a directory without .go files in the example.com/a. If we change to that directory and run go build . or go test ., the Go command actually should fetch example.com/a/b if it's not present in the cache. We may need to add a new requirement on module example.com/a/b as well.

Sorry this is a contrived example, but as long as we allow packages to be provided by any module whose path is a prefix, I think we'll have this problem.

[bcmills]

If we change to that directory and run go build . or go test ., the Go command actually should fetch example.com/a/b if it's not present in the cache.

I disagree. In general, we only allow paths starting with . in module mode to refer to packages within the same module. (Consider, for example, the behavior of go test ./....) Since example.com/a/b is not in module example.com/a, running go build . in within the ./b subdirectory of example.com/a should be an error.

[jayconrod]

We could fix this by requiring relative paths to be packages in the main module. It looks like if a relative path contains a wildcard, we expand it to paths in the main module containing go files, and we won't go out to the network. If a path does not contain a wildcard, we join it with the module path without checking if anything is there, treating it as a full package path thereafter.

So currently, go build . is equivalent to go build example.com/a/b in my example. But we could fix go build . while keeping the current behavior of go build example.com/a/b.

[bcmills]

Yes. The tricky part is the vendor directory: we should probably resolve it to the corresponding module path if -mod=vendor is set and the directory contains .go source files, and refuse to resolve it otherwise.

[bcmills]

I think I've spotted a simple fix for 1.13. Will send a CL tomorrow (assuming it works out).

[gopherbot]

Change https://golang.org/cl/185344 mentions this issue: cmd/go/internal/search: record errors in the Match struct

[gopherbot]

Change https://golang.org/cl/185343 mentions this issue: cmd/go/internal/modload: make packageNotInModuleError reasonable for the Target module

[gopherbot]

Change https://golang.org/cl/185345 mentions this issue: cmd/go: rationalize errors in internal/load and internal/modload

[gopherbot]

Change https://golang.org/cl/185417 mentions this issue: cmd/go: check for source files in relative paths before attempting to determine the package path

[gopherbot]

Change https://golang.org/cl/221458 mentions this issue: cmd/go/internal/search: cache IsLocalImport and IsAbs predicates in a Match field