cmd/go: build fetches a module that has been replaced

[pokstad]

What version of Go are you using (go version)?

go version go1.11 linux/amd64

Does this issue reproduce with the latest release?

Yes, if latest release is 1.11

What operating system and processor architecture are you using (go env)?

The following was created in the docker container golang:1.11 with GO111MODULE=on:

GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/go/src/my_private_project/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build292379904=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I am using the new module functionality to handle my project dependencies. One of my dependencies has an import path that is a gopkg.in redirect URL. My corporate network does not allow gopkg.in for security reasons, so in my go.mod file I use a replace statements:

replace (
	gopkg.in/yaml.v2 v2.0.0 => github.com/go-yaml/yaml v0.0.0-20180328195020-5420a8b6744d
	gopkg.in/yaml.v2 v2.2.1 => github.com/go-yaml/yaml v0.0.0-20180328195020-5420a8b6744d
	// ... etc
)

require (
	gopkg.in/yaml.v2 v2.2.1
	// ... etc
)

My dependency graph only lists the two versions of go-yaml that I included in my replacement statements:

$ go mod graph | grep gopkg.in/yaml
my_private_project gopkg.in/[email protected]
[email protected] gopkg.in/[email protected]
[email protected] gopkg.in/[email protected]
github.com/spf13/[email protected] gopkg.in/[email protected]
github.com/hashicorp/[email protected] gopkg.in/[email protected]
gopkg.in/[email protected] gopkg.in/[email protected]
[email protected] gopkg.in/[email protected]
[email protected] gopkg.in/[email protected]
gopkg.in/[email protected] gopkg.in/[email protected]_private_project

When performing a module based install, the following appears in the output:

$ GO111MODULE=on go build
Fetching https://gopkg.in/yaml.v2?go-get=1
https fetch failed: Get https://gopkg.in/yaml.v2?go-get=1: dial tcp 35.196.143.184:443: i/o timeout

The other packages then compile and the executable successfully builds.

What did you expect to see?

No references to any attempt of downloading the gopkg.in/yaml project. Instead, the direct GitHub link provided in the replacement statements should be fetched instead.

What did you see instead?

The go command attempts to reach the gopkg.in/yaml URL each time a build is attempted. It causes a longer than needed build time since the fetch times out. The build succeeds, but it isn't clear why since the fetch failed.

[bcmills]

Possibly related to #26241.

[bjm88]

This seems like a major issue in that every save in IDE causes build and its majorly delayed to get even basic responses on syntax errors to users. Breaks the expected and loved normally very fast compiling of go for engineers. Is there any other work around or standard way to reference local packages rather than ones that need an http fetch? Having to use replace for this instead of a native way in the require statement in go.mod seems like it would be more appropriate. Regardless of general design gap perhaps, can this specific fix be considered for 1.11.1 and not wait until 1.12?

Note I'm talking about doing a replace to relative path "../common_package"

[mibes]

If this issue is hindering your productivity, you can consider this temporary workaround.

Include the domain name that Go is trying to fetch in your /etc/hosts file (on Linux/Mac) and point it to your local machine. So something like this for the example above:

127.0.0.1 localhost gopkg.in

Please note that this will not fix the issue, but the fetch will fail instantly which brings the build speed back to what you are used to:

https fetch failed: Get https://gopkg.in/yaml.v2?go-get=1: dial tcp [::1]:443: connect: connection refused

Keep in mind that the above change affects all network operations.

Hope this helps short term.

[bjm88]

Very hacky @mibes :) . but thank you for the creative solution, guess thats what I'll do for now. It would require we actually change our package module paths to some fake domain and not our own real one, so slightly disruptive, but doable temporarily

[gopherbot]

Change https://golang.org/cl/140860 mentions this issue: cmd/go/internal/modload: skip fetches of replaced modules in moduleInfo

[Dreamwalker666]

+1 just hit this exact same problem. mine for local repos with it trying to fetch from the internet without resolving the "replace" first.

[dmitris]

in general it would be great (for security and network policy-related reasons) to have the following guarantee: if all your required dependencies in "external" domains (github.com, golang.org, gopkg.in etc.) have been replaced, go will not attempt any network connections to those domains but instead will just use the domain from the replacement. This way a CI system for example could check if there are any network connections to disallowed domains and if so, conclude the replacements are not complete and break the build. Or a build could be conducted in a network-sandboxed environment without any failures or slowdown from attempting to reach inaccessible and unnecessary domains (since all code is available from the replacement domains).

I know @bcmills mentoned using GOPROXY earlier in related discussions and it's likely another way to do this but would be useful to have the discussed limits of go's network "outreach" without requirement of additional "moving parts".

[stevenh]

go list also seems to download too which makes our build pipeline horrifically slow.

For now we can ensure GO111MODULE isn't set to avoid this but moving forward it seems that tools that aren't explicitly doing gets should not attempt to do any of this processing.

[Dreamwalker666]

It should always be local processing first then remote. What happens if your in an environment with no network support at all?

[myitcv]

@stevenh - the fix in https://go-review.googlesource.com/c/go/+/140860 will work for all of the go commands.

[stevenh]

Thanks @myitcv but still seeing go list pulling data from the net on each run even with that patch e.g.

> setenv GO111MODULE on
> /usr/bin/time -h go list -e -f '{{if .TestGoFiles }}{{.ImportPath}}{{end}}' ./... > f
go: finding github.com/multiplay/go-utils/cdata latest
go: finding github.com/multiplay/go-utils latest
        1.44s real              1.88s user              1.08s sys
> unsetenv GO111MODULE
> /usr/bin/time -h go list -e -f '{{if .TestGoFiles }}{{.ImportPath}}{{end}}' ./... > f
        0.87s real              0.78s user              0.42s sys

Should I raise a separate issue to track this?

[bcmills]

Yes, please do raise a separate issue (with steps to reproduce it).

The fix in CL 140860 addresses the known source of spurious fetches, but there may be others.

[john8329]

I had the same problem, fixed by manually compiling and using the latest version of the go sdk.