cmd/go : Errors in go get all cause the tool to stop and not finish the work

[ardan-bkennedy]

What version of Go are you using (go version)?

$ go version
go version go1.13 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/bill/Library/Caches/go-build"
GOENV="/Users/bill/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/bill/code/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/bill/code/go/src/github.com/ardanlabs/service/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/f8/nl6gsnzs1m7530bkx9ct8rzc0000gn/T/go-build275140276=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

using this project
https://github.com/ardanlabs/service

I ran go get -u all and received an error.

$ go get -u all
go: downloading golang.org/x/net v0.0.0-20190620200207-3b0461eec859
go: extracting golang.org/x/net v0.0.0-20190620200207-3b0461eec859
go: downloading golang.org/x/text v0.3.2
go: extracting golang.org/x/text v0.3.2
go: finding github.com/cznic/golex latest
go: finding github.com/cznic/internal latest
go: finding github.com/cznic/fileutil latest
go: finding golang.org/x/net latest
go: finding github.com/cznic/zappy latest
go: finding golang.org/x/crypto latest
go: finding github.com/cznic/mathutil latest
go: finding github.com/GuiaBolso/darwin latest
go: finding github.com/cznic/sortutil latest
go: finding github.com/golang/groupcache latest
go: finding github.com/cznic/b latest
go: finding github.com/cznic/strutil latest
go: finding gopkg.in/DATA-DOG/go-sqlmock.v1 v1.3.3
go: finding gopkg.in/DATA-DOG/go-sqlmock.v1 v1.3.3
go get all: gopkg.in/DATA-DOG/[email protected]: invalid version: go.mod has non-....v1 module path "github.com/DATA-DOG/go-sqlmock" at revision v1.3.3

When I run go get -u ./... everything succeeds.

$ go get -u ./...
go: finding github.com/GuiaBolso/darwin latest
go: finding github.com/golang/groupcache latest
go: finding golang.org/x/crypto latest

What did you expect to see?

I expected go get -u all to behave the same way as go get -u ./...

What did you see instead?

I saw the documented failure about the invalid version.

What's worse, go get stopped finishing it's update. This is the real problem for me.

More information

I understand the failure. I am using a dependency github.com/GuiaBolso/darwin that imported the DataDog go-sqlmock package via gopkg.in. The latest version of go-sqlmock has a go.mod file. There is an incompatibility as we are seeing.

To fix this I added a replace command in my go.mod file

https://github.com/ardanlabs/service/blob/master/go.mod

replace gopkg.in/DATA-DOG/go-sqlmock.v1 => gopkg.in/DATA-DOG/go-sqlmock.v1 v1.3.0

Version 1.3.0 is the highest version of go-sqlmock that does not have a go.mod file.

[seankhliao]

all includes all dependencies (direct and indirect), including test dependencies of your dependencies

./... only includes the dependencies needed to build your package, doesn't include test dependencies or those listed under build tags

[ardan-bkennedy]

Ok, I see. I've been trying to figure out these differences.

When it is said, update "indirect" dependencies, does that mean my cache contains versions of a deps, deps that are potentially higher than listed in that deps own module file?

Though there is still a problem because since this versioning issue exists, it is causing go get to fail and exit when it should continue it's work with other potential dependency updates.

[seankhliao]

version selection works by building out a graph of dependencies recursively

"indirect" means not directly imported by your packages (dependency of a dependency, dependency under specific conditions (test, build tags))

go get should fail since it cannot build out the graph of needed dependencies because it can't get the broken dependency (and therefore the dependencies it needs). Without this information, the resulting graph would be wrong, which is why it wants you to fix it before continuing

cache just saves you a roundtrip to remote servers, which version is used is still decided by building out he graph of needed dependencies

[ardan-bkennedy]

I get that, thanks! Maybe I am having UI / useability issues?

I felt like something was now partially updated :( I am left wondering if I am in an odd state? Did the tooling rollback or was nothing even updated? It would be nice to be told that. I am clueless.

I would like a warning with a prompt (Y/N) from the tooling anytime you are about to update anything indirectly. There are so many switches and combinations, you could do this when you don't want to.

In fact, maybe a mode that prompts anytime the go.mod or go.sum file will be touched. That would give me the opportunity to pause and try to understand what I just did. Eventually I could turn that off.

[bcmills]

This is the same underlying problem as #30831: prior to modules, the repository lacked an indication of its canonical import path. GOPATH mode did not verify import paths unless an // import comment was present, but in module mode we now have a path for every go.mod file, so the mismatched import path can be detected and made an explicit error.

(It is important to detect this error because packages within a repo or module may import other packages within that module, and if the paths don't match you end up with two copies of each package instead of just one.)

There are several possible short-term workarounds:

1. Explicitly replace the problematic module path with the last version at which its repo was compatible with that path.
2. Explicitly suppress the upgrade of the problematic module by requesting its last compatible version:

   go get -u all gopkg.in/DATA-DOG/[email protected]
   

3. Upgrade your transitive dependencies to a point at which they no longer use to the non-canonical import path.

I notice that you already found and applied workaround (1).

Long term, we need better support for repositories whose import paths have changed. That is tracked in #30831.

[bcmills]

Duplicate of #30831

[dolmen]

@ardan-bkennedy The import path for DATA-DOG/sqlmock has changed. The new import path is github.com/DATA-DOG/go-sqlmock.