cmd/go: go mod tidy misses adding certain lines to go.sum

[jsternberg]

What did you do?

The reproduction steps and results are located in this repository: https://github.com/jsternberg/gotidyissue

1. Create a project and add a dependency on a project.
2. Replace that module with another location.
3. Run go mod tidy.
4. Run a go command that compiles the module (such as test)

What did you expect to see?

I expected go mod tidy to produce the final go.mod and go.sum files.

What did you see instead?

It doesn't seem to add all entries to go.sum. It adds all of the go.mod entries, but it doesn't always get them all. This was one specific scenario using replace, but I think I've also seen it miss entries with non-replace directives.

I will add more to this issue as I run into them. We weren't sure if it was a bug when we were encountering it.

System details

go version go1.11.1 darwin/amd64
GOARCH="amd64"
GOBIN="/Users/jsternberg/go/pkg/bin/github.com/influxdata/platform"
GOCACHE="/Users/jsternberg/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/jsternberg/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.11.1/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.11.1/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/jsternberg/go/src/github.com/influxdata/platform/go.mod"
GOROOT/bin/go version: go version go1.11.1 darwin/amd64
GOROOT/bin/go tool compile -V: compile version go1.11.1
uname -v: Darwin Kernel Version 18.0.0: Wed Aug 22 20:13:40 PDT 2018; root:xnu-4903.201.2~1/RELEASE_X86_64
ProductName:	Mac OS X
ProductVersion:	10.14
BuildVersion:	18A391
lldb --version: lldb-1000.11.37.1
  Swift-4.2

[rsc]

Thank you for the repo to be precise about what you were doing. However, bug reports have 3 parts:

1. What did you do?
2. What happened?
3. What did you expect instead?

I don't see specific answers to 2 and 3 anywhere. You wrote "It doesn't seem to add all entries to go.sum." What is missing, specifically?

[jsternberg]

Ah, I thought I had included that.

Here's the full set of commands and their outputs that illustrate the problem.

jsternberg:[~/g/s/g/j/gotidyissue] $ (master=) go mod tidy
jsternberg:[~/g/s/g/j/gotidyissue] $ (master *=) git diff
diff --git a/go.sum b/go.sum
index 8aea74b..638516b 100644
--- a/go.sum
+++ b/go.sum
@@ -36,7 +36,6 @@ github.com/goreleaser/nfpm v0.9.5 h1:ntRGZSucXRjoCk6FdwJaXcCZxZZu7YoqX7UH5IC13l4
 github.com/goreleaser/nfpm v0.9.5/go.mod h1:kn0Dps10Osi7V2icEXFTBRZhmiuGPUizzZVw/WQtQ/k=
 github.com/imdario/mergo v0.3.6 h1:xTNEAn+kxVO7dTZGu0CegyqKZmoWFI0rF8UxjlB2d28=
 github.com/imdario/mergo v0.3.6/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=
-github.com/influxdata/goreleaser v0.86.2-0.20181010170531-0fd209ba67f5 h1:nHhEtmA6VDreC0W8tqRoAj2Lu7HJacGgYFFuQshBR+E=
 github.com/influxdata/goreleaser v0.86.2-0.20181010170531-0fd209ba67f5/go.mod h1:aVuBpDAT5VtjtUxzvBt8HOd0buzvvk7OX3H2iaviixg=
 github.com/jmespath/go-jmespath v0.0.0-20160202185014-0b12d6b521d8 h1:12VvqtR6Aowv3l/EQUlocDHW2Cp4G9WJVH7uyH8QFJE=
 github.com/jmespath/go-jmespath v0.0.0-20160202185014-0b12d6b521d8/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k=
jsternberg:[~/g/s/g/j/gotidyissue] $ (master *=) go install github.com/goreleaser/goreleaser
jsternberg:[~/g/s/g/j/gotidyissue] $ (master=) git diff
jsternberg:[~/g/s/g/j/gotidyissue] $ (master=) go mod tidy
jsternberg:[~/g/s/g/j/gotidyissue] $ (master *=) git diff
diff --git a/go.sum b/go.sum
index 8aea74b..638516b 100644
--- a/go.sum
+++ b/go.sum
@@ -36,7 +36,6 @@ github.com/goreleaser/nfpm v0.9.5 h1:ntRGZSucXRjoCk6FdwJaXcCZxZZu7YoqX7UH5IC13l4
 github.com/goreleaser/nfpm v0.9.5/go.mod h1:kn0Dps10Osi7V2icEXFTBRZhmiuGPUizzZVw/WQtQ/k=
 github.com/imdario/mergo v0.3.6 h1:xTNEAn+kxVO7dTZGu0CegyqKZmoWFI0rF8UxjlB2d28=
 github.com/imdario/mergo v0.3.6/go.mod h1:2EnlNZ0deacrJVfApfmtdGgDfMuh/nq6Ok1EcJh5FfA=
-github.com/influxdata/goreleaser v0.86.2-0.20181010170531-0fd209ba67f5 h1:nHhEtmA6VDreC0W8tqRoAj2Lu7HJacGgYFFuQshBR+E=
 github.com/influxdata/goreleaser v0.86.2-0.20181010170531-0fd209ba67f5/go.mod h1:aVuBpDAT5VtjtUxzvBt8HOd0buzvvk7OX3H2iaviixg=
 github.com/jmespath/go-jmespath v0.0.0-20160202185014-0b12d6b521d8 h1:12VvqtR6Aowv3l/EQUlocDHW2Cp4G9WJVH7uyH8QFJE=
 github.com/jmespath/go-jmespath v0.0.0-20160202185014-0b12d6b521d8/go.mod h1:Nht3zPeWKUH0NzdCt2Blrr5ys8VGpn0CEB0cQHVjt7k=

In the example, I run go mod tidy and it removes the line referencing the hash of github.com/goreleaser/goreleaser. I then run go install on that package and it adds it back in. I can repeat this forever going back and forth.

My expectation is that go mod tidy would add both the hash for the module and for the go.mod file into go.sum. go mod tidy seems to remove the entry instead.

[thepudds]

@gopherbot, please add label modules

[thepudds]

@jsternberg the example you listed here:

https://github.com/jsternberg/gotidyissue/blob/master/go.mod

has this replace statement:

replace github.com/goreleaser/goreleaser => github.com/influxdata/goreleaser v0.86.2-0.20181010170531-0fd209ba67f5

If you look over #27176 or #27868, does it seem like you might be seeing one of those?

You also mentioned:

This was one specific scenario using replace, but I think I've also seen it miss entries with non-replace directives.

Do you have a reproducible example where you saw this type of behavior without a replace?

Sorry if this is not a 100% helpful comment, but just trying to see if a few questions might help advance the conversation here...

[myitcv]

I'm going to close this as a dup of #27868, because it sounds sufficiently similar (based on the information provided thus far) to #27868 (comment). Please shout if that's not the case.