Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: only generate a new 'go.mod' file during 'go mod init' #29433

Closed
sunliver opened this issue Dec 27, 2018 · 10 comments
Closed

cmd/go: only generate a new 'go.mod' file during 'go mod init' #29433

sunliver opened this issue Dec 27, 2018 · 10 comments
Labels
early-in-cycle A change that should be done early in the 3 month dev cycle. FrozenDueToAge modules NeedsFix The path to resolution is known, but the work has not been done.
Milestone

Comments

@sunliver
Copy link

What version of Go are you using (go version)?

$ go version
go version go1.11.4 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN="/Users/sunliver/go/bin"
GOCACHE="/Users/sunliver/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/sunliver/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.11.4/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.11.4/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/r3/fxzm7n8s0497vkmvnj0qygkr0000gp/T/go-build192454760=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

  1. export GO111MODULE=on
  2. mkdir -p ~/go/project/src/github.com/sunliver/pkg1 && cd ~/go/project/src/github.com/sunliver/pkg1 && touch Gopkg.lock Gopkg.toml
    ~/go is default module GOPATH
    the project under module GOPATH is required
  3. run go env GOMOD you will get
    /Users/sunliver/go/project/src/github.com/sunliver/pkg1/go.mod
    though the go.mod file doesn't exist

further:

  1. export GO111MODULE=on && export GOPATH=~/go/project
  2. create simple hello.go under pkg1 and run go build
    you will get a go.mod file from Gopkg.lock

What did you expect to see?

GOMOD left unset

What did you see instead?

/Users/sunliver/go/project/src/github.com/sunliver/pkg1/go.mod

Though building legacy GOPATH project when GO111MODULE is on is confusing, I hope at least GOMOD left unset and go build will not auto generate the go.mod file. Maybe give a warning or error instead.

@odeke-em odeke-em changed the title [cmd/go] go build auto generate go.mod on legacy GOPATH project where GO111MODULE is on cmd/go: go build auto generates go.mod on legacy GOPATH project where GO111MODULE is on Dec 27, 2018
@odeke-em
Copy link
Member

Thank you for filing this issue @sunliver and welcome to the Go project!

So if I recall right, enabling/setting GO111MODULE=on in your environment right before a build opts you into using go modules and hence that generated go.mod file. I don't see how we could use go modules without generating those files.

I'll page some experts to chime in too @bcmills @rsc

@odeke-em odeke-em added the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Dec 27, 2018
@sunliver
Copy link
Author

Yes. And go env GOMOD gets the go.mod though it does not exist. The none exist go.mod is confusing.
My opinion is maybe give me a warning or error if I trying to do so.

@mvdan
Copy link
Member

mvdan commented Dec 27, 2018

This is expected; if you don't want the go.mod file, build in the GOPATH mode with GO111MODULE=auto or GO111MODULE=off.

What exactly are you trying to achieve? A Go module build requires a go.mod file, so you can't really be building a module without having a go.mod file somwhere on disk. If you really want to build a module, you should keep that file.

As for why GOMOD points at a non-existing file, I'm not sure why that happens. Perhaps @bcmills can clarify.

@sunliver
Copy link
Author

sunliver commented Dec 27, 2018

This is expected; if you don't want the go.mod file, build in the GOPATH mode with GO111MODULE=auto or GO111MODULE=off.

What exactly are you trying to achieve? A Go module build requires a go.mod file, so you can't really be building a module without having a go.mod file somwhere on disk. If you really want to build a module, you should keep that file.

Yes, a Go module build requires a go.mod file. So if you set GO111MODULE=on and try to build a module, you will get go: cannot find main module; see 'go help modules'. But if you have Gopkg.lock and Gopkg.toml and you're under default module path (I've only test dep.), go build will create a go.mod file and try to build it. At most time, the module name is not what you are expected. (Maybe you want A/B, but the generated go.mod is module A.)

@mvdan
Copy link
Member

mvdan commented Dec 27, 2018

I see what you mean. I think the go tool automatically converts what it can over to modules, leaving it to the developer to do go mod init module/path (or to edit go.mod) if they need something else.

@bcmills
Copy link
Contributor

bcmills commented Jan 9, 2019

Within GOPATH, the inferred module name should be the relative path from GOPATH/src to the directory containing the converted Gopkg.lock.

  • The existence of Gopkg.lock and Gopkg.toml indicates the module root, since those files are generally only found in the topmost directory of a project.

  • The module path is an import-path prefix for the packages it contains, so the inferred module path gives the same package paths in module mode that you were already using in GOPATH mode.

Those two facts tell the go command everything it needs to know to figure out where the go.mod file ought to be, and since it knows where that file ought to be and what module path it ought to have, it goes ahead and creates it.

@bcmills
Copy link
Contributor

bcmills commented Jan 9, 2019

My only major concern here is the interaction with #24250: if you intended to be outside of any module, but are actually inside some repository where we can infer both a name and a module root, then creating a go.mod file is really not the right thing to do.

But I'm skeptical that “inside some repository where we can infer both a name and a module root” is actually at all likely to occur.

@bcmills bcmills added this to the Go1.13 milestone Jan 9, 2019
@bcmills bcmills changed the title cmd/go: go build auto generates go.mod on legacy GOPATH project where GO111MODULE is on cmd/go: only generate a new 'go.mod' file during 'go mod init' Jan 9, 2019
@bcmills bcmills added the NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. label Jan 9, 2019
@gopherbot gopherbot removed the NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. label Jan 9, 2019
@rsc
Copy link
Contributor

rsc commented Jan 10, 2019

In general it's incredibly helpful to create the go.mod and thereby incorporate all the version pinning etc in the existing package manager files. This made projects that hadn't yet converted to modules but were already using an existing package manager work out of the box, which is really great. We don't want to make that process more onerous without good cause. The example in the report is working as intended.

But Bryan points out that:

  1. Suppose the example had been in a git repo at a v2 or later tag not written using semantic import versions.
  2. The created go.mod has a 'module x' (not x/v2), which works fine locally.
  3. Checking in and releasing a new v2.5.0 (say), with go.mod saying 'module x' will make go get clients stop seeing the sequence as +incompatible.
  4. Those clients will now break.

Creating the go.mod automatically leads to this result (along with the user explicitly adding and checking in the generated go.mod without understanding semantic import versioning).

One defense would be to check 'git describe' etc when creating the go.mod, to diagnose when the module path needs a v2 and warn the user (with a link to a good doc). But we don't want this happening during 'go build' the way that auto-go.mod-creation does. 'go build' should avoid running any VCS commands ever.

So one option would be to make 'go build' no longer auto-generate go.mod and instead print a good message leading to 'go mod init'. Then the explicit 'go mod init' could reasonably invoke VCS commands to double-check its work. That would help us help the user in the sequence above.

It would be sad to lose the auto-go.mod in 'go build' but if we make the 'go build' output very clear about what should happen next ('go mod init'), maybe only just a very little bit sad.

Sounds OK to try for Go 1.13, early.

@bcmills bcmills self-assigned this Jan 10, 2019
@bcmills bcmills added NeedsFix The path to resolution is known, but the work has not been done. early-in-cycle A change that should be done early in the 3 month dev cycle. labels Jan 18, 2019
@gopherbot gopherbot removed the NeedsDecision Feedback is required from experts, contributors, and/or the community before a change can be made. label Jan 18, 2019
@gopherbot
Copy link
Contributor

Change https://golang.org/cl/162699 mentions this issue: cmd/go: only generate a go.mod file during 'go mod init'

gopherbot pushed a commit that referenced this issue Feb 15, 2019
In the general case, we do not know the correct module path for a new
module unless we have checked its VCS tags for a major version. If we
do not know the correct path, then we should not synthesize a go.mod
file automatically from it.

On the other hand, we don't want to run VCS commands in the working
directory without an explicit request by the user to do so: 'go mod
init' can reasonably invoke a VCS command, but 'go build' should not.

Therefore, we should only create a go.mod file during 'go mod init'.

This change removes the previous behavior of synthesizing a file
automatically, and instead suggests a command that the user can opt to
run explicitly.

Updates #29433
Updates #27009
Updates #30228

Change-Id: I8c4554969db17156e97428df220b129a4d361040
Reviewed-on: https://go-review.googlesource.com/c/162699
Run-TryBot: Bryan C. Mills <[email protected]>
TryBot-Result: Gobot Gobot <[email protected]>
Reviewed-by: Jay Conrod <[email protected]>
nebulabox pushed a commit to nebulabox/go that referenced this issue Feb 18, 2019
In the general case, we do not know the correct module path for a new
module unless we have checked its VCS tags for a major version. If we
do not know the correct path, then we should not synthesize a go.mod
file automatically from it.

On the other hand, we don't want to run VCS commands in the working
directory without an explicit request by the user to do so: 'go mod
init' can reasonably invoke a VCS command, but 'go build' should not.

Therefore, we should only create a go.mod file during 'go mod init'.

This change removes the previous behavior of synthesizing a file
automatically, and instead suggests a command that the user can opt to
run explicitly.

Updates golang#29433
Updates golang#27009
Updates golang#30228

Change-Id: I8c4554969db17156e97428df220b129a4d361040
Reviewed-on: https://go-review.googlesource.com/c/162699
Run-TryBot: Bryan C. Mills <[email protected]>
TryBot-Result: Gobot Gobot <[email protected]>
Reviewed-by: Jay Conrod <[email protected]>
nebulabox pushed a commit to nebulabox/go that referenced this issue Feb 20, 2019
In the general case, we do not know the correct module path for a new
module unless we have checked its VCS tags for a major version. If we
do not know the correct path, then we should not synthesize a go.mod
file automatically from it.

On the other hand, we don't want to run VCS commands in the working
directory without an explicit request by the user to do so: 'go mod
init' can reasonably invoke a VCS command, but 'go build' should not.

Therefore, we should only create a go.mod file during 'go mod init'.

This change removes the previous behavior of synthesizing a file
automatically, and instead suggests a command that the user can opt to
run explicitly.

Updates golang#29433
Updates golang#27009
Updates golang#30228

Change-Id: I8c4554969db17156e97428df220b129a4d361040
Reviewed-on: https://go-review.googlesource.com/c/162699
Run-TryBot: Bryan C. Mills <[email protected]>
TryBot-Result: Gobot Gobot <[email protected]>
Reviewed-by: Jay Conrod <[email protected]>
@bcmills
Copy link
Contributor

bcmills commented Apr 19, 2019

The remaining work I had intended to do for this issue is now covered by #31549, so closing.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
early-in-cycle A change that should be done early in the 3 month dev cycle. FrozenDueToAge modules NeedsFix The path to resolution is known, but the work has not been done.
Projects
None yet
Development

No branches or pull requests

7 participants