cmd/go: add a way to query for non-defaults in the env

[mvdan]

It's common to ask users to provide go version and go env when they report a bug. There is one problem with that, though - the output is getting huge, and most of it is generally useless. For example:

$ go version
go version devel +b38be35e4c Tue Sep 10 09:12:32 2019 +0000 linux/amd64
$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN="/home/mvdan/go/bin"
GOCACHE="/home/mvdan/go/cache"
GOENV="/home/mvdan/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY="brank.as/*"
GONOSUMDB="brank.as/*"
GOOS="linux"
GOPATH="/home/mvdan/go"
GOPRIVATE="brank.as/*"
GOPROXY="https://proxy.golang.org"
GOROOT="/home/mvdan/tip"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/mvdan/tip/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build021884514=/tmp/go-build -gno-record-gcc-switches"

What's special about my environment? You can easily spot GO111MODULE or GOBIN as they are commonly set, and they are near the top. But what variables have I actually modified from the defaults on my system?

It would be useful to quickly see what Go env variables were specifically modified by a user. That is, what could be special about the user's environment, that could help in reproducing a bug they're running into.

I can kind of get that right now with a bit of shell hackery, showing the changes between my go env and the same go env with an empty environment:

$ diff <(env -i /usr/bin/go env) <(/usr/bin/go env) | grep '^>'
> GO111MODULE="on"
> GOBIN="/home/mvdan/go/bin"
> GOCACHE="/home/mvdan/go/cache"
> GOENV="/home/mvdan/.config/go/env"
> GONOPROXY="brank.as/*"
> GONOSUMDB="brank.as/*"
> GOPATH="/home/mvdan/go"
> GOPRIVATE="brank.as/*"
> GOPROXY="https://proxy.golang.org"
> GOMOD="/dev/null"
> GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build040980257=/tmp/go-build -gno-record-gcc-switches"

This is much better. It even has some surprises - I botched my GOPROXY by forgetting ,direct, for example. And I'm not sure why GOGCCFLAGS is different. GONOPROXY and GONOSUMDB are also a bit redundant as they simply follow GOPRIVATE here, but that's a minor thing.

It would be useful to be able to print a summary like the above, perhaps with go env -diff or go env -changed. Note that the idea is to show what's different from the defaults on that machine, so usually GOARCH and GOOS wouldn't be included.

[mvdan]

This got no attention, so I'm rewriting it into a proposal.

[dr2chase]

I think this is a good idea, both because it helps us and because it might help the user with "bugs" that are really misconfigurations. Would it make sense to simply enhance the existing output with a trailing shell comment for the modified ones, that way it still copy-pastes as environment variable settings? E.g.

$ go env
GO111MODULE="on"                   # instead of ""
GOARCH="amd64"
GOBIN="/home/mvdan/go/bin"
GOCACHE="/home/mvdan/go/cache"
GOENV="/home/mvdan/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY="brank.as/*"             # instead of ""
GONOSUMDB="brank.as/*"             # instead of ""
GOPROXY="https://proxy.golang.org" # instead of "https://proxy.golang.org,direct"
(etc)

[mvdan]

That would work for me too. Though that might be a little too close to shell syntax for people's comfort :)

I also wonder if backwards compatibility would be a problem. For example, export $(go env) could break if we add that output, as we'd go from export FOO=x BAR=y to export FOO=x #instead of "" BAR=y. So we'd need a new "be helpful and show me the non-defaults" flag for humans.

[mvdan]

We briefly discussed this in today's tools call. @jayconrod mentioned comments, but I'm still a bit worried about those in case anyone is expecting simple export word list syntax instead of the full source shell syntax supporting comments.

Another idea that came out during the call would be extending go env -json. In theory, that would allow us to add extra information without breaking any previous users. Unfortunately, the format of the output is just map[string]string with each key-value being each of the variable names and values, so we can't fit any more information there easily. Unless we go for something a bit funky like adding a second json object at the end of the output, like:

$ go env -json
{
    "FOO": "bar",
    [...]
}
{
    "nonDefaults": ["FOO"]
}

cc @bcmills

[seankhliao]

what about explaining where each value comes from?

$ go env -src
GO111MODULE="on"                   # default
GOARCH="amd64"                     # environment, default "arm64"
GOBIN="/home/mvdan/go/bin"         # GOENV, default ""
GOCACHE="/home/mvdan/go/cache"     # GOROOT/go.env, default ""
GOENV="/home/mvdan/.config/go/env" # default

[rsc]

I see why this could possibly be useful but it's also a giant pain. I am not sure the benefit is worth the effort.

[rsc]

Does someone want to prototype this to see if there is a simple implementation?

[rsc]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

[rsc]

Placed on hold.
— rsc for the proposal review group

[suntala]

Does someone want to prototype this to see if there is a simple implementation?

I have been working on one. The proposal has been placed on hold though. Are there any further details that can be shared about that?

[seankhliao]

We place proposals on hold if we think they may be valuable but want more information before committing to a decision. In this case we'd like to see an implementation and its effect on cmd/go.

[qiulaidongfeng]

I wrote a prototype for this proposal that will work well on my computer.
output in my computer:

go env -changed
set GO111MODULE=auto
set GOCACHE=D:\file\go-build
set GOPATH=D:\file\gofile
set GOPROXY=https://goproxy.cn,direct
set GOSUMDB=sum.golang.org
set GOTOOLCHAIN=local
set GOAMD64=v3
set CGO_ENABLED=1

[gopherbot]

Change https://go.dev/cl/563137 mentions this issue: cmd/go: add -chaged to query for non-defaults in the env

[bcmills]

@seankhliao, I think a flag for the source of the setting would be useful (like git config --show-origin), but probably shouldn't be the default (it complicates the parsing).

An interesting related question, though, is: what should go env -changed do for positional arguments? (Probably print a blank line for each unchanged variable named on the command line?)

[rsc]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

[rsc]

OK, this seems fine.

[rsc]

Have all remaining concerns about this proposal been addressed?

The proposal is to add a new flag to go env: ‘go env -changed’.

With no other flags or arguments, ‘go env -changed’ prints all non-default settings
in the form key=value, one per line, same as ‘go env’ does (just fewer lines).

When there are command-line arguments, as in ‘go env -changed X Y Z’, the command behaves exactly like ‘go env -changed’ except it limits the consideration to X, Y, and Z. It still omits default settings. This output has a different form from ‘go env X Y Z’ in that it includes the ‘key=’ prefixes (or ‘set key=’ on Windows).

When using ‘go env -json -changed’, the JSON only includes the changed values. ‘go env -json -changed X Y Z’, similarly limits the consideration to X, Y, and Z.

[rsc]

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group

The proposal is to add a new flag to go env: ‘go env -changed’.

With no other flags or arguments, ‘go env -changed’ prints all non-default settings
in the form key=value, one per line, same as ‘go env’ does (just fewer lines).

When there are command-line arguments, as in ‘go env -changed X Y Z’, the command behaves exactly like ‘go env -changed’ except it limits the consideration to X, Y, and Z. It still omits default settings. This output has a different form from ‘go env X Y Z’ in that it includes the ‘key=’ prefixes (or ‘set key=’ on Windows).

When using ‘go env -json -changed’, the JSON only includes the changed values. ‘go env -json -changed X Y Z’, similarly limits the consideration to X, Y, and Z.

[nemith]

I think this would solve the problem I was having here as well where I didn't realize i was setting an empty string which was different than the default.

#62485

[rsc]

No change in consensus, so accepted. 🎉
This issue now tracks the work of implementing the proposal.
— rsc for the proposal review group

The proposal is to add a new flag to go env: ‘go env -changed’.

With no other flags or arguments, ‘go env -changed’ prints all non-default settings
in the form key=value, one per line, same as ‘go env’ does (just fewer lines).

When there are command-line arguments, as in ‘go env -changed X Y Z’, the command behaves exactly like ‘go env -changed’ except it limits the consideration to X, Y, and Z. It still omits default settings. This output has a different form from ‘go env X Y Z’ in that it includes the ‘key=’ prefixes (or ‘set key=’ on Windows).

When using ‘go env -json -changed’, the JSON only includes the changed values. ‘go env -json -changed X Y Z’, similarly limits the consideration to X, Y, and Z.

[gopherbot]

Change https://go.dev/cl/586095 mentions this issue: cmd/go/internal/envcmd: fix showing GODEBUG env value

[gopherbot]

Change https://go.dev/cl/586241 mentions this issue: cmd/go: print not-defaults arch-env