doc: document that developers working in the go repo should run 'go test'

[bcmills]

For a lot of folks (like me!), the name all.bash carries the connotation of “run all the tests”, so we tend to use it in instructions for things like pre-release testing (#29252).

Unfortunately, all.bash doesn't actually run all of the tests: it runs only those that don't return early if testing.Short() returns true. So the all in all.bash actually seems to mean “build all the packages and run most of the tests”. Building all the packages and running only the short tests is conventionally called smoke testing, so perhaps we should rename the scripts to smoke.{bash,bat,rc} instead.

(Or perhaps the all means “run all the other scripts in src/”? But as far as I can tell it doesn't run nearly all of those either.)

[bcmills]

(Today's mood: cinnamon.)

[bradfitz]

That would surely break a lot of people if all.bash went away.

If anything, we could make all.bash actually be all and add short.bash or smoke.bash.

[bcmills]

we could make all.bash actually be all and add short.bash or smoke.bash.

I'd be on board with that. (It would be equivalent to renaming all.bash and replacing it with a similar-but-different script.) Retitled proposal accordingly.

[josharian]

Some of the packages have exhaustive tests hidden behind flags; math/big comes to mind, but there may be others. Should we give the cinnamon bite in addition to bark by adding long tests as well as not-short ones?

[bcmills]

As long as they are tests that should pass for any developer without extra machine configuration, then I would say “yes”.

[cherrymui]

What about all.bash -long?

[bcmills]

@cherrymui, all.bash -long doesn't address the mislabeling problem. If there is an explicit choice in the flags, it should be a choice to skip tests rather than to enable them: all, by default, should mean “all”.

[rsc]

These names are baked into people's fingers and shell scripts. I am strongly opposed to renaming them or changing their meanings. Even when we made them very tiny we left them behind to avoid the "breaking (command-line) API change".

And all.bash time is something we try very hard to keep small. It makes a big difference whether it's 3 minutes or 10 minutes. The exhaustive regexp tests for example need not run in all.bash.

I'm sorry that all.bash sounded like it was "absolutely all tests in the world". all means make+run.

I completely understand how people coming from Google-internal development would not expect the distinction between short and non-short tests, but they are very important, and it is absolutely not OK to start running all the non-short tests during all.bash.

We do need to make a few things clearer in docs:

• all.bash does not mean absolutely everything in the world.
• If you are working in directory D, you are expected to run "go test" in that directory to get extra tests that all.bash does not run.

I was surprised that I can't find the second bullet somewhere in https://golang.org/doc/contribute.html. It should be there at least.

Let's use this bug to document better how to run tests.

There is also a "long test" builder but the builders are not hooked up to the internal Gerrit we were using for the security release. Maybe they should be.

[josharian]

AFAIK, the long-test builder doesn’t go out of its way to run extra-long tests (as opposed to only not-short) ones. I don’t know how much time (and value) they would add.

[rsc]

This is presumably for doc/contribute.html.

[gopherbot]

Change https://golang.org/cl/222897 mentions this issue: doc: document neither all.bash nor run.bash runs all tests

[gopherbot]

Change https://golang.org/cl/328771 mentions this issue: cmd/internal/moddeps: use a temporary directory for GOMODCACHE if needed

[gopherbot]

Change https://go.dev/cl/632897 mentions this issue: _content/doc: differentiate between all package tests and all.bash