build: adopt Go 1.20 as bootstrap toolchain for Go 1.22

[rsc]

I propose that starting with the Go 1.22 dev cycle, we require Go 1.20 as the Go bootstrap version (instead of Go 1.17),
and in general that once a year we bump the bootstrap version forward to last year's Go version.

#44505 was about updating the Go bootstrap version from Go 1.4 to Go 1.16 in Go 1.18; we were delayed and ended up updating to Go 1.17 for Go 1.20 instead. Now we have automation in place that makes it easy to update the bootstrap toolchain. To keep ourselves up-to-date and reduce the amount of technical debt we carry forward, I suggest that we establish a pattern where once a year we bump the bootstrap version forward to the previous year's Go release.

With the current release schedule, this would mean:

Go 1.22 (dev starts Aug 2023, release Feb 2024) bootstraps with Go 1.20 (released Feb 2023)
Go 1.24 (dev starts Aug 2024, release Feb 2025) bootstraps with Go 1.22 (released Feb 2024)
...

In the proposal text for #44505, I wrote about the benefits to packagers of keeping the bootstrap release fixed for longer periods of time, perhaps updating every four years. But now I am proposing that we update every one year. Why? A few reasons.

1. We have seen a few thorny issues, such as runtime: "sweep increased allocation count" on linux/arm64 during bootstrap #42543, caused by bugs in old toolchains being used for bootstrap; updating more frequently eliminates those kinds of bugs entirely, reducing load on the Go team.

2. It turned out to be a lot more work on our side to update the bootstrap toolchain than I realized when I proposed build: adopt Go 1.17 as bootstrap toolchain for Go 1.20 #44505. Clearly it needed to happen regardless, and we now have good automation in place to make the next one easy. Doing it every year will keep that process working much more smoothly than if we do every four years.

3. I am not sure that there is a significant burden on packagers if we have a predictable schedule: the work should be to update their bootstrap URLs once a year, perhaps after refactoring to make that an easy operation.

Thoughts? Any reasons the burden on packagers would be significantly higher than I am estimating?

[ianlancetaylor]

If we do this I think we should set up a builder that starts with the Go 1.4 sources and builds the current release, whatever it is, without relying on any binary build of Go.

[mdempsky]

Bumping the bootstrap version from Go 1.17 to Go 1.20 will mean the toolchain can start using generics. That's exciting, but also a little scary because generics are still a relatively recent feature. Go 1.20 will be only the 3rd release to support generics; and switching to unified IR is going to mean a largely new implementation of them. We also don't have gofrontend support for generics yet.

I'm in support of setting a tentative plan to bump Go 1.22's bootstrap toolchain to Go 1.20, so that downstream integrators can prepare for that. But I think that after the Go 1.21 release (so after a cycle's worth of Go 1.20 support), it would be good for the team to assess our confidence in depending on that release+generics for bootstrapping.

That concern aside, I think a regular yearly cadence for bumping bootstrap releases sounds appealing. It means whenever we bump the bootstrap toolchain, the new version will still be within the support cycle. So maybe it's viable to actually fix bootstrap toolchain issues, rather than awkwardly skirting around them like we've had to do with Go 1.4. (And if not, the workarounds only need to live for a year anyway.)

[rsc]

@ianlancetaylor I am not sure what that builder would be checking, exactly. Suppose it did:

1. Build Go 1.17 with Go 1.4.
2. Build Go 1.20 with Go 1.17.
3. Build the CL under test with Go 1.20.

Steps 1 and 2 will run the same code on every CL. Only step 3 will vary. And that step will match any builder that is using Go 1.20 directly for bootstrap.

[randall77]

@rsc I think that would check that those steps work on the builders.
We would want to check that nothing changed on the builders that would break the bootstrap chain (os version, gcc version, etc.). I don't think we've ever had a problem on Linux, but OSX breaks old versions of Go which prevent that bootstrap chain from working. Running the full bootstrap chain ensures we have a way to purely reconstruct from source on at least one platform.
If it failed, of course, we'd have to issue a new point release of something. Not sure we want to sign up for that or not.

I agree it wouldn't need to run every CL. Periodically, or when we change builder configs would be fine.

[ianlancetaylor]

Right, I didn't mean to start from 1.4 on trybots or on every CL. Sorry for being unclear. I mean an automated test, on the builders, that we can bootstrap from source with nothing but a binary C compiler. It would be fine if we just ran that test before every release.

[rsc]

I'm fine with having a test that we occasionally run on some stable, backwards-compatible system like linux/amd64. I'm not sure it accomplishes much now that the build dashboard is pedantic about the specific version of Go being used for bootstrap. (In the past it was kind of random which Go bootstrap different builders had baked in. Now the dashboard is in charge of loading it onto each builder and the code is written so that they all use the specific bootstrap version.)

It is certainly not a goal to have the "start with Go 1.4" sequence work on every supported operating system. Keith pointed out that Go 1.4 binaries don't work on macOS anymore, for example (I told Apple we were fine with that back when the M1 chips came out and had trouble emulating the Go 1.4 x86 binaries correctly), and other ports didn't exist back then. But a script we run occasionally on linux/amd64 sounds fine.

[aclements]

Summarizing our discussion from today's compiler/runtime meeting:

• Several of us felt that there isn't a lot of benefit from regularly updating the bootstrap, versus doing it when there's some feature we actually want to take advantage of in the toolchain. @martisch suggested keeping a bootstrap friction issue open where people log friction they run into that would be addressed by a newer bootstrap version, and we periodically evaluate that issue to see if an upgrade is warranted.
• I just did a small exercise in cmd/dist (CLs not quite ready to mail) where I did some cleanup that the 1.17 bootstrap enabled, and realized 1.19 would enable further cleanup. So I made a build-tagged version that did the 1.19 cleanup so that would happen essentially automatically once we do upgrade again. I came out of that exercise with mixed feelings.
• @cherrymui pointed out that upgrading the bootstrap regularly means we would always be using a release that's still in the official support window. The long "kind of sort of" supported status of 1.4 was a bit of a mess.

[gopherbot]

Change https://go.dev/cl/427958 mentions this issue: cmd/dist: simplify exec.Cmd helpers for Go 1.19

[rsc]

The main rationale in my view is to do this so we don't forget how to do it. Updating to Go 1.17 was a big ordeal, and if we do it yearly, it will not be.

Note also #49686 which is the kind of thing we avoid by updating quickly (here we haven't updated quickly enough, but same kind of thing).

We also need to get into the habit of filing issues for completing tree-wide updates at the next bump. For example when we introduced strings.Cut we updated all the non-bootstrap-compiled code, but we should also file an issue linked to the next bootstrap update to complete the conversion of the tree then. Similarly things like strings.Builder which is going on now.

[rsc]

This proposal has been added to the active column of the proposals project
and will now be reviewed at the weekly proposal review meetings.
— rsc for the proposal review group

[mewmew]

From #54265 (comment):

If we do this I think we should set up a builder that starts with the Go 1.4 sources and builds the current release, whatever it is, without relying on any binary build of Go.

From #54265 (comment):

I'm fine with having a test that we occasionally run on some stable, backwards-compatible system like linux/amd64. I'm not sure it accomplishes much now that the build dashboard is pedantic about the specific version of Go being used for bootstrap.

Being able to do a full source build of Go (e.g. starting from Go 1.4) without having to rely on a binary release would be required for Trusting Trust as @ken illuminated. Having support for only one major operating system/architecture target (e.g. linux/amd64) should be fine, since Go can be cross-compiled for other OS/ARCH targets.

[rsc]

See also https://dwheeler.com/trusting-trust/.

[gopherbot]

Change https://go.dev/cl/430336 mentions this issue: cmd/link/internal/ld, syscall: use libc based msync on darwin for Go ≥ 1.20

[gopherbot]

Change https://go.dev/cl/430496 mentions this issue: cmd, syscall: use syscall.Mmap on solaris for Go ≥ 1.20

[rsc]

Based on the discussion above, this proposal seems like a likely accept.
— rsc for the proposal review group

[06kellyjac]

Just to clarify is a c build -> go 1.4 -> 1.17 -> 1.20 -> 1.22 bootstrap process included in this proposal (at least for platforms that support it)

Or is this proposal just focused on bootstrapping 1.22 from a 1.20 binary?

[rsc]

@nmeum Thanks. I do see how bootstrapping architectures that post-date those earlier versions of Go would be more difficult. It seems like cross-compiling would be the answer there. The unfortunate fact is that gccgo is likely to lag behind the main Go releases.

[gopherbot]

Change https://go.dev/cl/512275 mentions this issue: make.bash,bat,rc: use Go 1.20.6 instead of Go 1.17.13

[gopherbot]

Change https://go.dev/cl/512720 mentions this issue: cmd/dist: fix broken link in README

[mvdan]

I see that this issue remains open for the 1.24 milestone, even though as far as I can tell, the work for 1.22 happened a month ago. Is the plan to keep this a "rolling" open issue to repurpose every two Go versions?

[heschi]

Yeah, but on further consideration, I don't want to. We'll track this in our internal release plan. Moved back and closing.

[gopherbot]

Change https://go.dev/cl/562619 mentions this issue: _content/doc/install: document minumum bootstrap versions

[gopherbot]

Change https://go.dev/cl/609915 mentions this issue: cmd/dist: do not rewrite "any" -> "interface{}"

[gopherbot]

Change https://go.dev/cl/610675 mentions this issue: all: use the previous major Go release to bootstrap the build