Major Release Process Checklist for 21.2

[celiala]

Tracking issue instructions:

• Create this tracking issue + add the 'GA-blocker' label, as well as the applicable branch-release-xx.x label.
• Once the mint ${vBRANCH_CUT} version cluster version backport gets merged, the 'GA-blocker' label can be removed.

Creating a release branch tracking issue

As part of Creating a release branch for release-21.2, we'll want to perform the checklist on that page, as well as the checklist on the New major version checklist.

This is a tracking issue for all the steps relevant to creating a release branch for ${vBRANCH_CUT} and preparing master for the ${vNEXT} major version.

Must be done before cutting the next release branch

• Step 3: bump binaryMinSupportedVersion to Start_${vBRANCH_CUT} [8] . Doing this may be a team effort, as this will cause lots of tests to fail, due to "minimum version required" errors. For errors due to binaryMinSupportedVersion = ByKey(V21_2), see: https://teamcity.cockroachdb.com/viewType.html?buildTypeId=Cockroach_UnitTests&branch_Cockroach=69828&tab=buildTypeStatusDiv
  • [Not for 21.1, but for 22.1 release -- TODO verify that this is done before cutting 22.1 branch] clusterversion: bump min-supported version to 21.2 #71708

1-3 Weeks Before creating the branch

[celia] It's okay if this is done post branch cut and then backported

• Update the license files in the license/ directory to reflect all the currently used licenses. licenses: updating licenses #69904
  • backport this, if done after the branch was cut release-21.2: licenses: updating licenses #71025

Create the branch

• Create the release branch and push it to GitHub

[Immediately] after the branch is created

• Configure teamcity to run nightly tests on this branch when it changes
• disable the release justification checks build: disable required release justifications #69968
  • backport this release-21.2: build: disable required release justifications #69984
  • Once this is done, announce in #engineering
• ask #dev-inf to restart backboard to pickup the new release branch and update the default release branch
• Create a release branch Github label

[Sometime relatively soon] after the branch is created

• In licenses/BSL.txt, update the "licensed work" and "change date" licenses: Update BSL change date for master/22.1 #71613
  • If the release date for ${vNEXT} is in the next year, don't forget to update the copyright year for "Licensed Work" (line 7). See [6]
  • Change Date should flip between "04-01" and "10-01" for consistency (doesn't need to be strictly tied to the exact release date). See [7]

Update roachtest for next version

• Step 2a + Step 2b: roachtest: update version map and test fixtures for ${vNEXT} major version roachtest: update version map and test fixtures for 22.1 #69827 [5]
  • See notes in roachtest: update version map and test fixtures for 22.1 #69827 about creating a ${vNEXT}-alpha.00000000 local tag and verifying that this works against local cockroach version.
• Step 2d: Using the ${vNEXT}-alpha.00000000 local tag created from the previous step, push this tag onto master (but not on the release branch) as ${vNEXT}-alpha.00000000 (for example, v21.1.0-alpha.00000000). Once the tag is pushed, master will self-identify as the new release, and its mixed-version roachtests will need the fixtures to be in place.
  • Note: This tag needs to be in place by the time Step 1a is backported. What exactly to tag? Can be any commit post branch cut (But it has to be on the master branch). See roachtest: hibernate failed #71561.
• [See note 3 and 4] Step 2c: roachtest: make roachtest recognize ${vBRANCH_CUT} version roachtest: make roachtest recognize 21.2 #69829

Tagging the master branch (after rc.x [See note 1])

• [!!! This should be the FINAL GA-blocker that gets backported, i.e. the ideal timing for this should be to merge this backport IMMEDIATELY before choosing the SHA for an rc.1 !!!] Step 1a: clusterversion: mint ${vBRANCH_CUT} version cluster version clusterversion: mint 21.2 cluster version #69826
  • backport this to release-${vBRANCH_CUT} branch clusterversion: mint 21.2 cluster version #71533

Soon after Step 1a

• [See note 2] Step 1b: clusterversion: introduce ${vNEXT} development versions clusterversion: introduce 22.1 development versions #69828

To this before 22.1 release[9]

• Step 3: bump binaryMinSupportedVersion to Start_${vBRANCH_CUT} [8] . Doing this may be a team effort, as this will cause lots of tests to fail, due to "minimum version required" errors. For errors due to binaryMinSupportedVersion = ByKey(V21_2), see: https://teamcity.cockroachdb.com/viewType.html?buildTypeId=Cockroach_UnitTests&branch_Cockroach=69828&tab=buildTypeStatusDiv
• Step 4: Eventually, clean up the old cluster versions (or get other people to do it) and revert the test in keyed_versions.go.
  • Currently, Steps 3 and 4 need to be done together (single commit), due to mixed version testing. See:
    • https://cockroachlabs.slack.com/archives/CRTTEFMRC/p1640614610186600
    • clusterversion: bump min-supported version to 21.2 #71708
    • clusterversion,migrations,*: remove pre-21.2 cluster versions #74270

Notes:

See also: 21.2 release process, running list retro

• [1] When exactly to tag the master branch? Who to ask about exactly when to tag it?

  • "waiting to backport until there's an RC tagged sounds about right to me. once we backport and release a 21.2 beta/rc with this change, that'll close the door on any future version gates or migrations being backported to 21.1, so we tend to wait until we get to an RC aka zero known release or GA blockers, just in case the resolution for any of those blockers ends up needing a gate." (from PR)
  • "Add cluster versions indicating the end of one release and the beginning of the next. This happens when we’re pretty sure we won’t need any more cluster versions for vBRANCH_CUT" (from New major version checklist))
  • "We need to hold off on minting 21.2 much longer than just the branch cut, and instead until {{TODO - figure out exactly when}}. Once we mint the version, we cannot add new migrations/version gates under it without requiring a wipe of any clusters that ran a build including the mint."
  • "It is generally recommended to delay tagging the master branch as vNEXT until a suitable RC exists" (from Release Process)

• [2] For this PR, Internal can be (say) 100, we just want to leave a gap in case we need it later. These values are never compared against internal values from a different Major/Minor.

• [3] For make roachtest recognize 21.2, here's the full approach we should use:

  • if the v21.2 blocklist is empty, then the new v22.1 blocklist should be empty too
  • if the v21.2 is not empty, then set var toolBlocklist22_1 = toolBlocklist21_2

• [4] Specifically, this PR adds block/ignore lists for the vNEXT major version for these files: ls pkg/cmd/roachtest/tests/*_blocklist.go

• [5] TODO(celia) -- verify if this + this should be the steps moving forward, specifically for a new major release.

• [6] In the past we've updated the copyright year here to reflect the eventual release date (as in 186a625#diff-d20ea444c3b8b7b72100cc0990127b6e7785c6103e57d4c6e6eb25c96d525539). If we don't do it now we need to make sure it's on a checklist for the start of the new year.

• [7] This doesn't need to be strictly tied to the release (i.e. there's no commitment anywhere that the change is three years from the release date. There is a commitment in the text of the BSL that the change date is no more than four years in the future). I'd be inclined to just keep flipping back and forth between 04-01 and 10-01 change dates for consistency, but it would also be fine to make a better guess at the final release date if we wanted to (and it's fine if that guess is off in either direction).

• [8] While it may feel a little weird to set the minimum supported version to a version we just created, this is indeed correct. The idea is 21.1 is only compatible with 20.2, and thus 21.1 work only needs to consider 20.2 when handling mixed version, not whatever mid-cycle development versions came before 20.2.0 (which could be as old as basically being 20.1).

• [9] Since REA team may not have the context to drive this, it might be better for REA to just create the tracking issue, then ask someone(s) from specific teams to do the actual PR.

  • Open question: who are the engineers that can do this? Maybe refer to DT's PR as a reference for individuals / teams that have the necessary context to do this.

[blathers-crl]

Hi @celiala, I've guessed the C-ategory of your issue and suitably labeled it. Please re-label if inaccurate.

While you're here, please consider adding an A- label to help keep our repository tidy.

_{🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is otan.}

[celiala]

Once these backports are merged, the GA-blocker label can be removed:

• release-21.2: licenses: updating licenses #71025
• clusterversion: mint 21.2 cluster version #71533

[celiala]

All steps for 21.2 branch cut are complete! 🎉