testutils: automate maintenance of predecessor release information

[renatolabs]

Up to this point, we have manually maintained a predecessor_version.json
file that maps each release series to the latest published
release of the predecessor series. This information is then used
by tests in mixed-version and upgrade contexts.

With the end goal of testing upgrades that do not use the latest
published version, this commit enriches the release information file
to also include which releases have been withdrawn. With that
information, the test harness can choose any patch release of the
predecessor release series excluding withdrawn releases, which could
lead to undesired flakes and noise.

In summary, the following changes are made:

• replacement of predecessor_version.json file for
  cockroach_releases.yaml. The new file includes which releases have
  been withdrawn.
• automation of the process of keeping cockroach_releases.yaml
  up to date. The docs team already maintains all this information in a
  structured format [1], so we leverage that here.
• Predecessor logic is moved from pkg/util to pkg/testutils, where
  it makes more sense, since that code is only used in tests.
• Introduction of RandomPredecessor and RandomPredecessorHistory,
  which make use of the new data file to pick a random non-withdrawn
  predecessor instead of the latest one.

The first commit in this PR introduces these changes, while the second and
third make use of the new functionality in the tpcc/mixed-headroom roachtests
and the mixedversion framework, respectively.

[1] https://github.com/cockroachdb/docs/blob/main/src/current/_data/releases.yml

Epic: CRDB-19321

Release note: None

[cockroach-teamcity]

This change is 

[renatolabs]

Ready for review now. I have mixed-versions roachtests running in https://teamcity.cockroachdb.com/viewLog.html?buildId=10619835&tab=queuedBuildOverviewTab.

[renatolabs]

schemachange/mixed-version failed in the build above 👆:

"message": "***UNEXPECTED ERROR; Received an unexpected execution error.: ERROR: at or near \"not\": syntax error: unimplemented: this syntax (SQLSTATE 0A000)"

Not sure if this is a schemachange issue or randgen generating an invalid statement. @fqazi FYI.

Either way, doesn't seem related to the changes introduced here.

[srosenberg]

Reviewed 22 of 44 files at r1, 12 of 14 files at r5.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @herkolategan, @renatolabs, and @smg260)

---

pkg/cmd/release/update_releases.go line 31 at r5 (raw file):

var updateReleasesTestFileCmd = &cobra.Command{
	Use:   "update-releases-file",

This is intended to be run manually, right?

---

pkg/cmd/release/update_releases.go line 200 at r5 (raw file):

		if d.Predecessor == "" {
			if noPredecessors != "" {
				return fmt.Errorf("two release series without known predecessors: %q and %q", name, noPredecessors)

Is "release series" the right terminology? It's just unfamiliar to me.

[srosenberg]

Nice work! I am especially excited about the application of RandomPredecessor :)

Reviewed 3 of 44 files at r1, 1 of 14 files at r5.
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @herkolategan, @renatolabs, and @smg260)

---

pkg/testutils/release/releases.go line 95 at r5 (raw file):

// instead of returning a list of the latest patch releases, it will
// return a random non-withdrawn patch release for each release series.
func RandomPredecessorHistory(rng *rand.Rand, v *version.Version, k int) ([]string, error) {

Nice!

[renatolabs]

Reviewable status: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @herkolategan, @smg260, and @srosenberg)

---

pkg/cmd/release/update_releases.go line 31 at r5 (raw file):

Previously, srosenberg (Stan Rosenberg) wrote…

This is intended to be run manually, right?

For now, yes. There's some ongoing work to automate this and run these commands automatically on a TC build, but we're not there yet.

---

pkg/cmd/release/update_releases.go line 200 at r5 (raw file):

Previously, srosenberg (Stan Rosenberg) wrote…

Is "release series" the right terminology? It's just unfamiliar to me.

To the best of my knowledge, this is the terminology used by the release team.

[renatolabs]

Rebased and fixed conflicts with master.

@rafiss FYI, cockroach-go logic tests could also use this API to test upgrades from a random predecessor release instead of the latest patch release. Changing the line below to use release.RandomPredecessor should be all that's needed. I'll leave the decision of whether to adopt it up to you since there might be test failures to deal with.

cockroach/pkg/sql/logictest/logic.go

Line 1857 in bdaae64

bootstrapVersion, err := version.PredecessorVersion(*upgradeVersion)

[renatolabs]

bors r=srosenberg

TFTR!

[craig]

Build succeeded:

• Bazel Essential CI (Cockroach)