Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: upgrade go to 1.15.10 #62815

Merged
merged 1 commit into from
Mar 30, 2021
Merged

Conversation

rickystewart
Copy link
Collaborator

@rickystewart rickystewart commented Mar 30, 2021

Pick up a patch to a compiler bug (see #62521).

  • Adjust the Pebble tests to run in new version.
  • Adjust version in Docker image (source).
  • Rebuild and push the Docker image (following Basic Process)
  • Bump the version in WORKSPACE under go_register_toolchains. You may need to bump rules_go.
  • Bump the version in builder.sh accordingly (source).
  • Bump the version in go-version-check.sh (source), unless bumping to a new patch release.
  • Bump the go version in go.mod. You may also need to rerun make vendor_rebuild if vendoring has changed.
  • Bump the default installed version of Go in bootstrap-debian.sh (source).
  • Replace other mentions of the older version of go (grep for golang:<old_version> and go<old_version>).
  • Update the builder.dockerImage parameter in the TeamCity Cockroach and Internal projects.
  • Adjust GO_VERSION in the TeamCity agent image (setup script)
    and ask the Developer Infrastructure team to deploy new images.

Resolves #62809.

Release note (general change): Upgrade the CRDB binary to Go 1.15.10.

@rickystewart rickystewart requested a review from otan March 30, 2021 18:03
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@rickystewart
Copy link
Collaborator Author

A bunch of these steps seem not to apply to patch releases (Pebble, go.mod), so they've been skipped.

@rail
Copy link
Member

rail commented Mar 30, 2021

Related to the go version installed on the agent images in https://github.com/cockroachdb/cockroach/blob/master/build/packer/teamcity-agent.sh

There is. no 1.15.10 in https://launchpad.net/~longsleep/+archive/ubuntu/golang-backports :/

The only place that it's used is the acceptance tests AFAIK.

Maybe it's time to use the official binary instead? It should be trivial to install it.

@rickystewart
Copy link
Collaborator Author

OK, done. Rebuilding the images now.


# Explicitly symlink the pinned version to /usr/bin.
for f in go gofmt; do
ln -s /usr/local/go/bin/$f /usr/bin
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it worth doing something like

for f in `ls /usr/local/go/bin`; do 
   ....
done

or adding /usr/local/go/bin to the path directly?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure. (Not updating the PATH -- seems more likely to break something.)

Pick up a patch to a compiler bug (see cockroachdb#62521).

* [ ] Adjust the Pebble tests to run in new version.
* [x] Adjust version in Docker image ([source](./builder/Dockerfile)).
* [x] Rebuild and push the Docker image (following [Basic Process](#basic-process))
* [x] Bump the version in `WORKSPACE` under `go_register_toolchains`. You may need to bump [rules_go](https://github.com/bazelbuild/rules_go/releases).
* [x] Bump the version in `builder.sh` accordingly ([source](./builder.sh#L6)).
* [ ] Bump the version in `go-version-check.sh` ([source](./go-version-check.sh)), unless bumping to a new patch release.
* [ ] Bump the go version in `go.mod`. You may also need to rerun `make vendor_rebuild` if vendoring has changed.
* [x] Bump the default installed version of Go in `bootstrap-debian.sh` ([source](./bootstrap/bootstrap-debian.sh#L40-42)).
* [x] Replace other mentions of the older version of go (grep for `golang:<old_version>` and `go<old_version>`).
* [ ] Update the `builder.dockerImage` parameter in the TeamCity [`Cockroach`](https://teamcity.cockroachdb.com/admin/editProject.html?projectId=Cockroach&tab=projectParams) and [`Internal`](https://teamcity.cockroachdb.com/admin/editProject.html?projectId=Internal&tab=projectParams) projects.
* [ ] Adjust `GO_VERSION` in the TeamCity agent image ([setup script](./packer/teamcity-agent.sh))
  and ask the Developer Infrastructure team to deploy new images.

Resolves cockroachdb#62809.

Release note (general change): Upgrade the CRDB binary to Go 1.15.10.
@rickystewart
Copy link
Collaborator Author

bors r=otan

@craig
Copy link
Contributor

craig bot commented Mar 30, 2021

Build succeeded:

@craig craig bot merged commit fed2225 into cockroachdb:master Mar 30, 2021
rickystewart added a commit to rickystewart/cockroach that referenced this pull request Mar 31, 2021
In cockroachdb#62815, we migrated from an alternative way of installing golang, the
`longsleep/golang-backports` deb repo, to the currently recommended
install method found at https://golang.org/doc/install -- namely, we
download a tarball and then just unzip it in the right spot. This
works perfectly, *except* that the deb package had a dependency on build
tools like `gcc` and `make`, and certain build configurations had come
to depend on their global installation (namely, all those that don't use
`builder.sh` to run a build). This resulted in a couple of failures
being reported:

* https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_ExampleORMs/2834741
* https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_UnitTests_Acceptance/2834732

We just install [`build-essential`](https://packages.ubuntu.com/xenial/build-essential)
here, which is the easiest way to get all of that stuff.

Release note: None
craig bot pushed a commit that referenced this pull request Mar 31, 2021
62498: utilccl,kvccl: improve performance when checking enterprise features r=tbg a=erikgrinaker

**utilccl: cache license decoding**

Previously, the `utilccl` package would decode the license from the the
base64-encoded Protobuf representation in settings every time it was
needed, which was sufficient for its uses. However, recently there's
been a need to check whether enterprise features are enabled in hot
paths (e.g. with follower reads as seen in #62447), making the decoding
cost too great.

This patch adds `cluster.Settings.Cache` as a shared cache, and uses it
to cache decoded licenses with a private key type.

**utilccl,kvccl: add IsEnterpriseEnabled for faster license checks**

`utilccl.CheckEnterpriseEnabled()` is used to check whether a valid
enterprise license exists for a given feature. If no valid license is
found, it returns an error with specific details.

However, `kvccl` used this function in follower read hot paths, and
instantiating an error when follower reads are unavailable could have
significant overhead -- see e.g. #62447.

This patch adds `IsEnterpriseEnabled()`, which has the same behavior as
`CheckEnterpriseEnabled()` but returns a boolean instead. This is
significantly faster since we can avoid instantiating a custom error
each time. `kvccl` is also updated to use this in hot paths.

Resolves #62489.

Release note: None

62642: colserde: fix the edge case with nulls handling r=yuzefovich a=yuzefovich

When serializing the data of Bool, Bytes, Int, and Float types when they
don't have any nulls in the vector, we don't explicit specify the null
bitmap. Previously, when deserializing such vectors with no nulls we
would simply call `UnsetNulls` on the `coldata.Nulls` object that is
currently present. However, it is possible that already present nulls
object cannot support the desired batch length. This could lead to index
out of bounds accesses. Note that in the vast majority of cases this
likely doesn't happen in practice because we check `MaybeHasNulls`, and
that would return `false` making us omit the null checking code.

Fixes: #62636.

Release note (bug fix): Previously, CockroachDB could encounter an
internal error in rare circumstances when executing queries via the
vectorized engine that operate on columns of BOOL, BYTES, INT, and FLOAT
types that have a mix of NULL and non-NULL values.

62740: workload: add idle-conns flag for adding idle connections to tpcc r=rafiss a=RichardJCai

workload: add idle-conns flag for adding idle connections to tpcc

Release note: None

#62526

62814: tenantrate: add "test" that reports IOPS estimations r=RaduBerinde a=RaduBerinde

This change adds a "test" facility which takes the description of a
uniform workload (read percentage, read size, write size) and prints
out an estimation of the sustained IOPS and burst IO. This will allow
a better understanding of how changes to the settings or the mechanism
translate into IOPS changes.

Release note: None

62833: kvserver: deflake TestFollowerReadsWithStaleDescriptor r=aayushshah15 a=aayushshah15

A preceding change (#62696) introduced a flakey update to this test.
Prior to that change, this test was using 2 voting replicas but that
change tried to make it use 1 voter and 1 non-voter instead (as a litmus
test for the new syntax added in #62696).

The test rebalances a replica away from a node and ensures that a
historical read sent immediately afterwards gets re-routed to the
leaseholder replica, since the receiving store had its replica
destroyed. However, when we're using a non-voter in this test, that
non-voter may not have learned about this replication change by the time
it receives this historical query and that fails the assertion.

This commit re-organizes the test and fixes the flake.

Release note: None

62862: testutils: add skip.UnderBazelWithIssue r=rickystewart a=stevendanna

This is to skip individual tests under bazel. This seems a bit more
fine-grained than the broken_in_bazel tag in the bazel configuration
but also allows us to avoid skipping tests that work outside of bazel
in our main test suite.

Release note: None

62877: Added CACHE to SEQUENCE syntax diagrams r=ericharmeling a=ericharmeling

Follow-up of #56954.

Release justification: non-production code changes

Release note: None

62889: colexecerror: catch panics from packages in sql/sem folder r=yuzefovich a=yuzefovich

Previously, we would only catch panics from `sql/sem/tree` package.
Recently sqlsmith encountered a crash because of a panic in
`sql/sem/builtins` package, and I believe it is reasonable to catch
panics from that package as well as from `sql/sem/transform`, so we will
now be catching based on `sql/sem` prefix.

Addresses: #62846.

Release note: None

62898: build: install essential build tools in teamcity build agents r=jlinder a=rickystewart

In #62815, we migrated from an alternative way of installing golang, the
`longsleep/golang-backports` deb repo, to the currently recommended
install method found at https://golang.org/doc/install -- namely, we
download a tarball and then just unzip it in the right spot. This
works perfectly, *except* that the deb package had a dependency on build
tools like `gcc` and `make`, and certain build configurations had come
to depend on their global installation (namely, all those that don't use
`builder.sh` to run a build). This resulted in a couple of failures
being reported:

* https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_ExampleORMs/2834741
* https://teamcity.cockroachdb.com/buildConfiguration/Cockroach_UnitTests_Acceptance/2834732

We just install [`build-essential`](https://packages.ubuntu.com/xenial/build-essential)
here, which is the easiest way to get all of that stuff.

Release note: None

Co-authored-by: Erik Grinaker <[email protected]>
Co-authored-by: Yahor Yuzefovich <[email protected]>
Co-authored-by: richardjcai <[email protected]>
Co-authored-by: Radu Berinde <[email protected]>
Co-authored-by: Aayush Shah <[email protected]>
Co-authored-by: Steven Danna <[email protected]>
Co-authored-by: Eric Harmeling <[email protected]>
Co-authored-by: Ricky Stewart <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

build: bump Go version to latest patch release of 1.15
4 participants