Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

roachtest: version/sequence-upgrade failed #67182

Closed
cockroach-teamcity opened this issue Jul 2, 2021 · 9 comments · Fixed by #67276
Closed

roachtest: version/sequence-upgrade failed #67182

cockroach-teamcity opened this issue Jul 2, 2021 · 9 comments · Fixed by #67276
Assignees
@ajwerner
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)

Comments

@cockroach-teamcity
Copy link
Member

roachtest.version/sequence-upgrade failed with artifacts on master @ eb7f3fe2991992472a68d701d65543f7a9c7fb56:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/version/sequence-upgrade/run_1
	versionupgrade.go:344,versionupgrade.go:472,retry.go:197,versionupgrade.go:471,versionupgrade.go:214,sequence_upgrade.go:116,sequence_upgrade.go:28,test_runner.go:761: pq: operation "show cluster setting version" timed out after 2m0s: value differs between local setting ([18 8 8 20 16 2 24 0 32 20]) and KV ([18 8 8 20 16 2 24 0 32 0]); try again later (<nil> after 1m59.370508512s)
Reproduce

To reproduce, try: 

# From https://go.crdb.dev/p/roachstress, perhaps edited lightly.
caffeinate ./roachstress.sh version/sequence-upgrade

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!
@cockroach-teamcity cockroach-teamcity added branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. labels Jul 2, 2021
@blathers-crl blathers-crl bot added the T-sql-schema-deprecated Use T-sql-foundations instead label Jul 2, 2021
@cockroach-teamcity
Copy link
Member Author

roachtest.version/sequence-upgrade failed with artifacts on master @ eb7f3fe2991992472a68d701d65543f7a9c7fb56:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/version/sequence-upgrade/run_1
	versionupgrade.go:344,versionupgrade.go:472,retry.go:197,versionupgrade.go:471,versionupgrade.go:214,sequence_upgrade.go:116,sequence_upgrade.go:28,test_runner.go:761: pq: operation "show cluster setting version" timed out after 2m0s: value differs between local setting ([18 8 8 20 16 2 24 0 32 20]) and KV ([18 8 8 20 16 2 24 0 32 0]); try again later (<nil> after 1m59.401902425s)
Reproduce

To reproduce, try: 

# From https://go.crdb.dev/p/roachstress, perhaps edited lightly.
caffeinate ./roachstress.sh version/sequence-upgrade

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.version/sequence-upgrade failed with artifacts on master @ 7121f6d80505801cc432a178ad8b6dacbaac1448:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/version/sequence-upgrade/run_1
	versionupgrade.go:344,versionupgrade.go:472,retry.go:197,versionupgrade.go:471,versionupgrade.go:214,sequence_upgrade.go:116,sequence_upgrade.go:28,test_runner.go:761: pq: operation "show cluster setting version" timed out after 2m0s: value differs between local setting ([18 8 8 20 16 2 24 0 32 20]) and KV ([18 8 8 20 16 2 24 0 32 0]); try again later (<nil> after 1m58.012053871s)
Reproduce

To reproduce, try: 

# From https://go.crdb.dev/p/roachstress, perhaps edited lightly.
caffeinate ./roachstress.sh version/sequence-upgrade

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.version/sequence-upgrade failed with artifacts on master @ 84ec89c77841016da0b9c4c71772a4304bad45a5:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/version/sequence-upgrade/run_1
	versionupgrade.go:344,versionupgrade.go:472,retry.go:197,versionupgrade.go:471,versionupgrade.go:214,sequence_upgrade.go:116,sequence_upgrade.go:28,z_test_runner.go:765: pq: operation "show cluster setting version" timed out after 2m0s: value differs between local setting ([18 8 8 20 16 2 24 0 32 20]) and KV ([18 8 8 20 16 2 24 0 32 0]); try again later (<nil> after 1m58.899005248s)
Reproduce

To reproduce, try: 

# From https://go.crdb.dev/p/roachstress, perhaps edited lightly.
caffeinate ./roachstress.sh version/sequence-upgrade

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

@blathers-crl blathers-crl bot added the T-server-and-security DB Server & Security label Jul 6, 2021
@ajwerner ajwerner mentioned this issue Jul 6, 2021
Merged
@ajwerner ajwerner self-assigned this Jul 6, 2021
ajwerner added a commit to ajwerner/cockroach that referenced this issue Jul 6, 2021
@ajwerner
roachtest: remove test which exercised 20.2-21.1 upgrade
905c916 
We no longer use the master roachtest for old branches. Given that, the answer
is to just remove this test from master.

This test fails because in cockroachdb#67053 I removed compatibility between 20.2 and
master. We should never be trying to do that upgrade.

Fixes cockroachdb#67182

Release note: None
@cockroach-teamcity
Copy link
Member Author

roachtest.version/sequence-upgrade failed with artifacts on master @ ba689f91a5bbd7737a8a229522048b2ba91b2ec0:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/version/sequence-upgrade/run_1
	versionupgrade.go:344,versionupgrade.go:472,retry.go:197,versionupgrade.go:471,versionupgrade.go:214,sequence_upgrade.go:116,sequence_upgrade.go:28,z_test_runner.go:765: pq: operation "show cluster setting version" timed out after 2m0s: value differs between local setting ([18 8 8 20 16 2 24 0 32 20]) and KV ([18 8 8 20 16 2 24 0 32 0]); try again later (<nil> after 1m59.081082402s)
Reproduce

To reproduce, try: 

# From https://go.crdb.dev/p/roachstress, perhaps edited lightly.
caffeinate ./roachstress.sh version/sequence-upgrade

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.version/sequence-upgrade failed with artifacts on master @ 7d0fd136a538b22cbf9bfff03b2885b7783711aa:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/version/sequence-upgrade/run_1
	versionupgrade.go:343,versionupgrade.go:471,retry.go:197,versionupgrade.go:470,versionupgrade.go:213,sequence_upgrade.go:116,sequence_upgrade.go:28,z_test_runner.go:765: pq: operation "show cluster setting version" timed out after 2m0s: value differs between local setting ([18 8 8 20 16 2 24 0 32 20]) and KV ([18 8 8 20 16 2 24 0 32 0]); try again later (<nil> after 1m59.388897535s)
Reproduce

To reproduce, try: 

# From https://go.crdb.dev/p/roachstress, perhaps edited lightly.
caffeinate ./roachstress.sh version/sequence-upgrade

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.version/sequence-upgrade failed with artifacts on master @ c543d448266e2e1e3adefe3a55f920beacd848b3:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/version/sequence-upgrade/run_1
	versionupgrade.go:343,versionupgrade.go:471,retry.go:197,versionupgrade.go:470,versionupgrade.go:213,sequence_upgrade.go:116,sequence_upgrade.go:28,z_test_runner.go:765: pq: operation "show cluster setting version" timed out after 2m0s: value differs between local setting ([18 8 8 20 16 2 24 0 32 20]) and KV ([18 8 8 20 16 2 24 0 32 0]); try again later (<nil> after 1m59.462311015s)
Reproduce

To reproduce, try: 

# From https://go.crdb.dev/p/roachstress, perhaps edited lightly.
caffeinate ./roachstress.sh version/sequence-upgrade

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.version/sequence-upgrade failed with artifacts on master @ 0a48b0b74b0a6057f1d418875b97830359a52ec6:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/version/sequence-upgrade/run_1
	versionupgrade.go:343,versionupgrade.go:471,retry.go:197,versionupgrade.go:470,versionupgrade.go:213,sequence_upgrade.go:116,sequence_upgrade.go:28,z_test_runner.go:765: pq: operation "show cluster setting version" timed out after 2m0s: value differs between local setting ([18 8 8 20 16 2 24 0 32 20]) and KV ([18 8 8 20 16 2 24 0 32 0]); try again later (<nil> after 1m58.241509091s)
Reproduce

To reproduce, try: 

# From https://go.crdb.dev/p/roachstress, perhaps edited lightly.
caffeinate ./roachstress.sh version/sequence-upgrade

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.version/sequence-upgrade failed with artifacts on master @ 9ba3068987a973a92757db53800dfafbf8a262e7:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/version/sequence-upgrade/run_1
	versionupgrade.go:343,versionupgrade.go:471,retry.go:197,versionupgrade.go:470,versionupgrade.go:213,sequence_upgrade.go:116,sequence_upgrade.go:28,z_test_runner.go:765: pq: operation "show cluster setting version" timed out after 2m0s: value differs between local setting ([18 8 8 20 16 2 24 0 32 20]) and KV ([18 8 8 20 16 2 24 0 32 0]); try again later (<nil> after 1m59.77279872s)
Reproduce

To reproduce, try: 

# From https://go.crdb.dev/p/roachstress, perhaps edited lightly.
caffeinate ./roachstress.sh version/sequence-upgrade

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member Author

roachtest.version/sequence-upgrade failed with artifacts on master @ 5a0b11e4208ad8dc2a957f3f70199c2600155ba6:

The test failed on branch=master, cloud=gce:
test artifacts and logs in: /home/agent/work/.go/src/github.com/cockroachdb/cockroach/artifacts/version/sequence-upgrade/run_1
	versionupgrade.go:343,versionupgrade.go:471,retry.go:197,versionupgrade.go:470,versionupgrade.go:213,sequence_upgrade.go:116,sequence_upgrade.go:28,test_runner.go:765: pq: operation "show cluster setting version" timed out after 2m0s: value differs between local setting ([18 8 8 20 16 2 24 0 32 20]) and KV ([18 8 8 20 16 2 24 0 32 0]); try again later (<nil> after 1m58.739976165s)
Reproduce

To reproduce, try: 

# From https://go.crdb.dev/p/roachstress, perhaps edited lightly.
caffeinate ./roachstress.sh version/sequence-upgrade

/cc @cockroachdb/sql-schema

This test on roachdash | Improve this report!

craig bot pushed a commit that referenced this issue Jul 14, 2021
@chrisseto @ajwerner
@knz @rafiss @rail @rickystewart
Merge #66850 #67276 #67460 #67461 #67474 #67484 #67588 #67594 #67603
eef3cef 
66850: sqlproxyccl: statistically load balance across tenants r=chrisseto a=chrisseto

    Previously, the SQLProxy's Directory implementation would always
    select the first instance from the pods list. As part of the
    SQLProxy's duty is load balancing, this was inadequate. This commit
    upgrades the tenant dir to have pods report their CPU load. This
    load is then used to perform a weighted distribution across all
    running tenant pods.

Release note: None

67276: roachtest: remove test which exercised 20.2-21.1 upgrade r=postamar a=ajwerner

We no longer use the master roachtest for old branches. Given that, the answer
is to just remove this test from master.

This test fails because in #67053 I removed compatibility between 20.2 and
master. We should never be trying to do that upgrade.

Fixes #67182

Release note: None

67460: cli: move the demo cluster to new sub-package `democluster` (5/) r=otan a=knz

First 15 commits are from #67459 and prior.
Please ignore those (they will disappear once the prefix PR is merged)

Only the last 5 commits belong to this PR.

This is part of a family of PRs that aim to encapsulate CLI functionality into finer grained packages.

67461: cli: SQL shell cleanups and better encapsulation (6/) r=otan a=knz

First 20 commits are from #67460 and prior.
Please ignore those (they will disappear once the prefix PR is merged)

Only the last 4 commits belong to this PR.

This is part of a family of PRs that aim to encapsulate CLI functionality into finer grained packages.

67474: cli: new sub-package `clisqlcfg` to ease the creation of a standalone SQL shell (8/) r=otan a=knz

First 28 commits are from #67462 and prior.
Please ignore those (they will disappear once the prefix PR is merged)

Only the last 2 commits belong to this PR.

This is part of a family of PRs that aim to encapsulate CLI functionality into finer grained packages.



67484: clisqlclient: drop the dependency on 'security' r=otan a=knz

Fixes #67481.
Informs #67470.

This PR only contains 1 commit (the last).
All commits except for the last belong to #67469 and prior PRs. Ignore them during review.


The 'security' package is quite large. There is no reason to embark it
into a standalone SQL shell.

Towards achieving this, this patch removes the dependency on the
'security' package from `clisqlclient` by moving the password prompt
code into a sub-package.

Release note: None

67588: compare_test: account for more differences from postgres r=otan a=rafiss

- explicit cast for constants #62602 (comment)
- don't use postgis_script builtin #62602 (comment)
- handle differences in timezone offset display #62602 (comment)

Release note: None

67594: build: update go to 1.16.6 r=rail a=rail

* [ ] Adjust the Pebble tests to run in new version.
* [x] Adjust version in Docker image ([source](./builder/Dockerfile)).
* [x] Adjust version in the TeamCity agent image ([setup script](./packer/teamcity-agent.sh))
* [x] Rebuild and push the Docker image (following [Basic Process](#basic-process))
* [x] Bump the version in `WORKSPACE` under `go_register_toolchains`. You may need to bump [rules_go](https://github.com/bazelbuild/rules_go/releases).
* [x] Bump the version in `builder.sh` accordingly ([source](./builder.sh#L6)).
* [x] Bump the version in `go-version-check.sh` ([source](./go-version-check.sh)), unless bumping to a new patch release.
* [x] Bump the go version in `go.mod`. You may also need to rerun `make vendor_rebuild` if vendoring has changed.
* [x] Bump the default installed version of Go in `bootstrap-debian.sh` ([source](./bootstrap/bootstrap-debian.sh)).
* [x] Replace other mentions of the older version of go (grep for `golang:<old_version>` and `go<old_version>`).
* [ ] Update the `builder.dockerImage` parameter in the TeamCity [`Cockroach`](https://teamcity.cockroachdb.com/admin/editProject.html?projectId=Cockroach&tab=projectParams) and [`Internal`](https://teamcity.cockroachdb.com/admin/editProject.html?projectId=Internal&tab=projectParams) projects.
* [ ] Ask the Developer Infrastructure team to deploy new TeamCity agent images according to [packer/README.md](./packer/README.md)

Release note: None

67603: roachtest: re-mark `pkg/cmd/roachtest` tests as `broken_in_bazel` r=rail a=rickystewart

`faa4c2ccdaeca01eca3ccee29a8a64973cf8b043` removed this tag, which
caused all Bazel test runs to begin failing.

Closes #67595.

Release note: None

Co-authored-by: Chris Seto <[email protected]>
Co-authored-by: Andrew Werner <[email protected]>
Co-authored-by: Raphael 'kena' Poss <[email protected]>
Co-authored-by: Rafi Shamim <[email protected]>
Co-authored-by: Rail Aliiev <[email protected]>
Co-authored-by: Ricky Stewart <[email protected]>
@craig craig bot closed this as completed in 26d715c Jul 14, 2021
pawalt pushed a commit to pawalt/cockroach that referenced this issue Jul 22, 2021
@ajwerner @pawalt
roachtest: remove test which exercised 20.2-21.1 upgrade
6c32648 
We no longer use the master roachtest for old branches. Given that, the answer
is to just remove this test from master.

This test fails because in cockroachdb#67053 I removed compatibility between 20.2 and
master. We should never be trying to do that upgrade.

Fixes cockroachdb#67182

Release note: None
@exalate-issue-sync exalate-issue-sync bot added T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions) and removed T-server-and-security DB Server & Security T-sql-schema-deprecated Use T-sql-foundations instead labels May 10, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ajwerner ajwerner

Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. T-sql-foundations SQL Foundations Team (formerly SQL Schema + SQL Sessions)
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

roachtest: remove test which exercised 20.2-21.1 upgrade ajwerner/cockroach
2 participants
@ajwerner @cockroach-teamcity