roachtest: multitenant-upgrade failed [waiting for 71692] #71269

cockroach-teamcity · 2021-10-07T11:09:11Z

roachtest.multitenant-upgrade failed with artifacts on release-21.2 @ 82e8782453b1cf14460d93b6bf8328a7b2964575:

		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1991
		  | main.(*clusterImpl).RunE
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1970
		  | main.(*clusterImpl).Run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1891
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).stop
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:84
		  | runtime.gopanic
		  | 	/usr/local/go/src/runtime/panic.go:965
		  | main.(*testImpl).fatalfInner
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:284
		  | main.(*testImpl).Fatal
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:259
		  | main.(*clusterImpl).Run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1893
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).stop
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:84
		  | runtime.gopanic
		  | 	/usr/local/go/src/runtime/panic.go:965
		  | main.(*testImpl).fatalfInner
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:284
		  | main.(*testImpl).Fatal
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:259
		  | main.(*clusterImpl).Run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1893
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).stop
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:84
		  | runtime.gopanic
		  | 	/usr/local/go/src/runtime/panic.go:965
		  | main.(*testImpl).fatalfInner
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:284
		  | main.(*testImpl).Fatal
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:259
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).start.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:119
		  | github.com/cockroachdb/cockroach/pkg/util/retry.ForDuration
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/util/retry/retry.go:197
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).start
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:114
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runMultiTenantUpgrade
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:294
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerMultiTenantUpgrade.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:39
		  | main.(*testRunner).runTest.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:777
		  | runtime.goexit
		  | 	/usr/local/go/src/runtime/asm_amd64.s:1371
		Wraps: (4) cluster.RunL
		Wraps: (5) context canceled
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *errors.errorString

Reproduce

See: roachtest README

Same failure on other branches

roachtest: multitenant-upgrade failed #69920 roachtest: multitenant-upgrade failed [C-test-failure O-roachtest O-robot T-sql-experience branch-master]

/cc @cockroachdb/kv-triage _{This test on roachdash | Improve this report!}

The text was updated successfully, but these errors were encountered:

tbg · 2021-10-07T16:17:44Z

		  | ERROR: could not cleanup temporary directories from record file: could not lock temporary directory /home/ubuntu/cockroach-data/cockroach-temp048983879, may still be in use: resource temporarily unavailable

Odd problem, not a release-blocker, but going to keep monitoring this

cockroach-teamcity · 2021-10-08T11:22:21Z

roachtest.multitenant-upgrade failed with artifacts on release-21.2 @ 3cf3b0ea3b08cb24e4d6b84c6c237b856ce6b411:

		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1991
		  | main.(*clusterImpl).RunE
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1970
		  | main.(*clusterImpl).Run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1891
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).stop
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:84
		  | runtime.gopanic
		  | 	/usr/local/go/src/runtime/panic.go:965
		  | main.(*testImpl).fatalfInner
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:284
		  | main.(*testImpl).Fatal
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:259
		  | main.(*clusterImpl).Run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1893
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).stop
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:84
		  | runtime.gopanic
		  | 	/usr/local/go/src/runtime/panic.go:965
		  | main.(*testImpl).fatalfInner
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:284
		  | main.(*testImpl).Fatal
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:259
		  | main.(*clusterImpl).Run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1893
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).stop
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:84
		  | runtime.gopanic
		  | 	/usr/local/go/src/runtime/panic.go:965
		  | main.(*testImpl).fatalfInner
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:284
		  | main.(*testImpl).Fatal
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:259
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).start.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:119
		  | github.com/cockroachdb/cockroach/pkg/util/retry.ForDuration
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/util/retry/retry.go:197
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).start
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:114
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runMultiTenantUpgrade
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:294
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerMultiTenantUpgrade.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:39
		  | main.(*testRunner).runTest.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:777
		  | runtime.goexit
		  | 	/usr/local/go/src/runtime/asm_amd64.s:1371
		Wraps: (4) cluster.RunL
		Wraps: (5) context canceled
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *errors.errorString

Reproduce

See: roachtest README

Same failure on other branches

roachtest: multitenant-upgrade failed #69920 roachtest: multitenant-upgrade failed [C-test-failure O-roachtest O-robot T-sql-experience branch-master]

/cc @cockroachdb/kv-triage _{This test on roachdash | Improve this report!}

cockroach-teamcity · 2021-10-12T10:52:35Z

roachtest.multitenant-upgrade failed with artifacts on release-21.2 @ 2ef24098cc88bab3958ad905f46b0ed7e74bd7db:

		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1991
		  | main.(*clusterImpl).RunE
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1970
		  | main.(*clusterImpl).Run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1891
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).stop
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:84
		  | runtime.gopanic
		  | 	/usr/local/go/src/runtime/panic.go:965
		  | main.(*testImpl).fatalfInner
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:284
		  | main.(*testImpl).Fatal
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:259
		  | main.(*clusterImpl).Run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1893
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).stop
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:84
		  | runtime.gopanic
		  | 	/usr/local/go/src/runtime/panic.go:965
		  | main.(*testImpl).fatalfInner
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:284
		  | main.(*testImpl).Fatal
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:259
		  | main.(*clusterImpl).Run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1893
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).stop
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:84
		  | runtime.gopanic
		  | 	/usr/local/go/src/runtime/panic.go:965
		  | main.(*testImpl).fatalfInner
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:284
		  | main.(*testImpl).Fatal
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:259
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).start.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:119
		  | github.com/cockroachdb/cockroach/pkg/util/retry.ForDuration
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/util/retry/retry.go:197
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).start
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:114
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runMultiTenantUpgrade
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:294
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerMultiTenantUpgrade.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:39
		  | main.(*testRunner).runTest.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:777
		  | runtime.goexit
		  | 	/usr/local/go/src/runtime/asm_amd64.s:1371
		Wraps: (4) cluster.RunL
		Wraps: (5) context canceled
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *errors.errorString

Reproduce

See: roachtest README

Same failure on other branches

roachtest: multitenant-upgrade failed #69920 roachtest: multitenant-upgrade failed [C-test-failure O-roachtest O-robot T-sql-experience branch-master]

/cc @cockroachdb/kv-triage _{This test on roachdash | Improve this report!}

cockroach-teamcity · 2021-10-13T11:46:25Z

roachtest.multitenant-upgrade failed with artifacts on release-21.2 @ e67cf963427faf93670a4c87e0d02bc0ef213b1d:

		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1991
		  | main.(*clusterImpl).RunE
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1970
		  | main.(*clusterImpl).Run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1891
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).stop
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:84
		  | runtime.gopanic
		  | 	/usr/local/go/src/runtime/panic.go:965
		  | main.(*testImpl).fatalfInner
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:284
		  | main.(*testImpl).Fatal
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:259
		  | main.(*clusterImpl).Run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1893
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).stop
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:84
		  | runtime.gopanic
		  | 	/usr/local/go/src/runtime/panic.go:965
		  | main.(*testImpl).fatalfInner
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:284
		  | main.(*testImpl).Fatal
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:259
		  | main.(*clusterImpl).Run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1893
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).stop
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:84
		  | runtime.gopanic
		  | 	/usr/local/go/src/runtime/panic.go:965
		  | main.(*testImpl).fatalfInner
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:284
		  | main.(*testImpl).Fatal
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:259
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).start.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:119
		  | github.com/cockroachdb/cockroach/pkg/util/retry.ForDuration
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/util/retry/retry.go:197
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).start
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:114
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runMultiTenantUpgrade
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:294
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerMultiTenantUpgrade.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:39
		  | main.(*testRunner).runTest.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:777
		  | runtime.goexit
		  | 	/usr/local/go/src/runtime/asm_amd64.s:1371
		Wraps: (4) cluster.RunL
		Wraps: (5) context canceled
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *errors.errorString

Reproduce

See: roachtest README

Same failure on other branches

roachtest: multitenant-upgrade failed #69920 roachtest: multitenant-upgrade failed [C-test-failure O-roachtest O-robot T-sql-experience branch-master]

/cc @cockroachdb/kv-triage _{This test on roachdash | Improve this report!}

cockroach-teamcity · 2021-10-14T11:31:56Z

roachtest.multitenant-upgrade failed with artifacts on release-21.2 @ 2712f35c0c22919ed5547f9f64c03e907373e74f:

		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1991
		  | main.(*clusterImpl).RunE
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1970
		  | main.(*clusterImpl).Run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1891
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).stop
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:84
		  | runtime.gopanic
		  | 	/usr/local/go/src/runtime/panic.go:965
		  | main.(*testImpl).fatalfInner
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:284
		  | main.(*testImpl).Fatal
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:259
		  | main.(*clusterImpl).Run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1893
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).stop
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:84
		  | runtime.gopanic
		  | 	/usr/local/go/src/runtime/panic.go:965
		  | main.(*testImpl).fatalfInner
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:284
		  | main.(*testImpl).Fatal
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:259
		  | main.(*clusterImpl).Run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1893
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).stop
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:84
		  | runtime.gopanic
		  | 	/usr/local/go/src/runtime/panic.go:965
		  | main.(*testImpl).fatalfInner
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:284
		  | main.(*testImpl).Fatal
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:259
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).start.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:119
		  | github.com/cockroachdb/cockroach/pkg/util/retry.ForDuration
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/util/retry/retry.go:197
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).start
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:114
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runMultiTenantUpgrade
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:294
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerMultiTenantUpgrade.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:39
		  | main.(*testRunner).runTest.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:777
		  | runtime.goexit
		  | 	/usr/local/go/src/runtime/asm_amd64.s:1371
		Wraps: (4) cluster.RunL
		Wraps: (5) context canceled
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *errors.errorString

Reproduce

See: roachtest README

Same failure on other branches

roachtest: multitenant-upgrade failed #69920 roachtest: multitenant-upgrade failed [C-test-failure O-roachtest O-robot T-sql-experience branch-master]

/cc @cockroachdb/kv-triage _{This test on roachdash | Improve this report!}

tbg · 2021-10-14T19:48:43Z

Oh wait this is dumb. We're starting a tenant but it's pulling in the the default temp dir spec of cockroach-data, so it will actually race with the server that is running there and try to clean up its temp dir that it is very much still using.

cockroach-teamcity · 2021-10-16T10:33:26Z

roachtest.multitenant-upgrade failed with artifacts on release-21.2 @ b3c1760a030791143c57e89c73d1f26ae8babaf5:

		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1991
		  | main.(*clusterImpl).RunE
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1970
		  | main.(*clusterImpl).Run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1891
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).stop
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:84
		  | runtime.gopanic
		  | 	/usr/local/go/src/runtime/panic.go:965
		  | main.(*testImpl).fatalfInner
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:284
		  | main.(*testImpl).Fatal
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:259
		  | main.(*clusterImpl).Run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1893
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).stop
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:84
		  | runtime.gopanic
		  | 	/usr/local/go/src/runtime/panic.go:965
		  | main.(*testImpl).fatalfInner
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:284
		  | main.(*testImpl).Fatal
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:259
		  | main.(*clusterImpl).Run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1893
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).stop
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:84
		  | runtime.gopanic
		  | 	/usr/local/go/src/runtime/panic.go:965
		  | main.(*testImpl).fatalfInner
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:284
		  | main.(*testImpl).Fatal
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:259
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).start.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:119
		  | github.com/cockroachdb/cockroach/pkg/util/retry.ForDuration
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/util/retry/retry.go:197
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).start
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:114
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runMultiTenantUpgrade
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:294
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerMultiTenantUpgrade.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:39
		  | main.(*testRunner).runTest.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:777
		  | runtime.goexit
		  | 	/usr/local/go/src/runtime/asm_amd64.s:1371
		Wraps: (4) cluster.RunL
		Wraps: (5) context canceled
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *errors.errorString

Reproduce

See: roachtest README

Same failure on other branches

roachtest: multitenant-upgrade failed #69920 roachtest: multitenant-upgrade failed [C-test-failure O-roachtest O-robot T-sql-experience branch-master]

/cc @cockroachdb/kv-triage _{This test on roachdash | Improve this report!}

cockroach-teamcity · 2021-10-19T11:26:54Z

roachtest.multitenant-upgrade failed with artifacts on release-21.2 @ cb725a6acdc08257ad81273535dadad5d0a727dc:

		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1991
		  | main.(*clusterImpl).RunE
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1970
		  | main.(*clusterImpl).Run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1891
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).stop
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:84
		  | runtime.gopanic
		  | 	/usr/local/go/src/runtime/panic.go:965
		  | main.(*testImpl).fatalfInner
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:284
		  | main.(*testImpl).Fatal
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:259
		  | main.(*clusterImpl).Run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1893
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).stop
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:84
		  | runtime.gopanic
		  | 	/usr/local/go/src/runtime/panic.go:965
		  | main.(*testImpl).fatalfInner
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:284
		  | main.(*testImpl).Fatal
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:259
		  | main.(*clusterImpl).Run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1893
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).stop
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:84
		  | runtime.gopanic
		  | 	/usr/local/go/src/runtime/panic.go:965
		  | main.(*testImpl).fatalfInner
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:284
		  | main.(*testImpl).Fatal
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:259
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).start.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:119
		  | github.com/cockroachdb/cockroach/pkg/util/retry.ForDuration
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/util/retry/retry.go:197
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).start
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:114
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runMultiTenantUpgrade
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:294
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerMultiTenantUpgrade.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:39
		  | main.(*testRunner).runTest.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:777
		  | runtime.goexit
		  | 	/usr/local/go/src/runtime/asm_amd64.s:1371
		Wraps: (4) cluster.RunL
		Wraps: (5) context canceled
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *errors.errorString

Reproduce

See: roachtest README

/cc @cockroachdb/kv-triage _{This test on roachdash | Improve this report!}

cockroach-teamcity · 2021-10-20T11:22:47Z

roachtest.multitenant-upgrade failed with artifacts on release-21.2 @ ff34ea874d5c7efc47922f73b7a1d5ee1ef614fc:

		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1991
		  | main.(*clusterImpl).RunE
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1970
		  | main.(*clusterImpl).Run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1891
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).stop
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:84
		  | runtime.gopanic
		  | 	/usr/local/go/src/runtime/panic.go:965
		  | main.(*testImpl).fatalfInner
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:284
		  | main.(*testImpl).Fatal
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:259
		  | main.(*clusterImpl).Run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1893
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).stop
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:84
		  | runtime.gopanic
		  | 	/usr/local/go/src/runtime/panic.go:965
		  | main.(*testImpl).fatalfInner
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:284
		  | main.(*testImpl).Fatal
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:259
		  | main.(*clusterImpl).Run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1893
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).stop
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:84
		  | runtime.gopanic
		  | 	/usr/local/go/src/runtime/panic.go:965
		  | main.(*testImpl).fatalfInner
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:284
		  | main.(*testImpl).Fatal
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:259
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).start.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:119
		  | github.com/cockroachdb/cockroach/pkg/util/retry.ForDuration
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/util/retry/retry.go:197
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).start
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:114
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runMultiTenantUpgrade
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:294
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerMultiTenantUpgrade.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:39
		  | main.(*testRunner).runTest.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:777
		  | runtime.goexit
		  | 	/usr/local/go/src/runtime/asm_amd64.s:1371
		Wraps: (4) cluster.RunL
		Wraps: (5) context canceled
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *errors.errorString

Reproduce

See: roachtest README

/cc @cockroachdb/kv-triage _{This test on roachdash | Improve this report!}

cockroach-teamcity · 2021-10-21T11:38:04Z

roachtest.multitenant-upgrade failed with artifacts on release-21.2 @ c5a6b266917ee3846dbd7ae1126c6a5d55cf439b:

		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1991
		  | main.(*clusterImpl).RunE
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1970
		  | main.(*clusterImpl).Run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1891
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).stop
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:84
		  | runtime.gopanic
		  | 	/usr/local/go/src/runtime/panic.go:965
		  | main.(*testImpl).fatalfInner
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:284
		  | main.(*testImpl).Fatal
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:259
		  | main.(*clusterImpl).Run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1893
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).stop
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:84
		  | runtime.gopanic
		  | 	/usr/local/go/src/runtime/panic.go:965
		  | main.(*testImpl).fatalfInner
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:284
		  | main.(*testImpl).Fatal
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:259
		  | main.(*clusterImpl).Run
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/cluster.go:1893
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).stop
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:84
		  | runtime.gopanic
		  | 	/usr/local/go/src/runtime/panic.go:965
		  | main.(*testImpl).fatalfInner
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:284
		  | main.(*testImpl).Fatal
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_impl.go:259
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).start.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:119
		  | github.com/cockroachdb/cockroach/pkg/util/retry.ForDuration
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/util/retry/retry.go:197
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.(*tenantNode).start
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:114
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.runMultiTenantUpgrade
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:294
		  | github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests.registerMultiTenantUpgrade.func1
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/tests/multitenant_upgrade.go:39
		  | main.(*testRunner).runTest.func2
		  | 	/home/agent/work/.go/src/github.com/cockroachdb/cockroach/pkg/cmd/roachtest/test_runner.go:777
		  | runtime.goexit
		  | 	/usr/local/go/src/runtime/asm_amd64.s:1371
		Wraps: (4) cluster.RunL
		Wraps: (5) context canceled
		Error types: (1) *withstack.withStack (2) *errutil.withPrefix (3) *withstack.withStack (4) *errutil.withPrefix (5) *errors.errorString

Reproduce

See: roachtest README

/cc @cockroachdb/kv-triage _{This test on roachdash | Improve this report!}

cockroach-teamcity added branch-release-21.2 C-test-failure Broken test (automatically or manually discovered). O-roachtest O-robot Originated from a bot. release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. labels Oct 7, 2021

cockroach-teamcity mentioned this issue Oct 7, 2021

roachtest: multitenant-upgrade failed #69920

Closed

tbg removed the release-blocker Indicates a release-blocker. Use with branch-release-2x.x label to denote which branch is blocked. label Oct 7, 2021

tbg mentioned this issue Oct 15, 2021

cli: start-sql should default store to cockroach-data-<tenant> #71603

Closed

tbg mentioned this issue Oct 19, 2021

release-21.2: roachtest: fix multitenant-upgrade #71692

Merged

cockroach-teamcity mentioned this issue Oct 19, 2021

roachtest: multitenant-upgrade failed #71705

Closed

tbg changed the title ~~roachtest: multitenant-upgrade failed~~ roachtest: multitenant-upgrade failed [waiting for 71692] Oct 20, 2021

cockroach-teamcity mentioned this issue Nov 3, 2021

roachtest: multitenant-upgrade failed [caused by #72425] #72420

Closed

irfansharif mentioned this issue Nov 8, 2021

multitenant: unable to start secondary tenant pod with single node cluster #72518

Closed

tbg closed this as completed Nov 10, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

roachtest: multitenant-upgrade failed [waiting for 71692] #71269

roachtest: multitenant-upgrade failed [waiting for 71692] #71269

cockroach-teamcity commented Oct 7, 2021

tbg commented Oct 7, 2021

cockroach-teamcity commented Oct 8, 2021

cockroach-teamcity commented Oct 12, 2021

cockroach-teamcity commented Oct 13, 2021

cockroach-teamcity commented Oct 14, 2021

tbg commented Oct 14, 2021

cockroach-teamcity commented Oct 16, 2021

cockroach-teamcity commented Oct 19, 2021

cockroach-teamcity commented Oct 20, 2021

cockroach-teamcity commented Oct 21, 2021

roachtest: multitenant-upgrade failed [waiting for 71692] #71269

roachtest: multitenant-upgrade failed [waiting for 71692] #71269

Comments

cockroach-teamcity commented Oct 7, 2021

tbg commented Oct 7, 2021

cockroach-teamcity commented Oct 8, 2021

cockroach-teamcity commented Oct 12, 2021

cockroach-teamcity commented Oct 13, 2021

cockroach-teamcity commented Oct 14, 2021

tbg commented Oct 14, 2021

cockroach-teamcity commented Oct 16, 2021

cockroach-teamcity commented Oct 19, 2021

cockroach-teamcity commented Oct 20, 2021

cockroach-teamcity commented Oct 21, 2021