ccl/sqlproxyccl/tenant: TestWatchPods failed #69220
Assignees
Labels
branch-master
Failures and bugs on the master branch.
C-test-failure
Broken test (automatically or manually discovered).
O-robot
Originated from a bot.
Comments
cockroach-teamcity
added
branch-master
|
ccl/sqlproxyccl/tenant.TestWatchPods failed with artifacts on master @ c438563d64d6cd6493a59d84e65e62d3f6060725: ReproduceTo reproduce, try: make stressrace TESTS=TestWatchPods PKG=./pkg/ccl/sqlproxyccl/tenant TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1Parameters in this failure:
|
|
ccl/sqlproxyccl/tenant.TestWatchPods failed with artifacts on master @ 649a8ca38cdc0883dd7a7811b189ea53af19f644: ReproduceTo reproduce, try: make stressrace TESTS=TestWatchPods PKG=./pkg/ccl/sqlproxyccl/tenant TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1Parameters in this failure:
|
|
ccl/sqlproxyccl/tenant.TestWatchPods failed with artifacts on master @ 8ef90ca92676389da2c955054ab5d996bb9ae6ff: HelpSee also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)Parameters in this failure:
|
|
ccl/sqlproxyccl/tenant.TestWatchPods failed with artifacts on master @ 38fd4064f40c574d0b2aee02f414787a2760fcec: HelpSee also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)Parameters in this failure:
|
|
ccl/sqlproxyccl/tenant.TestWatchPods failed with artifacts on master @ 884088d64d0859d72e42ee4fde1cf65192f1cb99: HelpSee also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)Parameters in this failure:
|
|
ccl/sqlproxyccl/tenant.TestWatchPods failed with artifacts on master @ 150caaf45cf8633aca2e5d0ba7ef7bbc2a285efb: HelpSee also: [How To Investigate a Go Test Failure \(internal\)](https://cockroachlabs.atlassian.net/l/c/HgfXfJgM)Parameters in this failure:
|
|
This should be fixed by: #73946. Note that I wasn't able to reproduce this locally even after 5m of runs. If this appears again, we could reopen the issue. |
|
Actually, I was able to reproduce this on my linux machine. It didn't work on Mac. Applied the changes from #73946, and it looks good: |
craig bot
pushed a commit
that referenced
this issue
Dec 17, 2021
73500: kv,storage: persist gateway node id in transaction intents r=AlexTalks a=AlexTalks This change augments the `TxnMeta` protobuf structure to include the gateway node ID (responsible for initiating the transaction) when serializing the intent. By doing so, this commit enables the Contention Event Store proposed in #71965, utilizing option 2. Release note: None 73862: sql: add test asserting CREATE/USAGE on public schema r=otan a=rafiss refs #70266 The public schema currently always has CREATE/USAGE privileges for the public role. Add a test that confirms this. Release note: None 73873: scdeps: tighten dependencies, log more side effects r=postamar a=postamar This commit reworks the dependency injection for the event logger, among other declarative schema changer dependencies. It also makes the test dependencies more chatty in the side effects log. Release note: None 73932: ui: select grants tab on table details page r=maryliag a=maryliag Previosuly, when the grants view was selected on the Database Details page, it was going to the Table Details with the Overview tab selected. With this commit, if the view mode selected is Grant, the grant tab is selected on the Table Details page. Fixes #68829 Release note: None 73943: cli: support --locality and --max-offset flags with sql tenant pods r=nvanbenschoten a=nvanbenschoten This commit adds support for the `--locality` and `--max-offset` flags to the `cockroach mt start-sql` command. The first of these is important because tenant SQL pods should know where they reside. This will be important in the future for multi-region serverless and also for projects like #72593. The second of these is important because the SQL pod's max-offset setting needs to be the same as the host cluster's. If we want to be able to configure the host cluster's maximum clock offset to some non-default value, we'll need SQL pods to be configured identically. Validation of plumbing: ```sh ./cockroach start-single-node --insecure --max-offset=250ms ./cockroach sql --insecure -e 'select crdb_internal.create_tenant(2)' # verify --max-offset ./cockroach mt start-sql --insecure --tenant-id=2 --sql-addr=:26258 --http-addr=:0 # CRDB crashes with error "locally configured maximum clock offset (250ms) does not match that of node [::]:62744 (500ms)" ./cockroach mt start-sql --insecure --tenant-id=2 --sql-addr=:26258 --http-addr=:0 --max-offset=250ms # successful # verify --locality ./cockroach sql --insecure --port=26258 -e 'select gateway_region()' ERROR: gateway_region(): no region set on the locality flag on this node ./cockroach mt start-sql --insecure --tenant-id=2 --sql-addr=:26258 --http-addr=:0 --max-offset=250ms --locality=region=us-east1 ./cockroach sql --insecure --port=26258 -e 'select gateway_region()' gateway_region ------------------ us-east1 ``` 73946: ccl/sqlproxyccl: fix TestWatchPods under stressrace r=jaylim-crl a=jaylim-crl Fixes #69220. Regression from #67452. In #67452, we omitted DRAINING pods from the tenant directory. Whenever a pod goes into the DRAINING state, the pod watcher needs time to update the directory. Not waiting for that while calling EnsureTenantAddr produces a stale result. This commit updates TestWatchPods by polling on EnsureTenantAddr until the pod watcher updates the directory. Release note: None 73954: sqlsmith: don't compare voids for joins r=rafiss a=otan No comparison expr is defined on voids, so don't generate comparisons for them. Resolves #73901 Resolves #73898 Resolves #73777 Release note: None Co-authored-by: Alex Sarkesian <[email protected]> Co-authored-by: Rafi Shamim <[email protected]> Co-authored-by: Marius Posta <[email protected]> Co-authored-by: Marylia Gutierrez <[email protected]> Co-authored-by: Nathan VanBenschoten <[email protected]> Co-authored-by: Jay <[email protected]> Co-authored-by: Oliver Tan <[email protected]>
gustasva
pushed a commit
to gustasva/cockroach
that referenced
this issue
Jan 4, 2022
Fixes cockroachdb#69220. Regression from cockroachdb#67452. In cockroachdb#67452, we omitted DRAINING pods from the tenant directory. Whenever a pod goes into the DRAINING state, the pod watcher needs time to update the directory. Not waiting for that while calling EnsureTenantAddr produces a stale result. This commit updates TestWatchPods by polling on EnsureTenantAddr until that has been completed. Release note: None
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
ccl/sqlproxyccl/tenant.TestWatchPods failed with artifacts on master @ d18da6c092bf1522e7a6478fe3973817e318c247:
Reproduce
To reproduce, try:
Parameters in this failure:
This test on roachdash | Improve this report!
The text was updated successfully, but these errors were encountered: