Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql/tests: TestRandomSyntaxSchemaChangeColumn failed #63393

Closed
rafiss opened this issue Apr 9, 2021 · 47 comments · Fixed by #64877
Closed

sql/tests: TestRandomSyntaxSchemaChangeColumn failed #63393

rafiss opened this issue Apr 9, 2021 · 47 comments · Fixed by #64877
Assignees
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered).

Comments

@rafiss
Copy link
Collaborator

rafiss commented Apr 9, 2021

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ 9a335c3b2eaac8a5b36ed4e4299be0bbb3d9f548:

[14:52:40]
[post issues]
[14:52:40]
[post issues]         	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
[14:52:40]
[post issues]         
[14:52:40]
[post issues]         goroutine 8553553 [select, 2 minutes]:
[14:52:40]
[post issues]         github.com/cockroachdb/cockroach/pkg/kv/kvserver/protectedts/ptcache.(*Cache).periodicallyRefreshProtectedtsCache(0xc01a8acbb0, 0x5bcca60, 0xc00b9329f0)
[14:52:40]
[post issues]         	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/protectedts/ptcache/cache.go:160 +0x25b
[14:52:40]
[post issues]         github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc007dda200, 0x5bcca60, 0xc00b9329f0, 0x0, 0xc017a32600)
[14:52:40]
[post issues]         	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
[14:52:40]
[post issues]         created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
[14:52:40]
[post issues]         	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
[14:52:40]
[post issues]         
[14:52:40]
[post issues]         goroutine 8553583 [select, 1 minutes]:
[14:52:40]
[post issues]         github.com/cockroachdb/cockroach/pkg/sql/catalog/lease.(*Manager).PeriodicallyRefreshSomeLeases.func1(0x5bcca60, 0xc00b9329f0)
[14:52:40]
[post issues]         	/go/src/github.com/cockroachdb/cockroach/pkg/sql/catalog/lease/lease.go:1808 +0x285
[14:52:40]
[post issues]         github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc007dda200, 0x5bcca60, 0xc00b9329f0, 0x0, 0xc017a32e50)
[14:52:40]
[post issues]         	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
[14:52:40]
[post issues]         created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
[14:52:40]
[post issues]         	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
[14:52:40]
[post issues]         
[14:52:40]
[post issues]         goroutine 8553324 [select, 5 minutes]:
[14:52:40]
[post issues]         google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc008ed7ac0)
[14:52:40]
[post issues]         	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:69 +0xc8
[14:52:40]
[post issues]         created by google.golang.org/grpc.newCCBalancerWrapper
[14:52:40]
[post issues]         	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:60 +0x172
[14:52:40]
[post issues]         
[14:52:40]
[post issues]         goroutine 8553286 [select, 5 minutes]:
[14:52:40]
[post issues]         github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x5bcca60, 0xc00b9329f0)
[14:52:40]
[post issues]         	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:105 +0x13d
[14:52:40]
[post issues]         github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc007dda200, 0x5bcca60, 0xc00b9329f0, 0x0, 0xc006286c10)
[14:52:40]
[post issues]         	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
[14:52:40]
[post issues]         created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
[14:52:40]
[post issues]         	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
[14:52:40]
[post issues]         
[14:52:40]
[post issues]         goroutine 8553316 [chan receive, 5 minutes]:
[14:52:40]
[post issues]         github.com/cockroachdb/cockroach/pkg/server.(*Server).startServeUI.func1(0x5bcca60, 0xc00b9329f0)
[14:52:40]
[post issues]         	/go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:2022 +0x53
[14:52:40]
[post issues]         github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc007dda200, 0x5bcca60, 0xc00b9329f0, 0x0, 0xc004556160)
[14:52:40]
[post issues]         	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
[14:52:40]
[post issues]         created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
[14:52:40]
[post issues]         	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
[14:52:40]
[post issues]         
[14:52:40]
[post issues]     rsg_test.go:186: SELECT 1 executed successfully: probably a slow statement
[14:52:40]
[post issues]     rsg_test.go:755: Crash detected: 
[14:52:40]
[post issues]         ALTER TABLE ident.ident ADD IF NOT EXISTS SURVIVE INTERVAL DAY TO MINUTE [ ]
[14:52:40]
[post issues]         
[14:52:40]
[post issues]         Stack trace:
[14:52:40]
[post issues]         timeout: "ALTER TABLE ident.ident ADD IF NOT EXISTS SURVIVE INTERVAL DAY TO MINUTE [ ]". currently executing: map[ALTER TABLE ident.ident ADD IF NOT EXISTS SURVIVE INTERVAL DAY TO MINUTE [ ]:1]
[14:52:40]
[post issues]     rsg_test.go:764: 6923 executions, 1292 successful
[14:52:40]
[post issues]     rsg_test.go:401: -- test log scope end --
[14:52:40]
[post issues] test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestRandomSyntaxSchemaChangeColumn880718491
[14:52:40]
[post issues] --- FAIL: TestRandomSyntaxSchemaChangeColumn (357.99s)
[14:52:40]
[post issues] ```
[14:52:40]
[post issues] <details><summary>Reproduce</summary>
[14:52:40]
[post issues] <p>
[14:52:40]
[post issues] <p>To reproduce, try:
[14:52:40]
[post issues] ```bash
[14:52:40]
[post issues] make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1
[14:52:40]
[post issues] ```
[14:52:40]
[post issues] </p>
[14:52:40]
[post issues] </p>
[14:52:40]
[post issues] </details>
[14:52:40]
[post issues] <details><summary>Same failure on other branches</summary>
[14:52:40]
[post issues] <p>
[14:52:40]
[post issues] - #63202 sql/tests: TestRandomSyntaxSchemaChangeColumn failed [C-test-failure O-robot branch-release-20.1]</p>
[14:52:40]
[post issues] </details>
[14:52:40]
[post issues] [See this test on roachdash](https://roachdash.crdb.dev/?filter=status:open%20t:.*TestRandomSyntaxSchemaChangeColumn.*&sort=title+created&display=lastcommented+project)
[14:52:40]
[post issues] <sub>[Improve this report!](https://github.com/cockroachdb/cockroach/tree/master/pkg/cmd/internal/issues)</sub>
[14:52:40]
[post issues] ", Labels:["O-robot" "C-test-failure" "branch-master"], Assignee:"mjibson"}: POST https://api.github.com/repos/cockroachdb/cockroach/issues: 422 Validation Failed [{Resource:Issue Field:assignee Code:invalid Message:}]
[14:52:40]
[Run Random Syntax tests] artifacts (7s)
[14:52:47]
[Run Random Syntax tests] exit status
[14:52:47]
[Run Random Syntax tests] Process exited with code 2
[14:52:47]
[Step 2/2] Process exited with code 2 (Step: Command Line)
[14:52:47]
[Step 2/2] Step Command Line failed
[14:52:47]
Publishing internal artifacts (4s)
[14:52:47]
Publishing artifacts (5s)
[14:52:53]
Build finished

cc @ajwerner

@rafiss rafiss added C-test-failure Broken test (automatically or manually discovered). branch-master Failures and bugs on the master branch. labels Apr 9, 2021
@rafiss
Copy link
Collaborator Author

rafiss commented Apr 9, 2021

I'm creating this manually since it looks like the automatic issue posting failed.

[post issues] ", Labels:["O-robot" "C-test-failure" "branch-master"], Assignee:"mjibson"}: POST https://api.github.com/repos/cockroachdb/cockroach/issues: 422 Validation Failed [{Resource:Issue Field:assignee Code:invalid Message:}]

@tbg do you happen to know if there's something special to be done to make this test not try to assign Matt to the issue? I'm assuming the posting failed because he no longer is in the Github org, but I thought your recent changes got rid of issue assignees entirely.

@tbg
Copy link
Member

tbg commented Apr 9, 2021

Look at oldFriendsMap in issues.go.

Weird, my recent changes should have made the issue poster more resilient to invalid assignee, not less :-) I'll take a look early next week.

@tbg
Copy link
Member

tbg commented Apr 9, 2021

Ah I see what this is. Matt's Github login works and is found alright, but you can't assign cockroachdb/cockroach projects to him any more, so the creation fails. Sneaky.

tbg added a commit to tbg/cockroach that referenced this issue Apr 9, 2021
We've wanted to move off assigning authors for a while. It turns out
that I also just [broke] something about the assignments, so it's a
convenient time to make the switch.

We still ping the author on the issue, which is going to stay that
way until we've put the plumbing in place that will allow mentioning
and assigning into the project of teams from unit tests too (to date
this is only done from roachtest).

[broke]: cockroachdb#63393 (comment)

Release note: None
@tbg
Copy link
Member

tbg commented Apr 9, 2021

#63202 now fixes this in a commit I just pushed, by not even trying to assign these issues to the authors any more. It simply mentions the author in the comments instead, so it wouldn't have that issue. I mapped Matt to you, Rafi, in that same commit. Enjoy your weekend!

tbg added a commit to tbg/cockroach that referenced this issue Apr 12, 2021
We've wanted to move off assigning authors for a while. It turns out
that I also just [broke] something about the assignments, so it's a
convenient time to make the switch.

We still ping the author on the issue, which is going to stay that
way until we've put the plumbing in place that will allow mentioning
and assigning into the project of teams from unit tests too (to date
this is only done from roachtest).

[broke]: cockroachdb#63393 (comment)

Release note: None
@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ 7495d19e50d8fd14140220f3ba5c7ad951abafa6:

        goroutine 8914744 [select]:
        google.golang.org/grpc/internal/transport.(*controlBuffer).get(0xc00983ce60, 0x1, 0x0, 0x0, 0x0, 0x0)
        	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:395 +0x125
        google.golang.org/grpc/internal/transport.(*loopyWriter).run(0xc01434b500, 0x0, 0x0)
        	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/controlbuf.go:515 +0x1d3
        google.golang.org/grpc/internal/transport.newHTTP2Client.func3(0xc00f7fda40)
        	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_client.go:356 +0x7b
        created by google.golang.org/grpc/internal/transport.newHTTP2Client
        	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_client.go:354 +0x123c
        
        goroutine 8908392 [select]:
        github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).Start.func8(0x5bddd00, 0xc000074120)
        	/go/src/github.com/cockroachdb/cockroach/pkg/jobs/registry.go:727 +0x20d
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc011432200, 0x5bddd00, 0xc000074120, 0x0, 0xc013cf9e90)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8978568 [chan send, 1 minutes]:
        github.com/lib/pq.(*conn).watchCancel.func1(0xc0066e8240, 0xc00e1a5b80, 0xc00c457b00)
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/lib/pq/conn_go18.go:104 +0x185
        created by github.com/lib/pq.(*conn).watchCancel
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/lib/pq/conn_go18.go:93 +0xa5
        
        goroutine 8908331 [select, 5 minutes]:
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).addToPurgatoryLocked.func2(0x5bddd80, 0xc013c16990)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/queue.go:1152 +0x346
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc011432200, 0x5bddd80, 0xc013c16990, 0x0, 0xc021940a40)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8908120 [select, 5 minutes]:
        github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x5bddd80, 0xc0116e6600)
        	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:105 +0x13d
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc011432200, 0x5bddd80, 0xc0116e6600, 0x0, 0xc0057dafe0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
    rsg_test.go:186: SELECT 1 executed successfully: probably a slow statement
    rsg_test.go:755: Crash detected: 
        ALTER TABLE ident.ident ADD COLUMN ident INTERVAL CREATE IF NOT EXISTS FAMILY ident
        
        Stack trace:
        timeout: "ALTER TABLE ident.ident ADD COLUMN ident INTERVAL CREATE IF NOT EXISTS FAMILY ident". currently executing: map[ALTER TABLE ident.ident ADD COLUMN ident INTERVAL CREATE IF NOT EXISTS FAMILY ident:1]
    rsg_test.go:764: 7997 executions, 1473 successful
    rsg_test.go:401: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestRandomSyntaxSchemaChangeColumn788688690
--- FAIL: TestRandomSyntaxSchemaChangeColumn (355.99s)
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @rafiss

This test on roachdash | Improve this report!

@tbg
Copy link
Member

tbg commented Apr 14, 2021

^-- nice, that worked.

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ e29d694755e4800f07df139137c71cf741829dd0:

        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8980127 [sync.Cond.Wait]:
        runtime.goparkunlock(...)
        	/usr/local/go/src/runtime/proc.go:312
        sync.runtime_notifyListWait(0xc00d3b8590, 0x418d)
        	/usr/local/go/src/runtime/sema.go:513 +0xf8
        sync.(*Cond).Wait(0xc00d3b8580)
        	/usr/local/go/src/sync/cond.go:56 +0x9d
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc022510320, 0x5c150c0, 0xc019bd9590)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:251 +0x8b
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc022a0e200, 0x5c150c0, 0xc019bd9590, 0x0, 0xc006825b40)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 9062374 [select]:
        github.com/cockroachdb/cockroach/pkg/internal/client/requestbatcher.(*RequestBatcher).Send(0xc000fc4000, 0x5c15040, 0xc000074120, 0xb, 0x5d51240, 0xc00e13c3c0, 0x0, 0x1, 0x0, 0x7600000000000000)
        	/go/src/github.com/cockroachdb/cockroach/pkg/internal/client/requestbatcher/batcher.go:248 +0x1fc
        github.com/cockroachdb/cockroach/pkg/kv/kvserver/intentresolver.(*IntentResolver).gcTxnRecord(0xc026e22680, 0x5c15040, 0xc000074120, 0xb, 0xc0037aefc0, 0xce6f8a95823c9f9a, 0x0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/intentresolver/intent_resolver.go:716 +0x35e
        github.com/cockroachdb/cockroach/pkg/kv/kvserver/intentresolver.(*IntentResolver).cleanupFinishedTxnIntents.func2(0x5c15040, 0xc000074120)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/intentresolver/intent_resolver.go:752 +0x68
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc022a0e200, 0x5c15040, 0xc000074120, 0x0, 0xc00eff4330)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8980213 [select, 5 minutes]:
        github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord.(*DistSender).RangeFeed.func1(0x5c15000, 0xc02f1bc580, 0x100000001, 0x70000c034321c20)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/kvcoord/dist_sender_rangefeed.go:66 +0x14d
        github.com/cockroachdb/cockroach/pkg/util/ctxgroup.Group.GoCtx.func1(0xc022a0e200, 0x5495998)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/ctxgroup/ctxgroup.go:166 +0x3a
        golang.org/x/sync/errgroup.(*Group).Go.func1(0xc02901a1b0, 0xc02901a1e0)
        	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x59
        created by golang.org/x/sync/errgroup.(*Group).Go
        	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x66
        
    rsg_test.go:186: SELECT 1 executed successfully: probably a slow statement
    rsg_test.go:755: Crash detected: 
        ALTER TABLE ident.ident DROP IF EXISTS AFTER CASCADE
        
        Stack trace:
        timeout: "ALTER TABLE ident.ident DROP IF EXISTS AFTER CASCADE". currently executing: map[ALTER TABLE ident.ident DROP IF EXISTS AFTER CASCADE:1]
    rsg_test.go:764: 9250 executions, 1682 successful
    rsg_test.go:401: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestRandomSyntaxSchemaChangeColumn603151278
--- FAIL: TestRandomSyntaxSchemaChangeColumn (350.59s)
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @rafiss

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ e29d694755e4800f07df139137c71cf741829dd0:

        goroutine 8251892 [select, 5 minutes]:
        github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x5c150c0, 0xc0134e0660)
        	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:105 +0x13d
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc009cdc380, 0x5c150c0, 0xc0134e0660, 0x0, 0xc01882eee0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8251891 [select, 5 minutes]:
        github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x5c150c0, 0xc0134e0660)
        	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:105 +0x13d
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc009cdc380, 0x5c150c0, 0xc0134e0660, 0x0, 0xc01882eed0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8251902 [chan receive, 5 minutes]:
        github.com/cockroachdb/cmux.muxListener.Accept(...)
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/cmux/cmux.go:184
        crypto/tls.(*listener).Accept(0xc004242240, 0x4a9c380, 0xc01022c390, 0x464d140, 0x7d001c0)
        	/usr/local/go/src/crypto/tls/tls.go:67 +0x37
        net/http.(*Server).Serve(0xc024ad0e00, 0x5bce480, 0xc004242240, 0x0, 0x0)
        	/usr/local/go/src/net/http/server.go:2937 +0x266
        github.com/cockroachdb/cockroach/pkg/server.(*Server).startServeUI.func4(0x5c150c0, 0xc0134e0660)
        	/go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:2070 +0x3f
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc009cdc380, 0x5c150c0, 0xc0134e0660, 0x0, 0xc004242260)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8251896 [select]:
        github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed.(*RangeFeed).processEvents(0xc000983450, 0x5c15000, 0xc00850d600, 0xc0074ae360, 0xc00e4ae060, 0xc00e4ae0c0, 0x0, 0x0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed/rangefeed.go:303 +0x129
        github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed.(*RangeFeed).run(0xc000983450, 0x5c15000, 0xc00850d600)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed/rangefeed.go:214 +0x6e5
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc009cdc380, 0x5c15000, 0xc00850d600, 0x0, 0xc01882ef70)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
    rsg_test.go:186: SELECT 1 executed successfully: probably a slow statement
    rsg_test.go:755: Crash detected: 
        ALTER TABLE ident.ident ALTER COLUMN LEAST SET NOT NULL
        
        Stack trace:
        timeout: "ALTER TABLE ident.ident ALTER COLUMN LEAST SET NOT NULL". currently executing: map[ALTER TABLE ident.ident ALTER COLUMN LEAST SET NOT NULL:1]
    rsg_test.go:764: 5942 executions, 1091 successful
    rsg_test.go:401: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestRandomSyntaxSchemaChangeColumn392923736
--- FAIL: TestRandomSyntaxSchemaChangeColumn (349.67s)
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @rafiss

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ 3aad3f8a372090ddaedad0ad58a945fb399ddbc6:

        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:251 +0x8b
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc00bf4c280, 0x5c167a0, 0xc03b617050, 0x0, 0xc00261fd50)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8688715 [select, 5 minutes]:
        github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x5c167a0, 0xc03a4e0600)
        	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:105 +0x13d
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc00bf4c280, 0x5c167a0, 0xc03a4e0600, 0x0, 0xc00eace630)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8688707 [chan receive, 5 minutes]:
        github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop.func1(0x5c167a0, 0xc0328c0330)
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:176 +0x76
        runtime/pprof.Do(0x5c167a0, 0xc0328c0330, 0xc0003415a0, 0x1, 0x1, 0xc005a16fb8)
        	/usr/local/go/src/runtime/pprof/runtime.go:40 +0xcc
        github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop(0xc004e4f000)
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:175 +0x89
        created by github.com/cockroachdb/pebble.(*tableCacheShard).init
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:167 +0x1a5
        
        goroutine 8688753 [select, 5 minutes]:
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processLoop.func2(0x5c167a0, 0xc03b616c60)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/queue.go:818 +0x15f
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc00bf4c280, 0x5c167a0, 0xc03b616c60, 0x0, 0xc02fe46b40)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8688734 [chan receive, 5 minutes]:
        github.com/cockroachdb/cockroach/pkg/server.(*Server).startServeUI.func1(0x5c167a0, 0xc03a4e0600)
        	/go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:2022 +0x53
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc00bf4c280, 0x5c167a0, 0xc03a4e0600, 0x0, 0xc021999de0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
    rsg_test.go:186: SELECT 1 executed successfully: probably a slow statement
    rsg_test.go:755: Crash detected: 
        ALTER TABLE ident.ident ALTER VIRTUAL SET NOT NULL
        
        Stack trace:
        timeout: "ALTER TABLE ident.ident ALTER VIRTUAL SET NOT NULL". currently executing: map[ALTER TABLE ident.ident ALTER VIRTUAL SET NOT NULL:1]
    rsg_test.go:764: 9204 executions, 1827 successful
    rsg_test.go:401: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestRandomSyntaxSchemaChangeColumn347323964
--- FAIL: TestRandomSyntaxSchemaChangeColumn (347.35s)
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @rafiss

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ 4dc05cceea254854317a9374a4b88df06c0946a6:

        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/store_raft.go:649 +0x265
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc00e406200, 0x5c19900, 0xc0114a1ad0, 0x0, 0xc00fa056c0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8184341 [sync.Cond.Wait]:
        runtime.goparkunlock(...)
        	/usr/local/go/src/runtime/proc.go:312
        sync.runtime_notifyListWait(0xc007714350, 0x2fff)
        	/usr/local/go/src/runtime/sema.go:513 +0xf8
        sync.(*Cond).Wait(0xc007714340)
        	/usr/local/go/src/sync/cond.go:56 +0x9d
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc00be380a0, 0x5c19900, 0xc0114a1ad0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:251 +0x8b
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc00e406200, 0x5c19900, 0xc0114a1ad0, 0x0, 0xc00fa052b0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8184398 [sync.Cond.Wait]:
        runtime.goparkunlock(...)
        	/usr/local/go/src/runtime/proc.go:312
        sync.runtime_notifyListWait(0xc007714350, 0x2ffe)
        	/usr/local/go/src/runtime/sema.go:513 +0xf8
        sync.(*Cond).Wait(0xc007714340)
        	/usr/local/go/src/sync/cond.go:56 +0x9d
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc00be380a0, 0x5c19900, 0xc0114a1ad0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:251 +0x8b
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc00e406200, 0x5c19900, 0xc0114a1ad0, 0x0, 0xc00fa05640)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8190407 [select]:
        google.golang.org/grpc/internal/transport.(*http2Client).keepalive(0xc01c90d180)
        	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_client.go:1362 +0x193
        created by google.golang.org/grpc/internal/transport.newHTTP2Client
        	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_client.go:305 +0x1a3c
        
    rsg_test.go:186: SELECT 1 executed successfully: probably a slow statement
    rsg_test.go:755: Crash detected: 
        ALTER TABLE ident.ident ADD COLUMN IF NOT EXISTS OIDS INTERVAL
        
        Stack trace:
        timeout: "ALTER TABLE ident.ident ADD COLUMN IF NOT EXISTS OIDS INTERVAL". currently executing: map[ALTER TABLE ident.ident ADD COLUMN IF NOT EXISTS OIDS INTERVAL:1]
    rsg_test.go:764: 5979 executions, 1154 successful
    rsg_test.go:401: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestRandomSyntaxSchemaChangeColumn274973818
--- FAIL: TestRandomSyntaxSchemaChangeColumn (354.76s)
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @rafiss

This test on roachdash | Improve this report!

@ajwerner
Copy link
Contributor

@rafiss do you remember why you tagged me on this? @fqazi could you repro this and figure out what's going on?

@rafiss
Copy link
Collaborator Author

rafiss commented Apr 21, 2021

I think my initial tag was just to let you know that there was an issue with the automated issue posting from this test.

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ 66a1657247f8e1cd84ce22a32321fda6352d789f:

        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8204139 [select, 5 minutes]:
        github.com/cockroachdb/cockroach/pkg/security.(*CertificateManager).RegisterSignalHandler.func1(0xc01b5f2300, 0x5c40720, 0xc000074120, 0xc01b032000)
        	/go/src/github.com/cockroachdb/cockroach/pkg/security/certificate_manager.go:235 +0x159
        created by github.com/cockroachdb/cockroach/pkg/security.(*CertificateManager).RegisterSignalHandler
        	/go/src/github.com/cockroachdb/cockroach/pkg/security/certificate_manager.go:232 +0x65
        
        goroutine 8204141 [chan receive, 5 minutes]:
        github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop.func1(0x5c407a0, 0xc003b7a3f0)
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:176 +0x76
        runtime/pprof.Do(0x5c407a0, 0xc003b7a3f0, 0xc0005a47a0, 0x1, 0x1, 0xc0119057b8)
        	/usr/local/go/src/runtime/pprof/runtime.go:40 +0xcc
        github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop(0xc012d78e00)
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:175 +0x89
        created by github.com/cockroachdb/pebble.(*tableCacheShard).init
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:167 +0x1a5
        
        goroutine 8204167 [select]:
        github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed.(*RangeFeed).processEvents(0xc012b6a340, 0x5c406e0, 0xc02810de00, 0xc00b1200f0, 0xc019978060, 0xc0199780c0, 0x0, 0x0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed/rangefeed.go:303 +0x129
        github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed.(*RangeFeed).run(0xc012b6a340, 0x5c406e0, 0xc02810de00)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvclient/rangefeed/rangefeed.go:214 +0x6e5
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc01b5f2300, 0x5c406e0, 0xc02810de00, 0x0, 0xc01d352fd0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8204355 [select]:
        github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*registration).outputLoop(0xc023fc7ea0, 0x5c406e0, 0xc023494940, 0xc023494940, 0xc01d353fa0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/registry.go:254 +0x1f9
        github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*registration).runOutputLoop(0xc023fc7ea0, 0x5c406e0, 0xc023494940)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/registry.go:271 +0xc5
        github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed.(*Processor).run.func1(0x5c406e0, 0xc023494800)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/rangefeed/processor.go:259 +0x65
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc01b5f2300, 0x5c406e0, 0xc023494800, 0x0, 0xc013615920)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
    rsg_test.go:186: SELECT 1 executed successfully: probably a slow statement
    rsg_test.go:755: Crash detected: 
        ALTER TABLE ident.ident ALTER GOAL DROP NOT NULL
        
        Stack trace:
        timeout: "ALTER TABLE ident.ident ALTER GOAL DROP NOT NULL". currently executing: map[ALTER TABLE ident.ident ALTER GOAL DROP NOT NULL:1]
    rsg_test.go:764: 9985 executions, 1881 successful
    rsg_test.go:401: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestRandomSyntaxSchemaChangeColumn906812222
--- FAIL: TestRandomSyntaxSchemaChangeColumn (352.83s)
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @rafiss

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ 5f40d691ef9caf910f97fbad53802000c0ca72eb:

        	/usr/local/go/src/runtime/proc.go:312
        sync.runtime_notifyListWait(0xc021c041d0, 0x255e)
        	/usr/local/go/src/runtime/sema.go:513 +0xf8
        sync.(*Cond).Wait(0xc021c041c0)
        	/usr/local/go/src/sync/cond.go:56 +0x9d
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc0211dc140, 0x5d07440, 0xc0173b2db0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:251 +0x8b
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc023b8c300, 0x5d07440, 0xc0173b2db0, 0x0, 0xc0143f5ca0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8076634 [sync.Cond.Wait]:
        runtime.goparkunlock(...)
        	/usr/local/go/src/runtime/proc.go:312
        sync.runtime_notifyListWait(0xc021c041d0, 0x257a)
        	/usr/local/go/src/runtime/sema.go:513 +0xf8
        sync.(*Cond).Wait(0xc021c041c0)
        	/usr/local/go/src/sync/cond.go:56 +0x9d
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc0211dc140, 0x5d07440, 0xc0173b2db0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:251 +0x8b
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc023b8c300, 0x5d07440, 0xc0173b2db0, 0x0, 0xc0143f5e90)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8083655 [select, 5 minutes]:
        google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc01ef53f40)
        	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:69 +0xc8
        created by google.golang.org/grpc.newCCBalancerWrapper
        	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:60 +0x172
        
        goroutine 8076723 [select, 5 minutes]:
        github.com/cockroachdb/cockroach/pkg/sql.(*Server).PeriodicallyClearSQLStats.func1(0x5d07440, 0xc0169586c0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:757 +0x265
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc023b8c300, 0x5d07440, 0xc0169586c0, 0x0, 0xc004b782a0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
    rsg_test.go:186: SELECT 1 executed successfully: probably a slow statement
    rsg_test.go:755: Crash detected: 
        ALTER TABLE ident.ident DROP STRING
        
        Stack trace:
        timeout: "ALTER TABLE ident.ident DROP STRING". currently executing: map[ALTER TABLE ident.ident DROP STRING:1]
    rsg_test.go:764: 4276 executions, 770 successful
    rsg_test.go:401: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestRandomSyntaxSchemaChangeColumn419597926
--- FAIL: TestRandomSyntaxSchemaChangeColumn (359.79s)
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @rafiss

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ 9e821aa456bfbe8b5ec3a8ef79ae3954d52cb675:

        	/usr/local/go/src/net/http/server.go:2937 +0x266
        github.com/cockroachdb/cockroach/pkg/server.(*Server).startServeUI.func4(0x5d11280, 0xc01c8ba7b0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:2071 +0x3f
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc00f010200, 0x5d11280, 0xc01c8ba7b0, 0x0, 0xc014669d80)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8435279 [sync.Cond.Wait]:
        runtime.goparkunlock(...)
        	/usr/local/go/src/runtime/proc.go:312
        sync.runtime_notifyListWait(0xc0134440d0, 0x3065)
        	/usr/local/go/src/runtime/sema.go:513 +0xf8
        sync.(*Cond).Wait(0xc0134440c0)
        	/usr/local/go/src/sync/cond.go:56 +0x9d
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc0017d3220, 0x5d11280, 0xc004843230)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:251 +0x8b
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc00f010200, 0x5d11280, 0xc004843230, 0x0, 0xc003c79d70)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8435200 [select, 5 minutes]:
        github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x5d11280, 0xc01c8ba7b0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:103 +0x13d
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc00f010200, 0x5d11280, 0xc01c8ba7b0, 0x0, 0xc00980ecb0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8435178 [chan receive, 5 minutes]:
        github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop.func1(0x5d11280, 0xc011b8e210)
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:176 +0x76
        runtime/pprof.Do(0x5d11280, 0xc011b8e210, 0xc0003415c0, 0x1, 0x1, 0xc0316d3fb8)
        	/usr/local/go/src/runtime/pprof/runtime.go:40 +0xcc
        github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop(0xc004ef0500)
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:175 +0x89
        created by github.com/cockroachdb/pebble.(*tableCacheShard).init
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:167 +0x1a5
        
    rsg_test.go:186: SELECT 1 executed successfully: probably a slow statement
    rsg_test.go:755: Crash detected: 
        ALTER TABLE ident.ident ADD IF NOT EXISTS RESUME INTERVAL DAY ARRAY
        
        Stack trace:
        timeout: "ALTER TABLE ident.ident ADD IF NOT EXISTS RESUME INTERVAL DAY ARRAY". currently executing: map[ALTER TABLE ident.ident ADD IF NOT EXISTS RESUME INTERVAL DAY ARRAY:1]
    rsg_test.go:764: 5835 executions, 1076 successful
    rsg_test.go:401: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestRandomSyntaxSchemaChangeColumn317778154
--- FAIL: TestRandomSyntaxSchemaChangeColumn (357.60s)
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @rafiss

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ 3cd29547884feb24b150b67ca2a88073798a35fd:

        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/queue.go:818 +0x15f
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc00ce4a300, 0x5ca5ea0, 0xc00b7c6f90, 0x0, 0xc060f46100)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 9022981 [select]:
        github.com/cockroachdb/cockroach/pkg/util/retry.(*Retry).Next(0xc0026cb350, 0x1)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/retry/retry.go:128 +0x187
        github.com/cockroachdb/cockroach/pkg/sql.schemaChangeResumer.Resume.func1(0x7100000035, 0xc000000000, 0xc0173b5280, 0x5d0e3c0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/sql/schema_changer.go:2153 +0x5f3
        github.com/cockroachdb/cockroach/pkg/sql.schemaChangeResumer.Resume(0xc00d64bb80, 0x5ca5ea0, 0xc015773ad0, 0x4aac980, 0xc0149e6400, 0xc00d64bb80, 0xc0026cb830)
        	/go/src/github.com/cockroachdb/cockroach/pkg/sql/schema_changer.go:2274 +0x803
        github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).stepThroughStateMachine.func1(0xc054771e00, 0x5c4d860, 0xc0149e63f0, 0x5ca5ea0, 0xc015773ad0, 0x4aac980, 0xc0149e6400, 0xc0026cba60)
        	/go/src/github.com/cockroachdb/cockroach/pkg/jobs/registry.go:1150 +0xae
        github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).stepThroughStateMachine(0xc002f9a4e0, 0x5ca5ea0, 0xc0157739e0, 0x4aac980, 0xc0149e6400, 0x5c4d860, 0xc0149e63f0, 0xc00d64bb80, 0xc00d0cebd0, 0x7, ...)
        	/go/src/github.com/cockroachdb/cockroach/pkg/jobs/registry.go:1151 +0x877
        github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).runJob(0xc002f9a4e0, 0x5ca5de0, 0xc002a32280, 0x5c4d860, 0xc0149e63f0, 0xc00d64bb80, 0xc00d0cebd0, 0x7, 0xc0092274a0, 0x16, ...)
        	/go/src/github.com/cockroachdb/cockroach/pkg/jobs/adopt.go:255 +0x378
        github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).resumeJob.func1(0x5ca5de0, 0xc000c5b0c0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/jobs/adopt.go:214 +0x165
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc00ce4a300, 0x5ca5de0, 0xc000c5b0c0, 0x0, 0xc05be280a0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8973730 [sync.Cond.Wait]:
        runtime.goparkunlock(...)
        	/usr/local/go/src/runtime/proc.go:312
        sync.runtime_notifyListWait(0xc012ce8250, 0x23f2)
        	/usr/local/go/src/runtime/sema.go:513 +0xf8
        sync.(*Cond).Wait(0xc012ce8240)
        	/usr/local/go/src/sync/cond.go:56 +0x9d
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc00f568140, 0x5ca5ea0, 0xc00b7c73e0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:251 +0x8b
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc00ce4a300, 0x5ca5ea0, 0xc00b7c73e0, 0x0, 0xc0409799d0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
    rsg_test.go:186: SELECT 1 executed successfully: probably a slow statement
    rsg_test.go:755: Crash detected: 
        ALTER TABLE ident.ident ADD IF NOT EXISTS VOTERS BIT ( 442 ) ARRAY
        
        Stack trace:
        timeout: "ALTER TABLE ident.ident ADD IF NOT EXISTS VOTERS BIT ( 442 ) ARRAY". currently executing: map[ALTER TABLE ident.ident ADD IF NOT EXISTS VOTERS BIT ( 442 ) ARRAY:1]
    rsg_test.go:764: 4448 executions, 825 successful
    rsg_test.go:401: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestRandomSyntaxSchemaChangeColumn690796522
--- FAIL: TestRandomSyntaxSchemaChangeColumn (356.78s)
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @rafiss

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ 2d7f9a08bac49040a6d77c5465e2b4fd8fdac1f4:

        	/go/src/github.com/cockroachdb/cockroach/pkg/util/retry/retry.go:128 +0x187
        github.com/cockroachdb/cockroach/pkg/sql.schemaChangeResumer.Resume.func1(0x8d00000035, 0xc000000000, 0xc00d67d100, 0x5d0e540)
        	/go/src/github.com/cockroachdb/cockroach/pkg/sql/schema_changer.go:2159 +0x5f3
        github.com/cockroachdb/cockroach/pkg/sql.schemaChangeResumer.Resume(0xc015e69e00, 0x5ca6020, 0xc006e40f00, 0x4aaca40, 0xc00ef7a948, 0xc015e69e00, 0xc0012af830)
        	/go/src/github.com/cockroachdb/cockroach/pkg/sql/schema_changer.go:2280 +0x803
        github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).stepThroughStateMachine.func1(0xc006a28200, 0x5c4d9e0, 0xc00ef7a938, 0x5ca6020, 0xc006e40f00, 0x4aaca40, 0xc00ef7a948, 0xc002b13a60)
        	/go/src/github.com/cockroachdb/cockroach/pkg/jobs/registry.go:1150 +0xae
        github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).stepThroughStateMachine(0xc000f43860, 0x5ca6020, 0xc006e40e40, 0x4aaca40, 0xc00ef7a948, 0x5c4d9e0, 0xc00ef7a938, 0xc015e69e00, 0xc01d66d840, 0x7, ...)
        	/go/src/github.com/cockroachdb/cockroach/pkg/jobs/registry.go:1151 +0x877
        github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).runJob(0xc000f43860, 0x5ca5f60, 0xc029e0a3c0, 0x5c4d9e0, 0xc00ef7a938, 0xc015e69e00, 0xc01d66d840, 0x7, 0xc003f72a20, 0x16, ...)
        	/go/src/github.com/cockroachdb/cockroach/pkg/jobs/adopt.go:255 +0x378
        github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).resumeJob.func1(0x5ca5f60, 0xc00bc34f00)
        	/go/src/github.com/cockroachdb/cockroach/pkg/jobs/adopt.go:214 +0x165
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc024946300, 0x5ca5f60, 0xc00bc34f00, 0x0, 0xc00f3663c0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8611774 [select, 5 minutes]:
        github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x5ca6020, 0xc02d57a660)
        	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:103 +0x13d
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc024946300, 0x5ca6020, 0xc02d57a660, 0x0, 0xc009271b40)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8611825 [sync.Cond.Wait]:
        runtime.goparkunlock(...)
        	/usr/local/go/src/runtime/proc.go:312
        sync.runtime_notifyListWait(0xc00bff20d0, 0x2e02)
        	/usr/local/go/src/runtime/sema.go:513 +0xf8
        sync.(*Cond).Wait(0xc00bff20c0)
        	/usr/local/go/src/sync/cond.go:56 +0x9d
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc00074d7c0, 0x5ca6020, 0xc01a9c76e0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:251 +0x8b
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc024946300, 0x5ca6020, 0xc01a9c76e0, 0x0, 0xc01afebc20)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
    rsg_test.go:186: SELECT 1 executed successfully: probably a slow statement
    rsg_test.go:755: Crash detected: 
        ALTER TABLE ident.ident ALTER COLUMN ident SET NOT NULL
        
        Stack trace:
        timeout: "ALTER TABLE ident.ident ALTER COLUMN ident SET NOT NULL". currently executing: map[ALTER TABLE ident.ident ALTER COLUMN ident SET NOT NULL:1]
    rsg_test.go:764: 5247 executions, 998 successful
    rsg_test.go:401: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestRandomSyntaxSchemaChangeColumn324191524
--- FAIL: TestRandomSyntaxSchemaChangeColumn (360.02s)
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @rafiss

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ 2dbd33ce319439bc424a52896a03e0a7ccff07f1:

        	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:57 +0x59
        created by golang.org/x/sync/errgroup.(*Group).Go
        	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:54 +0x66
        
        goroutine 8551264 [select, 5 minutes]:
        google.golang.org/grpc.(*ccBalancerWrapper).watcher(0xc01b065c40)
        	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:69 +0xc8
        created by google.golang.org/grpc.newCCBalancerWrapper
        	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/balancer_conn_wrappers.go:60 +0x172
        
        goroutine 8544926 [sync.Cond.Wait]:
        runtime.goparkunlock(...)
        	/usr/local/go/src/runtime/proc.go:312
        sync.runtime_notifyListWait(0xc02c12c550, 0x2a90)
        	/usr/local/go/src/runtime/sema.go:513 +0xf8
        sync.(*Cond).Wait(0xc02c12c540)
        	/usr/local/go/src/sync/cond.go:56 +0x9d
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc03cdea3c0, 0x5ca6220, 0xc0080e19e0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:251 +0x8b
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc033be8200, 0x5ca6220, 0xc0080e19e0, 0x0, 0xc01bfc1c50)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8544880 [select, 5 minutes]:
        github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x5ca6220, 0xc017950720)
        	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:103 +0x13d
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc033be8200, 0x5ca6220, 0xc017950720, 0x0, 0xc00622ab20)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8545066 [select]:
        github.com/cockroachdb/cockroach/pkg/sql/stmtdiagnostics.(*Registry).poll(0xc01e4aeae0, 0x5ca6160, 0xc00f262c00)
        	/go/src/github.com/cockroachdb/cockroach/pkg/sql/stmtdiagnostics/statement_diagnostics.go:147 +0x2e5
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc033be8200, 0x5ca6160, 0xc00f262c00, 0x0, 0xc0032cc860)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
    rsg_test.go:186: SELECT 1 executed successfully: probably a slow statement
    rsg_test.go:755: Crash detected: 
        ALTER TABLE ident.ident ADD COLUMN IF NOT EXISTS ident STRING ( 359 ) ARRAY [ 132 ]
        
        Stack trace:
        timeout: "ALTER TABLE ident.ident ADD COLUMN IF NOT EXISTS ident STRING ( 359 ) ARRAY [ 132 ]". currently executing: map[ALTER TABLE ident.ident ADD COLUMN IF NOT EXISTS ident STRING ( 359 ) ARRAY [ 132 ]:1]
    rsg_test.go:764: 5337 executions, 983 successful
    rsg_test.go:401: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestRandomSyntaxSchemaChangeColumn509228950
--- FAIL: TestRandomSyntaxSchemaChangeColumn (357.46s)
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @rafiss

This test on roachdash | Improve this report!

tbg added a commit to tbg/cockroach that referenced this issue Apr 26, 2021
We've wanted to move off assigning authors for a while. It turns out
that I also just [broke] something about the assignments, so it's a
convenient time to make the switch.

We still ping the author on the issue, which is going to stay that
way until we've put the plumbing in place that will allow mentioning
and assigning into the project of teams from unit tests too (to date
this is only done from roachtest).

[broke]: cockroachdb#63393 (comment)

Release note: None
@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ fd2ab675441d1928650a0e3ed7eb3aa0395f51b2:

        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8660354 [select]:
        github.com/cockroachdb/cockroach/pkg/sql/sqlliveness/slinstance.(*Instance).heartbeatLoop(0xc049395620, 0x5cadd60, 0xc0582d9100)
        	/go/src/github.com/cockroachdb/cockroach/pkg/sql/sqlliveness/slinstance/slinstance.go:190 +0x19e
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc00d98a200, 0x5cade20, 0xc003202840, 0x0, 0xc01f11b660)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8660279 [sync.Cond.Wait]:
        runtime.goparkunlock(...)
        	/usr/local/go/src/runtime/proc.go:312
        sync.runtime_notifyListWait(0xc011124190, 0x245c)
        	/usr/local/go/src/runtime/sema.go:513 +0xf8
        sync.(*Cond).Wait(0xc011124180)
        	/usr/local/go/src/sync/cond.go:56 +0x9d
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc010f460a0, 0x5cade20, 0xc02ffc73e0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:251 +0x8b
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc00d98a200, 0x5cade20, 0xc02ffc73e0, 0x0, 0xc0190edc90)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8660287 [sync.Cond.Wait]:
        runtime.goparkunlock(...)
        	/usr/local/go/src/runtime/proc.go:312
        sync.runtime_notifyListWait(0xc011124190, 0x2442)
        	/usr/local/go/src/runtime/sema.go:513 +0xf8
        sync.(*Cond).Wait(0xc011124180)
        	/usr/local/go/src/sync/cond.go:56 +0x9d
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc010f460a0, 0x5cade20, 0xc02ffc73e0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:251 +0x8b
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc00d98a200, 0x5cade20, 0xc02ffc73e0, 0x0, 0xc0190edd10)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
    rsg_test.go:186: SELECT 1 executed successfully: probably a slow statement
    rsg_test.go:755: Crash detected: 
        ALTER TABLE ident.ident ADD COLUMN ident CHAR VARYING ( 66 ) ARRAY
        
        Stack trace:
        timeout: "ALTER TABLE ident.ident ADD COLUMN ident CHAR VARYING ( 66 ) ARRAY". currently executing: map[ALTER TABLE ident.ident ADD COLUMN ident CHAR VARYING ( 66 ) ARRAY:1]
    rsg_test.go:764: 4414 executions, 809 successful
    rsg_test.go:401: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestRandomSyntaxSchemaChangeColumn282009305
--- FAIL: TestRandomSyntaxSchemaChangeColumn (353.71s)
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @rafiss

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ 9ae5896d1944df379cc8005d597d0e89d710da81:

        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8779164 [chan send, 1 minutes]:
        github.com/lib/pq.(*conn).watchCancel.func1(0xc00d159bc0, 0xc00344d600, 0xc037288c60)
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/lib/pq/conn_go18.go:104 +0x185
        created by github.com/lib/pq.(*conn).watchCancel
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/lib/pq/conn_go18.go:93 +0xa5
        
        goroutine 8697047 [select, 5 minutes]:
        github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x5caeec0, 0xc020062600)
        	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:103 +0x13d
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc0198c2200, 0x5caeec0, 0xc020062600, 0x0, 0xc015f2c510)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8697092 [sync.Cond.Wait]:
        runtime.goparkunlock(...)
        	/usr/local/go/src/runtime/proc.go:312
        sync.runtime_notifyListWait(0xc00cf9a350, 0x4998)
        	/usr/local/go/src/runtime/sema.go:513 +0xf8
        sync.(*Cond).Wait(0xc00cf9a340)
        	/usr/local/go/src/sync/cond.go:56 +0x9d
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc03e4983c0, 0x5caeec0, 0xc017003620)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:251 +0x8b
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc0198c2200, 0x5caeec0, 0xc017003620, 0x0, 0xc02791bd30)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8697051 [select, 5 minutes]:
        github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x5caeec0, 0xc020062600)
        	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:103 +0x13d
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc0198c2200, 0x5caeec0, 0xc020062600, 0x0, 0xc015f2c550)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
    rsg_test.go:186: SELECT 1 executed successfully: probably a slow statement
    rsg_test.go:755: Crash detected: 
        ALTER TABLE ident.ident DROP COLUMN IF EXISTS IF RESTRICT
        
        Stack trace:
        timeout: "ALTER TABLE ident.ident DROP COLUMN IF EXISTS IF RESTRICT". currently executing: map[ALTER TABLE ident.ident DROP COLUMN IF EXISTS IF RESTRICT:1]
    rsg_test.go:764: 9432 executions, 1787 successful
    rsg_test.go:401: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestRandomSyntaxSchemaChangeColumn049238292
--- FAIL: TestRandomSyntaxSchemaChangeColumn (358.80s)
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @rafiss

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ f9dca40d3afb0a605eeee76bec4a07ff65d6fec0:

        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8797640 [chan receive, 5 minutes]:
        github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop.func1(0x5ccc2c0, 0xc011c56030)
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:176 +0x76
        runtime/pprof.Do(0x5ccc2c0, 0xc011c56030, 0xc00000ff40, 0x1, 0x1, 0xc004baefb8)
        	/usr/local/go/src/runtime/pprof/runtime.go:40 +0xcc
        github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop(0xc00fcc6300)
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:175 +0x89
        created by github.com/cockroachdb/pebble.(*tableCacheShard).init
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:167 +0x1a5
        
        goroutine 8797658 [chan receive, 5 minutes]:
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).Start.func1(0x5ccc2c0, 0xc014280e10)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:193 +0x4c
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc012506300, 0x5ccc2c0, 0xc014280e10, 0x0, 0xc019ae03a0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
        goroutine 8850920 [chan send, 1 minutes]:
        github.com/lib/pq.(*conn).watchCancel.func1(0xc027c84d80, 0xc0021ca000, 0xc00c1ae720)
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/lib/pq/conn_go18.go:104 +0x185
        created by github.com/lib/pq.(*conn).watchCancel
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/lib/pq/conn_go18.go:93 +0xa5
        
        goroutine 8797770 [sync.Cond.Wait]:
        runtime.goparkunlock(...)
        	/usr/local/go/src/runtime/proc.go:312
        sync.runtime_notifyListWait(0xc013df21d0, 0x2c4d)
        	/usr/local/go/src/runtime/sema.go:513 +0xf8
        sync.(*Cond).Wait(0xc013df21c0)
        	/usr/local/go/src/sync/cond.go:56 +0x9d
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc012f321e0, 0x5ccc2c0, 0xc014280e10)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:251 +0x8b
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc012506300, 0x5ccc2c0, 0xc014280e10, 0x0, 0xc05432fd60)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:351 +0xb9
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:346 +0xfc
        
    rsg_test.go:186: SELECT 1 executed successfully: probably a slow statement
    rsg_test.go:755: Crash detected: 
        ALTER TABLE ident.ident ADD COLUMN PRECISION STRING ( 202 ) ARRAY [ 209 ]
        
        Stack trace:
        timeout: "ALTER TABLE ident.ident ADD COLUMN PRECISION STRING ( 202 ) ARRAY [ 209 ]". currently executing: map[ALTER TABLE ident.ident ADD COLUMN PRECISION STRING ( 202 ) ARRAY [ 209 ]:1]
    rsg_test.go:764: 5706 executions, 1045 successful
    rsg_test.go:401: -- test log scope end --
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestRandomSyntaxSchemaChangeColumn489812988
--- FAIL: TestRandomSyntaxSchemaChangeColumn (345.88s)
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @cockroachdb/sql-queries @rafiss

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ 43a079db6dc1e6460a0bdb23a73bb69c81389532:

Random syntax error:

    rsg_test.go:758: Crash detected: server panic: statement exec timeout

Query:

        ALTER TABLE ident.ident ADD COLUMN IF NOT EXISTS TIMETZ BIT VARYING ( 411 ) ARRAY [ 9 ] CREATE FAMILY TIMETZ;
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @cockroachdb/sql-queries @rafiss

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ dee0558010170d938ead781d82514fd0bb3b0b88:

Random syntax error:

    rsg_test.go:758: Crash detected: server panic: statement exec timeout

Query:

        ALTER TABLE ident.ident ALTER COLUMN INT SET NOT NULL;
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @cockroachdb/sql-queries @rafiss

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ 329a636debe0d968f2f4d6fe0daef957c4c41583:

Random syntax error:

    rsg_test.go:758: Crash detected: server panic: statement exec timeout

Query:

        ALTER TABLE ident.ident ALTER COLUMN ident SET NOT NULL;
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @cockroachdb/sql-queries @rafiss

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ d302ba88430dd2438a5edcee15edf18286f81485:

Random syntax error:

    rsg_test.go:758: Crash detected: server panic: statement exec timeout

Query:

        ALTER TABLE ident.ident DROP GEOMETRY;
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @cockroachdb/sql-queries @rafiss

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ 704e841bbc440deb4321447d9f36fc6ba41c39df:

Random syntax error:

    rsg_test.go:758: Crash detected: server panic: statement exec timeout

Query:

        ALTER TABLE ident.ident ADD IF NOT EXISTS DATABASES VARBIT ( 366 ) [ ] CREATE FAMILY;
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @cockroachdb/sql-queries @rafiss

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ ac033403c8fbb699cff200316e432dc60b47be6e:

Random syntax error:

    rsg_test.go:758: Crash detected: server panic: statement exec timeout

Query:

        ALTER TABLE ident.ident ALTER ident SET NOT NULL;
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @cockroachdb/sql-queries @rafiss

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ 4da45c4e6b4ba223b832eb6a1f675addbc479f76:

Random syntax error:

    rsg_test.go:758: Crash detected: server panic: statement exec timeout

Query:

        ALTER TABLE ident.ident ALTER COLUMN ident SET NOT NULL;
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @cockroachdb/sql-queries @rafiss

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ 88754f2866fde1f37620b04c6a708eefc0b5be54:

Random syntax error:

    rsg_test.go:758: Crash detected: server panic: statement exec timeout

Query:

        ALTER TABLE ident.ident ADD ident BIT VARYING ARRAY;
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @cockroachdb/sql-queries @rafiss

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ 68b69b6e24c4016e724492b19a0567ad189fd8fd:

Random syntax error:

    rsg_test.go:758: Crash detected: server panic: statement exec timeout

Query:

        ALTER TABLE ident.ident ADD EXCLUDING INTERVAL DAY TO SECOND ARRAY CREATE IF NOT EXISTS FAMILY TIMESTAMP;
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @cockroachdb/sql-queries @rafiss

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ b706c913c07890d04d6bbec4d9d7cd8fea609b95:

Random syntax error:

    rsg_test.go:758: Crash detected: server panic: statement exec timeout

Query:

        ALTER TABLE ident.ident ADD POSITION BIT VARYING ( 416 ) ARRAY [ 339 ];
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @cockroachdb/sql-queries @rafiss

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ 25e26a5f0361fcfbf71cee86f963b93f806604d8:

Random syntax error:

    rsg_test.go:758: Crash detected: server panic: statement exec timeout

Query:

        ALTER TABLE ident.ident ADD COLUMN ALTER VARBIT ( 432 ) ARRAY;
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @cockroachdb/sql-queries @rafiss

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ eecead5c3aa93b5718a57c47970d67ee3035e794:

Random syntax error:

    rsg_test.go:758: Crash detected: server panic: statement exec timeout

Query:

        ALTER TABLE ident.ident DROP COLUMN COALESCE CASCADE;
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @cockroachdb/sql-queries rafiss

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ c305dbcff1bcf621ae8edf058b220033c98d6526:

Random syntax error:

    rsg_test.go:758: Crash detected: server panic: statement exec timeout

Query:

        ALTER TABLE ident.ident ADD IF NOT EXISTS USERS VARBIT ( 112 ) ARRAY;
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @cockroachdb/sql-queries rafiss

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ 73154b35a3f679861a102b7cfcbeaa9bfc1aa10f:

Random syntax error:

    rsg_test.go:758: Crash detected: server panic: statement exec timeout

Query:

        ALTER TABLE ident.ident ALTER COLUMN VARBIT SET NOT NULL;
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @cockroachdb/sql-experience rafiss

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ 3eeb35f3ea834228cd724f2a0e01c27b75bb6dd5:

Random syntax error:

    rsg_test.go:758: Crash detected: server panic: statement exec timeout

Query:

        ALTER TABLE ident.ident ADD COLUMN IF NOT EXISTS TIMETZ BIT ( 175 ) ARRAY [ 444 ];
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @cockroachdb/sql-experience rafiss

This test on roachdash | Improve this report!

@cockroach-teamcity
Copy link
Member

sql/tests.TestRandomSyntaxSchemaChangeColumn failed with artifacts on master @ d95eec9bf1b7b18792c9da528cb204c20804216f:

Random syntax error:

    rsg_test.go:758: Crash detected: server panic: pq: internal error: validation of NOT NULL constraint failed: validate check constraint: comparison overload not found (is, collatedstring{any}[], unknown)

Query:

        ALTER TABLE ident.ident ALTER COLUMN OVERLAY SET NOT NULL;
Reproduce

To reproduce, try:

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Same failure on other branches

Internal log

mjibson marked as alumn{us/a}; resolving to rafiss instead

/cc @cockroachdb/sql-experience rafiss

This test on roachdash | Improve this report!

craig bot pushed a commit that referenced this issue May 25, 2021
64877: sql/tests: TestRandomSyntaxSchemaChangeColumn fails due to timeout r=postamar a=fqazi

Fixes: #63393

Previously, there was a 1 minute timeout (in team city) on the DDL
alter statements run during this test under team city. With that
time limit we would observe failures on certain ALTER COLUMN ADD
statements, because the statement timeout could be too short.
To address this, this patch bumps the random column changes
statement timeout to 2 minutes, which resolves these failures.

Release note: None

65670: changefeedccl: Cleanup: deprecate `DeprecatedBoundaryReached`. r=miretskiy a=miretskiy

Remove `DeprecatedBoundaryReach`, and rely on `BoundaryType`
to detect schema changes.

Release Notes: None

65672: backupccl: fix TestCloudBackupRestoreGoogleCloudStorage r=dt a=pbardea

This test was previously failing due to credentials not being set.

Release note: None

Co-authored-by: Faizan Qazi <[email protected]>
Co-authored-by: Yevgeniy Miretskiy <[email protected]>
Co-authored-by: Paul Bardea <[email protected]>
@craig craig bot closed this as completed in 632b8ed May 25, 2021
fqazi added a commit to fqazi/cockroach that referenced this issue May 28, 2021
Fixes: cockroachdb#63393

Previously, there was a 1 minute timeout (in team city) on the DDL
alter statements run during this test under team city. With that
time limit we would observe failures on certain ALTER COLUMN ADD
statements, because the statement timeout could be too short.
To address this, this patch bumps the random column changes
statement timeout to 2 minutes, which resolves these failures.

Release note: None
fqazi added a commit to fqazi/cockroach that referenced this issue May 28, 2021
Fixes: cockroachdb#63393

Previously, there was a 1 minute timeout (in team city) on the DDL
alter statements run during this test under team city. With that
time limit we would observe failures on certain ALTER COLUMN ADD
statements, because the statement timeout could be too short.
To address this, this patch bumps the random column changes
statement timeout to 2 minutes, which resolves these failures.

Release note: None
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
branch-master Failures and bugs on the master branch. C-test-failure Broken test (automatically or manually discovered).
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants