Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sql/tests: TestRandomSyntaxSchemaChangeColumn failed #65632

Closed
cockroach-teamcity opened this issue May 24, 2021 · 8 comments
Closed

sql/tests: TestRandomSyntaxSchemaChangeColumn failed #65632

cockroach-teamcity opened this issue May 24, 2021 · 8 comments
Assignees
Labels
C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot.
Milestone

Comments

@cockroach-teamcity
Copy link
Member

(sql/tests).TestRandomSyntaxSchemaChangeColumn failed on release-20.2@eda73f20b651cb76dc42aedb194f4373cbd84945:

        internal/poll.(*FD).Read(0xc00e30b180, 0xc00cf2e000, 0x8b5, 0x8b5, 0x0, 0x0, 0x0)
        	/usr/local/go/src/internal/poll/fd_unix.go:159 +0x1a5
        net.(*netFD).Read(0xc00e30b180, 0xc00cf2e000, 0x8b5, 0x8b5, 0x203001, 0xa0a3fb, 0xc0094d2860)
        	/usr/local/go/src/net/fd_posix.go:55 +0x4f
        net.(*conn).Read(0xc005757ff8, 0xc00cf2e000, 0x8b5, 0x8b5, 0x0, 0x0, 0x0)
        	/usr/local/go/src/net/net.go:182 +0x8e
        crypto/tls.(*atLeastReader).Read(0xc006fef720, 0xc00cf2e000, 0x8b5, 0x8b5, 0x62, 0x8b0, 0xc0005937b0)
        	/usr/local/go/src/crypto/tls/conn.go:779 +0x62
        bytes.(*Buffer).ReadFrom(0xc0094d2980, 0x5f5bd20, 0xc006fef720, 0x773cc5, 0x4d5dba0, 0x5285320)
        	/usr/local/go/src/bytes/buffer.go:204 +0xb1
        crypto/tls.(*Conn).readFromUntil(0xc0094d2700, 0x5f64760, 0xc005757ff8, 0x5, 0xc005757ff8, 0x51)
        	/usr/local/go/src/crypto/tls/conn.go:801 +0xf3
        crypto/tls.(*Conn).readRecordOrCCS(0xc0094d2700, 0x0, 0x0, 0x0)
        	/usr/local/go/src/crypto/tls/conn.go:608 +0x115
        crypto/tls.(*Conn).readRecord(...)
        	/usr/local/go/src/crypto/tls/conn.go:576
        crypto/tls.(*Conn).Read(0xc0094d2700, 0xc0026b0000, 0x8000, 0x8000, 0x0, 0x0, 0x0)
        	/usr/local/go/src/crypto/tls/conn.go:1252 +0x15f
        bufio.(*Reader).Read(0xc007141bc0, 0xc001ac3ed8, 0x9, 0x9, 0x5fa7501, 0x1000500c0, 0x0)
        	/usr/local/go/src/bufio/bufio.go:227 +0x222
        io.ReadAtLeast(0x5f5bb00, 0xc007141bc0, 0xc001ac3ed8, 0x9, 0x9, 0x9, 0xc0028432d0, 0x0, 0x0)
        	/usr/local/go/src/io/io.go:314 +0x87
        io.ReadFull(...)
        	/usr/local/go/src/io/io.go:333
        golang.org/x/net/http2.readFrameHeader(0xc001ac3ed8, 0x9, 0x9, 0x5f5bb00, 0xc007141bc0, 0x0, 0x0, 0x0, 0x0)
        	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/net/http2/frame.go:237 +0x89
        golang.org/x/net/http2.(*Framer).ReadFrame(0xc001ac3ea0, 0xc00e411d70, 0x841d520, 0x0, 0x0)
        	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/net/http2/frame.go:492 +0xa5
        google.golang.org/grpc/internal/transport.(*http2Client).reader(0xc001be8000)
        	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_client.go:1273 +0x179
        created by google.golang.org/grpc/internal/transport.newHTTP2Client
        	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/internal/transport/http2_client.go:300 +0xd31
        
        goroutine 9576681 [chan receive, 5 minutes]:
        github.com/cockroachdb/cockroach/pkg/server.(*Server).startServeUI.func1(0x5feca60, 0xc00d52fd10)
        	/go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:1837 +0x53
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00a3494f0, 0xc00c6d4000, 0xc005ffe440)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:222 +0xfc
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:215 +0xa8
        
    rsg_test.go:186: SELECT 1 executed successfully: probably a slow statement
    rsg_test.go:752: Crash detected: 
        ALTER TABLE ident.ident ADD COLUMN VIEW CHARACTER VARYING ( 52 ) ARRAY [ 230 ] CREATE IF NOT EXISTS FAMILY TEMPORARY
        
        Stack trace:
        timeout: "ALTER TABLE ident.ident ADD COLUMN VIEW CHARACTER VARYING ( 52 ) ARRAY [ 230 ] CREATE IF NOT EXISTS FAMILY TEMPORARY". currently executing: map[ALTER TABLE ident.ident ADD COLUMN VIEW CHARACTER VARYING ( 52 ) ARRAY [ 230 ] CREATE IF NOT EXISTS FAMILY TEMPORARY:1]
    rsg_test.go:761: 6454 executions, 1291 successful
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestRandomSyntaxSchemaChangeColumn877739815
--- FAIL: TestRandomSyntaxSchemaChangeColumn (356.17s)

More

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity cockroach-teamcity added branch-release-20.2 C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot. labels May 24, 2021
@cockroach-teamcity cockroach-teamcity added this to the 20.2 milestone May 24, 2021
@tbg tbg removed their assignment May 25, 2021
@rytaft
Copy link
Collaborator

rytaft commented May 25, 2021

cc @fqazi

@cockroach-teamcity
Copy link
Member Author

(sql/tests).TestRandomSyntaxSchemaChangeColumn failed on release-20.2@265a0098de6ce55129fee67df8a38110f5d28c55:

        
        goroutine 9920190 [chan receive, 1 minutes]:
        github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).resumeJob.func2(0xc031a71020)
        	/go/src/github.com/cockroachdb/cockroach/pkg/jobs/adopt.go:201 +0x59
        created by github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).resumeJob
        	/go/src/github.com/cockroachdb/cockroach/pkg/jobs/adopt.go:199 +0x9c5
        
        goroutine 9875546 [chan receive, 5 minutes]:
        github.com/cockroachdb/cockroach/pkg/gossip.(*server).start.func3(0x5fecae0, 0xc02a7d1320)
        	/go/src/github.com/cockroachdb/cockroach/pkg/gossip/server.go:383 +0x5a
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc008dfd890, 0xc023ab4fc0, 0xc01b15c580)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:222 +0xfc
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:215 +0xa8
        
        goroutine 9875568 [select]:
        github.com/cockroachdb/cockroach/pkg/sql/sqlliveness/slstorage.(*Storage).deleteSessionsLoop(0xc001c4e540, 0x5fecae0, 0xc0224d41b0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/sql/sqlliveness/slstorage/slstorage.go:279 +0x245
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc023ab4fc0, 0x5fecae0, 0xc0224d41b0, 0xc022f9da60, 0x11, 0x0, 0x0, 0xc01f8f5570)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:347 +0xee
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:342 +0x133
        
        goroutine 9875668 [select, 5 minutes]:
        github.com/cockroachdb/cockroach/pkg/sql/flowinfra.(*FlowScheduler).Start.func1(0x5fecae0, 0xc0350018c0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/sql/flowinfra/flow_scheduler.go:160 +0x165
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc01f8f55e0, 0xc023ab4fc0, 0xc01f8f55d0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:222 +0xfc
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:215 +0xa8
        
        goroutine 9875526 [chan receive, 5 minutes]:
        github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop.func1(0x5fecae0, 0xc0350a20c0)
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:160 +0x76
        runtime/pprof.Do(0x5fecae0, 0xc0350a20c0, 0xc000110d20, 0x1, 0x1, 0xc001360fb8)
        	/usr/local/go/src/runtime/pprof/runtime.go:40 +0xcc
        github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop(0xc00d5061f0)
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:159 +0x89
        created by github.com/cockroachdb/pebble.(*tableCacheShard).init
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:151 +0x1a5
        
    rsg_test.go:186: SELECT 1 executed successfully: probably a slow statement
    rsg_test.go:752: Crash detected: 
        ALTER TABLE ident.ident DROP ISERROR
        
        Stack trace:
        timeout: "ALTER TABLE ident.ident DROP ISERROR". currently executing: map[ALTER TABLE ident.ident DROP ISERROR:1]
    rsg_test.go:761: 4198 executions, 870 successful
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestRandomSyntaxSchemaChangeColumn457842216
--- FAIL: TestRandomSyntaxSchemaChangeColumn (357.59s)

More

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(sql/tests).TestRandomSyntaxSchemaChangeColumn failed on release-20.2@deeb6c61c90795c9983c7e1f73ea35f0ddebb459:

        	/usr/local/go/src/sync/cond.go:56 +0x9d
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc00a5a9320, 0x5fed040, 0xc00ee77da0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:239 +0x85
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).Start.func2(0x5fed040, 0xc00ee77da0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:194 +0x3e
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc014ed1810, 0xc00a1d26c0, 0xc014ed1800)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:222 +0xfc
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:215 +0xa8
        
        goroutine 9703346 [chan receive, 6 minutes]:
        google.golang.org/grpc.(*addrConn).resetTransport(0xc00a8ad8c0)
        	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/clientconn.go:1156 +0x709
        created by google.golang.org/grpc.(*addrConn).connect
        	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/clientconn.go:799 +0x12a
        
        goroutine 9703462 [select, 5 minutes]:
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*baseQueue).processLoop.func1(0x5fed040, 0xc0115dbda0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/queue.go:814 +0x16a
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc01a20a5a0, 0xc00a1d26c0, 0xc016affe00)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:222 +0xfc
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:215 +0xa8
        
        goroutine 9703478 [select]:
        github.com/cockroachdb/cockroach/pkg/sql/stmtdiagnostics.(*Registry).poll(0xc00a368600, 0x5fecf80, 0xc009df2240)
        	/go/src/github.com/cockroachdb/cockroach/pkg/sql/stmtdiagnostics/statement_diagnostics.go:147 +0x2e5
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc00a1d26c0, 0x5fecf80, 0xc009df2240, 0xc02265aa40, 0x16, 0x0, 0x0, 0xc01131a950)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:347 +0xee
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:342 +0x133
        
        goroutine 9703475 [select, 6 minutes]:
        github.com/cockroachdb/cockroach/pkg/sql.(*Server).PeriodicallyClearSQLStats.func1(0x5fed040, 0xc00fa5cdb0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:777 +0x265
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc01131a8e0, 0xc00a1d26c0, 0xc011777620)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:222 +0xfc
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:215 +0xa8
        
    rsg_test.go:186: SELECT 1 executed successfully: probably a slow statement
    rsg_test.go:752: Crash detected: 
        ALTER TABLE ident.ident ADD ident CHAR ( 112 ) ARRAY [ 158 ]
        
        Stack trace:
        timeout: "ALTER TABLE ident.ident ADD ident CHAR ( 112 ) ARRAY [ 158 ]". currently executing: map[ALTER TABLE ident.ident ADD ident CHAR ( 112 ) ARRAY [ 158 ]:1]
    rsg_test.go:761: 5646 executions, 1077 successful
W210528 06:00:25.608394 9747432 kv/txn.go:631  [n1,client=127.0.0.1:38238,hostssl,user=root] failure aborting transaction: node unavailable; try another peer; abort caused by: context canceled
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestRandomSyntaxSchemaChangeColumn117155279
--- FAIL: TestRandomSyntaxSchemaChangeColumn (359.72s)

More

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@fqazi fqazi self-assigned this May 28, 2021
@cockroach-teamcity
Copy link
Member Author

(sql/tests).TestRandomSyntaxSchemaChangeColumn failed on release-20.2@31cc103513651e1d7693013709f911cc06e79415:

        github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x5fee040, 0xc008e21ec0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:104 +0x151
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00d0c88a0, 0xc00c6ae990, 0xc00d0c8890)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:222 +0xfc
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:215 +0xa8
        
        goroutine 9844351 [chan receive, 5 minutes]:
        github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop.func1(0x5fee040, 0xc00dbe0990)
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:160 +0x76
        runtime/pprof.Do(0x5fee040, 0xc00dbe0990, 0xc0003fb7e0, 0x1, 0x1, 0xc0015097b8)
        	/usr/local/go/src/runtime/pprof/runtime.go:40 +0xcc
        github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop(0xc0069f29f0)
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:159 +0x89
        created by github.com/cockroachdb/pebble.(*tableCacheShard).init
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:151 +0x1a5
        
        goroutine 9888314 [select]:
        github.com/cockroachdb/cockroach/pkg/internal/client/requestbatcher.(*RequestBatcher).Send(0xc0060bb440, 0x5fee000, 0xc00b0ecba0, 0x6, 0x6068fc0, 0xc00b0ecc00, 0x0, 0x1, 0xc002aaf500, 0xc00008c000)
        	/go/src/github.com/cockroachdb/cockroach/pkg/internal/client/requestbatcher/batcher.go:248 +0x1fc
        github.com/cockroachdb/cockroach/pkg/kv/kvserver/intentresolver.(*IntentResolver).gcTxnRecord(0xc005da48c0, 0x5fee000, 0xc00b0ecba0, 0x6, 0xc002d9d7a0, 0xc00b0ecba0, 0xc011ba6fb0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/intentresolver/intent_resolver.go:732 +0x365
        github.com/cockroachdb/cockroach/pkg/kv/kvserver/intentresolver.(*IntentResolver).cleanupFinishedTxnIntents.func2.1(0x5fee000, 0xc00b0ecba0, 0x4a817c800, 0x5fee000)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/intentresolver/intent_resolver.go:775 +0x50
        github.com/cockroachdb/cockroach/pkg/util/contextutil.RunWithTimeout(0x5fee000, 0xc00b0ecba0, 0x5388b27, 0x12, 0x4a817c800, 0xc00122c6e8, 0x0, 0x0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/contextutil/context.go:135 +0x9e
        github.com/cockroachdb/cockroach/pkg/kv/kvserver/intentresolver.(*IntentResolver).cleanupFinishedTxnIntents.func2(0x5fedfc0, 0xc000074120)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/intentresolver/intent_resolver.go:773 +0xc9
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc00c6ae990, 0x5fedfc0, 0xc000074120, 0xc009fcbe00, 0x33, 0x0, 0x0, 0xc00adb3ce0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:347 +0xee
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:342 +0x133
        
        goroutine 9844207 [chan receive, 5 minutes]:
        github.com/cockroachdb/cockroach/pkg/util/netutil.MakeServer.func2(0x5fee040, 0xc009cca330)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/netutil/net.go:107 +0x65
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc007cdeb80, 0xc00c6ae990, 0xc0182ed960)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:222 +0xfc
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:215 +0xa8
        
    rsg_test.go:186: SELECT 1 executed successfully: probably a slow statement
    rsg_test.go:752: Crash detected: 
        ALTER TABLE ident.ident ADD COLUMN EXISTS INTERVAL MONTH ARRAY CREATE FAMILY ESCAPE
        
        Stack trace:
        timeout: "ALTER TABLE ident.ident ADD COLUMN EXISTS INTERVAL MONTH ARRAY CREATE FAMILY ESCAPE". currently executing: map[ALTER TABLE ident.ident ADD COLUMN EXISTS INTERVAL MONTH ARRAY CREATE FAMILY ESCAPE:1]
    rsg_test.go:761: 3363 executions, 701 successful
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestRandomSyntaxSchemaChangeColumn933816773
--- FAIL: TestRandomSyntaxSchemaChangeColumn (354.62s)

More

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(sql/tests).TestRandomSyntaxSchemaChangeColumn failed on release-20.2@1a5d3d947cbcbcd30e692d5e1f4fb459e11c90b0:

        github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop.func1(0x5fee1e0, 0xc00c41cdb0)
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:160 +0x76
        runtime/pprof.Do(0x5fee1e0, 0xc00c41cdb0, 0xc000111f00, 0x1, 0x1, 0xc015a1d7b8)
        	/usr/local/go/src/runtime/pprof/runtime.go:40 +0xcc
        github.com/cockroachdb/pebble.(*tableCacheShard).releaseLoop(0xc0090850f8)
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:159 +0x89
        created by github.com/cockroachdb/pebble.(*tableCacheShard).init
        	/go/src/github.com/cockroachdb/cockroach/vendor/github.com/cockroachdb/pebble/table_cache.go:151 +0x1a5
        
        goroutine 9630035 [sync.Cond.Wait]:
        runtime.goparkunlock(...)
        	/usr/local/go/src/runtime/proc.go:312
        sync.runtime_notifyListWait(0xc0107c84d0, 0x2bb5)
        	/usr/local/go/src/runtime/sema.go:513 +0xf8
        sync.(*Cond).Wait(0xc0107c84c0)
        	/usr/local/go/src/sync/cond.go:56 +0x9d
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc008c04ab0, 0x5fee1e0, 0xc00e6321e0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:239 +0x85
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).Start.func2(0x5fee1e0, 0xc00e6321e0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:194 +0x3e
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc001a41890, 0xc0087a79e0, 0xc001a41880)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:222 +0xfc
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:215 +0xa8
        
        goroutine 9630036 [sync.Cond.Wait]:
        runtime.goparkunlock(...)
        	/usr/local/go/src/runtime/proc.go:312
        sync.runtime_notifyListWait(0xc0107c84d0, 0x2bc1)
        	/usr/local/go/src/runtime/sema.go:513 +0xf8
        sync.(*Cond).Wait(0xc0107c84c0)
        	/usr/local/go/src/sync/cond.go:56 +0x9d
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc008c04ab0, 0x5fee1e0, 0xc00ec3e030)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:239 +0x85
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).Start.func2(0x5fee1e0, 0xc00ec3e030)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:194 +0x3e
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc001a418b0, 0xc0087a79e0, 0xc001a418a0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:222 +0xfc
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:215 +0xa8
        
    rsg_test.go:186: SELECT 1 executed successfully: probably a slow statement
    rsg_test.go:752: Crash detected: 
        ALTER TABLE ident.ident ADD COLUMN IF NOT EXISTS ident BIT ( 297 ) ARRAY [ 12 ]
        
        Stack trace:
        timeout: "ALTER TABLE ident.ident ADD COLUMN IF NOT EXISTS ident BIT ( 297 ) ARRAY [ 12 ]". currently executing: map[ALTER TABLE ident.ident ADD COLUMN IF NOT EXISTS ident BIT ( 297 ) ARRAY [ 12 ]:1]
    rsg_test.go:761: 4203 executions, 817 successful
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestRandomSyntaxSchemaChangeColumn423469042
--- FAIL: TestRandomSyntaxSchemaChangeColumn (342.88s)

More

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(sql/tests).TestRandomSyntaxSchemaChangeColumn failed on release-20.2@951c99b9e0b69691790a626444c3132b815763b2:

        	/usr/local/go/src/runtime/sema.go:513 +0xf8
        sync.(*Cond).Wait(0xc012e9ef40)
        	/usr/local/go/src/sync/cond.go:56 +0x9d
        github.com/cockroachdb/cockroach/pkg/sql.(*StmtBuf).CurCmd(0xc012186f20, 0x0, 0x0, 0x0, 0x0, 0x0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_io.go:439 +0x9e
        github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).execCmd(0xc012c08000, 0x5fef100, 0xc012e9ef80, 0x0, 0x0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1415 +0x65
        github.com/cockroachdb/cockroach/pkg/sql.(*connExecutor).run(0xc012c08000, 0x5fef100, 0xc012e9ef80, 0xc00f522960, 0x5400, 0x15000, 0xc00f522a00, 0xc01bc6e720, 0x0, 0x0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:1391 +0x1fc
        github.com/cockroachdb/cockroach/pkg/sql.(*Server).ServeConn(0xc00f5b0b00, 0x5fef100, 0xc012e9ef80, 0xc012c08000, 0x5400, 0x15000, 0xc00f522a00, 0xc01bc6e720, 0x0, 0x0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/sql/conn_executor.go:508 +0xce
        github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*conn).processCommandsAsync.func1(0xc01adb217f, 0xc01cac3e00, 0x5fef100, 0xc012e9ef80, 0xc01bc6e720, 0xc00f5b0b00, 0xc012186900, 0x601af80, 0xc00a7aba10, 0xc012b77ec0, ...)
        	/go/src/github.com/cockroachdb/cockroach/pkg/sql/pgwire/conn.go:627 +0x3a5
        created by github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*conn).processCommandsAsync
        	/go/src/github.com/cockroachdb/cockroach/pkg/sql/pgwire/conn.go:555 +0x17e
        
        goroutine 9645869 [sync.Cond.Wait]:
        runtime.goparkunlock(...)
        	/usr/local/go/src/runtime/proc.go:312
        sync.runtime_notifyListWait(0xc00f6bb090, 0x3c30)
        	/usr/local/go/src/runtime/sema.go:513 +0xf8
        sync.(*Cond).Wait(0xc00f6bb080)
        	/usr/local/go/src/sync/cond.go:56 +0x9d
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc00f71e480, 0x5fef1c0, 0xc00afa15f0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:239 +0x85
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).Start.func2(0x5fef1c0, 0xc00afa15f0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:194 +0x3e
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00ff4f0f0, 0xc00ea80360, 0xc00ff4f0e0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:222 +0xfc
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:215 +0xa8
        
        goroutine 9645696 [select]:
        github.com/cockroachdb/cockroach/pkg/gossip.newInfoStore.func1(0x5fef1c0, 0xc00eb829c0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/gossip/infostore.go:190 +0x17b
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc008bc3b30, 0xc00ea80360, 0xc008bc3b20)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:222 +0xfc
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:215 +0xa8
        
    rsg_test.go:186: SELECT 1 executed successfully: probably a slow statement
    rsg_test.go:752: Crash detected: 
        ALTER TABLE ident.ident ADD COLUMN IF NOT EXISTS MINUTE BIT VARYING ( 497 ) ARRAY
        
        Stack trace:
        timeout: "ALTER TABLE ident.ident ADD COLUMN IF NOT EXISTS MINUTE BIT VARYING ( 497 ) ARRAY". currently executing: map[ALTER TABLE ident.ident ADD COLUMN IF NOT EXISTS MINUTE BIT VARYING ( 497 ) ARRAY:1]
    rsg_test.go:761: 6205 executions, 1211 successful
W210604 05:56:59.846566 9705294 kv/txn.go:631  [n1,client=127.0.0.1:40156,hostssl,user=root] failure aborting transaction: node unavailable; try another peer; abort caused by: context canceled
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestRandomSyntaxSchemaChangeColumn236086299
--- FAIL: TestRandomSyntaxSchemaChangeColumn (352.85s)

More

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@cockroach-teamcity
Copy link
Member Author

(sql/tests).TestRandomSyntaxSchemaChangeColumn failed on release-20.2@e3de35e8edc96eccee84826e667a9ba2cbeb1218:

        
        goroutine 9624533 [select, 5 minutes]:
        github.com/cockroachdb/cockroach/pkg/jobs.(*Registry).Start.func9(0x5fef1a0, 0xc000074120)
        	/go/src/github.com/cockroachdb/cockroach/pkg/jobs/registry.go:713 +0x34a
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc006217200, 0x5fef1a0, 0xc000074120, 0xc012d61c60, 0xf, 0x0, 0x0, 0xc01e6e3760)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:347 +0xee
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:342 +0x133
        
        goroutine 9624421 [sync.Cond.Wait]:
        runtime.goparkunlock(...)
        	/usr/local/go/src/runtime/proc.go:312
        sync.runtime_notifyListWait(0xc00c99e790, 0x32dc)
        	/usr/local/go/src/runtime/sema.go:513 +0xf8
        sync.(*Cond).Wait(0xc00c99e780)
        	/usr/local/go/src/sync/cond.go:56 +0x9d
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).worker(0xc00dae5680, 0x5fef220, 0xc00d8fee10)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:239 +0x85
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*raftScheduler).Start.func2(0x5fef220, 0xc00d8fee10)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/scheduler.go:194 +0x3e
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00acaa830, 0xc006217200, 0xc00acaa820)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:222 +0xfc
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:215 +0xa8
        
        goroutine 9624485 [sync.Cond.Wait]:
        runtime.goparkunlock(...)
        	/usr/local/go/src/runtime/proc.go:312
        sync.runtime_notifyListWait(0xc00c99e6d0, 0x249)
        	/usr/local/go/src/runtime/sema.go:513 +0xf8
        sync.(*Cond).Wait(0xc00c99e6c0)
        	/usr/local/go/src/sync/cond.go:56 +0x9d
        github.com/cockroachdb/cockroach/pkg/kv/kvserver/closedts/provider.(*Provider).Subscribe(0xc010721730, 0x5fef220, 0xc00a8f7b00, 0xc00fd14f60)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/closedts/provider/provider.go:296 +0x465
        github.com/cockroachdb/cockroach/pkg/kv/kvserver.(*Store).startClosedTimestampRangefeedSubscriber.func1(0x5fef220, 0xc00a8f7b00)
        	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvserver/store.go:1785 +0x57
        github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask.func1(0xc006217200, 0x5fef220, 0xc00a8f7b00, 0xc02092cab0, 0x25, 0x0, 0x0, 0xc0079091e0)
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:347 +0xee
        created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunAsyncTask
        	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:342 +0x133
        
    rsg_test.go:186: SELECT 1 executed successfully: probably a slow statement
    rsg_test.go:752: Crash detected: 
        ALTER TABLE ident.ident ADD ident GEOGRAPHY [ 313 ]
        
        Stack trace:
        timeout: "ALTER TABLE ident.ident ADD ident GEOGRAPHY [ 313 ]". currently executing: map[ALTER TABLE ident.ident ADD ident GEOGRAPHY [ 313 ]:1]
    rsg_test.go:761: 6213 executions, 1192 successful
test logs left over in: /go/src/github.com/cockroachdb/cockroach/artifacts/logTestRandomSyntaxSchemaChangeColumn149615781
--- FAIL: TestRandomSyntaxSchemaChangeColumn (351.46s)

More

make stressrace TESTS=TestRandomSyntaxSchemaChangeColumn PKG=./pkg/sql/tests TESTTIMEOUT=5m STRESSFLAGS='-timeout 5m' 2>&1

Related:

See this test on roachdash
powered by pkg/cmd/internal/issues

@fqazi
Copy link
Collaborator

fqazi commented Jun 7, 2021

Fixed with: #65838

@fqazi fqazi closed this as completed Jun 7, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot.
Projects
None yet
Development

No branches or pull requests

4 participants