Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

teamcity: failed tests on master: test/TestBackupRestoreCrossTableReferences, test/TestBackupRestoreCrossTableReferences/restore_everything_to_new_cluster, testrace/TestBackupRestoreCrossTableReferences, testrace/TestBackupRestoreCrossTableReferences/restore_everything_to_new_cluster #21340

Closed
cockroach-teamcity opened this issue Jan 9, 2018 · 0 comments
Closed

teamcity: failed tests on master: test/TestBackupRestoreCrossTableReferences, test/TestBackupRestoreCrossTableReferences/restore_everything_to_new_cluster, testrace/TestBackupRestoreCrossTableReferences, testrace/TestBackupRestoreCrossTableReferences/restore_everything_to_new_cluster #21340

cockroach-teamcity opened this issue Jan 9, 2018 · 0 comments
Labels
C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot.

Comments

@cockroach-teamcity
Copy link
Member

The following tests appear to have failed:

#472489:

--- FAIL: test/TestBackupRestoreCrossTableReferences (0.000s)
Test ended in panic.

------- Stdout: -------
W180109 09:21:33.866105 131059 server/status/runtime.go:109  Could not parse build timestamp: parsing time "" as "2006/01/02 15:04:05": cannot parse "" as "2006"
I180109 09:21:33.867873 131059 server/config.go:516  [n?] 1 storage engine initialized
I180109 09:21:33.867889 131059 server/config.go:519  [n?] RocksDB cache size: 128 MiB
I180109 09:21:33.867894 131059 server/config.go:519  [n?] store 0: in-memory, size 0 B
I180109 09:21:33.870025 131059 server/node.go:361  [n?] **** cluster cc94dc6e-4945-4492-a9cb-b4b8b110b074 has been created
I180109 09:21:33.870050 131059 server/server.go:934  [n?] **** add additional nodes by specifying --join=127.0.0.1:38531
I180109 09:21:33.870646 131059 storage/store.go:1209  [n1,s1] [n1,s1]: failed initial metrics computation: [n1,s1]: system config not yet available
I180109 09:21:33.870687 131059 server/node.go:486  [n1] initialized store [n1,s1]: disk (capacity=512 MiB, available=512 MiB, used=0 B, logicalBytes=3.2 KiB), ranges=1, leases=0, writes=0.00, bytesPerReplica={p10=3322.00 p25=3322.00 p50=3322.00 p75=3322.00 p90=3322.00}, writesPerReplica={p10=0.00 p25=0.00 p50=0.00 p75=0.00 p90=0.00}
I180109 09:21:33.870712 131059 server/node.go:339  [n1] node ID 1 initialized
I180109 09:21:33.870763 131059 gossip/gossip.go:332  [n1] NodeDescriptor set to node_id:1 address:<network_field:"tcp" address_field:"127.0.0.1:38531" > attrs:<> locality:<> ServerVersion:<major_val:1 minor_val:1 patch:0 unstable:8 > 
I180109 09:21:33.870872 131059 storage/stores.go:331  [n1] read 0 node addresses from persistent storage
I180109 09:21:33.871012 131059 server/node.go:627  [n1] connecting to gossip network to verify cluster ID...
I180109 09:21:33.871057 131059 server/node.go:652  [n1] node connected via gossip and verified as part of cluster "cc94dc6e-4945-4492-a9cb-b4b8b110b074"
I180109 09:21:33.871098 131059 server/node.go:428  [n1] node=1: started with [<no-attributes>=<in-mem>] engine(s) and attributes []
I180109 09:21:33.871156 131059 sql/distsql_physical_planner.go:122  [n1] creating DistSQLPlanner with address {tcp 127.0.0.1:38531}
I180109 09:21:33.873570 131167 storage/replica_command.go:819  [split,n1,s1,r1/1:/M{in-ax}] initiating a split of this range at key /System/"" [r2]
E180109 09:21:33.877887 131168 storage/queue.go:668  [replicate,n1,s1,r1/1:/{Min-System/}] range requires a replication change, but lacks a quorum of live replicas (0/1)
I180109 09:21:33.878077 131167 storage/replica_command.go:819  [split,n1,s1,r2/1:/{System/-Max}] initiating a split of this range at key /System/NodeLiveness [r3]
I180109 09:21:33.878526 131059 server/server.go:1161  [n1] starting https server at 127.0.0.1:37015
I180109 09:21:33.878558 131059 server/server.go:1162  [n1] starting grpc/postgres server at 127.0.0.1:38531
I180109 09:21:33.878567 131059 server/server.go:1163  [n1] advertising CockroachDB node at 127.0.0.1:38531
I180109 09:21:33.885012 131167 storage/replica_command.go:819  [split,n1,s1,r3/1:/{System/NodeL…-Max}] initiating a split of this range at key /System/NodeLivenessMax [r4]
W180109 09:21:33.885386 131318 storage/intent_resolver.go:324  [n1,s1] failed to push during intent resolution: failed to push "sql txn implicit" id=c90ba564 key=/Table/SystemConfigSpan/Start rw=true pri=0.00854502 iso=SERIALIZABLE stat=PENDING epo=0 ts=1515489693.880886054,0 orig=1515489693.880886054,0 max=1515489693.880886054,0 wto=false rop=false seq=7
I180109 09:21:33.886974 131059 sql/event_log.go:115  [n1] Event: "alter_table", target: 12, info: {TableName:eventlog Statement:ALTER TABLE system.eventlog ALTER COLUMN "uniqueID" SET DEFAULT uuid_v4() User:node MutationID:0 CascadeDroppedViews:[]}
I180109 09:21:33.888785 131167 storage/replica_command.go:819  [split,n1,s1,r4/1:/{System/NodeL…-Max}] initiating a split of this range at key /System/tsd [r5]
I180109 09:21:33.890568 131059 sql/lease.go:348  [n1] publish: descID=12 (eventlog) version=2 mtime=2018-01-09 09:21:33.890446992 +0000 UTC
I180109 09:21:33.893688 131167 storage/replica_command.go:819  [split,n1,s1,r5/1:/{System/tsd-Max}] initiating a split of this range at key /System/"tse" [r6]
I180109 09:21:33.899470 131167 storage/replica_command.go:819  [split,n1,s1,r6/1:/{System/tse-Max}] initiating a split of this range at key /Table/SystemConfigSpan/Start [r7]
W180109 09:21:33.905327 131352 storage/intent_resolver.go:324  [n1,s1] failed to push during intent resolution: failed to push "sql txn implicit" id=d937b43c key=/Table/SystemConfigSpan/Start rw=true pri=0.06994833 iso=SERIALIZABLE stat=PENDING epo=0 ts=1515489693.900599892,0 orig=1515489693.900599892,0 max=1515489693.900599892,0 wto=false rop=false seq=7
I180109 09:21:33.905789 131167 storage/replica_command.go:819  [split,n1,s1,r7/1:/{Table/System…-Max}] initiating a split of this range at key /Table/11 [r8]
I180109 09:21:33.907332 131059 sql/event_log.go:115  [n1] Event: "set_cluster_setting", target: 0, info: {SettingName:diagnostics.reporting.enabled Value:true User:node}
I180109 09:21:33.910140 131167 storage/replica_command.go:819  [split,n1,s1,r8/1:/{Table/11-Max}] initiating a split of this range at key /Table/12 [r9]
I180109 09:21:33.913898 131167 storage/replica_command.go:819  [split,n1,s1,r9/1:/{Table/12-Max}] initiating a split of this range at key /Table/13 [r10]
I180109 09:21:33.915454 131059 sql/event_log.go:115  [n1] Event: "set_cluster_setting", target: 0, info: {SettingName:version Value:$1 User:node}
I180109 09:21:33.917611 131167 storage/replica_command.go:819  [split,n1,s1,r10/1:/{Table/13-Max}] initiating a split of this range at key /Table/14 [r11]
I180109 09:21:33.918666 131059 sql/event_log.go:115  [n1] Event: "set_cluster_setting", target: 0, info: {SettingName:trace.debug.enable Value:false User:node}
I180109 09:21:33.921998 131167 storage/replica_command.go:819  [split,n1,s1,r11/1:/{Table/14-Max}] initiating a split of this range at key /Table/15 [r12]
I180109 09:21:33.925016 131167 storage/replica_command.go:819  [split,n1,s1,r12/1:/{Table/15-Max}] initiating a split of this range at key /Table/16 [r13]
I180109 09:21:33.928482 131167 storage/replica_command.go:819  [split,n1,s1,r13/1:/{Table/16-Max}] initiating a split of this range at key /Table/17 [r14]
I180109 09:21:33.929620 131059 sql/event_log.go:115  [n1] Event: "alter_table", target: 4, info: {TableName:users Statement:ALTER TABLE system.users ADD COLUMN IF NOT EXISTS "isRole" BOOL NOT NULL DEFAULT false User:node MutationID:1 CascadeDroppedViews:[]}
I180109 09:21:33.931520 131059 sql/lease.go:348  [n1] publish: descID=4 (users) version=2 mtime=2018-01-09 09:21:33.931429583 +0000 UTC
I180109 09:21:33.932068 131167 storage/replica_command.go:819  [split,n1,s1,r14/1:/{Table/17-Max}] initiating a split of this range at key /Table/18 [r15]
I180109 09:21:33.934899 131167 storage/replica_command.go:819  [split,n1,s1,r15/1:/{Table/18-Max}] initiating a split of this range at key /Table/19 [r16]
I180109 09:21:33.935361 131059 sql/lease.go:348  [n1] publish: descID=4 (users) version=3 mtime=2018-01-09 09:21:33.93528106 +0000 UTC
I180109 09:21:33.939322 131167 storage/replica_command.go:819  [split,n1,s1,r16/1:/{Table/19-Max}] initiating a split of this range at key /Table/20 [r17]
I180109 09:21:33.941019 131059 sql/backfill.go:132  [n1] Running backfill for "users", v=3, m=1
I180109 09:21:33.944056 131167 storage/replica_command.go:819  [split,n1,s1,r17/1:/{Table/20-Max}] initiating a split of this range at key /Table/21 [r18]
I180109 09:21:33.947029 131167 storage/replica_command.go:819  [split,n1,s1,r18/1:/{Table/21-Max}] initiating a split of this range at key /Table/22 [r19]
I180109 09:21:33.949266 131059 sql/lease.go:348  [n1] publish: descID=4 (users) version=4 mtime=2018-01-09 09:21:33.949181027 +0000 UTC
I180109 09:21:33.950080 131167 storage/replica_command.go:819  [split,n1,s1,r19/1:/{Table/22-Max}] initiating a split of this range at key /Table/23 [r20]
I180109 09:21:33.952548 131059 sql/event_log.go:115  [n1] Event: "finish_schema_change", target: 4, info: {MutationID:1}
I180109 09:21:33.953232 131059 sql/lease.go:274  publish (count leases): descID=4 name=users version=3 count=1
I180109 09:21:33.981830 131059 server/server.go:1232  [n1] done ensuring all necessary migrations have run
I180109 09:21:33.981870 131059 server/server.go:1235  [n1] serving sql connections
I180109 09:21:33.983974 131557 sql/event_log.go:115  [n1] Event: "node_join", target: 1, info: {Descriptor:{NodeID:1 Address:{NetworkField:tcp AddressField:127.0.0.1:38531} Attrs: Locality: ServerVersion:1.1-8} ClusterID:cc94dc6e-4945-4492-a9cb-b4b8b110b074 StartedAt:1515489693871071403 LastUp:1515489693871071403}
I180109 09:21:33.994395 131523 sql/event_log.go:115  [client=127.0.0.1:47286,user=root,n1] Event: "create_database", target: 50, info: {DatabaseName:data Statement:CREATE DATABASE data User:root}
I180109 09:21:33.995085 131167 storage/replica_command.go:819  [split,n1,s1,r20/1:/{Table/23-Max}] initiating a split of this range at key /Table/50 [r21]
I180109 09:21:33.997759 131523 sql/event_log.go:115  [client=127.0.0.1:47286,user=root,n1] Event: "create_table", target: 51, info: {TableName:bank Statement:CREATE TABLE bank (id INT PRIMARY KEY, balance INT, payload STRING, FAMILY (id, balance, payload)) User:root}
I180109 09:21:33.998465 131167 storage/replica_command.go:819  [split,n1,s1,r21/1:/{Table/50-Max}] initiating a split of this range at key /Table/51 [r22]
I180109 09:21:34.001662 131523 storage/replica_command.go:819  [n1,s1,r22/1:/{Table/51-Max}] initiating a split of this range at key /Table/51/1/3 [r23]
I180109 09:21:34.004569 131523 storage/replica_command.go:819  [n1,s1,r23/1:/{Table/51/1/3-Max}] initiating a split of this range at key /Table/51/1/6 [r24]
I180109 09:21:34.007061 131523 storage/replica_command.go:819  [n1,s1,r24/1:/{Table/51/1/6-Max}] initiating a split of this range at key /Table/51/1/9 [r25]
I180109 09:21:34.010260 131523 storage/replica_command.go:819  [n1,s1,r25/1:/{Table/51/1/9-Max}] initiating a split of this range at key /Table/51/1/12 [r26]
I180109 09:21:34.012969 131523 storage/replica_command.go:819  [n1,s1,r26/1:/{Table/51/1/12-Max}] initiating a split of this range at key /Table/51/1/15 [r27]
I180109 09:21:34.015850 131523 storage/replica_command.go:819  [n1,s1,r27/1:/{Table/51/1/15-Max}] initiating a split of this range at key /Table/51/1/18 [r28]
I180109 09:21:34.022412 131523 storage/replica_command.go:819  [n1,s1,r28/1:/{Table/51/1/18-Max}] initiating a split of this range at key /Table/51/1/21 [r29]
I180109 09:21:34.025262 131523 storage/replica_command.go:819  [n1,s1,r29/1:/{Table/51/1/21-Max}] initiating a split of this range at key /Table/51/1/24 [r30]
I180109 09:21:34.028766 131523 storage/replica_command.go:819  [n1,s1,r30/1:/{Table/51/1/24-Max}] initiating a split of this range at key /Table/51/1/27 [r31]
I180109 09:21:34.034479 131523 sql/event_log.go:115  [client=127.0.0.1:47286,user=root,n1] Event: "create_database", target: 52, info: {DatabaseName:store Statement:CREATE DATABASE store User:root}
I180109 09:21:34.034914 131167 storage/replica_command.go:819  [split,n1,s1,r31/1:/{Table/51/1/27-Max}] initiating a split of this range at key /Table/52 [r32]
I180109 09:21:34.036720 131523 sql/event_log.go:115  [client=127.0.0.1:47286,user=root,n1] Event: "create_database", target: 53, info: {DatabaseName:storestats Statement:CREATE DATABASE storestats User:root}
I180109 09:21:34.038601 131167 storage/replica_command.go:819  [split,n1,s1,r32/1:/{Table/52-Max}] initiating a split of this range at key /Table/53 [r33]
I180109 09:21:34.039958 131523 sql/event_log.go:115  [client=127.0.0.1:47286,user=root,n1] Event: "create_table", target: 54, info: {TableName:store.customers Statement:CREATE TABLE store.customers (id INT PRIMARY KEY, email STRING UNIQUE) User:root}
I180109 09:21:34.041581 131167 storage/replica_command.go:819  [split,n1,s1,r33/1:/{Table/53-Max}] initiating a split of this range at key /Table/54 [r34]
I180109 09:21:34.043523 131523 sql/event_log.go:115  [client=127.0.0.1:47286,user=root,n1] Event: "create_table", target: 55, info: {TableName:store.orders Statement:CREATE TABLE store.orders (id INT PRIMARY KEY, placed TIMESTAMP, INDEX (placed DESC), customerid INT, FOREIGN KEY (customerid) REFERENCES store.customers) User:root}
I180109 09:21:34.044993 131167 storage/replica_command.go:819  [split,n1,s1,r34/1:/{Table/54-Max}] initiating a split of this range at key /Table/55 [r35]
I180109 09:21:34.046211 131523 sql/lease.go:348  [client=127.0.0.1:47286,user=root,n1] publish: descID=54 (customers) version=2 mtime=2018-01-09 09:21:34.04612265 +0000 UTC
I180109 09:21:34.049740 131523 sql/lease.go:348  [client=127.0.0.1:47286,user=root,n1] publish: descID=55 (orders) version=2 mtime=2018-01-09 09:21:34.049648734 +0000 UTC
I180109 09:21:34.051503 131523 sql/lease.go:348  [client=127.0.0.1:47286,user=root,n1] publish: descID=55 (orders) version=3 mtime=2018-01-09 09:21:34.051418448 +0000 UTC
I180109 09:21:34.056141 131523 sql/event_log.go:115  [client=127.0.0.1:47286,user=root,n1] Event: "create_table", target: 56, info: {TableName:data.unused Statement:CREATE TABLE data.unused (id INT PRIMARY KEY) User:root}
I180109 09:21:34.056534 131167 storage/replica_command.go:819  [split,n1,s1,r35/1:/{Table/55-Max}] initiating a split of this range at key /Table/56 [r36]
I180109 09:21:34.062748 131523 sql/event_log.go:115  [client=127.0.0.1:47286,user=root,n1] Event: "create_table", target: 57, info: {TableName:store.receipts Statement:CREATE TABLE store.receipts (id INT PRIMARY KEY, reissue INT, dest STRING, orderid INT, FOREIGN KEY (reissue) REFERENCES store.receipts (id), FOREIGN KEY (dest) REFERENCES store.customers (email), FOREIGN KEY (orderid) REFERENCES store.orders) User:root}
I180109 09:21:34.063269 131167 storage/replica_command.go:819  [split,n1,s1,r36/1:/{Table/56-Max}] initiating a split of this range at key /Table/57 [r37]
I180109 09:21:34.065818 131523 sql/lease.go:348  [client=127.0.0.1:47286,user=root,n1] publish: descID=54 (customers) version=3 mtime=2018-01-09 09:21:34.065709627 +0000 UTC
I180109 09:21:34.069744 131523 sql/lease.go:348  [client=127.0.0.1:47286,user=root,n1] publish: descID=55 (orders) version=4 mtime=2018-01-09 09:21:34.06965851 +0000 UTC
I180109 09:21:34.072953 131523 sql/lease.go:348  [client=127.0.0.1:47286,user=root,n1] publish: descID=57 (receipts) version=2 mtime=2018-01-09 09:21:34.072868353 +0000 UTC
I180109 09:21:34.075669 131523 sql/lease.go:348  [client=127.0.0.1:47286,user=root,n1] publish: descID=57 (receipts) version=3 mtime=2018-01-09 09:21:34.075588734 +0000 UTC
I180109 09:21:34.081445 131523 sql/event_log.go:115  [client=127.0.0.1:47286,user=root,n1] Event: "create_view", target: 58, info: {ViewName:store.early_customers Statement:CREATE VIEW store.early_customers AS SELECT id FROM store.customers WHERE id < 5 User:root}
I180109 09:21:34.082047 131167 storage/replica_command.go:819  [split,n1,s1,r37/1:/{Table/57-Max}] initiating a split of this range at key /Table/58 [r38]
I180109 09:21:34.083345 131523 sql/lease.go:348  [client=127.0.0.1:47286,user=root,n1] publish: descID=54 (customers) version=4 mtime=2018-01-09 09:21:34.083264848 +0000 UTC
I180109 09:21:34.090238 131523 sql/event_log.go:115  [client=127.0.0.1:47286,user=root,n1] Event: "create_view", target: 59, info: {ViewName:storestats.ordercounts Statement:CREATE VIEW storestats.ordercounts AS SELECT c.id, c.email, count(o.id) FROM store.customers AS c LEFT JOIN store.orders AS o ON o.customerid = c.id GROUP BY c.id, c.email ORDER BY c.id, c.email User:root}
I180109 09:21:34.090580 131167 storage/replica_command.go:819  [split,n1,s1,r38/1:/{Table/58-Max}] initiating a split of this range at key /Table/59 [r39]
I180109 09:21:34.093128 131523 sql/lease.go:348  [client=127.0.0.1:47286,user=root,n1] publish: descID=54 (customers) version=5 mtime=2018-01-09 09:21:34.093034984 +0000 UTC
I180109 09:21:34.099468 131523 sql/lease.go:348  [client=127.0.0.1:47286,user=root,n1] publish: descID=55 (orders) version=5 mtime=2018-01-09 09:21:34.09938499 +0000 UTC
I180109 09:21:34.105201 131523 sql/event_log.go:115  [client=127.0.0.1:47286,user=root,n1] Event: "create_view", target: 60, info: {ViewName:store.unused_view Statement:CREATE VIEW store.unused_view AS SELECT id FROM store.customers WHERE false User:root}
I180109 09:21:34.105709 131167 storage/replica_command.go:819  [split,n1,s1,r39/1:/{Table/59-Max}] initiating a split of this range at key /Table/60 [r40]
I180109 09:21:34.107552 131523 sql/lease.go:348  [client=127.0.0.1:47286,user=root,n1] publish: descID=54 (customers) version=6 mtime=2018-01-09 09:21:34.107147156 +0000 UTC
I180109 09:21:34.244458 131978 ccl/storageccl/export.go:124  [n1,s1,r35/1:/Table/5{5-6}] export [/Table/55/3,/Table/55/4)
I180109 09:21:34.244546 131974 ccl/storageccl/export.go:124  [n1,s1,r34/1:/Table/5{4-5}] export [/Table/54/1,/Table/54/2)
I180109 09:21:34.244566 131975 ccl/storageccl/export.go:124  [n1,s1,r34/1:/Table/5{4-5}] export [/Table/54/2,/Table/54/3)
I180109 09:21:34.244603 131976 ccl/storageccl/export.go:124  [n1,s1,r35/1:/Table/5{5-6}] export [/Table/55/1,/Table/55/2)
I180109 09:21:34.244665 131977 ccl/storageccl/export.go:124  [n1,s1,r35/1:/Table/5{5-6}] export [/Table/55/2,/Table/55/3)
I180109 09:21:34.247714 131986 ccl/storageccl/export.go:124  [n1,s1,r37/1:/Table/5{7-8}] export [/Table/57/3,/Table/57/4)
I180109 09:21:34.247798 131793 ccl/storageccl/export.go:124  [n1,s1,r37/1:/Table/5{7-8}] export [/Table/57/2,/Table/57/3)
I180109 09:21:34.247834 131792 ccl/storageccl/export.go:124  [n1,s1,r37/1:/Table/5{7-8}] export [/Table/57/1,/Table/57/2)
I180109 09:21:34.247837 131923 ccl/storageccl/export.go:124  [n1,s1,r37/1:/Table/5{7-8}] export [/Table/57/4,/Table/57/5)
--- FAIL: test/TestBackupRestoreCrossTableReferences/restore_everything_to_new_cluster (0.000s)
Test ended in panic.

------- Stdout: -------
W180109 09:21:34.280807 131883 server/status/runtime.go:109  Could not parse build timestamp: parsing time "" as "2006/01/02 15:04:05": cannot parse "" as "2006"
I180109 09:21:34.286423 131883 server/config.go:516  [n?] 1 storage engine initialized
I180109 09:21:34.286448 131883 server/config.go:519  [n?] RocksDB cache size: 128 MiB
I180109 09:21:34.286455 131883 server/config.go:519  [n?] store 0: in-memory, size 0 B
I180109 09:21:34.289689 131883 server/node.go:361  [n?] **** cluster 178ecce7-47ba-4782-b8a8-ffc3ab67cc8d has been created
I180109 09:21:34.289723 131883 server/server.go:934  [n?] **** add additional nodes by specifying --join=127.0.0.1:37723
I180109 09:21:34.290471 131883 storage/store.go:1209  [n1,s1] [n1,s1]: failed initial metrics computation: [n1,s1]: system config not yet available
I180109 09:21:34.290513 131883 server/node.go:486  [n1] initialized store [n1,s1]: disk (capacity=512 MiB, available=512 MiB, used=0 B, logicalBytes=3.2 KiB), ranges=1, leases=0, writes=0.00, bytesPerReplica={p10=3322.00 p25=3322.00 p50=3322.00 p75=3322.00 p90=3322.00}, writesPerReplica={p10=0.00 p25=0.00 p50=0.00 p75=0.00 p90=0.00}
I180109 09:21:34.290532 131883 server/node.go:339  [n1] node ID 1 initialized
I180109 09:21:34.290584 131883 gossip/gossip.go:332  [n1] NodeDescriptor set to node_id:1 address:<network_field:"tcp" address_field:"127.0.0.1:37723" > attrs:<> locality:<> ServerVersion:<major_val:1 minor_val:1 patch:0 unstable:8 > 
I180109 09:21:34.290730 131883 storage/stores.go:331  [n1] read 0 node addresses from persistent storage
I180109 09:21:34.290821 131883 server/node.go:627  [n1] connecting to gossip network to verify cluster ID...
I180109 09:21:34.290848 131883 server/node.go:652  [n1] node connected via gossip and verified as part of cluster "178ecce7-47ba-4782-b8a8-ffc3ab67cc8d"
I180109 09:21:34.290870 131883 server/node.go:428  [n1] node=1: started with [<no-attributes>=<in-mem>] engine(s) and attributes []
I180109 09:21:34.290915 131883 sql/distsql_physical_planner.go:122  [n1] creating DistSQLPlanner with address {tcp 127.0.0.1:37723}
I180109 09:21:34.292112 131991 storage/replica_command.go:819  [split,n1,s1,r1/1:/M{in-ax}] initiating a split of this range at key /System/"" [r2]
I180109 09:21:34.293861 131883 server/server.go:1161  [n1] starting https server at 127.0.0.1:33523
I180109 09:21:34.293897 131883 server/server.go:1162  [n1] starting grpc/postgres server at 127.0.0.1:37723
I180109 09:21:34.293913 131883 server/server.go:1163  [n1] advertising CockroachDB node at 127.0.0.1:37723
I180109 09:21:34.298352 131991 storage/replica_command.go:819  [split,n1,s1,r2/1:/{System/-Max}] initiating a split of this range at key /System/NodeLiveness [r3]
I180109 09:21:34.304120 131991 storage/replica_command.go:819  [split,n1,s1,r3/1:/{System/NodeL…-Max}] initiating a split of this range at key /System/NodeLivenessMax [r4]
W180109 09:21:34.304480 131931 storage/intent_resolver.go:324  [n1,s1] failed to push during intent resolution: failed to push "sql txn implicit" id=4c4394e4 key=/Table/SystemConfigSpan/Start rw=true pri=0.05814885 iso=SERIALIZABLE stat=PENDING epo=0 ts=1515489694.299458114,0 orig=1515489694.299458114,0 max=1515489694.299458114,0 wto=false rop=false seq=7
I180109 09:21:34.308107 131883 sql/event_log.go:115  [n1] Event: "alter_table", target: 12, info: {TableName:eventlog Statement:ALTER TABLE system.eventlog ALTER COLUMN "uniqueID" SET DEFAULT uuid_v4() User:node MutationID:0 CascadeDroppedViews:[]}
I180109 09:21:34.310130 131991 storage/replica_command.go:819  [split,n1,s1,r4/1:/{System/NodeL…-Max}] initiating a split of this range at key /System/tsd [r5]
I180109 09:21:34.312299 131883 sql/lease.go:348  [n1] publish: descID=12 (eventlog) version=2 mtime=2018-01-09 09:21:34.312155844 +0000 UTC
I180109 09:21:34.314469 131991 storage/replica_command.go:819  [split,n1,s1,r5/1:/{System/tsd-Max}] initiating a split of this range at key /System/"tse" [r6]
I180109 09:21:34.320568 131991 storage/replica_command.go:819  [split,n1,s1,r6/1:/{System/tse-Max}] initiating a split of this range at key /Table/SystemConfigSpan/Start [r7]
I180109 09:21:34.324654 131991 storage/replica_command.go:819  [split,n1,s1,r7/1:/{Table/System…-Max}] initiating a split of this range at key /Table/11 [r8]
W180109 09:21:34.324961 131901 storage/intent_resolver.go:324  [n1,s1] failed to push during intent resolution: failed to push "sql txn implicit" id=ab50d140 key=/Table/SystemConfigSpan/Start rw=true pri=0.03907741 iso=SERIALIZABLE stat=PENDING epo=0 ts=1515489694.321792062,0 orig=1515489694.321792062,0 max=1515489694.321792062,0 wto=false rop=false seq=7
I180109 09:21:34.326085 131883 sql/event_log.go:115  [n1] Event: "set_cluster_setting", target: 0, info: {SettingName:diagnostics.reporting.enabled Value:true User:node}
I180109 09:21:34.331913 131991 storage/replica_command.go:819  [split,n1,s1,r8/1:/{Table/11-Max}] initiating a split of this range at key /Table/12 [r9]
I180109 09:21:34.333873 131883 sql/event_log.go:115  [n1] Event: "set_cluster_setting", target: 0, info: {SettingName:version Value:$1 User:node}
I180109 09:21:34.335944 131991 storage/replica_command.go:819  [split,n1,s1,r9/1:/{Table/12-Max}] initiating a split of this range at key /Table/13 [r10]
I180109 09:21:34.337386 131883 sql/event_log.go:115  [n1] Event: "set_cluster_setting", target: 0, info: {SettingName:trace.debug.enable Value:false User:node}
I180109 09:21:34.339027 131991 storage/replica_command.go:819  [split,n1,s1,r10/1:/{Table/13-Max}] initiating a split of this range at key /Table/14 [r11]
I180109 09:21:34.342373 131991 storage/replica_command.go:819  [split,n1,s1,r11/1:/{Table/14-Max}] initiating a split of this range at key /Table/15 [r12]
I180109 09:21:34.344988 131991 storage/replica_command.go:819  [split,n1,s1,r12/1:/{Table/15-Max}] initiating a split of this range at key /Table/16 [r13]
I180109 09:21:34.350603 131991 storage/replica_command.go:819  [split,n1,s1,r13/1:/{Table/16-Max}] initiating a split of this range at key /Table/17 [r14]
I180109 09:21:34.350678 131883 sql/event_log.go:115  [n1] Event: "alter_table", target: 4, info: {TableName:users Statement:ALTER TABLE system.users ADD COLUMN IF NOT EXISTS "isRole" BOOL NOT NULL DEFAULT false User:node MutationID:1 CascadeDroppedViews:[]}
I180109 09:21:34.354585 131883 sql/lease.go:348  [n1] publish: descID=4 (users) version=2 mtime=2018-01-09 09:21:34.354476404 +0000 UTC
I180109 09:21:34.355122 131991 storage/replica_command.go:819  [split,n1,s1,r14/1:/{Table/17-Max}] initiating a split of this range at key /Table/18 [r15]
I180109 09:21:34.358219 131991 storage/replica_command.go:819  [split,n1,s1,r15/1:/{Table/18-Max}] initiating a split of this range at key /Table/19 [r16]
I180109 09:21:34.358564 131883 sql/lease.go:348  [n1] publish: descID=4 (users) version=3 mtime=2018-01-09 09:21:34.358476879 +0000 UTC
I180109 09:21:34.361543 131991 storage/replica_command.go:819  [split,n1,s1,r16/1:/{Table/19-Max}] initiating a split of this range at key /Table/20 [r17]
I180109 09:21:34.361787 131883 sql/backfill.go:132  [n1] Running backfill for "users", v=3, m=1
I180109 09:21:34.364537 131991 storage/replica_command.go:819  [split,n1,s1,r17/1:/{Table/20-Max}] initiating a split of this range at key /Table/21 [r18]
I180109 09:21:34.367427 131991 storage/replica_command.go:819  [split,n1,s1,r18/1:/{Table/21-Max}] initiating a split of this range at key /Table/22 [r19]
I180109 09:21:34.368597 131883 sql/lease.go:348  [n1] publish: descID=4 (users) version=4 mtime=2018-01-09 09:21:34.368520284 +0000 UTC
I180109 09:21:34.370694 131991 storage/replica_command.go:819  [split,n1,s1,r19/1:/{Table/22-Max}] initiating a split of this range at key /Table/23 [r20]
I180109 09:21:34.372378 131883 sql/event_log.go:115  [n1] Event: "finish_schema_change", target: 4, info: {MutationID:1}
I180109 09:21:34.373389 131883 sql/lease.go:274  publish (count leases): descID=4 name=users version=3 count=1
I180109 09:21:34.404168 131883 server/server.go:1232  [n1] done ensuring all necessary migrations have run
I180109 09:21:34.404199 131883 server/server.go:1235  [n1] serving sql connections
I180109 09:21:34.406024 132352 sql/event_log.go:115  [n1] Event: "node_join", target: 1, info: {Descriptor:{NodeID:1 Address:{NetworkField:tcp AddressField:127.0.0.1:37723} Attrs: Locality: ServerVersion:1.1-8} ClusterID:178ecce7-47ba-4782-b8a8-ffc3ab67cc8d StartedAt:1515489694290854334 LastUp:1515489694290854334}
I180109 09:21:34.416437 132429 sql/event_log.go:115  [client=127.0.0.1:36528,user=root,n1] Event: "create_database", target: 50, info: {DatabaseName:store Statement:CREATE DATABASE store User:root}
I180109 09:21:34.416726 131991 storage/replica_command.go:819  [split,n1,s1,r20/1:/{Table/23-Max}] initiating a split of this range at key /Table/50 [r21]
I180109 09:21:34.423891 132430 storage/replica_command.go:819  [n1,s1,r21/1:/{Table/50-Max}] initiating a split of this range at key /Table/51 [r22]
I180109 09:21:34.429545 132430 storage/replica_command.go:819  [n1,s1,r22/1:/{Table/51-Max}] initiating a split of this range at key /Table/52/2 [r23]
I180109 09:21:34.429924 132432 storage/replica_command.go:819  [n1,s1,r22/1:/{Table/51-Max}] initiating a split of this range at key /Table/51/2 [r24]
W180109 09:21:34.436625 132479 storage/replica.go:2170  [n1,s1,r22/1:/Table/5{1-2/2}] context canceled before command queue: 1 QueryTxn
I180109 09:21:34.437111 132430 storage/replica_command.go:819  [n1,s1,r23/1:/{Table/52/2-Max}] initiating a split of this range at key /Table/53/2 [r25]
I180109 09:21:34.437312 132431 storage/replica_command.go:819  [n1,s1,r23/1:/{Table/52/2-Max}] initiating a split of this range at key /Table/52/3 [r26]
W180109 09:21:34.444062 132413 storage/replica.go:2170  [n1,s1,r23/1:/Table/5{2/2-3/2}] context canceled before command queue: 1 QueryTxn
I180109 09:21:34.489572 132432 storage/replica_command.go:819  [n1,s1,r22/1:/Table/5{1-2/2}] initiating a split of this range at key /Table/51/2 [r27]
panic: test timed out after 4m0s

goroutine 132502 [running]:
testing.startAlarm.func1()
	/usr/local/go/src/testing/testing.go:1145 +0xf9
created by time.goFunc
	/usr/local/go/src/time/sleep.go:170 +0x44

goroutine 1 [chan receive]:
testing.(*T).Run(0xc4200a0690, 0x23217f9, 0x25, 0x23cc1b8, 0x8d4f01)
	/usr/local/go/src/testing/testing.go:790 +0x2fc
testing.runTests.func1(0xc4200a0690)
	/usr/local/go/src/testing/testing.go:1004 +0x64
testing.tRunner(0xc4200a0690, 0xc420683db8)
	/usr/local/go/src/testing/testing.go:746 +0xd0
testing.runTests(0xc42024acc0, 0x33ba640, 0x38, 0x38, 0x0)
	/usr/local/go/src/testing/testing.go:1002 +0x2d8
testing.(*M).Run(0xc42097bf18, 0xc4202577f0)
	/usr/local/go/src/testing/testing.go:921 +0x111
github.com/cockroachdb/cockroach/pkg/ccl/sqlccl.TestMain(0xc420683f18)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/sqlccl/main_test.go:31 +0xda
main.main()
	github.com/cockroachdb/cockroach/pkg/ccl/sqlccl/_test/_testmain.go:172 +0xdb

goroutine 6 [chan receive]:
github.com/cockroachdb/cockroach/pkg/util/log.(*loggingT).flushDaemon(0x36075e0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/log/clog.go:1043 +0x81
created by github.com/cockroachdb/cockroach/pkg/util/log.init.0
	/go/src/github.com/cockroachdb/cockroach/pkg/util/log/clog.go:581 +0xbf

goroutine 15 [syscall, 4 minutes]:
os/signal.signal_recv(0xc4200ec640)
	/usr/local/go/src/runtime/sigqueue.go:131 +0xa6
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:22 +0x22
created by os/signal.init.0
	/usr/local/go/src/os/signal/signal_unix.go:28 +0x41

goroutine 131059 [chan receive]:
testing.(*T).Run(0xc42450c2d0, 0x2319a85, 0x21, 0xc42133f320, 0x30)
	/usr/local/go/src/testing/testing.go:790 +0x2fc
github.com/cockroachdb/cockroach/pkg/ccl/sqlccl_test.TestBackupRestoreCrossTableReferences(0xc42450c2d0)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/sqlccl/backup_test.go:1208 +0xdfb
testing.tRunner(0xc42450c2d0, 0x23cc1b8)
	/usr/local/go/src/testing/testing.go:746 +0xd0
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:789 +0x2de

goroutine 131032 [select]:
github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/transport.loopyWriter(0x33e5220, 0xc421482d80, 0xc421482d20, 0xc420c3bfb8)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/transport/transport.go:750 +0x2e4
github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/transport.newHTTP2Server.func1(0xc423ec4580)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/transport/http2_server.go:227 +0x5e
created by github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/transport.newHTTP2Server
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/transport/http2_server.go:226 +0x8d5

goroutine 34 [select, locked to thread]:
runtime.gopark(0x23d0038, 0x0, 0x22e9164, 0x6, 0x18, 0x1)
	/usr/local/go/src/runtime/proc.go:277 +0x12c
runtime.selectgo(0xc42046af50, 0xc420440480)
	/usr/local/go/src/runtime/select.go:395 +0x1138
runtime.ensureSigM.func1()
	/usr/local/go/src/runtime/signal_unix.go:511 +0x220
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:2337 +0x1

goroutine 131818 [select]:
github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/transport.loopyWriter(0x33e5220, 0xc423d7fe00, 0xc423d7fd70, 0xc42236ffb8)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/transport/transport.go:750 +0x2e4
github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/transport.newHTTP2Server.func1(0xc423ec42c0)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/transport/http2_server.go:227 +0x5e
created by github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/transport.newHTTP2Server
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/transport/http2_server.go:226 +0x8d5

goroutine 132179 [select]:
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x33e5220, 0xc422be23c0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:86 +0x167
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc420412430, 0xc421400a20, 0xc420412420)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:195 +0xf3
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:188 +0xad

goroutine 131229 [select]:
github.com/cockroachdb/cockroach/pkg/storage.(*Store).coalescedHeartbeatsLoop(0xc422434c00, 0x33e5220, 0xc4212bac00)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/store.go:3788 +0x16d
github.com/cockroachdb/cockroach/pkg/storage.(*Store).(github.com/cockroachdb/cockroach/pkg/storage.coalescedHeartbeatsLoop)-fm(0x33e5220, 0xc4212bac00)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/store.go:3734 +0x3e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc421592ec0, 0xc4204ac900, 0xc421592eb0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:195 +0xf3
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:188 +0xad

goroutine 131273 [select]:
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x33e5220, 0xc4213b8780)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:86 +0x167
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc420412dd0, 0xc4204ac900, 0xc420412dc0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:195 +0xf3
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:188 +0xad

goroutine 132011 [select]:
github.com/cockroachdb/cockroach/pkg/gossip.(*Gossip).manage.func1(0x33e5220, 0xc420f86000)
	/go/src/github.com/cockroachdb/cockroach/pkg/gossip/gossip.go:1189 +0x269
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc422af6460, 0xc421400a20, 0xc422af6450)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:195 +0xf3
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:188 +0xad

goroutine 131949 [select]:
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x33e5220, 0xc4232441b0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:86 +0x167
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc4204122f0, 0xc421400a20, 0xc4204122d0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:195 +0xf3
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:188 +0xad

goroutine 132118 [semacquire]:
sync.runtime_notifyListWait(0xc420f8f250, 0xc40000019d)
	/usr/local/go/src/runtime/sema.go:507 +0x110
sync.(*Cond).Wait(0xc420f8f240)
	/usr/local/go/src/sync/cond.go:56 +0x80
github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc4204ad050, 0x33e5220, 0xc423cf6a20)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:197 +0x7c
github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x33e5220, 0xc423cf6a20)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:166 +0x3e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc420b7e650, 0xc421400a20, 0xc420b7e640)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:195 +0xf3
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:188 +0xad

goroutine 132091 [IO wait]:
internal/poll.runtime_pollWait(0x2b4786dc6a30, 0x72, 0x0)
	/usr/local/go/src/runtime/netpoll.go:173 +0x57
internal/poll.(*pollDesc).wait(0xc421328118, 0x72, 0xffffffffffffff00, 0x33cf720, 0x33c2700)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:85 +0xae
internal/poll.(*pollDesc).waitRead(0xc421328118, 0xc422546c00, 0x400, 0x400)
	/usr/local/go/src/internal/poll/fd_poll_runtime.go:90 +0x3d
internal/poll.(*FD).Read(0xc421328100, 0xc422546c00, 0x400, 0x400, 0x0, 0x0, 0x0)
	/usr/local/go/src/internal/poll/fd_unix.go:125 +0x18a
net.(*netFD).Read(0xc421328100, 0xc422546c00, 0x400, 0x400, 0x0, 0xb1a2ee901e964bcb, 0xc420c3c9c8)
	/usr/local/go/src/net/fd_unix.go:202 +0x52
net.(*conn).Read(0xc421122008, 0xc422546c00, 0x400, 0x400, 0x0, 0x0, 0x0)
	/usr/local/go/src/net/net.go:176 +0x6d
crypto/tls.(*block).readFromUntil(0xc422be2570, 0x2b4788d63000, 0xc421122008, 0x5, 0xc421122008, 0xc420c3ca68)
	/usr/local/go/src/crypto/tls/conn.go:488 +0x95
crypto/tls.(*Conn).readRecord(0xc4210c4380, 0x23d0117, 0xc4210c44a0, 0x857967)
	/usr/local/go/src/crypto/tls/conn.go:590 +0xe0
crypto/tls.(*Conn).Read(0xc4210c4380, 0xc422b86000, 0x8000, 0x8000, 0x0, 0x0, 0x0)
	/usr/local/go/src/crypto/tls/conn.go:1134 +0x110
bufio.(*Reader).Read(0xc420fd5b00, 0xc421c131b8, 0x9, 0x9, 0x0, 0x8719e8, 0xc4210c5500)
	/usr/local/go/src/bufio/bufio.go:213 +0x30b
io.ReadAtLeast(0x33c45e0, 0xc420fd5b00, 0xc421c131b8, 0x9, 0x9, 0x9, 0xc4204141e0, 0xc420c3ce28, 0xd5f04e)
	/usr/local/go/src/io/io.go:309 +0x86
io.ReadFull(0x33c45e0, 0xc420fd5b00, 0xc421c131b8, 0x9, 0x9, 0x23cf590, 0x1800050100, 0xc400000001)
	/usr/local/go/src/io/io.go:327 +0x58
github.com/cockroachdb/cockroach/vendor/golang.org/x/net/http2.readFrameHeader(0xc421c131b8, 0x9, 0x9, 0x33c45e0, 0xc420fd5b00, 0x0, 0x0, 0x0, 0xc424c0c318)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/net/http2/frame.go:237 +0x7b
github.com/cockroachdb/cockroach/vendor/golang.org/x/net/http2.(*Framer).ReadFrame(0xc421c13180, 0xc423341a40, 0xc423341a40, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/net/http2/frame.go:492 +0xa4
github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/transport.(*http2Client).reader(0xc4212ce780)
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/transport/http2_client.go:1118 +0xbe
created by github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/transport.newHTTP2Client
	/go/src/github.com/cockroachdb/cockroach/vendor/google.golang.org/grpc/transport/http2_client.go:266 +0xc80

goroutine 131203 [semacquire]:
sync.runtime_notifyListWait(0xc423d235d0, 0xc400000328)
	/usr/local/go/src/runtime/sema.go:507 +0x110
sync.(*Cond).Wait(0xc423d235c0)
	/usr/local/go/src/sync/cond.go:56 +0x80
github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc42183ed80, 0x33e5220, 0xc4214b0450)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:197 +0x7c
github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x33e5220, 0xc4214b0450)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:166 +0x3e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc421592b70, 0xc4204ac900, 0xc421592b60)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:195 +0xf3
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:188 +0xad

goroutine 131272 [select]:
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x33e5220, 0xc4213b8750)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:86 +0x167
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc420412db0, 0xc4204ac900, 0xc420412da0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:195 +0xf3
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:188 +0xad

goroutine 132429 [select]:
github.com/cockroachdb/cockroach/pkg/sql.(*hookFnNode).Next(0xc423984cd0, 0x33e5160, 0xc4233572c0, 0xc42285c298, 0xc42285c228, 0xc423984cd0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/planhook.go:119 +0x172
github.com/cockroachdb/cockroach/pkg/sql.forEachRow(0x33e5160, 0xc4233572c0, 0xc42285c298, 0xc42285c228, 0x33dca20, 0xc423984cd0, 0xc42269e860, 0x0, 0xc42285ce90)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/executor.go:2074 +0x5d
github.com/cockroachdb/cockroach/pkg/sql.(*Executor).execClassic(0xc423ecdd40, 0xc42285c228, 0x33dca20, 0xc423984cd0, 0x33f0e80, 0xc4231850e0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/executor.go:2055 +0x29c
github.com/cockroachdb/cockroach/pkg/sql.(*Executor).execStmt(0xc423ecdd40, 0x33e74e0, 0xc423328f50, 0x33b16e0, 0x7, 0x7, 0xc4227bba60, 0x17, 0x150819d9007aa98d, 0x1, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/executor.go:2216 +0x709
github.com/cockroachdb/cockroach/pkg/sql.(*Executor).execStmtInOpenTxn(0xc423ecdd40, 0xc42285c000, 0x33e74e0, 0xc423328f50, 0x33b16e0, 0x7, 0x7, 0xc4227bba60, 0x17, 0x150819d9007aa98d, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/executor.go:1849 +0x758
github.com/cockroachdb/cockroach/pkg/sql.(*Executor).execSingleStatement(0xc423ecdd40, 0xc42285c000, 0x33e74e0, 0xc423328f50, 0x33b16e0, 0x7, 0x7, 0xc4227bba60, 0x17, 0x150819d9007aa98d, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/executor.go:1318 +0x6c8
github.com/cockroachdb/cockroach/pkg/sql.runTxnAttempt(0xc423ecdd40, 0xc42285c000, 0xc423984be0, 0x1, 0x1, 0xc42269f638, 0x1, 0x1, 0x0, 0x33e9c80, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/executor.go:1194 +0x41f
github.com/cockroachdb/cockroach/pkg/sql.runWithAutoRetry(0xc423ecdd40, 0xc42285c000, 0xc423984be0, 0x1, 0x1, 0xed1e60101, 0x0, 0xc42269f638, 0x3ff0000000000000, 0x970800, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/executor.go:1019 +0x3aa
github.com/cockroachdb/cockroach/pkg/sql.(*Executor).execParsed(0xc423ecdd40, 0xc42285c000, 0xc423984be0, 0x1, 0x1, 0xc42269f638, 0x0, 0xe14546, 0xbe8d404798d7786c)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/executor.go:835 +0x138
github.com/cockroachdb/cockroach/pkg/sql.(*Executor).execPrepared(0xc423ecdd40, 0xc42285c000, 0xc4216ea870, 0xc423277638, 0x9, 0xc42285e017)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/executor.go:677 +0x159
github.com/cockroachdb/cockroach/pkg/sql.(*Executor).ExecutePreparedStatement(0xc423ecdd40, 0xc42285c000, 0xc4216ea870, 0xc423277638, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/executor.go:654 +0x134
github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*v3Conn).handleExecute(0xc4231850e0, 0xc423185108, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/pgwire/v3.go:964 +0x2f6
github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*v3Conn).serve(0xc4231850e0, 0x33e5220, 0xc4242499b0, 0xc4226a60d0, 0x5400, 0x15000, 0xc423ecc178, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/pgwire/v3.go:532 +0xb6f
github.com/cockroachdb/cockroach/pkg/sql/pgwire.(*Server).ServeConn(0xc423ecc000, 0x33e5160, 0xc422332c40, 0x33f07c0, 0xc42149f180, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/pgwire/server.go:421 +0xbae
github.com/cockroachdb/cockroach/pkg/server.(*Server).Start.func17.1(0x33f1060, 0xc420a499a0)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/server.go:1249 +0x149
github.com/cockroachdb/cockroach/pkg/util/netutil.(*Server).ServeWith.func1(0xc421400a20, 0x33e5220, 0xc4241f23f0, 0xc422f7c000, 0x33f1060, 0xc420a499a0, 0xc4241f2420)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/netutil/net.go:140 +0xa9
created by github.com/cockroachdb/cockroach/pkg/util/netutil.(*Server).ServeWith
	/go/src/github.com/cockroachdb/cockroach/pkg/util/netutil/net.go:137 +0x263

goroutine 131223 [semacquire]:
sync.runtime_notifyListWait(0xc423d235d0, 0xc40000033b)
	/usr/local/go/src/runtime/sema.go:507 +0x110
sync.(*Cond).Wait(0xc423d235c0)
	/usr/local/go/src/sync/cond.go:56 +0x80
github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc42183ed80, 0x33e5220, 0xc4213b8570)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:197 +0x7c
github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x33e5220, 0xc4213b8570)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:166 +0x3e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc421592df0, 0xc4204ac900, 0xc421592de0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:195 +0xf3
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:188 +0xad

goroutine 131242 [select]:
github.com/cockroachdb/cockroach/pkg/sql/jobs.(*Registry).Start.func2(0x33e5220, 0xc4213a5380)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/jobs/registry.go:205 +0x13b
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc420412500, 0xc4204ac900, 0xc4211c1050)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:195 +0xf3
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:188 +0xad

goroutine 131210 [semacquire]:
sync.runtime_notifyListWait(0xc423d235d0, 0xc400000331)
	/usr/local/go/src/runtime/sema.go:507 +0x110
sync.(*Cond).Wait(0xc423d235c0)
	/usr/local/go/src/sync/cond.go:56 +0x80
github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc42183ed80, 0x33e5220, 0xc4214b0510)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:197 +0x7c
github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x33e5220, 0xc4214b0510)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:166 +0x3e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc421592c50, 0xc4204ac900, 0xc421592c40)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:195 +0xf3
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:188 +0xad

goroutine 132431 [select]:
github.com/cockroachdb/cockroach/pkg/util/retry.(*Retry).Next(0xc4209ffe30, 0x33d0960)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/retry/retry.go:119 +0x189
github.com/cockroachdb/cockroach/pkg/storage.(*Replica).AdminSplit(0xc421170000, 0x33e5220, 0xc4211165a0, 0xc420296158, 0x2, 0x8, 0x0, 0x0, 0x0, 0xc420296158, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/replica_command.go:693 +0x12e
github.com/cockroachdb/cockroach/pkg/storage.(*Replica).executeAdminBatch(0xc421170000, 0x33e5220, 0xc4211165a0, 0x150819d901b30365, 0x0, 0x100000001, 0x1, 0x17, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/replica.go:2445 +0xbec
github.com/cockroachdb/cockroach/pkg/storage.(*Replica).Send(0xc421170000, 0x33e5220, 0xc4211165a0, 0x150819d901b30365, 0x0, 0x100000001, 0x1, 0x17, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/replica.go:1915 +0x60f
github.com/cockroachdb/cockroach/pkg/storage.(*Store).Send(0xc4200a5400, 0x33e5220, 0xc421116570, 0x150819d901b30365, 0x0, 0x100000001, 0x1, 0x17, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/store.go:2585 +0x688
github.com/cockroachdb/cockroach/pkg/storage.(*Stores).Send(0xc422a60a50, 0x33e5220, 0xc4211164e0, 0x0, 0x0, 0x100000001, 0x1, 0x17, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/stores.go:214 +0x214
github.com/cockroachdb/cockroach/pkg/server.(*Node).batchInternal.func1(0x33e5220, 0xc4211164e0, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/node.go:860 +0x192
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunTaskWithErr(0xc421400a20, 0x33e5220, 0xc4211164e0, 0x22f84ea, 0x10, 0xc42210cfc8, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:245 +0xf7
github.com/cockroachdb/cockroach/pkg/server.(*Node).batchInternal(0xc422a08000, 0x33e5220, 0xc4211164e0, 0xc420d94648, 0xc4211164e0, 0x14c5f7e, 0x33e5220)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/node.go:847 +0x152
github.com/cockroachdb/cockroach/pkg/server.(*Node).Batch(0xc422a08000, 0x33e5220, 0xc4211164e0, 0xc420d94648, 0x1f, 0x0, 0x0)
	/go/src/github.com/cockroachdb/cockroach/pkg/server/node.go:888 +0x99
github.com/cockroachdb/cockroach/pkg/kv.(*grpcTransport).send.func1(0xc422a24380, 0xc420d94630, 0x33e5220, 0xc4211163c0, 0x0, 0xc42210d4a8, 0x8580c8)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/transport.go:243 +0x6b0
github.com/cockroachdb/cockroach/pkg/kv.(*grpcTransport).send(0xc422a24380, 0x33e5220, 0xc4211163c0, 0xc4203b4030, 0xf, 0xc423865b80, 0x0, 0x0, 0x100000001, 0x1, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/transport.go:271 +0x9a
github.com/cockroachdb/cockroach/pkg/kv.(*grpcTransport).SendNext(0xc422a24380, 0x33e5220, 0xc4211163c0, 0xc420e6c360)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/transport.go:208 +0x31b
github.com/cockroachdb/cockroach/pkg/kv.(*DistSender).sendToReplicas(0xc4218e05a0, 0x33e5220, 0xc4211163c0, 0xc4218e05f0, 0x17, 0xc4202e6280, 0x1, 0x1, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/dist_sender.go:1246 +0x3a4
github.com/cockroachdb/cockroach/pkg/kv.(*DistSender).sendRPC(0xc4218e05a0, 0x33e5220, 0xc4211163c0, 0x17, 0xc4202e6280, 0x1, 0x1, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/dist_sender.go:382 +0x27b
github.com/cockroachdb/cockroach/pkg/kv.(*DistSender).sendSingleRange(0xc4218e05a0, 0x33e5220, 0xc4211163c0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/dist_sender.go:446 +0x16e
github.com/cockroachdb/cockroach/pkg/kv.(*DistSender).sendPartialBatch(0xc4218e05a0, 0x33e5220, 0xc4211163c0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/dist_sender.go:1024 +0x347
github.com/cockroachdb/cockroach/pkg/kv.(*DistSender).divideAndSendBatchToRanges(0xc4218e05a0, 0x33e5220, 0xc4211163c0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/dist_sender.go:700 +0x16f8
github.com/cockroachdb/cockroach/pkg/kv.(*DistSender).Send(0xc4218e05a0, 0x33e5220, 0xc4211163c0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/dist_sender.go:621 +0x3f5
github.com/cockroachdb/cockroach/pkg/kv.(*TxnCoordSender).Send(0xc420e0c460, 0x33e5220, 0xc4211163c0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/kv/txn_coord_sender.go:444 +0x1ce
github.com/cockroachdb/cockroach/pkg/internal/client.(*DB).send(0xc4237fcd80, 0x33e5160, 0xc422332f40, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:546 +0x2c9
github.com/cockroachdb/cockroach/pkg/internal/client.(*DB).(github.com/cockroachdb/cockroach/pkg/internal/client.send)-fm(0x33e5160, 0xc422332f40, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
	/go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:480 +0x76
github.com/cockroachdb/cockroach/pkg/internal/client.sendAndFill(0x33e5160, 0xc422332f40, 0xc42210f4c0, 0xc422824700, 0xc420296158, 0xc4242680a0)
	/go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:457 +0xf8
github.com/cockroachdb/cockroach/pkg/internal/client.(*DB).Run(0xc4237fcd80, 0x33e5160, 0xc422332f40, 0xc422824700, 0xc4202e6220, 0xc4202e6220)
	/go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:480 +0x94
github.com/cockroachdb/cockroach/pkg/internal/client.(*DB).AdminSplit(0xc4237fcd80, 0x33e5160, 0xc422332f40, 0x21d07a0, 0xc4202e6200, 0x21d07a0, 0xc4202e6220, 0x2, 0x2)
	/go/src/github.com/cockroachdb/cockroach/pkg/internal/client/db.go:386 +0x98
github.com/cockroachdb/cockroach/pkg/ccl/sqlccl.splitAndScatter.func2(0x8, 0x23d0180)
	/go/src/github.com/cockroachdb/cockroach/pkg/ccl/sqlccl/restore.go:676 +0x41b
github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup.(*Group).Go.func1(0xc422332f80, 0xc42308cb60)
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:58 +0x57
created by github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup.(*Group).Go
	/go/src/github.com/cockroachdb/cockroach/vendor/golang.org/x/sync/errgroup/errgroup.go:55 +0x66

goroutine 131215 [semacquire]:
sync.runtime_notifyListWait(0xc423d235d0, 0xc400000335)
	/usr/local/go/src/runtime/sema.go:507 +0x110
sync.(*Cond).Wait(0xc423d235c0)
	/usr/local/go/src/sync/cond.go:56 +0x80
github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc42183ed80, 0x33e5220, 0xc4212bab10)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:197 +0x7c
github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x33e5220, 0xc4212bab10)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:166 +0x3e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc421592cf0, 0xc4204ac900, 0xc421592ce0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:195 +0xf3
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:188 +0xad

goroutine 131231 [select]:
github.com/cockroachdb/cockroach/pkg/storage.(*Store).startGossip.func4(0x33e5220, 0xc423cf60c0)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/store.go:1313 +0x3e9
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc421592f00, 0xc4204ac900, 0xc420075220)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:195 +0xf3
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:188 +0xad

goroutine 131948 [select]:
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x33e5220, 0xc4232441e0)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:86 +0x167
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc4204122c0, 0xc421400a20, 0xc4204122a0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:195 +0xf3
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:188 +0xad

goroutine 131085 [semacquire]:
sync.runtime_notifyListWait(0xc423d235d0, 0xc400000351)
	/usr/local/go/src/runtime/sema.go:507 +0x110
sync.(*Cond).Wait(0xc423d235c0)
	/usr/local/go/src/sync/cond.go:56 +0x80
github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc42183ed80, 0x33e5220, 0xc4228b1f20)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:197 +0x7c
github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x33e5220, 0xc4228b1f20)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:166 +0x3e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc4215926b0, 0xc4204ac900, 0xc4215926a0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:195 +0xf3
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:188 +0xad

goroutine 131217 [semacquire]:
sync.runtime_notifyListWait(0xc423d235d0, 0xc400000333)
	/usr/local/go/src/runtime/sema.go:507 +0x110
sync.(*Cond).Wait(0xc423d235c0)
	/usr/local/go/src/sync/cond.go:56 +0x80
github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc42183ed80, 0x33e5220, 0xc4214b0570)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:197 +0x7c
github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x33e5220, 0xc4214b0570)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:166 +0x3e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc421592d30, 0xc4204ac900, 0xc421592d20)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:195 +0xf3
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:188 +0xad

goroutine 131235 [select]:
github.com/cockroachdb/cockroach/pkg/storage/compactor.(*Compactor).Start.func1(0x33e5220, 0xc4249cb560)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/compactor/compactor.go:124 +0x1ce
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc421592fb0, 0xc4204ac900, 0xc4238e6bd0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:195 +0xf3
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:188 +0xad

goroutine 131282 [select]:
github.com/cockroachdb/cockroach/pkg/sql.(*DistSQLPlanner).initRunners.func1(0x33e5220, 0xc42247c180)
	/go/src/github.com/cockroachdb/cockroach/pkg/sql/distsql_running.go:86 +0x167
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc420412ef0, 0xc4204ac900, 0xc420412ee0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:195 +0xf3
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:188 +0xad

goroutine 131189 [semacquire]:
sync.runtime_notifyListWait(0xc423d235d0, 0xc400000352)
	/usr/local/go/src/runtime/sema.go:507 +0x110
sync.(*Cond).Wait(0xc423d235c0)
	/usr/local/go/src/sync/cond.go:56 +0x80
github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc42183ed80, 0x33e5220, 0xc4213b81e0)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:197 +0x7c
github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x33e5220, 0xc4213b81e0)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:166 +0x3e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc4215929b0, 0xc4204ac900, 0xc4215929a0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:195 +0xf3
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:188 +0xad

goroutine 131252 [select]:
github.com/cockroachdb/cockroach/pkg/storage.(*baseQueue).processLoop.func1(0x33e5220, 0xc4214b0db0)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/queue.go:514 +0x1bb
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc421433160, 0xc4204ac900, 0xc42033f7c0)
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:195 +0xf3
created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker
	/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:188 +0xad

goroutine 132123 [semacquire]:
sync.runtime_notifyListWait(0xc420f8f250, 0xc400000198)
	/usr/local/go/src/runtime/sema.go:507 +0x110
sync.(*Cond).Wait(0xc420f8f240)
	/usr/local/go/src/sync/cond.go:56 +0x80
github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).worker(0xc4204ad050, 0x33e5220, 0xc423286300)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:197 +0x7c
github.com/cockroachdb/cockroach/pkg/storage.(*raftScheduler).Start.func2(0x33e5220, 0xc423286300)
	/go/src/github.com/cockroachdb/cockroach/pkg/storage/scheduler.go:166 +0x3e
github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc420b7e6f0,

Please assign, take a look and update the issue accordingly.
@cockroach-teamcity cockroach-teamcity added O-robot Originated from a bot. C-test-failure Broken test (automatically or manually discovered). labels Jan 9, 2018
@benesch benesch mentioned this issue Jan 16, 2018
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
C-test-failure Broken test (automatically or manually discovered). O-robot Originated from a bot.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@cockroach-teamcity