Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

kvnemesis: add a multi-node variant of the test #44769

Merged
merged 1 commit into from
Feb 21, 2020
Merged

kvnemesis: add a multi-node variant of the test #44769

merged 1 commit into from
Feb 21, 2020

Conversation

danhhz
Copy link
Contributor

@danhhz danhhz commented Feb 5, 2020

To make this more interesting, at the same time, add support for the
ChangeReplicas command.

Release note: None

@danhhz danhhz requested a review from nvanbenschoten February 5, 2020 18:02
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@danhhz
Copy link
Contributor Author

danhhz commented Feb 5, 2020

Finally got around to cleaning up last week's flex friday project

@danhhz danhhz force-pushed the kvnemesis_multinode branch from 60a1b21 to b41eb17 Compare February 5, 2020 18:06
nvanbenschoten
nvanbenschoten approved these changes Feb 5, 2020
View reviewed changes
Copy link
Member

@nvanbenschoten nvanbenschoten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 12 of 12 files at r1.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @danhhz)

pkg/kv/kvnemesis/applier.go, line 48 at r1 (raw file):

// Applier, errors from a Step execution are saved in the Step itself.
func (a *Applier) Apply(ctx context.Context, step *Step) (retErr error) {
	a.mu.Lock()

nit: pull this into a method.

pkg/kv/kvnemesis/applier.go, line 207 at r1 (raw file):

		descs, _, err := client.RangeLookup(ctx, sender, key, roachpb.CONSISTENT, 0, false)
		if err != nil {
			log.Infof(ctx, "looking up descriptor for %s: %+v", key, err)

continue in each of these cases? Or if, else if, else?

pkg/kv/kvnemesis/generator.go, line 118 at r1 (raw file):

	config := newAllOperationsConfig()
	// TODO(dan): This sometimes returns "TransactionStatusError: already
	// committed".

Really? Even after the fix?

pkg/kv/kvnemesis/generator.go, line 330 at r1 (raw file):

	current := g.replicasFn(roachpb.Key(key))
	var changes []roachpb.ReplicationChange
	if len(current) > g.Config.NumReplicas {

Do we want to test atomic replica movement when len(current) == g.Config.NumReplicas?

pkg/kv/kvnemesis/kvnemesis_test.go, line 59 at r1 (raw file):

	defer scope.Close(t)

	// 4 nodes so we have somewhere to 3x replicated ranges to.

Is this missing a word?

pkg/kv/kvnemesis/kvnemesis_test.go, line 92 at r1 (raw file):

		q := fmt.Sprintf(`SET CLUSTER SETTING kv.closed_timestamp.target_duration = '%s'`, d)
		if _, err = sqlDB.Exec(q); err == nil {
			break

return nil?

pkg/kv/kvnemesis/kvnemesis_test.go, line 105 at r1 (raw file):

		q := fmt.Sprintf(`SET CLUSTER SETTING kv.closed_timestamp.target_duration TO DEFAULT`)
		if _, err = sqlDB.Exec(q); err == nil {
			break

return nil?

@danhhz danhhz force-pushed the kvnemesis_multinode branch from b41eb17 to 0aac581 Compare February 5, 2020 19:55
danhhz
danhhz commented Feb 5, 2020
View reviewed changes
Copy link
Contributor Author

@danhhz danhhz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @nvanbenschoten)

pkg/kv/kvnemesis/applier.go, line 48 at r1 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

nit: pull this into a method.

Done

pkg/kv/kvnemesis/applier.go, line 207 at r1 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

continue in each of these cases? Or if, else if, else?

Done

pkg/kv/kvnemesis/generator.go, line 118 at r1 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

Really? Even after the fix?

This was just code movement. Hard to tell if it's still an issue because the WriteTooOld one is much more common. I'll optimistically delete it and we can put it back if we see it again

pkg/kv/kvnemesis/generator.go, line 330 at r1 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

Do we want to test atomic replica movement when len(current) == g.Config.NumReplicas?

Ooo, good call.

pkg/kv/kvnemesis/kvnemesis_test.go, line 59 at r1 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

Is this missing a word?

Yup. Done

pkg/kv/kvnemesis/kvnemesis_test.go, line 92 at r1 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

return nil?

Yeah, oof. This worked but only accidentally

pkg/kv/kvnemesis/kvnemesis_test.go, line 105 at r1 (raw file):

Previously, nvanbenschoten (Nathan VanBenschoten) wrote…

return nil?

Ditto

pkg/kv/kvnemesis/watcher.go, line 106 at r2 (raw file):

func isRetryableRangeFeedErr(err error) bool {
	switch {

this is to make the linter happy. it suggested return !errors.Is(err, context.Canceled), but that's harder to read IMO

@danhhz
Copy link
Contributor Author

danhhz commented Feb 5, 2020

Repro'd the teamcity flake (or something that seems similar). A write from a committed txn is missing from rangefeed. The scan at the end of the test picks it up. There's no errors in the execution, so unlikely to be a nemesis issue. Maybe another rangefeed bug? I'll have to put in more logging to say.

I200205 20:14:28.047885 4919 kv/kvnemesis/kvnemesis.go:63
  1 OP  db0.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  1 OP    txn.Put(ctx, /Table/50/"e484508b", v-28) // nil
  1 OP    txn.Put(ctx, /Table/50/"e484508b", v-29) // nil
  1 OP    txn.Get(ctx, /Table/50/"e484508b") // ("v-29", nil)
  1 OP    b := &Batch{}
  1 OP    txn.CommitInBatch(ctx, b) // nil
  1 OP    return nil
  1 OP  }) // nil
...
I200205 20:14:29.023928 13 kv/kvnemesis/kvnemesis.go:87  kvs (recorded from rangefeed):
  /Table/50/"02c5bc5f" 1580933667.974282275,0 -> /BYTES/v-23
  /Table/50/"02c5bc5f" 1580933667.929227101,0 -> /BYTES/v-14
  /Table/50/"0c989982" 1580933667.940076146,1 -> /BYTES/v-4
  /Table/50/"0c989982" 1580933667.940076146,0 -> /BYTES/v-16
  /Table/50/"0c989982" 1580933667.928927189,1 -> /BYTES/v-11
  /Table/50/"1278069d" 1580933667.972101461,0 -> /BYTES/v-20
  /Table/50/"1278069d" 1580933667.912701473,0 -> /BYTES/v-6
  /Table/50/"1aba52f0" 1580933667.928927189,1 -> /BYTES/v-10
  /Table/50/"3b873180" 1580933667.928927189,1 -> /BYTES/v-12
  /Table/50/"6dbf0c5c" 1580933667.981691583,0 -> /BYTES/v-25
  /Table/50/"c9e1ee8c" 1580933667.928927189,1 -> /BYTES/v-9
  /Table/50/"c9e1ee8c" 1580933667.909137940,0 -> /BYTES/v-3
  /Table/50/"cfc8adbd" 1580933667.972101461,0 -> /BYTES/v-21
  /Table/50/"d518d349" 1580933667.986777467,0 -> /BYTES/v-27
  /Table/50/"d518d349" 1580933667.974282275,0 -> /BYTES/v-22
  /Table/50/"d518d349" 1580933667.929227101,0 -> /BYTES/v-15
  /Table/50/"e4f843de" 1580933667.971863036,0 -> /BYTES/v-19
  /Table/50/"f5d56fdf" 1580933667.929227101,0 -> /BYTES/v-13
  /Table/50/"fd31b74d" 1580933667.959842831,0 -> /BYTES/v-17
I200205 20:14:29.025448 13 kv/kvnemesis/kvnemesis.go:98  kvs (scan of latest values according to crdb):
  /Table/50/"02c5bc5f" 1580933667.974282275,0 -> /BYTES/v-23
  /Table/50/"0c989982" 1580933667.940076146,1 -> /BYTES/v-4
  /Table/50/"1278069d" 1580933667.972101461,0 -> /BYTES/v-20
  /Table/50/"1aba52f0" 1580933667.928927189,1 -> /BYTES/v-10
  /Table/50/"3b873180" 1580933667.928927189,1 -> /BYTES/v-12
  /Table/50/"6dbf0c5c" 1580933667.981691583,0 -> /BYTES/v-25
  /Table/50/"c9e1ee8c" 1580933667.928927189,1 -> /BYTES/v-9
  /Table/50/"cfc8adbd" 1580933667.972101461,0 -> /BYTES/v-21
  /Table/50/"d518d349" 1580933667.986777467,0 -> /BYTES/v-27
  /Table/50/"e484508b" 1580933668.035902630,0 -> /BYTES/v-29
  /Table/50/"e4f843de" 1580933667.971863036,0 -> /BYTES/v-19
  /Table/50/"f5d56fdf" 1580933667.929227101,0 -> /BYTES/v-13
  /Table/50/"fd31b74d" 1580933667.959842831,0 -> /BYTES/v-17
- committed txn missing write: /Table/50/"e484508b":missing->v-28 /Table/50/"e484508b":missing->v-29

@danhhz
Copy link
Contributor Author

danhhz commented Feb 5, 2020

Stressed this again with more logging and got no repros for 29m. Right as I was about to bail for the day, got a new failure. @nvanbenschoten isn't this fixed? I'm pulled pretty recently on master (7f684e4), definitely past #42939

panic: resolved timestamp 1580942700.328635872,0 equal to or above timestamp of operation {<nil> txn_id:8827da21-056f-455a-80c8-17b31ad9acc4 txn_key:"\272\022a77ce3da\000\001" txn_min_timestamp:<wall_time:1580942699322580444 > timestamp:<wall_time:1580942699322580444 >  <nil> <nil> <nil> <nil>} [recovered]

=== RUN   TestKVNemesisMultiNode
I200205 22:44:55.206857 24 base/addr_validation.go:296  [n?] server certificate addresses: IP=127.0.0.1,::1; DNS=localhost,*.local; CN=node
I200205 22:44:55.206901 24 base/addr_validation.go:342  [n?] web UI certificate addresses: IP=127.0.0.1,::1; DNS=localhost,*.local; CN=node
W200205 22:44:55.290330 24 server/status/runtime.go:308  [n?] Could not parse build timestamp: parsing time "" as "2006/01/02 15:04:05": cannot parse "" as "2006"
I200205 22:44:55.366485 24 server/server.go:988  [n?] monitoring forward clock jumps based on server.clock.forward_jump_check_enabled
I200205 22:44:55.431403 24 server/config.go:565  [n?] 1 storage engine initialized
I200205 22:44:55.431435 24 server/config.go:568  [n?] RocksDB cache size: 128 MiB
I200205 22:44:55.431450 24 server/config.go:568  [n?] store 0: in-memory, size 0 B
I200205 22:44:55.431639 24 server/server.go:1443  [n?] no stores bootstrapped and --join flag specified, awaiting init command or join with an already initialized node.
I200205 22:44:55.431690 24 server/server.go:1453  [n?] **** add additional nodes by specifying --join=127.0.0.1:37327
I200205 22:44:55.441581 24 server/node.go:334  [n?] **** cluster 91b82819-9cf2-41a2-9821-bcbbab030ef5 has been created
I200205 22:44:55.441856 24 gossip/gossip.go:395  [n1] NodeDescriptor set to node_id:1 address: attrs:<> locality: tiers: > ServerVersion: build_tag:"v20.1.0-alpha.20191118-1913-gd01dcf6-dirty" started_at:1580942695441702227 cluster_name:"" sql_address:
I200205 22:44:55.463470 24 server/node.go:438  [n1] initialized store [n1,s1]: disk (capacity=512 MiB, available=512 MiB, used=0 B, logicalBytes=13 KiB), ranges=28, leases=0, queries=0.00, writes=0.00, bytesPerReplica={p10=0.00 p25=0.00 p50=0.00 p75=0.00 p90=950.00 pMax=11033.00}, writesPerReplica={p10=0.00 p25=0.00 p50=0.00 p75=0.00 p90=0.00 pMax=0.00}
I200205 22:44:55.463573 24 storage/stores.go:240  [n1] read 0 node addresses from persistent storage
W200205 22:44:55.463611 201 storage/store.go:1538  [n1,s1,r6/1:/Table/{SystemCon…-11}] could not gossip system config: [NotLeaseHolderError] r6: replica (n1,s1):1 not lease holder; lease holder unknown
I200205 22:44:55.463708 24 server/node.go:655  [n1] connecting to gossip network to verify cluster ID...
I200205 22:44:55.464978 24 server/node.go:675  [n1] node connected via gossip and verified as part of cluster "91b82819-9cf2-41a2-9821-bcbbab030ef5"
I200205 22:44:55.465076 24 server/node.go:522  [n1] node=1: started with [=] engine(s) and attributes []
I200205 22:44:55.465139 24 server/server.go:2131  [n1] Could not start goroutine dumper worker due to: directory to store dumps could not be determined
I200205 22:44:55.465208 24 server/server.go:1573  [n1] starting https server at 127.0.0.1:46267 (use: 127.0.0.1:46267)
I200205 22:44:55.465246 24 server/server.go:1578  [n1] starting postgres server at 127.0.0.1:39597 (use: 127.0.0.1:39597)
I200205 22:44:55.465271 24 server/server.go:1580  [n1] starting grpc server at 127.0.0.1:37327
I200205 22:44:55.465296 24 server/server.go:1581  [n1] advertising CockroachDB node at 127.0.0.1:37327
I200205 22:44:55.543058 330 sql/event_log.go:132  [n1,intExec=optInToDiagnosticsStatReporting] Event: "set_cluster_setting", target: 0, info: {SettingName:diagnostics.reporting.enabled Value:true User:root}
I200205 22:44:55.587699 440 sql/event_log.go:132  [n1,intExec=set-setting] Event: "set_cluster_setting", target: 0, info: {SettingName:version Value:19.2-11 User:root}
I200205 22:44:55.608540 655 sql/event_log.go:132  [n1,intExec=initializeClusterSecret] Event: "set_cluster_setting", target: 0, info: {SettingName:cluster.secret Value:d080cebd-5fd6-4251-8059-cbac674c9939 User:root}
I200205 22:44:55.619973 710 sql/event_log.go:132  [n1,intExec=create-default-db] Event: "create_database", target: 50, info: {DatabaseName:defaultdb Statement:CREATE DATABASE IF NOT EXISTS defaultdb User:root}
I200205 22:44:55.632947 752 sql/event_log.go:132  [n1,intExec=create-default-db] Event: "create_database", target: 51, info: {DatabaseName:postgres Statement:CREATE DATABASE IF NOT EXISTS postgres User:root}
I200205 22:44:55.668158 24 server/server.go:1655  [n1] done ensuring all necessary migrations have run
I200205 22:44:55.668225 24 server/server.go:1936  [n1] serving sql connections
I200205 22:44:55.673584 950 server/server_update.go:55  [n1] no need to upgrade, cluster already at the newest version
I200205 22:44:55.675984 952 sql/event_log.go:132  [n1] Event: "node_join", target: 1, info: {Descriptor:{NodeID:1 Address:127.0.0.1:37327 Attrs: Locality:region=test,dc=dc1 ServerVersion:19.2-11 BuildTag:v20.1.0-alpha.20191118-1913-gd01dcf6-dirty StartedAt:1580942695441702227 LocalityAddress:[] ClusterName: SQLAddress:127.0.0.1:39597} ClusterID:91b82819-9cf2-41a2-9821-bcbbab030ef5 StartedAt:1580942695441702227 LastUp:1580942695441702227}
I200205 22:44:55.679843 24 base/addr_validation.go:296  [n?] server certificate addresses: IP=127.0.0.1,::1; DNS=localhost,*.local; CN=node
I200205 22:44:55.679863 24 base/addr_validation.go:342  [n?] web UI certificate addresses: IP=127.0.0.1,::1; DNS=localhost,*.local; CN=node
W200205 22:44:55.692842 24 server/status/runtime.go:308  [n?] Could not parse build timestamp: parsing time "" as "2006/01/02 15:04:05": cannot parse "" as "2006"
I200205 22:44:55.706377 24 server/server.go:988  [n?] monitoring forward clock jumps based on server.clock.forward_jump_check_enabled
I200205 22:44:55.727062 24 server/config.go:565  [n?] 1 storage engine initialized
I200205 22:44:55.727092 24 server/config.go:568  [n?] RocksDB cache size: 128 MiB
I200205 22:44:55.727103 24 server/config.go:568  [n?] store 0: in-memory, size 0 B
W200205 22:44:55.727200 24 gossip/gossip.go:1518  [n?] no incoming or outgoing connections
I200205 22:44:55.727258 24 server/server.go:1443  [n?] no stores bootstrapped and --join flag specified, awaiting init command or join with an already initialized node.
I200205 22:44:55.744945 1016 gossip/client.go:124  [n?] started gossip client to 127.0.0.1:37327
I200205 22:44:55.745373 1006 gossip/server.go:227  [n1] received initial cluster-verification connection from 127.0.0.1:35133
I200205 22:44:55.746099 24 server/node.go:655  [n?] connecting to gossip network to verify cluster ID...
I200205 22:44:55.746132 24 server/node.go:675  [n?] node connected via gossip and verified as part of cluster "91b82819-9cf2-41a2-9821-bcbbab030ef5"
I200205 22:44:55.747364 1006 gossip/server.go:227  [n1] received initial cluster-verification connection from 127.0.0.1:35133
I200205 22:44:55.813334 24 server/node.go:388  [n?] new node allocated ID 2
I200205 22:44:55.813415 24 gossip/gossip.go:395  [n2] NodeDescriptor set to node_id:2 address: attrs:<> locality: tiers: > ServerVersion: build_tag:"v20.1.0-alpha.20191118-1913-gd01dcf6-dirty" started_at:1580942695813347385 cluster_name:"" sql_address:
I200205 22:44:55.813467 24 storage/stores.go:240  [n2] read 0 node addresses from persistent storage
I200205 22:44:55.813557 24 storage/stores.go:259  [n2] wrote 1 node addresses to persistent storage
I200205 22:44:55.813942 27 storage/stores.go:259  [n1] wrote 1 node addresses to persistent storage
I200205 22:44:55.821444 24 server/node.go:630  [n2] bootstrapped store [n2,s2]
I200205 22:44:55.821590 24 server/node.go:522  [n2] node=2: started with [=] engine(s) and attributes []
I200205 22:44:55.821654 24 server/server.go:2131  [n2] Could not start goroutine dumper worker due to: directory to store dumps could not be determined
I200205 22:44:55.821711 24 server/server.go:1573  [n2] starting https server at 127.0.0.1:42165 (use: 127.0.0.1:42165)
I200205 22:44:55.821741 24 server/server.go:1578  [n2] starting postgres server at 127.0.0.1:39241 (use: 127.0.0.1:39241)
I200205 22:44:55.821769 24 server/server.go:1580  [n2] starting grpc server at 127.0.0.1:35133
I200205 22:44:55.821786 24 server/server.go:1581  [n2] advertising CockroachDB node at 127.0.0.1:35133
I200205 22:44:55.873285 24 server/server.go:1655  [n2] done ensuring all necessary migrations have run
I200205 22:44:55.873326 24 server/server.go:1936  [n2] serving sql connections
I200205 22:44:55.882906 24 base/addr_validation.go:296  [n?] server certificate addresses: IP=127.0.0.1,::1; DNS=localhost,*.local; CN=node
I200205 22:44:55.883766 24 base/addr_validation.go:342  [n?] web UI certificate addresses: IP=127.0.0.1,::1; DNS=localhost,*.local; CN=node
I200205 22:44:55.890618 1181 sql/event_log.go:132  [n2] Event: "node_join", target: 2, info: {Descriptor:{NodeID:2 Address:127.0.0.1:35133 Attrs: Locality:region=test,dc=dc2 ServerVersion:19.2-11 BuildTag:v20.1.0-alpha.20191118-1913-gd01dcf6-dirty StartedAt:1580942695813347385 LocalityAddress:[] ClusterName: SQLAddress:127.0.0.1:39241} ClusterID:91b82819-9cf2-41a2-9821-bcbbab030ef5 StartedAt:1580942695813347385 LastUp:1580942695813347385}
I200205 22:44:55.895723 1179 server/server_update.go:55  [n2] no need to upgrade, cluster already at the newest version
W200205 22:44:55.912219 24 server/status/runtime.go:308  [n?] Could not parse build timestamp: parsing time "" as "2006/01/02 15:04:05": cannot parse "" as "2006"
I200205 22:44:55.948138 24 server/server.go:988  [n?] monitoring forward clock jumps based on server.clock.forward_jump_check_enabled
I200205 22:44:55.973880 24 server/config.go:565  [n?] 1 storage engine initialized
I200205 22:44:55.973901 24 server/config.go:568  [n?] RocksDB cache size: 128 MiB
I200205 22:44:55.973911 24 server/config.go:568  [n?] store 0: in-memory, size 0 B
W200205 22:44:55.973996 24 gossip/gossip.go:1518  [n?] no incoming or outgoing connections
I200205 22:44:55.974079 24 server/server.go:1443  [n?] no stores bootstrapped and --join flag specified, awaiting init command or join with an already initialized node.
I200205 22:44:55.996695 1287 gossip/client.go:124  [n?] started gossip client to 127.0.0.1:37327
I200205 22:44:55.997201 1271 gossip/server.go:227  [n1] received initial cluster-verification connection from 127.0.0.1:40101
I200205 22:44:55.997963 24 server/node.go:655  [n?] connecting to gossip network to verify cluster ID...
I200205 22:44:55.998038 24 server/node.go:675  [n?] node connected via gossip and verified as part of cluster "91b82819-9cf2-41a2-9821-bcbbab030ef5"
I200205 22:44:55.998774 1271 gossip/server.go:227  [n1] received initial cluster-verification connection from 127.0.0.1:40101
I200205 22:44:56.032602 24 server/node.go:388  [n?] new node allocated ID 3
I200205 22:44:56.032745 24 gossip/gossip.go:395  [n3] NodeDescriptor set to node_id:3 address: attrs:<> locality: tiers: > ServerVersion: build_tag:"v20.1.0-alpha.20191118-1913-gd01dcf6-dirty" started_at:1580942696032651392 cluster_name:"" sql_address:
I200205 22:44:56.032863 24 storage/stores.go:240  [n3] read 0 node addresses from persistent storage
I200205 22:44:56.032933 24 storage/stores.go:259  [n3] wrote 2 node addresses to persistent storage
I200205 22:44:56.034956 24 server/node.go:630  [n3] bootstrapped store [n3,s3]
I200205 22:44:56.035131 24 server/node.go:522  [n3] node=3: started with [=] engine(s) and attributes []
I200205 22:44:56.035208 24 server/server.go:2131  [n3] Could not start goroutine dumper worker due to: directory to store dumps could not be determined
I200205 22:44:56.035295 24 server/server.go:1573  [n3] starting https server at 127.0.0.1:40543 (use: 127.0.0.1:40543)
I200205 22:44:56.035329 24 server/server.go:1578  [n3] starting postgres server at 127.0.0.1:36033 (use: 127.0.0.1:36033)
I200205 22:44:56.035349 24 server/server.go:1580  [n3] starting grpc server at 127.0.0.1:40101
I200205 22:44:56.035367 24 server/server.go:1581  [n3] advertising CockroachDB node at 127.0.0.1:40101
I200205 22:44:56.054654 27 storage/stores.go:259  [n1] wrote 2 node addresses to persistent storage
I200205 22:44:56.055067 292 storage/replica_command.go:1601  [n1,replicate,s1,r3/1:/System/{NodeLive…-tsd}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r3:/System/{NodeLivenessMax-tsd} [(n1,s1):1, next=2, gen=0]
I200205 22:44:56.058210 961 storage/stores.go:259  [n2] wrote 2 node addresses to persistent storage
I200205 22:44:56.059603 24 server/server.go:1655  [n3] done ensuring all necessary migrations have run
I200205 22:44:56.059643 24 server/server.go:1936  [n3] serving sql connections
I200205 22:44:56.062719 24 base/addr_validation.go:296  [n?] server certificate addresses: IP=127shutting down
.0.0.1,::1; DNS=localhost,*.local; CN=node
I200205 22:44:56.062747 24 base/addr_validation.go:342  [n?] web UI certificate addresses: IP=127.0.0.1,::1; DNS=localhost,*.local; CN=node
I200205 22:44:56.067368 292 storage/replica_raft.go:248  [n1,s1,r3/1:/System/{NodeLive…-tsd}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200205 22:44:56.081405 292 storage/store_snapshot.go:922  [n1,replicate,s1,r3/1:/System/{NodeLive…-tsd}] sending LEARNER snapshot 73e3df0a at applied index 43
I200205 22:44:56.082087 292 storage/store_snapshot.go:965  [n1,replicate,s1,r3/1:/System/{NodeLive…-tsd}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 33, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:56.083456 1560 storage/replica_raftstorage.go:780  [n2,s2,r3/2:{-}] applying LEARNER snapshot [id=73e3df0a index=43]
I200205 22:44:56.085451 1195 sql/event_log.go:132  [n3] Event: "node_join", target: 3, info: {Descriptor:{NodeID:3 Address:127.0.0.1:40101 Attrs: Locality:region=test,dc=dc3 ServerVersion:19.2-11 BuildTag:v20.1.0-alpha.20191118-1913-gd01dcf6-dirty StartedAt:1580942696032651392 LocalityAddress:[] ClusterName: SQLAddress:127.0.0.1:36033} ClusterID:91b82819-9cf2-41a2-9821-bcbbab030ef5 StartedAt:1580942696032651392 LastUp:1580942696032651392}
I200205 22:44:56.086906 1560 storage/replica_raftstorage.go:801  [n2,s2,r3/2:/System/{NodeLive…-tsd}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=73e3df0a index=43]
I200205 22:44:56.087637 292 storage/replica_command.go:1601  [n1,replicate,s1,r3/1:/System/{NodeLive…-tsd}] change replicas (add [(n2,s2):2] remove []): existing descriptor r3:/System/{NodeLivenessMax-tsd} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
W200205 22:44:56.092676 24 server/status/runtime.go:308  [n?] Could not parse build timestamp: parsing time "" as "2006/01/02 15:04:05": cannot parse "" as "2006"
I200205 22:44:56.096637 292 storage/replica_raft.go:248  [n1,s1,r3/1:/System/{NodeLive…-tsd}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200205 22:44:56.097302 292 storage/replica_command.go:1601  [n1,replicate,s1,r3/1:/System/{NodeLive…-tsd}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r3:/System/{NodeLivenessMax-tsd} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200205 22:44:56.107329 292 storage/replica_raft.go:248  [n1,s1,r3/1:/System/{NodeLive…-tsd}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3LEARNER] next=4
I200205 22:44:56.112000 292 storage/store_snapshot.go:922  [n1,replicate,s1,r3/1:/System/{NodeLive…-tsd}] sending LEARNER snapshot a4b4cd9a at applied index 47
I200205 22:44:56.112551 1605 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r3/1:/System/{NodeLive…-tsd}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200205 22:44:56.112843 292 storage/store_snapshot.go:965  [n1,replicate,s1,r3/1:/System/{NodeLive…-tsd}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 37, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:56.118781 1598 storage/replica_raftstorage.go:780  [n3,s3,r3/3:{-}] applying LEARNER snapshot [id=a4b4cd9a index=47]
I200205 22:44:56.121858 24 server/server.go:988  [n?] monitoring forward clock jumps based on server.clock.forward_jump_check_enabled
I200205 22:44:56.134162 1193 server/server_update.go:55  [n3] no need to upgrade, cluster already at the newest version
I200205 22:44:56.137293 1598 storage/replica_raftstorage.go:801  [n3,s3,r3/3:/System/{NodeLive…-tsd}] applied LEARNER snapshot [total=18ms ingestion=4@18ms id=a4b4cd9a index=47]
I200205 22:44:56.138298 292 storage/replica_command.go:1601  [n1,replicate,s1,r3/1:/System/{NodeLive…-tsd}] change replicas (add [(n3,s3):3] remove []): existing descriptor r3:/System/{NodeLivenessMax-tsd} [(n1,s1):1, (n2,s2):2, (n3,s3):3LEARNER, next=4, gen=3]
I200205 22:44:56.142507 292 storage/replica_raft.go:248  [n1,s1,r3/1:/System/{NodeLive…-tsd}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3] next=4
I200205 22:44:56.147707 292 storage/replica_command.go:1601  [n1,replicate,s1,r26/1:/NamespaceTable/{30-Max}] change replicas (add [(n3,s3):2LEARNER] remove []): existing descriptor r26:/NamespaceTable/{30-Max} [(n1,s1):1, next=2, gen=0]
I200205 22:44:56.159294 24 server/config.go:565  [n?] 1 storage engine initialized
I200205 22:44:56.159312 24 server/config.go:568  [n?] RocksDB cache size: 128 MiB
I200205 22:44:56.159322 24 server/config.go:568  [n?] store 0: in-memory, size 0 B
W200205 22:44:56.159404 24 gossip/gossip.go:1518  [n?] no incoming or outgoing connections
I200205 22:44:56.159452 24 server/server.go:1443  [n?] no stores bootstrapped and --join flag specified, awaiting init command or join with an already initialized node.
I200205 22:44:56.174131 1656 gossip/client.go:124  [n?] started gossip client to 127.0.0.1:37327
I200205 22:44:56.174813 1686 gossip/server.go:227  [n1] received initial cluster-verification connection from 127.0.0.1:37605
I200205 22:44:56.176571 292 storage/replica_raft.go:248  [n1,s1,r26/1:/NamespaceTable/{30-Max}] proposing SIMPLE(l2) ADD_REPLICA[(n3,s3):2LEARNER]: after=[(n1,s1):1 (n3,s3):2LEARNER] next=3
I200205 22:44:56.178257 24 server/node.go:655  [n?] connecting to gossip network to verify cluster ID...
I200205 22:44:56.178322 24 server/node.go:675  [n?] node connected via gossip and verified as part of cluster "91b82819-9cf2-41a2-9821-bcbbab030ef5"
I200205 22:44:56.178596 1686 gossip/server.go:227  [n1] received initial cluster-verification connection from 127.0.0.1:37605
I200205 22:44:56.178885 1648 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r26/1:/NamespaceTable/{30-Max}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):2LEARNER
I200205 22:44:56.179446 292 storage/store_snapshot.go:922  [n1,replicate,s1,r26/1:/NamespaceTable/{30-Max}] sending LEARNER snapshot 75e23931 at applied index 22
I200205 22:44:56.179636 292 storage/store_snapshot.go:965  [n1,replicate,s1,r26/1:/NamespaceTable/{30-Max}] streamed snapshot to (n3,s3):2LEARNER: kv pairs: 34, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:56.183588 1692 storage/replica_raftstorage.go:780  [n3,s3,r26/2:{-}] applying LEARNER snapshot [id=75e23931 index=22]
I200205 22:44:56.190357 1692 storage/replica_raftstorage.go:801  [n3,s3,r26/2:/NamespaceTable/{30-Max}] applied LEARNER snapshot [total=7ms ingestion=4@6ms id=75e23931 index=22]
I200205 22:44:56.190998 292 storage/replica_command.go:1601  [n1,replicate,s1,r26/1:/NamespaceTable/{30-Max}] change replicas (add [(n3,s3):2] remove []): existing descriptor r26:/NamespaceTable/{30-Max} [(n1,s1):1, (n3,s3):2LEARNER, next=3, gen=1]
I200205 22:44:56.194529 292 storage/replica_raft.go:248  [n1,s1,r26/1:/NamespaceTable/{30-Max}] proposing SIMPLE(v2) ADD_REPLICA[(n3,s3):2]: after=[(n1,s1):1 (n3,s3):2] next=3
I200205 22:44:56.195082 292 storage/replica_command.go:1601  [n1,replicate,s1,r26/1:/NamespaceTable/{30-Max}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r26:/NamespaceTable/{30-Max} [(n1,s1):1, (n3,s3):2, next=3, gen=2]
I200205 22:44:56.214174 292 storage/replica_raft.go:248  [n1,s1,r26/1:/NamespaceTable/{30-Max}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n2,s2):3LEARNER] next=4
I200205 22:44:56.216662 292 storage/store_snapshot.go:922  [n1,replicate,s1,r26/1:/NamespaceTable/{30-Max}] sending LEARNER snapshot a36bcbde at applied index 26
I200205 22:44:56.216889 292 storage/store_snapshot.go:965  [n1,replicate,s1,r26/1:/NamespaceTable/{30-Max}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 38, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:56.217216 1732 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r26/1:/NamespaceTable/{30-Max}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):3LEARNER
I200205 22:44:56.217782 1521 storage/replica_raftstorage.go:780  [n2,s2,r26/3:{-}] applying LEARNER snapshot [id=a36bcbde index=26]
I200205 22:44:56.220675 1521 storage/replica_raftstorage.go:801  [n2,s2,r26/3:/NamespaceTable/{30-Max}] applied LEARNER snapshot [total=3ms ingestion=4@2ms id=a36bcbde index=26]
I200205 22:44:56.222072 292 storage/replica_command.go:1601  [n1,replicate,s1,r26/1:/NamespaceTable/{30-Max}] change replicas (add [(n2,s2):3] remove []): existing descriptor r26:/NamespaceTable/{30-Max} [(n1,s1):1, (n3,s3):2, (n2,s2):3LEARNER, next=4, gen=3]
I200205 22:44:56.287506 24 server/node.go:388  [n?] new node allocated ID 4
I200205 22:44:56.287619 24 gossip/gossip.go:395  [n4] NodeDescriptor set to node_id:4 address: attrs:<> locality: tiers: > ServerVersion: build_tag:"v20.1.0-alpha.20191118-1913-gd01dcf6-dirty" started_at:1580942696287519230 cluster_name:"" sql_address:
I200205 22:44:56.287691 24 storage/stores.go:240  [n4] read 0 node addresses from persistent storage
I200205 22:44:56.287742 24 storage/stores.go:259  [n4] wrote 3 node addresses to persistent storage
I200205 22:44:56.288397 27 storage/stores.go:259  [n1] wrote 3 node addresses to persistent storage
I200205 22:44:56.289033 961 storage/stores.go:259  [n2] wrote 3 node addresses to persistent storage
I200205 22:44:56.290638 292 storage/replica_raft.go:248  [n1,s1,r26/1:/NamespaceTable/{30-Max}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n3,s3):2 (n2,s2):3] next=4
I200205 22:44:56.290982 24 server/node.go:630  [n4] bootstrapped store [n4,s4]
I200205 22:44:56.291151 24 server/node.go:522  [n4] node=4: started with [=] engine(s) and attributes []
I200205 22:44:56.291833 1037 storage/stores.go:259  [n3] wrote 3 node addresses to persistent storage
I200205 22:44:56.297002 24 server/server.go:2131  [n4] Could not start goroutine dumper worker due to: directory to store dumps could not be determined
I200205 22:44:56.297266 24 server/server.go:1573  [n4] starting https server at 127.0.0.1:33833 (use: 127.0.0.1:33833)
I200205 22:44:56.297340 24 server/server.go:1578  [n4] starting postgres server at 127.0.0.1:39357 (use: 127.0.0.1:39357)
I200205 22:44:56.297400 24 server/server.go:1580  [n4] starting grpc server at 127.0.0.1:37605
I200205 22:44:56.297432 24 server/server.go:1581  [n4] advertising CockroachDB node at 127.0.0.1:37605
I200205 22:44:56.324455 292 storage/replica_command.go:1601  [n1,replicate,s1,r27/1:/{NamespaceTab…-Table/32}] change replicas (add [(n3,s3):2LEARNER] remove []): existing descriptor r27:/{NamespaceTable/Max-Table/32} [(n1,s1):1, next=2, gen=0]
I200205 22:44:56.326888 24 server/server.go:1655  [n4] done ensuring all necessary migrations have run
I200205 22:44:56.326941 24 server/server.go:1936  [n4] serving sql connections
W200205 22:44:56.326994 24 jobs/registry.go:392  [n4] unable to get node liveness: node not in the liveness table
I200205 22:44:56.327195 292 storage/replica_raft.go:248  [n1,s1,r27/1:/{NamespaceTab…-Table/32}] proposing SIMPLE(l2) ADD_REPLICA[(n3,s3):2LEARNER]: after=[(n1,s1):1 (n3,s3):2LEARNER] next=3
I200205 22:44:56.329765 292 storage/store_snapshot.go:922  [n1,replicate,s1,r27/1:/{NamespaceTab…-Table/32}] sending LEARNER snapshot cbcbf0f6 at applied index 16
I200205 22:44:56.329864 292 storage/store_snapshot.go:965  [n1,replicate,s1,r27/1:/{NamespaceTab…-Table/32}] streamed snapshot to (n3,s3):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:56.330505 1821 storage/replica_raftstorage.go:780  [n3,s3,r27/2:{-}] applying LEARNER snapshot [id=cbcbf0f6 index=16]
I200205 22:44:56.335893 24 testutils/testcluster/testcluster.go:780  [n1,s1] only sees 3/4 available nodes
I200205 22:44:56.338953 1821 storage/replica_raftstorage.go:801  [n3,s3,r27/2:/{NamespaceTab…-Table/32}] applied LEARNER snapshot [total=8ms ingestion=4@1ms id=cbcbf0f6 index=16]
I200205 22:44:56.339454 292 storage/replica_command.go:1601  [n1,replicate,s1,r27/1:/{NamespaceTab…-Table/32}] change replicas (add [(n3,s3):2] remove []): existing descriptor r27:/{NamespaceTable/Max-Table/32} [(n1,s1):1, (n3,s3):2LEARNER, next=3, gen=1]
I200205 22:44:56.342586 292 storage/replica_raft.go:248  [n1,s1,r27/1:/{NamespaceTab…-Table/32}] proposing SIMPLE(v2) ADD_REPLICA[(n3,s3):2]: after=[(n1,s1):1 (n3,s3):2] next=3
I200205 22:44:56.345992 292 storage/replica_command.go:1601  [n1,replicate,s1,r27/1:/{NamespaceTab…-Table/32}] change replicas (add [(n4,s4):3LEARNER] remove []): existing descriptor r27:/{NamespaceTable/Max-Table/32} [(n1,s1):1, (n3,s3):2, next=3, gen=2]
I200205 22:44:56.359396 1762 server/server_update.go:55  [n4] no need to upgrade, cluster already at the newest version
I200205 22:44:56.359804 1940 sql/event_log.go:132  [n4] Event: "node_join", target: 4, info: {Descriptor:{NodeID:4 Address:127.0.0.1:37605 Attrs: Locality:region=test,dc=dc4 ServerVersion:19.2-11 BuildTag:v20.1.0-alpha.20191118-1913-gd01dcf6-dirty StartedAt:1580942696287519230 LocalityAddress:[] ClusterName: SQLAddress:127.0.0.1:39357} ClusterID:91b82819-9cf2-41a2-9821-bcbbab030ef5 StartedAt:1580942696287519230 LastUp:1580942696287519230}
I200205 22:44:56.365256 292 storage/replica_raft.go:248  [n1,s1,r27/1:/{NamespaceTab…-Table/32}] proposing SIMPLE(l3) ADD_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3LEARNER] next=4
I200205 22:44:56.400010 2094 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r27/1:/{NamespaceTab…-Table/32}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):3LEARNER
I200205 22:44:56.400128 292 storage/store_snapshot.go:922  [n1,replicate,s1,r27/1:/{NamespaceTab…-Table/32}] sending LEARNER snapshot f1bb54b3 at applied index 20
I200205 22:44:56.400265 292 storage/store_snapshot.go:965  [n1,replicate,s1,r27/1:/{NamespaceTab…-Table/32}] streamed snapshot to (n4,s4):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:56.400858 2086 storage/replica_raftstorage.go:780  [n4,s4,r27/3:{-}] applying LEARNER snapshot [id=f1bb54b3 index=20]
I200205 22:44:56.410087 2086 storage/replica_raftstorage.go:801  [n4,s4,r27/3:/{NamespaceTab…-Table/32}] applied LEARNER snapshot [total=9ms ingestion=4@9ms id=f1bb54b3 index=20]
I200205 22:44:56.411164 292 storage/replica_command.go:1601  [n1,replicate,s1,r27/1:/{NamespaceTab…-Table/32}] change replicas (add [(n4,s4):3] remove []): existing descriptor r27:/{NamespaceTable/Max-Table/32} [(n1,s1):1, (n3,s3):2, (n4,s4):3LEARNER, next=4, gen=3]
I200205 22:44:56.416197 292 storage/replica_raft.go:248  [n1,s1,r27/1:/{NamespaceTab…-Table/32}] proposing SIMPLE(v3) ADD_REPLICA[(n4,s4):3]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3] next=4
I200205 22:44:56.418700 292 storage/replica_command.go:1601  [n1,replicate,s1,r8/1:/Table/1{2-3}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r8:/Table/1{2-3} [(n1,s1):1, next=2, gen=0]
I200205 22:44:56.421264 292 storage/replica_raft.go:248  [n1,s1,r8/1:/Table/1{2-3}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200205 22:44:56.438765 292 storage/store_snapshot.go:922  [n1,replicate,s1,r8/1:/Table/1{2-3}] sending LEARNER snapshot bed8fec8 at applied index 41
I200205 22:44:56.438878 2049 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r8/1:/Table/1{2-3}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200205 22:44:56.438971 292 storage/store_snapshot.go:965  [n1,replicate,s1,r8/1:/Table/1{2-3}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 51, log entries: 0, rate-limit: 8.0 MiB/sec, 0.02s
I200205 22:44:56.439675 2147 storage/replica_raftstorage.go:780  [n4,s4,r8/2:{-}] applying LEARNER snapshot [id=bed8fec8 index=41]
I200205 22:44:56.442308 2147 storage/replica_raftstorage.go:801  [n4,s4,r8/2:/Table/1{2-3}] applied LEARNER snapshot [total=3ms ingestion=4@2ms id=bed8fec8 index=41]
I200205 22:44:56.445853 292 storage/replica_command.go:1601  [n1,replicate,s1,r8/1:/Table/1{2-3}] change replicas (add [(n4,s4):2] remove []): existing descriptor r8:/Table/1{2-3} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200205 22:44:56.449849 292 storage/replica_raft.go:248  [n1,s1,r8/1:/Table/1{2-3}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200205 22:44:56.451218 292 storage/replica_command.go:1601  [n1,replicate,s1,r8/1:/Table/1{2-3}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r8:/Table/1{2-3} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200205 22:44:56.461508 292 storage/replica_raft.go:248  [n1,s1,r8/1:/Table/1{2-3}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3LEARNER] next=4
I200205 22:44:56.467364 292 storage/store_snapshot.go:922  [n1,replicate,s1,r8/1:/Table/1{2-3}] sending LEARNER snapshot afa55dd3 at applied index 45
I200205 22:44:56.467605 292 storage/store_snapshot.go:965  [n1,replicate,s1,r8/1:/Table/1{2-3}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 55, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:56.468899 2226 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r8/1:/Table/1{2-3}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):3LEARNER
I200205 22:44:56.471647 2213 storage/replica_raftstorage.go:780  [n2,s2,r8/3:{-}] applying LEARNER snapshot [id=afa55dd3 index=45]
I200205 22:44:56.477673 2213 storage/replica_raftstorage.go:801  [n2,s2,r8/3:/Table/1{2-3}] applied LEARNER snapshot [total=6ms ingestion=4@5ms id=afa55dd3 index=45]
I200205 22:44:56.478187 292 storage/replica_command.go:1601  [n1,replicate,s1,r8/1:/Table/1{2-3}] change replicas (add [(n2,s2):3] remove []): existing descriptor r8:/Table/1{2-3} [(n1,s1):1, (n4,s4):2, (n2,s2):3LEARNER, next=4, gen=3]
I200205 22:44:56.484352 292 storage/replica_raft.go:248  [n1,s1,r8/1:/Table/1{2-3}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3] next=4
I200205 22:44:56.489560 292 storage/replica_command.go:1601  [n1,replicate,s1,r7/1:/Table/1{1-2}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r7:/Table/1{1-2} [(n1,s1):1, next=2, gen=0]
I200205 22:44:56.497954 292 storage/replica_raft.go:248  [n1,s1,r7/1:/Table/1{1-2}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200205 22:44:56.500204 292 storage/store_snapshot.go:922  [n1,replicate,s1,r7/1:/Table/1{1-2}] sending LEARNER snapshot 177ebbf3 at applied index 19
I200205 22:44:56.500313 2275 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r7/1:/Table/1{1-2}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200205 22:44:56.500321 292 storage/store_snapshot.go:965  [n1,replicate,s1,r7/1:/Table/1{1-2}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 7, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:56.503200 2245 storage/replica_raftstorage.go:780  [n4,s4,r7/2:{-}] applying LEARNER snapshot [id=177ebbf3 index=19]
I200205 22:44:56.507293 2245 storage/replica_raftstorage.go:801  [n4,s4,r7/2:/Table/1{1-2}] applied LEARNER snapshot [total=4ms ingestion=4@4ms id=177ebbf3 index=19]
I200205 22:44:56.507939 292 storage/replica_command.go:1601  [n1,replicate,s1,r7/1:/Table/1{1-2}] change replicas (add [(n4,s4):2] remove []): existing descriptor r7:/Table/1{1-2} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200205 22:44:56.510631 292 storage/replica_raft.go:248  [n1,s1,r7/1:/Table/1{1-2}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200205 22:44:56.511120 292 storage/replica_command.go:1601  [n1,replicate,s1,r7/1:/Table/1{1-2}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r7:/Table/1{1-2} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200205 22:44:56.526843 292 storage/replica_raft.go:248  [n1,s1,r7/1:/Table/1{1-2}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3LEARNER] next=4
I200205 22:44:56.529912 292 storage/store_snapshot.go:922  [n1,replicate,s1,r7/1:/Table/1{1-2}] sending LEARNER snapshot b0e561f7 at applied index 23
I200205 22:44:56.530047 292 storage/store_snapshot.go:965  [n1,replicate,s1,r7/1:/Table/1{1-2}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 11, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:56.530055 2268 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r7/1:/Table/1{1-2}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200205 22:44:56.533612 2293 storage/replica_raftstorage.go:780  [n3,s3,r7/3:{-}] applying LEARNER snapshot [id=b0e561f7 index=23]
I200205 22:44:56.537711 2293 storage/replica_raftstorage.go:801  [n3,s3,r7/3:/Table/1{1-2}] applied LEARNER snapshot [total=4ms ingestion=4@4ms id=b0e561f7 index=23]
I200205 22:44:56.538353 292 storage/replica_command.go:1601  [n1,replicate,s1,r7/1:/Table/1{1-2}] change replicas (add [(n3,s3):3] remove []): existing descriptor r7:/Table/1{1-2} [(n1,s1):1, (n4,s4):2, (n3,s3):3LEARNER, next=4, gen=3]
I200205 22:44:56.541199 292 storage/replica_raft.go:248  [n1,s1,r7/1:/Table/1{1-2}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3] next=4
I200205 22:44:56.545952 292 storage/replica_command.go:1601  [n1,replicate,s1,r19/1:/Table/2{3-4}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r19:/Table/2{3-4} [(n1,s1):1, next=2, gen=0]
I200205 22:44:56.555252 292 storage/replica_raft.go:248  [n1,s1,r19/1:/Table/2{3-4}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200205 22:44:56.557888 2252 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r19/1:/Table/2{3-4}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):2LEARNER
I200205 22:44:56.558204 292 storage/store_snapshot.go:922  [n1,replicate,s1,r19/1:/Table/2{3-4}] sending LEARNER snapshot 5723e08e at applied index 17
I200205 22:44:56.558397 292 storage/store_snapshot.go:965  [n1,replicate,s1,r19/1:/Table/2{3-4}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:56.561504 2356 storage/replica_raftstorage.go:780  [n2,s2,r19/2:{-}] applying LEARNER snapshot [id=5723e08e index=17]
I200205 22:44:56.565262 2356 storage/replica_raftstorage.go:801  [n2,s2,r19/2:/Table/2{3-4}] applied LEARNER snapshot [total=4ms ingestion=4@3ms id=5723e08e index=17]
I200205 22:44:56.566553 292 storage/replica_command.go:1601  [n1,replicate,s1,r19/1:/Table/2{3-4}] change replicas (add [(n2,s2):2] remove []): existing descriptor r19:/Table/2{3-4} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
I200205 22:44:56.569526 292 storage/replica_raft.go:248  [n1,s1,r19/1:/Table/2{3-4}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200205 22:44:56.570024 292 storage/replica_command.go:1601  [n1,replicate,s1,r19/1:/Table/2{3-4}] change replicas (add [(n4,s4):3LEARNER] remove []): existing descriptor r19:/Table/2{3-4} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200205 22:44:56.583148 292 storage/replica_raft.go:248  [n1,s1,r19/1:/Table/2{3-4}] proposing SIMPLE(l3) ADD_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3LEARNER] next=4
I200205 22:44:56.586722 292 storage/store_snapshot.go:922  [n1,replicate,s1,r19/1:/Table/2{3-4}] sending LEARNER snapshot 991d61fc at applied index 21
I200205 22:44:56.587158 2361 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r19/1:/Table/2{3-4}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):3LEARNER
I200205 22:44:56.587406 292 storage/store_snapshot.go:965  [n1,replicate,s1,r19/1:/Table/2{3-4}] streamed snapshot to (n4,s4):3LEARNER: kv pairs: 14, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:56.588129 2332 storage/replica_raftstorage.go:780  [n4,s4,r19/3:{-}] applying LEARNER snapshot [id=991d61fc index=21]
I200205 22:44:56.595119 2332 storage/replica_raftstorage.go:801  [n4,s4,r19/3:/Table/2{3-4}] applied LEARNER snapshot [total=7ms ingestion=4@7ms id=991d61fc index=21]
I200205 22:44:56.596120 292 storage/replica_command.go:1601  [n1,replicate,s1,r19/1:/Table/2{3-4}] change replicas (add [(n4,s4):3] remove []): existing descriptor r19:/Table/2{3-4} [(n1,s1):1, (n2,s2):2, (n4,s4):3LEARNER, next=4, gen=3]
I200205 22:44:56.598944 292 storage/replica_raft.go:248  [n1,s1,r19/1:/Table/2{3-4}] proposing SIMPLE(v3) ADD_REPLICA[(n4,s4):3]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3] next=4
I200205 22:44:56.601033 292 storage/replica_command.go:1601  [n1,replicate,s1,r17/1:/Table/2{1-2}] change replicas (add [(n3,s3):2LEARNER] remove []): existing descriptor r17:/Table/2{1-2} [(n1,s1):1, next=2, gen=0]
I200205 22:44:56.603725 292 storage/replica_raft.go:248  [n1,s1,r17/1:/Table/2{1-2}] proposing SIMPLE(l2) ADD_REPLICA[(n3,s3):2LEARNER]: after=[(n1,s1):1 (n3,s3):2LEARNER] next=3
I200205 22:44:56.614641 2365 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r17/1:/Table/2{1-2}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):2LEARNER
I200205 22:44:56.614841 292 storage/store_snapshot.go:922  [n1,replicate,s1,r17/1:/Table/2{1-2}] sending LEARNER snapshot bb3be127 at applied index 21
I200205 22:44:56.614961 292 storage/store_snapshot.go:965  [n1,replicate,s1,r17/1:/Table/2{1-2}] streamed snapshot to (n3,s3):2LEARNER: kv pairs: 11, log entries: 0, rate-limit: 8.0 MiB/sec, 0.01s
I200205 22:44:56.619771 2399 storage/replica_raftstorage.go:780  [n3,s3,r17/2:{-}] applying LEARNER snapshot [id=bb3be127 index=21]
I200205 22:44:56.624328 2399 storage/replica_raftstorage.go:801  [n3,s3,r17/2:/Table/2{1-2}] applied LEARNER snapshot [total=4ms ingestion=4@2ms id=bb3be127 index=21]
I200205 22:44:56.625002 292 storage/replica_command.go:1601  [n1,replicate,s1,r17/1:/Table/2{1-2}] change replicas (add [(n3,s3):2] remove []): existing descriptor r17:/Table/2{1-2} [(n1,s1):1, (n3,s3):2LEARNER, next=3, gen=1]
I200205 22:44:56.628220 292 storage/replica_raft.go:248  [n1,s1,r17/1:/Table/2{1-2}] proposing SIMPLE(v2) ADD_REPLICA[(n3,s3):2]: after=[(n1,s1):1 (n3,s3):2] next=3
I200205 22:44:56.628795 292 storage/replica_command.go:1601  [n1,replicate,s1,r17/1:/Table/2{1-2}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r17:/Table/2{1-2} [(n1,s1):1, (n3,s3):2, next=3, gen=2]
I200205 22:44:56.640773 292 storage/replica_raft.go:248  [n1,s1,r17/1:/Table/2{1-2}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n2,s2):3LEARNER] next=4
I200205 22:44:56.650265 292 storage/store_snapshot.go:922  [n1,replicate,s1,r17/1:/Table/2{1-2}] sending LEARNER snapshot 9219e582 at applied index 25
I200205 22:44:56.650941 292 storage/store_snapshot.go:965  [n1,replicate,s1,r17/1:/Table/2{1-2}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 15, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:56.650309 2421 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r17/1:/Table/2{1-2}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):3LEARNER
I200205 22:44:56.651697 2419 storage/replica_raftstorage.go:780  [n2,s2,r17/3:{-}] applying LEARNER snapshot [id=9219e582 index=25]
I200205 22:44:56.655075 2419 storage/replica_raftstorage.go:801  [n2,s2,r17/3:/Table/2{1-2}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=9219e582 index=25]
I200205 22:44:56.657333 292 storage/replica_command.go:1601  [n1,replicate,s1,r17/1:/Table/2{1-2}] change replicas (add [(n2,s2):3] remove []): existing descriptor r17:/Table/2{1-2} [(n1,s1):1, (n3,s3):2, (n2,s2):3LEARNER, next=4, gen=3]
I200205 22:44:56.659966 292 storage/replica_raft.go:248  [n1,s1,r17/1:/Table/2{1-2}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n3,s3):2 (n2,s2):3] next=4
I200205 22:44:56.662377 292 storage/queue.go:1135  [n1,replicate] purgatory is now empty
I200205 22:44:56.662790 2440 storage/replica_command.go:1601  [n1,replicate,s1,r9/1:/Table/1{3-4}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r9:/Table/1{3-4} [(n1,s1):1, next=2, gen=0]
I200205 22:44:56.666748 2440 storage/replica_raft.go:248  [n1,s1,r9/1:/Table/1{3-4}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200205 22:44:56.668541 2351 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r9/1:/Table/1{3-4}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):2LEARNER
I200205 22:44:56.668862 2440 storage/store_snapshot.go:922  [n1,replicate,s1,r9/1:/Table/1{3-4}] sending LEARNER snapshot 6b535744 at applied index 70
I200205 22:44:56.669319 2440 storage/store_snapshot.go:965  [n1,replicate,s1,r9/1:/Table/1{3-4}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 156, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:56.670294 2447 storage/replica_raftstorage.go:780  [n2,s2,r9/2:{-}] applying LEARNER snapshot [id=6b535744 index=70]
I200205 22:44:56.677368 2447 storage/replica_raftstorage.go:801  [n2,s2,r9/2:/Table/1{3-4}] applied LEARNER snapshot [total=7ms ingestion=4@7ms id=6b535744 index=70]
I200205 22:44:56.677898 2440 storage/replica_command.go:1601  [n1,replicate,s1,r9/1:/Table/1{3-4}] change replicas (add [(n2,s2):2] remove []): existing descriptor r9:/Table/1{3-4} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
I200205 22:44:56.680666 2440 storage/replica_raft.go:248  [n1,s1,r9/1:/Table/1{3-4}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200205 22:44:56.681201 2440 storage/replica_command.go:1601  [n1,replicate,s1,r9/1:/Table/1{3-4}] change replicas (add [(n4,s4):3LEARNER] remove []): existing descriptor r9:/Table/1{3-4} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200205 22:44:56.691382 2440 storage/replica_raft.go:248  [n1,s1,r9/1:/Table/1{3-4}] proposing SIMPLE(l3) ADD_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3LEARNER] next=4
I200205 22:44:56.694106 2440 storage/store_snapshot.go:922  [n1,replicate,s1,r9/1:/Table/1{3-4}] sending LEARNER snapshot 905db337 at applied index 77
I200205 22:44:56.694655 2440 storage/store_snapshot.go:965  [n1,replicate,s1,r9/1:/Table/1{3-4}] streamed snapshot to (n4,s4):3LEARNER: kv pairs: 165, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:56.694855 2505 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r9/1:/Table/1{3-4}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):3LEARNER
I200205 22:44:56.695568 2503 storage/replica_raftstorage.go:780  [n4,s4,r9/3:{-}] applying LEARNER snapshot [id=905db337 index=77]
I200205 22:44:56.699714 2503 storage/replica_raftstorage.go:801  [n4,s4,r9/3:/Table/1{3-4}] applied LEARNER snapshot [total=4ms ingestion=4@4ms id=905db337 index=77]
I200205 22:44:56.703955 2440 storage/replica_command.go:1601  [n1,replicate,s1,r9/1:/Table/1{3-4}] change replicas (add [(n4,s4):3] remove []): existing descriptor r9:/Table/1{3-4} [(n1,s1):1, (n2,s2):2, (n4,s4):3LEARNER, next=4, gen=3]
I200205 22:44:56.708414 2440 storage/replica_raft.go:248  [n1,s1,r9/1:/Table/1{3-4}] proposing SIMPLE(v3) ADD_REPLICA[(n4,s4):3]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3] next=4
I200205 22:44:56.711851 24 storage/replica_command.go:1601  [n1,replicate,s1,r5/1:/{Systemtse-Table/System…}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r5:/{Systemtse-Table/SystemConfigSpan/Start} [(n1,s1):1, next=2, gen=0]
I200205 22:44:56.771844 24 storage/replica_raft.go:248  [n1,s1,r5/1:/{Systemtse-Table/System…}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200205 22:44:56.784857 24 storage/store_snapshot.go:922  [n1,replicate,s1,r5/1:/{Systemtse-Table/System…}] sending LEARNER snapshot 806d4730 at applied index 15
I200205 22:44:56.785051 24 storage/store_snapshot.go:965  [n1,replicate,s1,r5/1:/{Systemtse-Table/System…}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.01s
I200205 22:44:56.785125 2479 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r5/1:/{Systemtse-Table/System…}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200205 22:44:56.785924 2553 storage/replica_raftstorage.go:780  [n4,s4,r5/2:{-}] applying LEARNER snapshot [id=806d4730 index=15]
I200205 22:44:56.792163 2553 storage/replica_raftstorage.go:801  [n4,s4,r5/2:/{Systemtse-Table/System…}] applied LEARNER snapshot [total=6ms ingestion=4@6ms id=806d4730 index=15]
I200205 22:44:56.796302 24 storage/replica_command.go:1601  [n1,replicate,s1,r5/1:/{Systemtse-Table/System…}] change replicas (add [(n4,s4):2] remove []): existing descriptor r5:/{Systemtse-Table/SystemConfigSpan/Start} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200205 22:44:56.812042 24 storage/replica_raft.go:248  [n1,s1,r5/1:/{Systemtse-Table/System…}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200205 22:44:56.812661 24 storage/replica_command.go:1601  [n1,replicate,s1,r5/1:/{Systemtse-Table/System…}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r5:/{Systemtse-Table/SystemConfigSpan/Start} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200205 22:44:56.824268 24 storage/replica_raft.go:248  [n1,s1,r5/1:/{Systemtse-Table/System…}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3LEARNER] next=4
I200205 22:44:56.826528 24 storage/store_snapshot.go:922  [n1,replicate,s1,r5/1:/{Systemtse-Table/System…}] sending LEARNER snapshot 5315130a at applied index 19
I200205 22:44:56.826662 24 storage/store_snapshot.go:965  [n1,replicate,s1,r5/1:/{Systemtse-Table/System…}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:56.826718 2481 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r5/1:/{Systemtse-Table/System…}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200205 22:44:56.827323 2535 storage/replica_raftstorage.go:780  [n3,s3,r5/3:{-}] applying LEARNER snapshot [id=5315130a index=19]
I200205 22:44:56.832920 2535 storage/replica_raftstorage.go:801  [n3,s3,r5/3:/{Systemtse-Table/System…}] applied LEARNER snapshot [total=6ms ingestion=4@5ms id=5315130a index=19]
I200205 22:44:56.834009 24 storage/replica_command.go:1601  [n1,replicate,s1,r5/1:/{Systemtse-Table/System…}] change replicas (add [(n3,s3):3] remove []): existing descriptor r5:/{Systemtse-Table/SystemConfigSpan/Start} [(n1,s1):1, (n4,s4):2, (n3,s3):3LEARNER, next=4, gen=3]
I200205 22:44:56.837801 24 storage/replica_raft.go:248  [n1,s1,r5/1:/{Systemtse-Table/System…}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3] next=4
I200205 22:44:56.842306 24 storage/replica_command.go:1601  [n1,replicate,s1,r4/1:/System{/tsd-tse}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r4:/System{/tsd-tse} [(n1,s1):1, next=2, gen=0]
I200205 22:44:56.846973 24 storage/replica_raft.go:248  [n1,s1,r4/1:/System{/tsd-tse}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200205 22:44:56.850051 2592 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r4/1:/System{/tsd-tse}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200205 22:44:56.850056 24 storage/store_snapshot.go:922  [n1,replicate,s1,r4/1:/System{/tsd-tse}] sending LEARNER snapshot 38787a38 at applied index 27
I200205 22:44:56.901424 24 storage/store_snapshot.go:965  [n1,replicate,s1,r4/1:/System{/tsd-tse}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 2594, log entries: 0, rate-limit: 8.0 MiB/sec, 0.05s
I200205 22:44:56.902652 2432 storage/replica_raftstorage.go:780  [n4,s4,r4/2:{-}] applying LEARNER snapshot [id=38787a38 index=27]
I200205 22:44:56.911700 2432 storage/replica_raftstorage.go:801  [n4,s4,r4/2:/System{/tsd-tse}] applied LEARNER snapshot [total=9ms ingestion=4@9ms id=38787a38 index=27]
I200205 22:44:56.913146 24 storage/replica_command.go:1601  [n1,replicate,s1,r4/1:/System{/tsd-tse}] change replicas (add [(n4,s4):2] remove []): existing descriptor r4:/System{/tsd-tse} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200205 22:44:56.918318 24 storage/replica_raft.go:248  [n1,s1,r4/1:/System{/tsd-tse}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200205 22:44:56.918847 24 storage/replica_command.go:1601  [n1,replicate,s1,r4/1:/System{/tsd-tse}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r4:/System{/tsd-tse} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200205 22:44:56.931440 24 storage/replica_raft.go:248  [n1,s1,r4/1:/System{/tsd-tse}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3LEARNER] next=4
I200205 22:44:56.933886 24 storage/store_snapshot.go:922  [n1,replicate,s1,r4/1:/System{/tsd-tse}] sending LEARNER snapshot ad392440 at applied index 31
I200205 22:44:56.937134 2606 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r4/1:/System{/tsd-tse}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200205 22:44:56.942228 109 gossip/gossip.go:1532  [n1] node has connected to cluster via gossip
I200205 22:44:56.942384 109 storage/stores.go:259  [n1] wrote 3 node addresses to persistent storage
I200205 22:44:56.998606 24 storage/store_snapshot.go:965  [n1,replicate,s1,r4/1:/System{/tsd-tse}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 2598, log entries: 0, rate-limit: 8.0 MiB/sec, 0.07s
I200205 22:44:56.998989 2675 storage/replica_raftstorage.go:780  [n3,s3,r4/3:{-}] applying LEARNER snapshot [id=ad392440 index=31]
I200205 22:44:57.003323 2675 storage/replica_raftstorage.go:801  [n3,s3,r4/3:/System{/tsd-tse}] applied LEARNER snapshot [total=4ms ingestion=4@4ms id=ad392440 index=31]
I200205 22:44:57.003924 24 storage/replica_command.go:1601  [n1,replicate,s1,r4/1:/System{/tsd-tse}] change replicas (add [(n3,s3):3] remove []): existing descriptor r4:/System{/tsd-tse} [(n1,s1):1, (n4,s4):2, (n3,s3):3LEARNER, next=4, gen=3]
I200205 22:44:57.008126 24 storage/replica_raft.go:248  [n1,s1,r4/1:/System{/tsd-tse}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3] next=4
I200205 22:44:57.010917 24 storage/replica_command.go:1601  [n1,replicate,s1,r6/1:/Table/{SystemCon…-11}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r6:/Table/{SystemConfigSpan/Start-11} [(n1,s1):1, next=2, gen=0]
I200205 22:44:57.020778 24 storage/replica_raft.go:248  [n1,s1,r6/1:/Table/{SystemCon…-11}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200205 22:44:57.022769 24 storage/store_snapshot.go:922  [n1,replicate,s1,r6/1:/Table/{SystemCon…-11}] sending LEARNER snapshot ffcabd54 at applied index 33
I200205 22:44:57.022883 2713 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r6/1:/Table/{SystemCon…-11}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):2LEARNER
I200205 22:44:57.022993 24 storage/store_snapshot.go:965  [n1,replicate,s1,r6/1:/Table/{SystemCon…-11}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 50, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:57.023779 2615 storage/replica_raftstorage.go:780  [n2,s2,r6/2:{-}] applying LEARNER snapshot [id=ffcabd54 index=33]
I200205 22:44:57.031478 2615 storage/replica_raftstorage.go:801  [n2,s2,r6/2:/Table/{SystemCon…-11}] applied LEARNER snapshot [total=8ms ingestion=4@7ms id=ffcabd54 index=33]
I200205 22:44:57.032369 24 storage/replica_command.go:1601  [n1,replicate,s1,r6/1:/Table/{SystemCon…-11}] change replicas (add [(n2,s2):2] remove []): existing descriptor r6:/Table/{SystemConfigSpan/Start-11} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
I200205 22:44:57.036242 24 storage/replica_raft.go:248  [n1,s1,r6/1:/Table/{SystemCon…-11}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200205 22:44:57.036783 24 storage/replica_command.go:1601  [n1,replicate,s1,r6/1:/Table/{SystemCon…-11}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r6:/Table/{SystemConfigSpan/Start-11} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200205 22:44:57.046245 24 storage/replica_raft.go:248  [n1,s1,r6/1:/Table/{SystemCon…-11}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3LEARNER] next=4
I200205 22:44:57.048146 24 storage/store_snapshot.go:922  [n1,replicate,s1,r6/1:/Table/{SystemCon…-11}] sending LEARNER snapshot 90e0373d at applied index 37
I200205 22:44:57.048256 2686 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r6/1:/Table/{SystemCon…-11}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200205 22:44:57.048443 24 storage/store_snapshot.go:965  [n1,replicate,s1,r6/1:/Table/{SystemCon…-11}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 54, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:57.049276 2684 storage/replica_raftstorage.go:780  [n3,s3,r6/3:{-}] applying LEARNER snapshot [id=90e0373d index=37]
I200205 22:44:57.056492 2684 storage/replica_raftstorage.go:801  [n3,s3,r6/3:/Table/{SystemCon…-11}] applied LEARNER snapshot [total=7ms ingestion=4@7ms id=90e0373d index=37]
I200205 22:44:57.057159 24 storage/replica_command.go:1601  [n1,replicate,s1,r6/1:/Table/{SystemCon…-11}] change replicas (add [(n3,s3):3] remove []): existing descriptor r6:/Table/{SystemConfigSpan/Start-11} [(n1,s1):1, (n2,s2):2, (n3,s3):3LEARNER, next=4, gen=3]
I200205 22:44:57.061359 24 storage/replica_raft.go:248  [n1,s1,r6/1:/Table/{SystemCon…-11}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3] next=4
I200205 22:44:57.064211 24 storage/replica_command.go:1601  [n1,replicate,s1,r14/1:/Table/1{8-9}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r14:/Table/1{8-9} [(n1,s1):1, next=2, gen=0]
I200205 22:44:57.126997 24 storage/replica_raft.go:248  [n1,s1,r14/1:/Table/1{8-9}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200205 22:44:57.128286 2804 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r14/1:/Table/1{8-9}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):2LEARNER
I200205 22:44:57.128476 24 storage/store_snapshot.go:922  [n1,replicate,s1,r14/1:/Table/1{8-9}] sending LEARNER snapshot 94a50402 at applied index 15
I200205 22:44:57.128648 24 storage/store_snapshot.go:965  [n1,replicate,s1,r14/1:/Table/1{8-9}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:57.129302 2751 storage/replica_raftstorage.go:780  [n2,s2,r14/2:{-}] applying LEARNER snapshot [id=94a50402 index=15]
I200205 22:44:57.132287 2751 storage/replica_raftstorage.go:801  [n2,s2,r14/2:/Table/1{8-9}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=94a50402 index=15]
I200205 22:44:57.132877 24 storage/replica_command.go:1601  [n1,replicate,s1,r14/1:/Table/1{8-9}] change replicas (add [(n2,s2):2] remove []): existing descriptor r14:/Table/1{8-9} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
I200205 22:44:57.141859 24 storage/replica_raft.go:248  [n1,s1,r14/1:/Table/1{8-9}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200205 22:44:57.142368 24 storage/replica_command.go:1601  [n1,replicate,s1,r14/1:/Table/1{8-9}] change replicas (add [(n4,s4):3LEARNER] remove []): existing descriptor r14:/Table/1{8-9} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200205 22:44:57.155259 24 storage/replica_raft.go:248  [n1,s1,r14/1:/Table/1{8-9}] proposing SIMPLE(l3) ADD_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3LEARNER] next=4
I200205 22:44:57.157499 2653 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r14/1:/Table/1{8-9}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):3LEARNER
I200205 22:44:57.157617 24 storage/store_snapshot.go:922  [n1,replicate,s1,r14/1:/Table/1{8-9}] sending LEARNER snapshot d06660c8 at applied index 19
I200205 22:44:57.157744 24 storage/store_snapshot.go:965  [n1,replicate,s1,r14/1:/Table/1{8-9}] streamed snapshot to (n4,s4):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:57.158467 2831 storage/replica_raftstorage.go:780  [n4,s4,r14/3:{-}] applying LEARNER snapshot [id=d06660c8 index=19]
I200205 22:44:57.168382 2831 storage/replica_raftstorage.go:801  [n4,s4,r14/3:/Table/1{8-9}] applied LEARNER snapshot [total=10ms ingestion=4@9ms id=d06660c8 index=19]
I200205 22:44:57.169000 24 storage/replica_command.go:1601  [n1,replicate,s1,r14/1:/Table/1{8-9}] change replicas (add [(n4,s4):3] remove []): existing descriptor r14:/Table/1{8-9} [(n1,s1):1, (n2,s2):2, (n4,s4):3LEARNER, next=4, gen=3]
I200205 22:44:57.177519 24 storage/replica_raft.go:248  [n1,s1,r14/1:/Table/1{8-9}] proposing SIMPLE(v3) ADD_REPLICA[(n4,s4):3]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3] next=4
I200205 22:44:57.180656 24 storage/replica_command.go:1601  [n1,replicate,s1,r25/1:/{Table/29-NamespaceTab…}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r25:/{Table/29-NamespaceTable/30} [(n1,s1):1, next=2, gen=0]
I200205 22:44:57.192789 24 storage/replica_raft.go:248  [n1,s1,r25/1:/{Table/29-NamespaceTab…}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200205 22:44:57.195538 24 storage/store_snapshot.go:922  [n1,replicate,s1,r25/1:/{Table/29-NamespaceTab…}] sending LEARNER snapshot d4c96c39 at applied index 15
I200205 22:44:57.195661 24 storage/store_snapshot.go:965  [n1,replicate,s1,r25/1:/{Table/29-NamespaceTab…}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:57.195838 2886 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r25/1:/{Table/29-NamespaceTab…}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):2LEARNER
I200205 22:44:57.196313 2900 storage/replica_raftstorage.go:780  [n2,s2,r25/2:{-}] applying LEARNER snapshot [id=d4c96c39 index=15]
I200205 22:44:57.205368 2900 storage/replica_raftstorage.go:801  [n2,s2,r25/2:/{Table/29-NamespaceTab…}] applied LEARNER snapshot [total=9ms ingestion=4@6ms id=d4c96c39 index=15]
I200205 22:44:57.208365 24 storage/replica_command.go:1601  [n1,replicate,s1,r25/1:/{Table/29-NamespaceTab…}] change replicas (add [(n2,s2):2] remove []): existing descriptor r25:/{Table/29-NamespaceTable/30} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
I200205 22:44:57.221851 24 storage/replica_raft.go:248  [n1,s1,r25/1:/{Table/29-NamespaceTab…}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200205 22:44:57.222381 24 storage/replica_command.go:1601  [n1,replicate,s1,r25/1:/{Table/29-NamespaceTab…}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r25:/{Table/29-NamespaceTable/30} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200205 22:44:57.247671 24 storage/replica_raft.go:248  [n1,s1,r25/1:/{Table/29-NamespaceTab…}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3LEARNER] next=4
I200205 22:44:57.250035 24 storage/store_snapshot.go:922  [n1,replicate,s1,r25/1:/{Table/29-NamespaceTab…}] sending LEARNER snapshot 674feaa5 at applied index 19
I200205 22:44:57.250224 24 storage/store_snapshot.go:965  [n1,replicate,s1,r25/1:/{Table/29-NamespaceTab…}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:57.250850 2934 storage/replica_raftstorage.go:780  [n3,s3,r25/3:{-}] applying LEARNER snapshot [id=674feaa5 index=19]
I200205 22:44:57.251931 2936 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r25/1:/{Table/29-NamespaceTab…}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200205 22:44:57.255871 2934 storage/replica_raftstorage.go:801  [n3,s3,r25/3:/{Table/29-NamespaceTab…}] applied LEARNER snapshot [total=5ms ingestion=4@5ms id=674feaa5 index=19]
I200205 22:44:57.256369 24 storage/replica_command.go:1601  [n1,replicate,s1,r25/1:/{Table/29-NamespaceTab…}] change replicas (add [(n3,s3):3] remove []): existing descriptor r25:/{Table/29-NamespaceTable/30} [(n1,s1):1, (n2,s2):2, (n3,s3):3LEARNER, next=4, gen=3]
I200205 22:44:57.263888 24 storage/replica_raft.go:248  [n1,s1,r25/1:/{Table/29-NamespaceTab…}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3] next=4
I200205 22:44:57.269259 24 storage/replica_command.go:1601  [n1,replicate,s1,r16/1:/Table/2{0-1}] change replicas (add [(n3,s3):2LEARNER] remove []): existing descriptor r16:/Table/2{0-1} [(n1,s1):1, next=2, gen=0]
I200205 22:44:57.286636 24 storage/replica_raft.go:248  [n1,s1,r16/1:/Table/2{0-1}] proposing SIMPLE(l2) ADD_REPLICA[(n3,s3):2LEARNER]: after=[(n1,s1):1 (n3,s3):2LEARNER] next=3
I200205 22:44:57.288254 24 storage/store_snapshot.go:922  [n1,replicate,s1,r16/1:/Table/2{0-1}] sending LEARNER snapshot 90d35740 at applied index 15
I200205 22:44:57.288368 24 storage/store_snapshot.go:965  [n1,replicate,s1,r16/1:/Table/2{0-1}] streamed snapshot to (n3,s3):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:57.288434 2974 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r16/1:/Table/2{0-1}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):2LEARNER
I200205 22:44:57.288990 2972 storage/replica_raftstorage.go:780  [n3,s3,r16/2:{-}] applying LEARNER snapshot [id=90d35740 index=15]
I200205 22:44:57.297122 2972 storage/replica_raftstorage.go:801  [n3,s3,r16/2:/Table/2{0-1}] applied LEARNER snapshot [total=8ms ingestion=4@8ms id=90d35740 index=15]
I200205 22:44:57.298719 24 storage/replica_command.go:1601  [n1,replicate,s1,r16/1:/Table/2{0-1}] change replicas (add [(n3,s3):2] remove []): existing descriptor r16:/Table/2{0-1} [(n1,s1):1, (n3,s3):2LEARNER, next=3, gen=1]
I200205 22:44:57.309375 24 storage/replica_raft.go:248  [n1,s1,r16/1:/Table/2{0-1}] proposing SIMPLE(v2) ADD_REPLICA[(n3,s3):2]: after=[(n1,s1):1 (n3,s3):2] next=3
I200205 22:44:57.309926 24 storage/replica_command.go:1601  [n1,replicate,s1,r16/1:/Table/2{0-1}] change replicas (add [(n4,s4):3LEARNER] remove []): existing descriptor r16:/Table/2{0-1} [(n1,s1):1, (n3,s3):2, next=3, gen=2]
I200205 22:44:57.326430 24 storage/replica_raft.go:248  [n1,s1,r16/1:/Table/2{0-1}] proposing SIMPLE(l3) ADD_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3LEARNER] next=4
I200205 22:44:57.330026 24 storage/store_snapshot.go:922  [n1,replicate,s1,r16/1:/Table/2{0-1}] sending LEARNER snapshot 92c9af34 at applied index 19
I200205 22:44:57.330161 24 storage/store_snapshot.go:965  [n1,replicate,s1,r16/1:/Table/2{0-1}] streamed snapshot to (n4,s4):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:57.330224 3000 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r16/1:/Table/2{0-1}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):3LEARNER
I200205 22:44:57.330838 3028 storage/replica_raftstorage.go:780  [n4,s4,r16/3:{-}] applying LEARNER snapshot [id=92c9af34 index=19]
I200205 22:44:57.335174 3028 storage/replica_raftstorage.go:801  [n4,s4,r16/3:/Table/2{0-1}] applied LEARNER snapshot [total=4ms ingestion=4@4ms id=92c9af34 index=19]
I200205 22:44:57.341719 24 storage/replica_command.go:1601  [n1,replicate,s1,r16/1:/Table/2{0-1}] change replicas (add [(n4,s4):3] remove []): existing descriptor r16:/Table/2{0-1} [(n1,s1):1, (n3,s3):2, (n4,s4):3LEARNER, next=4, gen=3]
I200205 22:44:57.352611 24 storage/replica_raft.go:248  [n1,s1,r16/1:/Table/2{0-1}] proposing SIMPLE(v3) ADD_REPLICA[(n4,s4):3]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3] next=4
I200205 22:44:57.357659 24 storage/replica_command.go:1601  [n1,replicate,s1,r12/1:/Table/1{6-7}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r12:/Table/1{6-7} [(n1,s1):1, next=2, gen=0]
I200205 22:44:57.366818 24 storage/replica_raft.go:248  [n1,s1,r12/1:/Table/1{6-7}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200205 22:44:57.368448 3023 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r12/1:/Table/1{6-7}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200205 22:44:57.368548 24 storage/store_snapshot.go:922  [n1,replicate,s1,r12/1:/Table/1{6-7}] sending LEARNER snapshot c81d2724 at applied index 15
I200205 22:44:57.368654 24 storage/store_snapshot.go:965  [n1,replicate,s1,r12/1:/Table/1{6-7}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:57.369267 3021 storage/replica_raftstorage.go:780  [n4,s4,r12/2:{-}] applying LEARNER snapshot [id=c81d2724 index=15]
I200205 22:44:57.379367 3021 storage/replica_raftstorage.go:801  [n4,s4,r12/2:/Table/1{6-7}] applied LEARNER snapshot [total=10ms ingestion=4@10ms id=c81d2724 index=15]
I200205 22:44:57.380175 24 storage/replica_command.go:1601  [n1,replicate,s1,r12/1:/Table/1{6-7}] change replicas (add [(n4,s4):2] remove []): existing descriptor r12:/Table/1{6-7} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200205 22:44:57.388287 24 storage/replica_raft.go:248  [n1,s1,r12/1:/Table/1{6-7}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200205 22:44:57.388799 24 storage/replica_command.go:1601  [n1,replicate,s1,r12/1:/Table/1{6-7}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r12:/Table/1{6-7} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200205 22:44:57.404533 24 storage/replica_raft.go:248  [n1,s1,r12/1:/Table/1{6-7}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3LEARNER] next=4
I200205 22:44:57.408990 24 storage/store_snapshot.go:922  [n1,replicate,s1,r12/1:/Table/1{6-7}] sending LEARNER snapshot 52fdf47a at applied index 19
I200205 22:44:57.409123 24 storage/store_snapshot.go:965  [n1,replicate,s1,r12/1:/Table/1{6-7}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:57.409233 3120 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r12/1:/Table/1{6-7}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):3LEARNER
I200205 22:44:57.409673 3103 storage/replica_raftstorage.go:780  [n2,s2,r12/3:{-}] applying LEARNER snapshot [id=52fdf47a index=19]
I200205 22:44:57.418057 3103 storage/replica_raftstorage.go:801  [n2,s2,r12/3:/Table/1{6-7}] applied LEARNER snapshot [total=8ms ingestion=4@8ms id=52fdf47a index=19]
I200205 22:44:57.419183 24 storage/replica_command.go:1601  [n1,replicate,s1,r12/1:/Table/1{6-7}] change replicas (add [(n2,s2):3] remove []): existing descriptor r12:/Table/1{6-7} [(n1,s1):1, (n4,s4):2, (n2,s2):3LEARNER, next=4, gen=3]
I200205 22:44:57.443930 24 storage/replica_raft.go:248  [n1,s1,r12/1:/Table/1{6-7}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3] next=4
I200205 22:44:57.452910 24 storage/replica_command.go:1601  [n1,replicate,s1,r1/1:/{Min-System/NodeL…}] change replicas (add [(n3,s3):2LEARNER] remove []): existing descriptor r1:/{Min-System/NodeLiveness} [(n1,s1):1, next=2, gen=0]
I200205 22:44:57.469522 24 storage/replica_raft.go:248  [n1,s1,r1/1:/{Min-System/NodeL…}] proposing SIMPLE(l2) ADD_REPLICA[(n3,s3):2LEARNER]: after=[(n1,s1):1 (n3,s3):2LEARNER] next=3
I200205 22:44:57.471043 24 storage/store_snapshot.go:922  [n1,replicate,s1,r1/1:/{Min-System/NodeL…}] sending LEARNER snapshot 959b8ba3 at applied index 140
I200205 22:44:57.471095 2991 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r1/1:/{Min-System/NodeL…}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):2LEARNER
I200205 22:44:57.471309 24 storage/store_snapshot.go:965  [n1,replicate,s1,r1/1:/{Min-System/NodeL…}] streamed snapshot to (n3,s3):2LEARNER: kv pairs: 97, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:57.478742 3174 storage/replica_raftstorage.go:780  [n3,s3,r1/2:{-}] applying LEARNER snapshot [id=959b8ba3 index=140]
I200205 22:44:57.484022 3174 storage/replica_raftstorage.go:801  [n3,s3,r1/2:/{Min-System/NodeL…}] applied LEARNER snapshot [total=5ms ingestion=4@5ms id=959b8ba3 index=140]
I200205 22:44:57.484635 24 storage/replica_command.go:1601  [n1,replicate,s1,r1/1:/{Min-System/NodeL…}] change replicas (add [(n3,s3):2] remove []): existing descriptor r1:/{Min-System/NodeLiveness} [(n1,s1):1, (n3,s3):2LEARNER, next=3, gen=1]
I200205 22:44:57.497771 24 storage/replica_raft.go:248  [n1,s1,r1/1:/{Min-System/NodeL…}] proposing SIMPLE(v2) ADD_REPLICA[(n3,s3):2]: after=[(n1,s1):1 (n3,s3):2] next=3
I200205 22:44:57.504372 24 storage/replica_command.go:1601  [n1,replicate,s1,r1/1:/{Min-System/NodeL…}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r1:/{Min-System/NodeLiveness} [(n1,s1):1, (n3,s3):2, next=3, gen=2]
I200205 22:44:57.522585 24 storage/replica_raft.go:248  [n1,s1,r1/1:/{Min-System/NodeL…}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n2,s2):3LEARNER] next=4
I200205 22:44:57.524680 24 storage/store_snapshot.go:922  [n1,replicate,s1,r1/1:/{Min-System/NodeL…}] sending LEARNER snapshot 242e57d8 at applied index 146
I200205 22:44:57.524939 3252 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r1/1:/{Min-System/NodeL…}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):3LEARNER
I200205 22:44:57.525000 24 storage/store_snapshot.go:965  [n1,replicate,s1,r1/1:/{Min-System/NodeL…}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 105, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:57.525854 3237 storage/replica_raftstorage.go:780  [n2,s2,r1/3:{-}] applying LEARNER snapshot [id=242e57d8 index=146]
I200205 22:44:57.529951 3237 storage/replica_raftstorage.go:801  [n2,s2,r1/3:/{Min-System/NodeL…}] applied LEARNER snapshot [total=4ms ingestion=4@4ms id=242e57d8 index=146]
I200205 22:44:57.530435 24 storage/replica_command.go:1601  [n1,replicate,s1,r1/1:/{Min-System/NodeL…}] change replicas (add [(n2,s2):3] remove []): existing descriptor r1:/{Min-System/NodeLiveness} [(n1,s1):1, (n3,s3):2, (n2,s2):3LEARNER, next=4, gen=3]
I200205 22:44:57.541502 24 storage/replica_raft.go:248  [n1,s1,r1/1:/{Min-System/NodeL…}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n3,s3):2 (n2,s2):3] next=4
I200205 22:44:57.544861 24 storage/replica_command.go:1601  [n1,replicate,s1,r10/1:/Table/1{4-5}] change replicas (add [(n3,s3):2LEARNER] remove []): existing descriptor r10:/Table/1{4-5} [(n1,s1):1, next=2, gen=0]
I200205 22:44:57.556755 24 storage/replica_raft.go:248  [n1,s1,r10/1:/Table/1{4-5}] proposing SIMPLE(l2) ADD_REPLICA[(n3,s3):2LEARNER]: after=[(n1,s1):1 (n3,s3):2LEARNER] next=3
I200205 22:44:57.570165 3272 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r10/1:/Table/1{4-5}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):2LEARNER
I200205 22:44:57.581077 24 storage/store_snapshot.go:922  [n1,replicate,s1,r10/1:/Table/1{4-5}] sending LEARNER snapshot 52999a41 at applied index 15
I200205 22:44:57.581217 24 storage/store_snapshot.go:965  [n1,replicate,s1,r10/1:/Table/1{4-5}] streamed snapshot to (n3,s3):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.02s
I200205 22:44:57.588916 3288 storage/replica_raftstorage.go:780  [n3,s3,r10/2:{-}] applying LEARNER snapshot [id=52999a41 index=15]
I200205 22:44:57.593742 3288 storage/replica_raftstorage.go:801  [n3,s3,r10/2:/Table/1{4-5}] applied LEARNER snapshot [total=5ms ingestion=4@4ms id=52999a41 index=15]
I200205 22:44:57.596386 24 storage/replica_command.go:1601  [n1,replicate,s1,r10/1:/Table/1{4-5}] change replicas (add [(n3,s3):2] remove []): existing descriptor r10:/Table/1{4-5} [(n1,s1):1, (n3,s3):2LEARNER, next=3, gen=1]
I200205 22:44:57.642529 24 storage/replica_raft.go:248  [n1,s1,r10/1:/Table/1{4-5}] proposing SIMPLE(v2) ADD_REPLICA[(n3,s3):2]: after=[(n1,s1):1 (n3,s3):2] next=3
I200205 22:44:57.643316 24 storage/replica_command.go:1601  [n1,replicate,s1,r10/1:/Table/1{4-5}] change replicas (add [(n4,s4):3LEARNER] remove []): existing descriptor r10:/Table/1{4-5} [(n1,s1):1, (n3,s3):2, next=3, gen=2]
I200205 22:44:57.678212 24 storage/replica_raft.go:248  [n1,s1,r10/1:/Table/1{4-5}] proposing SIMPLE(l3) ADD_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3LEARNER] next=4
I200205 22:44:57.681111 3350 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r10/1:/Table/1{4-5}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):3LEARNER
I200205 22:44:57.681241 24 storage/store_snapshot.go:922  [n1,replicate,s1,r10/1:/Table/1{4-5}] sending LEARNER snapshot b64afdc2 at applied index 20
I200205 22:44:57.681370 24 storage/store_snapshot.go:965  [n1,replicate,s1,r10/1:/Table/1{4-5}] streamed snapshot to (n4,s4):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:57.681924 3348 storage/replica_raftstorage.go:780  [n4,s4,r10/3:{-}] applying LEARNER snapshot [id=b64afdc2 index=20]
I200205 22:44:57.691240 3348 storage/replica_raftstorage.go:801  [n4,s4,r10/3:/Table/1{4-5}] applied LEARNER snapshot [total=9ms ingestion=4@9ms id=b64afdc2 index=20]
I200205 22:44:57.691857 24 storage/replica_command.go:1601  [n1,replicate,s1,r10/1:/Table/1{4-5}] change replicas (add [(n4,s4):3] remove []): existing descriptor r10:/Table/1{4-5} [(n1,s1):1, (n3,s3):2, (n4,s4):3LEARNER, next=4, gen=3]
I200205 22:44:57.711940 24 storage/replica_raft.go:248  [n1,s1,r10/1:/Table/1{4-5}] proposing SIMPLE(v3) ADD_REPLICA[(n4,s4):3]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3] next=4
I200205 22:44:57.724040 24 storage/replica_command.go:1601  [n1,replicate,s1,r20/1:/Table/2{4-5}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r20:/Table/2{4-5} [(n1,s1):1, next=2, gen=0]
I200205 22:44:57.732345 24 storage/replica_raft.go:248  [n1,s1,r20/1:/Table/2{4-5}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200205 22:44:57.745293 3219 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r20/1:/Table/2{4-5}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200205 22:44:57.745402 24 storage/store_snapshot.go:922  [n1,replicate,s1,r20/1:/Table/2{4-5}] sending LEARNER snapshot 56d95360 at applied index 15
I200205 22:44:57.745510 24 storage/store_snapshot.go:965  [n1,replicate,s1,r20/1:/Table/2{4-5}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.01s
I200205 22:44:57.746030 3217 storage/replica_raftstorage.go:780  [n4,s4,r20/2:{-}] applying LEARNER snapshot [id=56d95360 index=15]
I200205 22:44:57.749299 3217 storage/replica_raftstorage.go:801  [n4,s4,r20/2:/Table/2{4-5}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=56d95360 index=15]
I200205 22:44:57.750525 24 storage/replica_command.go:1601  [n1,replicate,s1,r20/1:/Table/2{4-5}] change replicas (add [(n4,s4):2] remove []): existing descriptor r20:/Table/2{4-5} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200205 22:44:57.760171 24 storage/replica_raft.go:248  [n1,s1,r20/1:/Table/2{4-5}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200205 22:44:57.760735 24 storage/replica_command.go:1601  [n1,replicate,s1,r20/1:/Table/2{4-5}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r20:/Table/2{4-5} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200205 22:44:57.783854 1014 gossip/gossip.go:1532  [n2] node has connected to cluster via gossip
I200205 22:44:57.783992 1014 storage/stores.go:259  [n2] wrote 3 node addresses to persistent storage
I200205 22:44:57.787303 24 storage/replica_raft.go:248  [n1,s1,r20/1:/Table/2{4-5}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3LEARNER] next=4
I200205 22:44:57.790911 24 storage/store_snapshot.go:922  [n1,replicate,s1,r20/1:/Table/2{4-5}] sending LEARNER snapshot 7ccf456d at applied index 19
I200205 22:44:57.791065 3339 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r20/1:/Table/2{4-5}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):3LEARNER
I200205 22:44:57.791079 24 storage/store_snapshot.go:965  [n1,replicate,s1,r20/1:/Table/2{4-5}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:57.791790 3422 storage/replica_raftstorage.go:780  [n2,s2,r20/3:{-}] applying LEARNER snapshot [id=7ccf456d index=19]
I200205 22:44:57.798368 3422 storage/replica_raftstorage.go:801  [n2,s2,r20/3:/Table/2{4-5}] applied LEARNER snapshot [total=7ms ingestion=4@6ms id=7ccf456d index=19]
I200205 22:44:57.798796 24 storage/replica_command.go:1601  [n1,replicate,s1,r20/1:/Table/2{4-5}] change replicas (add [(n2,s2):3] remove []): existing descriptor r20:/Table/2{4-5} [(n1,s1):1, (n4,s4):2, (n2,s2):3LEARNER, next=4, gen=3]
I200205 22:44:57.814812 24 storage/replica_raft.go:248  [n1,s1,r20/1:/Table/2{4-5}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3] next=4
I200205 22:44:57.819449 24 storage/replica_command.go:1601  [n1,replicate,s1,r15/1:/Table/{19-20}] change replicas (add [(n3,s3):2LEARNER] remove []): existing descriptor r15:/Table/{19-20} [(n1,s1):1, next=2, gen=0]
I200205 22:44:57.831891 24 storage/replica_raft.go:248  [n1,s1,r15/1:/Table/{19-20}] proposing SIMPLE(l2) ADD_REPLICA[(n3,s3):2LEARNER]: after=[(n1,s1):1 (n3,s3):2LEARNER] next=3
I200205 22:44:57.835193 24 storage/store_snapshot.go:922  [n1,replicate,s1,r15/1:/Table/{19-20}] sending LEARNER snapshot 8f2d75ab at applied index 15
I200205 22:44:57.835292 24 storage/store_snapshot.go:965  [n1,replicate,s1,r15/1:/Table/{19-20}] streamed snapshot to (n3,s3):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:57.835307 3497 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r15/1:/Table/{19-20}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):2LEARNER
I200205 22:44:57.835819 3450 storage/replica_raftstorage.go:780  [n3,s3,r15/2:{-}] applying LEARNER snapshot [id=8f2d75ab index=15]
I200205 22:44:57.847924 3542 storage/replica_command.go:1601  [n2,replicate,s2,r17/3:/Table/2{1-2}] change replicas (add [(n4,s4):4LEARNER] remove []): existing descriptor r17:/Table/2{1-2} [(n1,s1):1, (n3,s3):2, (n2,s2):3, next=4, gen=4]
I200205 22:44:57.858624 3450 storage/replica_raftstorage.go:801  [n3,s3,r15/2:/Table/{19-20}] applied LEARNER snapshot [total=23ms ingestion=4@14ms id=8f2d75ab index=15]
I200205 22:44:57.859435 24 storage/replica_command.go:1601  [n1,replicate,s1,r15/1:/Table/{19-20}] change replicas (add [(n3,s3):2] remove []): existing descriptor r15:/Table/{19-20} [(n1,s1):1, (n3,s3):2LEARNER, next=3, gen=1]
I200205 22:44:57.878702 24 storage/replica_raft.go:248  [n1,s1,r15/1:/Table/{19-20}] proposing SIMPLE(v2) ADD_REPLICA[(n3,s3):2]: after=[(n1,s1):1 (n3,s3):2] next=3
I200205 22:44:57.879431 24 storage/replica_command.go:1601  [n1,replicate,s1,r15/1:/Table/{19-20}] change replicas (add [(n4,s4):3LEARNER] remove []): existing descriptor r15:/Table/{19-20} [(n1,s1):1, (n3,s3):2, next=3, gen=2]
I200205 22:44:57.896959 24 storage/replica_raft.go:248  [n1,s1,r15/1:/Table/{19-20}] proposing SIMPLE(l3) ADD_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3LEARNER] next=4
I200205 22:44:57.907691 24 storage/store_snapshot.go:922  [n1,replicate,s1,r15/1:/Table/{19-20}] sending LEARNER snapshot 37c49a87 at applied index 20
I200205 22:44:57.907818 24 storage/store_snapshot.go:965  [n1,replicate,s1,r15/1:/Table/{19-20}] streamed snapshot to (n4,s4):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.01s
I200205 22:44:57.907971 3580 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r15/1:/Table/{19-20}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):3LEARNER
I200205 22:44:57.908463 3607 storage/replica_raftstorage.go:780  [n4,s4,r15/3:{-}] applying LEARNER snapshot [id=37c49a87 index=20]
I200205 22:44:57.926026 3542 storage/replica_raft.go:248  [n2,s2,r17/3:/Table/2{1-2}] proposing SIMPLE(l4) ADD_REPLICA[(n4,s4):4LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n2,s2):3 (n4,s4):4LEARNER] next=5
I200205 22:44:57.941137 3607 storage/replica_raftstorage.go:801  [n4,s4,r15/3:/Table/{19-20}] applied LEARNER snapshot [total=33ms ingestion=4@31ms id=37c49a87 index=20]
I200205 22:44:57.946932 24 storage/replica_command.go:1601  [n1,replicate,s1,r15/1:/Table/{19-20}] change replicas (add [(n4,s4):3] remove []): existing descriptor r15:/Table/{19-20} [(n1,s1):1, (n3,s3):2, (n4,s4):3LEARNER, next=4, gen=3]
I200205 22:44:57.947004 3542 storage/store_snapshot.go:922  [n2,replicate,s2,r17/3:/Table/2{1-2}] sending LEARNER snapshot ad04b7fb at applied index 36
I200205 22:44:57.947176 3542 storage/store_snapshot.go:965  [n2,replicate,s2,r17/3:/Table/2{1-2}] streamed snapshot to (n4,s4):4LEARNER: kv pairs: 16, log entries: 0, rate-limit: 8.0 MiB/sec, 0.02s
I200205 22:44:57.947546 3626 storage/raft_snapshot_queue.go:169  [n2,raftsnapshot,s2,r17/3:/Table/2{1-2}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):4LEARNER
I200205 22:44:57.947866 3584 storage/replica_raftstorage.go:780  [n4,s4,r17/4:{-}] applying LEARNER snapshot [id=ad04b7fb index=36]
I200205 22:44:57.956612 24 storage/replica_raft.go:248  [n1,s1,r15/1:/Table/{19-20}] proposing SIMPLE(v3) ADD_REPLICA[(n4,s4):3]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3] next=4
I200205 22:44:57.957941 3584 storage/replica_raftstorage.go:801  [n4,s4,r17/4:/Table/2{1-2}] applied LEARNER snapshot [total=10ms ingestion=4@10ms id=ad04b7fb index=36]
I200205 22:44:57.958580 3542 storage/replica_command.go:1601  [n2,replicate,s2,r17/3:/Table/2{1-2}] change replicas (add [(n4,s4):4VOTER_INCOMING] remove [(n1,s1):1VOTER_DEMOTING]): existing descriptor r17:/Table/2{1-2} [(n1,s1):1, (n3,s3):2, (n2,s2):3, (n4,s4):4LEARNER, next=5, gen=5]
I200205 22:44:57.959660 24 storage/replica_command.go:1601  [n1,replicate,s1,r2/1:/System/NodeLiveness{-Max}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r2:/System/NodeLiveness{-Max} [(n1,s1):1, next=2, gen=0]
I200205 22:44:57.971733 24 storage/replica_raft.go:248  [n1,s1,r2/1:/System/NodeLiveness{-Max}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200205 22:44:57.972789 3542 storage/replica_raft.go:248  [n2,s2,r17/3:/Table/2{1-2}] proposing ENTER_JOINT(r1 l1 v4) ADD_REPLICA[(n4,s4):4VOTER_INCOMING], REMOVE_REPLICA[(n1,s1):1VOTER_DEMOTING]: after=[(n1,s1):1VOTER_DEMOTING (n3,s3):2 (n2,s2):3 (n4,s4):4VOTER_INCOMING] next=5
I200205 22:44:57.973642 1285 gossip/gossip.go:1532  [n3] node has connected to cluster via gossip
I200205 22:44:57.973817 1285 storage/stores.go:259  [n3] wrote 3 node addresses to persistent storage
I200205 22:44:57.974956 3542 storage/replica_command.go:1601  [n2,replicate,s2,r17/3:/Table/2{1-2}] change replicas (add [] remove []): existing descriptor r17:/Table/2{1-2} [(n1,s1):1VOTER_DEMOTING, (n3,s3):2, (n2,s2):3, (n4,s4):4VOTER_INCOMING, next=5, gen=6]
I200205 22:44:57.975996 3721 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r2/1:/System/NodeLiveness{-Max}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):2LEARNER
I200205 22:44:57.977113 24 storage/store_snapshot.go:922  [n1,replicate,s1,r2/1:/System/NodeLiveness{-Max}] sending LEARNER snapshot b8d945c4 at applied index 23
I200205 22:44:57.977287 24 storage/store_snapshot.go:965  [n1,replicate,s1,r2/1:/System/NodeLiveness{-Max}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:57.978078 3618 storage/replica_raftstorage.go:780  [n2,s2,r2/2:{-}] applying LEARNER snapshot [id=b8d945c4 index=23]
I200205 22:44:57.984082 3542 storage/replica_raft.go:248  [n2,s2,r17/3:/Table/2{1-2}] proposing LEAVE_JOINT: after=[(n1,s1):1LEARNER (n3,s3):2 (n2,s2):3 (n4,s4):4] next=5
I200205 22:44:57.985422 3542 storage/replica_command.go:1601  [n2,replicate,s2,r17/3:/Table/2{1-2}] change replicas (add [] remove [(n1,s1):1LEARNER]): existing descriptor r17:/Table/2{1-2} [(n1,s1):1LEARNER, (n3,s3):2, (n2,s2):3, (n4,s4):4, next=5, gen=7]
I200205 22:44:57.990965 3618 storage/replica_raftstorage.go:801  [n2,s2,r2/2:/System/NodeLiveness{-Max}] applied LEARNER snapshot [total=13ms ingestion=4@13ms id=b8d945c4 index=23]
I200205 22:44:57.993403 24 storage/replica_command.go:1601  [n1,replicate,s1,r2/1:/System/NodeLiveness{-Max}] change replicas (add [(n2,s2):2] remove []): existing descriptor r2:/System/NodeLiveness{-Max} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
I200205 22:44:58.001620 3542 storage/replica_raft.go:248  [n2,s2,r17/3:/Table/2{1-2}] proposing SIMPLE(r1) REMOVE_REPLICA[(n1,s1):1LEARNER]: after=[(n4,s4):4 (n3,s3):2 (n2,s2):3] next=5
I200205 22:44:58.002720 150 storage/store_remove_replica.go:122  [n1,s1,r17/1:/Table/2{1-2}] removing replica r17/1
I200205 22:44:58.014094 24 storage/replica_raft.go:248  [n1,s1,r2/1:/System/NodeLiveness{-Max}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200205 22:44:58.014664 24 storage/replica_command.go:1601  [n1,replicate,s1,r2/1:/System/NodeLiveness{-Max}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r2:/System/NodeLiveness{-Max} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200205 22:44:58.032961 24 storage/replica_raft.go:248  [n1,s1,r2/1:/System/NodeLiveness{-Max}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3LEARNER] next=4
I200205 22:44:58.037570 24 storage/store_snapshot.go:922  [n1,replicate,s1,r2/1:/System/NodeLiveness{-Max}] sending LEARNER snapshot 5ef8d9fd at applied index 27
I200205 22:44:58.037726 24 storage/store_snapshot.go:965  [n1,replicate,s1,r2/1:/System/NodeLiveness{-Max}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 14, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:58.037726 3770 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r2/1:/System/NodeLiveness{-Max}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200205 22:44:58.038291 3768 storage/replica_raftstorage.go:780  [n3,s3,r2/3:{-}] applying LEARNER snapshot [id=5ef8d9fd index=27]
I200205 22:44:58.042738 3768 storage/replica_raftstorage.go:801  [n3,s3,r2/3:/System/NodeLiveness{-Max}] applied LEARNER snapshot [total=4ms ingestion=4@3ms id=5ef8d9fd index=27]
I200205 22:44:58.044910 24 storage/replica_command.go:1601  [n1,replicate,s1,r2/1:/System/NodeLiveness{-Max}] change replicas (add [(n3,s3):3] remove []): existing descriptor r2:/System/NodeLiveness{-Max} [(n1,s1):1, (n2,s2):2, (n3,s3):3LEARNER, next=4, gen=3]
I200205 22:44:58.058573 24 storage/replica_raft.go:248  [n1,s1,r2/1:/System/NodeLiveness{-Max}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3] next=4
I200205 22:44:58.066305 24 storage/replica_command.go:1601  [n1,replicate,s1,r18/1:/Table/2{2-3}] change replicas (add [(n3,s3):2LEARNER] remove []): existing descriptor r18:/Table/2{2-3} [(n1,s1):1, next=2, gen=0]
I200205 22:44:58.077779 24 storage/replica_raft.go:248  [n1,s1,r18/1:/Table/2{2-3}] proposing SIMPLE(l2) ADD_REPLICA[(n3,s3):2LEARNER]: after=[(n1,s1):1 (n3,s3):2LEARNER] next=3
I200205 22:44:58.079321 24 storage/store_snapshot.go:922  [n1,replicate,s1,r18/1:/Table/2{2-3}] sending LEARNER snapshot 1e558107 at applied index 15
I200205 22:44:58.079420 3823 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r18/1:/Table/2{2-3}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):2LEARNER
I200205 22:44:58.079429 24 storage/store_snapshot.go:965  [n1,replicate,s1,r18/1:/Table/2{2-3}] streamed snapshot to (n3,s3):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:58.081547 3821 storage/replica_raftstorage.go:780  [n3,s3,r18/2:{-}] applying LEARNER snapshot [id=1e558107 index=15]
I200205 22:44:58.084509 3821 storage/replica_raftstorage.go:801  [n3,s3,r18/2:/Table/2{2-3}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=1e558107 index=15]
I200205 22:44:58.085653 24 storage/replica_command.go:1601  [n1,replicate,s1,r18/1:/Table/2{2-3}] change replicas (add [(n3,s3):2] remove []): existing descriptor r18:/Table/2{2-3} [(n1,s1):1, (n3,s3):2LEARNER, next=3, gen=1]
I200205 22:44:58.095366 24 storage/replica_raft.go:248  [n1,s1,r18/1:/Table/2{2-3}] proposing SIMPLE(v2) ADD_REPLICA[(n3,s3):2]: after=[(n1,s1):1 (n3,s3):2] next=3
I200205 22:44:58.096059 24 storage/replica_command.go:1601  [n1,replicate,s1,r18/1:/Table/2{2-3}] change replicas (add [(n4,s4):3LEARNER] remove []): existing descriptor r18:/Table/2{2-3} [(n1,s1):1, (n3,s3):2, next=3, gen=2]
I200205 22:44:58.099209 3699 storage/replica_command.go:1601  [n3,replicate,s3,r27/2:/{NamespaceTab…-Table/32}] change replicas (add [(n2,s2):4LEARNER] remove []): existing descriptor r27:/{NamespaceTable/Max-Table/32} [(n1,s1):1, (n3,s3):2, (n4,s4):3, next=4, gen=4]
I200205 22:44:58.124224 24 storage/replica_raft.go:248  [n1,s1,r18/1:/Table/2{2-3}] proposing SIMPLE(l3) ADD_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3LEARNER] next=4
I200205 22:44:58.125908 24 storage/store_snapshot.go:922  [n1,replicate,s1,r18/1:/Table/2{2-3}] sending LEARNER snapshot 24849f97 at applied index 19
I200205 22:44:58.126013 3930 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r18/1:/Table/2{2-3}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):3LEARNER
I200205 22:44:58.126316 24 storage/store_snapshot.go:965  [n1,replicate,s1,r18/1:/Table/2{2-3}] streamed snapshot to (n4,s4):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:58.127592 1615 gossip/gossip.go:1532  [n4] node has connected to cluster via gossip
I200205 22:44:58.127685 3885 storage/replica_raftstorage.go:780  [n4,s4,r18/3:{-}] applying LEARNER snapshot [id=24849f97 index=19]
I200205 22:44:58.127883 1615 storage/stores.go:259  [n4] wrote 3 node addresses to persistent storage
I200205 22:44:58.131534 3885 storage/replica_raftstorage.go:801  [n4,s4,r18/3:/Table/2{2-3}] applied LEARNER snapshot [total=4ms ingestion=4@3ms id=24849f97 index=19]
I200205 22:44:58.132536 24 storage/replica_command.go:1601  [n1,replicate,s1,r18/1:/Table/2{2-3}] change replicas (add [(n4,s4):3] remove []): existing descriptor r18:/Table/2{2-3} [(n1,s1):1, (n3,s3):2, (n4,s4):3LEARNER, next=4, gen=3]
I200205 22:44:58.142668 24 storage/replica_raft.go:248  [n1,s1,r18/1:/Table/2{2-3}] proposing SIMPLE(v3) ADD_REPLICA[(n4,s4):3]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3] next=4
I200205 22:44:58.153298 24 storage/replica_command.go:1601  [n1,replicate,s1,r21/1:/Table/2{5-6}] change replicas (add [(n3,s3):2LEARNER] remove []): existing descriptor r21:/Table/2{5-6} [(n1,s1):1, next=2, gen=0]
I200205 22:44:58.167984 3699 storage/replica_raft.go:248  [n3,s3,r27/2:/{NamespaceTab…-Table/32}] proposing SIMPLE(l4) ADD_REPLICA[(n2,s2):4LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3 (n2,s2):4LEARNER] next=5
I200205 22:44:58.169757 3990 storage/raft_snapshot_queue.go:169  [n3,raftsnapshot,s3,r27/2:/{NamespaceTab…-Table/32}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):4LEARNER
I200205 22:44:58.170123 3699 storage/store_snapshot.go:922  [n3,replicate,s3,r27/2:/{NamespaceTab…-Table/32}] sending LEARNER snapshot aa157038 at applied index 31
I200205 22:44:58.170263 3699 storage/store_snapshot.go:965  [n3,replicate,s3,r27/2:/{NamespaceTab…-Table/32}] streamed snapshot to (n2,s2):4LEARNER: kv pairs: 11, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:58.171006 3992 storage/replica_raftstorage.go:780  [n2,s2,r27/4:{-}] applying LEARNER snapshot [id=aa157038 index=31]
I200205 22:44:58.171869 24 storage/replica_raft.go:248  [n1,s1,r21/1:/Table/2{5-6}] proposing SIMPLE(l2) ADD_REPLICA[(n3,s3):2LEARNER]: after=[(n1,s1):1 (n3,s3):2LEARNER] next=3
I200205 22:44:58.173574 24 storage/store_snapshot.go:922  [n1,replicate,s1,r21/1:/Table/2{5-6}] sending LEARNER snapshot 4eeda1ed at applied index 15
I200205 22:44:58.173680 24 storage/store_snapshot.go:965  [n1,replicate,s1,r21/1:/Table/2{5-6}] streamed snapshot to (n3,s3):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:58.174258 4006 storage/replica_raftstorage.go:780  [n3,s3,r21/2:{-}] applying LEARNER snapshot [id=4eeda1ed index=15]
I200205 22:44:58.180198 3968 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r21/1:/Table/2{5-6}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):2LEARNER
I200205 22:44:58.187494 4006 storage/replica_raftstorage.go:801  [n3,s3,r21/2:/Table/2{5-6}] applied LEARNER snapshot [total=13ms ingestion=4@13ms id=4eeda1ed index=15]
I200205 22:44:58.191927 3992 storage/replica_raftstorage.go:801  [n2,s2,r27/4:/{NamespaceTab…-Table/32}] applied LEARNER snapshot [total=21ms ingestion=4@21ms id=aa157038 index=31]
I200205 22:44:58.197455 24 storage/replica_command.go:1601  [n1,replicate,s1,r21/1:/Table/2{5-6}] change replicas (add [(n3,s3):2] remove []): existing descriptor r21:/Table/2{5-6} [(n1,s1):1, (n3,s3):2LEARNER, next=3, gen=1]
I200205 22:44:58.209153 24 storage/replica_raft.go:248  [n1,s1,r21/1:/Table/2{5-6}] proposing SIMPLE(v2) ADD_REPLICA[(n3,s3):2]: after=[(n1,s1):1 (n3,s3):2] next=3
I200205 22:44:58.209674 24 storage/replica_command.go:1601  [n1,replicate,s1,r21/1:/Table/2{5-6}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r21:/Table/2{5-6} [(n1,s1):1, (n3,s3):2, next=3, gen=2]
I200205 22:44:58.217711 3699 storage/replica_command.go:1601  [n3,replicate,s3,r27/2:/{NamespaceTab…-Table/32}] change replicas (add [(n2,s2):4VOTER_INCOMING] remove [(n1,s1):1VOTER_DEMOTING]): existing descriptor r27:/{NamespaceTable/Max-Table/32} [(n1,s1):1, (n3,s3):2, (n4,s4):3, (n2,s2):4LEARNER, next=5, gen=5]
I200205 22:44:58.225236 24 storage/replica_raft.go:248  [n1,s1,r21/1:/Table/2{5-6}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n2,s2):3LEARNER] next=4
I200205 22:44:58.227426 24 storage/store_snapshot.go:922  [n1,replicate,s1,r21/1:/Table/2{5-6}] sending LEARNER snapshot 789a0fc1 at applied index 19
I200205 22:44:58.227538 24 storage/store_snapshot.go:965  [n1,replicate,s1,r21/1:/Table/2{5-6}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:58.227571 4071 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r21/1:/Table/2{5-6}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):3LEARNER
I200205 22:44:58.228175 4069 storage/replica_raftstorage.go:780  [n2,s2,r21/3:{-}] applying LEARNER snapshot [id=789a0fc1 index=19]
I200205 22:44:58.231041 4069 storage/replica_raftstorage.go:801  [n2,s2,r21/3:/Table/2{5-6}] applied LEARNER snapshot [total=3ms ingestion=4@2ms id=789a0fc1 index=19]
I200205 22:44:58.241440 24 storage/replica_command.go:1601  [n1,replicate,s1,r21/1:/Table/2{5-6}] change replicas (add [(n2,s2):3] remove []): existing descriptor r21:/Table/2{5-6} [(n1,s1):1, (n3,s3):2, (n2,s2):3LEARNER, next=4, gen=3]
I200205 22:44:58.259428 24 storage/replica_raft.go:248  [n1,s1,r21/1:/Table/2{5-6}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n3,s3):2 (n2,s2):3] next=4
I200205 22:44:58.264352 24 storage/replica_command.go:1601  [n1,replicate,s1,r13/1:/Table/1{7-8}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r13:/Table/1{7-8} [(n1,s1):1, next=2, gen=0]
I200205 22:44:58.275695 3699 storage/replica_raft.go:248  [n3,s3,r27/2:/{NamespaceTab…-Table/32}] proposing ENTER_JOINT(r1 l1 v4) ADD_REPLICA[(n2,s2):4VOTER_INCOMING], REMOVE_REPLICA[(n1,s1):1VOTER_DEMOTING]: after=[(n1,s1):1VOTER_DEMOTING (n3,s3):2 (n4,s4):3 (n2,s2):4VOTER_INCOMING] next=5
I200205 22:44:58.278028 3699 storage/replica_command.go:1601  [n3,replicate,s3,r27/2:/{NamespaceTab…-Table/32}] change replicas (add [] remove []): existing descriptor r27:/{NamespaceTable/Max-Table/32} [(n1,s1):1VOTER_DEMOTING, (n3,s3):2, (n4,s4):3, (n2,s2):4VOTER_INCOMING, next=5, gen=6]
I200205 22:44:58.289195 24 storage/replica_raft.go:248  [n1,s1,r13/1:/Table/1{7-8}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200205 22:44:58.290300 3699 storage/replica_raft.go:248  [n3,s3,r27/2:/{NamespaceTab…-Table/32}] proposing LEAVE_JOINT: after=[(n1,s1):1LEARNER (n3,s3):2 (n4,s4):3 (n2,s2):4] next=5
I200205 22:44:58.292784 3699 storage/replica_command.go:1601  [n3,replicate,s3,r27/2:/{NamespaceTab…-Table/32}] change replicas (add [] remove [(n1,s1):1LEARNER]): existing descriptor r27:/{NamespaceTable/Max-Table/32} [(n1,s1):1LEARNER, (n3,s3):2, (n4,s4):3, (n2,s2):4, next=5, gen=7]
I200205 22:44:58.295682 4058 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r13/1:/Table/1{7-8}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200205 22:44:58.298667 24 storage/store_snapshot.go:922  [n1,replicate,s1,r13/1:/Table/1{7-8}] sending LEARNER snapshot 41845fdd at applied index 16
I200205 22:44:58.299783 24 storage/store_snapshot.go:965  [n1,replicate,s1,r13/1:/Table/1{7-8}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:58.301861 4151 storage/replica_raftstorage.go:780  [n4,s4,r13/2:{-}] applying LEARNER snapshot [id=41845fdd index=16]
I200205 22:44:58.327261 3699 storage/replica_raft.go:248  [n3,s3,r27/2:/{NamespaceTab…-Table/32}] proposing SIMPLE(r1) REMOVE_REPLICA[(n1,s1):1LEARNER]: after=[(n2,s2):4 (n3,s3):2 (n4,s4):3] next=5
I200205 22:44:58.328336 155 storage/store_remove_replica.go:122  [n1,s1,r27/1:/{NamespaceTab…-Table/32}] removing replica r27/1
I200205 22:44:58.333535 4051 storage/replica_command.go:1601  [n4,replicate,s4,r9/3:/Table/1{3-4}] change replicas (add [(n3,s3):4LEARNER] remove []): existing descriptor r9:/Table/1{3-4} [(n1,s1):1, (n2,s2):2, (n4,s4):3, next=4, gen=4]
I200205 22:44:58.335809 4151 storage/replica_raftstorage.go:801  [n4,s4,r13/2:/Table/1{7-8}] applied LEARNER snapshot [total=34ms ingestion=4@33ms id=41845fdd index=16]
I200205 22:44:58.336321 24 storage/replica_command.go:1601  [n1,replicate,s1,r13/1:/Table/1{7-8}] change replicas (add [(n4,s4):2] remove []): existing descriptor r13:/Table/1{7-8} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200205 22:44:58.348465 24 storage/replica_raft.go:248  [n1,s1,r13/1:/Table/1{7-8}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200205 22:44:58.349205 24 storage/replica_command.go:1601  [n1,replicate,s1,r13/1:/Table/1{7-8}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r13:/Table/1{7-8} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200205 22:44:58.360304 24 storage/replica_raft.go:248  [n1,s1,r13/1:/Table/1{7-8}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3LEARNER] next=4
I200205 22:44:58.362477 24 storage/store_snapshot.go:922  [n1,replicate,s1,r13/1:/Table/1{7-8}] sending LEARNER snapshot 5bdfecfc at applied index 20
I200205 22:44:58.362675 24 storage/store_snapshot.go:965  [n1,replicate,s1,r13/1:/Table/1{7-8}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:58.362765 4096 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r13/1:/Table/1{7-8}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200205 22:44:58.363171 4174 storage/replica_raftstorage.go:780  [n3,s3,r13/3:{-}] applying LEARNER snapshot [id=5bdfecfc index=20]
I200205 22:44:58.370869 4174 storage/replica_raftstorage.go:801  [n3,s3,r13/3:/Table/1{7-8}] applied LEARNER snapshot [total=8ms ingestion=4@7ms id=5bdfecfc index=20]
I200205 22:44:58.372863 24 storage/replica_command.go:1601  [n1,replicate,s1,r13/1:/Table/1{7-8}] change replicas (add [(n3,s3):3] remove []): existing descriptor r13:/Table/1{7-8} [(n1,s1):1, (n4,s4):2, (n3,s3):3LEARNER, next=4, gen=3]
I200205 22:44:58.386660 24 storage/replica_raft.go:248  [n1,s1,r13/1:/Table/1{7-8}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3] next=4
I200205 22:44:58.390465 24 storage/replica_command.go:1601  [n1,replicate,s1,r11/1:/Table/1{5-6}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r11:/Table/1{5-6} [(n1,s1):1, next=2, gen=0]
I200205 22:44:58.406689 24 storage/replica_raft.go:248  [n1,s1,r11/1:/Table/1{5-6}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200205 22:44:58.408395 24 storage/store_snapshot.go:922  [n1,replicate,s1,r11/1:/Table/1{5-6}] sending LEARNER snapshot 9676793a at applied index 15
I200205 22:44:58.408519 24 storage/store_snapshot.go:965  [n1,replicate,s1,r11/1:/Table/1{5-6}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:58.408535 4312 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r11/1:/Table/1{5-6}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200205 22:44:58.409156 4310 storage/replica_raftstorage.go:780  [n4,s4,r11/2:{-}] applying LEARNER snapshot [id=9676793a index=15]
I200205 22:44:58.412853 4310 storage/replica_raftstorage.go:801  [n4,s4,r11/2:/Table/1{5-6}] applied LEARNER snapshot [total=4ms ingestion=4@3ms id=9676793a index=15]
I200205 22:44:58.421360 24 storage/replica_command.go:1601  [n1,replicate,s1,r11/1:/Table/1{5-6}] change replicas (add [(n4,s4):2] remove []): existing descriptor r11:/Table/1{5-6} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200205 22:44:58.430651 4051 storage/replica_raft.go:248  [n4,s4,r9/3:/Table/1{3-4}] proposing SIMPLE(l4) ADD_REPLICA[(n3,s3):4LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3 (n3,s3):4LEARNER] next=5
I200205 22:44:58.433988 4321 storage/raft_snapshot_queue.go:169  [n4,raftsnapshot,s4,r9/3:/Table/1{3-4}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):4LEARNER
I200205 22:44:58.434146 4051 storage/store_snapshot.go:922  [n4,replicate,s4,r9/3:/Table/1{3-4}] sending LEARNER snapshot d5492aa0 at applied index 233
I200205 22:44:58.435579 4051 storage/store_snapshot.go:965  [n4,replicate,s4,r9/3:/Table/1{3-4}] streamed snapshot to (n3,s3):4LEARNER: kv pairs: 521, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:58.437195 4319 storage/replica_raftstorage.go:780  [n3,s3,r9/4:{-}] applying LEARNER snapshot [id=d5492aa0 index=233]
I200205 22:44:58.440257 4319 storage/replica_raftstorage.go:801  [n3,s3,r9/4:/Table/1{3-4}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=d5492aa0 index=233]
I200205 22:44:58.443808 4051 storage/replica_command.go:1601  [n4,replicate,s4,r9/3:/Table/1{3-4}] change replicas (add [(n3,s3):4VOTER_INCOMING] remove [(n1,s1):1VOTER_DEMOTING]): existing descriptor r9:/Table/1{3-4} [(n1,s1):1, (n2,s2):2, (n4,s4):3, (n3,s3):4LEARNER, next=5, gen=5]
I200205 22:44:58.445911 24 storage/replica_raft.go:248  [n1,s1,r11/1:/Table/1{5-6}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200205 22:44:58.449841 24 storage/replica_command.go:1601  [n1,replicate,s1,r11/1:/Table/1{5-6}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r11:/Table/1{5-6} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200205 22:44:58.493529 24 storage/replica_raft.go:248  [n1,s1,r11/1:/Table/1{5-6}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3LEARNER] next=4
I200205 22:44:58.501089 24 storage/store_snapshot.go:922  [n1,replicate,s1,r11/1:/Table/1{5-6}] sending LEARNER snapshot c93c2ab5 at applied index 20
I200205 22:44:58.501241 24 storage/store_snapshot.go:965  [n1,replicate,s1,r11/1:/Table/1{5-6}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:58.501855 4393 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r11/1:/Table/1{5-6}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):3LEARNER
I200205 22:44:58.502163 4391 storage/replica_raftstorage.go:780  [n2,s2,r11/3:{-}] applying LEARNER snapshot [id=c93c2ab5 index=20]
I200205 22:44:58.506620 4391 storage/replica_raftstorage.go:801  [n2,s2,r11/3:/Table/1{5-6}] applied LEARNER snapshot [total=4ms ingestion=4@4ms id=c93c2ab5 index=20]
I200205 22:44:58.507069 4051 storage/replica_raft.go:248  [n4,s4,r9/3:/Table/1{3-4}] proposing ENTER_JOINT(r1 l1 v4) ADD_REPLICA[(n3,s3):4VOTER_INCOMING], REMOVE_REPLICA[(n1,s1):1VOTER_DEMOTING]: after=[(n1,s1):1VOTER_DEMOTING (n2,s2):2 (n4,s4):3 (n3,s3):4VOTER_INCOMING] next=5
I200205 22:44:58.515001 24 storage/replica_command.go:1601  [n1,replicate,s1,r11/1:/Table/1{5-6}] change replicas (add [(n2,s2):3] remove []): existing descriptor r11:/Table/1{5-6} [(n1,s1):1, (n4,s4):2, (n2,s2):3LEARNER, next=4, gen=3]
I200205 22:44:58.520994 4051 storage/replica_command.go:1601  [n4,replicate,s4,r9/3:/Table/1{3-4}] change replicas (add [] remove []): existing descriptor r9:/Table/1{3-4} [(n1,s1):1VOTER_DEMOTING, (n2,s2):2, (n4,s4):3, (n3,s3):4VOTER_INCOMING, next=5, gen=6]
I200205 22:44:58.550755 4051 storage/replica_raft.go:248  [n4,s4,r9/3:/Table/1{3-4}] proposing LEAVE_JOINT: after=[(n1,s1):1LEARNER (n2,s2):2 (n4,s4):3 (n3,s3):4] next=5
I200205 22:44:58.553293 4051 storage/replica_command.go:1601  [n4,replicate,s4,r9/3:/Table/1{3-4}] change replicas (add [] remove [(n1,s1):1LEARNER]): existing descriptor r9:/Table/1{3-4} [(n1,s1):1LEARNER, (n2,s2):2, (n4,s4):3, (n3,s3):4, next=5, gen=7]
I200205 22:44:58.563311 24 storage/replica_raft.go:248  [n1,s1,r11/1:/Table/1{5-6}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3] next=4
I200205 22:44:58.582155 24 storage/replica_command.go:1601  [n1,replicate,s1,r24/1:/Table/2{8-9}] change replicas (add [(n3,s3):2LEARNER] remove []): existing descriptor r24:/Table/2{8-9} [(n1,s1):1, next=2, gen=0]
I200205 22:44:58.584382 4051 storage/replica_raft.go:248  [n4,s4,r9/3:/Table/1{3-4}] proposing SIMPLE(r1) REMOVE_REPLICA[(n1,s1):1LEARNER]: after=[(n3,s3):4 (n2,s2):2 (n4,s4):3] next=5
I200205 22:44:58.594792 143 storage/store_remove_replica.go:122  [n1,s1,r9/1:/Table/1{3-4}] removing replica r9/1
I200205 22:44:58.600877 24 storage/replica_raft.go:248  [n1,s1,r24/1:/Table/2{8-9}] proposing SIMPLE(l2) ADD_REPLICA[(n3,s3):2LEARNER]: after=[(n1,s1):1 (n3,s3):2LEARNER] next=3
I200205 22:44:58.603141 24 storage/store_snapshot.go:922  [n1,replicate,s1,r24/1:/Table/2{8-9}] sending LEARNER snapshot 0b231ec9 at applied index 15
I200205 22:44:58.603240 4401 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r24/1:/Table/2{8-9}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):2LEARNER
I200205 22:44:58.603303 24 storage/store_snapshot.go:965  [n1,replicate,s1,r24/1:/Table/2{8-9}] streamed snapshot to (n3,s3):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:58.606397 4387 storage/replica_raftstorage.go:780  [n3,s3,r24/2:{-}] applying LEARNER snapshot [id=0b231ec9 index=15]
I200205 22:44:58.614214 4387 storage/replica_raftstorage.go:801  [n3,s3,r24/2:/Table/2{8-9}] applied LEARNER snapshot [total=8ms ingestion=4@7ms id=0b231ec9 index=15]
I200205 22:44:58.615288 24 storage/replica_command.go:1601  [n1,replicate,s1,r24/1:/Table/2{8-9}] change replicas (add [(n3,s3):2] remove []): existing descriptor r24:/Table/2{8-9} [(n1,s1):1, (n3,s3):2LEARNER, next=3, gen=1]
I200205 22:44:58.625194 24 storage/replica_raft.go:248  [n1,s1,r24/1:/Table/2{8-9}] proposing SIMPLE(v2) ADD_REPLICA[(n3,s3):2]: after=[(n1,s1):1 (n3,s3):2] next=3
I200205 22:44:58.625818 24 storage/replica_command.go:1601  [n1,replicate,s1,r24/1:/Table/2{8-9}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r24:/Table/2{8-9} [(n1,s1):1, (n3,s3):2, next=3, gen=2]
I200205 22:44:58.640001 24 storage/replica_raft.go:248  [n1,s1,r24/1:/Table/2{8-9}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n2,s2):3LEARNER] next=4
I200205 22:44:58.643983 24 storage/store_snapshot.go:922  [n1,replicate,s1,r24/1:/Table/2{8-9}] sending LEARNER snapshot 764a7c75 at applied index 19
I200205 22:44:58.644127 24 storage/store_snapshot.go:965  [n1,replicate,s1,r24/1:/Table/2{8-9}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:58.644775 4521 storage/replica_raftstorage.go:780  [n2,s2,r24/3:{-}] applying LEARNER snapshot [id=764a7c75 index=19]
I200205 22:44:58.645255 4507 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r24/1:/Table/2{8-9}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):3LEARNER
I200205 22:44:58.658512 4521 storage/replica_raftstorage.go:801  [n2,s2,r24/3:/Table/2{8-9}] applied LEARNER snapshot [total=14ms ingestion=4@13ms id=764a7c75 index=19]
I200205 22:44:58.659375 24 storage/replica_command.go:1601  [n1,replicate,s1,r24/1:/Table/2{8-9}] change replicas (add [(n2,s2):3] remove []): existing descriptor r24:/Table/2{8-9} [(n1,s1):1, (n3,s3):2, (n2,s2):3LEARNER, next=4, gen=3]
I200205 22:44:58.684146 24 storage/replica_raft.go:248  [n1,s1,r24/1:/Table/2{8-9}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n3,s3):2 (n2,s2):3] next=4
I200205 22:44:58.687336 24 storage/replica_command.go:1601  [n1,replicate,s1,r23/1:/Table/2{7-8}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r23:/Table/2{7-8} [(n1,s1):1, next=2, gen=0]
I200205 22:44:58.710433 24 storage/replica_raft.go:248  [n1,s1,r23/1:/Table/2{7-8}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200205 22:44:58.715307 4525 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r23/1:/Table/2{7-8}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200205 22:44:58.716043 24 storage/store_snapshot.go:922  [n1,replicate,s1,r23/1:/Table/2{7-8}] sending LEARNER snapshot c5f4971e at applied index 15
I200205 22:44:58.716159 24 storage/store_snapshot.go:965  [n1,replicate,s1,r23/1:/Table/2{7-8}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:58.716777 4527 storage/replica_raftstorage.go:780  [n4,s4,r23/2:{-}] applying LEARNER snapshot [id=c5f4971e index=15]
I200205 22:44:58.720358 4527 storage/replica_raftstorage.go:801  [n4,s4,r23/2:/Table/2{7-8}] applied LEARNER snapshot [total=4ms ingestion=4@3ms id=c5f4971e index=15]
I200205 22:44:58.721983 24 storage/replica_command.go:1601  [n1,replicate,s1,r23/1:/Table/2{7-8}] change replicas (add [(n4,s4):2] remove []): existing descriptor r23:/Table/2{7-8} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200205 22:44:58.741135 24 storage/replica_raft.go:248  [n1,s1,r23/1:/Table/2{7-8}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200205 22:44:58.741701 24 storage/replica_command.go:1601  [n1,replicate,s1,r23/1:/Table/2{7-8}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r23:/Table/2{7-8} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200205 22:44:58.766162 24 storage/replica_raft.go:248  [n1,s1,r23/1:/Table/2{7-8}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3LEARNER] next=4
I200205 22:44:58.778716 24 storage/store_snapshot.go:922  [n1,replicate,s1,r23/1:/Table/2{7-8}] sending LEARNER snapshot 0916f9b3 at applied index 19
I200205 22:44:58.778854 24 storage/store_snapshot.go:965  [n1,replicate,s1,r23/1:/Table/2{7-8}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.01s
I200205 22:44:58.778988 4660 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r23/1:/Table/2{7-8}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):3LEARNER
I200205 22:44:58.779384 4578 storage/replica_raftstorage.go:780  [n2,s2,r23/3:{-}] applying LEARNER snapshot [id=0916f9b3 index=19]
I200205 22:44:58.784909 4578 storage/replica_raftstorage.go:801  [n2,s2,r23/3:/Table/2{7-8}] applied LEARNER snapshot [total=5ms ingestion=4@4ms id=0916f9b3 index=19]
I200205 22:44:58.794350 24 storage/replica_command.go:1601  [n1,replicate,s1,r23/1:/Table/2{7-8}] change replicas (add [(n2,s2):3] remove []): existing descriptor r23:/Table/2{7-8} [(n1,s1):1, (n4,s4):2, (n2,s2):3LEARNER, next=4, gen=3]
I200205 22:44:58.806842 24 storage/replica_raft.go:248  [n1,s1,r23/1:/Table/2{7-8}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3] next=4
I200205 22:44:58.809662 24 storage/replica_command.go:1601  [n1,replicate,s1,r22/1:/Table/2{6-7}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r22:/Table/2{6-7} [(n1,s1):1, next=2, gen=0]
I200205 22:44:58.833964 24 storage/replica_raft.go:248  [n1,s1,r22/1:/Table/2{6-7}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200205 22:44:58.835420 24 storage/store_snapshot.go:922  [n1,replicate,s1,r22/1:/Table/2{6-7}] sending LEARNER snapshot 415c55f3 at applied index 15
I200205 22:44:58.835534 24 storage/store_snapshot.go:965  [n1,replicate,s1,r22/1:/Table/2{6-7}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:58.835662 4593 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r22/1:/Table/2{6-7}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):2LEARNER
I200205 22:44:58.836037 4591 storage/replica_raftstorage.go:780  [n2,s2,r22/2:{-}] applying LEARNER snapshot [id=415c55f3 index=15]
I200205 22:44:58.842828 4591 storage/replica_raftstorage.go:801  [n2,s2,r22/2:/Table/2{6-7}] applied LEARNER snapshot [total=7ms ingestion=4@6ms id=415c55f3 index=15]
I200205 22:44:58.845317 24 storage/replica_command.go:1601  [n1,replicate,s1,r22/1:/Table/2{6-7}] change replicas (add [(n2,s2):2] remove []): existing descriptor r22:/Table/2{6-7} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
I200205 22:44:58.853380 24 storage/replica_raft.go:248  [n1,s1,r22/1:/Table/2{6-7}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200205 22:44:58.853908 24 storage/replica_command.go:1601  [n1,replicate,s1,r22/1:/Table/2{6-7}] change replicas (add [(n4,s4):3LEARNER] remove []): existing descriptor r22:/Table/2{6-7} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200205 22:44:58.871635 24 storage/replica_raft.go:248  [n1,s1,r22/1:/Table/2{6-7}] proposing SIMPLE(l3) ADD_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3LEARNER] next=4
I200205 22:44:58.877212 24 storage/store_snapshot.go:922  [n1,replicate,s1,r22/1:/Table/2{6-7}] sending LEARNER snapshot 625ec55b at applied index 20
I200205 22:44:58.877288 4758 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r22/1:/Table/2{6-7}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):3LEARNER
I200205 22:44:58.877338 24 storage/store_snapshot.go:965  [n1,replicate,s1,r22/1:/Table/2{6-7}] streamed snapshot to (n4,s4):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:58.877890 4683 storage/replica_raftstorage.go:780  [n4,s4,r22/3:{-}] applying LEARNER snapshot [id=625ec55b index=20]
I200205 22:44:58.887931 4683 storage/replica_raftstorage.go:801  [n4,s4,r22/3:/Table/2{6-7}] applied LEARNER snapshot [total=10ms ingestion=4@10ms id=625ec55b index=20]
I200205 22:44:58.898521 24 storage/replica_command.go:1601  [n1,replicate,s1,r22/1:/Table/2{6-7}] change replicas (add [(n4,s4):3] remove []): existing descriptor r22:/Table/2{6-7} [(n1,s1):1, (n2,s2):2, (n4,s4):3LEARNER, next=4, gen=3]
I200205 22:44:58.914712 24 storage/replica_raft.go:248  [n1,s1,r22/1:/Table/2{6-7}] proposing SIMPLE(v3) ADD_REPLICA[(n4,s4):3]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3] next=4
I200205 22:44:58.918354 24 storage/replica_command.go:1601  [n1,replicate,s1,r28/1:/{Table/32-Max}] change replicas (add [(n3,s3):2LEARNER] remove []): existing descriptor r28:/{Table/32-Max} [(n1,s1):1, next=2, gen=0]
I200205 22:44:58.945237 24 storage/replica_raft.go:248  [n1,s1,r28/1:/{Table/32-Max}] proposing SIMPLE(l2) ADD_REPLICA[(n3,s3):2LEARNER]: after=[(n1,s1):1 (n3,s3):2LEARNER] next=3
I200205 22:44:58.948408 4799 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r28/1:/{Table/32-Max}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):2LEARNER
I200205 22:44:58.948492 24 storage/store_snapshot.go:922  [n1,replicate,s1,r28/1:/{Table/32-Max}] sending LEARNER snapshot f14d6359 at applied index 15
I200205 22:44:58.951121 24 storage/store_snapshot.go:965  [n1,replicate,s1,r28/1:/{Table/32-Max}] streamed snapshot to (n3,s3):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.01s
I200205 22:44:58.954577 4766 storage/replica_raftstorage.go:780  [n3,s3,r28/2:{-}] applying LEARNER snapshot [id=f14d6359 index=15]
I200205 22:44:58.974984 4766 storage/replica_raftstorage.go:801  [n3,s3,r28/2:/{Table/32-Max}] applied LEARNER snapshot [total=20ms ingestion=4@20ms id=f14d6359 index=15]
I200205 22:44:58.975684 24 storage/replica_command.go:1601  [n1,replicate,s1,r28/1:/{Table/32-Max}] change replicas (add [(n3,s3):2] remove []): existing descriptor r28:/{Table/32-Max} [(n1,s1):1, (n3,s3):2LEARNER, next=3, gen=1]
I200205 22:44:58.994919 24 storage/replica_raft.go:248  [n1,s1,r28/1:/{Table/32-Max}] proposing SIMPLE(v2) ADD_REPLICA[(n3,s3):2]: after=[(n1,s1):1 (n3,s3):2] next=3
I200205 22:44:58.995827 24 storage/replica_command.go:1601  [n1,replicate,s1,r28/1:/{Table/32-Max}] change replicas (add [(n4,s4):3LEARNER] remove []): existing descriptor r28:/{Table/32-Max} [(n1,s1):1, (n3,s3):2, next=3, gen=2]
I200205 22:44:59.010207 24 storage/replica_raft.go:248  [n1,s1,r28/1:/{Table/32-Max}] proposing SIMPLE(l3) ADD_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3LEARNER] next=4
I200205 22:44:59.012034 4735 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r28/1:/{Table/32-Max}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):3LEARNER
I200205 22:44:59.012157 24 storage/store_snapshot.go:922  [n1,replicate,s1,r28/1:/{Table/32-Max}] sending LEARNER snapshot d7324087 at applied index 19
I200205 22:44:59.012404 24 storage/store_snapshot.go:965  [n1,replicate,s1,r28/1:/{Table/32-Max}] streamed snapshot to (n4,s4):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:59.013538 4733 storage/replica_raftstorage.go:780  [n4,s4,r28/3:{-}] applying LEARNER snapshot [id=d7324087 index=19]
I200205 22:44:59.018755 4733 storage/replica_raftstorage.go:801  [n4,s4,r28/3:/{Table/32-Max}] applied LEARNER snapshot [total=5ms ingestion=4@5ms id=d7324087 index=19]
I200205 22:44:59.019249 24 storage/replica_command.go:1601  [n1,replicate,s1,r28/1:/{Table/32-Max}] change replicas (add [(n4,s4):3] remove []): existing descriptor r28:/{Table/32-Max} [(n1,s1):1, (n3,s3):2, (n4,s4):3LEARNER, next=4, gen=3]
I200205 22:44:59.034096 24 storage/replica_raft.go:248  [n1,s1,r28/1:/{Table/32-Max}] proposing SIMPLE(v3) ADD_REPLICA[(n4,s4):3]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3] next=4
I200205 22:44:59.037628 24 storage/replica_command.go:1601  [n3,replicate,s3,r15/2:/Table/{19-20}] change replicas (add [(n2,s2):4LEARNER] remove []): existing descriptor r15:/Table/{19-20} [(n1,s1):1, (n3,s3):2, (n4,s4):3, next=4, gen=4]
I200205 22:44:59.050248 24 storage/replica_raft.go:248  [n3,s3,r15/2:/Table/{19-20}] proposing SIMPLE(l4) ADD_REPLICA[(n2,s2):4LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3 (n2,s2):4LEARNER] next=5
I200205 22:44:59.057587 24 storage/store_snapshot.go:922  [n3,replicate,s3,r15/2:/Table/{19-20}] sending LEARNER snapshot 894b0058 at applied index 29
I200205 22:44:59.057748 24 storage/store_snapshot.go:965  [n3,replicate,s3,r15/2:/Table/{19-20}] streamed snapshot to (n2,s2):4LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:44:59.059092 4932 storage/raft_snapshot_queue.go:169  [n3,raftsnapshot,s3,r15/2:/Table/{19-20}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):4LEARNER
I200205 22:44:59.059291 4815 storage/replica_raftstorage.go:780  [n2,s2,r15/4:{-}] applying LEARNER snapshot [id=894b0058 index=29]
I200205 22:44:59.062630 4815 storage/replica_raftstorage.go:801  [n2,s2,r15/4:/Table/{19-20}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=894b0058 index=29]
I200205 22:44:59.064213 24 storage/replica_command.go:1601  [n3,replicate,s3,r15/2:/Table/{19-20}] change replicas (add [(n2,s2):4VOTER_INCOMING] remove [(n1,s1):1VOTER_DEMOTING]): existing descriptor r15:/Table/{19-20} [(n1,s1):1, (n3,s3):2, (n4,s4):3, (n2,s2):4LEARNER, next=5, gen=5]
I200205 22:44:59.089763 24 storage/replica_raft.go:248  [n3,s3,r15/2:/Table/{19-20}] proposing ENTER_JOINT(r1 l1 v4) ADD_REPLICA[(n2,s2):4VOTER_INCOMING], REMOVE_REPLICA[(n1,s1):1VOTER_DEMOTING]: after=[(n1,s1):1VOTER_DEMOTING (n3,s3):2 (n4,s4):3 (n2,s2):4VOTER_INCOMING] next=5
I200205 22:44:59.090669 24 storage/replica_command.go:1601  [n3,replicate,s3,r15/2:/Table/{19-20}] change replicas (add [] remove []): existing descriptor r15:/Table/{19-20} [(n1,s1):1VOTER_DEMOTING, (n3,s3):2, (n4,s4):3, (n2,s2):4VOTER_INCOMING, next=5, gen=6]
I200205 22:44:59.104663 24 storage/replica_raft.go:248  [n3,s3,r15/2:/Table/{19-20}] proposing LEAVE_JOINT: after=[(n1,s1):1LEARNER (n3,s3):2 (n4,s4):3 (n2,s2):4] next=5
I200205 22:44:59.106929 24 storage/replica_command.go:1601  [n3,replicate,s3,r15/2:/Table/{19-20}] change replicas (add [] remove [(n1,s1):1LEARNER]): existing descriptor r15:/Table/{19-20} [(n1,s1):1LEARNER, (n3,s3):2, (n4,s4):3, (n2,s2):4, next=5, gen=7]
I200205 22:44:59.115489 24 storage/replica_raft.go:248  [n3,s3,r15/2:/Table/{19-20}] proposing SIMPLE(r1) REMOVE_REPLICA[(n1,s1):1LEARNER]: after=[(n2,s2):4 (n3,s3):2 (n4,s4):3] next=5
I200205 22:44:59.118895 143 storage/store_remove_replica.go:122  [n1,s1,r15/1:/Table/{19-20}] removing replica r15/1
I200205 22:44:59.140211 24 testutils/testcluster/testcluster.go:760  WaitForFullReplication took: 2.804287288s
I200205 22:44:59.186826 4996 sql/event_log.go:132  [n1,client=127.0.0.1:52400,user=root] Event: "set_cluster_setting", target: 0, info: {SettingName:kv.rangefeed.enabled Value:true User:root}
I200205 22:44:59.193449 24 kv/kvnemesis/watcher.go:131  watcher waiting for 1580942699.193423374,0
I200205 22:44:59.193979 5002 storage/replica_rangefeed.go:607  [n1,s1,r28/1:/{Table/32-Max}] RangeFeed closed timestamp is empty
I200205 22:44:59.214580 4996 sql/event_log.go:132  [n1,client=127.0.0.1:52400,user=root] Event: "set_cluster_setting", target: 0, info: {SettingName:kv.closed_timestamp.target_duration Value:00:00:00.001 User:root}
I200205 22:44:59.219914 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942696.051513234,0 lagging by 3.168354796s
I200205 22:44:59.220504 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.218651327,0 lagging by 1.836734ms
I200205 22:44:59.220531 5205 kv/kvnemesis/watcher.go:203  watcher notifying 1580942699.193423374,0
I200205 22:44:59.221284 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.219434602,0 lagging by 1.83526ms
I200205 22:44:59.221593 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.220212970,0 lagging by 1.366439ms
I200205 22:44:59.222150 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.220492459,0 lagging by 1.643217ms
I200205 22:44:59.222204 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.220849474,0 lagging by 1.341382ms
I200205 22:44:59.222521 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.221151291,0 lagging by 1.341991ms
I200205 22:44:59.222909 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.221446939,0 lagging by 1.447282ms
I200205 22:44:59.223183 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.221851479,0 lagging by 1.318209ms
I200205 22:44:59.224048 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.222084041,0 lagging by 1.948142ms
I200205 22:44:59.224402 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.222976896,0 lagging by 1.411116ms
I200205 22:44:59.225308 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.223326164,0 lagging by 1.966646ms
I200205 22:44:59.226994 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.224229184,0 lagging by 2.750002ms
I200205 22:44:59.233681 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.225888583,0 lagging by 7.776675ms
I200205 22:44:59.233709 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.227633345,0 lagging by 6.066029ms
I200205 22:44:59.233727 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.228526297,0 lagging by 5.192468ms
I200205 22:44:59.233743 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.228834793,0 lagging by 4.901845ms
I200205 22:44:59.233760 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.229158135,0 lagging by 4.592211ms
I200205 22:44:59.235194 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.229496204,0 lagging by 5.682063ms
I200205 22:44:59.235228 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.230775076,0 lagging by 4.443491ms
I200205 22:44:59.235255 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.231291037,0 lagging by 3.954378ms
I200205 22:44:59.235281 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.233030124,0 lagging by 2.242463ms
I200205 22:44:59.235306 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.233390098,0 lagging by 1.906679ms
I200205 22:44:59.235329 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.233720952,0 lagging by 1.600457ms
I200205 22:44:59.237740 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.234155732,0 lagging by 3.568906ms
I200205 22:44:59.239369 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.236672461,0 lagging by 2.681901ms
I200205 22:44:59.245408 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.238248731,0 lagging by 6.423795ms
I200205 22:44:59.245490 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.241325597,0 lagging by 4.151133ms
I200205 22:44:59.247259 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.243508040,0 lagging by 3.737339ms
I200205 22:44:59.247879 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.246169211,0 lagging by 1.695162ms
I200205 22:44:59.248146 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.246748508,0 lagging by 1.38352ms
I200205 22:44:59.248417 5205 kv/kvnemesis/watcher.go:197  watcher reached frontier 1580942699.247080509,0 lagging by 1.322205ms
I200205 22:44:59.251605 4996 sql/event_log.go:132  [n1,client=127.0.0.1:52400,user=root] Event: "set_cluster_setting", target: 0, info: {SettingName:kv.closed_timestamp.target_duration Value:DEFAULT User:root}
I200205 22:44:59.254305 5406 kv/kvnemesis/kvnemesis.go:63
  0 OP  db3.AdminMerge(ctx, /Table/50/"56715736") // storage/replica_command.go:776: merge failed: cannot merge final range
I200205 22:44:59.254485 5407 kv/kvnemesis/kvnemesis.go:63
  1 OP  db1.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  1 OP    b := &Batch{}
  1 OP    b.Get(ctx, /Table/50/"ec8754d4") // (nil, nil)
  1 OP    txn.CommitInBatch(ctx, b) // nil
  1 OP    return nil
  1 OP  }) // nil
I200205 22:44:59.255174 5410 kv/kvnemesis/kvnemesis.go:63
  4 OP  db0.Put(ctx, /Table/50/"fcc312b6", v-0) // nil
I200205 22:44:59.255356 5003 storage/rangefeed/processor.go:580  [n1,s1,r28/1:/{Table/32-Max},rangefeed] publishValue /Table/50/"fcc312b6":1580942699.253214408,0 -> {׺XYv-0 0,0}
I200205 22:44:59.255499 5205 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"fcc312b6" 1580942699.253214408,0 -> /BYTES/v-0 (prev /)
I200205 22:44:59.255595 5408 kv/kvnemesis/kvnemesis.go:63
  2 OP  db2.Get(ctx, /Table/50/"fcc312b6") // ("v-0", nil)
I200205 22:44:59.256774 5408 storage/replica_command.go:396  [n1,s1,r28/1:/{Table/32-Max}] initiating a split of this range at key /Table/50/"199e6d89" [r29] (manual)
I200205 22:44:59.258372 5409 kv/kvnemesis/kvnemesis.go:63
  3 OP  db0.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  3 OP    txn.Get(ctx, /Table/50/"851160cf") // (nil, nil)
  3 OP    txn.Put(ctx, /Table/50/"96405bb4", v-1) // nil
  3 OP    txn.Put(ctx, /Table/50/"fcc312b6", v-2) // nil
  3 OP    return nil
  3 OP  }) // nil
I200205 22:44:59.260901 5003 storage/rangefeed/processor.go:580  [n1,s1,r28/1:/{Table/32-Max},rangefeed] publishValue /Table/50/"96405bb4":1580942699.254027375,1 -> {�av-1 0,0}
I200205 22:44:59.260933 5003 storage/rangefeed/processor.go:580  [n1,s1,r28/1:/{Table/32-Max},rangefeed] publishValue /Table/50/"fcc312b6":1580942699.254027375,1 -> {9�9uv-2 0,0}
I200205 22:44:59.260970 5205 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"96405bb4" 1580942699.254027375,1 -> /BYTES/v-1 (prev /)
I200205 22:44:59.260994 5205 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"fcc312b6" 1580942699.254027375,1 -> /BYTES/v-2 (prev /BYTES/v-0)
I200205 22:44:59.270866 5003 storage/rangefeed/processor.go:580  [n1,s1,r28/1:/{Table/32-Max},rangefeed] publishValue /Table/50/"fcc312b6":1580942699.258458092,0 -> {�Ь�v-5 0,0}
I200205 22:44:59.270918 5205 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"fcc312b6" 1580942699.258458092,0 -> /BYTES/v-5 (prev /BYTES/v-2)
I200205 22:44:59.271810 5409 kv/kvnemesis/kvnemesis.go:63
  3 OP  db1.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  3 OP    txn.Get(ctx, /Table/50/"33d98d0e") // (nil, nil)
  3 OP    b := &Batch{}
  3 OP    b.Put(ctx, /Table/50/"fcc312b6", v-5) // nil
  3 OP    txn.CommitInBatch(ctx, b) // nil
  3 OP    return nil
  3 OP  }) // nil
I200205 22:44:59.277918 5408 kv/kvnemesis/kvnemesis.go:63
  2 OP  db0.AdminSplit(ctx, /Table/50/"199e6d89") // nil
I200205 22:44:59.280391 5408 kv/kvnemesis/kvnemesis.go:63
  2 OP  db3.AdminSplit(ctx, /Table/50/"199e6d89") // nil
I200205 22:44:59.283934 5408 storage/replica_command.go:396  [n1,s1,r29/1:/{Table/50/"19…-Max}] initiating a split of this range at key /Table/50/"5b46b7f0" [r30] (manual)
I200205 22:44:59.287007 5205 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"96405bb4" 1580942699.254027375,1 -> /BYTES/v-1 (prev /)
I200205 22:44:59.287043 5205 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"fcc312b6" 1580942699.253214408,0 -> /BYTES/v-0 (prev /)
I200205 22:44:59.287066 5205 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"fcc312b6" 1580942699.254027375,1 -> /BYTES/v-2 (prev /BYTES/v-0)
I200205 22:44:59.287088 5205 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"fcc312b6" 1580942699.258458092,0 -> /BYTES/v-5 (prev /BYTES/v-2)
I200205 22:44:59.295376 5410 kv/kvnemesis/kvnemesis.go:63
  4 OP  db3.AdminChangeReplicas(ctx, /Table/50/"d5d803a6", [{ADD_REPLICA n2,s2} {REMOVE_REPLICA n4,s4}]) // change replicas of r28 failed: descriptor changed: [expected] r28:/{Table/32-Max} [(n1,s1):1, (n3,s3):2, (n4,s4):3, next=4, gen=4] != [actual] r28:/Table/{32-50/"199e6d89"} [(n1,s1):1, (n3,s3):2, (n4,s4):3, next=4, gen=5]
I200205 22:44:59.299447 5676 storage/rangefeed/processor.go:580  [n1,s1,r29/1:/{Table/50/"19…-Max},rangefeed] publishValue /Table/50/"fcc312b6":1580942699.258458092,1 -> {�ל@v-4 0,0}
I200205 22:44:59.299547 5205 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"fcc312b6" 1580942699.258458092,1 -> /BYTES/v-4 (prev /BYTES/v-5)
I200205 22:44:59.301525 5407 kv/kvnemesis/kvnemesis.go:63
  1 OP  db2.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  1 OP    b := &Batch{}
  1 OP    b.Put(ctx, /Table/50/"fcc312b6", v-4) // nil
  1 OP    b.Get(ctx, /Table/50/"f7a3b888") // (nil, nil)
  1 OP    txn.CommitInBatch(ctx, b) // nil
  1 OP    return nil
  1 OP  }) // nil
I200205 22:44:59.301587 5407 kv/kvnemesis/kvnemesis.go:63
  1 OP  db2.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  1 OP    return nil
  1 OP  }) // nil
I200205 22:44:59.303138 5409 kv/kvnemesis/kvnemesis.go:63
  3 OP  db2.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  3 OP    txn.Put(ctx, /Table/50/"2a453a5d", v-6) // nil
  3 OP    return nil
  3 OP  }) // nil
I200205 22:44:59.304884 5676 storage/rangefeed/processor.go:580  [n1,s1,r29/1:/{Table/50/"19…-Max},rangefeed] publishValue /Table/50/"2a453a5d":1580942699.271863929,0 -> {d�ڮv-6 0,0}
I200205 22:44:59.304976 5205 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"2a453a5d" 1580942699.271863929,0 -> /BYTES/v-6 (prev /)
I200205 22:44:59.308524 5407 kv/kvnemesis/kvnemesis.go:63
  1 OP  db3.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  1 OP    txn.Get(ctx, /Table/50/"96405bb4") // ("v-1", nil)
  1 OP    txn.Get(ctx, /Table/50/"96405bb4") // ("v-1", nil)
  1 OP    txn.Put(ctx, /Table/50/"1eb1ce34", v-8) // nil
  1 OP    return errors.New("rollback")
  1 OP  }) // rollback
I200205 22:44:59.310663 5408 kv/kvnemesis/kvnemesis.go:63
  2 OP  db0.AdminSplit(ctx, /Table/50/"5b46b7f0") // nil
I200205 22:44:59.312969 5205 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"2a453a5d" 1580942699.271863929,0 -> /BYTES/v-6 (prev /)
I200205 22:44:59.313062 5205 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"96405bb4" 1580942699.254027375,1 -> /BYTES/v-1 (prev /)
I200205 22:44:59.313103 5205 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"fcc312b6" 1580942699.253214408,0 -> /BYTES/v-0 (prev /)
I200205 22:44:59.313132 5205 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"fcc312b6" 1580942699.254027375,1 -> /BYTES/v-2 (prev /BYTES/v-0)
I200205 22:44:59.313156 5205 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"fcc312b6" 1580942699.258458092,0 -> /BYTES/v-5 (prev /BYTES/v-2)
I200205 22:44:59.313184 5205 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"fcc312b6" 1580942699.258458092,1 -> /BYTES/v-4 (prev /BYTES/v-5)
I200205 22:44:59.321677 5407 kv/kvnemesis/kvnemesis.go:63
  1 OP  db1.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  1 OP    txn.Put(ctx, /Table/50/"1eb1ce34", v-11) // nil
  1 OP    txn.Get(ctx, /Table/50/"2fe55735") // (nil, nil)
  1 OP    return nil
  1 OP  }) // nil
I200205 22:44:59.322482 5407 kv/kvnemesis/kvnemesis.go:63
  1 OP  db3.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  1 OP    b := &Batch{}
  1 OP    b.Get(ctx, /Table/50/"2a453a5d") // ("v-6", nil)
  1 OP    txn.CommitInBatch(ctx, b) // nil
  1 OP    return nil
  1 OP  }) // nil
I200205 22:44:59.322872 5746 storage/rangefeed/processor.go:580  [n1,s1,r29/1:/Table/50/"{199e6d8…-5b46b7f…},rangefeed] publishValue /Table/50/"1eb1ce34":1580942699.308585216,0 -> F��v-11 0,0}
I200205 22:44:59.322931 5205 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"1eb1ce34" 1580942699.308585216,0 -> /BYTES/v-11 (prev /)
I200205 22:45:01.554419 5408 kv/kvnemesis/kvnemesis.go:63
  2 OP  db2.AdminMerge(ctx, /Table/50/"c38112af") // storage/replica_command.go:776: merge failed: cannot merge final range
I200205 22:45:01.565126 5408 kv/kvnemesis/kvnemesis.go:63
  2 OP  db1.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  2 OP    txn.Get(ctx, /Table/50/"823aaa3f") // (nil, nil)
  2 OP    txn.Get(ctx, /Table/50/"789266c8") // (nil, nil)
  2 OP    txn.Get(ctx, /Table/50/"6c82ac6b") // (nil, nil)
  2 OP    return errors.New("rollback")
  2 OP  }) // rollback
I200205 22:45:01.567894 5408 kv/kvnemesis/kvnemesis.go:63
  2 OP  db2.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  2 OP    txn.Put(ctx, /Table/50/"1e379de0", v-16) // nil
  2 OP    txn.Get(ctx, /Table/50/"d970f137") // (nil, nil)
  2 OP    b := &Batch{}
  2 OP    b.Get(ctx, /Table/50/"c99747f1") // (nil, nil)
  2 OP    txn.CommitInBatch(ctx, b) // nil
  2 OP    return nil
  2 OP  }) // nil
I200205 22:45:01.568040 5746 storage/rangefeed/processor.go:580  [n1,s1,r29/1:/Table/50/"{199e6d8…-5b46b7f…},rangefeed] publishValue /Table/50/"1e379de0":1580942701.565156555,0 -> {]���v-16 0,0}
I200205 22:45:01.568105 5205 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"1e379de0" 1580942701.565156555,0 -> /BYTES/v-16 (prev /)
I200205 22:45:01.569006 5408 kv/kvnemesis/kvnemesis.go:63
  2 OP  db3.Get(ctx, /Table/50/"1e379de0") // ("v-16", nil)
I200205 22:45:01.570518 6025 storage/replica_command.go:662  [n3,s3,r29/2:/Table/50/"{199e6d8…-5b46b7f…}] initiating a merge of r30:/{Table/50/"5b46b7f0"-Max} [(n1,s1):1, (n3,s3):2, (n4,s4):3, next=4, gen=6, sticky=9223372036.854775807,2147483647] into this range (manual)
I200205 22:45:01.582611 1232 storage/store_remove_replica.go:122  [n3,s3,r29/2:/Table/50/"{199e6d8…-5b46b7f…}] removing replica r30/2
I200205 22:45:01.583779 140 storage/store_remove_replica.go:122  [n1,s1,r29/1:/Table/50/"{199e6d8…-5b46b7f…}] removing replica r30/1
I200205 22:45:01.584126 5408 kv/kvnemesis/kvnemesis.go:63
  2 OP  db0.AdminMerge(ctx, /Table/50/"199e6d89") // nil
I200205 22:45:01.584847 1845 storage/store_remove_replica.go:122  [n4,s4,r29/3:/Table/50/"{199e6d8…-5b46b7f…}] removing replica r30/3
I200205 22:45:01.585912 5408 kv/kvnemesis/kvnemesis.go:63
  2 OP  db1.Get(ctx, /Table/50/"132c6b43") // (nil, nil)
I200205 22:45:01.586968 5408 kv/kvnemesis/kvnemesis.go:63
  2 OP  db2.Put(ctx, /Table/50/"444c245f", v-17) // nil
I200205 22:45:01.587480 5205 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"1e379de0" 1580942701.565156555,0 -> /BYTES/v-16 (prev /)
I200205 22:45:01.587743 6104 storage/rangefeed/processor.go:580  [n1,s1,r29/1:/{Table/50/"19…-Max},rangefeed] publishValue /Table/50/"444c245f":1580942701.585965273,0 -> {�v�/v-17 0,0}
I200205 22:45:01.587810 5205 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"444c245f" 1580942701.585965273,0 -> /BYTES/v-17 (prev /)
I200205 22:45:01.598270 5808 storage/replica_command.go:1601  [n3,s3,r29/2:/{Table/50/"19…-Max}] change replicas (add [(n2,s2):4LEARNER] remove []): existing descriptor r29:/{Table/50/"199e6d89"-Max} [(n1,s1):1, (n3,s3):2, (n4,s4):3, next=4, gen=7, sticky=9223372036.854775807,2147483647]
I200205 22:45:01.614919 5808 storage/replica_raft.go:248  [n3,s3,r29/2:/{Table/50/"19…-Max}] proposing SIMPLE(l4) ADD_REPLICA[(n2,s2):4LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3 (n2,s2):4LEARNER] next=5
I200205 22:45:01.619338 5808 storage/store_snapshot.go:922  [n3,s3,r29/2:/{Table/50/"19…-Max}] sending LEARNER snapshot 7e25c3af at applied index 35
I200205 22:45:01.619514 5808 storage/store_snapshot.go:965  [n3,s3,r29/2:/{Table/50/"19…-Max}] streamed snapshot to (n2,s2):4LEARNER: kv pairs: 21, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200205 22:45:01.619692 6149 storage/raft_snapshot_queue.go:169  [n3,raftsnapshot,s3,r29/2:/{Table/50/"19…-Max}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):4LEARNER
I200205 22:45:01.620179 6109 storage/replica_raftstorage.go:780  [n2,s2,r29/4:{-}] applying LEARNER snapshot [id=7e25c3af index=35]
I200205 22:45:01.628251 6109 storage/replica_raftstorage.go:801  [n2,s2,r29/4:/{Table/50/"19…-Max}] applied LEARNER snapshot [total=8ms ingestion=4@8ms id=7e25c3af index=35]
I200205 22:45:01.629538 5808 storage/replica_command.go:1601  [n3,s3,r29/2:/{Table/50/"19…-Max}] change replicas (add [(n2,s2):4VOTER_INCOMING] remove [(n1,s1):1VOTER_DEMOTING]): existing descriptor r29:/{Table/50/"199e6d89"-Max} [(n1,s1):1, (n3,s3):2, (n4,s4):3, (n2,s2):4LEARNER, next=5, gen=8, sticky=9223372036.854775807,2147483647]
I200205 22:45:01.646696 5808 storage/replica_raft.go:248  [n3,s3,r29/2:/{Table/50/"19…-Max}] proposing ENTER_JOINT(r1 l1 v4) ADD_REPLICA[(n2,s2):4VOTER_INCOMING], REMOVE_REPLICA[(n1,s1):1VOTER_DEMOTING]: after=[(n1,s1):1VOTER_DEMOTING (n3,s3):2 (n4,s4):3 (n2,s2):4VOTER_INCOMING] next=5
I200205 22:45:01.647743 5808 storage/replica_command.go:1601  [n3,s3,r29/2:/{Table/50/"19…-Max}] change replicas (add [] remove []): existing descriptor r29:/{Table/50/"199e6d89"-Max} [(n1,s1):1VOTER_DEMOTING, (n3,s3):2, (n4,s4):3, (n2,s2):4VOTER_INCOMING, next=5, gen=9, sticky=9223372036.854775807,2147483647]
I200205 22:45:01.657691 5808 storage/replica_raft.go:248  [n3,s3,r29/2:/{Table/50/"19…-Max}] proposing LEAVE_JOINT: after=[(n1,s1):1LEARNER (n3,s3):2 (n4,s4):3 (n2,s2):4] next=5
I200205 22:45:01.658750 5808 storage/replica_command.go:1601  [n3,s3,r29/2:/{Table/50/"19…-Max}] change replicas (add [] remove [(n1,s1):1LEARNER]): existing descriptor r29:/{Table/50/"199e6d89"-Max} [(n1,s1):1LEARNER, (n3,s3):2, (n4,s4):3, (n2,s2):4, next=5, gen=10, sticky=9223372036.854775807,2147483647]
I200205 22:45:01.668310 5808 storage/replica_raft.go:248  [n3,s3,r29/2:/{Table/50/"19…-Max}] proposing SIMPLE(r1) REMOVE_REPLICA[(n1,s1):1LEARNER]: after=[(n2,s2):4 (n3,s3):2 (n4,s4):3] next=5
I200205 22:45:01.670829 5408 kv/kvnemesis/kvnemesis.go:63
  2 OP  db3.AdminChangeReplicas(ctx, /Table/50/"901c52e8", [{ADD_REPLICA n2,s2} {REMOVE_REPLICA n1,s1}]) // nil
I200205 22:45:01.670993 1593 storage/store_remove_replica.go:122  [n1,s1,r29/1:/{Table/50/"19…-Max}] removing replica r29/1
I200205 22:45:01.671505 1593 storage/replica_destroy.go:217  [n1,s1,r29/1:/{Table/50/"19…-Max}] removed 15 (6+9) keys in 0ms [clear=0ms commit=0ms]
I200205 22:45:01.672296 5408 kv/kvnemesis/kvnemesis.go:63
  2 OP  db0.Put(ctx, /Table/50/"1eb1ce34", v-18) // nil
I200205 22:45:01.684912 5408 kv/kvnemesis/kvnemesis.go:63
  2 OP  db1.AdminMerge(ctx, /Table/50/"04e7cf0d") // storage/replica_command.go:776: merge failed: ranges not collocated; (n1,s1):1,(n3,s3):2,(n4,s4):3 != (n2,s2):4,(n3,s3):2,(n4,s4):3
I200205 22:45:01.706350 5409 kv/kvnemesis/kvnemesis.go:63
  3 OP  db0.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  3 OP    txn.Put(ctx, /Table/50/"823aaa3f", v-9) // nil
  3 OP    txn.Get(ctx, /Table/50/"1eb1ce34") // ("v-18", nil)
  3 OP    txn.Get(ctx, /Table/50/"806583b9") // (nil, nil)
  3 OP    b := &Batch{}
  3 OP    b.Put(ctx, /Table/50/"2a453a5d", v-10) // nil
  3 OP    txn.CommitInBatch(ctx, b) // nil
  3 OP    return nil
  3 OP  }) // nil
I200205 22:45:01.829713 5205 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"1e379de0" 1580942701.565156555,0 -> /BYTES/v-16 (prev /)
I200205 22:45:01.829759 5205 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"1eb1ce34" 1580942701.671345439,0 -> /BYTES/v-18 (prev /BYTES/v-11)
I200205 22:45:01.829792 5205 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"2a453a5d" 1580942701.703632020,0 -> /BYTES/v-10 (prev /BYTES/v-6)
I200205 22:45:01.829824 5205 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"444c245f" 1580942701.585965273,0 -> /BYTES/v-17 (prev /)
panic: resolved timestamp 1580942700.328635872,0 equal to or above timestamp of operation { txn_id:8827da21-056f-455a-80c8-17b31ad9acc4 txn_key:"\272\022a77ce3da\000\001" txn_min_timestamp: timestamp:     } [recovered]
	panic: resolved timestamp 1580942700.328635872,0 equal to or above timestamp of operation { txn_id:8827da21-056f-455a-80c8-17b31ad9acc4 txn_key:"\272\022a77ce3da\000\001" txn_min_timestamp: timestamp:     }

goroutine 6244 [running]:

github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).Recover(0xc001454750, 0x4363a00, 0xc00516adb0)

/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:182 +0x11f

panic(0x3627ea0, 0xc0049739f0)

/usr/local/go/src/runtime/panic.go:679 +0x1b2

github.com/cockroachdb/cockroach/pkg/storage/rangefeed.(*resolvedTimestamp).assertOpAboveRTS(0xc0047bfae0, 0x0, 0xc003f6d540, 0x0, 0x0, 0x0, 0x0, 0x15f0a302435e9ddc, 0xc000000000)

/go/src/github.com/cockroachdb/cockroach/pkg/storage/rangefeed/resolved_timestamp.go:250 +0x199

github.com/cockroachdb/cockroach/pkg/storage/rangefeed.(*resolvedTimestamp).consumeLogicalOp(0xc0047bfae0, 0x0, 0xc003f6d540, 0x0, 0x0, 0x0, 0x0, 0x389c1e0)

/go/src/github.com/cockroachdb/cockroach/pkg/storage/rangefeed/resolved_timestamp.go:145 +0x443

github.com/cockroachdb/cockroach/pkg/storage/rangefeed.(*resolvedTimestamp).ConsumeLogicalOp(0xc0047bfae0, 0x0, 0xc003f6d540, 0x0, 0x0, 0x0, 0x0, 0x6e3244)

/go/src/github.com/cockroachdb/cockroach/pkg/storage/rangefeed/resolved_timestamp.go:131 +0x4d

github.com/cockroachdb/cockroach/pkg/storage/rangefeed.(*Processor).consumeLogicalOps(0xc0047bfa20, 0x4363940, 0xc005260bc0, 0xc0051a9950, 0x1, 0x1)

/go/src/github.com/cockroachdb/cockroach/pkg/storage/rangefeed/processor.go:556 +0x1a4

github.com/cockroachdb/cockroach/pkg/storage/rangefeed.(*Processor).consumeEvent(0xc0047bfa20, 0x4363940, 0xc005260bc0, 0xc0051a9950, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0, ...)

/go/src/github.com/cockroachdb/cockroach/pkg/storage/rangefeed/processor.go:505 +0x21e

github.com/cockroachdb/cockroach/pkg/storage/rangefeed.(*Processor).Start.func1(0x4363a00, 0xc00516adb0)

/go/src/github.com/cockroachdb/cockroach/pkg/storage/rangefeed/processor.go:253 +0xad5

github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker.func1(0xc00458d100, 0xc001454750, 0xc00516ad80)

/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:197 +0x13e

created by github.com/cockroachdb/cockroach/pkg/util/stop.(*Stopper).RunWorker

/go/src/github.com/cockroachdb/cockroach/pkg/util/stop/stopper.go:190 +0xa8

@danhhz danhhz force-pushed the kvnemesis_multinode branch from d79d9d4 to 8776e10 Compare February 5, 2020 22:52
@danhhz
Copy link
Contributor Author

danhhz commented Feb 6, 2020

Okay, got a repro of the original problem. The missing value doesn't show up in my log line at the very beginning of rangefeed.(*Processor).publishValue. So, seems that either it's not getting logged as a logical op, or it's not making it out of the event channel.

@nvanbenschoten
Copy link
Member

Is it at all interesting that the transaction re-wrote the same key twice? Is there a pattern there?

The missing value doesn't show up in my log line at the very beginning of rangefeed.(*Processor).publishValue

Did you see the corresponding MVCCWriteIntentOp ops? Did you see the corresponding MVCCCommitIntentOp ops? I'd suggest using txn IDs to try to line these things up.

Were there any interesting Range state transitions going on at the same time?

panic: resolved timestamp 1580942700.328635872,0 equal to or above timestamp of operation { txn_id:8827da21-056f-455a-80c8-17b31ad9acc4 txn_key:"\272\022a77ce3da\000\001" txn_min_timestamp:<wall_time:1580942699322580444 > timestamp:<wall_time:1580942699322580444 > } [recovered]

isn't this fixed

This assertion could catch all kinds of bugs. Pretty much any invariant violation related to txns at the MVCC layer or invariant violation related to closed timestamps could trigger it.

@danhhz danhhz force-pushed the kvnemesis_multinode branch from 8776e10 to cb47fcd Compare February 6, 2020 16:46
@danhhz
Copy link
Contributor Author

danhhz commented Feb 6, 2020

Is it at all interesting that the transaction re-wrote the same key twice? Is there a pattern there?

I vague recall seeing this happen where the missing key hadn't been rewritten, but not sure about that. This latest repro had an overwrite, but it was an earlier txn.

Did you see the corresponding MVCCWriteIntentOp ops? Did you see the corresponding MVCCCommitIntentOp ops? I'd suggest using txn IDs to try to line these things up.

I didn't have logging for this at the time. This sometimes doesn't repro in an hour+ of roachprod-stressrace on 10 nodes, so it's reasonably rare. I pushed my latest logging to this PR (haven't gotten a repro with it yet), would you mind taking a look and suggesting anything I'm missing? I really should be working on something else right now so hoping to get this figured out in the next repro : - p

Were there any interesting Range state transitions going on at the same time?

Hard to say, there was definitely a split and an upreplicate going on around the same time. I think I need to print each op before and after it runs, instead of just after. Then I could see definitively what this overlapped. Right now I can only look at things that finished shortly after, for some vague definition of "shortly after".

This assertion could catch all kinds of bugs. Pretty much any invariant violation related to txns at the MVCC layer or invariant violation related to closed timestamps could trigger it.

Yeah, makes sense. I've only seen it the once, but it does sounds like there's something there. I'll file an issue.

Here's the full logs of this morning's repro btw, nothing particularly interesting in here besides this:

W200206 15:30:03.620834 5332 storage/intentresolver/intent_resolver.go:820  failed to gc transaction record: could not GC completed transaction anchored at /Table/50/"ecc9fb13": node unavailable; try another peer

=== RUN   TestKVNemesisMultiNode
I200206 15:29:59.013835 12 base/addr_validation.go:296  [n?] server certificate addresses: IP=127.0.0.1,::1; DNS=localhost,*.local; CN=node
I200206 15:29:59.013869 12 base/addr_validation.go:342  [n?] web UI certificate addresses: IP=127.0.0.1,::1; DNS=localhost,*.local; CN=node
W200206 15:29:59.023828 12 server/status/runtime.go:308  [n?] Could not parse build timestamp: parsing time "" as "2006/01/02 15:04:05": cannot parse "" as "2006"
I200206 15:29:59.038510 12 server/server.go:988  [n?] monitoring forward clock jumps based on server.clock.forward_jump_check_enabled
I200206 15:29:59.046860 12 server/config.go:565  [n?] 1 storage engine initialized
I200206 15:29:59.046889 12 server/config.go:568  [n?] RocksDB cache size: 128 MiB
I200206 15:29:59.046899 12 server/config.go:568  [n?] store 0: in-memory, size 0 B
I200206 15:29:59.047196 12 server/server.go:1443  [n?] no stores bootstrapped and --join flag specified, awaiting init command or join with an already initialized node.
I200206 15:29:59.047993 12 server/server.go:1453  [n?] **** add additional nodes by specifying --join=127.0.0.1:44373
I200206 15:29:59.057946 12 server/node.go:334  [n?] **** cluster 497e2b2e-4a10-49f2-9788-ec20c9a7b0a6 has been created
I200206 15:29:59.058253 12 gossip/gossip.go:395  [n1] NodeDescriptor set to node_id:1 address: attrs:<> locality: tiers: > ServerVersion: build_tag:"v20.1.0-alpha.20191118-1912-g8776e10-dirty" started_at:1581002999058058183 cluster_name:"" sql_address:
I200206 15:29:59.065661 12 server/node.go:438  [n1] initialized store [n1,s1]: disk (capacity=512 MiB, available=512 MiB, used=0 B, logicalBytes=13 KiB), ranges=28, leases=0, queries=0.00, writes=0.00, bytesPerReplica={p10=0.00 p25=0.00 p50=0.00 p75=0.00 p90=950.00 pMax=11033.00}, writesPerReplica={p10=0.00 p25=0.00 p50=0.00 p75=0.00 p90=0.00 pMax=0.00}
I200206 15:29:59.065791 12 storage/stores.go:240  [n1] read 0 node addresses from persistent storage
I200206 15:29:59.065928 12 server/node.go:655  [n1] connecting to gossip network to verify cluster ID...
I200206 15:29:59.065976 12 server/node.go:675  [n1] node connected via gossip and verified as part of cluster "497e2b2e-4a10-49f2-9788-ec20c9a7b0a6"
I200206 15:29:59.066067 12 server/node.go:522  [n1] node=1: started with [=] engine(s) and attributes []
I200206 15:29:59.066162 12 server/server.go:2131  [n1] Could not start goroutine dumper worker due to: directory to store dumps could not be determined
I200206 15:29:59.066286 12 server/server.go:1573  [n1] starting https server at 127.0.0.1:40283 (use: 127.0.0.1:40283)
I200206 15:29:59.066356 12 server/server.go:1578  [n1] starting postgres server at 127.0.0.1:44467 (use: 127.0.0.1:44467)
I200206 15:29:59.066391 12 server/server.go:1580  [n1] starting grpc server at 127.0.0.1:44373
I200206 15:29:59.066439 12 server/server.go:1581  [n1] advertising CockroachDB node at 127.0.0.1:44373
W200206 15:29:59.074763 197 storage/store.go:1538  [n1,s1,r6/1:/Table/{SystemCon…-11}] could not gossip system config: [NotLeaseHolderError] r6: replica (n1,s1):1 not lease holder; lease holder unknown
I200206 15:29:59.100505 283 sql/event_log.go:132  [n1,intExec=optInToDiagnosticsStatReporting] Event: "set_cluster_setting", target: 0, info: {SettingName:diagnostics.reporting.enabled Value:true User:root}
I200206 15:29:59.110547 403 sql/event_log.go:132  [n1,intExec=set-setting] Event: "set_cluster_setting", target: 0, info: {SettingName:version Value:19.2-11 User:root}
I200206 15:29:59.134689 645 sql/event_log.go:132  [n1,intExec=initializeClusterSecret] Event: "set_cluster_setting", target: 0, info: {SettingName:cluster.secret Value:a72e6490-8612-4741-9858-9b9dd1d4274e User:root}
I200206 15:29:59.140259 661 sql/event_log.go:132  [n1,intExec=create-default-db] Event: "create_database", target: 50, info: {DatabaseName:defaultdb Statement:CREATE DATABASE IF NOT EXISTS defaultdb User:root}
I200206 15:29:59.152057 666 sql/event_log.go:132  [n1,intExec=create-default-db] Event: "create_database", target: 51, info: {DatabaseName:postgres Statement:CREATE DATABASE IF NOT EXISTS postgres User:root}
I200206 15:29:59.168282 12 server/server.go:1655  [n1] done ensuring all necessary migrations have run
I200206 15:29:59.168323 12 server/server.go:1936  [n1] serving sql connections
I200206 15:29:59.172058 946 sql/event_log.go:132  [n1] Event: "node_join", target: 1, info: {Descriptor:{NodeID:1 Address:127.0.0.1:44373 Attrs: Locality:region=test,dc=dc1 ServerVersion:19.2-11 BuildTag:v20.1.0-alpha.20191118-1912-g8776e10-dirty StartedAt:1581002999058058183 LocalityAddress:[] ClusterName: SQLAddress:127.0.0.1:44467} ClusterID:497e2b2e-4a10-49f2-9788-ec20c9a7b0a6 StartedAt:1581002999058058183 LastUp:1581002999058058183}
I200206 15:29:59.173785 640 server/server_update.go:55  [n1] no need to upgrade, cluster already at the newest version
I200206 15:29:59.174902 12 base/addr_validation.go:296  [n?] server certificate addresses: IP=127.0.0.1,::1; DNS=localhost,*.local; CN=node
I200206 15:29:59.174930 12 base/addr_validation.go:342  [n?] web UI certificate addresses: IP=127.0.0.1,::1; DNS=localhost,*.local; CN=node
W200206 15:29:59.183939 12 server/status/runtime.go:308  [n?] Could not parse build timestamp: parsing time "" as "2006/01/02 15:04:05": cannot parse "" as "2006"
I200206 15:29:59.211038 12 server/server.go:988  [n?] monitoring forward clock jumps based on server.clock.forward_jump_check_enabled
I200206 15:29:59.230492 12 server/config.go:565  [n?] 1 storage engine initialized
I200206 15:29:59.230518 12 server/config.go:568  [n?] RocksDB cache size: 128 MiB
I200206 15:29:59.230526 12 server/config.go:568  [n?] store 0: in-memory, size 0 B
W200206 15:29:59.230606 12 gossip/gossip.go:1518  [n?] no incoming or outgoing connections
I200206 15:29:59.230658 12 server/server.go:1443  [n?] no stores bootstrapped and --join flag specified, awaiting init command or join with an already initialized node.
I200206 15:29:59.239815 957 gossip/client.go:124  [n?] started gossip client to 127.0.0.1:44373
I200206 15:29:59.240257 1053 gossip/server.go:227  [n1] received initial cluster-verification connection from 127.0.0.1:43543
I200206 15:29:59.240809 12 server/node.go:655  [n?] connecting to gossip network to verify cluster ID...
I200206 15:29:59.241372 12 server/node.go:675  [n?] node connected via gossip and verified as part of cluster "497e2b2e-4a10-49f2-9788-ec20c9a7b0a6"
I200206 15:29:59.241450 1053 gossip/server.go:227  [n1] received initial cluster-verification connection from 127.0.0.1:43543
I200206 15:29:59.269655 12 server/node.go:388  [n?] new node allocated ID 2
I200206 15:29:59.269743 12 gossip/gossip.go:395  [n2] NodeDescriptor set to node_id:2 address: attrs:<> locality: tiers: > ServerVersion: build_tag:"v20.1.0-alpha.20191118-1912-g8776e10-dirty" started_at:1581002999269676328 cluster_name:"" sql_address:
I200206 15:29:59.269793 12 storage/stores.go:240  [n2] read 0 node addresses from persistent storage
I200206 15:29:59.269853 12 storage/stores.go:259  [n2] wrote 1 node addresses to persistent storage
I200206 15:29:59.270199 15 storage/stores.go:259  [n1] wrote 1 node addresses to persistent storage
I200206 15:29:59.272290 12 server/node.go:630  [n2] bootstrapped store [n2,s2]
I200206 15:29:59.272427 12 server/node.go:522  [n2] node=2: started with [=] engine(s) and attributes []
I200206 15:29:59.272502 12 server/server.go:2131  [n2] Could not start goroutine dumper worker due to: directory to store dumps could not be determined
I200206 15:29:59.272565 12 server/server.go:1573  [n2] starting https server at 127.0.0.1:33057 (use: 127.0.0.1:33057)
I200206 15:29:59.272605 12 server/server.go:1578  [n2] starting postgres server at 127.0.0.1:44861 (use: 127.0.0.1:44861)
I200206 15:29:59.272627 12 server/server.go:1580  [n2] starting grpc server at 127.0.0.1:43543
I200206 15:29:59.272652 12 server/server.go:1581  [n2] advertising CockroachDB node at 127.0.0.1:43543
I200206 15:29:59.297547 12 server/server.go:1655  [n2] done ensuring all necessary migrations have run
I200206 15:29:59.297601 12 server/server.go:1936  [n2] serving sql connections
W200206 15:29:59.304165 12 jobs/registry.go:392  [n2] unable to get node liveness: node not in the liveness table
I200206 15:29:59.307086 12 base/addr_validation.go:296  [n?] server certificate addresses: IP=127.0.0.1,::1; DNS=localhost,*.local; CN=node
I200206 15:29:59.307105 12 base/addr_validation.go:342  [n?] web UI certificate addresses: IP=127.0.0.1,::1; DNS=localhost,*.local; CN=node
W200206 15:29:59.345922 12 server/status/runtime.go:308  [n?] Could not parse build timestamp: parsing time "" as "2006/01/02 15:04:05": cannot parse "" as "2006"
I200206 15:29:59.357325 1163 server/server_update.go:55  [n2] no need to upgrade, cluster already at the newest version
I200206 15:29:59.357440 1165 sql/event_log.go:132  [n2] Event: "node_join", target: 2, info: {Descriptor:{NodeID:2 Address:127.0.0.1:43543 Attrs: Locality:region=test,dc=dc2 ServerVersion:19.2-11 BuildTag:v20.1.0-alpha.20191118-1912-g8776e10-dirty StartedAt:1581002999269676328 LocalityAddress:[] ClusterName: SQLAddress:127.0.0.1:44861} ClusterID:497e2b2e-4a10-49f2-9788-ec20c9a7b0a6 StartedAt:1581002999269676328 LastUp:1581002999269676328}
I200206 15:29:59.357843 12 server/server.go:988  [n?] monitoring forward clock jumps based on server.clock.forward_jump_check_enabled
I200206 15:29:59.378440 12 server/config.go:565  [n?] 1 storage engine initialized
I200206 15:29:59.378476 12 server/config.go:568  [n?] RocksDB cache size: 128 MiB
I200206 15:29:59.378485 12 server/config.go:568  [n?] store 0: in-memory, size 0 B
W200206 15:29:59.378563 12 gossip/gossip.go:1518  [n?] no incoming or outgoing connections
I200206 15:29:59.378614 12 server/server.go:1443  [n?] no stores bootstrapped and --join flag specified, awaiting init command or join with an already initialized node.
I200206 15:29:59.387838 1210 gossip/client.go:124  [n?] started gossip client to 127.0.0.1:44373
I200206 15:29:59.388267 1290 gossip/server.go:227  [n1] received initial cluster-verification connection from 127.0.0.1:35593
I200206 15:29:59.388941 12 server/node.go:655  [n?] connecting to gossip network to verify cluster ID...
I200206 15:29:59.388963 12 server/node.go:675  [n?] node connected via gossip and verified as part of cluster "497e2b2e-4a10-49f2-9788-ec20c9a7b0a6"
I200206 15:29:59.389426 1290 gossip/server.go:227  [n1] received initial cluster-verification connection from 127.0.0.1:35593
I200206 15:29:59.414475 12 server/node.go:388  [n?] new node allocated ID 3
I200206 15:29:59.414550 12 gossip/gossip.go:395  [n3] NodeDescriptor set to node_id:3 address: attrs:<> locality: tiers: > ServerVersion: build_tag:"v20.1.0-alpha.20191118-1912-g8776e10-dirty" started_at:1581002999414492013 cluster_name:"" sql_address:
I200206 15:29:59.414637 12 storage/stores.go:240  [n3] read 0 node addresses from persistent storage
I200206 15:29:59.414673 12 storage/stores.go:259  [n3] wrote 2 node addresses to persistent storage
I200206 15:29:59.415113 15 storage/stores.go:259  [n1] wrote 2 node addresses to persistent storage
I200206 15:29:59.415365 955 storage/stores.go:259  [n2] wrote 2 node addresses to persistent storage
I200206 15:29:59.416554 12 server/node.go:630  [n3] bootstrapped store [n3,s3]
I200206 15:29:59.416671 12 server/node.go:522  [n3] node=3: started with [=] engine(s) and attributes []
I200206 15:29:59.416731 12 server/server.go:2131  [n3] Could not start goroutine dumper worker due to: directory to store dumps could not be determined
I200206 15:29:59.416782 12 server/server.go:1573  [n3] starting https server at 127.0.0.1:44727 (use: 127.0.0.1:44727)
I200206 15:29:59.416808 12 server/server.go:1578  [n3] starting postgres server at 127.0.0.1:40173 (use: 127.0.0.1:40173)
I200206 15:29:59.416824 12 server/server.go:1580  [n3] starting grpc server at 127.0.0.1:35593
I200206 15:29:59.416839 12 server/server.go:1581  [n3] advertising CockroachDB node at 127.0.0.1:35593
I200206 15:29:59.437489 281 storage/replica_command.go:1601  [n1,replicate,s1,r19/1:/Table/2{3-4}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r19:/Table/2{3-4} [(n1,s1):1, next=2, gen=0]
I200206 15:29:59.440256 12 server/server.go:1655  [n3] done ensuring all necessary migrations have run
I200206 15:29:59.440330 12 server/server.go:1936  [n3] serving sql connections
I200206 15:29:59.454819 281 storage/replica_raft.go:248  [n1,s1,r19/1:/Table/2{3-4}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200206 15:29:59.456638 12 base/addr_validation.go:296  [n?] server certificate addresses: IP=127.0.0.1,::1; DNS=localhost,*.local; CN=node
I200206 15:29:59.456687 12 base/addr_validation.go:342  [n?] web UI certificate addresses: IP=127.0.0.1,::1; DNS=localhost,*.local; CN=node
I200206 15:29:59.462327 1281 server/server_update.go:55  [n3] no need to upgrade, cluster already at the newest version
I200206 15:29:59.463620 1475 sql/event_log.go:132  [n3] Event: "node_join", target: 3, info: {Descriptor:{NodeID:3 Address:127.0.0.1:35593 Attrs: Locality:region=test,dc=dc3 ServerVersion:19.2-11 BuildTag:v20.1.0-alpha.20191118-1912-g8776e10-dirty StartedAt:1581002999414492013 LocalityAddress:[] ClusterName: SQLAddress:127.0.0.1:40173} ClusterID:497e2b2e-4a10-49f2-9788-ec20c9a7b0a6 StartedAt:1581002999414492013 LastUp:1581002999414492013}
I200206 15:29:59.467187 1571 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r19/1:/Table/2{3-4}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):2LEARNER
I200206 15:29:59.467342 281 storage/store_snapshot.go:922  [n1,replicate,s1,r19/1:/Table/2{3-4}] sending LEARNER snapshot d7cc8a71 at applied index 17
I200206 15:29:59.467469 281 storage/store_snapshot.go:965  [n1,replicate,s1,r19/1:/Table/2{3-4}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:29:59.468120 1562 storage/replica_raftstorage.go:780  [n2,s2,r19/2:{-}] applying LEARNER snapshot [id=d7cc8a71 index=17]
W200206 15:29:59.470161 12 server/status/runtime.go:308  [n?] Could not parse build timestamp: parsing time "" as "2006/01/02 15:04:05": cannot parse "" as "2006"
I200206 15:29:59.478778 12 server/server.go:988  [n?] monitoring forward clock jumps based on server.clock.forward_jump_check_enabled
I200206 15:29:59.492571 1562 storage/replica_raftstorage.go:801  [n2,s2,r19/2:/Table/2{3-4}] applied LEARNER snapshot [total=24ms ingestion=4@11ms id=d7cc8a71 index=17]
I200206 15:29:59.512402 281 storage/replica_command.go:1601  [n1,replicate,s1,r19/1:/Table/2{3-4}] change replicas (add [(n2,s2):2] remove []): existing descriptor r19:/Table/2{3-4} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
I200206 15:29:59.515991 12 server/config.go:565  [n?] 1 storage engine initialized
I200206 15:29:59.516026 12 server/config.go:568  [n?] RocksDB cache size: 128 MiB
I200206 15:29:59.516035 12 server/config.go:568  [n?] store 0: in-memory, size 0 B
W200206 15:29:59.516107 12 gossip/gossip.go:1518  [n?] no incoming or outgoing connections
I200206 15:29:59.516156 12 server/server.go:1443  [n?] no stores bootstrapped and --join flag specified, awaiting init command or join with an already initialized node.
I200206 15:29:59.515998 281 storage/replica_raft.go:248  [n1,s1,r19/1:/Table/2{3-4}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200206 15:29:59.516778 281 storage/replica_command.go:1601  [n1,replicate,s1,r19/1:/Table/2{3-4}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r19:/Table/2{3-4} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200206 15:29:59.526028 281 storage/replica_raft.go:248  [n1,s1,r19/1:/Table/2{3-4}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3LEARNER] next=4
I200206 15:29:59.528982 1577 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r19/1:/Table/2{3-4}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200206 15:29:59.529135 281 storage/store_snapshot.go:922  [n1,replicate,s1,r19/1:/Table/2{3-4}] sending LEARNER snapshot 03a39421 at applied index 21
I200206 15:29:59.529264 281 storage/store_snapshot.go:965  [n1,replicate,s1,r19/1:/Table/2{3-4}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 14, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:29:59.529866 1456 storage/replica_raftstorage.go:780  [n3,s3,r19/3:{-}] applying LEARNER snapshot [id=03a39421 index=21]
I200206 15:29:59.532148 1456 storage/replica_raftstorage.go:801  [n3,s3,r19/3:/Table/2{3-4}] applied LEARNER snapshot [total=2ms ingestion=4@2ms id=03a39421 index=21]
I200206 15:29:59.532909 281 storage/replica_command.go:1601  [n1,replicate,s1,r19/1:/Table/2{3-4}] change replicas (add [(n3,s3):3] remove []): existing descriptor r19:/Table/2{3-4} [(n1,s1):1, (n2,s2):2, (n3,s3):3LEARNER, next=4, gen=3]
I200206 15:29:59.537802 281 storage/replica_raft.go:248  [n1,s1,r19/1:/Table/2{3-4}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3] next=4
I200206 15:29:59.538852 281 storage/replica_command.go:1601  [n1,replicate,s1,r17/1:/Table/2{1-2}] change replicas (add [(n3,s3):2LEARNER] remove []): existing descriptor r17:/Table/2{1-2} [(n1,s1):1, next=2, gen=0]
I200206 15:29:59.541310 1517 gossip/client.go:124  [n?] started gossip client to 127.0.0.1:44373
I200206 15:29:59.541337 281 storage/replica_raft.go:248  [n1,s1,r17/1:/Table/2{1-2}] proposing SIMPLE(l2) ADD_REPLICA[(n3,s3):2LEARNER]: after=[(n1,s1):1 (n3,s3):2LEARNER] next=3
I200206 15:29:59.543271 1691 gossip/server.go:227  [n1] received initial cluster-verification connection from 127.0.0.1:35387
I200206 15:29:59.543929 12 server/node.go:655  [n?] connecting to gossip network to verify cluster ID...
I200206 15:29:59.543956 12 server/node.go:675  [n?] node connected via gossip and verified as part of cluster "497e2b2e-4a10-49f2-9788-ec20c9a7b0a6"
I200206 15:29:59.544587 1691 gossip/server.go:227  [n1] received initial cluster-verification connection from 127.0.0.1:35387
I200206 15:29:59.551158 281 storage/store_snapshot.go:922  [n1,replicate,s1,r17/1:/Table/2{1-2}] sending LEARNER snapshot 35eec863 at applied index 21
I200206 15:29:59.551261 1632 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r17/1:/Table/2{1-2}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):2LEARNER
I200206 15:29:59.553516 1549 storage/replica_raftstorage.go:780  [n3,s3,r17/2:{-}] applying LEARNER snapshot [id=35eec863 index=21]
I200206 15:29:59.553673 281 storage/store_snapshot.go:965  [n1,replicate,s1,r17/1:/Table/2{1-2}] streamed snapshot to (n3,s3):2LEARNER: kv pairs: 11, log entries: 0, rate-limit: 8.0 MiB/sec, 0.01s
I200206 15:29:59.585916 1549 storage/replica_raftstorage.go:801  [n3,s3,r17/2:/Table/2{1-2}] applied LEARNER snapshot [total=32ms ingestion=4@21ms id=35eec863 index=21]
I200206 15:29:59.588654 281 storage/replica_command.go:1601  [n1,replicate,s1,r17/1:/Table/2{1-2}] change replicas (add [(n3,s3):2] remove []): existing descriptor r17:/Table/2{1-2} [(n1,s1):1, (n3,s3):2LEARNER, next=3, gen=1]
I200206 15:29:59.591036 281 storage/replica_raft.go:248  [n1,s1,r17/1:/Table/2{1-2}] proposing SIMPLE(v2) ADD_REPLICA[(n3,s3):2]: after=[(n1,s1):1 (n3,s3):2] next=3
I200206 15:29:59.591480 281 storage/replica_command.go:1601  [n1,replicate,s1,r17/1:/Table/2{1-2}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r17:/Table/2{1-2} [(n1,s1):1, (n3,s3):2, next=3, gen=2]
I200206 15:29:59.599631 281 storage/replica_raft.go:248  [n1,s1,r17/1:/Table/2{1-2}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n2,s2):3LEARNER] next=4
I200206 15:29:59.599692 12 server/node.go:388  [n?] new node allocated ID 4
I200206 15:29:59.599761 12 gossip/gossip.go:395  [n4] NodeDescriptor set to node_id:4 address: attrs:<> locality: tiers: > ServerVersion: build_tag:"v20.1.0-alpha.20191118-1912-g8776e10-dirty" started_at:1581002999599705044 cluster_name:"" sql_address:
I200206 15:29:59.599872 12 storage/stores.go:240  [n4] read 0 node addresses from persistent storage
I200206 15:29:59.599914 12 storage/stores.go:259  [n4] wrote 3 node addresses to persistent storage
I200206 15:29:59.601923 15 storage/stores.go:259  [n1] wrote 3 node addresses to persistent storage
I200206 15:29:59.602101 12 server/node.go:630  [n4] bootstrapped store [n4,s4]
I200206 15:29:59.602191 12 server/node.go:522  [n4] node=4: started with [=] engine(s) and attributes []
I200206 15:29:59.602239 12 server/server.go:2131  [n4] Could not start goroutine dumper worker due to: directory to store dumps could not be determined
I200206 15:29:59.602315 12 server/server.go:1573  [n4] starting https server at 127.0.0.1:43043 (use: 127.0.0.1:43043)
I200206 15:29:59.602340 12 server/server.go:1578  [n4] starting postgres server at 127.0.0.1:43479 (use: 127.0.0.1:43479)
I200206 15:29:59.602358 12 server/server.go:1580  [n4] starting grpc server at 127.0.0.1:35387
I200206 15:29:59.602372 12 server/server.go:1581  [n4] advertising CockroachDB node at 127.0.0.1:35387
I200206 15:29:59.602382 955 storage/stores.go:259  [n2] wrote 3 node addresses to persistent storage
I200206 15:29:59.602594 1035 storage/stores.go:259  [n3] wrote 3 node addresses to persistent storage
I200206 15:29:59.621374 1634 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r17/1:/Table/2{1-2}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):3LEARNER
I200206 15:29:59.626829 281 storage/store_snapshot.go:922  [n1,replicate,s1,r17/1:/Table/2{1-2}] sending LEARNER snapshot 4aee132b at applied index 25
I200206 15:29:59.627059 281 storage/store_snapshot.go:965  [n1,replicate,s1,r17/1:/Table/2{1-2}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 15, log entries: 0, rate-limit: 8.0 MiB/sec, 0.03s
I200206 15:29:59.627989 1880 storage/replica_raftstorage.go:780  [n2,s2,r17/3:{-}] applying LEARNER snapshot [id=4aee132b index=25]
I200206 15:29:59.631993 1880 storage/replica_raftstorage.go:801  [n2,s2,r17/3:/Table/2{1-2}] applied LEARNER snapshot [total=4ms ingestion=4@4ms id=4aee132b index=25]
I200206 15:29:59.632575 281 storage/replica_command.go:1601  [n1,replicate,s1,r17/1:/Table/2{1-2}] change replicas (add [(n2,s2):3] remove []): existing descriptor r17:/Table/2{1-2} [(n1,s1):1, (n3,s3):2, (n2,s2):3LEARNER, next=4, gen=3]
I200206 15:29:59.636321 12 server/server.go:1655  [n4] done ensuring all necessary migrations have run
I200206 15:29:59.636388 12 server/server.go:1936  [n4] serving sql connections
I200206 15:29:59.670252 281 storage/replica_raft.go:248  [n1,s1,r17/1:/Table/2{1-2}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n3,s3):2 (n2,s2):3] next=4
I200206 15:29:59.678323 1890 sql/event_log.go:132  [n4] Event: "node_join", target: 4, info: {Descriptor:{NodeID:4 Address:127.0.0.1:35387 Attrs: Locality:region=test,dc=dc4 ServerVersion:19.2-11 BuildTag:v20.1.0-alpha.20191118-1912-g8776e10-dirty StartedAt:1581002999599705044 LocalityAddress:[] ClusterName: SQLAddress:127.0.0.1:43479} ClusterID:497e2b2e-4a10-49f2-9788-ec20c9a7b0a6 StartedAt:1581002999599705044 LastUp:1581002999599705044}
I200206 15:29:59.695697 281 storage/replica_command.go:1601  [n1,replicate,s1,r27/1:/{NamespaceTab…-Table/32}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r27:/{NamespaceTable/Max-Table/32} [(n1,s1):1, next=2, gen=0]
I200206 15:29:59.698146 281 storage/replica_raft.go:248  [n1,s1,r27/1:/{NamespaceTab…-Table/32}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200206 15:29:59.704735 2056 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r27/1:/{NamespaceTab…-Table/32}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200206 15:29:59.705349 281 storage/store_snapshot.go:922  [n1,replicate,s1,r27/1:/{NamespaceTab…-Table/32}] sending LEARNER snapshot cce879cc at applied index 16
I200206 15:29:59.705521 281 storage/store_snapshot.go:965  [n1,replicate,s1,r27/1:/{NamespaceTab…-Table/32}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:29:59.706214 2038 storage/replica_raftstorage.go:780  [n4,s4,r27/2:{-}] applying LEARNER snapshot [id=cce879cc index=16]
I200206 15:29:59.708455 1888 server/server_update.go:55  [n4] no need to upgrade, cluster already at the newest version
I200206 15:29:59.709070 2038 storage/replica_raftstorage.go:801  [n4,s4,r27/2:/{NamespaceTab…-Table/32}] applied LEARNER snapshot [total=3ms ingestion=4@2ms id=cce879cc index=16]
I200206 15:29:59.709888 281 storage/replica_command.go:1601  [n1,replicate,s1,r27/1:/{NamespaceTab…-Table/32}] change replicas (add [(n4,s4):2] remove []): existing descriptor r27:/{NamespaceTable/Max-Table/32} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200206 15:29:59.712891 281 storage/replica_raft.go:248  [n1,s1,r27/1:/{NamespaceTab…-Table/32}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200206 15:29:59.713331 281 storage/replica_command.go:1601  [n1,replicate,s1,r27/1:/{NamespaceTab…-Table/32}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r27:/{NamespaceTable/Max-Table/32} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200206 15:29:59.724292 281 storage/replica_raft.go:248  [n1,s1,r27/1:/{NamespaceTab…-Table/32}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3LEARNER] next=4
I200206 15:29:59.726530 2069 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r27/1:/{NamespaceTab…-Table/32}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):3LEARNER
I200206 15:29:59.726676 281 storage/store_snapshot.go:922  [n1,replicate,s1,r27/1:/{NamespaceTab…-Table/32}] sending LEARNER snapshot ac581eb3 at applied index 20
I200206 15:29:59.726828 281 storage/store_snapshot.go:965  [n1,replicate,s1,r27/1:/{NamespaceTab…-Table/32}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:29:59.727407 2099 storage/replica_raftstorage.go:780  [n2,s2,r27/3:{-}] applying LEARNER snapshot [id=ac581eb3 index=20]
I200206 15:29:59.733669 2099 storage/replica_raftstorage.go:801  [n2,s2,r27/3:/{NamespaceTab…-Table/32}] applied LEARNER snapshot [total=6ms ingestion=4@6ms id=ac581eb3 index=20]
I200206 15:29:59.734221 281 storage/replica_command.go:1601  [n1,replicate,s1,r27/1:/{NamespaceTab…-Table/32}] change replicas (add [(n2,s2):3] remove []): existing descriptor r27:/{NamespaceTable/Max-Table/32} [(n1,s1):1, (n4,s4):2, (n2,s2):3LEARNER, next=4, gen=3]
I200206 15:29:59.742620 281 storage/replica_raft.go:248  [n1,s1,r27/1:/{NamespaceTab…-Table/32}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3] next=4
I200206 15:29:59.747236 281 storage/replica_command.go:1601  [n1,replicate,s1,r8/1:/Table/1{2-3}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r8:/Table/1{2-3} [(n1,s1):1, next=2, gen=0]
I200206 15:29:59.751304 281 storage/replica_raft.go:248  [n1,s1,r8/1:/Table/1{2-3}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200206 15:29:59.752767 281 storage/store_snapshot.go:922  [n1,replicate,s1,r8/1:/Table/1{2-3}] sending LEARNER snapshot fb33e47a at applied index 40
I200206 15:29:59.752988 281 storage/store_snapshot.go:965  [n1,replicate,s1,r8/1:/Table/1{2-3}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 51, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:29:59.753025 2090 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r8/1:/Table/1{2-3}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200206 15:29:59.753649 2088 storage/replica_raftstorage.go:780  [n4,s4,r8/2:{-}] applying LEARNER snapshot [id=fb33e47a index=40]
I200206 15:29:59.762159 2088 storage/replica_raftstorage.go:801  [n4,s4,r8/2:/Table/1{2-3}] applied LEARNER snapshot [total=7ms ingestion=4@6ms id=fb33e47a index=40]
I200206 15:29:59.762817 281 storage/replica_command.go:1601  [n1,replicate,s1,r8/1:/Table/1{2-3}] change replicas (add [(n4,s4):2] remove []): existing descriptor r8:/Table/1{2-3} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200206 15:29:59.765834 281 storage/replica_raft.go:248  [n1,s1,r8/1:/Table/1{2-3}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200206 15:29:59.766299 281 storage/replica_command.go:1601  [n1,replicate,s1,r8/1:/Table/1{2-3}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r8:/Table/1{2-3} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200206 15:29:59.778255 281 storage/replica_raft.go:248  [n1,s1,r8/1:/Table/1{2-3}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3LEARNER] next=4
I200206 15:29:59.780652 1952 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r8/1:/Table/1{2-3}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200206 15:29:59.780789 281 storage/store_snapshot.go:922  [n1,replicate,s1,r8/1:/Table/1{2-3}] sending LEARNER snapshot 8f0cd685 at applied index 44
I200206 15:29:59.781065 281 storage/store_snapshot.go:965  [n1,replicate,s1,r8/1:/Table/1{2-3}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 55, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:29:59.781834 2160 storage/replica_raftstorage.go:780  [n3,s3,r8/3:{-}] applying LEARNER snapshot [id=8f0cd685 index=44]
I200206 15:29:59.816928 2160 storage/replica_raftstorage.go:801  [n3,s3,r8/3:/Table/1{2-3}] applied LEARNER snapshot [total=35ms ingestion=4@5ms id=8f0cd685 index=44]
I200206 15:29:59.822347 281 storage/replica_command.go:1601  [n1,replicate,s1,r8/1:/Table/1{2-3}] change replicas (add [(n3,s3):3] remove []): existing descriptor r8:/Table/1{2-3} [(n1,s1):1, (n4,s4):2, (n3,s3):3LEARNER, next=4, gen=3]
I200206 15:29:59.824847 281 storage/replica_raft.go:248  [n1,s1,r8/1:/Table/1{2-3}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3] next=4
I200206 15:29:59.827086 281 storage/replica_command.go:1601  [n1,replicate,s1,r7/1:/Table/1{1-2}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r7:/Table/1{1-2} [(n1,s1):1, next=2, gen=0]
I200206 15:29:59.829742 281 storage/replica_raft.go:248  [n1,s1,r7/1:/Table/1{1-2}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200206 15:29:59.831090 281 storage/store_snapshot.go:922  [n1,replicate,s1,r7/1:/Table/1{1-2}] sending LEARNER snapshot 912d9205 at applied index 19
I200206 15:29:59.831183 281 storage/store_snapshot.go:965  [n1,replicate,s1,r7/1:/Table/1{1-2}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 7, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:29:59.831208 2261 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r7/1:/Table/1{1-2}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):2LEARNER
I200206 15:29:59.831683 2259 storage/replica_raftstorage.go:780  [n2,s2,r7/2:{-}] applying LEARNER snapshot [id=912d9205 index=19]
I200206 15:29:59.834190 2259 storage/replica_raftstorage.go:801  [n2,s2,r7/2:/Table/1{1-2}] applied LEARNER snapshot [total=2ms ingestion=4@2ms id=912d9205 index=19]
I200206 15:29:59.836665 281 storage/replica_command.go:1601  [n1,replicate,s1,r7/1:/Table/1{1-2}] change replicas (add [(n2,s2):2] remove []): existing descriptor r7:/Table/1{1-2} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
I200206 15:29:59.843014 281 storage/replica_raft.go:248  [n1,s1,r7/1:/Table/1{1-2}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200206 15:29:59.843629 281 storage/replica_command.go:1601  [n1,replicate,s1,r7/1:/Table/1{1-2}] change replicas (add [(n4,s4):3LEARNER] remove []): existing descriptor r7:/Table/1{1-2} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200206 15:29:59.858345 281 storage/replica_raft.go:248  [n1,s1,r7/1:/Table/1{1-2}] proposing SIMPLE(l3) ADD_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3LEARNER] next=4
I200206 15:29:59.860732 2275 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r7/1:/Table/1{1-2}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):3LEARNER
I200206 15:29:59.860788 281 storage/store_snapshot.go:922  [n1,replicate,s1,r7/1:/Table/1{1-2}] sending LEARNER snapshot 85d8b9ac at applied index 23
I200206 15:29:59.860914 281 storage/store_snapshot.go:965  [n1,replicate,s1,r7/1:/Table/1{1-2}] streamed snapshot to (n4,s4):3LEARNER: kv pairs: 11, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:29:59.861446 2265 storage/replica_raftstorage.go:780  [n4,s4,r7/3:{-}] applying LEARNER snapshot [id=85d8b9ac index=23]
I200206 15:29:59.865016 2265 storage/replica_raftstorage.go:801  [n4,s4,r7/3:/Table/1{1-2}] applied LEARNER snapshot [total=4ms ingestion=4@3ms id=85d8b9ac index=23]
I200206 15:29:59.865511 281 storage/replica_command.go:1601  [n1,replicate,s1,r7/1:/Table/1{1-2}] change replicas (add [(n4,s4):3] remove []): existing descriptor r7:/Table/1{1-2} [(n1,s1):1, (n2,s2):2, (n4,s4):3LEARNER, next=4, gen=3]
I200206 15:29:59.868005 281 storage/replica_raft.go:248  [n1,s1,r7/1:/Table/1{1-2}] proposing SIMPLE(v3) ADD_REPLICA[(n4,s4):3]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3] next=4
I200206 15:29:59.869821 281 storage/queue.go:1135  [n1,replicate] purgatory is now empty
I200206 15:29:59.870208 2325 storage/replica_command.go:1601  [n1,replicate,s1,r9/1:/Table/1{3-4}] change replicas (add [(n3,s3):2LEARNER] remove []): existing descriptor r9:/Table/1{3-4} [(n1,s1):1, next=2, gen=0]
I200206 15:29:59.872951 2325 storage/replica_raft.go:248  [n1,s1,r9/1:/Table/1{3-4}] proposing SIMPLE(l2) ADD_REPLICA[(n3,s3):2LEARNER]: after=[(n1,s1):1 (n3,s3):2LEARNER] next=3
I200206 15:29:59.876461 2325 storage/store_snapshot.go:922  [n1,replicate,s1,r9/1:/Table/1{3-4}] sending LEARNER snapshot d243cb98 at applied index 55
I200206 15:29:59.876872 2325 storage/store_snapshot.go:965  [n1,replicate,s1,r9/1:/Table/1{3-4}] streamed snapshot to (n3,s3):2LEARNER: kv pairs: 116, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:29:59.877452 2329 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r9/1:/Table/1{3-4}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):2LEARNER
I200206 15:29:59.877621 2216 storage/replica_raftstorage.go:780  [n3,s3,r9/2:{-}] applying LEARNER snapshot [id=d243cb98 index=55]
I200206 15:29:59.880730 2216 storage/replica_raftstorage.go:801  [n3,s3,r9/2:/Table/1{3-4}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=d243cb98 index=55]
I200206 15:29:59.881336 2325 storage/replica_command.go:1601  [n1,replicate,s1,r9/1:/Table/1{3-4}] change replicas (add [(n3,s3):2] remove []): existing descriptor r9:/Table/1{3-4} [(n1,s1):1, (n3,s3):2LEARNER, next=3, gen=1]
I200206 15:29:59.885398 2325 storage/replica_raft.go:248  [n1,s1,r9/1:/Table/1{3-4}] proposing SIMPLE(v2) ADD_REPLICA[(n3,s3):2]: after=[(n1,s1):1 (n3,s3):2] next=3
I200206 15:29:59.885949 2325 storage/replica_command.go:1601  [n1,replicate,s1,r9/1:/Table/1{3-4}] change replicas (add [(n4,s4):3LEARNER] remove []): existing descriptor r9:/Table/1{3-4} [(n1,s1):1, (n3,s3):2, next=3, gen=2]
I200206 15:29:59.894891 2325 storage/replica_raft.go:248  [n1,s1,r9/1:/Table/1{3-4}] proposing SIMPLE(l3) ADD_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3LEARNER] next=4
I200206 15:29:59.899479 2325 storage/store_snapshot.go:922  [n1,replicate,s1,r9/1:/Table/1{3-4}] sending LEARNER snapshot ad510d90 at applied index 62
I200206 15:29:59.899830 2325 storage/store_snapshot.go:965  [n1,replicate,s1,r9/1:/Table/1{3-4}] streamed snapshot to (n4,s4):3LEARNER: kv pairs: 125, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:29:59.900796 2336 storage/replica_raftstorage.go:780  [n4,s4,r9/3:{-}] applying LEARNER snapshot [id=ad510d90 index=62]
I200206 15:29:59.902171 2336 storage/replica_raftstorage.go:801  [n4,s4,r9/3:/Table/1{3-4}] applied LEARNER snapshot [total=1ms ingestion=4@1ms id=ad510d90 index=62]
I200206 15:29:59.902597 2325 storage/replica_command.go:1601  [n1,replicate,s1,r9/1:/Table/1{3-4}] change replicas (add [(n4,s4):3] remove []): existing descriptor r9:/Table/1{3-4} [(n1,s1):1, (n3,s3):2, (n4,s4):3LEARNER, next=4, gen=3]
I200206 15:29:59.906511 2325 storage/replica_raft.go:248  [n1,s1,r9/1:/Table/1{3-4}] proposing SIMPLE(v3) ADD_REPLICA[(n4,s4):3]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3] next=4
I200206 15:29:59.913474 12 storage/replica_command.go:1601  [n1,replicate,s1,r25/1:/{Table/29-NamespaceTab…}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r25:/{Table/29-NamespaceTable/30} [(n1,s1):1, next=2, gen=0]
I200206 15:29:59.968284 12 storage/replica_raft.go:248  [n1,s1,r25/1:/{Table/29-NamespaceTab…}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200206 15:29:59.969787 12 storage/store_snapshot.go:922  [n1,replicate,s1,r25/1:/{Table/29-NamespaceTab…}] sending LEARNER snapshot 57cfde0f at applied index 15
I200206 15:29:59.969905 12 storage/store_snapshot.go:965  [n1,replicate,s1,r25/1:/{Table/29-NamespaceTab…}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:29:59.969945 2349 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r25/1:/{Table/29-NamespaceTab…}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200206 15:29:59.970438 2347 storage/replica_raftstorage.go:780  [n4,s4,r25/2:{-}] applying LEARNER snapshot [id=57cfde0f index=15]
I200206 15:29:59.973246 2347 storage/replica_raftstorage.go:801  [n4,s4,r25/2:/{Table/29-NamespaceTab…}] applied LEARNER snapshot [total=3ms ingestion=4@2ms id=57cfde0f index=15]
I200206 15:29:59.979862 12 storage/replica_command.go:1601  [n1,replicate,s1,r25/1:/{Table/29-NamespaceTab…}] change replicas (add [(n4,s4):2] remove []): existing descriptor r25:/{Table/29-NamespaceTable/30} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200206 15:29:59.983524 12 storage/replica_raft.go:248  [n1,s1,r25/1:/{Table/29-NamespaceTab…}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200206 15:29:59.984026 12 storage/replica_command.go:1601  [n1,replicate,s1,r25/1:/{Table/29-NamespaceTab…}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r25:/{Table/29-NamespaceTable/30} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200206 15:29:59.992915 12 storage/replica_raft.go:248  [n1,s1,r25/1:/{Table/29-NamespaceTab…}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3LEARNER] next=4
I200206 15:29:59.994548 12 storage/store_snapshot.go:922  [n1,replicate,s1,r25/1:/{Table/29-NamespaceTab…}] sending LEARNER snapshot 684943f9 at applied index 19
I200206 15:29:59.994683 12 storage/store_snapshot.go:965  [n1,replicate,s1,r25/1:/{Table/29-NamespaceTab…}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:29:59.994845 2436 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r25/1:/{Table/29-NamespaceTab…}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):3LEARNER
I200206 15:29:59.996097 2421 storage/replica_raftstorage.go:780  [n2,s2,r25/3:{-}] applying LEARNER snapshot [id=684943f9 index=19]
I200206 15:30:00.003141 2421 storage/replica_raftstorage.go:801  [n2,s2,r25/3:/{Table/29-NamespaceTab…}] applied LEARNER snapshot [total=7ms ingestion=4@6ms id=684943f9 index=19]
I200206 15:30:00.003551 12 storage/replica_command.go:1601  [n1,replicate,s1,r25/1:/{Table/29-NamespaceTab…}] change replicas (add [(n2,s2):3] remove []): existing descriptor r25:/{Table/29-NamespaceTable/30} [(n1,s1):1, (n4,s4):2, (n2,s2):3LEARNER, next=4, gen=3]
I200206 15:30:00.006950 12 storage/replica_raft.go:248  [n1,s1,r25/1:/{Table/29-NamespaceTab…}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3] next=4
I200206 15:30:00.009568 12 storage/replica_command.go:1601  [n1,replicate,s1,r10/1:/Table/1{4-5}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r10:/Table/1{4-5} [(n1,s1):1, next=2, gen=0]
I200206 15:30:00.013517 12 storage/replica_raft.go:248  [n1,s1,r10/1:/Table/1{4-5}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200206 15:30:00.015003 12 storage/store_snapshot.go:922  [n1,replicate,s1,r10/1:/Table/1{4-5}] sending LEARNER snapshot b1c98b00 at applied index 15
I200206 15:30:00.015106 12 storage/store_snapshot.go:965  [n1,replicate,s1,r10/1:/Table/1{4-5}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:00.015276 2423 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r10/1:/Table/1{4-5}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):2LEARNER
I200206 15:30:00.015748 2225 storage/replica_raftstorage.go:780  [n2,s2,r10/2:{-}] applying LEARNER snapshot [id=b1c98b00 index=15]
I200206 15:30:00.018173 2225 storage/replica_raftstorage.go:801  [n2,s2,r10/2:/Table/1{4-5}] applied LEARNER snapshot [total=2ms ingestion=4@2ms id=b1c98b00 index=15]
I200206 15:30:00.018624 12 storage/replica_command.go:1601  [n1,replicate,s1,r10/1:/Table/1{4-5}] change replicas (add [(n2,s2):2] remove []): existing descriptor r10:/Table/1{4-5} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
I200206 15:30:00.024560 12 storage/replica_raft.go:248  [n1,s1,r10/1:/Table/1{4-5}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200206 15:30:00.025041 12 storage/replica_command.go:1601  [n1,replicate,s1,r10/1:/Table/1{4-5}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r10:/Table/1{4-5} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200206 15:30:00.039047 12 storage/replica_raft.go:248  [n1,s1,r10/1:/Table/1{4-5}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3LEARNER] next=4
I200206 15:30:00.041789 12 storage/store_snapshot.go:922  [n1,replicate,s1,r10/1:/Table/1{4-5}] sending LEARNER snapshot 3b30eafd at applied index 19
I200206 15:30:00.041921 12 storage/store_snapshot.go:965  [n1,replicate,s1,r10/1:/Table/1{4-5}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:00.042092 2501 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r10/1:/Table/1{4-5}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200206 15:30:00.042701 2499 storage/replica_raftstorage.go:780  [n3,s3,r10/3:{-}] applying LEARNER snapshot [id=3b30eafd index=19]
I200206 15:30:00.046371 2499 storage/replica_raftstorage.go:801  [n3,s3,r10/3:/Table/1{4-5}] applied LEARNER snapshot [total=4ms ingestion=4@3ms id=3b30eafd index=19]
I200206 15:30:00.047764 12 storage/replica_command.go:1601  [n1,replicate,s1,r10/1:/Table/1{4-5}] change replicas (add [(n3,s3):3] remove []): existing descriptor r10:/Table/1{4-5} [(n1,s1):1, (n2,s2):2, (n3,s3):3LEARNER, next=4, gen=3]
I200206 15:30:00.054183 12 storage/replica_raft.go:248  [n1,s1,r10/1:/Table/1{4-5}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3] next=4
I200206 15:30:00.056631 12 storage/replica_command.go:1601  [n1,replicate,s1,r28/1:/{Table/32-Max}] change replicas (add [(n3,s3):2LEARNER] remove []): existing descriptor r28:/{Table/32-Max} [(n1,s1):1, next=2, gen=0]
I200206 15:30:00.060492 12 storage/replica_raft.go:248  [n1,s1,r28/1:/{Table/32-Max}] proposing SIMPLE(l2) ADD_REPLICA[(n3,s3):2LEARNER]: after=[(n1,s1):1 (n3,s3):2LEARNER] next=3
I200206 15:30:00.062748 12 storage/store_snapshot.go:922  [n1,replicate,s1,r28/1:/{Table/32-Max}] sending LEARNER snapshot 59f9379e at applied index 15
I200206 15:30:00.062855 12 storage/store_snapshot.go:965  [n1,replicate,s1,r28/1:/{Table/32-Max}] streamed snapshot to (n3,s3):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:00.062861 2535 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r28/1:/{Table/32-Max}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):2LEARNER
I200206 15:30:00.063373 2533 storage/replica_raftstorage.go:780  [n3,s3,r28/2:{-}] applying LEARNER snapshot [id=59f9379e index=15]
I200206 15:30:00.065968 2533 storage/replica_raftstorage.go:801  [n3,s3,r28/2:/{Table/32-Max}] applied LEARNER snapshot [total=3ms ingestion=4@2ms id=59f9379e index=15]
I200206 15:30:00.067294 12 storage/replica_command.go:1601  [n1,replicate,s1,r28/1:/{Table/32-Max}] change replicas (add [(n3,s3):2] remove []): existing descriptor r28:/{Table/32-Max} [(n1,s1):1, (n3,s3):2LEARNER, next=3, gen=1]
I200206 15:30:00.076644 12 storage/replica_raft.go:248  [n1,s1,r28/1:/{Table/32-Max}] proposing SIMPLE(v2) ADD_REPLICA[(n3,s3):2]: after=[(n1,s1):1 (n3,s3):2] next=3
I200206 15:30:00.077262 12 storage/replica_command.go:1601  [n1,replicate,s1,r28/1:/{Table/32-Max}] change replicas (add [(n4,s4):3LEARNER] remove []): existing descriptor r28:/{Table/32-Max} [(n1,s1):1, (n3,s3):2, next=3, gen=2]
I200206 15:30:00.089506 12 storage/replica_raft.go:248  [n1,s1,r28/1:/{Table/32-Max}] proposing SIMPLE(l3) ADD_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3LEARNER] next=4
I200206 15:30:00.096675 2597 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r28/1:/{Table/32-Max}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):3LEARNER
I200206 15:30:00.096997 12 storage/store_snapshot.go:922  [n1,replicate,s1,r28/1:/{Table/32-Max}] sending LEARNER snapshot fa8386ab at applied index 20
I200206 15:30:00.097123 12 storage/store_snapshot.go:965  [n1,replicate,s1,r28/1:/{Table/32-Max}] streamed snapshot to (n4,s4):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:00.097726 2599 storage/replica_raftstorage.go:780  [n4,s4,r28/3:{-}] applying LEARNER snapshot [id=fa8386ab index=20]
I200206 15:30:00.108011 2599 storage/replica_raftstorage.go:801  [n4,s4,r28/3:/{Table/32-Max}] applied LEARNER snapshot [total=10ms ingestion=4@10ms id=fa8386ab index=20]
I200206 15:30:00.108611 12 storage/replica_command.go:1601  [n1,replicate,s1,r28/1:/{Table/32-Max}] change replicas (add [(n4,s4):3] remove []): existing descriptor r28:/{Table/32-Max} [(n1,s1):1, (n3,s3):2, (n4,s4):3LEARNER, next=4, gen=3]
I200206 15:30:00.112020 12 storage/replica_raft.go:248  [n1,s1,r28/1:/{Table/32-Max}] proposing SIMPLE(v3) ADD_REPLICA[(n4,s4):3]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3] next=4
I200206 15:30:00.114763 12 storage/replica_command.go:1601  [n1,replicate,s1,r23/1:/Table/2{7-8}] change replicas (add [(n3,s3):2LEARNER] remove []): existing descriptor r23:/Table/2{7-8} [(n1,s1):1, next=2, gen=0]
I200206 15:30:00.120795 12 storage/replica_raft.go:248  [n1,s1,r23/1:/Table/2{7-8}] proposing SIMPLE(l2) ADD_REPLICA[(n3,s3):2LEARNER]: after=[(n1,s1):1 (n3,s3):2LEARNER] next=3
I200206 15:30:00.122493 12 storage/store_snapshot.go:922  [n1,replicate,s1,r23/1:/Table/2{7-8}] sending LEARNER snapshot f4df5488 at applied index 15
I200206 15:30:00.122593 12 storage/store_snapshot.go:965  [n1,replicate,s1,r23/1:/Table/2{7-8}] streamed snapshot to (n3,s3):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:00.123244 2539 storage/replica_raftstorage.go:780  [n3,s3,r23/2:{-}] applying LEARNER snapshot [id=f4df5488 index=15]
I200206 15:30:00.125199 2589 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r23/1:/Table/2{7-8}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):2LEARNER
I200206 15:30:00.135557 2539 storage/replica_raftstorage.go:801  [n3,s3,r23/2:/Table/2{7-8}] applied LEARNER snapshot [total=12ms ingestion=4@12ms id=f4df5488 index=15]
I200206 15:30:00.137333 12 storage/replica_command.go:1601  [n1,replicate,s1,r23/1:/Table/2{7-8}] change replicas (add [(n3,s3):2] remove []): existing descriptor r23:/Table/2{7-8} [(n1,s1):1, (n3,s3):2LEARNER, next=3, gen=1]
I200206 15:30:00.141797 12 storage/replica_raft.go:248  [n1,s1,r23/1:/Table/2{7-8}] proposing SIMPLE(v2) ADD_REPLICA[(n3,s3):2]: after=[(n1,s1):1 (n3,s3):2] next=3
I200206 15:30:00.142395 12 storage/replica_command.go:1601  [n1,replicate,s1,r23/1:/Table/2{7-8}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r23:/Table/2{7-8} [(n1,s1):1, (n3,s3):2, next=3, gen=2]
I200206 15:30:00.152820 12 storage/replica_raft.go:248  [n1,s1,r23/1:/Table/2{7-8}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n2,s2):3LEARNER] next=4
I200206 15:30:00.155008 12 storage/store_snapshot.go:922  [n1,replicate,s1,r23/1:/Table/2{7-8}] sending LEARNER snapshot 901ef7af at applied index 19
I200206 15:30:00.155167 12 storage/store_snapshot.go:965  [n1,replicate,s1,r23/1:/Table/2{7-8}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:00.155377 2650 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r23/1:/Table/2{7-8}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):3LEARNER
I200206 15:30:00.155860 2632 storage/replica_raftstorage.go:780  [n2,s2,r23/3:{-}] applying LEARNER snapshot [id=901ef7af index=19]
I200206 15:30:00.158506 2632 storage/replica_raftstorage.go:801  [n2,s2,r23/3:/Table/2{7-8}] applied LEARNER snapshot [total=3ms ingestion=4@2ms id=901ef7af index=19]
I200206 15:30:00.159598 12 storage/replica_command.go:1601  [n1,replicate,s1,r23/1:/Table/2{7-8}] change replicas (add [(n2,s2):3] remove []): existing descriptor r23:/Table/2{7-8} [(n1,s1):1, (n3,s3):2, (n2,s2):3LEARNER, next=4, gen=3]
I200206 15:30:00.173958 12 storage/replica_raft.go:248  [n1,s1,r23/1:/Table/2{7-8}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n3,s3):2 (n2,s2):3] next=4
I200206 15:30:00.178227 12 storage/replica_command.go:1601  [n1,replicate,s1,r12/1:/Table/1{6-7}] change replicas (add [(n3,s3):2LEARNER] remove []): existing descriptor r12:/Table/1{6-7} [(n1,s1):1, next=2, gen=0]
I200206 15:30:00.185247 12 storage/replica_raft.go:248  [n1,s1,r12/1:/Table/1{6-7}] proposing SIMPLE(l2) ADD_REPLICA[(n3,s3):2LEARNER]: after=[(n1,s1):1 (n3,s3):2LEARNER] next=3
I200206 15:30:00.190776 2674 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r12/1:/Table/1{6-7}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):2LEARNER
I200206 15:30:00.190822 12 storage/store_snapshot.go:922  [n1,replicate,s1,r12/1:/Table/1{6-7}] sending LEARNER snapshot 228b43e8 at applied index 15
I200206 15:30:00.190918 12 storage/store_snapshot.go:965  [n1,replicate,s1,r12/1:/Table/1{6-7}] streamed snapshot to (n3,s3):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.01s
I200206 15:30:00.191479 2728 storage/replica_raftstorage.go:780  [n3,s3,r12/2:{-}] applying LEARNER snapshot [id=228b43e8 index=15]
I200206 15:30:00.199239 2728 storage/replica_raftstorage.go:801  [n3,s3,r12/2:/Table/1{6-7}] applied LEARNER snapshot [total=8ms ingestion=4@7ms id=228b43e8 index=15]
I200206 15:30:00.200433 12 storage/replica_command.go:1601  [n1,replicate,s1,r12/1:/Table/1{6-7}] change replicas (add [(n3,s3):2] remove []): existing descriptor r12:/Table/1{6-7} [(n1,s1):1, (n3,s3):2LEARNER, next=3, gen=1]
I200206 15:30:00.205023 12 storage/replica_raft.go:248  [n1,s1,r12/1:/Table/1{6-7}] proposing SIMPLE(v2) ADD_REPLICA[(n3,s3):2]: after=[(n1,s1):1 (n3,s3):2] next=3
I200206 15:30:00.205484 12 storage/replica_command.go:1601  [n1,replicate,s1,r12/1:/Table/1{6-7}] change replicas (add [(n4,s4):3LEARNER] remove []): existing descriptor r12:/Table/1{6-7} [(n1,s1):1, (n3,s3):2, next=3, gen=2]
I200206 15:30:00.219046 12 storage/replica_raft.go:248  [n1,s1,r12/1:/Table/1{6-7}] proposing SIMPLE(l3) ADD_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3LEARNER] next=4
I200206 15:30:00.221077 12 storage/store_snapshot.go:922  [n1,replicate,s1,r12/1:/Table/1{6-7}] sending LEARNER snapshot cd15f4e7 at applied index 19
I200206 15:30:00.221180 12 storage/store_snapshot.go:965  [n1,replicate,s1,r12/1:/Table/1{6-7}] streamed snapshot to (n4,s4):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:00.221341 2769 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r12/1:/Table/1{6-7}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):3LEARNER
I200206 15:30:00.222014 2767 storage/replica_raftstorage.go:780  [n4,s4,r12/3:{-}] applying LEARNER snapshot [id=cd15f4e7 index=19]
I200206 15:30:00.226231 2767 storage/replica_raftstorage.go:801  [n4,s4,r12/3:/Table/1{6-7}] applied LEARNER snapshot [total=4ms ingestion=4@4ms id=cd15f4e7 index=19]
I200206 15:30:00.226798 12 storage/replica_command.go:1601  [n1,replicate,s1,r12/1:/Table/1{6-7}] change replicas (add [(n4,s4):3] remove []): existing descriptor r12:/Table/1{6-7} [(n1,s1):1, (n3,s3):2, (n4,s4):3LEARNER, next=4, gen=3]
I200206 15:30:00.233559 12 storage/replica_raft.go:248  [n1,s1,r12/1:/Table/1{6-7}] proposing SIMPLE(v3) ADD_REPLICA[(n4,s4):3]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3] next=4
I200206 15:30:00.238684 12 storage/replica_command.go:1601  [n1,replicate,s1,r26/1:/NamespaceTable/{30-Max}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r26:/NamespaceTable/{30-Max} [(n1,s1):1, next=2, gen=0]
I200206 15:30:00.248139 12 storage/replica_raft.go:248  [n1,s1,r26/1:/NamespaceTable/{30-Max}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200206 15:30:00.250949 12 storage/store_snapshot.go:922  [n1,replicate,s1,r26/1:/NamespaceTable/{30-Max}] sending LEARNER snapshot 7ea02195 at applied index 22
I200206 15:30:00.250984 2807 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r26/1:/NamespaceTable/{30-Max}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):2LEARNER
I200206 15:30:00.251304 12 storage/store_snapshot.go:965  [n1,replicate,s1,r26/1:/NamespaceTable/{30-Max}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 34, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:00.257029 2706 storage/replica_raftstorage.go:780  [n2,s2,r26/2:{-}] applying LEARNER snapshot [id=7ea02195 index=22]
I200206 15:30:00.260147 2706 storage/replica_raftstorage.go:801  [n2,s2,r26/2:/NamespaceTable/{30-Max}] applied LEARNER snapshot [total=3ms ingestion=4@2ms id=7ea02195 index=22]
I200206 15:30:00.261535 12 storage/replica_command.go:1601  [n1,replicate,s1,r26/1:/NamespaceTable/{30-Max}] change replicas (add [(n2,s2):2] remove []): existing descriptor r26:/NamespaceTable/{30-Max} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
I200206 15:30:00.287968 12 storage/replica_raft.go:248  [n1,s1,r26/1:/NamespaceTable/{30-Max}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200206 15:30:00.291136 12 storage/replica_command.go:1601  [n1,replicate,s1,r26/1:/NamespaceTable/{30-Max}] change replicas (add [(n4,s4):3LEARNER] remove []): existing descriptor r26:/NamespaceTable/{30-Max} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200206 15:30:00.304615 12 storage/replica_raft.go:248  [n1,s1,r26/1:/NamespaceTable/{30-Max}] proposing SIMPLE(l3) ADD_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3LEARNER] next=4
I200206 15:30:00.307035 12 storage/store_snapshot.go:922  [n1,replicate,s1,r26/1:/NamespaceTable/{30-Max}] sending LEARNER snapshot 8134341c at applied index 26
I200206 15:30:00.307187 12 storage/store_snapshot.go:965  [n1,replicate,s1,r26/1:/NamespaceTable/{30-Max}] streamed snapshot to (n4,s4):3LEARNER: kv pairs: 38, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:00.307231 2716 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r26/1:/NamespaceTable/{30-Max}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):3LEARNER
I200206 15:30:00.307863 2810 storage/replica_raftstorage.go:780  [n4,s4,r26/3:{-}] applying LEARNER snapshot [id=8134341c index=26]
I200206 15:30:00.318936 2810 storage/replica_raftstorage.go:801  [n4,s4,r26/3:/NamespaceTable/{30-Max}] applied LEARNER snapshot [total=11ms ingestion=4@11ms id=8134341c index=26]
I200206 15:30:00.319468 12 storage/replica_command.go:1601  [n1,replicate,s1,r26/1:/NamespaceTable/{30-Max}] change replicas (add [(n4,s4):3] remove []): existing descriptor r26:/NamespaceTable/{30-Max} [(n1,s1):1, (n2,s2):2, (n4,s4):3LEARNER, next=4, gen=3]
I200206 15:30:00.325311 12 storage/replica_raft.go:248  [n1,s1,r26/1:/NamespaceTable/{30-Max}] proposing SIMPLE(v3) ADD_REPLICA[(n4,s4):3]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3] next=4
I200206 15:30:00.328064 12 storage/replica_command.go:1601  [n1,replicate,s1,r3/1:/System/{NodeLive…-tsd}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r3:/System/{NodeLivenessMax-tsd} [(n1,s1):1, next=2, gen=0]
I200206 15:30:00.408804 12 storage/replica_raft.go:248  [n1,s1,r3/1:/System/{NodeLive…-tsd}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200206 15:30:00.410185 12 storage/store_snapshot.go:922  [n1,replicate,s1,r3/1:/System/{NodeLive…-tsd}] sending LEARNER snapshot 27099eed at applied index 46
I200206 15:30:00.410299 2720 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r3/1:/System/{NodeLive…-tsd}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):2LEARNER
I200206 15:30:00.410810 12 storage/store_snapshot.go:965  [n1,replicate,s1,r3/1:/System/{NodeLive…-tsd}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 36, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:00.412095 2718 storage/replica_raftstorage.go:780  [n2,s2,r3/2:{-}] applying LEARNER snapshot [id=27099eed index=46]
I200206 15:30:00.416021 2718 storage/replica_raftstorage.go:801  [n2,s2,r3/2:/System/{NodeLive…-tsd}] applied LEARNER snapshot [total=4ms ingestion=4@4ms id=27099eed index=46]
I200206 15:30:00.418928 12 storage/replica_command.go:1601  [n1,replicate,s1,r3/1:/System/{NodeLive…-tsd}] change replicas (add [(n2,s2):2] remove []): existing descriptor r3:/System/{NodeLivenessMax-tsd} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
I200206 15:30:00.432461 12 storage/replica_raft.go:248  [n1,s1,r3/1:/System/{NodeLive…-tsd}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200206 15:30:00.432971 12 storage/replica_command.go:1601  [n1,replicate,s1,r3/1:/System/{NodeLive…-tsd}] change replicas (add [(n4,s4):3LEARNER] remove []): existing descriptor r3:/System/{NodeLivenessMax-tsd} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200206 15:30:00.443489 12 storage/replica_raft.go:248  [n1,s1,r3/1:/System/{NodeLive…-tsd}] proposing SIMPLE(l3) ADD_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3LEARNER] next=4
I200206 15:30:00.445575 12 storage/store_snapshot.go:922  [n1,replicate,s1,r3/1:/System/{NodeLive…-tsd}] sending LEARNER snapshot 0f58d8b7 at applied index 50
I200206 15:30:00.446213 12 storage/store_snapshot.go:965  [n1,replicate,s1,r3/1:/System/{NodeLive…-tsd}] streamed snapshot to (n4,s4):3LEARNER: kv pairs: 40, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:00.446376 2921 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r3/1:/System/{NodeLive…-tsd}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):3LEARNER
I200206 15:30:00.447767 2919 storage/replica_raftstorage.go:780  [n4,s4,r3/3:{-}] applying LEARNER snapshot [id=0f58d8b7 index=50]
I200206 15:30:00.452243 2919 storage/replica_raftstorage.go:801  [n4,s4,r3/3:/System/{NodeLive…-tsd}] applied LEARNER snapshot [total=4ms ingestion=4@4ms id=0f58d8b7 index=50]
I200206 15:30:00.452864 12 storage/replica_command.go:1601  [n1,replicate,s1,r3/1:/System/{NodeLive…-tsd}] change replicas (add [(n4,s4):3] remove []): existing descriptor r3:/System/{NodeLivenessMax-tsd} [(n1,s1):1, (n2,s2):2, (n4,s4):3LEARNER, next=4, gen=3]
I200206 15:30:00.463581 12 storage/replica_raft.go:248  [n1,s1,r3/1:/System/{NodeLive…-tsd}] proposing SIMPLE(v3) ADD_REPLICA[(n4,s4):3]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3] next=4
I200206 15:30:00.465862 12 storage/replica_command.go:1601  [n1,replicate,s1,r24/1:/Table/2{8-9}] change replicas (add [(n3,s3):2LEARNER] remove []): existing descriptor r24:/Table/2{8-9} [(n1,s1):1, next=2, gen=0]
I200206 15:30:00.481564 12 storage/replica_raft.go:248  [n1,s1,r24/1:/Table/2{8-9}] proposing SIMPLE(l2) ADD_REPLICA[(n3,s3):2LEARNER]: after=[(n1,s1):1 (n3,s3):2LEARNER] next=3
I200206 15:30:00.486825 12 storage/store_snapshot.go:922  [n1,replicate,s1,r24/1:/Table/2{8-9}] sending LEARNER snapshot 4353750c at applied index 16
I200206 15:30:00.486919 2848 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r24/1:/Table/2{8-9}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):2LEARNER
I200206 15:30:00.486963 12 storage/store_snapshot.go:965  [n1,replicate,s1,r24/1:/Table/2{8-9}] streamed snapshot to (n3,s3):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:00.487495 2888 storage/replica_raftstorage.go:780  [n3,s3,r24/2:{-}] applying LEARNER snapshot [id=4353750c index=16]
I200206 15:30:00.491920 2888 storage/replica_raftstorage.go:801  [n3,s3,r24/2:/Table/2{8-9}] applied LEARNER snapshot [total=4ms ingestion=4@4ms id=4353750c index=16]
I200206 15:30:00.492448 12 storage/replica_command.go:1601  [n1,replicate,s1,r24/1:/Table/2{8-9}] change replicas (add [(n3,s3):2] remove []): existing descriptor r24:/Table/2{8-9} [(n1,s1):1, (n3,s3):2LEARNER, next=3, gen=1]
I200206 15:30:00.498209 12 storage/replica_raft.go:248  [n1,s1,r24/1:/Table/2{8-9}] proposing SIMPLE(v2) ADD_REPLICA[(n3,s3):2]: after=[(n1,s1):1 (n3,s3):2] next=3
I200206 15:30:00.498647 12 storage/replica_command.go:1601  [n1,replicate,s1,r24/1:/Table/2{8-9}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r24:/Table/2{8-9} [(n1,s1):1, (n3,s3):2, next=3, gen=2]
I200206 15:30:00.517152 12 storage/replica_raft.go:248  [n1,s1,r24/1:/Table/2{8-9}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n2,s2):3LEARNER] next=4
I200206 15:30:00.518589 12 storage/store_snapshot.go:922  [n1,replicate,s1,r24/1:/Table/2{8-9}] sending LEARNER snapshot 884b6383 at applied index 20
I200206 15:30:00.518734 12 storage/store_snapshot.go:965  [n1,replicate,s1,r24/1:/Table/2{8-9}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:00.518767 2962 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r24/1:/Table/2{8-9}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):3LEARNER
I200206 15:30:00.519301 3019 storage/replica_raftstorage.go:780  [n2,s2,r24/3:{-}] applying LEARNER snapshot [id=884b6383 index=20]
I200206 15:30:00.522843 3019 storage/replica_raftstorage.go:801  [n2,s2,r24/3:/Table/2{8-9}] applied LEARNER snapshot [total=4ms ingestion=4@3ms id=884b6383 index=20]
I200206 15:30:00.523568 12 storage/replica_command.go:1601  [n1,replicate,s1,r24/1:/Table/2{8-9}] change replicas (add [(n2,s2):3] remove []): existing descriptor r24:/Table/2{8-9} [(n1,s1):1, (n3,s3):2, (n2,s2):3LEARNER, next=4, gen=3]
I200206 15:30:00.528189 12 storage/replica_raft.go:248  [n1,s1,r24/1:/Table/2{8-9}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n3,s3):2 (n2,s2):3] next=4
I200206 15:30:00.533952 12 storage/replica_command.go:1601  [n1,replicate,s1,r22/1:/Table/2{6-7}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r22:/Table/2{6-7} [(n1,s1):1, next=2, gen=0]
I200206 15:30:00.543902 12 storage/replica_raft.go:248  [n1,s1,r22/1:/Table/2{6-7}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200206 15:30:00.546260 12 storage/store_snapshot.go:922  [n1,replicate,s1,r22/1:/Table/2{6-7}] sending LEARNER snapshot 27efe2f8 at applied index 15
I200206 15:30:00.546351 12 storage/store_snapshot.go:965  [n1,replicate,s1,r22/1:/Table/2{6-7}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:00.546865 3079 storage/replica_raftstorage.go:780  [n2,s2,r22/2:{-}] applying LEARNER snapshot [id=27efe2f8 index=15]
I200206 15:30:00.546908 3023 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r22/1:/Table/2{6-7}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):2LEARNER
I200206 15:30:00.554295 3079 storage/replica_raftstorage.go:801  [n2,s2,r22/2:/Table/2{6-7}] applied LEARNER snapshot [total=7ms ingestion=4@7ms id=27efe2f8 index=15]
I200206 15:30:00.554824 12 storage/replica_command.go:1601  [n1,replicate,s1,r22/1:/Table/2{6-7}] change replicas (add [(n2,s2):2] remove []): existing descriptor r22:/Table/2{6-7} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
I200206 15:30:00.564129 12 storage/replica_raft.go:248  [n1,s1,r22/1:/Table/2{6-7}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200206 15:30:00.566151 12 storage/replica_command.go:1601  [n1,replicate,s1,r22/1:/Table/2{6-7}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r22:/Table/2{6-7} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200206 15:30:00.579958 12 storage/replica_raft.go:248  [n1,s1,r22/1:/Table/2{6-7}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3LEARNER] next=4
I200206 15:30:00.583303 12 storage/store_snapshot.go:922  [n1,replicate,s1,r22/1:/Table/2{6-7}] sending LEARNER snapshot ccf129bc at applied index 19
I200206 15:30:00.583414 12 storage/store_snapshot.go:965  [n1,replicate,s1,r22/1:/Table/2{6-7}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:00.583585 3058 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r22/1:/Table/2{6-7}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200206 15:30:00.584109 3056 storage/replica_raftstorage.go:780  [n3,s3,r22/3:{-}] applying LEARNER snapshot [id=ccf129bc index=19]
I200206 15:30:00.588311 3056 storage/replica_raftstorage.go:801  [n3,s3,r22/3:/Table/2{6-7}] applied LEARNER snapshot [total=4ms ingestion=4@3ms id=ccf129bc index=19]
I200206 15:30:00.590116 12 storage/replica_command.go:1601  [n1,replicate,s1,r22/1:/Table/2{6-7}] change replicas (add [(n3,s3):3] remove []): existing descriptor r22:/Table/2{6-7} [(n1,s1):1, (n2,s2):2, (n3,s3):3LEARNER, next=4, gen=3]
I200206 15:30:00.604442 12 storage/replica_raft.go:248  [n1,s1,r22/1:/Table/2{6-7}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3] next=4
I200206 15:30:00.609625 104 gossip/gossip.go:1532  [n1] node has connected to cluster via gossip
I200206 15:30:00.609742 104 storage/stores.go:259  [n1] wrote 3 node addresses to persistent storage
I200206 15:30:00.615327 12 storage/replica_command.go:1601  [n1,replicate,s1,r20/1:/Table/2{4-5}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r20:/Table/2{4-5} [(n1,s1):1, next=2, gen=0]
I200206 15:30:00.621007 12 storage/replica_raft.go:248  [n1,s1,r20/1:/Table/2{4-5}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200206 15:30:00.622459 12 storage/store_snapshot.go:922  [n1,replicate,s1,r20/1:/Table/2{4-5}] sending LEARNER snapshot 7962e520 at applied index 15
I200206 15:30:00.622550 12 storage/store_snapshot.go:965  [n1,replicate,s1,r20/1:/Table/2{4-5}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:00.622599 3103 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r20/1:/Table/2{4-5}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200206 15:30:00.623046 3089 storage/replica_raftstorage.go:780  [n4,s4,r20/2:{-}] applying LEARNER snapshot [id=7962e520 index=15]
I200206 15:30:00.648390 3089 storage/replica_raftstorage.go:801  [n4,s4,r20/2:/Table/2{4-5}] applied LEARNER snapshot [total=25ms ingestion=4@24ms id=7962e520 index=15]
I200206 15:30:00.649051 12 storage/replica_command.go:1601  [n1,replicate,s1,r20/1:/Table/2{4-5}] change replicas (add [(n4,s4):2] remove []): existing descriptor r20:/Table/2{4-5} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200206 15:30:00.663556 12 storage/replica_raft.go:248  [n1,s1,r20/1:/Table/2{4-5}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200206 15:30:00.664169 12 storage/replica_command.go:1601  [n1,replicate,s1,r20/1:/Table/2{4-5}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r20:/Table/2{4-5} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200206 15:30:00.673625 12 storage/replica_raft.go:248  [n1,s1,r20/1:/Table/2{4-5}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3LEARNER] next=4
I200206 15:30:00.676235 12 storage/store_snapshot.go:922  [n1,replicate,s1,r20/1:/Table/2{4-5}] sending LEARNER snapshot c7efe7e5 at applied index 19
I200206 15:30:00.676332 3176 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r20/1:/Table/2{4-5}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200206 15:30:00.676444 12 storage/store_snapshot.go:965  [n1,replicate,s1,r20/1:/Table/2{4-5}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:00.677010 3174 storage/replica_raftstorage.go:780  [n3,s3,r20/3:{-}] applying LEARNER snapshot [id=c7efe7e5 index=19]
I200206 15:30:00.681778 3174 storage/replica_raftstorage.go:801  [n3,s3,r20/3:/Table/2{4-5}] applied LEARNER snapshot [total=5ms ingestion=4@4ms id=c7efe7e5 index=19]
I200206 15:30:00.682277 12 storage/replica_command.go:1601  [n1,replicate,s1,r20/1:/Table/2{4-5}] change replicas (add [(n3,s3):3] remove []): existing descriptor r20:/Table/2{4-5} [(n1,s1):1, (n4,s4):2, (n3,s3):3LEARNER, next=4, gen=3]
I200206 15:30:00.695221 12 storage/replica_raft.go:248  [n1,s1,r20/1:/Table/2{4-5}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3] next=4
I200206 15:30:00.699838 12 storage/replica_command.go:1601  [n1,replicate,s1,r6/1:/Table/{SystemCon…-11}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r6:/Table/{SystemConfigSpan/Start-11} [(n1,s1):1, next=2, gen=0]
I200206 15:30:00.709416 12 storage/replica_raft.go:248  [n1,s1,r6/1:/Table/{SystemCon…-11}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200206 15:30:00.711118 12 storage/store_snapshot.go:922  [n1,replicate,s1,r6/1:/Table/{SystemCon…-11}] sending LEARNER snapshot 3ed98359 at applied index 33
I200206 15:30:00.711340 12 storage/store_snapshot.go:965  [n1,replicate,s1,r6/1:/Table/{SystemCon…-11}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 50, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:00.711663 3193 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r6/1:/Table/{SystemCon…-11}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200206 15:30:00.713574 3191 storage/replica_raftstorage.go:780  [n4,s4,r6/2:{-}] applying LEARNER snapshot [id=3ed98359 index=33]
I200206 15:30:00.722644 3191 storage/replica_raftstorage.go:801  [n4,s4,r6/2:/Table/{SystemCon…-11}] applied LEARNER snapshot [total=9ms ingestion=4@8ms id=3ed98359 index=33]
I200206 15:30:00.723036 12 storage/replica_command.go:1601  [n1,replicate,s1,r6/1:/Table/{SystemCon…-11}] change replicas (add [(n4,s4):2] remove []): existing descriptor r6:/Table/{SystemConfigSpan/Start-11} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200206 15:30:00.741708 12 storage/replica_raft.go:248  [n1,s1,r6/1:/Table/{SystemCon…-11}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200206 15:30:00.742764 12 storage/replica_command.go:1601  [n1,replicate,s1,r6/1:/Table/{SystemCon…-11}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r6:/Table/{SystemConfigSpan/Start-11} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200206 15:30:00.756124 12 storage/replica_raft.go:248  [n1,s1,r6/1:/Table/{SystemCon…-11}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3LEARNER] next=4
I200206 15:30:00.759325 12 storage/store_snapshot.go:922  [n1,replicate,s1,r6/1:/Table/{SystemCon…-11}] sending LEARNER snapshot de22a42d at applied index 37
I200206 15:30:00.760202 12 storage/store_snapshot.go:965  [n1,replicate,s1,r6/1:/Table/{SystemCon…-11}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 54, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:00.760659 3209 storage/replica_raftstorage.go:780  [n2,s2,r6/3:{-}] applying LEARNER snapshot [id=de22a42d index=37]
I200206 15:30:00.773098 3209 storage/replica_raftstorage.go:801  [n2,s2,r6/3:/Table/{SystemCon…-11}] applied LEARNER snapshot [total=12ms ingestion=4@12ms id=de22a42d index=37]
I200206 15:30:00.778025 12 storage/replica_command.go:1601  [n1,replicate,s1,r6/1:/Table/{SystemCon…-11}] change replicas (add [(n2,s2):3] remove []): existing descriptor r6:/Table/{SystemConfigSpan/Start-11} [(n1,s1):1, (n4,s4):2, (n2,s2):3LEARNER, next=4, gen=3]
I200206 15:30:00.789737 12 storage/replica_raft.go:248  [n1,s1,r6/1:/Table/{SystemCon…-11}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3] next=4
I200206 15:30:00.795080 12 storage/replica_command.go:1601  [n1,replicate,s1,r15/1:/Table/{19-20}] change replicas (add [(n3,s3):2LEARNER] remove []): existing descriptor r15:/Table/{19-20} [(n1,s1):1, next=2, gen=0]
I200206 15:30:00.872145 12 storage/replica_raft.go:248  [n1,s1,r15/1:/Table/{19-20}] proposing SIMPLE(l2) ADD_REPLICA[(n3,s3):2LEARNER]: after=[(n1,s1):1 (n3,s3):2LEARNER] next=3
I200206 15:30:00.873433 3232 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r15/1:/Table/{19-20}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):2LEARNER
I200206 15:30:00.874050 12 storage/store_snapshot.go:922  [n1,replicate,s1,r15/1:/Table/{19-20}] sending LEARNER snapshot 942d67ed at applied index 16
I200206 15:30:00.874150 12 storage/store_snapshot.go:965  [n1,replicate,s1,r15/1:/Table/{19-20}] streamed snapshot to (n3,s3):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:00.874854 3282 storage/replica_raftstorage.go:780  [n3,s3,r15/2:{-}] applying LEARNER snapshot [id=942d67ed index=16]
I200206 15:30:00.881520 3282 storage/replica_raftstorage.go:801  [n3,s3,r15/2:/Table/{19-20}] applied LEARNER snapshot [total=7ms ingestion=4@3ms id=942d67ed index=16]
I200206 15:30:00.883896 12 storage/replica_command.go:1601  [n1,replicate,s1,r15/1:/Table/{19-20}] change replicas (add [(n3,s3):2] remove []): existing descriptor r15:/Table/{19-20} [(n1,s1):1, (n3,s3):2LEARNER, next=3, gen=1]
I200206 15:30:00.893350 12 storage/replica_raft.go:248  [n1,s1,r15/1:/Table/{19-20}] proposing SIMPLE(v2) ADD_REPLICA[(n3,s3):2]: after=[(n1,s1):1 (n3,s3):2] next=3
I200206 15:30:00.893822 12 storage/replica_command.go:1601  [n1,replicate,s1,r15/1:/Table/{19-20}] change replicas (add [(n4,s4):3LEARNER] remove []): existing descriptor r15:/Table/{19-20} [(n1,s1):1, (n3,s3):2, next=3, gen=2]
I200206 15:30:00.915264 12 storage/replica_raft.go:248  [n1,s1,r15/1:/Table/{19-20}] proposing SIMPLE(l3) ADD_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3LEARNER] next=4
I200206 15:30:00.918665 12 storage/store_snapshot.go:922  [n1,replicate,s1,r15/1:/Table/{19-20}] sending LEARNER snapshot 85beabf4 at applied index 20
I200206 15:30:00.918785 3328 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r15/1:/Table/{19-20}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):3LEARNER
I200206 15:30:00.918808 12 storage/store_snapshot.go:965  [n1,replicate,s1,r15/1:/Table/{19-20}] streamed snapshot to (n4,s4):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:00.919347 3326 storage/replica_raftstorage.go:780  [n4,s4,r15/3:{-}] applying LEARNER snapshot [id=85beabf4 index=20]
I200206 15:30:00.924114 3326 storage/replica_raftstorage.go:801  [n4,s4,r15/3:/Table/{19-20}] applied LEARNER snapshot [total=5ms ingestion=4@4ms id=85beabf4 index=20]
I200206 15:30:00.926404 1208 gossip/gossip.go:1532  [n3] node has connected to cluster via gossip
I200206 15:30:00.926560 1208 storage/stores.go:259  [n3] wrote 3 node addresses to persistent storage
I200206 15:30:00.927488 12 storage/replica_command.go:1601  [n1,replicate,s1,r15/1:/Table/{19-20}] change replicas (add [(n4,s4):3] remove []): existing descriptor r15:/Table/{19-20} [(n1,s1):1, (n3,s3):2, (n4,s4):3LEARNER, next=4, gen=3]
I200206 15:30:00.933803 12 storage/replica_raft.go:248  [n1,s1,r15/1:/Table/{19-20}] proposing SIMPLE(v3) ADD_REPLICA[(n4,s4):3]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3] next=4
I200206 15:30:00.938538 12 storage/replica_command.go:1601  [n1,replicate,s1,r16/1:/Table/2{0-1}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r16:/Table/2{0-1} [(n1,s1):1, next=2, gen=0]
I200206 15:30:00.947505 12 storage/replica_raft.go:248  [n1,s1,r16/1:/Table/2{0-1}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200206 15:30:00.948795 12 storage/store_snapshot.go:922  [n1,replicate,s1,r16/1:/Table/2{0-1}] sending LEARNER snapshot 68233477 at applied index 15
I200206 15:30:00.948886 3399 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r16/1:/Table/2{0-1}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):2LEARNER
I200206 15:30:00.948890 12 storage/store_snapshot.go:965  [n1,replicate,s1,r16/1:/Table/2{0-1}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:00.949484 3414 storage/replica_raftstorage.go:780  [n2,s2,r16/2:{-}] applying LEARNER snapshot [id=68233477 index=15]
I200206 15:30:00.953324 3414 storage/replica_raftstorage.go:801  [n2,s2,r16/2:/Table/2{0-1}] applied LEARNER snapshot [total=4ms ingestion=4@3ms id=68233477 index=15]
I200206 15:30:00.953740 12 storage/replica_command.go:1601  [n1,replicate,s1,r16/1:/Table/2{0-1}] change replicas (add [(n2,s2):2] remove []): existing descriptor r16:/Table/2{0-1} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
I200206 15:30:00.960316 12 storage/replica_raft.go:248  [n1,s1,r16/1:/Table/2{0-1}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200206 15:30:00.960786 12 storage/replica_command.go:1601  [n1,replicate,s1,r16/1:/Table/2{0-1}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r16:/Table/2{0-1} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200206 15:30:00.974004 12 storage/replica_raft.go:248  [n1,s1,r16/1:/Table/2{0-1}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3LEARNER] next=4
I200206 15:30:00.976924 12 storage/store_snapshot.go:922  [n1,replicate,s1,r16/1:/Table/2{0-1}] sending LEARNER snapshot 250412cb at applied index 19
I200206 15:30:00.977033 12 storage/store_snapshot.go:965  [n1,replicate,s1,r16/1:/Table/2{0-1}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:00.977556 3451 storage/replica_raftstorage.go:780  [n3,s3,r16/3:{-}] applying LEARNER snapshot [id=250412cb index=19]
I200206 15:30:00.978352 3435 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r16/1:/Table/2{0-1}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200206 15:30:00.980808 3451 storage/replica_raftstorage.go:801  [n3,s3,r16/3:/Table/2{0-1}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=250412cb index=19]
I200206 15:30:00.981990 12 storage/replica_command.go:1601  [n1,replicate,s1,r16/1:/Table/2{0-1}] change replicas (add [(n3,s3):3] remove []): existing descriptor r16:/Table/2{0-1} [(n1,s1):1, (n2,s2):2, (n3,s3):3LEARNER, next=4, gen=3]
I200206 15:30:01.008173 12 storage/replica_raft.go:248  [n1,s1,r16/1:/Table/2{0-1}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3] next=4
I200206 15:30:01.010932 12 storage/replica_command.go:1601  [n1,replicate,s1,r14/1:/Table/1{8-9}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r14:/Table/1{8-9} [(n1,s1):1, next=2, gen=0]
I200206 15:30:01.023133 12 storage/replica_raft.go:248  [n1,s1,r14/1:/Table/1{8-9}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200206 15:30:01.025078 12 storage/store_snapshot.go:922  [n1,replicate,s1,r14/1:/Table/1{8-9}] sending LEARNER snapshot ca941085 at applied index 15
I200206 15:30:01.025198 12 storage/store_snapshot.go:965  [n1,replicate,s1,r14/1:/Table/1{8-9}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:01.025202 3407 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r14/1:/Table/1{8-9}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200206 15:30:01.025621 3541 storage/replica_raftstorage.go:780  [n4,s4,r14/2:{-}] applying LEARNER snapshot [id=ca941085 index=15]
I200206 15:30:01.028976 3541 storage/replica_raftstorage.go:801  [n4,s4,r14/2:/Table/1{8-9}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=ca941085 index=15]
I200206 15:30:01.029451 12 storage/replica_command.go:1601  [n1,replicate,s1,r14/1:/Table/1{8-9}] change replicas (add [(n4,s4):2] remove []): existing descriptor r14:/Table/1{8-9} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200206 15:30:01.037796 12 storage/replica_raft.go:248  [n1,s1,r14/1:/Table/1{8-9}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200206 15:30:01.038309 12 storage/replica_command.go:1601  [n1,replicate,s1,r14/1:/Table/1{8-9}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r14:/Table/1{8-9} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200206 15:30:01.052422 12 storage/replica_raft.go:248  [n1,s1,r14/1:/Table/1{8-9}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3LEARNER] next=4
I200206 15:30:01.054662 12 storage/store_snapshot.go:922  [n1,replicate,s1,r14/1:/Table/1{8-9}] sending LEARNER snapshot 09e8a8ba at applied index 19
I200206 15:30:01.054794 12 storage/store_snapshot.go:965  [n1,replicate,s1,r14/1:/Table/1{8-9}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:01.054963 3578 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r14/1:/Table/1{8-9}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):3LEARNER
I200206 15:30:01.055274 3576 storage/replica_raftstorage.go:780  [n2,s2,r14/3:{-}] applying LEARNER snapshot [id=09e8a8ba index=19]
I200206 15:30:01.065833 3576 storage/replica_raftstorage.go:801  [n2,s2,r14/3:/Table/1{8-9}] applied LEARNER snapshot [total=11ms ingestion=4@10ms id=09e8a8ba index=19]
I200206 15:30:01.066667 12 storage/replica_command.go:1601  [n1,replicate,s1,r14/1:/Table/1{8-9}] change replicas (add [(n2,s2):3] remove []): existing descriptor r14:/Table/1{8-9} [(n1,s1):1, (n4,s4):2, (n2,s2):3LEARNER, next=4, gen=3]
I200206 15:30:01.074080 12 storage/replica_raft.go:248  [n1,s1,r14/1:/Table/1{8-9}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3] next=4
I200206 15:30:01.076330 12 storage/replica_command.go:1601  [n1,replicate,s1,r2/1:/System/NodeLiveness{-Max}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r2:/System/NodeLiveness{-Max} [(n1,s1):1, next=2, gen=0]
I200206 15:30:01.092087 12 storage/replica_raft.go:248  [n1,s1,r2/1:/System/NodeLiveness{-Max}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200206 15:30:01.094100 3547 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r2/1:/System/NodeLiveness{-Max}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200206 15:30:01.094184 12 storage/store_snapshot.go:922  [n1,replicate,s1,r2/1:/System/NodeLiveness{-Max}] sending LEARNER snapshot 1d70a4c4 at applied index 22
I200206 15:30:01.094331 12 storage/store_snapshot.go:965  [n1,replicate,s1,r2/1:/System/NodeLiveness{-Max}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:01.094966 3545 storage/replica_raftstorage.go:780  [n4,s4,r2/2:{-}] applying LEARNER snapshot [id=1d70a4c4 index=22]
I200206 15:30:01.098658 3545 storage/replica_raftstorage.go:801  [n4,s4,r2/2:/System/NodeLiveness{-Max}] applied LEARNER snapshot [total=4ms ingestion=4@3ms id=1d70a4c4 index=22]
I200206 15:30:01.099568 12 storage/replica_command.go:1601  [n1,replicate,s1,r2/1:/System/NodeLiveness{-Max}] change replicas (add [(n4,s4):2] remove []): existing descriptor r2:/System/NodeLiveness{-Max} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200206 15:30:01.108036 12 storage/replica_raft.go:248  [n1,s1,r2/1:/System/NodeLiveness{-Max}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200206 15:30:01.108471 12 storage/replica_command.go:1601  [n1,replicate,s1,r2/1:/System/NodeLiveness{-Max}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r2:/System/NodeLiveness{-Max} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200206 15:30:01.125533 12 storage/replica_raft.go:248  [n1,s1,r2/1:/System/NodeLiveness{-Max}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3LEARNER] next=4
I200206 15:30:01.127188 12 storage/store_snapshot.go:922  [n1,replicate,s1,r2/1:/System/NodeLiveness{-Max}] sending LEARNER snapshot 13544365 at applied index 26
I200206 15:30:01.127316 12 storage/store_snapshot.go:965  [n1,replicate,s1,r2/1:/System/NodeLiveness{-Max}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 14, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:01.127500 3686 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r2/1:/System/NodeLiveness{-Max}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200206 15:30:01.128148 3673 storage/replica_raftstorage.go:780  [n3,s3,r2/3:{-}] applying LEARNER snapshot [id=13544365 index=26]
I200206 15:30:01.139907 3673 storage/replica_raftstorage.go:801  [n3,s3,r2/3:/System/NodeLiveness{-Max}] applied LEARNER snapshot [total=12ms ingestion=4@11ms id=13544365 index=26]
I200206 15:30:01.156900 12 storage/replica_command.go:1601  [n1,replicate,s1,r2/1:/System/NodeLiveness{-Max}] change replicas (add [(n3,s3):3] remove []): existing descriptor r2:/System/NodeLiveness{-Max} [(n1,s1):1, (n4,s4):2, (n3,s3):3LEARNER, next=4, gen=3]
I200206 15:30:01.171316 12 storage/replica_raft.go:248  [n1,s1,r2/1:/System/NodeLiveness{-Max}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3] next=4
I200206 15:30:01.174082 12 storage/replica_command.go:1601  [n1,replicate,s1,r4/1:/System{/tsd-tse}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r4:/System{/tsd-tse} [(n1,s1):1, next=2, gen=0]
I200206 15:30:01.189893 12 storage/replica_raft.go:248  [n1,s1,r4/1:/System{/tsd-tse}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200206 15:30:01.191420 12 storage/store_snapshot.go:922  [n1,replicate,s1,r4/1:/System{/tsd-tse}] sending LEARNER snapshot 82197ea2 at applied index 26
I200206 15:30:01.191541 3667 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r4/1:/System{/tsd-tse}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):2LEARNER
I200206 15:30:01.272945 12 storage/store_snapshot.go:965  [n1,replicate,s1,r4/1:/System{/tsd-tse}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 2594, log entries: 0, rate-limit: 8.0 MiB/sec, 0.08s
I200206 15:30:01.273426 3696 storage/replica_raftstorage.go:780  [n2,s2,r4/2:{-}] applying LEARNER snapshot [id=82197ea2 index=26]
I200206 15:30:01.276407 3696 storage/replica_raftstorage.go:801  [n2,s2,r4/2:/System{/tsd-tse}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=82197ea2 index=26]
I200206 15:30:01.279092 12 storage/replica_command.go:1601  [n1,replicate,s1,r4/1:/System{/tsd-tse}] change replicas (add [(n2,s2):2] remove []): existing descriptor r4:/System{/tsd-tse} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
I200206 15:30:01.286003 12 storage/replica_raft.go:248  [n1,s1,r4/1:/System{/tsd-tse}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200206 15:30:01.286493 12 storage/replica_command.go:1601  [n1,replicate,s1,r4/1:/System{/tsd-tse}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r4:/System{/tsd-tse} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200206 15:30:01.295997 12 storage/replica_raft.go:248  [n1,s1,r4/1:/System{/tsd-tse}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3LEARNER] next=4
I200206 15:30:01.299784 12 storage/store_snapshot.go:922  [n1,replicate,s1,r4/1:/System{/tsd-tse}] sending LEARNER snapshot c7c4b96a at applied index 30
I200206 15:30:01.299949 3782 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r4/1:/System{/tsd-tse}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200206 15:30:01.359600 12 storage/store_snapshot.go:965  [n1,replicate,s1,r4/1:/System{/tsd-tse}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 2598, log entries: 0, rate-limit: 8.0 MiB/sec, 0.06s
I200206 15:30:01.360311 3723 storage/replica_raftstorage.go:780  [n3,s3,r4/3:{-}] applying LEARNER snapshot [id=c7c4b96a index=30]
I200206 15:30:01.380806 3723 storage/replica_raftstorage.go:801  [n3,s3,r4/3:/System{/tsd-tse}] applied LEARNER snapshot [total=20ms ingestion=4@18ms id=c7c4b96a index=30]
I200206 15:30:01.381868 12 storage/replica_command.go:1601  [n1,replicate,s1,r4/1:/System{/tsd-tse}] change replicas (add [(n3,s3):3] remove []): existing descriptor r4:/System{/tsd-tse} [(n1,s1):1, (n2,s2):2, (n3,s3):3LEARNER, next=4, gen=3]
I200206 15:30:01.395169 12 storage/replica_raft.go:248  [n1,s1,r4/1:/System{/tsd-tse}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3] next=4
I200206 15:30:01.398429 12 storage/replica_command.go:1601  [n1,replicate,s1,r11/1:/Table/1{5-6}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r11:/Table/1{5-6} [(n1,s1):1, next=2, gen=0]
I200206 15:30:01.408831 12 storage/replica_raft.go:248  [n1,s1,r11/1:/Table/1{5-6}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200206 15:30:01.410122 12 storage/store_snapshot.go:922  [n1,replicate,s1,r11/1:/Table/1{5-6}] sending LEARNER snapshot 2f208b9a at applied index 15
I200206 15:30:01.410231 12 storage/store_snapshot.go:965  [n1,replicate,s1,r11/1:/Table/1{5-6}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:01.410392 3793 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r11/1:/Table/1{5-6}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200206 15:30:01.411239 3791 storage/replica_raftstorage.go:780  [n4,s4,r11/2:{-}] applying LEARNER snapshot [id=2f208b9a index=15]
I200206 15:30:01.415028 3791 storage/replica_raftstorage.go:801  [n4,s4,r11/2:/Table/1{5-6}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=2f208b9a index=15]
I200206 15:30:01.418655 12 storage/replica_command.go:1601  [n1,replicate,s1,r11/1:/Table/1{5-6}] change replicas (add [(n4,s4):2] remove []): existing descriptor r11:/Table/1{5-6} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200206 15:30:01.428128 12 storage/replica_raft.go:248  [n1,s1,r11/1:/Table/1{5-6}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200206 15:30:01.430201 12 storage/replica_command.go:1601  [n1,replicate,s1,r11/1:/Table/1{5-6}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r11:/Table/1{5-6} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200206 15:30:01.439258 12 storage/replica_raft.go:248  [n1,s1,r11/1:/Table/1{5-6}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3LEARNER] next=4
I200206 15:30:01.441137 12 storage/store_snapshot.go:922  [n1,replicate,s1,r11/1:/Table/1{5-6}] sending LEARNER snapshot b300a565 at applied index 19
I200206 15:30:01.441242 12 storage/store_snapshot.go:965  [n1,replicate,s1,r11/1:/Table/1{5-6}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:01.441294 3912 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r11/1:/Table/1{5-6}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):3LEARNER
I200206 15:30:01.442859 3958 storage/replica_raftstorage.go:780  [n2,s2,r11/3:{-}] applying LEARNER snapshot [id=b300a565 index=19]
I200206 15:30:01.451251 3958 storage/replica_raftstorage.go:801  [n2,s2,r11/3:/Table/1{5-6}] applied LEARNER snapshot [total=8ms ingestion=4@8ms id=b300a565 index=19]
I200206 15:30:01.451697 12 storage/replica_command.go:1601  [n1,replicate,s1,r11/1:/Table/1{5-6}] change replicas (add [(n2,s2):3] remove []): existing descriptor r11:/Table/1{5-6} [(n1,s1):1, (n4,s4):2, (n2,s2):3LEARNER, next=4, gen=3]
I200206 15:30:01.459446 12 storage/replica_raft.go:248  [n1,s1,r11/1:/Table/1{5-6}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3] next=4
I200206 15:30:01.462283 12 storage/replica_command.go:1601  [n1,replicate,s1,r18/1:/Table/2{2-3}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r18:/Table/2{2-3} [(n1,s1):1, next=2, gen=0]
I200206 15:30:01.468265 3828 storage/replica_command.go:1601  [n3,replicate,s3,r8/3:/Table/1{2-3}] change replicas (add [(n2,s2):4LEARNER] remove []): existing descriptor r8:/Table/1{2-3} [(n1,s1):1, (n4,s4):2, (n3,s3):3, next=4, gen=4]
I200206 15:30:01.471801 12 storage/replica_raft.go:248  [n1,s1,r18/1:/Table/2{2-3}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200206 15:30:01.472965 12 storage/store_snapshot.go:922  [n1,replicate,s1,r18/1:/Table/2{2-3}] sending LEARNER snapshot 1e358c27 at applied index 16
I200206 15:30:01.473097 3969 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r18/1:/Table/2{2-3}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200206 15:30:01.473774 3967 storage/replica_raftstorage.go:780  [n4,s4,r18/2:{-}] applying LEARNER snapshot [id=1e358c27 index=16]
I200206 15:30:01.473850 12 storage/store_snapshot.go:965  [n1,replicate,s1,r18/1:/Table/2{2-3}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:01.479953 3967 storage/replica_raftstorage.go:801  [n4,s4,r18/2:/Table/2{2-3}] applied LEARNER snapshot [total=6ms ingestion=4@6ms id=1e358c27 index=16]
I200206 15:30:01.481598 1013 gossip/gossip.go:1532  [n2] node has connected to cluster via gossip
I200206 15:30:01.481727 1013 storage/stores.go:259  [n2] wrote 3 node addresses to persistent storage
I200206 15:30:01.482886 12 storage/replica_command.go:1601  [n1,replicate,s1,r18/1:/Table/2{2-3}] change replicas (add [(n4,s4):2] remove []): existing descriptor r18:/Table/2{2-3} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200206 15:30:01.491909 12 storage/replica_raft.go:248  [n1,s1,r18/1:/Table/2{2-3}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200206 15:30:01.492365 12 storage/replica_command.go:1601  [n1,replicate,s1,r18/1:/Table/2{2-3}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r18:/Table/2{2-3} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200206 15:30:01.498913 12 storage/replica_raft.go:248  [n1,s1,r18/1:/Table/2{2-3}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3LEARNER] next=4
I200206 15:30:01.500469 3999 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r18/1:/Table/2{2-3}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200206 15:30:01.500599 12 storage/store_snapshot.go:922  [n1,replicate,s1,r18/1:/Table/2{2-3}] sending LEARNER snapshot 84acd98f at applied index 20
I200206 15:30:01.500737 12 storage/store_snapshot.go:965  [n1,replicate,s1,r18/1:/Table/2{2-3}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:01.501296 3885 storage/replica_raftstorage.go:780  [n3,s3,r18/3:{-}] applying LEARNER snapshot [id=84acd98f index=20]
I200206 15:30:01.510998 3885 storage/replica_raftstorage.go:801  [n3,s3,r18/3:/Table/2{2-3}] applied LEARNER snapshot [total=10ms ingestion=4@9ms id=84acd98f index=20]
I200206 15:30:01.511467 12 storage/replica_command.go:1601  [n1,replicate,s1,r18/1:/Table/2{2-3}] change replicas (add [(n3,s3):3] remove []): existing descriptor r18:/Table/2{2-3} [(n1,s1):1, (n4,s4):2, (n3,s3):3LEARNER, next=4, gen=3]
I200206 15:30:01.528151 12 storage/replica_raft.go:248  [n1,s1,r18/1:/Table/2{2-3}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3] next=4
I200206 15:30:01.530947 12 storage/replica_command.go:1601  [n1,replicate,s1,r21/1:/Table/2{5-6}] change replicas (add [(n3,s3):2LEARNER] remove []): existing descriptor r21:/Table/2{5-6} [(n1,s1):1, next=2, gen=0]
I200206 15:30:01.537314 12 storage/replica_raft.go:248  [n1,s1,r21/1:/Table/2{5-6}] proposing SIMPLE(l2) ADD_REPLICA[(n3,s3):2LEARNER]: after=[(n1,s1):1 (n3,s3):2LEARNER] next=3
I200206 15:30:01.538646 12 storage/store_snapshot.go:922  [n1,replicate,s1,r21/1:/Table/2{5-6}] sending LEARNER snapshot 68b03b4b at applied index 15
I200206 15:30:01.538765 12 storage/store_snapshot.go:965  [n1,replicate,s1,r21/1:/Table/2{5-6}] streamed snapshot to (n3,s3):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:01.538943 3988 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r21/1:/Table/2{5-6}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):2LEARNER
I200206 15:30:01.539262 4076 storage/replica_raftstorage.go:780  [n3,s3,r21/2:{-}] applying LEARNER snapshot [id=68b03b4b index=15]
I200206 15:30:01.539386 1640 gossip/gossip.go:1532  [n4] node has connected to cluster via gossip
I200206 15:30:01.539558 1640 storage/stores.go:259  [n4] wrote 3 node addresses to persistent storage
I200206 15:30:01.548080 4076 storage/replica_raftstorage.go:801  [n3,s3,r21/2:/Table/2{5-6}] applied LEARNER snapshot [total=9ms ingestion=4@8ms id=68b03b4b index=15]
I200206 15:30:01.548614 12 storage/replica_command.go:1601  [n1,replicate,s1,r21/1:/Table/2{5-6}] change replicas (add [(n3,s3):2] remove []): existing descriptor r21:/Table/2{5-6} [(n1,s1):1, (n3,s3):2LEARNER, next=3, gen=1]
I200206 15:30:01.560588 12 storage/replica_raft.go:248  [n1,s1,r21/1:/Table/2{5-6}] proposing SIMPLE(v2) ADD_REPLICA[(n3,s3):2]: after=[(n1,s1):1 (n3,s3):2] next=3
I200206 15:30:01.561018 12 storage/replica_command.go:1601  [n1,replicate,s1,r21/1:/Table/2{5-6}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r21:/Table/2{5-6} [(n1,s1):1, (n3,s3):2, next=3, gen=2]
I200206 15:30:01.580253 12 storage/replica_raft.go:248  [n1,s1,r21/1:/Table/2{5-6}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n2,s2):3LEARNER] next=4
I200206 15:30:01.582690 12 storage/store_snapshot.go:922  [n1,replicate,s1,r21/1:/Table/2{5-6}] sending LEARNER snapshot 5e6af044 at applied index 19
I200206 15:30:01.582788 4031 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r21/1:/Table/2{5-6}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):3LEARNER
I200206 15:30:01.582895 12 storage/store_snapshot.go:965  [n1,replicate,s1,r21/1:/Table/2{5-6}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:01.583287 4126 storage/replica_raftstorage.go:780  [n2,s2,r21/3:{-}] applying LEARNER snapshot [id=5e6af044 index=19]
I200206 15:30:01.587183 4126 storage/replica_raftstorage.go:801  [n2,s2,r21/3:/Table/2{5-6}] applied LEARNER snapshot [total=4ms ingestion=4@4ms id=5e6af044 index=19]
I200206 15:30:01.591102 12 storage/replica_command.go:1601  [n1,replicate,s1,r21/1:/Table/2{5-6}] change replicas (add [(n2,s2):3] remove []): existing descriptor r21:/Table/2{5-6} [(n1,s1):1, (n3,s3):2, (n2,s2):3LEARNER, next=4, gen=3]
I200206 15:30:01.603049 3828 storage/replica_raft.go:248  [n3,s3,r8/3:/Table/1{2-3}] proposing SIMPLE(l4) ADD_REPLICA[(n2,s2):4LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3 (n2,s2):4LEARNER] next=5
I200206 15:30:01.605071 3828 storage/store_snapshot.go:922  [n3,replicate,s3,r8/3:/Table/1{2-3}] sending LEARNER snapshot 16413db4 at applied index 55
I200206 15:30:01.605163 4131 storage/raft_snapshot_queue.go:169  [n3,raftsnapshot,s3,r8/3:/Table/1{2-3}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):4LEARNER
I200206 15:30:01.605335 3828 storage/store_snapshot.go:965  [n3,replicate,s3,r8/3:/Table/1{2-3}] streamed snapshot to (n2,s2):4LEARNER: kv pairs: 56, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:01.606315 4129 storage/replica_raftstorage.go:780  [n2,s2,r8/4:{-}] applying LEARNER snapshot [id=16413db4 index=55]
I200206 15:30:01.611261 4129 storage/replica_raftstorage.go:801  [n2,s2,r8/4:/Table/1{2-3}] applied LEARNER snapshot [total=5ms ingestion=4@5ms id=16413db4 index=55]
I200206 15:30:01.614053 3828 storage/replica_command.go:1601  [n3,replicate,s3,r8/3:/Table/1{2-3}] change replicas (add [(n2,s2):4VOTER_INCOMING] remove [(n1,s1):1VOTER_DEMOTING]): existing descriptor r8:/Table/1{2-3} [(n1,s1):1, (n4,s4):2, (n3,s3):3, (n2,s2):4LEARNER, next=5, gen=5]
I200206 15:30:01.627260 12 storage/replica_raft.go:248  [n1,s1,r21/1:/Table/2{5-6}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n3,s3):2 (n2,s2):3] next=4
I200206 15:30:01.631369 12 storage/replica_command.go:1601  [n1,replicate,s1,r1/1:/{Min-System/NodeL…}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r1:/{Min-System/NodeLiveness} [(n1,s1):1, next=2, gen=0]
I200206 15:30:01.637377 3828 storage/replica_raft.go:248  [n3,s3,r8/3:/Table/1{2-3}] proposing ENTER_JOINT(r1 l1 v4) ADD_REPLICA[(n2,s2):4VOTER_INCOMING], REMOVE_REPLICA[(n1,s1):1VOTER_DEMOTING]: after=[(n1,s1):1VOTER_DEMOTING (n4,s4):2 (n3,s3):3 (n2,s2):4VOTER_INCOMING] next=5
I200206 15:30:01.638383 3828 storage/replica_command.go:1601  [n3,replicate,s3,r8/3:/Table/1{2-3}] change replicas (add [] remove []): existing descriptor r8:/Table/1{2-3} [(n1,s1):1VOTER_DEMOTING, (n4,s4):2, (n3,s3):3, (n2,s2):4VOTER_INCOMING, next=5, gen=6]
I200206 15:30:01.647318 3828 storage/replica_raft.go:248  [n3,s3,r8/3:/Table/1{2-3}] proposing LEAVE_JOINT: after=[(n1,s1):1LEARNER (n4,s4):2 (n3,s3):3 (n2,s2):4] next=5
I200206 15:30:01.647628 12 storage/replica_raft.go:248  [n1,s1,r1/1:/{Min-System/NodeL…}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200206 15:30:01.648165 3828 storage/replica_command.go:1601  [n3,replicate,s3,r8/3:/Table/1{2-3}] change replicas (add [] remove [(n1,s1):1LEARNER]): existing descriptor r8:/Table/1{2-3} [(n1,s1):1LEARNER, (n4,s4):2, (n3,s3):3, (n2,s2):4, next=5, gen=7]
I200206 15:30:01.649587 12 storage/store_snapshot.go:922  [n1,replicate,s1,r1/1:/{Min-System/NodeL…}] sending LEARNER snapshot 8de904e4 at applied index 226
I200206 15:30:01.649881 4255 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r1/1:/{Min-System/NodeL…}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200206 15:30:01.649892 12 storage/store_snapshot.go:965  [n1,replicate,s1,r1/1:/{Min-System/NodeL…}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 141, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:01.650588 4253 storage/replica_raftstorage.go:780  [n4,s4,r1/2:{-}] applying LEARNER snapshot [id=8de904e4 index=226]
I200206 15:30:01.654415 4253 storage/replica_raftstorage.go:801  [n4,s4,r1/2:/{Min-System/NodeL…}] applied LEARNER snapshot [total=4ms ingestion=4@4ms id=8de904e4 index=226]
I200206 15:30:01.664024 3828 storage/replica_raft.go:248  [n3,s3,r8/3:/Table/1{2-3}] proposing SIMPLE(r1) REMOVE_REPLICA[(n1,s1):1LEARNER]: after=[(n2,s2):4 (n4,s4):2 (n3,s3):3] next=5
I200206 15:30:01.665311 1660 storage/store_remove_replica.go:122  [n1,s1,r8/1:/Table/1{2-3}] removing replica r8/1
I200206 15:30:01.665868 1660 storage/replica_destroy.go:217  [n1,s1,r8/1:/Table/1{2-3}] removed 52 (45+7) keys in 0ms [clear=0ms commit=0ms]
I200206 15:30:01.667536 12 storage/replica_command.go:1601  [n1,replicate,s1,r1/1:/{Min-System/NodeL…}] change replicas (add [(n4,s4):2] remove []): existing descriptor r1:/{Min-System/NodeLiveness} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200206 15:30:01.680001 12 storage/replica_raft.go:248  [n1,s1,r1/1:/{Min-System/NodeL…}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200206 15:30:01.680674 12 storage/replica_command.go:1601  [n1,replicate,s1,r1/1:/{Min-System/NodeL…}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r1:/{Min-System/NodeLiveness} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200206 15:30:01.691679 12 storage/replica_raft.go:248  [n1,s1,r1/1:/{Min-System/NodeL…}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3LEARNER] next=4
I200206 15:30:01.693826 12 storage/store_snapshot.go:922  [n1,replicate,s1,r1/1:/{Min-System/NodeL…}] sending LEARNER snapshot 0795ba88 at applied index 236
I200206 15:30:01.695140 12 storage/store_snapshot.go:965  [n1,replicate,s1,r1/1:/{Min-System/NodeL…}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 149, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:01.694233 4317 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r1/1:/{Min-System/NodeL…}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200206 15:30:01.696264 4191 storage/replica_raftstorage.go:780  [n3,s3,r1/3:{-}] applying LEARNER snapshot [id=0795ba88 index=236]
I200206 15:30:01.699443 4191 storage/replica_raftstorage.go:801  [n3,s3,r1/3:/{Min-System/NodeL…}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=0795ba88 index=236]
I200206 15:30:01.699934 12 storage/replica_command.go:1601  [n1,replicate,s1,r1/1:/{Min-System/NodeL…}] change replicas (add [(n3,s3):3] remove []): existing descriptor r1:/{Min-System/NodeLiveness} [(n1,s1):1, (n4,s4):2, (n3,s3):3LEARNER, next=4, gen=3]
I200206 15:30:01.717304 12 storage/replica_raft.go:248  [n1,s1,r1/1:/{Min-System/NodeL…}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3] next=4
I200206 15:30:01.720198 12 storage/replica_command.go:1601  [n1,replicate,s1,r13/1:/Table/1{7-8}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r13:/Table/1{7-8} [(n1,s1):1, next=2, gen=0]
I200206 15:30:01.729175 12 storage/replica_raft.go:248  [n1,s1,r13/1:/Table/1{7-8}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200206 15:30:01.732082 12 storage/store_snapshot.go:922  [n1,replicate,s1,r13/1:/Table/1{7-8}] sending LEARNER snapshot dfa24126 at applied index 15
I200206 15:30:01.732183 12 storage/store_snapshot.go:965  [n1,replicate,s1,r13/1:/Table/1{7-8}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:01.732275 4210 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r13/1:/Table/1{7-8}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200206 15:30:01.732775 4390 storage/replica_raftstorage.go:780  [n4,s4,r13/2:{-}] applying LEARNER snapshot [id=dfa24126 index=15]
I200206 15:30:01.741205 4390 storage/replica_raftstorage.go:801  [n4,s4,r13/2:/Table/1{7-8}] applied LEARNER snapshot [total=8ms ingestion=4@8ms id=dfa24126 index=15]
I200206 15:30:01.743273 12 storage/replica_command.go:1601  [n1,replicate,s1,r13/1:/Table/1{7-8}] change replicas (add [(n4,s4):2] remove []): existing descriptor r13:/Table/1{7-8} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200206 15:30:01.763048 12 storage/replica_raft.go:248  [n1,s1,r13/1:/Table/1{7-8}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200206 15:30:01.764112 12 storage/replica_command.go:1601  [n1,replicate,s1,r13/1:/Table/1{7-8}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r13:/Table/1{7-8} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200206 15:30:01.774329 12 storage/replica_raft.go:248  [n1,s1,r13/1:/Table/1{7-8}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3LEARNER] next=4
I200206 15:30:01.775960 12 storage/store_snapshot.go:922  [n1,replicate,s1,r13/1:/Table/1{7-8}] sending LEARNER snapshot 81fa6bdc at applied index 19
I200206 15:30:01.776019 4470 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r13/1:/Table/1{7-8}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):3LEARNER
I200206 15:30:01.776108 12 storage/store_snapshot.go:965  [n1,replicate,s1,r13/1:/Table/1{7-8}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:01.776606 4453 storage/replica_raftstorage.go:780  [n2,s2,r13/3:{-}] applying LEARNER snapshot [id=81fa6bdc index=19]
I200206 15:30:01.780869 4453 storage/replica_raftstorage.go:801  [n2,s2,r13/3:/Table/1{7-8}] applied LEARNER snapshot [total=4ms ingestion=4@4ms id=81fa6bdc index=19]
I200206 15:30:01.781295 12 storage/replica_command.go:1601  [n1,replicate,s1,r13/1:/Table/1{7-8}] change replicas (add [(n2,s2):3] remove []): existing descriptor r13:/Table/1{7-8} [(n1,s1):1, (n4,s4):2, (n2,s2):3LEARNER, next=4, gen=3]
I200206 15:30:01.807460 12 storage/replica_raft.go:248  [n1,s1,r13/1:/Table/1{7-8}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3] next=4
I200206 15:30:01.810429 12 storage/replica_command.go:1601  [n1,replicate,s1,r5/1:/{Systemtse-Table/System…}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r5:/{Systemtse-Table/SystemConfigSpan/Start} [(n1,s1):1, next=2, gen=0]
I200206 15:30:01.822205 12 storage/replica_raft.go:248  [n1,s1,r5/1:/{Systemtse-Table/System…}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200206 15:30:01.828115 4507 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r5/1:/{Systemtse-Table/System…}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200206 15:30:01.828262 12 storage/store_snapshot.go:922  [n1,replicate,s1,r5/1:/{Systemtse-Table/System…}] sending LEARNER snapshot 351bdb63 at applied index 15
I200206 15:30:01.829975 12 storage/store_snapshot.go:965  [n1,replicate,s1,r5/1:/{Systemtse-Table/System…}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.01s
I200206 15:30:01.830541 4505 storage/replica_raftstorage.go:780  [n4,s4,r5/2:{-}] applying LEARNER snapshot [id=351bdb63 index=15]
I200206 15:30:01.837348 4505 storage/replica_raftstorage.go:801  [n4,s4,r5/2:/{Systemtse-Table/System…}] applied LEARNER snapshot [total=7ms ingestion=4@6ms id=351bdb63 index=15]
I200206 15:30:01.837854 12 storage/replica_command.go:1601  [n1,replicate,s1,r5/1:/{Systemtse-Table/System…}] change replicas (add [(n4,s4):2] remove []): existing descriptor r5:/{Systemtse-Table/SystemConfigSpan/Start} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200206 15:30:01.846841 12 storage/replica_raft.go:248  [n1,s1,r5/1:/{Systemtse-Table/System…}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200206 15:30:01.847345 12 storage/replica_command.go:1601  [n1,replicate,s1,r5/1:/{Systemtse-Table/System…}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r5:/{Systemtse-Table/SystemConfigSpan/Start} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200206 15:30:01.873399 12 storage/replica_raft.go:248  [n1,s1,r5/1:/{Systemtse-Table/System…}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3LEARNER] next=4
I200206 15:30:01.877334 12 storage/store_snapshot.go:922  [n1,replicate,s1,r5/1:/{Systemtse-Table/System…}] sending LEARNER snapshot 13e1789f at applied index 20
I200206 15:30:01.877413 4465 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r5/1:/{Systemtse-Table/System…}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):3LEARNER
I200206 15:30:01.877446 12 storage/store_snapshot.go:965  [n1,replicate,s1,r5/1:/{Systemtse-Table/System…}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:01.878050 4387 storage/replica_raftstorage.go:780  [n2,s2,r5/3:{-}] applying LEARNER snapshot [id=13e1789f index=20]
I200206 15:30:01.881178 4387 storage/replica_raftstorage.go:801  [n2,s2,r5/3:/{Systemtse-Table/System…}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=13e1789f index=20]
I200206 15:30:01.881944 12 storage/replica_command.go:1601  [n1,replicate,s1,r5/1:/{Systemtse-Table/System…}] change replicas (add [(n2,s2):3] remove []): existing descriptor r5:/{Systemtse-Table/SystemConfigSpan/Start} [(n1,s1):1, (n4,s4):2, (n2,s2):3LEARNER, next=4, gen=3]
I200206 15:30:01.896920 12 storage/replica_raft.go:248  [n1,s1,r5/1:/{Systemtse-Table/System…}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3] next=4
I200206 15:30:01.972830 12 storage/replica_command.go:1601  [n2,replicate,s2,r7/2:/Table/1{1-2}] change replicas (add [(n3,s3):4LEARNER] remove []): existing descriptor r7:/Table/1{1-2} [(n1,s1):1, (n2,s2):2, (n4,s4):3, next=4, gen=4]
I200206 15:30:02.073662 12 storage/replica_raft.go:248  [n2,s2,r7/2:/Table/1{1-2}] proposing SIMPLE(l4) ADD_REPLICA[(n3,s3):4LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3 (n3,s3):4LEARNER] next=5
I200206 15:30:02.077054 12 storage/store_snapshot.go:922  [n2,replicate,s2,r7/2:/Table/1{1-2}] sending LEARNER snapshot ded90613 at applied index 32
I200206 15:30:02.077196 12 storage/store_snapshot.go:965  [n2,replicate,s2,r7/2:/Table/1{1-2}] streamed snapshot to (n3,s3):4LEARNER: kv pairs: 11, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:02.077559 4576 storage/raft_snapshot_queue.go:169  [n2,raftsnapshot,s2,r7/2:/Table/1{1-2}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):4LEARNER
I200206 15:30:02.077729 4625 storage/replica_raftstorage.go:780  [n3,s3,r7/4:{-}] applying LEARNER snapshot [id=ded90613 index=32]
I200206 15:30:02.084486 4625 storage/replica_raftstorage.go:801  [n3,s3,r7/4:/Table/1{1-2}] applied LEARNER snapshot [total=7ms ingestion=4@6ms id=ded90613 index=32]
I200206 15:30:02.084990 12 storage/replica_command.go:1601  [n2,replicate,s2,r7/2:/Table/1{1-2}] change replicas (add [(n3,s3):4VOTER_INCOMING] remove [(n1,s1):1VOTER_DEMOTING]): existing descriptor r7:/Table/1{1-2} [(n1,s1):1, (n2,s2):2, (n4,s4):3, (n3,s3):4LEARNER, next=5, gen=5]
I200206 15:30:02.099997 12 storage/replica_raft.go:248  [n2,s2,r7/2:/Table/1{1-2}] proposing ENTER_JOINT(r1 l1 v4) ADD_REPLICA[(n3,s3):4VOTER_INCOMING], REMOVE_REPLICA[(n1,s1):1VOTER_DEMOTING]: after=[(n1,s1):1VOTER_DEMOTING (n2,s2):2 (n4,s4):3 (n3,s3):4VOTER_INCOMING] next=5
I200206 15:30:02.101054 12 storage/replica_command.go:1601  [n2,replicate,s2,r7/2:/Table/1{1-2}] change replicas (add [] remove []): existing descriptor r7:/Table/1{1-2} [(n1,s1):1VOTER_DEMOTING, (n2,s2):2, (n4,s4):3, (n3,s3):4VOTER_INCOMING, next=5, gen=6]
I200206 15:30:02.178433 12 storage/replica_raft.go:248  [n2,s2,r7/2:/Table/1{1-2}] proposing LEAVE_JOINT: after=[(n1,s1):1LEARNER (n2,s2):2 (n4,s4):3 (n3,s3):4] next=5
I200206 15:30:02.179391 12 storage/replica_command.go:1601  [n2,replicate,s2,r7/2:/Table/1{1-2}] change replicas (add [] remove [(n1,s1):1LEARNER]): existing descriptor r7:/Table/1{1-2} [(n1,s1):1LEARNER, (n2,s2):2, (n4,s4):3, (n3,s3):4, next=5, gen=7]
I200206 15:30:02.195041 12 storage/replica_raft.go:248  [n2,s2,r7/2:/Table/1{1-2}] proposing SIMPLE(r1) REMOVE_REPLICA[(n1,s1):1LEARNER]: after=[(n3,s3):4 (n2,s2):2 (n4,s4):3] next=5
I200206 15:30:02.196277 138 storage/store_remove_replica.go:122  [n1,s1,r7/1:/Table/1{1-2}] removing replica r7/1
I200206 15:30:02.197486 12 storage/replica_command.go:1601  [n2,replicate,s2,r22/2:/Table/2{6-7}] change replicas (add [(n4,s4):4LEARNER] remove []): existing descriptor r22:/Table/2{6-7} [(n1,s1):1, (n2,s2):2, (n3,s3):3, next=4, gen=4]
I200206 15:30:02.203445 12 storage/replica_raft.go:248  [n2,s2,r22/2:/Table/2{6-7}] proposing SIMPLE(l4) ADD_REPLICA[(n4,s4):4LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3 (n4,s4):4LEARNER] next=5
I200206 15:30:02.205885 12 storage/store_snapshot.go:922  [n2,replicate,s2,r22/2:/Table/2{6-7}] sending LEARNER snapshot 21a6ee2c at applied index 28
I200206 15:30:02.206005 12 storage/store_snapshot.go:965  [n2,replicate,s2,r22/2:/Table/2{6-7}] streamed snapshot to (n4,s4):4LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:02.206171 4700 storage/raft_snapshot_queue.go:169  [n2,raftsnapshot,s2,r22/2:/Table/2{6-7}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):4LEARNER
I200206 15:30:02.206682 4690 storage/replica_raftstorage.go:780  [n4,s4,r22/4:{-}] applying LEARNER snapshot [id=21a6ee2c index=28]
I200206 15:30:02.211088 4690 storage/replica_raftstorage.go:801  [n4,s4,r22/4:/Table/2{6-7}] applied LEARNER snapshot [total=4ms ingestion=4@4ms id=21a6ee2c index=28]
I200206 15:30:02.215168 12 storage/replica_command.go:1601  [n2,replicate,s2,r22/2:/Table/2{6-7}] change replicas (add [(n4,s4):4VOTER_INCOMING] remove [(n1,s1):1VOTER_DEMOTING]): existing descriptor r22:/Table/2{6-7} [(n1,s1):1, (n2,s2):2, (n3,s3):3, (n4,s4):4LEARNER, next=5, gen=5]
I200206 15:30:02.234315 12 storage/replica_raft.go:248  [n2,s2,r22/2:/Table/2{6-7}] proposing ENTER_JOINT(r1 l1 v4) ADD_REPLICA[(n4,s4):4VOTER_INCOMING], REMOVE_REPLICA[(n1,s1):1VOTER_DEMOTING]: after=[(n1,s1):1VOTER_DEMOTING (n2,s2):2 (n3,s3):3 (n4,s4):4VOTER_INCOMING] next=5
I200206 15:30:02.236084 12 storage/replica_command.go:1601  [n2,replicate,s2,r22/2:/Table/2{6-7}] change replicas (add [] remove []): existing descriptor r22:/Table/2{6-7} [(n1,s1):1VOTER_DEMOTING, (n2,s2):2, (n3,s3):3, (n4,s4):4VOTER_INCOMING, next=5, gen=6]
I200206 15:30:02.245057 12 storage/replica_raft.go:248  [n2,s2,r22/2:/Table/2{6-7}] proposing LEAVE_JOINT: after=[(n1,s1):1LEARNER (n2,s2):2 (n3,s3):3 (n4,s4):4] next=5
I200206 15:30:02.246079 12 storage/replica_command.go:1601  [n2,replicate,s2,r22/2:/Table/2{6-7}] change replicas (add [] remove [(n1,s1):1LEARNER]): existing descriptor r22:/Table/2{6-7} [(n1,s1):1LEARNER, (n2,s2):2, (n3,s3):3, (n4,s4):4, next=5, gen=7]
I200206 15:30:02.258687 12 storage/replica_raft.go:248  [n2,s2,r22/2:/Table/2{6-7}] proposing SIMPLE(r1) REMOVE_REPLICA[(n1,s1):1LEARNER]: after=[(n4,s4):4 (n2,s2):2 (n3,s3):3] next=5
I200206 15:30:02.260362 177 storage/store_remove_replica.go:122  [n1,s1,r22/1:/Table/2{6-7}] removing replica r22/1
I200206 15:30:02.263048 12 storage/replica_command.go:1601  [n4,replicate,s4,r26/3:/NamespaceTable/{30-Max}] change replicas (add [(n3,s3):4LEARNER] remove []): existing descriptor r26:/NamespaceTable/{30-Max} [(n1,s1):1, (n2,s2):2, (n4,s4):3, next=4, gen=4]
I200206 15:30:02.327222 12 storage/replica_raft.go:248  [n4,s4,r26/3:/NamespaceTable/{30-Max}] proposing SIMPLE(l4) ADD_REPLICA[(n3,s3):4LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3 (n3,s3):4LEARNER] next=5
I200206 15:30:02.329160 12 storage/store_snapshot.go:922  [n4,replicate,s4,r26/3:/NamespaceTable/{30-Max}] sending LEARNER snapshot f8feb64c at applied index 35
I200206 15:30:02.329204 4763 storage/raft_snapshot_queue.go:169  [n4,raftsnapshot,s4,r26/3:/NamespaceTable/{30-Max}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):4LEARNER
I200206 15:30:02.329587 12 storage/store_snapshot.go:965  [n4,replicate,s4,r26/3:/NamespaceTable/{30-Max}] streamed snapshot to (n3,s3):4LEARNER: kv pairs: 38, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:02.329986 4720 storage/replica_raftstorage.go:780  [n3,s3,r26/4:{-}] applying LEARNER snapshot [id=f8feb64c index=35]
I200206 15:30:02.333404 4720 storage/replica_raftstorage.go:801  [n3,s3,r26/4:/NamespaceTable/{30-Max}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=f8feb64c index=35]
I200206 15:30:02.334133 12 storage/replica_command.go:1601  [n4,replicate,s4,r26/3:/NamespaceTable/{30-Max}] change replicas (add [(n3,s3):4VOTER_INCOMING] remove [(n1,s1):1VOTER_DEMOTING]): existing descriptor r26:/NamespaceTable/{30-Max} [(n1,s1):1, (n2,s2):2, (n4,s4):3, (n3,s3):4LEARNER, next=5, gen=5]
I200206 15:30:02.347475 12 storage/replica_raft.go:248  [n4,s4,r26/3:/NamespaceTable/{30-Max}] proposing ENTER_JOINT(r1 l1 v4) ADD_REPLICA[(n3,s3):4VOTER_INCOMING], REMOVE_REPLICA[(n1,s1):1VOTER_DEMOTING]: after=[(n1,s1):1VOTER_DEMOTING (n2,s2):2 (n4,s4):3 (n3,s3):4VOTER_INCOMING] next=5
I200206 15:30:02.348441 12 storage/replica_command.go:1601  [n4,replicate,s4,r26/3:/NamespaceTable/{30-Max}] change replicas (add [] remove []): existing descriptor r26:/NamespaceTable/{30-Max} [(n1,s1):1VOTER_DEMOTING, (n2,s2):2, (n4,s4):3, (n3,s3):4VOTER_INCOMING, next=5, gen=6]
I200206 15:30:02.356417 12 storage/replica_raft.go:248  [n4,s4,r26/3:/NamespaceTable/{30-Max}] proposing LEAVE_JOINT: after=[(n1,s1):1LEARNER (n2,s2):2 (n4,s4):3 (n3,s3):4] next=5
I200206 15:30:02.357189 12 storage/replica_command.go:1601  [n4,replicate,s4,r26/3:/NamespaceTable/{30-Max}] change replicas (add [] remove [(n1,s1):1LEARNER]): existing descriptor r26:/NamespaceTable/{30-Max} [(n1,s1):1LEARNER, (n2,s2):2, (n4,s4):3, (n3,s3):4, next=5, gen=7]
I200206 15:30:02.369016 12 storage/replica_raft.go:248  [n4,s4,r26/3:/NamespaceTable/{30-Max}] proposing SIMPLE(r1) REMOVE_REPLICA[(n1,s1):1LEARNER]: after=[(n3,s3):4 (n2,s2):2 (n4,s4):3] next=5
I200206 15:30:02.370306 173 storage/store_remove_replica.go:122  [n1,s1,r26/1:/NamespaceTable/{30-Max}] removing replica r26/1
I200206 15:30:02.381933 12 testutils/testcluster/testcluster.go:760  WaitForFullReplication took: 2.744847576s
I200206 15:30:02.437619 4825 sql/event_log.go:132  [n1,client=127.0.0.1:54138,user=root] Event: "set_cluster_setting", target: 0, info: {SettingName:kv.rangefeed.enabled Value:true User:root}
I200206 15:30:02.439228 12 kv/kvnemesis/watcher.go:131  watcher waiting for 1581003002.439192645,0
I200206 15:30:02.439696 4938 kv/kvnemesis/watcher.go:197  watcher reached frontier 1581003000.112853297,0 lagging by 2.326822526s
I200206 15:30:02.457464 4825 sql/event_log.go:132  [n1,client=127.0.0.1:54138,user=root] Event: "set_cluster_setting", target: 0, info: {SettingName:kv.closed_timestamp.target_duration Value:00:00:00.001 User:root}
I200206 15:30:02.460274 202 storage/replica_rangefeed.go:609  [n1,s1,r28/1:/{Table/32-Max}] RangeFeed closed timestamp 1581003000.112853297,0 is behind by 2.347380023s
I200206 15:30:02.462274 4938 kv/kvnemesis/watcher.go:197  watcher reached frontier 1581003002.458888518,0 lagging by 3.370138ms
I200206 15:30:02.462303 4938 kv/kvnemesis/watcher.go:203  watcher notifying 1581003002.439192645,0
I200206 15:30:02.464565 4938 kv/kvnemesis/watcher.go:197  watcher reached frontier 1581003002.459184996,0 lagging by 5.366646ms
I200206 15:30:02.467218 4938 kv/kvnemesis/watcher.go:197  watcher reached frontier 1581003002.462137840,0 lagging by 5.067058ms
I200206 15:30:02.468864 4938 kv/kvnemesis/watcher.go:197  watcher reached frontier 1581003002.464231071,0 lagging by 4.617029ms
I200206 15:30:02.469496 4938 kv/kvnemesis/watcher.go:197  watcher reached frontier 1581003002.466134575,0 lagging by 3.347063ms
I200206 15:30:02.470537 4938 kv/kvnemesis/watcher.go:197  watcher reached frontier 1581003002.468306106,0 lagging by 2.216788ms
I200206 15:30:02.477101 4938 kv/kvnemesis/watcher.go:197  watcher reached frontier 1581003002.470634030,0 lagging by 6.453501ms
I200206 15:30:02.479410 4938 kv/kvnemesis/watcher.go:197  watcher reached frontier 1581003002.470946569,0 lagging by 8.450786ms
I200206 15:30:02.481470 4938 kv/kvnemesis/watcher.go:197  watcher reached frontier 1581003002.477699108,0 lagging by 3.756668ms
I200206 15:30:02.481860 4938 kv/kvnemesis/watcher.go:197  watcher reached frontier 1581003002.477966789,0 lagging by 3.878713ms
I200206 15:30:02.482737 4938 kv/kvnemesis/watcher.go:197  watcher reached frontier 1581003002.480740093,0 lagging by 1.957723ms
I200206 15:30:02.485243 4938 kv/kvnemesis/watcher.go:197  watcher reached frontier 1581003002.481585655,0 lagging by 3.643806ms
I200206 15:30:02.486407 4938 kv/kvnemesis/watcher.go:197  watcher reached frontier 1581003002.481938746,0 lagging by 4.454914ms
I200206 15:30:02.487765 4938 kv/kvnemesis/watcher.go:197  watcher reached frontier 1581003002.485791872,0 lagging by 1.960375ms
I200206 15:30:02.498902 4938 kv/kvnemesis/watcher.go:197  watcher reached frontier 1581003002.496758197,0 lagging by 2.12773ms
I200206 15:30:02.502095 4825 sql/event_log.go:132  [n1,client=127.0.0.1:54138,user=root] Event: "set_cluster_setting", target: 0, info: {SettingName:kv.closed_timestamp.target_duration Value:DEFAULT User:root}
I200206 15:30:02.506993 5529 kv/kvnemesis/kvnemesis.go:63
  1 OP  db1.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  1 OP    txn.Get(ctx, /Table/50/"12e0a088") // (nil, nil)
  1 OP    return nil
  1 OP  }) // nil
I200206 15:30:02.507268 5531 kv/kvnemesis/kvnemesis.go:63
  3 OP  db0.Put(ctx, /Table/50/"aa639ab9", v-8) // nil
I200206 15:30:02.507408 5531 kv/kvnemesis/kvnemesis.go:63
  3 OP  db2.Get(ctx, /Table/50/"aa639ab9") // ("v-8", nil)
I200206 15:30:02.507575 5099 storage/rangefeed/processor.go:580  [n1,s1,r28/1:/{Table/32-Max},rangefeed] publishValue /Table/50/"4742afb5":1581003002.505260319,0 -> {h���v-7 0,0}
I200206 15:30:02.507618 5099 storage/rangefeed/processor.go:580  [n1,s1,r28/1:/{Table/32-Max},rangefeed] publishValue /Table/50/"aa639ab9":1581003002.505622371,0 -> {�h�Rv-8 0,0}
I200206 15:30:02.507672 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"4742afb5" 1581003002.505260319,0 -> /BYTES/v-7 (prev /)
I200206 15:30:02.507707 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"aa639ab9" 1581003002.505622371,0 -> /BYTES/v-8 (prev /)
I200206 15:30:02.507575 5530 kv/kvnemesis/kvnemesis.go:63
  2 OP  db3.Put(ctx, /Table/50/"4742afb5", v-7) // nil
I200206 15:30:02.507766 5529 kv/kvnemesis/kvnemesis.go:63
  1 OP  db1.AdminMerge(ctx, /Table/50/"f6c86655") // storage/replica_command.go:776: merge failed: cannot merge final range
I200206 15:30:02.508039 5528 kv/kvnemesis/kvnemesis.go:63
  0 OP  db2.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  0 OP    txn.Get(ctx, /Table/50/"2fe7652d") // (nil, nil)
  0 OP    txn.Get(ctx, /Table/50/"0d027936") // (nil, nil)
  0 OP    txn.Get(ctx, /Table/50/"2ec0477d") // (nil, nil)
  0 OP    b := &Batch{}
  0 OP    b.Put(ctx, /Table/50/"d63e2120", v-5) // nil
  0 OP    b.Put(ctx, /Table/50/"4742afb5", v-6) // nil
  0 OP    txn.CommitInBatch(ctx, b) // nil
  0 OP    return nil
  0 OP  }) // nil
I200206 15:30:02.508341 5099 storage/rangefeed/processor.go:580  [n1,s1,r28/1:/{Table/32-Max},rangefeed] publishValue /Table/50/"d63e2120":1581003002.505260319,1 -> {'�Hv-5 0,0}
I200206 15:30:02.508378 5099 storage/rangefeed/processor.go:580  [n1,s1,r28/1:/{Table/32-Max},rangefeed] publishValue /Table/50/"4742afb5":1581003002.505260319,1 -> {�OWv-6 0,0}
I200206 15:30:02.508416 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"d63e2120" 1581003002.505260319,1 -> /BYTES/v-5 (prev /)
I200206 15:30:02.508439 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"4742afb5" 1581003002.505260319,1 -> /BYTES/v-6 (prev /BYTES/v-7)
I200206 15:30:02.509558 5099 storage/rangefeed/processor.go:580  [n1,s1,r28/1:/{Table/32-Max},rangefeed] publishValue /Table/50/"4742afb5":1581003002.507695025,0 -> {�R�v-9 0,0}
I200206 15:30:02.519570 5528 kv/kvnemesis/kvnemesis.go:63
  0 OP  db2.Get(ctx, /Table/50/"2ec0477d") // ("v-10", nil)
I200206 15:30:02.509855 5531 kv/kvnemesis/kvnemesis.go:63
  3 OP  db3.Put(ctx, /Table/50/"4742afb5", v-9) // nil
I200206 15:30:02.512407 5530 kv/kvnemesis/kvnemesis.go:63
  2 OP  db0.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  2 OP    txn.Put(ctx, /Table/50/"2ec0477d", v-10) // nil
  2 OP    txn.Put(ctx, /Table/50/"4742afb5", v-11) // nil
  2 OP    return nil
  2 OP  }) // nil
I200206 15:30:02.520575 5530 kv/kvnemesis/kvnemesis.go:63
  2 OP  db1.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  2 OP    b := &Batch{}
  2 OP    b.Get(ctx, /Table/50/"d17eb9f3") // (nil, nil)
  2 OP    txn.CommitInBatch(ctx, b) // nil
  2 OP    return nil
  2 OP  }) // nil
I200206 15:30:02.520750 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"4742afb5" 1581003002.507695025,0 -> /BYTES/v-9 (prev /BYTES/v-6)
I200206 15:30:02.520778 5099 storage/rangefeed/processor.go:580  [n1,s1,r28/1:/{Table/32-Max},rangefeed] publishValue /Table/50/"2ec0477d":1581003002.507757974,0 -> {̡2�v-10 0,0}
I200206 15:30:02.520801 5099 storage/rangefeed/processor.go:580  [n1,s1,r28/1:/{Table/32-Max},rangefeed] publishValue /Table/50/"4742afb5":1581003002.507757974,0 -> {9�hv-11 0,0}
I200206 15:30:02.520830 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"2ec0477d" 1581003002.507757974,0 -> /BYTES/v-10 (prev /)
I200206 15:30:02.520847 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"4742afb5" 1581003002.507757974,0 -> /BYTES/v-11 (prev /BYTES/v-9)
I200206 15:30:02.521015 5531 kv/kvnemesis/kvnemesis.go:63
  3 OP  db0.Put(ctx, /Table/50/"2ec0477d", v-12) // nil
I200206 15:30:02.521091 5099 storage/rangefeed/processor.go:580  [n1,s1,r28/1:/{Table/32-Max},rangefeed] publishValue /Table/50/"2ec0477d":1581003002.520387801,0 -> {"�S�v-12 0,0}
I200206 15:30:02.521130 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"2ec0477d" 1581003002.520387801,0 -> /BYTES/v-12 (prev /BYTES/v-10)
I200206 15:30:02.524076 5531 kv/kvnemesis/kvnemesis.go:63
  3 OP  db3.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  3 OP    txn.Get(ctx, /Table/50/"4742afb5") // ("v-11", nil)
  3 OP    txn.Get(ctx, /Table/50/"03670547") // (nil, nil)
  3 OP    b := &Batch{}
  3 OP    b.Put(ctx, /Table/50/"ecc9fb13", v-14) // nil
  3 OP    txn.CommitInBatch(ctx, b) // nil
  3 OP    return nil
  3 OP  }) // nil
I200206 15:30:02.524123 5099 storage/rangefeed/processor.go:580  [n1,s1,r28/1:/{Table/32-Max},rangefeed] publishValue /Table/50/"ecc9fb13":1581003002.521041425,0 -> {�)�Qv-14 0,0}
I200206 15:30:02.524171 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"ecc9fb13" 1581003002.521041425,0 -> /BYTES/v-14 (prev /)
I200206 15:30:02.528565 5531 kv/kvnemesis/kvnemesis.go:63
  3 OP  db0.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  3 OP    txn.Get(ctx, /Table/50/"2ec0477d") // ("v-12", nil)
  3 OP    txn.Put(ctx, /Table/50/"4742afb5", v-15) // nil
  3 OP    txn.Get(ctx, /Table/50/"6cc8cc58") // (nil, nil)
  3 OP    return nil
  3 OP  }) // nil
I200206 15:30:02.529901 5532 kv/kvnemesis/kvnemesis.go:63
  4 OP  db0.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  4 OP    txn.Put(ctx, /Table/50/"342b25a0", v-0) // nil
  4 OP    txn.Put(ctx, /Table/50/"aa639ab9", v-1) // nil
  4 OP    txn.Put(ctx, /Table/50/"d63e2120", v-2) // nil
  4 OP    b := &Batch{}
  4 OP    b.Put(ctx, /Table/50/"d63e2120", v-3) // nil
  4 OP    b.Get(ctx, /Table/50/"16e746da") // (nil, nil)
  4 OP    b.Put(ctx, /Table/50/"2ec0477d", v-4) // nil
  4 OP    txn.CommitInBatch(ctx, b) // nil
  4 OP    return nil
  4 OP  }) // nil
I200206 15:30:02.530034 5099 storage/rangefeed/processor.go:580  [n1,s1,r28/1:/{Table/32-Max},rangefeed] publishValue /Table/50/"4742afb5":1581003002.524113383,0 -> {>��qv-15 0,0}
I200206 15:30:02.530449 5099 storage/rangefeed/processor.go:580  [n1,s1,r28/1:/{Table/32-Max},rangefeed] publishValue /Table/50/"2ec0477d":1581003002.524113383,1 -> {�E�-v-4 0,0}
I200206 15:30:02.530472 5099 storage/rangefeed/processor.go:580  [n1,s1,r28/1:/{Table/32-Max},rangefeed] publishValue /Table/50/"342b25a0":1581003002.524113383,1 -> {|
                                                                                                                                                                        {���v-1 0,0}
I200206 15:30:02.530531 5099 storage/rangefeed/processor.go:580  [n1,s1,r28/1:/{Table/32-Max},rangefeed] publishValue /Table/50/"d63e2120":1581003002.524113383,1 -> {�DE}v-3 0,0}
I200206 15:30:02.530592 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"4742afb5" 1581003002.524113383,0 -> /BYTES/v-15 (prev /BYTES/v-11)
I200206 15:30:02.530620 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"2ec0477d" 1581003002.524113383,1 -> /BYTES/v-4 (prev /BYTES/v-12)
I200206 15:30:02.530641 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"342b25a0" 1581003002.524113383,1 -> /BYTES/v-0 (prev /)
I200206 15:30:02.530680 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"aa639ab9" 1581003002.524113383,1 -> /BYTES/v-1 (prev /BYTES/v-8)
I200206 15:30:02.530703 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"d63e2120" 1581003002.524113383,1 -> /BYTES/v-3 (prev /BYTES/v-5)
I200206 15:30:02.540213 5530 kv/kvnemesis/kvnemesis.go:63
  2 OP  db2.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  2 OP    txn.Put(ctx, /Table/50/"d63e2120", v-13) // nil
  2 OP    return nil
  2 OP  }) // nil
I200206 15:30:02.544429 5099 storage/rangefeed/processor.go:580  [n1,s1,r28/1:/{Table/32-Max},rangefeed] publishValue /Table/50/"d63e2120":1581003002.524113383,2 -> {q���v-13 0,0}
I200206 15:30:02.544481 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"d63e2120" 1581003002.524113383,2 -> /BYTES/v-13 (prev /BYTES/v-3)
I200206 15:30:02.548574 5530 kv/kvnemesis/kvnemesis.go:63
  2 OP  db3.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  2 OP    txn.Get(ctx, /Table/50/"2ec0477d") // ("v-4", nil)
  2 OP    txn.Get(ctx, /Table/50/"443025cb") // (nil, nil)
  2 OP    txn.Put(ctx, /Table/50/"342b25a0", v-16) // nil
  2 OP    return errors.New("rollback")
  2 OP  }) // rollback
I200206 15:30:02.548658 5530 kv/kvnemesis/kvnemesis.go:63
  2 OP  db0.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  2 OP    return errors.New("rollback")
  2 OP  }) // rollback
I200206 15:30:02.570767 5584 storage/replica_command.go:396  [n3,s3,r28/2:/{Table/32-Max}] initiating a split of this range at key /Table/50/"b90affa3" [r33] (manual)
I200206 15:30:02.571638 5543 storage/replica_command.go:396  [n3,s3,r28/2:/{Table/32-Max}] initiating a split of this range at key /Table/50/"501a7034" [r29] (manual)
I200206 15:30:02.571751 5356 storage/replica_command.go:396  [n3,s3,r28/2:/{Table/32-Max}] initiating a split of this range at key /Table/50/"501a7034" [r30] (manual)
I200206 15:30:02.571828 5532 storage/replica_command.go:396  [n3,s3,r28/2:/{Table/32-Max}] initiating a split of this range at key /Table/50/"d4ac286f" [r31] (manual)
I200206 15:30:02.571901 5374 storage/replica_command.go:396  [n3,s3,r28/2:/{Table/32-Max}] initiating a split of this range at key /Table/50/"b90affa3" [r32] (manual)
I200206 15:30:02.585480 5530 kv/kvnemesis/kvnemesis.go:63
  2 OP  db1.AdminSplit(ctx, /Table/50/"b90affa3") // nil
I200206 15:30:02.585758 5530 storage/replica_command.go:396  [n3,s3,r33/2:/{Table/50/"b9…-Max}] initiating a split of this range at key /Table/50/"d4ac286f" [r34] (manual)
I200206 15:30:02.593396 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"d63e2120" 1581003002.505260319,1 -> /BYTES/v-5 (prev /)
I200206 15:30:02.593427 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"d63e2120" 1581003002.524113383,1 -> /BYTES/v-3 (prev /BYTES/v-5)
I200206 15:30:02.593445 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"d63e2120" 1581003002.524113383,2 -> /BYTES/v-13 (prev /BYTES/v-3)
I200206 15:30:02.593460 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"ecc9fb13" 1581003002.521041425,0 -> /BYTES/v-14 (prev /)
I200206 15:30:02.595813 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"2ec0477d" 1581003002.507757974,0 -> /BYTES/v-10 (prev /)
I200206 15:30:02.595939 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"2ec0477d" 1581003002.520387801,0 -> /BYTES/v-12 (prev /BYTES/v-10)
I200206 15:30:02.595976 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"2ec0477d" 1581003002.524113383,1 -> /BYTES/v-4 (prev /BYTES/v-12)
I200206 15:30:02.596006 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"342b25a0" 1581003002.524113383,1 -> /BYTES/v-0 (prev /)
I200206 15:30:02.596037 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"4742afb5" 1581003002.505260319,0 -> /BYTES/v-7 (prev /)
I200206 15:30:02.596066 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"4742afb5" 1581003002.505260319,1 -> /BYTES/v-6 (prev /BYTES/v-7)
I200206 15:30:02.596093 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"4742afb5" 1581003002.507695025,0 -> /BYTES/v-9 (prev /BYTES/v-6)
I200206 15:30:02.596120 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"4742afb5" 1581003002.507757974,0 -> /BYTES/v-11 (prev /BYTES/v-9)
I200206 15:30:02.596147 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"4742afb5" 1581003002.524113383,0 -> /BYTES/v-15 (prev /BYTES/v-11)
I200206 15:30:02.596174 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"aa639ab9" 1581003002.505622371,0 -> /BYTES/v-8 (prev /)
I200206 15:30:02.596200 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"aa639ab9" 1581003002.524113383,1 -> /BYTES/v-1 (prev /BYTES/v-8)
I200206 15:30:02.616203 5530 kv/kvnemesis/kvnemesis.go:63
  2 OP  db2.AdminSplit(ctx, /Table/50/"d4ac286f") // nil
I200206 15:30:02.616805 5530 kv/kvnemesis/kvnemesis.go:63
  2 OP  db3.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  2 OP    b := &Batch{}
  2 OP    b.Get(ctx, /Table/50/"93e944df") // (nil, nil)
  2 OP    txn.CommitInBatch(ctx, b) // nil
  2 OP    return nil
  2 OP  }) // nil
I200206 15:30:02.624150 5530 kv/kvnemesis/kvnemesis.go:63
  2 OP  db0.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  2 OP    txn.Put(ctx, /Table/50/"f01fb10c", v-17) // nil
  2 OP    return nil
  2 OP  }) // nil
I200206 15:30:02.625738 5530 kv/kvnemesis/kvnemesis.go:63
  2 OP  db1.Get(ctx, /Table/50/"aa639ab9") // ("v-1", nil)
I200206 15:30:02.628535 5530 kv/kvnemesis/kvnemesis.go:63
  2 OP  db2.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  2 OP    txn.Put(ctx, /Table/50/"ecc9fb13", v-18) // nil
  2 OP    txn.Put(ctx, /Table/50/"867a0fe9", v-19) // nil
  2 OP    b := &Batch{}
  2 OP    b.Put(ctx, /Table/50/"4742afb5", v-20) // nil
  2 OP    b.Get(ctx, /Table/50/"4ac24129") // (nil, nil)
  2 OP    txn.CommitInBatch(ctx, b) // nil
  2 OP    return nil
  2 OP  }) // nil
I200206 15:30:02.629982 5532 kv/kvnemesis/kvnemesis.go:63
  4 OP  db2.AdminSplit(ctx, /Table/50/"d4ac286f") // nil
I200206 15:30:02.631327 5623 storage/replica_command.go:1601  [n3,s3,r28/2:/Table/{32-50/"b90af…}] change replicas (add [(n2,s2):4LEARNER] remove []): existing descriptor r28:/Table/{32-50/"b90affa3"} [(n1,s1):1, (n3,s3):2, (n4,s4):3, next=4, gen=5]
I200206 15:30:02.640993 5675 storage/rangefeed/processor.go:580  [n1,s1,r28/1:/Table/{32-50/"b90af…},rangefeed] publishValue /Table/50/"4742afb5":1581003002.625775977,0 -> {e�~=v-20 0,0}
I200206 15:30:02.641032 5675 storage/rangefeed/processor.go:580  [n1,s1,r28/1:/Table/{32-50/"b90af…},rangefeed] publishValue /Table/50/"867a0fe9":1581003002.625775977,0 -> {�}C�v-19 0,0}
I200206 15:30:02.641068 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"4742afb5" 1581003002.625775977,0 -> /BYTES/v-20 (prev /BYTES/v-15)
I200206 15:30:02.641088 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"867a0fe9" 1581003002.625775977,0 -> /BYTES/v-19 (prev /)
I200206 15:30:02.642075 5532 kv/kvnemesis/kvnemesis.go:63
  4 OP  db0.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  4 OP    txn.Get(ctx, /Table/50/"867a0fe9") // ("v-19", nil)
  4 OP    b := &Batch{}
  4 OP    txn.CommitInBatch(ctx, b) // nil
  4 OP    return nil
  4 OP  }) // nil
I200206 15:30:02.647063 5531 kv/kvnemesis/kvnemesis.go:63
  3 OP  db1.AdminSplit(ctx, /Table/50/"b90affa3") // nil
I200206 15:30:02.648735 5356 storage/replica_command.go:396  [n3,s3,r28/2:/Table/{32-50/"b90af…}] initiating a split of this range at key /Table/50/"501a7034" [r35] (manual)
I200206 15:30:02.656487 5543 storage/replica_command.go:396  [n3,s3,r28/2:/Table/{32-50/"b90af…}] initiating a split of this range at key /Table/50/"501a7034" [r36] (manual)
I200206 15:30:02.671314 5623 storage/replica_raft.go:248  [n3,s3,r28/2:/Table/{32-50/"b90af…}] proposing SIMPLE(l4) ADD_REPLICA[(n2,s2):4LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3 (n2,s2):4LEARNER] next=5
I200206 15:30:02.676126 5623 storage/store_snapshot.go:922  [n3,s3,r28/2:/Table/{32-50/"b90af…}] sending LEARNER snapshot 6a165b59 at applied index 59
I200206 15:30:02.676242 5754 storage/raft_snapshot_queue.go:169  [n3,raftsnapshot,s3,r28/2:/Table/{32-50/"b90af…}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):4LEARNER
I200206 15:30:02.676315 5623 storage/store_snapshot.go:965  [n3,s3,r28/2:/Table/{32-50/"b90af…}] streamed snapshot to (n2,s2):4LEARNER: kv pairs: 25, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200206 15:30:02.676788 5769 storage/replica_raftstorage.go:780  [n2,s2,r28/4:{-}] applying LEARNER snapshot [id=6a165b59 index=59]
I200206 15:30:02.679897 5769 storage/replica_raftstorage.go:801  [n2,s2,r28/4:/Table/{32-50/"b90af…}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=6a165b59 index=59]
I200206 15:30:02.680433 5623 storage/replica_command.go:1601  [n3,s3,r28/2:/Table/{32-50/"b90af…}] change replicas (add [(n2,s2):4VOTER_INCOMING] remove [(n4,s4):3VOTER_DEMOTING]): existing descriptor r28:/Table/{32-50/"b90affa3"} [(n1,s1):1, (n3,s3):2, (n4,s4):3, (n2,s2):4LEARNER, next=5, gen=6]
I200206 15:30:02.709543 5623 storage/replica_raft.go:248  [n3,s3,r28/2:/Table/{32-50/"b90af…}] proposing ENTER_JOINT(r3 l3 v4) ADD_REPLICA[(n2,s2):4VOTER_INCOMING], REMOVE_REPLICA[(n4,s4):3VOTER_DEMOTING]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3VOTER_DEMOTING (n2,s2):4VOTER_INCOMING] next=5
I200206 15:30:02.711975 5623 storage/replica_command.go:1601  [n3,s3,r28/2:/Table/{32-50/"b90af…}] change replicas (add [] remove []): existing descriptor r28:/Table/{32-50/"b90affa3"} [(n1,s1):1, (n3,s3):2, (n4,s4):3VOTER_DEMOTING, (n2,s2):4VOTER_INCOMING, next=5, gen=7]
I200206 15:30:02.725966 5623 storage/replica_raft.go:248  [n3,s3,r28/2:/Table/{32-50/"b90af…}] proposing LEAVE_JOINT: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3LEARNER (n2,s2):4] next=5
I200206 15:30:02.729407 5623 storage/replica_command.go:1601  [n3,s3,r28/2:/Table/{32-50/"b90af…}] change replicas (add [] remove [(n4,s4):3LEARNER]): existing descriptor r28:/Table/{32-50/"b90affa3"} [(n1,s1):1, (n3,s3):2, (n4,s4):3LEARNER, (n2,s2):4, next=5, gen=8]
I200206 15:30:02.735456 5532 kv/kvnemesis/kvnemesis.go:63
  4 OP  db1.AdminChangeReplicas(ctx, /Table/50/"5c5a568c", [{REMOVE_REPLICA n2,s2}]) // change replicas of r28 failed: descriptor changed: [expected] r28:/Table/{32-50/"b90affa3"} [(n1,s1):1, (n3,s3):2, (n4,s4):3, (n2,s2):4LEARNER, next=5, gen=6] != [actual] r28:/Table/{32-50/"b90affa3"} [(n1,s1):1, (n3,s3):2, (n4,s4):3VOTER_DEMOTING, (n2,s2):4VOTER_INCOMING, next=5, gen=7]
I200206 15:30:02.741803 5623 storage/replica_raft.go:248  [n3,s3,r28/2:/Table/{32-50/"b90af…}] proposing SIMPLE(r3) REMOVE_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n2,s2):4] next=5
I200206 15:30:02.743168 5530 kv/kvnemesis/kvnemesis.go:63
  2 OP  db3.AdminChangeReplicas(ctx, /Table/50/"3a392574", [{ADD_REPLICA n2,s2} {REMOVE_REPLICA n4,s4}]) // nil
I200206 15:30:02.743424 1808 storage/store_remove_replica.go:122  [n4,s4,r28/3:/Table/{32-50/"b90af…}] removing replica r28/3
I200206 15:30:02.804018 5543 storage/replica_command.go:396  [n3,s3,r28/2:/Table/{32-50/"b90af…}] initiating a split of this range at key /Table/50/"501a7034" [r37] (manual)
I200206 15:30:02.804476 5356 storage/replica_command.go:396  [n3,s3,r28/2:/Table/{32-50/"b90af…}] initiating a split of this range at key /Table/50/"501a7034" [r38] (manual)
I200206 15:30:02.818066 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"2ec0477d" 1581003002.507757974,0 -> /BYTES/v-10 (prev /)
I200206 15:30:02.818097 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"2ec0477d" 1581003002.520387801,0 -> /BYTES/v-12 (prev /BYTES/v-10)
I200206 15:30:02.818117 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"2ec0477d" 1581003002.524113383,1 -> /BYTES/v-4 (prev /BYTES/v-12)
I200206 15:30:02.818132 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"342b25a0" 1581003002.524113383,1 -> /BYTES/v-0 (prev /)
I200206 15:30:02.818146 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"4742afb5" 1581003002.505260319,0 -> /BYTES/v-7 (prev /)
I200206 15:30:02.818163 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"4742afb5" 1581003002.505260319,1 -> /BYTES/v-6 (prev /BYTES/v-7)
I200206 15:30:02.818178 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"4742afb5" 1581003002.507695025,0 -> /BYTES/v-9 (prev /BYTES/v-6)
I200206 15:30:02.818194 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"4742afb5" 1581003002.507757974,0 -> /BYTES/v-11 (prev /BYTES/v-9)
I200206 15:30:02.818209 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"4742afb5" 1581003002.524113383,0 -> /BYTES/v-15 (prev /BYTES/v-11)
I200206 15:30:02.818233 5529 kv/kvnemesis/kvnemesis.go:63
  1 OP  db1.AdminSplit(ctx, /Table/50/"501a7034") // nil
I200206 15:30:02.818242 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"4742afb5" 1581003002.625775977,0 -> /BYTES/v-20 (prev /BYTES/v-15)
I200206 15:30:02.818909 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"867a0fe9" 1581003002.625775977,0 -> /BYTES/v-19 (prev /)
I200206 15:30:02.818946 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"aa639ab9" 1581003002.505622371,0 -> /BYTES/v-8 (prev /)
I200206 15:30:02.818972 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"aa639ab9" 1581003002.524113383,1 -> /BYTES/v-1 (prev /BYTES/v-8)
I200206 15:30:03.081406 5528 kv/kvnemesis/kvnemesis.go:63
  0 OP  db3.AdminSplit(ctx, /Table/50/"501a7034") // nil
I200206 15:30:03.081445 12 kv/kvnemesis/watcher.go:131  watcher waiting for 1581003003.081382768,0
I200206 15:30:03.097870 4825 sql/event_log.go:132  [n1,client=127.0.0.1:54138,user=root] Event: "set_cluster_setting", target: 0, info: {SettingName:kv.closed_timestamp.target_duration Value:00:00:00.001 User:root}
I200206 15:30:03.565624 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"d63e2120" 1581003002.505260319,1 -> /BYTES/v-5 (prev /)
I200206 15:30:03.565666 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"d63e2120" 1581003002.524113383,1 -> /BYTES/v-3 (prev /BYTES/v-5)
I200206 15:30:03.565683 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"d63e2120" 1581003002.524113383,2 -> /BYTES/v-13 (prev /BYTES/v-3)
I200206 15:30:03.565698 4938 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"ecc9fb13" 1581003002.521041425,0 -> /BYTES/v-14 (prev /)
I200206 15:30:03.566302 4938 kv/kvnemesis/watcher.go:197  watcher reached frontier 1581003003.562028727,0 lagging by 4.259746ms
I200206 15:30:03.566319 4938 kv/kvnemesis/watcher.go:203  watcher notifying 1581003003.081382768,0
I200206 15:30:03.605490 4825 sql/event_log.go:132  [n1,client=127.0.0.1:54138,user=root] Event: "set_cluster_setting", target: 0, info: {SettingName:kv.closed_timestamp.target_duration Value:DEFAULT User:root}
I200206 15:30:03.609156 12 kv/kvnemesis/kvnemesis.go:86  reproduction steps:
g := ctxgroup.WithContext(ctx)
g.GoCtx(func(ctx context.Context) error {
  db2.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    txn.Get(ctx, /Table/50/"2fe7652d") // (nil, nil)
    txn.Get(ctx, /Table/50/"0d027936") // (nil, nil)
    txn.Get(ctx, /Table/50/"2ec0477d") // (nil, nil)
    b := &Batch{}
    b.Put(ctx, /Table/50/"d63e2120", v-5) // nil
    b.Put(ctx, /Table/50/"4742afb5", v-6) // nil
    txn.CommitInBatch(ctx, b) // nil
    return nil
  }) // nil
  db2.Get(ctx, /Table/50/"2ec0477d") // ("v-10", nil)
  db3.AdminSplit(ctx, /Table/50/"501a7034") // nil
  return nil
})
g.GoCtx(func(ctx context.Context) error {
  db1.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    txn.Get(ctx, /Table/50/"12e0a088") // (nil, nil)
    return nil
  }) // nil
  db1.AdminMerge(ctx, /Table/50/"f6c86655") // storage/replica_command.go:776: merge failed: cannot merge final range
  db1.AdminSplit(ctx, /Table/50/"501a7034") // nil
  return nil
})
g.GoCtx(func(ctx context.Context) error {
  db3.Put(ctx, /Table/50/"4742afb5", v-7) // nil
  db0.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    txn.Put(ctx, /Table/50/"2ec0477d", v-10) // nil
    txn.Put(ctx, /Table/50/"4742afb5", v-11) // nil
    return nil
  }) // nil
  db1.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    b := &Batch{}
    b.Get(ctx, /Table/50/"d17eb9f3") // (nil, nil)
    txn.CommitInBatch(ctx, b) // nil
    return nil
  }) // nil
  db2.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    txn.Put(ctx, /Table/50/"d63e2120", v-13) // nil
    return nil
  }) // nil
  db3.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    txn.Get(ctx, /Table/50/"2ec0477d") // ("v-4", nil)
    txn.Get(ctx, /Table/50/"443025cb") // (nil, nil)
    txn.Put(ctx, /Table/50/"342b25a0", v-16) // nil
    return errors.New("rollback")
  }) // rollback
  db0.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    return errors.New("rollback")
  }) // rollback
  db1.AdminSplit(ctx, /Table/50/"b90affa3") // nil
  db2.AdminSplit(ctx, /Table/50/"d4ac286f") // nil
  db3.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    b := &Batch{}
    b.Get(ctx, /Table/50/"93e944df") // (nil, nil)
    txn.CommitInBatch(ctx, b) // nil
    return nil
  }) // nil
  db0.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    txn.Put(ctx, /Table/50/"f01fb10c", v-17) // nil
    return nil
  }) // nil
  db1.Get(ctx, /Table/50/"aa639ab9") // ("v-1", nil)
  db2.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    txn.Put(ctx, /Table/50/"ecc9fb13", v-18) // nil
    txn.Put(ctx, /Table/50/"867a0fe9", v-19) // nil
    b := &Batch{}
    b.Put(ctx, /Table/50/"4742afb5", v-20) // nil
    b.Get(ctx, /Table/50/"4ac24129") // (nil, nil)
    txn.CommitInBatch(ctx, b) // nil
    return nil
  }) // nil
  db3.AdminChangeReplicas(ctx, /Table/50/"3a392574", [{ADD_REPLICA n2,s2} {REMOVE_REPLICA n4,s4}]) // nil
  return nil
})
g.GoCtx(func(ctx context.Context) error {
  db0.Put(ctx, /Table/50/"aa639ab9", v-8) // nil
  db2.Get(ctx, /Table/50/"aa639ab9") // ("v-8", nil)
  db3.Put(ctx, /Table/50/"4742afb5", v-9) // nil
  db0.Put(ctx, /Table/50/"2ec0477d", v-12) // nil
  db3.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    txn.Get(ctx, /Table/50/"4742afb5") // ("v-11", nil)
    txn.Get(ctx, /Table/50/"03670547") // (nil, nil)
    b := &Batch{}
    b.Put(ctx, /Table/50/"ecc9fb13", v-14) // nil
    txn.CommitInBatch(ctx, b) // nil
    return nil
  }) // nil
  db0.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    txn.Get(ctx, /Table/50/"2ec0477d") // ("v-12", nil)
    txn.Put(ctx, /Table/50/"4742afb5", v-15) // nil
    txn.Get(ctx, /Table/50/"6cc8cc58") // (nil, nil)
    return nil
  }) // nil
  db1.AdminSplit(ctx, /Table/50/"b90affa3") // nil
  return nil
})
g.GoCtx(func(ctx context.Context) error {
  db0.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    txn.Put(ctx, /Table/50/"342b25a0", v-0) // nil
    txn.Put(ctx, /Table/50/"aa639ab9", v-1) // nil
    txn.Put(ctx, /Table/50/"d63e2120", v-2) // nil
    b := &Batch{}
    b.Put(ctx, /Table/50/"d63e2120", v-3) // nil
    b.Get(ctx, /Table/50/"16e746da") // (nil, nil)
    b.Put(ctx, /Table/50/"2ec0477d", v-4) // nil
    txn.CommitInBatch(ctx, b) // nil
    return nil
  }) // nil
  db2.AdminSplit(ctx, /Table/50/"d4ac286f") // nil
  db0.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    txn.Get(ctx, /Table/50/"867a0fe9") // ("v-19", nil)
    b := &Batch{}
    txn.CommitInBatch(ctx, b) // nil
    return nil
  }) // nil
  db1.AdminChangeReplicas(ctx, /Table/50/"5c5a568c", [{REMOVE_REPLICA n2,s2}]) // change replicas of r28 failed: descriptor changed: [expected] r28:/Table/{32-50/"b90affa3"} [(n1,s1):1, (n3,s3):2, (n4,s4):3, (n2,s2):4LEARNER, next=5, gen=6] != [actual] r28:/Table/{32-50/"b90affa3"} [(n1,s1):1, (n3,s3):2, (n4,s4):3VOTER_DEMOTING, (n2,s2):4VOTER_INCOMING, next=5, gen=7]
  return nil
})
g.Wait()
I200206 15:30:03.609289 12 kv/kvnemesis/kvnemesis.go:87  kvs (recorded from rangefeed):
  /Table/50/"2ec0477d" 1581003002.524113383,1 -> /BYTES/v-4
  /Table/50/"2ec0477d" 1581003002.520387801,0 -> /BYTES/v-12
  /Table/50/"2ec0477d" 1581003002.507757974,0 -> /BYTES/v-10
  /Table/50/"342b25a0" 1581003002.524113383,1 -> /BYTES/v-0
  /Table/50/"4742afb5" 1581003002.625775977,0 -> /BYTES/v-20
  /Table/50/"4742afb5" 1581003002.524113383,0 -> /BYTES/v-15
  /Table/50/"4742afb5" 1581003002.507757974,0 -> /BYTES/v-11
  /Table/50/"4742afb5" 1581003002.507695025,0 -> /BYTES/v-9
  /Table/50/"4742afb5" 1581003002.505260319,1 -> /BYTES/v-6
  /Table/50/"4742afb5" 1581003002.505260319,0 -> /BYTES/v-7
  /Table/50/"867a0fe9" 1581003002.625775977,0 -> /BYTES/v-19
  /Table/50/"aa639ab9" 1581003002.524113383,1 -> /BYTES/v-1
  /Table/50/"aa639ab9" 1581003002.505622371,0 -> /BYTES/v-8
  /Table/50/"d63e2120" 1581003002.524113383,2 -> /BYTES/v-13
  /Table/50/"d63e2120" 1581003002.524113383,1 -> /BYTES/v-3
  /Table/50/"d63e2120" 1581003002.505260319,1 -> /BYTES/v-5
  /Table/50/"ecc9fb13" 1581003002.521041425,0 -> /BYTES/v-14
I200206 15:30:03.614213 12 kv/kvnemesis/kvnemesis.go:98  kvs (scan of latest values according to crdb):
  /Table/50/"2ec0477d" 1581003002.524113383,1 -> /BYTES/v-4
  /Table/50/"342b25a0" 1581003002.524113383,1 -> /BYTES/v-0
  /Table/50/"4742afb5" 1581003002.625775977,0 -> /BYTES/v-20
  /Table/50/"867a0fe9" 1581003002.625775977,0 -> /BYTES/v-19
  /Table/50/"aa639ab9" 1581003002.524113383,1 -> /BYTES/v-1
  /Table/50/"d63e2120" 1581003002.524113383,2 -> /BYTES/v-13
  /Table/50/"ecc9fb13" 1581003002.625775977,0 -> /BYTES/v-18
  /Table/50/"f01fb10c" 1581003002.616876328,0 -> /BYTES/v-17
I200206 15:30:03.614527 12 util/stop/stopper.go:539  quiescing
I200206 15:30:03.614590 6940 util/stop/stopper.go:539  quiescing
I200206 15:30:03.614673 6937 util/stop/stopper.go:539  quiescing
I200206 15:30:03.614789 6938 util/stop/stopper.go:539  quiescing
I200206 15:30:03.614879 6939 util/stop/stopper.go:539  quiescing
W200206 15:30:03.615089 2310 storage/raft_transport.go:637  [n4] while processing outgoing Raft queue to node 2: rpc error: code = Canceled desc = grpc: the client connection is closing:
W200206 15:30:03.615137 2246 storage/raft_transport.go:637  [n4] while processing outgoing Raft queue to node 3: rpc error: code = Canceled desc = grpc: the client connection is closing:
W200206 15:30:03.615268 2889 storage/raft_transport.go:637  [n4] while processing outgoing Raft queue to node 2: rpc error: code = Canceled desc = grpc: the client connection is closing:
W200206 15:30:03.615344 2034 storage/raft_transport.go:637  [n4] while processing outgoing Raft queue to node 1: rpc error: code = Canceled desc = grpc: the client connection is closing:
W200206 15:30:03.615434 2510 storage/raft_transport.go:637  [n4] while processing outgoing Raft queue to node 3: rpc error: code = Canceled desc = grpc: the client connection is closing:
W200206 15:30:03.615513 2039 storage/raft_transport.go:637  [n4] while processing outgoing Raft queue to node 1: rpc error: code = Canceled desc = grpc: the client connection is closing:
W200206 15:30:03.615783 2192 storage/raft_transport.go:637  [n3] while processing outgoing Raft queue to node 4: EOF:
W200206 15:30:03.615894 2291 storage/raft_transport.go:637  [n2] while processing outgoing Raft queue to node 4: EOF:
W200206 15:30:03.615943 2387 storage/raft_transport.go:637  [n2] while processing outgoing Raft queue to node 4: EOF:
W200206 15:30:03.615978 2739 storage/raft_transport.go:637  [n1] while processing outgoing Raft queue to node 4: EOF:
W200206 15:30:03.616013 2513 storage/raft_transport.go:637  [n3] while processing outgoing Raft queue to node 4: EOF:
W200206 15:30:03.616264 1640 gossip/gossip.go:1518  [n4] no incoming or outgoing connections
W200206 15:30:03.619608 1968 storage/raft_transport.go:637  [n1] while processing outgoing Raft queue to node 4: EOF:
W200206 15:30:03.619899 1572 storage/raft_transport.go:637  [n2] while processing outgoing Raft queue to node 1: rpc error: code = Canceled desc = grpc: the client connection is closing:
W200206 15:30:03.620061 3204 storage/raft_transport.go:637  [n2] while processing outgoing Raft queue to node 3: rpc error: code = Canceled desc = grpc: the client connection is closing:
W200206 15:30:03.620160 1993 storage/raft_transport.go:637  [n2] while processing outgoing Raft queue to node 3: rpc error: code = Canceled desc = grpc: the client connection is closing:
W200206 15:30:03.620305 1013 gossip/gossip.go:1518  [n2] no incoming or outgoing connections
W200206 15:30:03.620440 1564 storage/raft_transport.go:637  [n2] while processing outgoing Raft queue to node 1: rpc error: code = Canceled desc = grpc: the client connection is closing:
W200206 15:30:03.620834 5332 storage/intentresolver/intent_resolver.go:820  failed to gc transaction record: could not GC completed transaction anchored at /Table/50/"ecc9fb13": node unavailable; try another peer
W200206 15:30:03.621174 1578 storage/raft_transport.go:637  [n3] while processing outgoing Raft queue to node 1: rpc error: code = Canceled desc = grpc: the client connection is closing:
W200206 15:30:03.621463 1658 storage/raft_transport.go:637  [n1] while processing outgoing Raft queue to node 3: EOF:
W200206 15:30:03.621635 2100 storage/raft_transport.go:637  [n1] while processing outgoing Raft queue to node 3: EOF:
W200206 15:30:03.622063 1343 storage/raft_transport.go:637  [n1] while processing outgoing Raft queue to node 2: rpc error: code = Canceled desc = grpc: the client connection is closing:
W200206 15:30:03.622195 1664 storage/raft_transport.go:637  [n3] while processing outgoing Raft queue to node 1: EOF:
W200206 15:30:03.622355 1208 gossip/gossip.go:1518  [n3] no incoming or outgoing connections
W200206 15:30:03.622592 2101 storage/raft_transport.go:637  [n1] while processing outgoing Raft queue to node 2: rpc error: code = Canceled desc = grpc: the client connection is closing:
W200206 15:30:03.622755 1976 storage/raft_transport.go:637  [n3] while processing outgoing Raft queue to node 2: rpc error: code = Canceled desc = grpc: the client connection is closing:
W200206 15:30:03.622774 7204 storage/intentresolver/intent_resolver.go:820  failed to gc transaction record: could not GC completed transaction anchored at /Table/SystemConfigSpan/Start: node unavailable; try another peer
W200206 15:30:03.623121 2342 storage/raft_transport.go:637  [n3] while processing outgoing Raft queue to node 2: EOF:
I200206 15:30:03.629262 7006 rpc/nodedialer/nodedialer.go:160  [ct-client] unable to connect to n1: failed to connect to n1 at 127.0.0.1:44373: context canceled
I200206 15:30:03.629415 7007 rpc/nodedialer/nodedialer.go:160  [ct-client] unable to connect to n2: failed to connect to n2 at 127.0.0.1:43543: context canceled
I200206 15:30:03.632114 7005 rpc/nodedialer/nodedialer.go:160  [ct-client] unable to connect to n4: failed to connect to n4 at 127.0.0.1:35387: context canceled
--- FAIL: TestKVNemesisMultiNode (4.66s)
    kvnemesis_test.go:79: failure:
        error with attached stack trace:
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.(*validator).checkCommittedTxn
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/validator.go:272
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.(*validator).checkAtomic
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/validator.go:231
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.(*validator).processOp
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/validator.go:217
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.Validate
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/validator.go:69
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.RunNemesis
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/kvnemesis.go:83
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.TestKVNemesisMultiNode
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/kvnemesis_test.go:75
            testing.tRunner
            	/usr/local/go/src/testing/testing.go:909
            runtime.goexit
            	/usr/local/go/src/runtime/asm_amd64.s:1357
          - error with embedded safe details: %s: %s
            -- arg 1: 
            -- arg 2: /Table/50/"f01fb10c":missing->v-17
          - committed txn missing write: /Table/50/"f01fb10c":missing->v-17
    kvnemesis_test.go:79: failure:
        error with attached stack trace:
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.(*validator).checkCommittedTxn
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/validator.go:272
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.(*validator).checkAtomic
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/validator.go:231
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.(*validator).processOp
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/validator.go:217
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.Validate
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/validator.go:69
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.RunNemesis
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/kvnemesis.go:83
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.TestKVNemesisMultiNode
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/kvnemesis_test.go:75
            testing.tRunner
            	/usr/local/go/src/testing/testing.go:909
            runtime.goexit
            	/usr/local/go/src/runtime/asm_amd64.s:1357
          - error with embedded safe details: %s: %s
            -- arg 1: 
            -- arg 2: /Table/50/"4742afb5":1581003002.625775977,0->v-20 /Table/50/"867a0fe9":1581003002.625775977,0->v-19 /Table/50/"ecc9fb13":missing->v-18
          - committed txn missing write: /Table/50/"4742afb5":1581003002.625775977,0->v-20 /Table/50/"867a0fe9":1581003002.625775977,0->v-19 /Table/50/"ecc9fb13":missing->v-18

@danhhz
Copy link
Contributor Author

danhhz commented Feb 7, 2020

Got another repro with more debugging around registrations and de-registrations. Interestingly, this involves keys that had never been written to before, so that seems to have been a coincidence in the previous repros.

Looking at key "4493fa8c", the range splits, and both sides are reconnected. The IntentWrite and IntentCommit are seen by two replicas, both skip over it because there is no processor. Presumably these are the two replicas the rangefeed is not connected to. The third replica doesn't seem to get the logical op. That's one question.

Second, we either shouldn't have advanced the resolved timestamp or we should have gotten the write in a catchup scan. A split happens later (2->3 ranges), but not on the side with any of the 3 missing keys, so we don't see what it would have reconnected at. Not sure why the resolved timestamp is advancing for the span covering the keys that ended up missing.

I200207 16:33:53.912490 1 rand.go:85  Random seed: -3438010863189673986
=== RUN   TestKVNemesisMultiNode
I200207 16:33:53.915726 14 base/addr_validation.go:296  [n?] server certificate addresses: IP=127.0.0.1,::1; DNS=localhost,*.local; CN=node
I200207 16:33:53.915760 14 base/addr_validation.go:342  [n?] web UI certificate addresses: IP=127.0.0.1,::1; DNS=localhost,*.local; CN=node
W200207 16:33:53.939693 14 server/status/runtime.go:308  [n?] Could not parse build timestamp: parsing time "" as "2006/01/02 15:04:05": cannot parse "" as "2006"
I200207 16:33:53.954615 14 server/server.go:988  [n?] monitoring forward clock jumps based on server.clock.forward_jump_check_enabled
I200207 16:33:53.985500 14 server/config.go:565  [n?] 1 storage engine initialized
I200207 16:33:53.985553 14 server/config.go:568  [n?] RocksDB cache size: 128 MiB
I200207 16:33:53.985581 14 server/config.go:568  [n?] store 0: in-memory, size 0 B
I200207 16:33:53.986078 14 server/server.go:1443  [n?] no stores bootstrapped and --join flag specified, awaiting init command or join with an already initialized node.
I200207 16:33:53.987637 14 server/server.go:1453  [n?] **** add additional nodes by specifying --join=127.0.0.1:33107
I200207 16:33:54.001369 14 server/node.go:334  [n?] **** cluster a2dd6d0a-267f-4814-8be5-9559f1904555 has been created
I200207 16:33:54.017773 14 gossip/gossip.go:395  [n1] NodeDescriptor set to node_id:1 address: attrs:<> locality: tiers: > ServerVersion: build_tag:"v20.1.0-alpha.20191118-1912-g1139636-dirty" started_at:1581093234001553726 cluster_name:"" sql_address:
W200207 16:33:54.042242 177 storage/store.go:1538  [n1,s1,r6/1:/Table/{SystemCon…-11}] could not gossip system config: [NotLeaseHolderError] r6: replica (n1,s1):1 not lease holder; lease holder unknown
I200207 16:33:54.042560 14 server/node.go:438  [n1] initialized store [n1,s1]: disk (capacity=512 MiB, available=512 MiB, used=0 B, logicalBytes=13 KiB), ranges=28, leases=2, queries=0.00, writes=0.00, bytesPerReplica={p10=0.00 p25=0.00 p50=0.00 p75=0.00 p90=950.00 pMax=11033.00}, writesPerReplica={p10=0.00 p25=0.00 p50=0.00 p75=0.00 p90=0.00 pMax=0.00}
I200207 16:33:54.042650 14 storage/stores.go:240  [n1] read 0 node addresses from persistent storage
I200207 16:33:54.042759 14 server/node.go:655  [n1] connecting to gossip network to verify cluster ID...
I200207 16:33:54.042784 14 server/node.go:675  [n1] node connected via gossip and verified as part of cluster "a2dd6d0a-267f-4814-8be5-9559f1904555"
I200207 16:33:54.042880 14 server/node.go:522  [n1] node=1: started with [=] engine(s) and attributes []
I200207 16:33:54.042953 14 server/server.go:2131  [n1] Could not start goroutine dumper worker due to: directory to store dumps could not be determined
I200207 16:33:54.043043 14 server/server.go:1573  [n1] starting https server at 127.0.0.1:35245 (use: 127.0.0.1:35245)
I200207 16:33:54.043094 14 server/server.go:1578  [n1] starting postgres server at 127.0.0.1:43443 (use: 127.0.0.1:43443)
I200207 16:33:54.043118 14 server/server.go:1580  [n1] starting grpc server at 127.0.0.1:33107
I200207 16:33:54.043154 14 server/server.go:1581  [n1] advertising CockroachDB node at 127.0.0.1:33107
I200207 16:33:54.128119 317 sql/event_log.go:132  [n1,intExec=optInToDiagnosticsStatReporting] Event: "set_cluster_setting", target: 0, info: {SettingName:diagnostics.reporting.enabled Value:true User:root}
I200207 16:33:54.138090 58 sql/event_log.go:132  [n1,intExec=set-setting] Event: "set_cluster_setting", target: 0, info: {SettingName:version Value:19.2-11 User:root}
I200207 16:33:54.162766 649 sql/event_log.go:132  [n1,intExec=initializeClusterSecret] Event: "set_cluster_setting", target: 0, info: {SettingName:cluster.secret Value:3f837ac2-9fd5-4321-b328-a8cb8043f153 User:root}
I200207 16:33:54.182460 572 sql/event_log.go:132  [n1,intExec=create-default-db] Event: "create_database", target: 50, info: {DatabaseName:defaultdb Statement:CREATE DATABASE IF NOT EXISTS defaultdb User:root}
I200207 16:33:54.216010 739 sql/event_log.go:132  [n1,intExec=create-default-db] Event: "create_database", target: 51, info: {DatabaseName:postgres Statement:CREATE DATABASE IF NOT EXISTS postgres User:root}
I200207 16:33:54.269834 14 server/server.go:1655  [n1] done ensuring all necessary migrations have run
I200207 16:33:54.269888 14 server/server.go:1936  [n1] serving sql connections
I200207 16:33:54.273382 14 base/addr_validation.go:296  [n?] server certificate addresses: IP=127.0.0.1,::1; DNS=localhost,*.local; CN=node
I200207 16:33:54.273409 14 base/addr_validation.go:342  [n?] web UI certificate addresses: IP=127.0.0.1,::1; DNS=localhost,*.local; CN=node
I200207 16:33:54.279741 950 server/server_update.go:55  [n1] no need to upgrade, cluster already at the newest version
I200207 16:33:54.284926 952 sql/event_log.go:132  [n1] Event: "node_join", target: 1, info: {Descriptor:{NodeID:1 Address:127.0.0.1:33107 Attrs: Locality:region=test,dc=dc1 ServerVersion:19.2-11 BuildTag:v20.1.0-alpha.20191118-1912-g1139636-dirty StartedAt:1581093234001553726 LocalityAddress:[] ClusterName: SQLAddress:127.0.0.1:43443} ClusterID:a2dd6d0a-267f-4814-8be5-9559f1904555 StartedAt:1581093234001553726 LastUp:1581093234001553726}
W200207 16:33:54.289587 14 server/status/runtime.go:308  [n?] Could not parse build timestamp: parsing time "" as "2006/01/02 15:04:05": cannot parse "" as "2006"
I200207 16:33:54.322884 14 server/server.go:988  [n?] monitoring forward clock jumps based on server.clock.forward_jump_check_enabled
I200207 16:33:54.370169 14 server/config.go:565  [n?] 1 storage engine initialized
I200207 16:33:54.370219 14 server/config.go:568  [n?] RocksDB cache size: 128 MiB
I200207 16:33:54.370236 14 server/config.go:568  [n?] store 0: in-memory, size 0 B
W200207 16:33:54.370329 14 gossip/gossip.go:1518  [n?] no incoming or outgoing connections
I200207 16:33:54.370392 14 server/server.go:1443  [n?] no stores bootstrapped and --join flag specified, awaiting init command or join with an already initialized node.
I200207 16:33:54.384387 913 gossip/client.go:124  [n?] started gossip client to 127.0.0.1:33107
I200207 16:33:54.384924 987 gossip/server.go:227  [n1] received initial cluster-verification connection from 127.0.0.1:44965
I200207 16:33:54.385581 14 server/node.go:655  [n?] connecting to gossip network to verify cluster ID...
I200207 16:33:54.385621 14 server/node.go:675  [n?] node connected via gossip and verified as part of cluster "a2dd6d0a-267f-4814-8be5-9559f1904555"
I200207 16:33:54.396900 987 gossip/server.go:227  [n1] received initial cluster-verification connection from 127.0.0.1:44965
I200207 16:33:54.408412 14 server/node.go:388  [n?] new node allocated ID 2
I200207 16:33:54.408496 14 gossip/gossip.go:395  [n2] NodeDescriptor set to node_id:2 address: attrs:<> locality: tiers: > ServerVersion: build_tag:"v20.1.0-alpha.20191118-1912-g1139636-dirty" started_at:1581093234408427300 cluster_name:"" sql_address:
I200207 16:33:54.408565 14 storage/stores.go:240  [n2] read 0 node addresses from persistent storage
I200207 16:33:54.408638 14 storage/stores.go:259  [n2] wrote 1 node addresses to persistent storage
I200207 16:33:54.409089 66 storage/stores.go:259  [n1] wrote 1 node addresses to persistent storage
I200207 16:33:54.410405 14 server/node.go:630  [n2] bootstrapped store [n2,s2]
I200207 16:33:54.410531 14 server/node.go:522  [n2] node=2: started with [=] engine(s) and attributes []
I200207 16:33:54.410604 14 server/server.go:2131  [n2] Could not start goroutine dumper worker due to: directory to store dumps could not be determined
I200207 16:33:54.410666 14 server/server.go:1573  [n2] starting https server at 127.0.0.1:44523 (use: 127.0.0.1:44523)
I200207 16:33:54.410699 14 server/server.go:1578  [n2] starting postgres server at 127.0.0.1:37143 (use: 127.0.0.1:37143)
I200207 16:33:54.410721 14 server/server.go:1580  [n2] starting grpc server at 127.0.0.1:44965
I200207 16:33:54.410742 14 server/server.go:1581  [n2] advertising CockroachDB node at 127.0.0.1:44965
I200207 16:33:54.425901 14 server/server.go:1655  [n2] done ensuring all necessary migrations have run
I200207 16:33:54.425946 14 server/server.go:1936  [n2] serving sql connections
I200207 16:33:54.468918 1041 server/server_update.go:55  [n2] no need to upgrade, cluster already at the newest version
I200207 16:33:54.470419 14 base/addr_validation.go:296  [n?] server certificate addresses: IP=127.0.0.1,::1; DNS=localhost,*.local; CN=node
I200207 16:33:54.470456 14 base/addr_validation.go:342  [n?] web UI certificate addresses: IP=127.0.0.1,::1; DNS=localhost,*.local; CN=node
I200207 16:33:54.471323 1171 sql/event_log.go:132  [n2] Event: "node_join", target: 2, info: {Descriptor:{NodeID:2 Address:127.0.0.1:44965 Attrs: Locality:region=test,dc=dc2 ServerVersion:19.2-11 BuildTag:v20.1.0-alpha.20191118-1912-g1139636-dirty StartedAt:1581093234408427300 LocalityAddress:[] ClusterName: SQLAddress:127.0.0.1:37143} ClusterID:a2dd6d0a-267f-4814-8be5-9559f1904555 StartedAt:1581093234408427300 LastUp:1581093234408427300}
W200207 16:33:54.528138 14 server/status/runtime.go:308  [n?] Could not parse build timestamp: parsing time "" as "2006/01/02 15:04:05": cannot parse "" as "2006"
I200207 16:33:54.539308 14 server/server.go:988  [n?] monitoring forward clock jumps based on server.clock.forward_jump_check_enabled
I200207 16:33:54.553282 14 server/config.go:565  [n?] 1 storage engine initialized
I200207 16:33:54.553311 14 server/config.go:568  [n?] RocksDB cache size: 128 MiB
I200207 16:33:54.553321 14 server/config.go:568  [n?] store 0: in-memory, size 0 B
W200207 16:33:54.553402 14 gossip/gossip.go:1518  [n?] no incoming or outgoing connections
I200207 16:33:54.553459 14 server/server.go:1443  [n?] no stores bootstrapped and --join flag specified, awaiting init command or join with an already initialized node.
I200207 16:33:54.568535 1274 gossip/client.go:124  [n?] started gossip client to 127.0.0.1:33107
I200207 16:33:54.568754 1232 gossip/server.go:227  [n1] received initial cluster-verification connection from 127.0.0.1:38505
I200207 16:33:54.570090 14 server/node.go:655  [n?] connecting to gossip network to verify cluster ID...
I200207 16:33:54.570128 14 server/node.go:675  [n?] node connected via gossip and verified as part of cluster "a2dd6d0a-267f-4814-8be5-9559f1904555"
I200207 16:33:54.570989 1232 gossip/server.go:227  [n1] received initial cluster-verification connection from 127.0.0.1:38505
I200207 16:33:54.594569 14 server/node.go:388  [n?] new node allocated ID 3
I200207 16:33:54.594664 14 gossip/gossip.go:395  [n3] NodeDescriptor set to node_id:3 address: attrs:<> locality: tiers: > ServerVersion: build_tag:"v20.1.0-alpha.20191118-1912-g1139636-dirty" started_at:1581093234594591865 cluster_name:"" sql_address:
I200207 16:33:54.594724 14 storage/stores.go:240  [n3] read 0 node addresses from persistent storage
I200207 16:33:54.594762 14 storage/stores.go:259  [n3] wrote 2 node addresses to persistent storage
I200207 16:33:54.595312 66 storage/stores.go:259  [n1] wrote 2 node addresses to persistent storage
I200207 16:33:54.595723 961 storage/stores.go:259  [n2] wrote 2 node addresses to persistent storage
I200207 16:33:54.596323 14 server/node.go:630  [n3] bootstrapped store [n3,s3]
I200207 16:33:54.596447 14 server/node.go:522  [n3] node=3: started with [=] engine(s) and attributes []
I200207 16:33:54.596520 14 server/server.go:2131  [n3] Could not start goroutine dumper worker due to: directory to store dumps could not be determined
I200207 16:33:54.596580 14 server/server.go:1573  [n3] starting https server at 127.0.0.1:38209 (use: 127.0.0.1:38209)
I200207 16:33:54.596622 14 server/server.go:1578  [n3] starting postgres server at 127.0.0.1:39083 (use: 127.0.0.1:39083)
I200207 16:33:54.596643 14 server/server.go:1580  [n3] starting grpc server at 127.0.0.1:38505
I200207 16:33:54.596662 14 server/server.go:1581  [n3] advertising CockroachDB node at 127.0.0.1:38505
I200207 16:33:54.602381 301 storage/replica_command.go:1601  [n1,replicate,s1,r27/1:/{NamespaceTab…-Table/32}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r27:/{NamespaceTable/Max-Table/32} [(n1,s1):1, next=2, gen=0]
I200207 16:33:54.608165 301 storage/replica_raft.go:248  [n1,s1,r27/1:/{NamespaceTab…-Table/32}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200207 16:33:54.614699 14 server/server.go:1655  [n3] done ensuring all necessary migrations have run
I200207 16:33:54.614825 14 server/server.go:1936  [n3] serving sql connections
I200207 16:33:54.618822 14 base/addr_validation.go:296  [n?] server certificate addresses: IP=127.0.0.1,::1; DNS=localhost,*.local; CN=node
I200207 16:33:54.618857 14 base/addr_validation.go:342  [n?] web UI certificate addresses: IP=127.0.0.1,::1; DNS=localhost,*.local; CN=node
I200207 16:33:54.633974 301 storage/store_snapshot.go:922  [n1,replicate,s1,r27/1:/{NamespaceTab…-Table/32}] sending LEARNER snapshot eca5a087 at applied index 16
I200207 16:33:54.634224 301 storage/store_snapshot.go:965  [n1,replicate,s1,r27/1:/{NamespaceTab…-Table/32}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.02s
I200207 16:33:54.649727 1494 storage/replica_raftstorage.go:780  [n2,s2,r27/2:{-}] applying LEARNER snapshot [id=eca5a087 index=16]
I200207 16:33:54.650053 1524 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r27/1:/{NamespaceTab…-Table/32}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):2LEARNER
I200207 16:33:54.655900 1494 storage/replica_raftstorage.go:801  [n2,s2,r27/2:/{NamespaceTab…-Table/32}] applied LEARNER snapshot [total=6ms ingestion=4@4ms id=eca5a087 index=16]
I200207 16:33:54.660547 301 storage/replica_command.go:1601  [n1,replicate,s1,r27/1:/{NamespaceTab…-Table/32}] change replicas (add [(n2,s2):2] remove []): existing descriptor r27:/{NamespaceTable/Max-Table/32} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
I200207 16:33:54.665386 1481 sql/event_log.go:132  [n3] Event: "node_join", target: 3, info: {Descriptor:{NodeID:3 Address:127.0.0.1:38505 Attrs: Locality:region=test,dc=dc3 ServerVersion:19.2-11 BuildTag:v20.1.0-alpha.20191118-1912-g1139636-dirty StartedAt:1581093234594591865 LocalityAddress:[] ClusterName: SQLAddress:127.0.0.1:39083} ClusterID:a2dd6d0a-267f-4814-8be5-9559f1904555 StartedAt:1581093234594591865 LastUp:1581093234594591865}
I200207 16:33:54.674299 1479 server/server_update.go:55  [n3] no need to upgrade, cluster already at the newest version
I200207 16:33:54.677067 301 storage/replica_raft.go:248  [n1,s1,r27/1:/{NamespaceTab…-Table/32}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200207 16:33:54.678000 301 storage/replica_command.go:1601  [n1,replicate,s1,r27/1:/{NamespaceTab…-Table/32}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r27:/{NamespaceTable/Max-Table/32} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200207 16:33:54.690343 301 storage/replica_raft.go:248  [n1,s1,r27/1:/{NamespaceTab…-Table/32}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3LEARNER] next=4
I200207 16:33:54.693505 301 storage/store_snapshot.go:922  [n1,replicate,s1,r27/1:/{NamespaceTab…-Table/32}] sending LEARNER snapshot e9624e9f at applied index 20
I200207 16:33:54.693621 301 storage/store_snapshot.go:965  [n1,replicate,s1,r27/1:/{NamespaceTab…-Table/32}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:54.693694 1579 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r27/1:/{NamespaceTab…-Table/32}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200207 16:33:54.694219 1608 storage/replica_raftstorage.go:780  [n3,s3,r27/3:{-}] applying LEARNER snapshot [id=e9624e9f index=20]
I200207 16:33:54.698653 1608 storage/replica_raftstorage.go:801  [n3,s3,r27/3:/{NamespaceTab…-Table/32}] applied LEARNER snapshot [total=4ms ingestion=4@4ms id=e9624e9f index=20]
I200207 16:33:54.699447 301 storage/replica_command.go:1601  [n1,replicate,s1,r27/1:/{NamespaceTab…-Table/32}] change replicas (add [(n3,s3):3] remove []): existing descriptor r27:/{NamespaceTable/Max-Table/32} [(n1,s1):1, (n2,s2):2, (n3,s3):3LEARNER, next=4, gen=3]
I200207 16:33:54.707589 301 storage/replica_raft.go:248  [n1,s1,r27/1:/{NamespaceTab…-Table/32}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3] next=4
W200207 16:33:54.708154 14 server/status/runtime.go:308  [n?] Could not parse build timestamp: parsing time "" as "2006/01/02 15:04:05": cannot parse "" as "2006"
I200207 16:33:54.709422 301 storage/replica_command.go:1601  [n1,replicate,s1,r8/1:/Table/1{2-3}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r8:/Table/1{2-3} [(n1,s1):1, next=2, gen=0]
I200207 16:33:54.712380 301 storage/replica_raft.go:248  [n1,s1,r8/1:/Table/1{2-3}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200207 16:33:54.717637 1505 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r8/1:/Table/1{2-3}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):2LEARNER
I200207 16:33:54.717903 301 storage/store_snapshot.go:922  [n1,replicate,s1,r8/1:/Table/1{2-3}] sending LEARNER snapshot 84d3d71f at applied index 37
I200207 16:33:54.718166 301 storage/store_snapshot.go:965  [n1,replicate,s1,r8/1:/Table/1{2-3}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 46, log entries: 0, rate-limit: 8.0 MiB/sec, 0.01s
I200207 16:33:54.718933 1566 storage/replica_raftstorage.go:780  [n2,s2,r8/2:{-}] applying LEARNER snapshot [id=84d3d71f index=37]
I200207 16:33:54.721025 14 server/server.go:988  [n?] monitoring forward clock jumps based on server.clock.forward_jump_check_enabled
I200207 16:33:54.721784 1566 storage/replica_raftstorage.go:801  [n2,s2,r8/2:/Table/1{2-3}] applied LEARNER snapshot [total=3ms ingestion=4@2ms id=84d3d71f index=37]
I200207 16:33:54.722340 301 storage/replica_command.go:1601  [n1,replicate,s1,r8/1:/Table/1{2-3}] change replicas (add [(n2,s2):2] remove []): existing descriptor r8:/Table/1{2-3} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
I200207 16:33:54.764498 301 storage/replica_raft.go:248  [n1,s1,r8/1:/Table/1{2-3}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200207 16:33:54.765089 301 storage/replica_command.go:1601  [n1,replicate,s1,r8/1:/Table/1{2-3}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r8:/Table/1{2-3} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200207 16:33:54.789442 301 storage/replica_raft.go:248  [n1,s1,r8/1:/Table/1{2-3}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3LEARNER] next=4
I200207 16:33:54.794932 1656 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r8/1:/Table/1{2-3}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200207 16:33:54.795063 301 storage/store_snapshot.go:922  [n1,replicate,s1,r8/1:/Table/1{2-3}] sending LEARNER snapshot dc15355e at applied index 41
I200207 16:33:54.795368 301 storage/store_snapshot.go:965  [n1,replicate,s1,r8/1:/Table/1{2-3}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 50, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:54.796121 1600 storage/replica_raftstorage.go:780  [n3,s3,r8/3:{-}] applying LEARNER snapshot [id=dc15355e index=41]
I200207 16:33:54.798944 1600 storage/replica_raftstorage.go:801  [n3,s3,r8/3:/Table/1{2-3}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=dc15355e index=41]
I200207 16:33:54.799642 301 storage/replica_command.go:1601  [n1,replicate,s1,r8/1:/Table/1{2-3}] change replicas (add [(n3,s3):3] remove []): existing descriptor r8:/Table/1{2-3} [(n1,s1):1, (n2,s2):2, (n3,s3):3LEARNER, next=4, gen=3]
I200207 16:33:54.802767 301 storage/replica_raft.go:248  [n1,s1,r8/1:/Table/1{2-3}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3] next=4
I200207 16:33:54.806370 301 storage/replica_command.go:1601  [n1,replicate,s1,r7/1:/Table/1{1-2}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r7:/Table/1{1-2} [(n1,s1):1, next=2, gen=0]
I200207 16:33:54.809193 301 storage/replica_raft.go:248  [n1,s1,r7/1:/Table/1{1-2}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200207 16:33:54.810535 301 storage/store_snapshot.go:922  [n1,replicate,s1,r7/1:/Table/1{1-2}] sending LEARNER snapshot dda8a1c2 at applied index 19
I200207 16:33:54.810653 1717 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r7/1:/Table/1{1-2}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):2LEARNER
I200207 16:33:54.810667 301 storage/store_snapshot.go:965  [n1,replicate,s1,r7/1:/Table/1{1-2}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 7, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:54.811307 1714 storage/replica_raftstorage.go:780  [n2,s2,r7/2:{-}] applying LEARNER snapshot [id=dda8a1c2 index=19]
I200207 16:33:54.812730 14 server/config.go:565  [n?] 1 storage engine initialized
I200207 16:33:54.812750 14 server/config.go:568  [n?] RocksDB cache size: 128 MiB
I200207 16:33:54.812759 14 server/config.go:568  [n?] store 0: in-memory, size 0 B
W200207 16:33:54.812854 14 gossip/gossip.go:1518  [n?] no incoming or outgoing connections
I200207 16:33:54.812901 14 server/server.go:1443  [n?] no stores bootstrapped and --join flag specified, awaiting init command or join with an already initialized node.
I200207 16:33:54.818702 1714 storage/replica_raftstorage.go:801  [n2,s2,r7/2:/Table/1{1-2}] applied LEARNER snapshot [total=7ms ingestion=4@7ms id=dda8a1c2 index=19]
I200207 16:33:54.819248 301 storage/replica_command.go:1601  [n1,replicate,s1,r7/1:/Table/1{1-2}] change replicas (add [(n2,s2):2] remove []): existing descriptor r7:/Table/1{1-2} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
I200207 16:33:54.822194 301 storage/replica_raft.go:248  [n1,s1,r7/1:/Table/1{1-2}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200207 16:33:54.822764 301 storage/replica_command.go:1601  [n1,replicate,s1,r7/1:/Table/1{1-2}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r7:/Table/1{1-2} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200207 16:33:54.825040 1718 gossip/client.go:124  [n?] started gossip client to 127.0.0.1:33107
I200207 16:33:54.825275 1781 gossip/server.go:227  [n1] received initial cluster-verification connection from 127.0.0.1:37689
I200207 16:33:54.826314 14 server/node.go:655  [n?] connecting to gossip network to verify cluster ID...
I200207 16:33:54.826339 14 server/node.go:675  [n?] node connected via gossip and verified as part of cluster "a2dd6d0a-267f-4814-8be5-9559f1904555"
I200207 16:33:54.826369 1781 gossip/server.go:227  [n1] received initial cluster-verification connection from 127.0.0.1:37689
I200207 16:33:54.836267 301 storage/replica_raft.go:248  [n1,s1,r7/1:/Table/1{1-2}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3LEARNER] next=4
I200207 16:33:54.838083 301 storage/store_snapshot.go:922  [n1,replicate,s1,r7/1:/Table/1{1-2}] sending LEARNER snapshot 9d938b3a at applied index 23
I200207 16:33:54.838186 1827 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r7/1:/Table/1{1-2}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200207 16:33:54.838258 301 storage/store_snapshot.go:965  [n1,replicate,s1,r7/1:/Table/1{1-2}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 11, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:54.838991 1760 storage/replica_raftstorage.go:780  [n3,s3,r7/3:{-}] applying LEARNER snapshot [id=9d938b3a index=23]
I200207 16:33:54.850471 14 server/node.go:388  [n?] new node allocated ID 4
I200207 16:33:54.850553 14 gossip/gossip.go:395  [n4] NodeDescriptor set to node_id:4 address: attrs:<> locality: tiers: > ServerVersion: build_tag:"v20.1.0-alpha.20191118-1912-g1139636-dirty" started_at:1581093234850481611 cluster_name:"" sql_address:
I200207 16:33:54.850612 14 storage/stores.go:240  [n4] read 0 node addresses from persistent storage
I200207 16:33:54.850639 14 storage/stores.go:259  [n4] wrote 3 node addresses to persistent storage
I200207 16:33:54.851038 66 storage/stores.go:259  [n1] wrote 3 node addresses to persistent storage
I200207 16:33:54.851363 1180 storage/stores.go:259  [n3] wrote 3 node addresses to persistent storage
I200207 16:33:54.851749 961 storage/stores.go:259  [n2] wrote 3 node addresses to persistent storage
I200207 16:33:54.852949 1760 storage/replica_raftstorage.go:801  [n3,s3,r7/3:/Table/1{1-2}] applied LEARNER snapshot [total=14ms ingestion=4@12ms id=9d938b3a index=23]
I200207 16:33:54.853640 301 storage/replica_command.go:1601  [n1,replicate,s1,r7/1:/Table/1{1-2}] change replicas (add [(n3,s3):3] remove []): existing descriptor r7:/Table/1{1-2} [(n1,s1):1, (n2,s2):2, (n3,s3):3LEARNER, next=4, gen=3]
I200207 16:33:54.853723 14 server/node.go:630  [n4] bootstrapped store [n4,s4]
I200207 16:33:54.853885 14 server/node.go:522  [n4] node=4: started with [=] engine(s) and attributes []
I200207 16:33:54.853955 14 server/server.go:2131  [n4] Could not start goroutine dumper worker due to: directory to store dumps could not be determined
I200207 16:33:54.854025 14 server/server.go:1573  [n4] starting https server at 127.0.0.1:40531 (use: 127.0.0.1:40531)
I200207 16:33:54.854059 14 server/server.go:1578  [n4] starting postgres server at 127.0.0.1:34835 (use: 127.0.0.1:34835)
I200207 16:33:54.854080 14 server/server.go:1580  [n4] starting grpc server at 127.0.0.1:37689
I200207 16:33:54.854098 14 server/server.go:1581  [n4] advertising CockroachDB node at 127.0.0.1:37689
I200207 16:33:54.862200 301 storage/replica_raft.go:248  [n1,s1,r7/1:/Table/1{1-2}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3] next=4
I200207 16:33:54.864362 14 server/server.go:1655  [n4] done ensuring all necessary migrations have run
I200207 16:33:54.864393 14 server/server.go:1936  [n4] serving sql connections
I200207 16:33:54.865016 301 storage/replica_command.go:1601  [n1,replicate,s1,r19/1:/Table/2{3-4}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r19:/Table/2{3-4} [(n1,s1):1, next=2, gen=0]
I200207 16:33:54.872170 301 storage/replica_raft.go:248  [n1,s1,r19/1:/Table/2{3-4}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200207 16:33:54.874524 301 storage/store_snapshot.go:922  [n1,replicate,s1,r19/1:/Table/2{3-4}] sending LEARNER snapshot 332372c7 at applied index 17
I200207 16:33:54.874643 301 storage/store_snapshot.go:965  [n1,replicate,s1,r19/1:/Table/2{3-4}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:54.874648 1848 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r19/1:/Table/2{3-4}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):2LEARNER
I200207 16:33:54.875467 1959 storage/replica_raftstorage.go:780  [n2,s2,r19/2:{-}] applying LEARNER snapshot [id=332372c7 index=17]
I200207 16:33:54.924684 1990 server/server_update.go:55  [n4] no need to upgrade, cluster already at the newest version
I200207 16:33:54.926646 1959 storage/replica_raftstorage.go:801  [n2,s2,r19/2:/Table/2{3-4}] applied LEARNER snapshot [total=51ms ingestion=4@51ms id=332372c7 index=17]
I200207 16:33:54.927496 301 storage/replica_command.go:1601  [n1,replicate,s1,r19/1:/Table/2{3-4}] change replicas (add [(n2,s2):2] remove []): existing descriptor r19:/Table/2{3-4} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
I200207 16:33:54.949513 1992 sql/event_log.go:132  [n4] Event: "node_join", target: 4, info: {Descriptor:{NodeID:4 Address:127.0.0.1:37689 Attrs: Locality:region=test,dc=dc4 ServerVersion:19.2-11 BuildTag:v20.1.0-alpha.20191118-1912-g1139636-dirty StartedAt:1581093234850481611 LocalityAddress:[] ClusterName: SQLAddress:127.0.0.1:34835} ClusterID:a2dd6d0a-267f-4814-8be5-9559f1904555 StartedAt:1581093234850481611 LastUp:1581093234850481611}
I200207 16:33:54.955855 301 storage/replica_raft.go:248  [n1,s1,r19/1:/Table/2{3-4}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200207 16:33:54.956681 301 storage/replica_command.go:1601  [n1,replicate,s1,r19/1:/Table/2{3-4}] change replicas (add [(n4,s4):3LEARNER] remove []): existing descriptor r19:/Table/2{3-4} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200207 16:33:54.972085 301 storage/replica_raft.go:248  [n1,s1,r19/1:/Table/2{3-4}] proposing SIMPLE(l3) ADD_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3LEARNER] next=4
I200207 16:33:55.041328 301 storage/store_snapshot.go:922  [n1,replicate,s1,r19/1:/Table/2{3-4}] sending LEARNER snapshot a0eeae5f at applied index 21
I200207 16:33:55.041483 301 storage/store_snapshot.go:965  [n1,replicate,s1,r19/1:/Table/2{3-4}] streamed snapshot to (n4,s4):3LEARNER: kv pairs: 14, log entries: 0, rate-limit: 8.0 MiB/sec, 0.01s
I200207 16:33:55.042256 2187 storage/replica_raftstorage.go:780  [n4,s4,r19/3:{-}] applying LEARNER snapshot [id=a0eeae5f index=21]
I200207 16:33:55.043579 2187 storage/replica_raftstorage.go:801  [n4,s4,r19/3:/Table/2{3-4}] applied LEARNER snapshot [total=1ms ingestion=4@1ms id=a0eeae5f index=21]
I200207 16:33:55.044270 301 storage/replica_command.go:1601  [n1,replicate,s1,r19/1:/Table/2{3-4}] change replicas (add [(n4,s4):3] remove []): existing descriptor r19:/Table/2{3-4} [(n1,s1):1, (n2,s2):2, (n4,s4):3LEARNER, next=4, gen=3]
I200207 16:33:55.047317 301 storage/replica_raft.go:248  [n1,s1,r19/1:/Table/2{3-4}] proposing SIMPLE(v3) ADD_REPLICA[(n4,s4):3]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3] next=4
I200207 16:33:55.063110 301 storage/replica_command.go:1601  [n1,replicate,s1,r17/1:/Table/2{1-2}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r17:/Table/2{1-2} [(n1,s1):1, next=2, gen=0]
I200207 16:33:55.066166 301 storage/replica_raft.go:248  [n1,s1,r17/1:/Table/2{1-2}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200207 16:33:55.068049 301 storage/store_snapshot.go:922  [n1,replicate,s1,r17/1:/Table/2{1-2}] sending LEARNER snapshot 65b36dc8 at applied index 21
I200207 16:33:55.068164 301 storage/store_snapshot.go:965  [n1,replicate,s1,r17/1:/Table/2{1-2}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 11, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:55.069064 2128 storage/replica_raftstorage.go:780  [n4,s4,r17/2:{-}] applying LEARNER snapshot [id=65b36dc8 index=21]
I200207 16:33:55.073383 2128 storage/replica_raftstorage.go:801  [n4,s4,r17/2:/Table/2{1-2}] applied LEARNER snapshot [total=4ms ingestion=4@4ms id=65b36dc8 index=21]
I200207 16:33:55.073773 2261 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r17/1:/Table/2{1-2}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200207 16:33:55.074096 301 storage/replica_command.go:1601  [n1,replicate,s1,r17/1:/Table/2{1-2}] change replicas (add [(n4,s4):2] remove []): existing descriptor r17:/Table/2{1-2} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200207 16:33:55.076885 301 storage/replica_raft.go:248  [n1,s1,r17/1:/Table/2{1-2}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200207 16:33:55.077462 301 storage/replica_command.go:1601  [n1,replicate,s1,r17/1:/Table/2{1-2}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r17:/Table/2{1-2} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200207 16:33:55.090926 301 storage/replica_raft.go:248  [n1,s1,r17/1:/Table/2{1-2}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3LEARNER] next=4
I200207 16:33:55.093429 301 storage/store_snapshot.go:922  [n1,replicate,s1,r17/1:/Table/2{1-2}] sending LEARNER snapshot b0d2b55b at applied index 25
I200207 16:33:55.093693 301 storage/store_snapshot.go:965  [n1,replicate,s1,r17/1:/Table/2{1-2}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 15, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:55.094282 2223 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r17/1:/Table/2{1-2}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200207 16:33:55.094723 2253 storage/replica_raftstorage.go:780  [n3,s3,r17/3:{-}] applying LEARNER snapshot [id=b0d2b55b index=25]
I200207 16:33:55.099660 2253 storage/replica_raftstorage.go:801  [n3,s3,r17/3:/Table/2{1-2}] applied LEARNER snapshot [total=5ms ingestion=4@4ms id=b0d2b55b index=25]
I200207 16:33:55.103270 301 storage/replica_command.go:1601  [n1,replicate,s1,r17/1:/Table/2{1-2}] change replicas (add [(n3,s3):3] remove []): existing descriptor r17:/Table/2{1-2} [(n1,s1):1, (n4,s4):2, (n3,s3):3LEARNER, next=4, gen=3]
I200207 16:33:55.107659 301 storage/replica_raft.go:248  [n1,s1,r17/1:/Table/2{1-2}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3] next=4
I200207 16:33:55.110557 301 storage/replica_command.go:1601  [n1,replicate,s1,r3/1:/System/{NodeLive…-tsd}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r3:/System/{NodeLivenessMax-tsd} [(n1,s1):1, next=2, gen=0]
I200207 16:33:55.116097 301 storage/replica_raft.go:248  [n1,s1,r3/1:/System/{NodeLive…-tsd}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200207 16:33:55.118072 301 storage/store_snapshot.go:922  [n1,replicate,s1,r3/1:/System/{NodeLive…-tsd}] sending LEARNER snapshot 659426ce at applied index 50
I200207 16:33:55.118231 2356 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r3/1:/System/{NodeLive…-tsd}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200207 16:33:55.118770 301 storage/store_snapshot.go:965  [n1,replicate,s1,r3/1:/System/{NodeLive…-tsd}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 38, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:55.120557 2324 storage/replica_raftstorage.go:780  [n4,s4,r3/2:{-}] applying LEARNER snapshot [id=659426ce index=50]
I200207 16:33:55.124108 2324 storage/replica_raftstorage.go:801  [n4,s4,r3/2:/System/{NodeLive…-tsd}] applied LEARNER snapshot [total=4ms ingestion=4@3ms id=659426ce index=50]
I200207 16:33:55.124724 301 storage/replica_command.go:1601  [n1,replicate,s1,r3/1:/System/{NodeLive…-tsd}] change replicas (add [(n4,s4):2] remove []): existing descriptor r3:/System/{NodeLivenessMax-tsd} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200207 16:33:55.130224 301 storage/replica_raft.go:248  [n1,s1,r3/1:/System/{NodeLive…-tsd}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200207 16:33:55.131030 301 storage/replica_command.go:1601  [n1,replicate,s1,r3/1:/System/{NodeLive…-tsd}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r3:/System/{NodeLivenessMax-tsd} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200207 16:33:55.140815 301 storage/replica_raft.go:248  [n1,s1,r3/1:/System/{NodeLive…-tsd}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3LEARNER] next=4
I200207 16:33:55.143577 301 storage/store_snapshot.go:922  [n1,replicate,s1,r3/1:/System/{NodeLive…-tsd}] sending LEARNER snapshot b32f2399 at applied index 54
I200207 16:33:55.144293 301 storage/store_snapshot.go:965  [n1,replicate,s1,r3/1:/System/{NodeLive…-tsd}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 42, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:55.144605 2320 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r3/1:/System/{NodeLive…-tsd}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):3LEARNER
I200207 16:33:55.145882 2318 storage/replica_raftstorage.go:780  [n2,s2,r3/3:{-}] applying LEARNER snapshot [id=b32f2399 index=54]
I200207 16:33:55.153793 2318 storage/replica_raftstorage.go:801  [n2,s2,r3/3:/System/{NodeLive…-tsd}] applied LEARNER snapshot [total=8ms ingestion=4@7ms id=b32f2399 index=54]
I200207 16:33:55.154377 301 storage/replica_command.go:1601  [n1,replicate,s1,r3/1:/System/{NodeLive…-tsd}] change replicas (add [(n2,s2):3] remove []): existing descriptor r3:/System/{NodeLivenessMax-tsd} [(n1,s1):1, (n4,s4):2, (n2,s2):3LEARNER, next=4, gen=3]
I200207 16:33:55.157046 301 storage/replica_raft.go:248  [n1,s1,r3/1:/System/{NodeLive…-tsd}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3] next=4
I200207 16:33:55.162785 301 storage/replica_command.go:1601  [n1,replicate,s1,r26/1:/NamespaceTable/{30-Max}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r26:/NamespaceTable/{30-Max} [(n1,s1):1, next=2, gen=0]
I200207 16:33:55.165480 301 storage/replica_raft.go:248  [n1,s1,r26/1:/NamespaceTable/{30-Max}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200207 16:33:55.171107 301 storage/store_snapshot.go:922  [n1,replicate,s1,r26/1:/NamespaceTable/{30-Max}] sending LEARNER snapshot 0c3f20eb at applied index 23
I200207 16:33:55.171319 301 storage/store_snapshot.go:965  [n1,replicate,s1,r26/1:/NamespaceTable/{30-Max}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 34, log entries: 0, rate-limit: 8.0 MiB/sec, 0.01s
I200207 16:33:55.171596 2419 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r26/1:/NamespaceTable/{30-Max}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200207 16:33:55.172160 2404 storage/replica_raftstorage.go:780  [n4,s4,r26/2:{-}] applying LEARNER snapshot [id=0c3f20eb index=23]
I200207 16:33:55.175828 2404 storage/replica_raftstorage.go:801  [n4,s4,r26/2:/NamespaceTable/{30-Max}] applied LEARNER snapshot [total=4ms ingestion=4@2ms id=0c3f20eb index=23]
I200207 16:33:55.176357 301 storage/replica_command.go:1601  [n1,replicate,s1,r26/1:/NamespaceTable/{30-Max}] change replicas (add [(n4,s4):2] remove []): existing descriptor r26:/NamespaceTable/{30-Max} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200207 16:33:55.179223 301 storage/replica_raft.go:248  [n1,s1,r26/1:/NamespaceTable/{30-Max}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200207 16:33:55.179725 301 storage/replica_command.go:1601  [n1,replicate,s1,r26/1:/NamespaceTable/{30-Max}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r26:/NamespaceTable/{30-Max} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200207 16:33:55.191537 301 storage/replica_raft.go:248  [n1,s1,r26/1:/NamespaceTable/{30-Max}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3LEARNER] next=4
I200207 16:33:55.193354 301 storage/store_snapshot.go:922  [n1,replicate,s1,r26/1:/NamespaceTable/{30-Max}] sending LEARNER snapshot b171435f at applied index 27
I200207 16:33:55.193629 301 storage/store_snapshot.go:965  [n1,replicate,s1,r26/1:/NamespaceTable/{30-Max}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 38, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:55.194343 2439 storage/replica_raftstorage.go:780  [n3,s3,r26/3:{-}] applying LEARNER snapshot [id=b171435f index=27]
I200207 16:33:55.195288 2421 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r26/1:/NamespaceTable/{30-Max}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200207 16:33:55.202955 2439 storage/replica_raftstorage.go:801  [n3,s3,r26/3:/NamespaceTable/{30-Max}] applied LEARNER snapshot [total=9ms ingestion=4@8ms id=b171435f index=27]
I200207 16:33:55.203671 301 storage/replica_command.go:1601  [n1,replicate,s1,r26/1:/NamespaceTable/{30-Max}] change replicas (add [(n3,s3):3] remove []): existing descriptor r26:/NamespaceTable/{30-Max} [(n1,s1):1, (n4,s4):2, (n3,s3):3LEARNER, next=4, gen=3]
I200207 16:33:55.206848 301 storage/replica_raft.go:248  [n1,s1,r26/1:/NamespaceTable/{30-Max}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3] next=4
I200207 16:33:55.209309 301 storage/queue.go:1135  [n1,replicate] purgatory is now empty
I200207 16:33:55.209695 2451 storage/replica_command.go:1601  [n1,replicate,s1,r9/1:/Table/1{3-4}] change replicas (add [(n3,s3):2LEARNER] remove []): existing descriptor r9:/Table/1{3-4} [(n1,s1):1, next=2, gen=0]
I200207 16:33:55.260115 2451 storage/replica_raft.go:248  [n1,s1,r9/1:/Table/1{3-4}] proposing SIMPLE(l2) ADD_REPLICA[(n3,s3):2LEARNER]: after=[(n1,s1):1 (n3,s3):2LEARNER] next=3
I200207 16:33:55.261946 2451 storage/store_snapshot.go:922  [n1,replicate,s1,r9/1:/Table/1{3-4}] sending LEARNER snapshot 7446db45 at applied index 73
I200207 16:33:55.262066 2476 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r9/1:/Table/1{3-4}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):2LEARNER
I200207 16:33:55.262381 2451 storage/store_snapshot.go:965  [n1,replicate,s1,r9/1:/Table/1{3-4}] streamed snapshot to (n3,s3):2LEARNER: kv pairs: 151, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:55.263234 2474 storage/replica_raftstorage.go:780  [n3,s3,r9/2:{-}] applying LEARNER snapshot [id=7446db45 index=73]
I200207 16:33:55.270389 2474 storage/replica_raftstorage.go:801  [n3,s3,r9/2:/Table/1{3-4}] applied LEARNER snapshot [total=7ms ingestion=4@7ms id=7446db45 index=73]
I200207 16:33:55.272081 2451 storage/replica_command.go:1601  [n1,replicate,s1,r9/1:/Table/1{3-4}] change replicas (add [(n3,s3):2] remove []): existing descriptor r9:/Table/1{3-4} [(n1,s1):1, (n3,s3):2LEARNER, next=3, gen=1]
I200207 16:33:55.276457 2451 storage/replica_raft.go:248  [n1,s1,r9/1:/Table/1{3-4}] proposing SIMPLE(v2) ADD_REPLICA[(n3,s3):2]: after=[(n1,s1):1 (n3,s3):2] next=3
I200207 16:33:55.277037 2451 storage/replica_command.go:1601  [n1,replicate,s1,r9/1:/Table/1{3-4}] change replicas (add [(n4,s4):3LEARNER] remove []): existing descriptor r9:/Table/1{3-4} [(n1,s1):1, (n3,s3):2, next=3, gen=2]
I200207 16:33:55.286713 2451 storage/replica_raft.go:248  [n1,s1,r9/1:/Table/1{3-4}] proposing SIMPLE(l3) ADD_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3LEARNER] next=4
I200207 16:33:55.298193 2451 storage/store_snapshot.go:922  [n1,replicate,s1,r9/1:/Table/1{3-4}] sending LEARNER snapshot b2c0d54a at applied index 79
I200207 16:33:55.298361 2430 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r9/1:/Table/1{3-4}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):3LEARNER
I200207 16:33:55.298809 2451 storage/store_snapshot.go:965  [n1,replicate,s1,r9/1:/Table/1{3-4}] streamed snapshot to (n4,s4):3LEARNER: kv pairs: 165, log entries: 0, rate-limit: 8.0 MiB/sec, 0.01s
I200207 16:33:55.305218 2428 storage/replica_raftstorage.go:780  [n4,s4,r9/3:{-}] applying LEARNER snapshot [id=b2c0d54a index=79]
I200207 16:33:55.309167 2428 storage/replica_raftstorage.go:801  [n4,s4,r9/3:/Table/1{3-4}] applied LEARNER snapshot [total=4ms ingestion=4@2ms id=b2c0d54a index=79]
I200207 16:33:55.310340 2451 storage/replica_command.go:1601  [n1,replicate,s1,r9/1:/Table/1{3-4}] change replicas (add [(n4,s4):3] remove []): existing descriptor r9:/Table/1{3-4} [(n1,s1):1, (n3,s3):2, (n4,s4):3LEARNER, next=4, gen=3]
I200207 16:33:55.316812 2451 storage/replica_raft.go:248  [n1,s1,r9/1:/Table/1{3-4}] proposing SIMPLE(v3) ADD_REPLICA[(n4,s4):3]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3] next=4
I200207 16:33:55.320635 14 storage/replica_command.go:1601  [n1,replicate,s1,r13/1:/Table/1{7-8}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r13:/Table/1{7-8} [(n1,s1):1, next=2, gen=0]
I200207 16:33:55.384015 14 storage/replica_raft.go:248  [n1,s1,r13/1:/Table/1{7-8}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200207 16:33:55.385680 14 storage/store_snapshot.go:922  [n1,replicate,s1,r13/1:/Table/1{7-8}] sending LEARNER snapshot 84021fb7 at applied index 15
I200207 16:33:55.385794 14 storage/store_snapshot.go:965  [n1,replicate,s1,r13/1:/Table/1{7-8}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:55.386014 2539 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r13/1:/Table/1{7-8}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):2LEARNER
I200207 16:33:55.386631 2537 storage/replica_raftstorage.go:780  [n2,s2,r13/2:{-}] applying LEARNER snapshot [id=84021fb7 index=15]
I200207 16:33:55.391425 2537 storage/replica_raftstorage.go:801  [n2,s2,r13/2:/Table/1{7-8}] applied LEARNER snapshot [total=5ms ingestion=4@4ms id=84021fb7 index=15]
I200207 16:33:55.391893 14 storage/replica_command.go:1601  [n1,replicate,s1,r13/1:/Table/1{7-8}] change replicas (add [(n2,s2):2] remove []): existing descriptor r13:/Table/1{7-8} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
I200207 16:33:55.403580 14 storage/replica_raft.go:248  [n1,s1,r13/1:/Table/1{7-8}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200207 16:33:55.404180 14 storage/replica_command.go:1601  [n1,replicate,s1,r13/1:/Table/1{7-8}] change replicas (add [(n4,s4):3LEARNER] remove []): existing descriptor r13:/Table/1{7-8} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200207 16:33:55.412390 14 storage/replica_raft.go:248  [n1,s1,r13/1:/Table/1{7-8}] proposing SIMPLE(l3) ADD_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3LEARNER] next=4
I200207 16:33:55.417745 14 storage/store_snapshot.go:922  [n1,replicate,s1,r13/1:/Table/1{7-8}] sending LEARNER snapshot 3eef0a3d at applied index 19
I200207 16:33:55.417869 14 storage/store_snapshot.go:965  [n1,replicate,s1,r13/1:/Table/1{7-8}] streamed snapshot to (n4,s4):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:55.417916 2579 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r13/1:/Table/1{7-8}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):3LEARNER
I200207 16:33:55.418602 2465 storage/replica_raftstorage.go:780  [n4,s4,r13/3:{-}] applying LEARNER snapshot [id=3eef0a3d index=19]
I200207 16:33:55.422831 2465 storage/replica_raftstorage.go:801  [n4,s4,r13/3:/Table/1{7-8}] applied LEARNER snapshot [total=4ms ingestion=4@4ms id=3eef0a3d index=19]
I200207 16:33:55.427888 14 storage/replica_command.go:1601  [n1,replicate,s1,r13/1:/Table/1{7-8}] change replicas (add [(n4,s4):3] remove []): existing descriptor r13:/Table/1{7-8} [(n1,s1):1, (n2,s2):2, (n4,s4):3LEARNER, next=4, gen=3]
I200207 16:33:55.438974 14 storage/replica_raft.go:248  [n1,s1,r13/1:/Table/1{7-8}] proposing SIMPLE(v3) ADD_REPLICA[(n4,s4):3]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3] next=4
I200207 16:33:55.445206 14 storage/replica_command.go:1601  [n1,replicate,s1,r1/1:/{Min-System/NodeL…}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r1:/{Min-System/NodeLiveness} [(n1,s1):1, next=2, gen=0]
I200207 16:33:55.450761 14 storage/replica_raft.go:248  [n1,s1,r1/1:/{Min-System/NodeL…}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200207 16:33:55.453446 14 storage/store_snapshot.go:922  [n1,replicate,s1,r1/1:/{Min-System/NodeL…}] sending LEARNER snapshot 0d8e0a5b at applied index 92
I200207 16:33:55.454280 14 storage/store_snapshot.go:965  [n1,replicate,s1,r1/1:/{Min-System/NodeL…}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 73, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:55.455743 2630 storage/replica_raftstorage.go:780  [n4,s4,r1/2:{-}] applying LEARNER snapshot [id=0d8e0a5b index=92]
I200207 16:33:55.469272 2630 storage/replica_raftstorage.go:801  [n4,s4,r1/2:/{Min-System/NodeL…}] applied LEARNER snapshot [total=13ms ingestion=4@13ms id=0d8e0a5b index=92]
I200207 16:33:55.473867 14 storage/replica_command.go:1601  [n1,replicate,s1,r1/1:/{Min-System/NodeL…}] change replicas (add [(n4,s4):2] remove []): existing descriptor r1:/{Min-System/NodeLiveness} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200207 16:33:55.484588 14 storage/replica_raft.go:248  [n1,s1,r1/1:/{Min-System/NodeL…}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200207 16:33:55.485463 14 storage/replica_command.go:1601  [n1,replicate,s1,r1/1:/{Min-System/NodeL…}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r1:/{Min-System/NodeLiveness} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200207 16:33:55.496091 14 storage/replica_raft.go:248  [n1,s1,r1/1:/{Min-System/NodeL…}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3LEARNER] next=4
I200207 16:33:55.498172 14 storage/store_snapshot.go:922  [n1,replicate,s1,r1/1:/{Min-System/NodeL…}] sending LEARNER snapshot 11f9b3fa at applied index 98
I200207 16:33:55.498294 2665 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r1/1:/{Min-System/NodeL…}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200207 16:33:55.498473 14 storage/store_snapshot.go:965  [n1,replicate,s1,r1/1:/{Min-System/NodeL…}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 81, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:55.499370 2686 storage/replica_raftstorage.go:780  [n3,s3,r1/3:{-}] applying LEARNER snapshot [id=11f9b3fa index=98]
I200207 16:33:55.504577 2686 storage/replica_raftstorage.go:801  [n3,s3,r1/3:/{Min-System/NodeL…}] applied LEARNER snapshot [total=5ms ingestion=4@4ms id=11f9b3fa index=98]
I200207 16:33:55.505795 14 storage/replica_command.go:1601  [n1,replicate,s1,r1/1:/{Min-System/NodeL…}] change replicas (add [(n3,s3):3] remove []): existing descriptor r1:/{Min-System/NodeLiveness} [(n1,s1):1, (n4,s4):2, (n3,s3):3LEARNER, next=4, gen=3]
I200207 16:33:55.515640 14 storage/replica_raft.go:248  [n1,s1,r1/1:/{Min-System/NodeL…}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3] next=4
I200207 16:33:55.528814 14 storage/replica_command.go:1601  [n1,replicate,s1,r28/1:/{Table/32-Max}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r28:/{Table/32-Max} [(n1,s1):1, next=2, gen=0]
I200207 16:33:55.549273 14 storage/replica_raft.go:248  [n1,s1,r28/1:/{Table/32-Max}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200207 16:33:55.551164 14 storage/store_snapshot.go:922  [n1,replicate,s1,r28/1:/{Table/32-Max}] sending LEARNER snapshot e3ea06f5 at applied index 15
I200207 16:33:55.551281 14 storage/store_snapshot.go:965  [n1,replicate,s1,r28/1:/{Table/32-Max}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:55.551344 2573 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r28/1:/{Table/32-Max}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):2LEARNER
I200207 16:33:55.552040 2721 storage/replica_raftstorage.go:780  [n2,s2,r28/2:{-}] applying LEARNER snapshot [id=e3ea06f5 index=15]
I200207 16:33:55.554872 2721 storage/replica_raftstorage.go:801  [n2,s2,r28/2:/{Table/32-Max}] applied LEARNER snapshot [total=3ms ingestion=4@2ms id=e3ea06f5 index=15]
I200207 16:33:55.555991 14 storage/replica_command.go:1601  [n1,replicate,s1,r28/1:/{Table/32-Max}] change replicas (add [(n2,s2):2] remove []): existing descriptor r28:/{Table/32-Max} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
I200207 16:33:55.562189 14 storage/replica_raft.go:248  [n1,s1,r28/1:/{Table/32-Max}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200207 16:33:55.562806 14 storage/replica_command.go:1601  [n1,replicate,s1,r28/1:/{Table/32-Max}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r28:/{Table/32-Max} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200207 16:33:55.583558 14 storage/replica_raft.go:248  [n1,s1,r28/1:/{Table/32-Max}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3LEARNER] next=4
I200207 16:33:55.586727 14 storage/store_snapshot.go:922  [n1,replicate,s1,r28/1:/{Table/32-Max}] sending LEARNER snapshot f16582d1 at applied index 19
I200207 16:33:55.586857 14 storage/store_snapshot.go:965  [n1,replicate,s1,r28/1:/{Table/32-Max}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:55.587082 2706 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r28/1:/{Table/32-Max}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200207 16:33:55.587860 2704 storage/replica_raftstorage.go:780  [n3,s3,r28/3:{-}] applying LEARNER snapshot [id=f16582d1 index=19]
I200207 16:33:55.591132 2704 storage/replica_raftstorage.go:801  [n3,s3,r28/3:/{Table/32-Max}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=f16582d1 index=19]
I200207 16:33:55.592048 14 storage/replica_command.go:1601  [n1,replicate,s1,r28/1:/{Table/32-Max}] change replicas (add [(n3,s3):3] remove []): existing descriptor r28:/{Table/32-Max} [(n1,s1):1, (n2,s2):2, (n3,s3):3LEARNER, next=4, gen=3]
I200207 16:33:55.602661 14 storage/replica_raft.go:248  [n1,s1,r28/1:/{Table/32-Max}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3] next=4
I200207 16:33:55.605296 14 storage/replica_command.go:1601  [n1,replicate,s1,r4/1:/System{/tsd-tse}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r4:/System{/tsd-tse} [(n1,s1):1, next=2, gen=0]
I200207 16:33:55.612603 14 storage/replica_raft.go:248  [n1,s1,r4/1:/System{/tsd-tse}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200207 16:33:55.614541 14 storage/store_snapshot.go:922  [n1,replicate,s1,r4/1:/System{/tsd-tse}] sending LEARNER snapshot 04c91510 at applied index 27
I200207 16:33:55.614752 2853 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r4/1:/System{/tsd-tse}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):2LEARNER
I200207 16:33:55.694931 14 storage/store_snapshot.go:965  [n1,replicate,s1,r4/1:/System{/tsd-tse}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 2594, log entries: 0, rate-limit: 8.0 MiB/sec, 0.08s
I200207 16:33:55.700633 2851 storage/replica_raftstorage.go:780  [n2,s2,r4/2:{-}] applying LEARNER snapshot [id=04c91510 index=27]
I200207 16:33:55.720400 2851 storage/replica_raftstorage.go:801  [n2,s2,r4/2:/System{/tsd-tse}] applied LEARNER snapshot [total=20ms ingestion=4@7ms id=04c91510 index=27]
I200207 16:33:55.728691 14 storage/replica_command.go:1601  [n1,replicate,s1,r4/1:/System{/tsd-tse}] change replicas (add [(n2,s2):2] remove []): existing descriptor r4:/System{/tsd-tse} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
I200207 16:33:55.736406 14 storage/replica_raft.go:248  [n1,s1,r4/1:/System{/tsd-tse}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200207 16:33:55.736994 14 storage/replica_command.go:1601  [n1,replicate,s1,r4/1:/System{/tsd-tse}] change replicas (add [(n4,s4):3LEARNER] remove []): existing descriptor r4:/System{/tsd-tse} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200207 16:33:55.757235 14 storage/replica_raft.go:248  [n1,s1,r4/1:/System{/tsd-tse}] proposing SIMPLE(l3) ADD_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3LEARNER] next=4
I200207 16:33:55.760095 2861 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r4/1:/System{/tsd-tse}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):3LEARNER
I200207 16:33:55.761065 14 storage/store_snapshot.go:922  [n1,replicate,s1,r4/1:/System{/tsd-tse}] sending LEARNER snapshot 6dad92a3 at applied index 31
I200207 16:33:55.821309 14 storage/store_snapshot.go:965  [n1,replicate,s1,r4/1:/System{/tsd-tse}] streamed snapshot to (n4,s4):3LEARNER: kv pairs: 2598, log entries: 0, rate-limit: 8.0 MiB/sec, 0.06s
I200207 16:33:55.821774 2872 storage/replica_raftstorage.go:780  [n4,s4,r4/3:{-}] applying LEARNER snapshot [id=6dad92a3 index=31]
I200207 16:33:55.825953 2872 storage/replica_raftstorage.go:801  [n4,s4,r4/3:/System{/tsd-tse}] applied LEARNER snapshot [total=4ms ingestion=4@4ms id=6dad92a3 index=31]
I200207 16:33:55.827788 14 storage/replica_command.go:1601  [n1,replicate,s1,r4/1:/System{/tsd-tse}] change replicas (add [(n4,s4):3] remove []): existing descriptor r4:/System{/tsd-tse} [(n1,s1):1, (n2,s2):2, (n4,s4):3LEARNER, next=4, gen=3]
I200207 16:33:55.839576 14 storage/replica_raft.go:248  [n1,s1,r4/1:/System{/tsd-tse}] proposing SIMPLE(v3) ADD_REPLICA[(n4,s4):3]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3] next=4
I200207 16:33:55.842614 14 storage/replica_command.go:1601  [n1,replicate,s1,r20/1:/Table/2{4-5}] change replicas (add [(n3,s3):2LEARNER] remove []): existing descriptor r20:/Table/2{4-5} [(n1,s1):1, next=2, gen=0]
I200207 16:33:55.854522 14 storage/replica_raft.go:248  [n1,s1,r20/1:/Table/2{4-5}] proposing SIMPLE(l2) ADD_REPLICA[(n3,s3):2LEARNER]: after=[(n1,s1):1 (n3,s3):2LEARNER] next=3
I200207 16:33:55.860512 2783 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r20/1:/Table/2{4-5}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):2LEARNER
I200207 16:33:55.860670 14 storage/store_snapshot.go:922  [n1,replicate,s1,r20/1:/Table/2{4-5}] sending LEARNER snapshot 29823e8c at applied index 15
I200207 16:33:55.860784 14 storage/store_snapshot.go:965  [n1,replicate,s1,r20/1:/Table/2{4-5}] streamed snapshot to (n3,s3):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:55.861544 2800 storage/replica_raftstorage.go:780  [n3,s3,r20/2:{-}] applying LEARNER snapshot [id=29823e8c index=15]
I200207 16:33:55.865337 2800 storage/replica_raftstorage.go:801  [n3,s3,r20/2:/Table/2{4-5}] applied LEARNER snapshot [total=4ms ingestion=4@3ms id=29823e8c index=15]
I200207 16:33:55.869207 14 storage/replica_command.go:1601  [n1,replicate,s1,r20/1:/Table/2{4-5}] change replicas (add [(n3,s3):2] remove []): existing descriptor r20:/Table/2{4-5} [(n1,s1):1, (n3,s3):2LEARNER, next=3, gen=1]
I200207 16:33:55.895091 14 storage/replica_raft.go:248  [n1,s1,r20/1:/Table/2{4-5}] proposing SIMPLE(v2) ADD_REPLICA[(n3,s3):2]: after=[(n1,s1):1 (n3,s3):2] next=3
I200207 16:33:55.895615 14 storage/replica_command.go:1601  [n1,replicate,s1,r20/1:/Table/2{4-5}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r20:/Table/2{4-5} [(n1,s1):1, (n3,s3):2, next=3, gen=2]
I200207 16:33:55.933267 14 storage/replica_raft.go:248  [n1,s1,r20/1:/Table/2{4-5}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n2,s2):3LEARNER] next=4
I200207 16:33:55.935642 2880 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r20/1:/Table/2{4-5}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):3LEARNER
I200207 16:33:55.935750 14 storage/store_snapshot.go:922  [n1,replicate,s1,r20/1:/Table/2{4-5}] sending LEARNER snapshot 88727678 at applied index 19
I200207 16:33:55.935890 14 storage/store_snapshot.go:965  [n1,replicate,s1,r20/1:/Table/2{4-5}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:55.936512 2967 storage/replica_raftstorage.go:780  [n2,s2,r20/3:{-}] applying LEARNER snapshot [id=88727678 index=19]
I200207 16:33:55.953071 2967 storage/replica_raftstorage.go:801  [n2,s2,r20/3:/Table/2{4-5}] applied LEARNER snapshot [total=17ms ingestion=4@10ms id=88727678 index=19]
I200207 16:33:55.955626 14 storage/replica_command.go:1601  [n1,replicate,s1,r20/1:/Table/2{4-5}] change replicas (add [(n2,s2):3] remove []): existing descriptor r20:/Table/2{4-5} [(n1,s1):1, (n3,s3):2, (n2,s2):3LEARNER, next=4, gen=3]
I200207 16:33:55.963226 14 storage/replica_raft.go:248  [n1,s1,r20/1:/Table/2{4-5}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n3,s3):2 (n2,s2):3] next=4
I200207 16:33:55.967776 14 storage/replica_command.go:1601  [n1,replicate,s1,r16/1:/Table/2{0-1}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r16:/Table/2{0-1} [(n1,s1):1, next=2, gen=0]
I200207 16:33:55.976578 14 storage/replica_raft.go:248  [n1,s1,r16/1:/Table/2{0-1}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200207 16:33:55.978142 3018 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r16/1:/Table/2{0-1}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200207 16:33:55.981443 14 storage/store_snapshot.go:922  [n1,replicate,s1,r16/1:/Table/2{0-1}] sending LEARNER snapshot b6bd1437 at applied index 15
I200207 16:33:55.981565 14 storage/store_snapshot.go:965  [n1,replicate,s1,r16/1:/Table/2{0-1}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:55.982141 3003 storage/replica_raftstorage.go:780  [n4,s4,r16/2:{-}] applying LEARNER snapshot [id=b6bd1437 index=15]
I200207 16:33:55.985195 3003 storage/replica_raftstorage.go:801  [n4,s4,r16/2:/Table/2{0-1}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=b6bd1437 index=15]
I200207 16:33:55.985883 14 storage/replica_command.go:1601  [n1,replicate,s1,r16/1:/Table/2{0-1}] change replicas (add [(n4,s4):2] remove []): existing descriptor r16:/Table/2{0-1} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200207 16:33:55.999687 14 storage/replica_raft.go:248  [n1,s1,r16/1:/Table/2{0-1}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200207 16:33:56.000290 14 storage/replica_command.go:1601  [n1,replicate,s1,r16/1:/Table/2{0-1}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r16:/Table/2{0-1} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200207 16:33:56.021931 14 storage/replica_raft.go:248  [n1,s1,r16/1:/Table/2{0-1}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3LEARNER] next=4
I200207 16:33:56.033381 3053 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r16/1:/Table/2{0-1}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):3LEARNER
I200207 16:33:56.033943 14 storage/store_snapshot.go:922  [n1,replicate,s1,r16/1:/Table/2{0-1}] sending LEARNER snapshot 691ac78b at applied index 19
I200207 16:33:56.034132 14 storage/store_snapshot.go:965  [n1,replicate,s1,r16/1:/Table/2{0-1}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:56.042112 3055 storage/replica_raftstorage.go:780  [n2,s2,r16/3:{-}] applying LEARNER snapshot [id=691ac78b index=19]
I200207 16:33:56.055545 107 gossip/gossip.go:1532  [n1] node has connected to cluster via gossip
I200207 16:33:56.055800 107 storage/stores.go:259  [n1] wrote 3 node addresses to persistent storage
I200207 16:33:56.066944 3055 storage/replica_raftstorage.go:801  [n2,s2,r16/3:/Table/2{0-1}] applied LEARNER snapshot [total=25ms ingestion=4@24ms id=691ac78b index=19]
I200207 16:33:56.073955 14 storage/replica_command.go:1601  [n1,replicate,s1,r16/1:/Table/2{0-1}] change replicas (add [(n2,s2):3] remove []): existing descriptor r16:/Table/2{0-1} [(n1,s1):1, (n4,s4):2, (n2,s2):3LEARNER, next=4, gen=3]
I200207 16:33:56.081344 14 storage/replica_raft.go:248  [n1,s1,r16/1:/Table/2{0-1}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3] next=4
I200207 16:33:56.089179 14 storage/replica_command.go:1601  [n1,replicate,s1,r25/1:/{Table/29-NamespaceTab…}] change replicas (add [(n3,s3):2LEARNER] remove []): existing descriptor r25:/{Table/29-NamespaceTable/30} [(n1,s1):1, next=2, gen=0]
I200207 16:33:56.131211 14 storage/replica_raft.go:248  [n1,s1,r25/1:/{Table/29-NamespaceTab…}] proposing SIMPLE(l2) ADD_REPLICA[(n3,s3):2LEARNER]: after=[(n1,s1):1 (n3,s3):2LEARNER] next=3
I200207 16:33:56.133051 911 gossip/gossip.go:1532  [n2] node has connected to cluster via gossip
I200207 16:33:56.133367 911 storage/stores.go:259  [n2] wrote 3 node addresses to persistent storage
I200207 16:33:56.133059 3075 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r25/1:/{Table/29-NamespaceTab…}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):2LEARNER
I200207 16:33:56.133586 14 storage/store_snapshot.go:922  [n1,replicate,s1,r25/1:/{Table/29-NamespaceTab…}] sending LEARNER snapshot 063afdd0 at applied index 15
I200207 16:33:56.133716 14 storage/store_snapshot.go:965  [n1,replicate,s1,r25/1:/{Table/29-NamespaceTab…}] streamed snapshot to (n3,s3):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:56.134336 3073 storage/replica_raftstorage.go:780  [n3,s3,r25/2:{-}] applying LEARNER snapshot [id=063afdd0 index=15]
I200207 16:33:56.137630 3073 storage/replica_raftstorage.go:801  [n3,s3,r25/2:/{Table/29-NamespaceTab…}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=063afdd0 index=15]
I200207 16:33:56.144604 14 storage/replica_command.go:1601  [n1,replicate,s1,r25/1:/{Table/29-NamespaceTab…}] change replicas (add [(n3,s3):2] remove []): existing descriptor r25:/{Table/29-NamespaceTable/30} [(n1,s1):1, (n3,s3):2LEARNER, next=3, gen=1]
I200207 16:33:56.150594 14 storage/replica_raft.go:248  [n1,s1,r25/1:/{Table/29-NamespaceTab…}] proposing SIMPLE(v2) ADD_REPLICA[(n3,s3):2]: after=[(n1,s1):1 (n3,s3):2] next=3
I200207 16:33:56.151180 14 storage/replica_command.go:1601  [n1,replicate,s1,r25/1:/{Table/29-NamespaceTab…}] change replicas (add [(n4,s4):3LEARNER] remove []): existing descriptor r25:/{Table/29-NamespaceTable/30} [(n1,s1):1, (n3,s3):2, next=3, gen=2]
I200207 16:33:56.161243 14 storage/replica_raft.go:248  [n1,s1,r25/1:/{Table/29-NamespaceTab…}] proposing SIMPLE(l3) ADD_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3LEARNER] next=4
I200207 16:33:56.172637 14 storage/store_snapshot.go:922  [n1,replicate,s1,r25/1:/{Table/29-NamespaceTab…}] sending LEARNER snapshot ff4a965b at applied index 19
I200207 16:33:56.172767 14 storage/store_snapshot.go:965  [n1,replicate,s1,r25/1:/{Table/29-NamespaceTab…}] streamed snapshot to (n4,s4):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:56.172973 2962 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r25/1:/{Table/29-NamespaceTab…}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):3LEARNER
I200207 16:33:56.174077 2960 storage/replica_raftstorage.go:780  [n4,s4,r25/3:{-}] applying LEARNER snapshot [id=ff4a965b index=19]
I200207 16:33:56.177026 2960 storage/replica_raftstorage.go:801  [n4,s4,r25/3:/{Table/29-NamespaceTab…}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=ff4a965b index=19]
I200207 16:33:56.177563 14 storage/replica_command.go:1601  [n1,replicate,s1,r25/1:/{Table/29-NamespaceTab…}] change replicas (add [(n4,s4):3] remove []): existing descriptor r25:/{Table/29-NamespaceTable/30} [(n1,s1):1, (n3,s3):2, (n4,s4):3LEARNER, next=4, gen=3]
I200207 16:33:56.186340 14 storage/replica_raft.go:248  [n1,s1,r25/1:/{Table/29-NamespaceTab…}] proposing SIMPLE(v3) ADD_REPLICA[(n4,s4):3]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3] next=4
I200207 16:33:56.189532 14 storage/replica_command.go:1601  [n1,replicate,s1,r21/1:/Table/2{5-6}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r21:/Table/2{5-6} [(n1,s1):1, next=2, gen=0]
I200207 16:33:56.203319 14 storage/replica_raft.go:248  [n1,s1,r21/1:/Table/2{5-6}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200207 16:33:56.208458 3183 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r21/1:/Table/2{5-6}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):2LEARNER
I200207 16:33:56.208613 14 storage/store_snapshot.go:922  [n1,replicate,s1,r21/1:/Table/2{5-6}] sending LEARNER snapshot 386831d9 at applied index 17
I200207 16:33:56.208735 14 storage/store_snapshot.go:965  [n1,replicate,s1,r21/1:/Table/2{5-6}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 7, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:56.219156 3193 storage/replica_raftstorage.go:780  [n2,s2,r21/2:{-}] applying LEARNER snapshot [id=386831d9 index=17]
I200207 16:33:56.242787 3193 storage/replica_raftstorage.go:801  [n2,s2,r21/2:/Table/2{5-6}] applied LEARNER snapshot [total=24ms ingestion=4@23ms id=386831d9 index=17]
I200207 16:33:56.243250 14 storage/replica_command.go:1601  [n1,replicate,s1,r21/1:/Table/2{5-6}] change replicas (add [(n2,s2):2] remove []): existing descriptor r21:/Table/2{5-6} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
I200207 16:33:56.251708 14 storage/replica_raft.go:248  [n1,s1,r21/1:/Table/2{5-6}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200207 16:33:56.254290 14 storage/replica_command.go:1601  [n1,replicate,s1,r21/1:/Table/2{5-6}] change replicas (add [(n4,s4):3LEARNER] remove []): existing descriptor r21:/Table/2{5-6} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200207 16:33:56.266693 14 storage/replica_raft.go:248  [n1,s1,r21/1:/Table/2{5-6}] proposing SIMPLE(l3) ADD_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3LEARNER] next=4
I200207 16:33:56.269381 3225 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r21/1:/Table/2{5-6}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):3LEARNER
I200207 16:33:56.269586 14 storage/store_snapshot.go:922  [n1,replicate,s1,r21/1:/Table/2{5-6}] sending LEARNER snapshot 1fff347b at applied index 21
I200207 16:33:56.269714 14 storage/store_snapshot.go:965  [n1,replicate,s1,r21/1:/Table/2{5-6}] streamed snapshot to (n4,s4):3LEARNER: kv pairs: 11, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:56.270541 3256 storage/replica_raftstorage.go:780  [n4,s4,r21/3:{-}] applying LEARNER snapshot [id=1fff347b index=21]
I200207 16:33:56.275535 3256 storage/replica_raftstorage.go:801  [n4,s4,r21/3:/Table/2{5-6}] applied LEARNER snapshot [total=5ms ingestion=4@4ms id=1fff347b index=21]
I200207 16:33:56.281209 14 storage/replica_command.go:1601  [n1,replicate,s1,r21/1:/Table/2{5-6}] change replicas (add [(n4,s4):3] remove []): existing descriptor r21:/Table/2{5-6} [(n1,s1):1, (n2,s2):2, (n4,s4):3LEARNER, next=4, gen=3]
I200207 16:33:56.287579 14 storage/replica_raft.go:248  [n1,s1,r21/1:/Table/2{5-6}] proposing SIMPLE(v3) ADD_REPLICA[(n4,s4):3]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3] next=4
I200207 16:33:56.290625 14 storage/replica_command.go:1601  [n1,replicate,s1,r23/1:/Table/2{7-8}] change replicas (add [(n3,s3):2LEARNER] remove []): existing descriptor r23:/Table/2{7-8} [(n1,s1):1, next=2, gen=0]
I200207 16:33:56.297025 14 storage/replica_raft.go:248  [n1,s1,r23/1:/Table/2{7-8}] proposing SIMPLE(l2) ADD_REPLICA[(n3,s3):2LEARNER]: after=[(n1,s1):1 (n3,s3):2LEARNER] next=3
I200207 16:33:56.300601 14 storage/store_snapshot.go:922  [n1,replicate,s1,r23/1:/Table/2{7-8}] sending LEARNER snapshot 06308782 at applied index 15
I200207 16:33:56.300713 14 storage/store_snapshot.go:965  [n1,replicate,s1,r23/1:/Table/2{7-8}] streamed snapshot to (n3,s3):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:56.300773 3266 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r23/1:/Table/2{7-8}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):2LEARNER
I200207 16:33:56.301293 3264 storage/replica_raftstorage.go:780  [n3,s3,r23/2:{-}] applying LEARNER snapshot [id=06308782 index=15]
I200207 16:33:56.304616 3264 storage/replica_raftstorage.go:801  [n3,s3,r23/2:/Table/2{7-8}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=06308782 index=15]
I200207 16:33:56.309607 14 storage/replica_command.go:1601  [n1,replicate,s1,r23/1:/Table/2{7-8}] change replicas (add [(n3,s3):2] remove []): existing descriptor r23:/Table/2{7-8} [(n1,s1):1, (n3,s3):2LEARNER, next=3, gen=1]
I200207 16:33:56.324830 14 storage/replica_raft.go:248  [n1,s1,r23/1:/Table/2{7-8}] proposing SIMPLE(v2) ADD_REPLICA[(n3,s3):2]: after=[(n1,s1):1 (n3,s3):2] next=3
I200207 16:33:56.325509 14 storage/replica_command.go:1601  [n1,replicate,s1,r23/1:/Table/2{7-8}] change replicas (add [(n4,s4):3LEARNER] remove []): existing descriptor r23:/Table/2{7-8} [(n1,s1):1, (n3,s3):2, next=3, gen=2]
I200207 16:33:56.335351 14 storage/replica_raft.go:248  [n1,s1,r23/1:/Table/2{7-8}] proposing SIMPLE(l3) ADD_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3LEARNER] next=4
I200207 16:33:56.338249 14 storage/store_snapshot.go:922  [n1,replicate,s1,r23/1:/Table/2{7-8}] sending LEARNER snapshot 4dd9cd1c at applied index 19
I200207 16:33:56.338388 3338 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r23/1:/Table/2{7-8}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):3LEARNER
I200207 16:33:56.338398 14 storage/store_snapshot.go:965  [n1,replicate,s1,r23/1:/Table/2{7-8}] streamed snapshot to (n4,s4):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:56.338999 3336 storage/replica_raftstorage.go:780  [n4,s4,r23/3:{-}] applying LEARNER snapshot [id=4dd9cd1c index=19]
I200207 16:33:56.342256 3336 storage/replica_raftstorage.go:801  [n4,s4,r23/3:/Table/2{7-8}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=4dd9cd1c index=19]
I200207 16:33:56.347336 14 storage/replica_command.go:1601  [n1,replicate,s1,r23/1:/Table/2{7-8}] change replicas (add [(n4,s4):3] remove []): existing descriptor r23:/Table/2{7-8} [(n1,s1):1, (n3,s3):2, (n4,s4):3LEARNER, next=4, gen=3]
I200207 16:33:56.359159 14 storage/replica_raft.go:248  [n1,s1,r23/1:/Table/2{7-8}] proposing SIMPLE(v3) ADD_REPLICA[(n4,s4):3]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3] next=4
I200207 16:33:56.362707 14 storage/replica_command.go:1601  [n1,replicate,s1,r12/1:/Table/1{6-7}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r12:/Table/1{6-7} [(n1,s1):1, next=2, gen=0]
I200207 16:33:56.374943 14 storage/replica_raft.go:248  [n1,s1,r12/1:/Table/1{6-7}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200207 16:33:56.376778 14 storage/store_snapshot.go:922  [n1,replicate,s1,r12/1:/Table/1{6-7}] sending LEARNER snapshot 8612a807 at applied index 15
I200207 16:33:56.376855 3380 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r12/1:/Table/1{6-7}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):2LEARNER
I200207 16:33:56.377018 14 storage/store_snapshot.go:965  [n1,replicate,s1,r12/1:/Table/1{6-7}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:56.377800 3353 storage/replica_raftstorage.go:780  [n2,s2,r12/2:{-}] applying LEARNER snapshot [id=8612a807 index=15]
I200207 16:33:56.381595 3353 storage/replica_raftstorage.go:801  [n2,s2,r12/2:/Table/1{6-7}] applied LEARNER snapshot [total=4ms ingestion=4@3ms id=8612a807 index=15]
I200207 16:33:56.382250 14 storage/replica_command.go:1601  [n1,replicate,s1,r12/1:/Table/1{6-7}] change replicas (add [(n2,s2):2] remove []): existing descriptor r12:/Table/1{6-7} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
I200207 16:33:56.392059 14 storage/replica_raft.go:248  [n1,s1,r12/1:/Table/1{6-7}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200207 16:33:56.392624 14 storage/replica_command.go:1601  [n1,replicate,s1,r12/1:/Table/1{6-7}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r12:/Table/1{6-7} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200207 16:33:56.393838 1272 gossip/gossip.go:1532  [n3] node has connected to cluster via gossip
I200207 16:33:56.394001 1272 storage/stores.go:259  [n3] wrote 3 node addresses to persistent storage
I200207 16:33:56.408709 14 storage/replica_raft.go:248  [n1,s1,r12/1:/Table/1{6-7}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3LEARNER] next=4
I200207 16:33:56.411518 14 storage/store_snapshot.go:922  [n1,replicate,s1,r12/1:/Table/1{6-7}] sending LEARNER snapshot 063588ef at applied index 19
I200207 16:33:56.411658 3314 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r12/1:/Table/1{6-7}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200207 16:33:56.411731 14 storage/store_snapshot.go:965  [n1,replicate,s1,r12/1:/Table/1{6-7}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:56.412385 3434 storage/replica_raftstorage.go:780  [n3,s3,r12/3:{-}] applying LEARNER snapshot [id=063588ef index=19]
I200207 16:33:56.417799 3460 storage/replica_command.go:1601  [n2,replicate,s2,r19/2:/Table/2{3-4}] change replicas (add [(n3,s3):4LEARNER] remove []): existing descriptor r19:/Table/2{3-4} [(n1,s1):1, (n2,s2):2, (n4,s4):3, next=4, gen=4]
I200207 16:33:56.419559 3434 storage/replica_raftstorage.go:801  [n3,s3,r12/3:/Table/1{6-7}] applied LEARNER snapshot [total=7ms ingestion=4@4ms id=063588ef index=19]
I200207 16:33:56.420006 14 storage/replica_command.go:1601  [n1,replicate,s1,r12/1:/Table/1{6-7}] change replicas (add [(n3,s3):3] remove []): existing descriptor r12:/Table/1{6-7} [(n1,s1):1, (n2,s2):2, (n3,s3):3LEARNER, next=4, gen=3]
I200207 16:33:56.427968 14 storage/replica_raft.go:248  [n1,s1,r12/1:/Table/1{6-7}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3] next=4
I200207 16:33:56.430632 14 storage/replica_command.go:1601  [n1,replicate,s1,r2/1:/System/NodeLiveness{-Max}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r2:/System/NodeLiveness{-Max} [(n1,s1):1, next=2, gen=0]
I200207 16:33:56.454679 14 storage/replica_raft.go:248  [n1,s1,r2/1:/System/NodeLiveness{-Max}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200207 16:33:56.456407 14 storage/store_snapshot.go:922  [n1,replicate,s1,r2/1:/System/NodeLiveness{-Max}] sending LEARNER snapshot 1050eb1e at applied index 24
I200207 16:33:56.456530 14 storage/store_snapshot.go:965  [n1,replicate,s1,r2/1:/System/NodeLiveness{-Max}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:56.456569 3540 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r2/1:/System/NodeLiveness{-Max}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):2LEARNER
I200207 16:33:56.457180 3483 storage/replica_raftstorage.go:780  [n2,s2,r2/2:{-}] applying LEARNER snapshot [id=1050eb1e index=24]
I200207 16:33:56.460518 3483 storage/replica_raftstorage.go:801  [n2,s2,r2/2:/System/NodeLiveness{-Max}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=1050eb1e index=24]
I200207 16:33:56.461030 14 storage/replica_command.go:1601  [n1,replicate,s1,r2/1:/System/NodeLiveness{-Max}] change replicas (add [(n2,s2):2] remove []): existing descriptor r2:/System/NodeLiveness{-Max} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
I200207 16:33:56.473957 14 storage/replica_raft.go:248  [n1,s1,r2/1:/System/NodeLiveness{-Max}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200207 16:33:56.475502 14 storage/replica_command.go:1601  [n1,replicate,s1,r2/1:/System/NodeLiveness{-Max}] change replicas (add [(n4,s4):3LEARNER] remove []): existing descriptor r2:/System/NodeLiveness{-Max} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200207 16:33:56.490352 14 storage/replica_raft.go:248  [n1,s1,r2/1:/System/NodeLiveness{-Max}] proposing SIMPLE(l3) ADD_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3LEARNER] next=4
I200207 16:33:56.494268 14 storage/store_snapshot.go:922  [n1,replicate,s1,r2/1:/System/NodeLiveness{-Max}] sending LEARNER snapshot 47962fa0 at applied index 28
I200207 16:33:56.494394 3564 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r2/1:/System/NodeLiveness{-Max}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):3LEARNER
I200207 16:33:56.494421 14 storage/store_snapshot.go:965  [n1,replicate,s1,r2/1:/System/NodeLiveness{-Max}] streamed snapshot to (n4,s4):3LEARNER: kv pairs: 14, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:56.495268 3547 storage/replica_raftstorage.go:780  [n4,s4,r2/3:{-}] applying LEARNER snapshot [id=47962fa0 index=28]
I200207 16:33:56.506951 3460 storage/replica_raft.go:248  [n2,s2,r19/2:/Table/2{3-4}] proposing SIMPLE(l4) ADD_REPLICA[(n3,s3):4LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3 (n3,s3):4LEARNER] next=5
I200207 16:33:56.509760 3460 storage/store_snapshot.go:922  [n2,replicate,s2,r19/2:/Table/2{3-4}] sending LEARNER snapshot 7547f6a4 at applied index 32
I200207 16:33:56.509964 3460 storage/store_snapshot.go:965  [n2,replicate,s2,r19/2:/Table/2{3-4}] streamed snapshot to (n3,s3):4LEARNER: kv pairs: 15, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:56.510621 3535 storage/replica_raftstorage.go:780  [n3,s3,r19/4:{-}] applying LEARNER snapshot [id=7547f6a4 index=32]
I200207 16:33:56.516529 3535 storage/replica_raftstorage.go:801  [n3,s3,r19/4:/Table/2{3-4}] applied LEARNER snapshot [total=6ms ingestion=4@5ms id=7547f6a4 index=32]
I200207 16:33:56.517741 3460 storage/replica_command.go:1601  [n2,replicate,s2,r19/2:/Table/2{3-4}] change replicas (add [(n3,s3):4VOTER_INCOMING] remove [(n1,s1):1VOTER_DEMOTING]): existing descriptor r19:/Table/2{3-4} [(n1,s1):1, (n2,s2):2, (n4,s4):3, (n3,s3):4LEARNER, next=5, gen=5]
I200207 16:33:56.521113 3547 storage/replica_raftstorage.go:801  [n4,s4,r2/3:/System/NodeLiveness{-Max}] applied LEARNER snapshot [total=26ms ingestion=4@12ms id=47962fa0 index=28]
I200207 16:33:56.521656 14 storage/replica_command.go:1601  [n1,replicate,s1,r2/1:/System/NodeLiveness{-Max}] change replicas (add [(n4,s4):3] remove []): existing descriptor r2:/System/NodeLiveness{-Max} [(n1,s1):1, (n2,s2):2, (n4,s4):3LEARNER, next=4, gen=3]
I200207 16:33:56.533925 14 storage/replica_raft.go:248  [n1,s1,r2/1:/System/NodeLiveness{-Max}] proposing SIMPLE(v3) ADD_REPLICA[(n4,s4):3]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3] next=4
I200207 16:33:56.543044 14 storage/replica_command.go:1601  [n1,replicate,s1,r22/1:/Table/2{6-7}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r22:/Table/2{6-7} [(n1,s1):1, next=2, gen=0]
I200207 16:33:56.545875 3460 storage/replica_raft.go:248  [n2,s2,r19/2:/Table/2{3-4}] proposing ENTER_JOINT(r1 l1 v4) ADD_REPLICA[(n3,s3):4VOTER_INCOMING], REMOVE_REPLICA[(n1,s1):1VOTER_DEMOTING]: after=[(n1,s1):1VOTER_DEMOTING (n2,s2):2 (n4,s4):3 (n3,s3):4VOTER_INCOMING] next=5
I200207 16:33:56.547469 3460 storage/replica_command.go:1601  [n2,replicate,s2,r19/2:/Table/2{3-4}] change replicas (add [] remove []): existing descriptor r19:/Table/2{3-4} [(n1,s1):1VOTER_DEMOTING, (n2,s2):2, (n4,s4):3, (n3,s3):4VOTER_INCOMING, next=5, gen=6]
I200207 16:33:56.551656 14 storage/replica_raft.go:248  [n1,s1,r22/1:/Table/2{6-7}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200207 16:33:56.553469 14 storage/store_snapshot.go:922  [n1,replicate,s1,r22/1:/Table/2{6-7}] sending LEARNER snapshot 3fc56e77 at applied index 15
I200207 16:33:56.553581 14 storage/store_snapshot.go:965  [n1,replicate,s1,r22/1:/Table/2{6-7}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:56.554830 3598 storage/replica_raftstorage.go:780  [n4,s4,r22/2:{-}] applying LEARNER snapshot [id=3fc56e77 index=15]
I200207 16:33:56.556358 3600 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r22/1:/Table/2{6-7}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200207 16:33:56.558290 3598 storage/replica_raftstorage.go:801  [n4,s4,r22/2:/Table/2{6-7}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=3fc56e77 index=15]
I200207 16:33:56.559218 14 storage/replica_command.go:1601  [n1,replicate,s1,r22/1:/Table/2{6-7}] change replicas (add [(n4,s4):2] remove []): existing descriptor r22:/Table/2{6-7} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200207 16:33:56.564369 3460 storage/replica_raft.go:248  [n2,s2,r19/2:/Table/2{3-4}] proposing LEAVE_JOINT: after=[(n1,s1):1LEARNER (n2,s2):2 (n4,s4):3 (n3,s3):4] next=5
I200207 16:33:56.565927 3460 storage/replica_command.go:1601  [n2,replicate,s2,r19/2:/Table/2{3-4}] change replicas (add [] remove [(n1,s1):1LEARNER]): existing descriptor r19:/Table/2{3-4} [(n1,s1):1LEARNER, (n2,s2):2, (n4,s4):3, (n3,s3):4, next=5, gen=7]
I200207 16:33:56.573540 14 storage/replica_raft.go:248  [n1,s1,r22/1:/Table/2{6-7}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200207 16:33:56.574157 14 storage/replica_command.go:1601  [n1,replicate,s1,r22/1:/Table/2{6-7}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r22:/Table/2{6-7} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200207 16:33:56.580422 3460 storage/replica_raft.go:248  [n2,s2,r19/2:/Table/2{3-4}] proposing SIMPLE(r1) REMOVE_REPLICA[(n1,s1):1LEARNER]: after=[(n3,s3):4 (n2,s2):2 (n4,s4):3] next=5
I200207 16:33:56.583319 154 storage/store_remove_replica.go:122  [n1,s1,r19/1:/Table/2{3-4}] removing replica r19/1
I200207 16:33:56.589168 14 storage/replica_raft.go:248  [n1,s1,r22/1:/Table/2{6-7}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3LEARNER] next=4
I200207 16:33:56.599275 14 storage/store_snapshot.go:922  [n1,replicate,s1,r22/1:/Table/2{6-7}] sending LEARNER snapshot a6bcd80e at applied index 19
I200207 16:33:56.599804 14 storage/store_snapshot.go:965  [n1,replicate,s1,r22/1:/Table/2{6-7}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:56.601117 3729 storage/replica_raftstorage.go:780  [n3,s3,r22/3:{-}] applying LEARNER snapshot [id=a6bcd80e index=19]
I200207 16:33:56.604717 3615 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r22/1:/Table/2{6-7}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200207 16:33:56.609038 3729 storage/replica_raftstorage.go:801  [n3,s3,r22/3:/Table/2{6-7}] applied LEARNER snapshot [total=8ms ingestion=4@7ms id=a6bcd80e index=19]
I200207 16:33:56.613560 14 storage/replica_command.go:1601  [n1,replicate,s1,r22/1:/Table/2{6-7}] change replicas (add [(n3,s3):3] remove []): existing descriptor r22:/Table/2{6-7} [(n1,s1):1, (n4,s4):2, (n3,s3):3LEARNER, next=4, gen=3]
I200207 16:33:56.658580 14 storage/replica_raft.go:248  [n1,s1,r22/1:/Table/2{6-7}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3] next=4
I200207 16:33:56.661741 14 storage/replica_command.go:1601  [n1,replicate,s1,r18/1:/Table/2{2-3}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r18:/Table/2{2-3} [(n1,s1):1, next=2, gen=0]
I200207 16:33:56.677704 14 storage/replica_raft.go:248  [n1,s1,r18/1:/Table/2{2-3}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200207 16:33:56.679396 14 storage/store_snapshot.go:922  [n1,replicate,s1,r18/1:/Table/2{2-3}] sending LEARNER snapshot 67b8b222 at applied index 15
I200207 16:33:56.679524 14 storage/store_snapshot.go:965  [n1,replicate,s1,r18/1:/Table/2{2-3}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:56.680043 3747 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r18/1:/Table/2{2-3}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):2LEARNER
I200207 16:33:56.680361 3731 storage/replica_raftstorage.go:780  [n2,s2,r18/2:{-}] applying LEARNER snapshot [id=67b8b222 index=15]
I200207 16:33:56.685985 3731 storage/replica_raftstorage.go:801  [n2,s2,r18/2:/Table/2{2-3}] applied LEARNER snapshot [total=6ms ingestion=4@5ms id=67b8b222 index=15]
I200207 16:33:56.686884 14 storage/replica_command.go:1601  [n1,replicate,s1,r18/1:/Table/2{2-3}] change replicas (add [(n2,s2):2] remove []): existing descriptor r18:/Table/2{2-3} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
I200207 16:33:56.703897 14 storage/replica_raft.go:248  [n1,s1,r18/1:/Table/2{2-3}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200207 16:33:56.704786 14 storage/replica_command.go:1601  [n1,replicate,s1,r18/1:/Table/2{2-3}] change replicas (add [(n4,s4):3LEARNER] remove []): existing descriptor r18:/Table/2{2-3} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200207 16:33:56.721070 14 storage/replica_raft.go:248  [n1,s1,r18/1:/Table/2{2-3}] proposing SIMPLE(l3) ADD_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3LEARNER] next=4
I200207 16:33:56.725318 3861 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r18/1:/Table/2{2-3}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):3LEARNER
I200207 16:33:56.726084 14 storage/store_snapshot.go:922  [n1,replicate,s1,r18/1:/Table/2{2-3}] sending LEARNER snapshot 29d64b9d at applied index 19
I200207 16:33:56.726287 14 storage/store_snapshot.go:965  [n1,replicate,s1,r18/1:/Table/2{2-3}] streamed snapshot to (n4,s4):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:56.726920 3792 storage/replica_raftstorage.go:780  [n4,s4,r18/3:{-}] applying LEARNER snapshot [id=29d64b9d index=19]
I200207 16:33:56.746075 3792 storage/replica_raftstorage.go:801  [n4,s4,r18/3:/Table/2{2-3}] applied LEARNER snapshot [total=19ms ingestion=4@19ms id=29d64b9d index=19]
I200207 16:33:56.746682 14 storage/replica_command.go:1601  [n1,replicate,s1,r18/1:/Table/2{2-3}] change replicas (add [(n4,s4):3] remove []): existing descriptor r18:/Table/2{2-3} [(n1,s1):1, (n2,s2):2, (n4,s4):3LEARNER, next=4, gen=3]
I200207 16:33:56.755041 14 storage/replica_raft.go:248  [n1,s1,r18/1:/Table/2{2-3}] proposing SIMPLE(v3) ADD_REPLICA[(n4,s4):3]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3] next=4
I200207 16:33:56.768862 14 storage/replica_command.go:1601  [n1,replicate,s1,r11/1:/Table/1{5-6}] change replicas (add [(n2,s2):2LEARNER] remove []): existing descriptor r11:/Table/1{5-6} [(n1,s1):1, next=2, gen=0]
I200207 16:33:56.783399 14 storage/replica_raft.go:248  [n1,s1,r11/1:/Table/1{5-6}] proposing SIMPLE(l2) ADD_REPLICA[(n2,s2):2LEARNER]: after=[(n1,s1):1 (n2,s2):2LEARNER] next=3
I200207 16:33:56.785796 3896 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r11/1:/Table/1{5-6}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):2LEARNER
I200207 16:33:56.820781 14 storage/store_snapshot.go:922  [n1,replicate,s1,r11/1:/Table/1{5-6}] sending LEARNER snapshot 199774d4 at applied index 15
I200207 16:33:56.820936 14 storage/store_snapshot.go:965  [n1,replicate,s1,r11/1:/Table/1{5-6}] streamed snapshot to (n2,s2):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.04s
I200207 16:33:56.830086 3898 storage/replica_raftstorage.go:780  [n2,s2,r11/2:{-}] applying LEARNER snapshot [id=199774d4 index=15]
I200207 16:33:56.855836 3898 storage/replica_raftstorage.go:801  [n2,s2,r11/2:/Table/1{5-6}] applied LEARNER snapshot [total=26ms ingestion=4@25ms id=199774d4 index=15]
I200207 16:33:56.856520 14 storage/replica_command.go:1601  [n1,replicate,s1,r11/1:/Table/1{5-6}] change replicas (add [(n2,s2):2] remove []): existing descriptor r11:/Table/1{5-6} [(n1,s1):1, (n2,s2):2LEARNER, next=3, gen=1]
I200207 16:33:56.858020 3924 storage/replica_command.go:1601  [n4,replicate,s4,r13/3:/Table/1{7-8}] change replicas (add [(n3,s3):4LEARNER] remove []): existing descriptor r13:/Table/1{7-8} [(n1,s1):1, (n2,s2):2, (n4,s4):3, next=4, gen=4]
I200207 16:33:56.862913 14 storage/replica_raft.go:248  [n1,s1,r11/1:/Table/1{5-6}] proposing SIMPLE(v2) ADD_REPLICA[(n2,s2):2]: after=[(n1,s1):1 (n2,s2):2] next=3
I200207 16:33:56.865122 14 storage/replica_command.go:1601  [n1,replicate,s1,r11/1:/Table/1{5-6}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r11:/Table/1{5-6} [(n1,s1):1, (n2,s2):2, next=3, gen=2]
I200207 16:33:56.894610 14 storage/replica_raft.go:248  [n1,s1,r11/1:/Table/1{5-6}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3LEARNER] next=4
I200207 16:33:56.900152 14 storage/store_snapshot.go:922  [n1,replicate,s1,r11/1:/Table/1{5-6}] sending LEARNER snapshot 599428a5 at applied index 19
I200207 16:33:56.900295 14 storage/store_snapshot.go:965  [n1,replicate,s1,r11/1:/Table/1{5-6}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:56.900325 3974 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r11/1:/Table/1{5-6}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200207 16:33:56.900953 3972 storage/replica_raftstorage.go:780  [n3,s3,r11/3:{-}] applying LEARNER snapshot [id=599428a5 index=19]
I200207 16:33:56.907691 3972 storage/replica_raftstorage.go:801  [n3,s3,r11/3:/Table/1{5-6}] applied LEARNER snapshot [total=7ms ingestion=4@6ms id=599428a5 index=19]
I200207 16:33:56.908186 14 storage/replica_command.go:1601  [n1,replicate,s1,r11/1:/Table/1{5-6}] change replicas (add [(n3,s3):3] remove []): existing descriptor r11:/Table/1{5-6} [(n1,s1):1, (n2,s2):2, (n3,s3):3LEARNER, next=4, gen=3]
I200207 16:33:56.914251 14 storage/replica_raft.go:248  [n1,s1,r11/1:/Table/1{5-6}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n2,s2):2 (n3,s3):3] next=4
I200207 16:33:56.922506 14 storage/replica_command.go:1601  [n1,replicate,s1,r24/1:/Table/2{8-9}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r24:/Table/2{8-9} [(n1,s1):1, next=2, gen=0]
W200207 16:33:56.937782 3986 storage/intentresolver/intent_resolver.go:515  failed to push during intent resolution: failed to push meta={id=86ddd5be key=/Local/Range/Table/28/RangeDescriptor pri=0.04088068 epo=0 ts=1581093236.922349073,0 min=1581093236.922349073,0 seq=7} rw=true stat=PENDING rts=0,0 wto=false max=0,0
I200207 16:33:56.938017 14 storage/replica_raft.go:248  [n1,s1,r24/1:/Table/2{8-9}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200207 16:33:56.943495 3996 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r24/1:/Table/2{8-9}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200207 16:33:56.943747 14 storage/store_snapshot.go:922  [n1,replicate,s1,r24/1:/Table/2{8-9}] sending LEARNER snapshot 2280e65a at applied index 15
I200207 16:33:56.943897 14 storage/store_snapshot.go:965  [n1,replicate,s1,r24/1:/Table/2{8-9}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.01s
I200207 16:33:56.944545 4009 storage/replica_raftstorage.go:780  [n4,s4,r24/2:{-}] applying LEARNER snapshot [id=2280e65a index=15]
I200207 16:33:56.978680 4009 storage/replica_raftstorage.go:801  [n4,s4,r24/2:/Table/2{8-9}] applied LEARNER snapshot [total=34ms ingestion=4@34ms id=2280e65a index=15]
I200207 16:33:56.982115 14 storage/replica_command.go:1601  [n1,replicate,s1,r24/1:/Table/2{8-9}] change replicas (add [(n4,s4):2] remove []): existing descriptor r24:/Table/2{8-9} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200207 16:33:56.982132 3924 storage/replica_raft.go:248  [n4,s4,r13/3:/Table/1{7-8}] proposing SIMPLE(l4) ADD_REPLICA[(n3,s3):4LEARNER]: after=[(n1,s1):1 (n2,s2):2 (n4,s4):3 (n3,s3):4LEARNER] next=5
I200207 16:33:56.984758 3924 storage/store_snapshot.go:922  [n4,replicate,s4,r13/3:/Table/1{7-8}] sending LEARNER snapshot c717aa7f at applied index 31
I200207 16:33:56.984920 3924 storage/store_snapshot.go:965  [n4,replicate,s4,r13/3:/Table/1{7-8}] streamed snapshot to (n3,s3):4LEARNER: kv pairs: 11, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:56.985523 4027 storage/replica_raftstorage.go:780  [n3,s3,r13/4:{-}] applying LEARNER snapshot [id=c717aa7f index=31]
I200207 16:33:56.986654 3917 storage/raft_snapshot_queue.go:169  [n4,raftsnapshot,s4,r13/3:/Table/1{7-8}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):4LEARNER
I200207 16:33:56.989004 4027 storage/replica_raftstorage.go:801  [n3,s3,r13/4:/Table/1{7-8}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=c717aa7f index=31]
I200207 16:33:56.999400 3924 storage/replica_command.go:1601  [n4,replicate,s4,r13/3:/Table/1{7-8}] change replicas (add [(n3,s3):4VOTER_INCOMING] remove [(n1,s1):1VOTER_DEMOTING]): existing descriptor r13:/Table/1{7-8} [(n1,s1):1, (n2,s2):2, (n4,s4):3, (n3,s3):4LEARNER, next=5, gen=5]
I200207 16:33:57.001860 14 storage/replica_raft.go:248  [n1,s1,r24/1:/Table/2{8-9}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200207 16:33:57.005168 14 storage/replica_command.go:1601  [n1,replicate,s1,r24/1:/Table/2{8-9}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r24:/Table/2{8-9} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200207 16:33:57.037970 3924 storage/replica_raft.go:248  [n4,s4,r13/3:/Table/1{7-8}] proposing ENTER_JOINT(r1 l1 v4) ADD_REPLICA[(n3,s3):4VOTER_INCOMING], REMOVE_REPLICA[(n1,s1):1VOTER_DEMOTING]: after=[(n1,s1):1VOTER_DEMOTING (n2,s2):2 (n4,s4):3 (n3,s3):4VOTER_INCOMING] next=5
I200207 16:33:57.039765 3924 storage/replica_command.go:1601  [n4,replicate,s4,r13/3:/Table/1{7-8}] change replicas (add [] remove []): existing descriptor r13:/Table/1{7-8} [(n1,s1):1VOTER_DEMOTING, (n2,s2):2, (n4,s4):3, (n3,s3):4VOTER_INCOMING, next=5, gen=6]
I200207 16:33:57.052533 14 storage/replica_raft.go:248  [n1,s1,r24/1:/Table/2{8-9}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3LEARNER] next=4
I200207 16:33:57.059042 14 storage/store_snapshot.go:922  [n1,replicate,s1,r24/1:/Table/2{8-9}] sending LEARNER snapshot fa136ab3 at applied index 20
I200207 16:33:57.059162 14 storage/store_snapshot.go:965  [n1,replicate,s1,r24/1:/Table/2{8-9}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:57.059205 4137 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r24/1:/Table/2{8-9}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):3LEARNER
I200207 16:33:57.059712 4135 storage/replica_raftstorage.go:780  [n2,s2,r24/3:{-}] applying LEARNER snapshot [id=fa136ab3 index=20]
I200207 16:33:57.062916 4135 storage/replica_raftstorage.go:801  [n2,s2,r24/3:/Table/2{8-9}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=fa136ab3 index=20]
I200207 16:33:57.069527 14 storage/replica_command.go:1601  [n1,replicate,s1,r24/1:/Table/2{8-9}] change replicas (add [(n2,s2):3] remove []): existing descriptor r24:/Table/2{8-9} [(n1,s1):1, (n4,s4):2, (n2,s2):3LEARNER, next=4, gen=3]
I200207 16:33:57.080029 14 storage/replica_raft.go:248  [n1,s1,r24/1:/Table/2{8-9}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3] next=4
I200207 16:33:57.082315 14 storage/replica_command.go:1601  [n1,replicate,s1,r6/1:/Table/{SystemCon…-11}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r6:/Table/{SystemConfigSpan/Start-11} [(n1,s1):1, next=2, gen=0]
I200207 16:33:57.097871 14 storage/replica_raft.go:248  [n1,s1,r6/1:/Table/{SystemCon…-11}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200207 16:33:57.100643 3924 storage/replica_raft.go:248  [n4,s4,r13/3:/Table/1{7-8}] proposing LEAVE_JOINT: after=[(n1,s1):1LEARNER (n2,s2):2 (n4,s4):3 (n3,s3):4] next=5
I200207 16:33:57.102370 14 storage/store_snapshot.go:922  [n1,replicate,s1,r6/1:/Table/{SystemCon…-11}] sending LEARNER snapshot 03f0b103 at applied index 34
I200207 16:33:57.102642 14 storage/store_snapshot.go:965  [n1,replicate,s1,r6/1:/Table/{SystemCon…-11}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 50, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:57.102988 4153 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r6/1:/Table/{SystemCon…-11}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200207 16:33:57.103961 4175 storage/replica_raftstorage.go:780  [n4,s4,r6/2:{-}] applying LEARNER snapshot [id=03f0b103 index=34]
I200207 16:33:57.105860 3924 storage/replica_command.go:1601  [n4,replicate,s4,r13/3:/Table/1{7-8}] change replicas (add [] remove [(n1,s1):1LEARNER]): existing descriptor r13:/Table/1{7-8} [(n1,s1):1LEARNER, (n2,s2):2, (n4,s4):3, (n3,s3):4, next=5, gen=7]
I200207 16:33:57.121722 4175 storage/replica_raftstorage.go:801  [n4,s4,r6/2:/Table/{SystemCon…-11}] applied LEARNER snapshot [total=18ms ingestion=4@17ms id=03f0b103 index=34]
I200207 16:33:57.131718 14 storage/replica_command.go:1601  [n1,replicate,s1,r6/1:/Table/{SystemCon…-11}] change replicas (add [(n4,s4):2] remove []): existing descriptor r6:/Table/{SystemConfigSpan/Start-11} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200207 16:33:57.152764 3924 storage/replica_raft.go:248  [n4,s4,r13/3:/Table/1{7-8}] proposing SIMPLE(r1) REMOVE_REPLICA[(n1,s1):1LEARNER]: after=[(n3,s3):4 (n2,s2):2 (n4,s4):3] next=5
I200207 16:33:57.154165 167 storage/store_remove_replica.go:122  [n1,s1,r13/1:/Table/1{7-8}] removing replica r13/1
I200207 16:33:57.159318 14 storage/replica_raft.go:248  [n1,s1,r6/1:/Table/{SystemCon…-11}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200207 16:33:57.159982 14 storage/replica_command.go:1601  [n1,replicate,s1,r6/1:/Table/{SystemCon…-11}] change replicas (add [(n3,s3):3LEARNER] remove []): existing descriptor r6:/Table/{SystemConfigSpan/Start-11} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200207 16:33:57.180145 14 storage/replica_raft.go:248  [n1,s1,r6/1:/Table/{SystemCon…-11}] proposing SIMPLE(l3) ADD_REPLICA[(n3,s3):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3LEARNER] next=4
I200207 16:33:57.190881 14 storage/store_snapshot.go:922  [n1,replicate,s1,r6/1:/Table/{SystemCon…-11}] sending LEARNER snapshot 480340ac at applied index 38
I200207 16:33:57.191220 14 storage/store_snapshot.go:965  [n1,replicate,s1,r6/1:/Table/{SystemCon…-11}] streamed snapshot to (n3,s3):3LEARNER: kv pairs: 54, log entries: 0, rate-limit: 8.0 MiB/sec, 0.01s
I200207 16:33:57.204861 4162 storage/replica_raftstorage.go:780  [n3,s3,r6/3:{-}] applying LEARNER snapshot [id=480340ac index=38]
I200207 16:33:57.205083 4131 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r6/1:/Table/{SystemCon…-11}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):3LEARNER
I200207 16:33:57.255048 4162 storage/replica_raftstorage.go:801  [n3,s3,r6/3:/Table/{SystemCon…-11}] applied LEARNER snapshot [total=50ms ingestion=4@50ms id=480340ac index=38]
I200207 16:33:57.261602 14 storage/replica_command.go:1601  [n1,replicate,s1,r6/1:/Table/{SystemCon…-11}] change replicas (add [(n3,s3):3] remove []): existing descriptor r6:/Table/{SystemConfigSpan/Start-11} [(n1,s1):1, (n4,s4):2, (n3,s3):3LEARNER, next=4, gen=3]
I200207 16:33:57.272278 14 storage/replica_raft.go:248  [n1,s1,r6/1:/Table/{SystemCon…-11}] proposing SIMPLE(v3) ADD_REPLICA[(n3,s3):3]: after=[(n1,s1):1 (n4,s4):2 (n3,s3):3] next=4
I200207 16:33:57.276646 14 storage/replica_command.go:1601  [n1,replicate,s1,r14/1:/Table/1{8-9}] change replicas (add [(n3,s3):2LEARNER] remove []): existing descriptor r14:/Table/1{8-9} [(n1,s1):1, next=2, gen=0]
I200207 16:33:57.303082 1757 gossip/gossip.go:1532  [n4] node has connected to cluster via gossip
I200207 16:33:57.303271 1757 storage/stores.go:259  [n4] wrote 3 node addresses to persistent storage
I200207 16:33:57.334408 14 storage/replica_raft.go:248  [n1,s1,r14/1:/Table/1{8-9}] proposing SIMPLE(l2) ADD_REPLICA[(n3,s3):2LEARNER]: after=[(n1,s1):1 (n3,s3):2LEARNER] next=3
I200207 16:33:57.336131 14 storage/store_snapshot.go:922  [n1,replicate,s1,r14/1:/Table/1{8-9}] sending LEARNER snapshot 93240fa3 at applied index 15
I200207 16:33:57.336278 14 storage/store_snapshot.go:965  [n1,replicate,s1,r14/1:/Table/1{8-9}] streamed snapshot to (n3,s3):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:57.336497 4310 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r14/1:/Table/1{8-9}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):2LEARNER
I200207 16:33:57.337091 4308 storage/replica_raftstorage.go:780  [n3,s3,r14/2:{-}] applying LEARNER snapshot [id=93240fa3 index=15]
I200207 16:33:57.343664 4308 storage/replica_raftstorage.go:801  [n3,s3,r14/2:/Table/1{8-9}] applied LEARNER snapshot [total=7ms ingestion=4@6ms id=93240fa3 index=15]
I200207 16:33:57.344113 14 storage/replica_command.go:1601  [n1,replicate,s1,r14/1:/Table/1{8-9}] change replicas (add [(n3,s3):2] remove []): existing descriptor r14:/Table/1{8-9} [(n1,s1):1, (n3,s3):2LEARNER, next=3, gen=1]
I200207 16:33:57.353618 14 storage/replica_raft.go:248  [n1,s1,r14/1:/Table/1{8-9}] proposing SIMPLE(v2) ADD_REPLICA[(n3,s3):2]: after=[(n1,s1):1 (n3,s3):2] next=3
I200207 16:33:57.354168 14 storage/replica_command.go:1601  [n1,replicate,s1,r14/1:/Table/1{8-9}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r14:/Table/1{8-9} [(n1,s1):1, (n3,s3):2, next=3, gen=2]
I200207 16:33:57.365367 14 storage/replica_raft.go:248  [n1,s1,r14/1:/Table/1{8-9}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n2,s2):3LEARNER] next=4
I200207 16:33:57.369478 14 storage/store_snapshot.go:922  [n1,replicate,s1,r14/1:/Table/1{8-9}] sending LEARNER snapshot 2caf9908 at applied index 19
I200207 16:33:57.369662 14 storage/store_snapshot.go:965  [n1,replicate,s1,r14/1:/Table/1{8-9}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:57.369995 4256 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r14/1:/Table/1{8-9}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):3LEARNER
I200207 16:33:57.371078 4254 storage/replica_raftstorage.go:780  [n2,s2,r14/3:{-}] applying LEARNER snapshot [id=2caf9908 index=19]
I200207 16:33:57.374429 4254 storage/replica_raftstorage.go:801  [n2,s2,r14/3:/Table/1{8-9}] applied LEARNER snapshot [total=3ms ingestion=4@3ms id=2caf9908 index=19]
I200207 16:33:57.374852 14 storage/replica_command.go:1601  [n1,replicate,s1,r14/1:/Table/1{8-9}] change replicas (add [(n2,s2):3] remove []): existing descriptor r14:/Table/1{8-9} [(n1,s1):1, (n3,s3):2, (n2,s2):3LEARNER, next=4, gen=3]
I200207 16:33:57.394611 14 storage/replica_raft.go:248  [n1,s1,r14/1:/Table/1{8-9}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n3,s3):2 (n2,s2):3] next=4
I200207 16:33:57.401183 14 storage/replica_command.go:1601  [n1,replicate,s1,r15/1:/Table/{19-20}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r15:/Table/{19-20} [(n1,s1):1, next=2, gen=0]
I200207 16:33:57.421819 14 storage/replica_raft.go:248  [n1,s1,r15/1:/Table/{19-20}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200207 16:33:57.426305 14 storage/store_snapshot.go:922  [n1,replicate,s1,r15/1:/Table/{19-20}] sending LEARNER snapshot b8b6bd09 at applied index 15
I200207 16:33:57.426422 14 storage/store_snapshot.go:965  [n1,replicate,s1,r15/1:/Table/{19-20}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:57.426442 4360 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r15/1:/Table/{19-20}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200207 16:33:57.426970 4289 storage/replica_raftstorage.go:780  [n4,s4,r15/2:{-}] applying LEARNER snapshot [id=b8b6bd09 index=15]
I200207 16:33:57.435309 4289 storage/replica_raftstorage.go:801  [n4,s4,r15/2:/Table/{19-20}] applied LEARNER snapshot [total=8ms ingestion=4@8ms id=b8b6bd09 index=15]
I200207 16:33:57.450109 14 storage/replica_command.go:1601  [n1,replicate,s1,r15/1:/Table/{19-20}] change replicas (add [(n4,s4):2] remove []): existing descriptor r15:/Table/{19-20} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200207 16:33:57.466001 14 storage/replica_raft.go:248  [n1,s1,r15/1:/Table/{19-20}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200207 16:33:57.466540 14 storage/replica_command.go:1601  [n1,replicate,s1,r15/1:/Table/{19-20}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r15:/Table/{19-20} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200207 16:33:57.473015 4379 storage/replica_command.go:1601  [n2,replicate,s2,r3/3:/System/{NodeLive…-tsd}] change replicas (add [(n3,s3):4LEARNER] remove []): existing descriptor r3:/System/{NodeLivenessMax-tsd} [(n1,s1):1, (n4,s4):2, (n2,s2):3, next=4, gen=4]
I200207 16:33:57.496503 14 storage/replica_raft.go:248  [n1,s1,r15/1:/Table/{19-20}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3LEARNER] next=4
I200207 16:33:57.499368 14 storage/store_snapshot.go:922  [n1,replicate,s1,r15/1:/Table/{19-20}] sending LEARNER snapshot 6eda13c1 at applied index 19
I200207 16:33:57.499511 14 storage/store_snapshot.go:965  [n1,replicate,s1,r15/1:/Table/{19-20}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:57.499511 4447 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r15/1:/Table/{19-20}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):3LEARNER
I200207 16:33:57.500230 4461 storage/replica_raftstorage.go:780  [n2,s2,r15/3:{-}] applying LEARNER snapshot [id=6eda13c1 index=19]
I200207 16:33:57.521568 4461 storage/replica_raftstorage.go:801  [n2,s2,r15/3:/Table/{19-20}] applied LEARNER snapshot [total=21ms ingestion=4@14ms id=6eda13c1 index=19]
I200207 16:33:57.522302 14 storage/replica_command.go:1601  [n1,replicate,s1,r15/1:/Table/{19-20}] change replicas (add [(n2,s2):3] remove []): existing descriptor r15:/Table/{19-20} [(n1,s1):1, (n4,s4):2, (n2,s2):3LEARNER, next=4, gen=3]
I200207 16:33:57.543325 14 storage/replica_raft.go:248  [n1,s1,r15/1:/Table/{19-20}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3] next=4
I200207 16:33:57.544127 4379 storage/replica_raft.go:248  [n2,s2,r3/3:/System/{NodeLive…-tsd}] proposing SIMPLE(l4) ADD_REPLICA[(n3,s3):4LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3 (n3,s3):4LEARNER] next=5
I200207 16:33:57.546178 14 storage/replica_command.go:1601  [n1,replicate,s1,r10/1:/Table/1{4-5}] change replicas (add [(n3,s3):2LEARNER] remove []): existing descriptor r10:/Table/1{4-5} [(n1,s1):1, next=2, gen=0]
I200207 16:33:57.547378 4491 storage/raft_snapshot_queue.go:169  [n2,raftsnapshot,s2,r3/3:/System/{NodeLive…-tsd}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):4LEARNER
I200207 16:33:57.547760 4379 storage/store_snapshot.go:922  [n2,replicate,s2,r3/3:/System/{NodeLive…-tsd}] sending LEARNER snapshot 58c55003 at applied index 63
I200207 16:33:57.548916 4379 storage/store_snapshot.go:965  [n2,replicate,s2,r3/3:/System/{NodeLive…-tsd}] streamed snapshot to (n3,s3):4LEARNER: kv pairs: 42, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:57.550187 4548 storage/replica_raftstorage.go:780  [n3,s3,r3/4:{-}] applying LEARNER snapshot [id=58c55003 index=63]
I200207 16:33:57.553749 4548 storage/replica_raftstorage.go:801  [n3,s3,r3/4:/System/{NodeLive…-tsd}] applied LEARNER snapshot [total=4ms ingestion=4@3ms id=58c55003 index=63]
I200207 16:33:57.554934 4379 storage/replica_command.go:1601  [n2,replicate,s2,r3/3:/System/{NodeLive…-tsd}] change replicas (add [(n3,s3):4VOTER_INCOMING] remove [(n1,s1):1VOTER_DEMOTING]): existing descriptor r3:/System/{NodeLivenessMax-tsd} [(n1,s1):1, (n4,s4):2, (n2,s2):3, (n3,s3):4LEARNER, next=5, gen=5]
I200207 16:33:57.561561 14 storage/replica_raft.go:248  [n1,s1,r10/1:/Table/1{4-5}] proposing SIMPLE(l2) ADD_REPLICA[(n3,s3):2LEARNER]: after=[(n1,s1):1 (n3,s3):2LEARNER] next=3
I200207 16:33:57.563313 14 storage/store_snapshot.go:922  [n1,replicate,s1,r10/1:/Table/1{4-5}] sending LEARNER snapshot 97c61885 at applied index 15
I200207 16:33:57.563399 4569 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r10/1:/Table/1{4-5}] skipping snapshot; replica is likely a learner in the process of being added: (n3,s3):2LEARNER
I200207 16:33:57.563413 14 storage/store_snapshot.go:965  [n1,replicate,s1,r10/1:/Table/1{4-5}] streamed snapshot to (n3,s3):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:57.575214 4530 storage/replica_raftstorage.go:780  [n3,s3,r10/2:{-}] applying LEARNER snapshot [id=97c61885 index=15]
I200207 16:33:57.582195 4530 storage/replica_raftstorage.go:801  [n3,s3,r10/2:/Table/1{4-5}] applied LEARNER snapshot [total=7ms ingestion=4@5ms id=97c61885 index=15]
I200207 16:33:57.583454 14 storage/replica_command.go:1601  [n1,replicate,s1,r10/1:/Table/1{4-5}] change replicas (add [(n3,s3):2] remove []): existing descriptor r10:/Table/1{4-5} [(n1,s1):1, (n3,s3):2LEARNER, next=3, gen=1]
I200207 16:33:57.594978 4379 storage/replica_raft.go:248  [n2,s2,r3/3:/System/{NodeLive…-tsd}] proposing ENTER_JOINT(r1 l1 v4) ADD_REPLICA[(n3,s3):4VOTER_INCOMING], REMOVE_REPLICA[(n1,s1):1VOTER_DEMOTING]: after=[(n1,s1):1VOTER_DEMOTING (n4,s4):2 (n2,s2):3 (n3,s3):4VOTER_INCOMING] next=5
I200207 16:33:57.596809 4379 storage/replica_command.go:1601  [n2,replicate,s2,r3/3:/System/{NodeLive…-tsd}] change replicas (add [] remove []): existing descriptor r3:/System/{NodeLivenessMax-tsd} [(n1,s1):1VOTER_DEMOTING, (n4,s4):2, (n2,s2):3, (n3,s3):4VOTER_INCOMING, next=5, gen=6]
I200207 16:33:57.599311 4622 storage/replica_command.go:1601  [n3,replicate,s3,r9/2:/Table/1{3-4}] change replicas (add [(n2,s2):4LEARNER] remove []): existing descriptor r9:/Table/1{3-4} [(n1,s1):1, (n3,s3):2, (n4,s4):3, next=4, gen=4]
I200207 16:33:57.599443 14 storage/replica_raft.go:248  [n1,s1,r10/1:/Table/1{4-5}] proposing SIMPLE(v2) ADD_REPLICA[(n3,s3):2]: after=[(n1,s1):1 (n3,s3):2] next=3
I200207 16:33:57.614242 14 storage/replica_command.go:1601  [n1,replicate,s1,r10/1:/Table/1{4-5}] change replicas (add [(n4,s4):3LEARNER] remove []): existing descriptor r10:/Table/1{4-5} [(n1,s1):1, (n3,s3):2, next=3, gen=2]
I200207 16:33:57.621615 4379 storage/replica_raft.go:248  [n2,s2,r3/3:/System/{NodeLive…-tsd}] proposing LEAVE_JOINT: after=[(n1,s1):1LEARNER (n4,s4):2 (n2,s2):3 (n3,s3):4] next=5
I200207 16:33:57.623328 4379 storage/replica_command.go:1601  [n2,replicate,s2,r3/3:/System/{NodeLive…-tsd}] change replicas (add [] remove [(n1,s1):1LEARNER]): existing descriptor r3:/System/{NodeLivenessMax-tsd} [(n1,s1):1LEARNER, (n4,s4):2, (n2,s2):3, (n3,s3):4, next=5, gen=7]
I200207 16:33:57.638548 14 storage/replica_raft.go:248  [n1,s1,r10/1:/Table/1{4-5}] proposing SIMPLE(l3) ADD_REPLICA[(n4,s4):3LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3LEARNER] next=4
I200207 16:33:57.662416 4697 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r10/1:/Table/1{4-5}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):3LEARNER
I200207 16:33:57.671472 4379 storage/replica_raft.go:248  [n2,s2,r3/3:/System/{NodeLive…-tsd}] proposing SIMPLE(r1) REMOVE_REPLICA[(n1,s1):1LEARNER]: after=[(n3,s3):4 (n4,s4):2 (n2,s2):3] next=5
I200207 16:33:57.673679 133 storage/store_remove_replica.go:122  [n1,s1,r3/1:/System/{NodeLive…-tsd}] removing replica r3/1
I200207 16:33:57.676164 14 storage/store_snapshot.go:922  [n1,replicate,s1,r10/1:/Table/1{4-5}] sending LEARNER snapshot 8b32d585 at applied index 20
I200207 16:33:57.676302 14 storage/store_snapshot.go:965  [n1,replicate,s1,r10/1:/Table/1{4-5}] streamed snapshot to (n4,s4):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.02s
I200207 16:33:57.682129 4667 storage/replica_raftstorage.go:780  [n4,s4,r10/3:{-}] applying LEARNER snapshot [id=8b32d585 index=20]
I200207 16:33:57.686548 4667 storage/replica_raftstorage.go:801  [n4,s4,r10/3:/Table/1{4-5}] applied LEARNER snapshot [total=4ms ingestion=4@3ms id=8b32d585 index=20]
I200207 16:33:57.687066 14 storage/replica_command.go:1601  [n1,replicate,s1,r10/1:/Table/1{4-5}] change replicas (add [(n4,s4):3] remove []): existing descriptor r10:/Table/1{4-5} [(n1,s1):1, (n3,s3):2, (n4,s4):3LEARNER, next=4, gen=3]
I200207 16:33:57.709955 4622 storage/replica_raft.go:248  [n3,s3,r9/2:/Table/1{3-4}] proposing SIMPLE(l4) ADD_REPLICA[(n2,s2):4LEARNER]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3 (n2,s2):4LEARNER] next=5
I200207 16:33:57.710872 14 storage/replica_raft.go:248  [n1,s1,r10/1:/Table/1{4-5}] proposing SIMPLE(v3) ADD_REPLICA[(n4,s4):3]: after=[(n1,s1):1 (n3,s3):2 (n4,s4):3] next=4
I200207 16:33:57.712004 4622 storage/store_snapshot.go:922  [n3,replicate,s3,r9/2:/Table/1{3-4}] sending LEARNER snapshot d6ede117 at applied index 269
I200207 16:33:57.712411 4775 storage/raft_snapshot_queue.go:169  [n3,raftsnapshot,s3,r9/2:/Table/1{3-4}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):4LEARNER
I200207 16:33:57.713417 4622 storage/store_snapshot.go:965  [n3,replicate,s3,r9/2:/Table/1{3-4}] streamed snapshot to (n2,s2):4LEARNER: kv pairs: 621, log entries: 0, rate-limit: 8.0 MiB/sec, 0.00s
I200207 16:33:57.713996 14 storage/replica_command.go:1601  [n1,replicate,s1,r5/1:/{Systemtse-Table/System…}] change replicas (add [(n4,s4):2LEARNER] remove []): existing descriptor r5:/{Systemtse-Table/SystemConfigSpan/Start} [(n1,s1):1, next=2, gen=0]
I200207 16:33:57.716749 4773 storage/replica_raftstorage.go:780  [n2,s2,r9/4:{-}] applying LEARNER snapshot [id=d6ede117 index=269]
I200207 16:33:57.744118 4773 storage/replica_raftstorage.go:801  [n2,s2,r9/4:/Table/1{3-4}] applied LEARNER snapshot [total=27ms ingestion=4@10ms id=d6ede117 index=269]
I200207 16:33:57.744905 14 storage/replica_raft.go:248  [n1,s1,r5/1:/{Systemtse-Table/System…}] proposing SIMPLE(l2) ADD_REPLICA[(n4,s4):2LEARNER]: after=[(n1,s1):1 (n4,s4):2LEARNER] next=3
I200207 16:33:57.745308 4622 storage/replica_command.go:1601  [n3,replicate,s3,r9/2:/Table/1{3-4}] change replicas (add [(n2,s2):4VOTER_INCOMING] remove [(n1,s1):1VOTER_DEMOTING]): existing descriptor r9:/Table/1{3-4} [(n1,s1):1, (n3,s3):2, (n4,s4):3, (n2,s2):4LEARNER, next=5, gen=5]
I200207 16:33:57.764479 14 storage/store_snapshot.go:922  [n1,replicate,s1,r5/1:/{Systemtse-Table/System…}] sending LEARNER snapshot a1ac971d at applied index 15
I200207 16:33:57.765401 14 storage/store_snapshot.go:965  [n1,replicate,s1,r5/1:/{Systemtse-Table/System…}] streamed snapshot to (n4,s4):2LEARNER: kv pairs: 6, log entries: 0, rate-limit: 8.0 MiB/sec, 0.01s
I200207 16:33:57.764996 4691 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r5/1:/{Systemtse-Table/System…}] skipping snapshot; replica is likely a learner in the process of being added: (n4,s4):2LEARNER
I200207 16:33:57.770823 4689 storage/replica_raftstorage.go:780  [n4,s4,r5/2:{-}] applying LEARNER snapshot [id=a1ac971d index=15]
I200207 16:33:57.801585 4689 storage/replica_raftstorage.go:801  [n4,s4,r5/2:/{Systemtse-Table/System…}] applied LEARNER snapshot [total=31ms ingestion=4@24ms id=a1ac971d index=15]
I200207 16:33:57.803793 14 storage/replica_command.go:1601  [n1,replicate,s1,r5/1:/{Systemtse-Table/System…}] change replicas (add [(n4,s4):2] remove []): existing descriptor r5:/{Systemtse-Table/SystemConfigSpan/Start} [(n1,s1):1, (n4,s4):2LEARNER, next=3, gen=1]
I200207 16:33:57.814848 4622 storage/replica_raft.go:248  [n3,s3,r9/2:/Table/1{3-4}] proposing ENTER_JOINT(r1 l1 v4) ADD_REPLICA[(n2,s2):4VOTER_INCOMING], REMOVE_REPLICA[(n1,s1):1VOTER_DEMOTING]: after=[(n1,s1):1VOTER_DEMOTING (n3,s3):2 (n4,s4):3 (n2,s2):4VOTER_INCOMING] next=5
I200207 16:33:57.817115 4622 storage/replica_command.go:1601  [n3,replicate,s3,r9/2:/Table/1{3-4}] change replicas (add [] remove []): existing descriptor r9:/Table/1{3-4} [(n1,s1):1VOTER_DEMOTING, (n3,s3):2, (n4,s4):3, (n2,s2):4VOTER_INCOMING, next=5, gen=6]
I200207 16:33:57.828364 4622 storage/replica_raft.go:248  [n3,s3,r9/2:/Table/1{3-4}] proposing LEAVE_JOINT: after=[(n1,s1):1LEARNER (n3,s3):2 (n4,s4):3 (n2,s2):4] next=5
I200207 16:33:57.829333 4622 storage/replica_command.go:1601  [n3,replicate,s3,r9/2:/Table/1{3-4}] change replicas (add [] remove [(n1,s1):1LEARNER]): existing descriptor r9:/Table/1{3-4} [(n1,s1):1LEARNER, (n3,s3):2, (n4,s4):3, (n2,s2):4, next=5, gen=7]
I200207 16:33:57.836557 14 storage/replica_raft.go:248  [n1,s1,r5/1:/{Systemtse-Table/System…}] proposing SIMPLE(v2) ADD_REPLICA[(n4,s4):2]: after=[(n1,s1):1 (n4,s4):2] next=3
I200207 16:33:57.837098 14 storage/replica_command.go:1601  [n1,replicate,s1,r5/1:/{Systemtse-Table/System…}] change replicas (add [(n2,s2):3LEARNER] remove []): existing descriptor r5:/{Systemtse-Table/SystemConfigSpan/Start} [(n1,s1):1, (n4,s4):2, next=3, gen=2]
I200207 16:33:57.861190 4622 storage/replica_raft.go:248  [n3,s3,r9/2:/Table/1{3-4}] proposing SIMPLE(r1) REMOVE_REPLICA[(n1,s1):1LEARNER]: after=[(n2,s2):4 (n3,s3):2 (n4,s4):3] next=5
I200207 16:33:57.862629 14 storage/replica_raft.go:248  [n1,s1,r5/1:/{Systemtse-Table/System…}] proposing SIMPLE(l3) ADD_REPLICA[(n2,s2):3LEARNER]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3LEARNER] next=4
I200207 16:33:57.867976 126 storage/store_remove_replica.go:122  [n1,s1,r9/1:/Table/1{3-4}] removing replica r9/1
I200207 16:33:57.869044 14 storage/store_snapshot.go:922  [n1,replicate,s1,r5/1:/{Systemtse-Table/System…}] sending LEARNER snapshot d55c365f at applied index 19
I200207 16:33:57.895161 14 storage/store_snapshot.go:965  [n1,replicate,s1,r5/1:/{Systemtse-Table/System…}] streamed snapshot to (n2,s2):3LEARNER: kv pairs: 10, log entries: 0, rate-limit: 8.0 MiB/sec, 0.03s
I200207 16:33:57.869283 4877 storage/raft_snapshot_queue.go:169  [n1,raftsnapshot,s1,r5/1:/{Systemtse-Table/System…}] skipping snapshot; replica is likely a learner in the process of being added: (n2,s2):3LEARNER
I200207 16:33:57.895994 4875 storage/replica_raftstorage.go:780  [n2,s2,r5/3:{-}] applying LEARNER snapshot [id=d55c365f index=19]
I200207 16:33:57.900244 4875 storage/replica_raftstorage.go:801  [n2,s2,r5/3:/{Systemtse-Table/System…}] applied LEARNER snapshot [total=4ms ingestion=4@4ms id=d55c365f index=19]
I200207 16:33:57.901770 14 storage/replica_command.go:1601  [n1,replicate,s1,r5/1:/{Systemtse-Table/System…}] change replicas (add [(n2,s2):3] remove []): existing descriptor r5:/{Systemtse-Table/SystemConfigSpan/Start} [(n1,s1):1, (n4,s4):2, (n2,s2):3LEARNER, next=4, gen=3]
I200207 16:33:57.914655 14 storage/replica_raft.go:248  [n1,s1,r5/1:/{Systemtse-Table/System…}] proposing SIMPLE(v3) ADD_REPLICA[(n2,s2):3]: after=[(n1,s1):1 (n4,s4):2 (n2,s2):3] next=4
I200207 16:33:57.953469 14 testutils/testcluster/testcluster.go:760  WaitForFullReplication took: 3.088431654s
I200207 16:33:57.990556 4931 sql/event_log.go:132  [n1,client=127.0.0.1:36458,user=root] Event: "set_cluster_setting", target: 0, info: {SettingName:kv.rangefeed.enabled Value:true User:root}
I200207 16:33:57.992895 14 kv/kvnemesis/watcher.go:131  watcher waiting for 1581093237.992869126,0
I200207 16:33:57.993075 5174 kv/dist_sender_rangefeed.go:151  [n1] RangeFeed dial /Table/5{0-1} 1581093237.992869126,0
I200207 16:33:57.993502 5176 storage/rangefeed/registry.go:451  register /Table/5{0-1} 1581093237.992869126,0
I200207 16:33:57.993688 5139 kv/kvnemesis/watcher.go:197  watcher reached frontier 1581093235.603544763,0 lagging by 2.390122653s
I200207 16:33:57.997996 169 storage/replica_rangefeed.go:505  [n1,s1,r8/1:/Table/1{2-3}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:57.998054 169 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.973526Z/"\xda\xf1\xa0\xcb\xeb\x85KY\xb5fɮ\x9f_ -"/0:1581093237.973526183,0 -> / 50fbf56f-91a5-415c-9222-2aec2765ff23
I200207 16:33:57.998097 169 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.973526Z/"\xda\xf1\xa0\xcb\xeb\x85KY\xb5fɮ\x9f_ -"/2/1:1581093237.973526183,0 -> / 50fbf56f-91a5-415c-9222-2aec2765ff23
I200207 16:33:57.998134 169 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.973526Z/"\xda\xf1\xa0\xcb\xeb\x85KY\xb5fɮ\x9f_ -"/3/1:1581093237.973526183,0 -> / 50fbf56f-91a5-415c-9222-2aec2765ff23
I200207 16:33:57.998169 169 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.973526Z/"\xda\xf1\xa0\xcb\xeb\x85KY\xb5fɮ\x9f_ -"/4/1:1581093237.973526183,0 -> / 50fbf56f-91a5-415c-9222-2aec2765ff23
I200207 16:33:57.998213 169 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.973526Z/"\xda\xf1\xa0\xcb\xeb\x85KY\xb5fɮ\x9f_ -"/5/1:1581093237.973526183,0 -> / 50fbf56f-91a5-415c-9222-2aec2765ff23
I200207 16:33:57.998691 1394 storage/replica_rangefeed.go:505  [n3,s3,r8/3:/Table/1{2-3}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:57.998735 1103 storage/replica_rangefeed.go:505  [n2,s2,r8/2:/Table/1{2-3}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:57.998784 1103 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.973526Z/"\xda\xf1\xa0\xcb\xeb\x85KY\xb5fɮ\x9f_ -"/0:1581093237.973526183,0 -> / 50fbf56f-91a5-415c-9222-2aec2765ff23
I200207 16:33:57.998815 1103 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.973526Z/"\xda\xf1\xa0\xcb\xeb\x85KY\xb5fɮ\x9f_ -"/2/1:1581093237.973526183,0 -> / 50fbf56f-91a5-415c-9222-2aec2765ff23
I200207 16:33:57.998838 1103 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.973526Z/"\xda\xf1\xa0\xcb\xeb\x85KY\xb5fɮ\x9f_ -"/3/1:1581093237.973526183,0 -> / 50fbf56f-91a5-415c-9222-2aec2765ff23
I200207 16:33:57.998862 1103 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.973526Z/"\xda\xf1\xa0\xcb\xeb\x85KY\xb5fɮ\x9f_ -"/4/1:1581093237.973526183,0 -> / 50fbf56f-91a5-415c-9222-2aec2765ff23
I200207 16:33:57.998898 1103 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.973526Z/"\xda\xf1\xa0\xcb\xeb\x85KY\xb5fɮ\x9f_ -"/5/1:1581093237.973526183,0 -> / 50fbf56f-91a5-415c-9222-2aec2765ff23
I200207 16:33:57.998740 1394 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.973526Z/"\xda\xf1\xa0\xcb\xeb\x85KY\xb5fɮ\x9f_ -"/0:1581093237.973526183,0 -> / 50fbf56f-91a5-415c-9222-2aec2765ff23
I200207 16:33:57.998987 1394 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.973526Z/"\xda\xf1\xa0\xcb\xeb\x85KY\xb5fɮ\x9f_ -"/2/1:1581093237.973526183,0 -> / 50fbf56f-91a5-415c-9222-2aec2765ff23
I200207 16:33:57.999018 1394 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.973526Z/"\xda\xf1\xa0\xcb\xeb\x85KY\xb5fɮ\x9f_ -"/3/1:1581093237.973526183,0 -> / 50fbf56f-91a5-415c-9222-2aec2765ff23
I200207 16:33:57.999041 1394 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.973526Z/"\xda\xf1\xa0\xcb\xeb\x85KY\xb5fɮ\x9f_ -"/4/1:1581093237.973526183,0 -> / 50fbf56f-91a5-415c-9222-2aec2765ff23
I200207 16:33:57.999069 1394 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.973526Z/"\xda\xf1\xa0\xcb\xeb\x85KY\xb5fɮ\x9f_ -"/5/1:1581093237.973526183,0 -> / 50fbf56f-91a5-415c-9222-2aec2765ff23
I200207 16:33:58.016136 1889 storage/replica_rangefeed.go:505  [n4,s4,r6/2:/Table/{SystemCon…-11}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.016188 1889 storage/replica_rangefeed.go:506  [n4,s4,r6/2:/Table/{SystemCon…-11}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093237.994276700,0 1581093237.994276700,0 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.016506 164 storage/replica_rangefeed.go:505  [n1,s1,r6/1:/Table/{SystemCon…-11}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.016542 164 storage/replica_rangefeed.go:506  [n1,s1,r6/1:/Table/{SystemCon…-11}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093237.994276700,0 1581093237.994276700,0 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.016711 1398 storage/replica_rangefeed.go:505  [n3,s3,r6/3:/Table/{SystemCon…-11}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.016752 1398 storage/replica_rangefeed.go:506  [n3,s3,r6/3:/Table/{SystemCon…-11}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093237.994276700,0 1581093237.994276700,0 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.021499 115 storage/replica_rangefeed.go:505  [n1,s1,r8/1:/Table/1{2-3}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.021574 115 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093237.994276700,0 1581093237.994276700,0 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.021641 115 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093237.994276700,0 1581093237.994276700,0 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.021710 115 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093237.994276700,0 1581093237.994276700,0 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.021778 115 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093237.994276700,0 1581093237.994276700,0 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.021842 115 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093237.994276700,0 1581093237.994276700,0 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.022375 1399 storage/replica_rangefeed.go:505  [n3,s3,r8/3:/Table/1{2-3}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.022452 1399 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093237.994276700,0 1581093237.994276700,0 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.022520 1399 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093237.994276700,0 1581093237.994276700,0 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.022587 1399 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093237.994276700,0 1581093237.994276700,0 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.022626 1399 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093237.994276700,0 1581093237.994276700,0 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.022700 1399 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093237.994276700,0 1581093237.994276700,0 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.022955 1113 storage/replica_rangefeed.go:505  [n2,s2,r8/2:/Table/1{2-3}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.023024 1113 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093237.994276700,0 1581093237.994276700,0 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.023092 1113 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093237.994276700,0 1581093237.994276700,0 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.023791 1113 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093237.994276700,0 1581093237.994276700,0 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.023858 1113 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093237.994276700,0 1581093237.994276700,0 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.023896 1113 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093237.994276700,0 1581093237.994276700,0 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.025367 1913 storage/replica_rangefeed.go:505  [n4,s4,r6/2:/Table/{SystemCon…-11}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.025445 1913 storage/replica_rangefeed.go:506  [n4,s4,r6/2:/Table/{SystemCon…-11}] WIP MVCCCommitIntentOp /Table/6/1/"kv.closed_timestamp.target_duration"/0:1581093237.994276700,0 -> / 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.026335 1374 storage/replica_rangefeed.go:505  [n3,s3,r6/3:/Table/{SystemCon…-11}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.026379 1374 storage/replica_rangefeed.go:506  [n3,s3,r6/3:/Table/{SystemCon…-11}] WIP MVCCCommitIntentOp /Table/6/1/"kv.closed_timestamp.target_duration"/0:1581093237.994276700,0 -> / 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.026586 118 storage/replica_rangefeed.go:505  [n1,s1,r6/1:/Table/{SystemCon…-11}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.026628 118 storage/replica_rangefeed.go:506  [n1,s1,r6/1:/Table/{SystemCon…-11}] WIP MVCCCommitIntentOp /Table/6/1/"kv.closed_timestamp.target_duration"/0:1581093237.994276700,0 -> / 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.029162 182 storage/replica_rangefeed.go:615  [n1,s1,r28/1:/{Table/32-Max}] RangeFeed closed timestamp 1581093235.603544763,0 is behind by 2.425554751s
I200207 16:33:58.035167 5139 kv/kvnemesis/watcher.go:197  watcher reached frontier 1581093238.028817739,0 lagging by 6.332513ms
I200207 16:33:58.035262 77 storage/replica_rangefeed.go:505  [n1,s1,r8/1:/Table/1{2-3}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.035305 77 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.994277Z/"J\x1a\x9f6n\"@\xbd\x8cIe\xdd8\u007f\xb7w"/0:1581093237.994276700,0 -> / 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.035332 77 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.994277Z/"J\x1a\x9f6n\"@\xbd\x8cIe\xdd8\u007f\xb7w"/2/1:1581093237.994276700,0 -> / 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.035358 77 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.994277Z/"J\x1a\x9f6n\"@\xbd\x8cIe\xdd8\u007f\xb7w"/3/1:1581093237.994276700,0 -> / 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.035382 77 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.994277Z/"J\x1a\x9f6n\"@\xbd\x8cIe\xdd8\u007f\xb7w"/4/1:1581093237.994276700,0 -> / 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.035405 77 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.994277Z/"J\x1a\x9f6n\"@\xbd\x8cIe\xdd8\u007f\xb7w"/5/1:1581093237.994276700,0 -> / 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.035476 1114 storage/replica_rangefeed.go:505  [n2,s2,r8/2:/Table/1{2-3}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.035507 1114 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.994277Z/"J\x1a\x9f6n\"@\xbd\x8cIe\xdd8\u007f\xb7w"/0:1581093237.994276700,0 -> / 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.035548 1114 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.994277Z/"J\x1a\x9f6n\"@\xbd\x8cIe\xdd8\u007f\xb7w"/2/1:1581093237.994276700,0 -> / 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.035572 1114 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.994277Z/"J\x1a\x9f6n\"@\xbd\x8cIe\xdd8\u007f\xb7w"/3/1:1581093237.994276700,0 -> / 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.035594 1114 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.994277Z/"J\x1a\x9f6n\"@\xbd\x8cIe\xdd8\u007f\xb7w"/4/1:1581093237.994276700,0 -> / 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.035616 1114 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.994277Z/"J\x1a\x9f6n\"@\xbd\x8cIe\xdd8\u007f\xb7w"/5/1:1581093237.994276700,0 -> / 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.035642 1403 storage/replica_rangefeed.go:505  [n3,s3,r8/3:/Table/1{2-3}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.035680 1403 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.994277Z/"J\x1a\x9f6n\"@\xbd\x8cIe\xdd8\u007f\xb7w"/0:1581093237.994276700,0 -> / 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.035710 1403 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.994277Z/"J\x1a\x9f6n\"@\xbd\x8cIe\xdd8\u007f\xb7w"/2/1:1581093237.994276700,0 -> / 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.035736 1403 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.994277Z/"J\x1a\x9f6n\"@\xbd\x8cIe\xdd8\u007f\xb7w"/3/1:1581093237.994276700,0 -> / 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.035760 1403 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.994277Z/"J\x1a\x9f6n\"@\xbd\x8cIe\xdd8\u007f\xb7w"/4/1:1581093237.994276700,0 -> / 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.035791 1403 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:57.994277Z/"J\x1a\x9f6n\"@\xbd\x8cIe\xdd8\u007f\xb7w"/5/1:1581093237.994276700,0 -> / 6cadeff7-19f2-4655-8bfd-975e6445bc08
I200207 16:33:58.039295 4931 sql/event_log.go:132  [n1,client=127.0.0.1:36458,user=root] Event: "set_cluster_setting", target: 0, info: {SettingName:kv.closed_timestamp.target_duration Value:00:00:00.001 User:root}
I200207 16:33:58.041361 5139 kv/kvnemesis/watcher.go:197  watcher reached frontier 1581093238.029032282,0 lagging by 12.312611ms
I200207 16:33:58.041403 5139 kv/kvnemesis/watcher.go:203  watcher notifying 1581093237.992869126,0
I200207 16:33:58.042326 5139 kv/kvnemesis/watcher.go:197  watcher reached frontier 1581093238.035291064,0 lagging by 7.021389ms
I200207 16:33:58.054598 5139 kv/kvnemesis/watcher.go:197  watcher reached frontier 1581093238.046912227,0 lagging by 7.667721ms
I200207 16:33:58.056513 5139 kv/kvnemesis/watcher.go:197  watcher reached frontier 1581093238.047266091,0 lagging by 9.231618ms
I200207 16:33:58.056514 1928 storage/replica_rangefeed.go:505  [n4,s4,r6/2:/Table/{SystemCon…-11}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.056559 1928 storage/replica_rangefeed.go:506  [n4,s4,r6/2:/Table/{SystemCon…-11}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.045860135,2 1581093238.041716484,0 ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.056788 126 storage/replica_rangefeed.go:505  [n1,s1,r6/1:/Table/{SystemCon…-11}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.056822 126 storage/replica_rangefeed.go:506  [n1,s1,r6/1:/Table/{SystemCon…-11}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.045860135,2 1581093238.041716484,0 ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.057001 1392 storage/replica_rangefeed.go:505  [n3,s3,r6/3:/Table/{SystemCon…-11}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.057030 1392 storage/replica_rangefeed.go:506  [n3,s3,r6/3:/Table/{SystemCon…-11}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.045860135,2 1581093238.041716484,0 ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.064187 136 storage/replica_rangefeed.go:505  [n1,s1,r8/1:/Table/1{2-3}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.064239 136 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.059948926,2 1581093238.041716484,0 ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.064264 136 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.059948926,2 1581093238.041716484,0 ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.064290 136 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.059948926,2 1581093238.041716484,0 ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.064316 136 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.059948926,2 1581093238.041716484,0 ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.064343 136 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.059948926,2 1581093238.041716484,0 ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.064426 1412 storage/replica_rangefeed.go:505  [n3,s3,r8/3:/Table/1{2-3}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.064468 1412 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.059948926,2 1581093238.041716484,0 ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.064498 1412 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.059948926,2 1581093238.041716484,0 ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.064525 1412 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.059948926,2 1581093238.041716484,0 ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.064549 1412 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.059948926,2 1581093238.041716484,0 ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.064572 1412 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.059948926,2 1581093238.041716484,0 ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.064915 1117 storage/replica_rangefeed.go:505  [n2,s2,r8/2:/Table/1{2-3}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.064955 1117 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.059948926,2 1581093238.041716484,0 ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.064978 1117 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.059948926,2 1581093238.041716484,0 ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.065005 1117 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.059948926,2 1581093238.041716484,0 ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.065032 1117 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.059948926,2 1581093238.041716484,0 ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.065053 1117 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.059948926,2 1581093238.041716484,0 ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.067102 1922 storage/replica_rangefeed.go:505  [n4,s4,r6/2:/Table/{SystemCon…-11}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.067140 1922 storage/replica_rangefeed.go:506  [n4,s4,r6/2:/Table/{SystemCon…-11}] WIP MVCCCommitIntentOp /Table/6/1/"kv.closed_timestamp.target_duration"/0:1581093238.059948926,2 -> / ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.067614 4931 sql/event_log.go:132  [n1,client=127.0.0.1:36458,user=root] Event: "set_cluster_setting", target: 0, info: {SettingName:kv.closed_timestamp.target_duration Value:DEFAULT User:root}
I200207 16:33:58.067862 1411 storage/replica_rangefeed.go:505  [n3,s3,r6/3:/Table/{SystemCon…-11}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.067894 1411 storage/replica_rangefeed.go:506  [n3,s3,r6/3:/Table/{SystemCon…-11}] WIP MVCCCommitIntentOp /Table/6/1/"kv.closed_timestamp.target_duration"/0:1581093238.059948926,2 -> / ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.069874 5139 kv/kvnemesis/watcher.go:197  watcher reached frontier 1581093238.059657892,0 lagging by 10.200001ms
I200207 16:33:58.070312 5417 kv/kvnemesis/applier.go:88  Applier txn de3e47be-fcf1-4c54-8883-05bd15f696f7
I200207 16:33:58.070585 5413 kv/kvnemesis/applier.go:88  Applier txn 44492280-b337-410c-91c0-f4ccce67af71
I200207 16:33:58.071175 5416 kv/kvnemesis/applier.go:88  Applier txn c9b135d3-a9b6-48cb-b04a-7d5e03c7a0de
I200207 16:33:58.071232 5416 kv/kvnemesis/kvnemesis.go:63
  3 OP  db0.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  3 OP    return nil
  3 OP  }) // nil
I200207 16:33:58.071732 1063 storage/replica_rangefeed.go:505  [n2,s2,r28/2:/{Table/32-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.071763 1063 storage/replica_rangefeed.go:506  [n2,s2,r28/2:/{Table/32-Max}] WIP MVCCWriteIntentOp /Table/50/"25567fd6" 1581093238.070282014,0 1581093238.070282014,0 de3e47be-fcf1-4c54-8883-05bd15f696f7
I200207 16:33:58.071982 1413 storage/replica_rangefeed.go:505  [n3,s3,r28/3:/{Table/32-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.072011 1413 storage/replica_rangefeed.go:506  [n3,s3,r28/3:/{Table/32-Max}] WIP MVCCWriteIntentOp /Table/50/"25567fd6" 1581093238.070282014,0 1581093238.070282014,0 de3e47be-fcf1-4c54-8883-05bd15f696f7
I200207 16:33:58.072288 131 storage/rangefeed/processor.go:443  WIP MVCCWriteIntentOp /Table/50/"25567fd6" 1581093238.070282014,0 1581093238.070282014,0 de3e47be-fcf1-4c54-8883-05bd15f696f7
I200207 16:33:58.072519 1065 storage/replica_rangefeed.go:505  [n2,s2,r3/3:/System/{NodeLive…-tsd}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.072543 1065 storage/replica_rangefeed.go:506  [n2,s2,r3/3:/System/{NodeLive…-tsd}] WIP MVCCWriteValueOp  /System/"range-idgen":1581093238.071802321,0 -> /
I200207 16:33:58.072669 5622 storage/replica_command.go:396  [n2,s2,r28/2:/{Table/32-Max}] initiating a split of this range at key /Table/50/"06af801b" [r29] (manual)
I200207 16:33:58.072903 132 storage/replica_rangefeed.go:505  [n1,s1,r6/1:/Table/{SystemCon…-11}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.072935 132 storage/replica_rangefeed.go:506  [n1,s1,r6/1:/Table/{SystemCon…-11}] WIP MVCCCommitIntentOp /Table/6/1/"kv.closed_timestamp.target_duration"/0:1581093238.059948926,2 -> / ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.072989 1836 storage/replica_rangefeed.go:505  [n4,s4,r3/2:/System/{NodeLive…-tsd}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.073015 1836 storage/replica_rangefeed.go:506  [n4,s4,r3/2:/System/{NodeLive…-tsd}] WIP MVCCWriteValueOp  /System/"range-idgen":1581093238.071802321,0 -> /
I200207 16:33:58.072989 1389 storage/replica_rangefeed.go:505  [n3,s3,r3/4:/System/{NodeLive…-tsd}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.073092 1389 storage/replica_rangefeed.go:506  [n3,s3,r3/4:/System/{NodeLive…-tsd}] WIP MVCCWriteValueOp  /System/"range-idgen":1581093238.071802321,0 -> /
I200207 16:33:58.074442 143 storage/replica_rangefeed.go:505  [n1,s1,r8/1:/Table/1{2-3}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.074496 143 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.041716Z/"\xad\xd8ih\xea#O\xa0\xb1GZ0\x13\x06\xd9\xce"/0:1581093238.059948926,2 -> / ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.074529 143 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.041716Z/"\xad\xd8ih\xea#O\xa0\xb1GZ0\x13\x06\xd9\xce"/2/1:1581093238.059948926,2 -> / ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.074563 143 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.041716Z/"\xad\xd8ih\xea#O\xa0\xb1GZ0\x13\x06\xd9\xce"/3/1:1581093238.059948926,2 -> / ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.074594 143 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.041716Z/"\xad\xd8ih\xea#O\xa0\xb1GZ0\x13\x06\xd9\xce"/4/1:1581093238.059948926,2 -> / ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.074618 143 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.041716Z/"\xad\xd8ih\xea#O\xa0\xb1GZ0\x13\x06\xd9\xce"/5/1:1581093238.059948926,2 -> / ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.074822 1069 storage/replica_rangefeed.go:505  [n2,s2,r8/2:/Table/1{2-3}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.074892 1069 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.041716Z/"\xad\xd8ih\xea#O\xa0\xb1GZ0\x13\x06\xd9\xce"/0:1581093238.059948926,2 -> / ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.074964 1069 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.041716Z/"\xad\xd8ih\xea#O\xa0\xb1GZ0\x13\x06\xd9\xce"/2/1:1581093238.059948926,2 -> / ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.075016 1312 storage/replica_rangefeed.go:505  [n3,s3,r8/3:/Table/1{2-3}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.075055 1312 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.041716Z/"\xad\xd8ih\xea#O\xa0\xb1GZ0\x13\x06\xd9\xce"/0:1581093238.059948926,2 -> / ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.075076 1069 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.041716Z/"\xad\xd8ih\xea#O\xa0\xb1GZ0\x13\x06\xd9\xce"/3/1:1581093238.059948926,2 -> / ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.075156 1069 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.041716Z/"\xad\xd8ih\xea#O\xa0\xb1GZ0\x13\x06\xd9\xce"/4/1:1581093238.059948926,2 -> / ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.075230 1069 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.041716Z/"\xad\xd8ih\xea#O\xa0\xb1GZ0\x13\x06\xd9\xce"/5/1:1581093238.059948926,2 -> / ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.075080 1312 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.041716Z/"\xad\xd8ih\xea#O\xa0\xb1GZ0\x13\x06\xd9\xce"/2/1:1581093238.059948926,2 -> / ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.075422 1312 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.041716Z/"\xad\xd8ih\xea#O\xa0\xb1GZ0\x13\x06\xd9\xce"/3/1:1581093238.059948926,2 -> / ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.075502 1312 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.041716Z/"\xad\xd8ih\xea#O\xa0\xb1GZ0\x13\x06\xd9\xce"/4/1:1581093238.059948926,2 -> / ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.075592 1312 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.041716Z/"\xad\xd8ih\xea#O\xa0\xb1GZ0\x13\x06\xd9\xce"/5/1:1581093238.059948926,2 -> / ddb21bbd-265d-4bbf-89a8-d66bc5528a1c
I200207 16:33:58.082056 1073 storage/replica_rangefeed.go:505  [n2,s2,r28/2:/{Table/32-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.082108 1073 storage/replica_rangefeed.go:506  [n2,s2,r28/2:/{Table/32-Max}] WIP MVCCCommitIntentOp /Table/50/"25567fd6":1581093238.070282014,0 -> / de3e47be-fcf1-4c54-8883-05bd15f696f7
I200207 16:33:58.082540 1417 storage/replica_rangefeed.go:505  [n3,s3,r28/3:/{Table/32-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.082579 1417 storage/replica_rangefeed.go:506  [n3,s3,r28/3:/{Table/32-Max}] WIP MVCCCommitIntentOp /Table/50/"25567fd6":1581093238.070282014,0 -> / de3e47be-fcf1-4c54-8883-05bd15f696f7
I200207 16:33:58.082845 133 storage/replica_rangefeed.go:564  WIP handleLogicalOpLogRaftMuLocked /Table/50/"25567fd6":1581093238.070282014,0 -> /BYTES/v-0 
I200207 16:33:58.082881 133 storage/rangefeed/processor.go:443  WIP MVCCCommitIntentOp /Table/50/"25567fd6":1581093238.070282014,0 -> /BYTES/v-0 de3e47be-fcf1-4c54-8883-05bd15f696f7
I200207 16:33:58.082976 5176 storage/rangefeed/processor.go:603  [n1,s1,r28/1:/{Table/32-Max},rangefeed] publishValue /Table/50/"25567fd6":1581093238.070282014,0 -> /BYTES/v-0
I200207 16:33:58.083038 5139 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"25567fd6" 1581093238.070282014,0 -> /BYTES/v-0 (prev /)
I200207 16:33:58.083211 5417 kv/kvnemesis/kvnemesis.go:63
  4 OP  db0.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  4 OP    txn.Put(ctx, /Table/50/"25567fd6", v-0) // nil
  4 OP    txn.Get(ctx, /Table/50/"dd91893a") // (nil, nil)
  4 OP    txn.Get(ctx, /Table/50/"00fbca0e") // (nil, nil)
  4 OP    b := &Batch{}
  4 OP    b.Get(ctx, /Table/50/"0d087981") // (nil, nil)
  4 OP    txn.CommitInBatch(ctx, b) // nil
  4 OP    return nil
  4 OP  }) // nil
I200207 16:33:58.084817 5417 kv/kvnemesis/kvnemesis.go:63
  4 OP  db2.Get(ctx, /Table/50/"184059ab") // (nil, nil)
I200207 16:33:58.092869 5417 kv/kvnemesis/kvnemesis.go:63
  4 OP  db3.Get(ctx, /Table/50/"4915918b") // (nil, nil)
I200207 16:33:58.092936 5417 kv/kvnemesis/applier.go:88  Applier txn 5cbefdc8-71fe-43ed-be20-d081bf4268a0
I200207 16:33:58.097265 1078 storage/replica_rangefeed.go:505  [n2,s2,r28/2:/{Table/32-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.097305 1078 storage/replica_rangefeed.go:506  [n2,s2,r28/2:/{Table/32-Max}] WIP MVCCWriteValueOp  /Table/50/"25567fd6":1581093238.092918251,0 -> /
I200207 16:33:58.097698 151 storage/replica_rangefeed.go:564  WIP handleLogicalOpLogRaftMuLocked /Table/50/"25567fd6":1581093238.092918251,0 -> /BYTES/v-1 
I200207 16:33:58.097720 151 storage/rangefeed/processor.go:443  WIP MVCCWriteValueOp  /Table/50/"25567fd6":1581093238.092918251,0 -> /BYTES/v-1
I200207 16:33:58.097847 5176 storage/rangefeed/processor.go:603  [n1,s1,r28/1:/{Table/32-Max},rangefeed] publishValue /Table/50/"25567fd6":1581093238.092918251,0 -> /BYTES/v-1
I200207 16:33:58.097907 5139 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"25567fd6" 1581093238.092918251,0 -> /BYTES/v-1 (prev /BYTES/v-0)
I200207 16:33:58.098070 5417 kv/kvnemesis/kvnemesis.go:63
  4 OP  db0.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  4 OP    txn.Get(ctx, /Table/50/"32825014") // (nil, nil)
  4 OP    txn.Get(ctx, /Table/50/"f16eda4e") // (nil, nil)
  4 OP    txn.Get(ctx, /Table/50/"e9b876b3") // (nil, nil)
  4 OP    b := &Batch{}
  4 OP    b.Get(ctx, /Table/50/"25567fd6") // ("v-0", nil)
  4 OP    b.Get(ctx, /Table/50/"ead4a2f3") // (nil, nil)
  4 OP    b.Put(ctx, /Table/50/"25567fd6", v-1) // nil
  4 OP    txn.CommitInBatch(ctx, b) // nil
  4 OP    return nil
  4 OP  }) // nil
I200207 16:33:58.098471 1307 storage/replica_rangefeed.go:505  [n3,s3,r28/3:/{Table/32-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.098500 1307 storage/replica_rangefeed.go:506  [n3,s3,r28/3:/{Table/32-Max}] WIP MVCCWriteValueOp  /Table/50/"25567fd6":1581093238.092918251,0 -> /
I200207 16:33:58.103434 5417 kv/kvnemesis/kvnemesis.go:63
  4 OP  db1.AdminMerge(ctx, /Table/50/"105bb8ed") // storage/replica_command.go:776: merge failed: cannot merge final range
I200207 16:33:58.103517 5417 kv/kvnemesis/applier.go:88  Applier txn 4df3c74b-7723-4ddc-977c-529e540fbf96
I200207 16:33:58.105469 1081 storage/replica_rangefeed.go:505  [n2,s2,r28/2:/{Table/32-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.105507 1081 storage/replica_rangefeed.go:506  [n2,s2,r28/2:/{Table/32-Max}] WIP MVCCWriteIntentOp /Table/50/"25567fd6" 1581093238.103499287,0 1581093238.103499287,0 4df3c74b-7723-4ddc-977c-529e540fbf96
I200207 16:33:58.105745 148 storage/rangefeed/processor.go:443  WIP MVCCWriteIntentOp /Table/50/"25567fd6" 1581093238.103499287,0 1581093238.103499287,0 4df3c74b-7723-4ddc-977c-529e540fbf96
I200207 16:33:58.106096 1309 storage/replica_rangefeed.go:505  [n3,s3,r28/3:/{Table/32-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.106169 1309 storage/replica_rangefeed.go:506  [n3,s3,r28/3:/{Table/32-Max}] WIP MVCCWriteIntentOp /Table/50/"25567fd6" 1581093238.103499287,0 1581093238.103499287,0 4df3c74b-7723-4ddc-977c-529e540fbf96
I200207 16:33:58.107436 5417 kv/kvnemesis/kvnemesis.go:63
  4 OP  db2.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  4 OP    txn.Put(ctx, /Table/50/"25567fd6", v-2) // nil
  4 OP    return nil
  4 OP  }) // nil
I200207 16:33:58.107489 5417 kv/kvnemesis/applier.go:88  Applier txn 541853a2-6fc8-45fe-a5d5-f1a100f79953
I200207 16:33:58.110521 1091 storage/replica_rangefeed.go:505  [n2,s2,r28/2:/{Table/32-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.110558 1091 storage/replica_rangefeed.go:506  [n2,s2,r28/2:/{Table/32-Max}] WIP MVCCCommitIntentOp /Table/50/"25567fd6":1581093238.103499287,0 -> / 4df3c74b-7723-4ddc-977c-529e540fbf96
I200207 16:33:58.111226 1366 storage/replica_rangefeed.go:505  [n3,s3,r28/3:/{Table/32-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.111256 1366 storage/replica_rangefeed.go:506  [n3,s3,r28/3:/{Table/32-Max}] WIP MVCCCommitIntentOp /Table/50/"25567fd6":1581093238.103499287,0 -> / 4df3c74b-7723-4ddc-977c-529e540fbf96
I200207 16:33:58.111961 158 storage/replica_rangefeed.go:564  WIP handleLogicalOpLogRaftMuLocked /Table/50/"25567fd6":1581093238.103499287,0 -> /BYTES/v-2 
I200207 16:33:58.112023 158 storage/rangefeed/processor.go:443  WIP MVCCCommitIntentOp /Table/50/"25567fd6":1581093238.103499287,0 -> /BYTES/v-2 4df3c74b-7723-4ddc-977c-529e540fbf96
I200207 16:33:58.112133 5176 storage/rangefeed/processor.go:603  [n1,s1,r28/1:/{Table/32-Max},rangefeed] publishValue /Table/50/"25567fd6":1581093238.103499287,0 -> /BYTES/v-2
I200207 16:33:58.112222 5139 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"25567fd6" 1581093238.103499287,0 -> /BYTES/v-2 (prev /BYTES/v-1)
I200207 16:33:58.117244 5417 kv/kvnemesis/kvnemesis.go:63
  4 OP  db3.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  4 OP    txn.Get(ctx, /Table/50/"25567fd6") // ("v-2", nil)
  4 OP    txn.Get(ctx, /Table/50/"1a68b653") // (nil, nil)
  4 OP    return nil
  4 OP  }) // nil
I200207 16:33:58.117575 5417 kv/kvnemesis/applier.go:88  Applier txn b9eda0f5-9a6d-40e7-8baa-832b60c8c579
I200207 16:33:58.117738 5417 kv/kvnemesis/kvnemesis.go:63
  4 OP  db0.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  4 OP    return errors.New("rollback")
  4 OP  }) // rollback
I200207 16:33:58.119285 5416 storage/replica_command.go:396  [n2,s2,r28/2:/{Table/32-Max}] initiating a split of this range at key /Table/50/"06af801b" [r30] (manual)
I200207 16:33:58.121534 5279 storage/replica_command.go:396  [n2,s2,r28/2:/{Table/32-Max}] initiating a split of this range at key /Table/50/"105bb8ed" [r31] (manual)
I200207 16:33:58.124551 5417 kv/kvnemesis/kvnemesis.go:63
  4 OP  db1.AdminMerge(ctx, /Table/50/"06af801b") // storage/replica_command.go:776: merge failed: cannot merge final range
I200207 16:33:58.124604 5417 kv/kvnemesis/applier.go:88  Applier txn b77d2434-a69d-43f5-8faa-672414f69bae
I200207 16:33:58.125570 5413 kv/kvnemesis/kvnemesis.go:63
  0 OP  db1.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  0 OP    txn.Get(ctx, /Table/50/"917c48c6") // (nil, nil)
  0 OP    return errors.New("rollback")
  0 OP  }) // rollback
I200207 16:33:58.125688 5413 kv/kvnemesis/applier.go:88  Applier txn 7222459e-c104-45d3-9869-c5816c0cf318
I200207 16:33:58.131494 1102 storage/replica_rangefeed.go:505  [n2,s2,r28/2:/{Table/32-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.131537 1102 storage/replica_rangefeed.go:506  [n2,s2,r28/2:/{Table/32-Max}] WIP MVCCWriteIntentOp /Table/50/"681f3c98" 1581093238.124590463,0 1581093238.124590463,0 b77d2434-a69d-43f5-8faa-672414f69bae
I200207 16:33:58.131764 1372 storage/replica_rangefeed.go:505  [n3,s3,r28/3:/{Table/32-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.131792 1372 storage/replica_rangefeed.go:506  [n3,s3,r28/3:/{Table/32-Max}] WIP MVCCWriteIntentOp /Table/50/"681f3c98" 1581093238.124590463,0 1581093238.124590463,0 b77d2434-a69d-43f5-8faa-672414f69bae
I200207 16:33:58.132103 171 storage/rangefeed/processor.go:443  WIP MVCCWriteIntentOp /Table/50/"681f3c98" 1581093238.124590463,0 1581093238.124590463,0 b77d2434-a69d-43f5-8faa-672414f69bae
I200207 16:33:58.132450 1104 storage/replica_rangefeed.go:505  [n2,s2,r28/2:/{Table/32-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.132480 1104 storage/replica_rangefeed.go:506  [n2,s2,r28/2:/{Table/32-Max}] WIP MVCCWriteIntentOp /Table/50/"25567fd6" 1581093238.125651983,0 1581093238.125651983,0 7222459e-c104-45d3-9869-c5816c0cf318
I200207 16:33:58.132797 154 storage/rangefeed/processor.go:443  WIP MVCCWriteIntentOp /Table/50/"25567fd6" 1581093238.125651983,0 1581093238.125651983,0 7222459e-c104-45d3-9869-c5816c0cf318
I200207 16:33:58.132992 1311 storage/replica_rangefeed.go:505  [n3,s3,r28/3:/{Table/32-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.133033 1311 storage/replica_rangefeed.go:506  [n3,s3,r28/3:/{Table/32-Max}] WIP MVCCWriteIntentOp /Table/50/"25567fd6" 1581093238.125651983,0 1581093238.125651983,0 7222459e-c104-45d3-9869-c5816c0cf318
I200207 16:33:58.133421 1108 storage/replica_rangefeed.go:505  [n2,s2,r28/2:/{Table/32-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.133456 1108 storage/replica_rangefeed.go:506  [n2,s2,r28/2:/{Table/32-Max}] WIP MVCCAbortIntentOp 7222459e-c104-45d3-9869-c5816c0cf318
I200207 16:33:58.133596 163 storage/rangefeed/processor.go:443  WIP MVCCAbortIntentOp 7222459e-c104-45d3-9869-c5816c0cf318
I200207 16:33:58.133899 1368 storage/replica_rangefeed.go:505  [n3,s3,r28/3:/{Table/32-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.133920 1368 storage/replica_rangefeed.go:506  [n3,s3,r28/3:/{Table/32-Max}] WIP MVCCAbortIntentOp 7222459e-c104-45d3-9869-c5816c0cf318
I200207 16:33:58.134049 5413 kv/kvnemesis/kvnemesis.go:63
  0 OP  db3.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  0 OP    txn.Get(ctx, /Table/50/"231d22e7") // (nil, nil)
  0 OP    txn.Get(ctx, /Table/50/"25567fd6") // ("v-2", nil)
  0 OP    txn.Put(ctx, /Table/50/"25567fd6", v-5) // nil
  0 OP    return errors.New("rollback")
  0 OP  }) // rollback
I200207 16:33:58.138847 1114 storage/replica_rangefeed.go:505  [n2,s2,r28/2:/{Table/32-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.138891 1114 storage/replica_rangefeed.go:506  [n2,s2,r28/2:/{Table/32-Max}] WIP MVCCWriteValueOp  /Table/50/"472df5f4":1581093238.134363957,0 -> /
I200207 16:33:58.138906 1114 storage/replica_rangefeed.go:505  [n2,s2,r28/2:/{Table/32-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.138930 1114 storage/replica_rangefeed.go:506  [n2,s2,r28/2:/{Table/32-Max}] WIP MVCCWriteIntentOp /Table/50/"681f3c98" 1581093238.124590463,0 1581093238.124590463,0 b77d2434-a69d-43f5-8faa-672414f69bae
I200207 16:33:58.139226 5413 kv/kvnemesis/kvnemesis.go:63
  0 OP  db0.Put(ctx, /Table/50/"472df5f4", v-6) // nil
I200207 16:33:58.139271 5413 kv/kvnemesis/applier.go:88  Applier txn 1cc91982-dee8-4d12-89b9-f885de4fbc1d
I200207 16:33:58.139521 1371 storage/replica_rangefeed.go:505  [n3,s3,r9/2:/Table/1{3-4}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.139568 1371 storage/replica_rangefeed.go:506  [n3,s3,r9/2:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/32/RangeDescriptor 1581093238.119357861,0 1581093238.119357861,0 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.139601 1371 storage/replica_rangefeed.go:506  [n3,s3,r9/2:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/32/RangeDescriptor 1581093238.119357861,0 1581093238.119357861,0 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.139629 1371 storage/replica_rangefeed.go:506  [n3,s3,r9/2:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/32/RangeDescriptor 1581093238.119357861,0 1581093238.119357861,0 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.139670 1371 storage/replica_rangefeed.go:506  [n3,s3,r9/2:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/32/RangeDescriptor 1581093238.119357861,0 1581093238.119357861,0 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.139695 1371 storage/replica_rangefeed.go:506  [n3,s3,r9/2:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/32/RangeDescriptor 1581093238.119357861,0 1581093238.119357861,0 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.139236 1373 storage/replica_rangefeed.go:505  [n3,s3,r28/3:/{Table/32-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.139767 1373 storage/replica_rangefeed.go:506  [n3,s3,r28/3:/{Table/32-Max}] WIP MVCCWriteValueOp  /Table/50/"472df5f4":1581093238.134363957,0 -> /
I200207 16:33:58.139799 1373 storage/replica_rangefeed.go:505  [n3,s3,r28/3:/{Table/32-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.139823 1373 storage/replica_rangefeed.go:506  [n3,s3,r28/3:/{Table/32-Max}] WIP MVCCWriteIntentOp /Table/50/"681f3c98" 1581093238.124590463,0 1581093238.124590463,0 b77d2434-a69d-43f5-8faa-672414f69bae
I200207 16:33:58.139725 1371 storage/replica_rangefeed.go:506  [n3,s3,r9/2:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/32/RangeDescriptor 1581093238.119357861,0 1581093238.119357861,0 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.139912 1884 storage/replica_rangefeed.go:505  [n4,s4,r9/3:/Table/1{3-4}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.139955 1884 storage/replica_rangefeed.go:506  [n4,s4,r9/3:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/32/RangeDescriptor 1581093238.119357861,0 1581093238.119357861,0 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.139976 1884 storage/replica_rangefeed.go:506  [n4,s4,r9/3:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/32/RangeDescriptor 1581093238.119357861,0 1581093238.119357861,0 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.139996 1884 storage/replica_rangefeed.go:506  [n4,s4,r9/3:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/32/RangeDescriptor 1581093238.119357861,0 1581093238.119357861,0 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.140017 1884 storage/replica_rangefeed.go:506  [n4,s4,r9/3:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/32/RangeDescriptor 1581093238.119357861,0 1581093238.119357861,0 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.140036 1884 storage/replica_rangefeed.go:506  [n4,s4,r9/3:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/32/RangeDescriptor 1581093238.119357861,0 1581093238.119357861,0 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.140058 1884 storage/replica_rangefeed.go:506  [n4,s4,r9/3:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/32/RangeDescriptor 1581093238.119357861,0 1581093238.119357861,0 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.140393 1381 storage/replica_rangefeed.go:505  [n3,s3,r1/3:/{Min-System/NodeL…}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.140436 1381 storage/replica_rangefeed.go:506  [n3,s3,r1/3:/{Min-System/NodeL…}] WIP MVCCWriteIntentOp /Local/Range/Table/32/RangeDescriptor 1581093238.119357861,0 1581093238.119357861,0 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.140459 1056 storage/replica_rangefeed.go:505  [n2,s2,r9/4:/Table/1{3-4}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.140505 1056 storage/replica_rangefeed.go:506  [n2,s2,r9/4:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/32/RangeDescriptor 1581093238.119357861,0 1581093238.119357861,0 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.140534 1056 storage/replica_rangefeed.go:506  [n2,s2,r9/4:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/32/RangeDescriptor 1581093238.119357861,0 1581093238.119357861,0 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.140563 1056 storage/replica_rangefeed.go:506  [n2,s2,r9/4:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/32/RangeDescriptor 1581093238.119357861,0 1581093238.119357861,0 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.140588 1056 storage/replica_rangefeed.go:506  [n2,s2,r9/4:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/32/RangeDescriptor 1581093238.119357861,0 1581093238.119357861,0 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.140617 1056 storage/replica_rangefeed.go:506  [n2,s2,r9/4:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/32/RangeDescriptor 1581093238.119357861,0 1581093238.119357861,0 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.140642 1056 storage/replica_rangefeed.go:506  [n2,s2,r9/4:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/32/RangeDescriptor 1581093238.119357861,0 1581093238.119357861,0 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.140765 167 storage/replica_rangefeed.go:505  [n1,s1,r1/1:/{Min-System/NodeL…}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.140798 1056 storage/replica_rangefeed.go:505  [n2,s2,r28/2:/{Table/32-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.140808 167 storage/replica_rangefeed.go:506  [n1,s1,r1/1:/{Min-System/NodeL…}] WIP MVCCWriteIntentOp /Local/Range/Table/32/RangeDescriptor 1581093238.119357861,0 1581093238.119357861,0 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.140840 167 storage/replica_rangefeed.go:506  [n1,s1,r1/1:/{Min-System/NodeL…}] WIP MVCCWriteIntentOp /Local/Range/Table/32/RangeDescriptor 1581093238.119357861,0 1581093238.119357861,0 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.140467 1381 storage/replica_rangefeed.go:506  [n3,s3,r1/3:/{Min-System/NodeL…}] WIP MVCCWriteIntentOp /Local/Range/Table/32/RangeDescriptor 1581093238.119357861,0 1581093238.119357861,0 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.140816 1056 storage/replica_rangefeed.go:506  [n2,s2,r28/2:/{Table/32-Max}] WIP MVCCAbortIntentOp b77d2434-a69d-43f5-8faa-672414f69bae
I200207 16:33:58.140981 1056 storage/replica_rangefeed.go:506  [n2,s2,r28/2:/{Table/32-Max}] WIP MVCCAbortIntentOp b77d2434-a69d-43f5-8faa-672414f69bae
I200207 16:33:58.141481 1841 storage/replica_rangefeed.go:505  [n4,s4,r1/2:/{Min-System/NodeL…}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.141526 1841 storage/replica_rangefeed.go:506  [n4,s4,r1/2:/{Min-System/NodeL…}] WIP MVCCWriteIntentOp /Local/Range/Table/32/RangeDescriptor 1581093238.119357861,0 1581093238.119357861,0 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.141549 1841 storage/replica_rangefeed.go:506  [n4,s4,r1/2:/{Min-System/NodeL…}] WIP MVCCWriteIntentOp /Local/Range/Table/32/RangeDescriptor 1581093238.119357861,0 1581093238.119357861,0 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.141649 5417 kv/kvnemesis/kvnemesis.go:63
  4 OP  db2.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  4 OP    txn.Put(ctx, /Table/50/"681f3c98", v-3) // nil
  4 OP    txn.Put(ctx, /Table/50/"c2f94d6d", v-4) // nil
  4 OP    txn.Get(ctx, /Table/50/"681f3c98") // ("v-3", nil)
  4 OP    return errors.New("rollback")
  4 OP  }) // rollback
I200207 16:33:58.141947 1410 storage/replica_rangefeed.go:505  [n3,s3,r28/3:/{Table/32-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.141980 1410 storage/replica_rangefeed.go:506  [n3,s3,r28/3:/{Table/32-Max}] WIP MVCCAbortIntentOp b77d2434-a69d-43f5-8faa-672414f69bae
I200207 16:33:58.141993 1410 storage/replica_rangefeed.go:506  [n3,s3,r28/3:/{Table/32-Max}] WIP MVCCAbortIntentOp b77d2434-a69d-43f5-8faa-672414f69bae
I200207 16:33:58.142668 1060 storage/replica_rangefeed.go:505  [n2,s2,r28/2:/{Table/32-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.142787 1060 storage/replica_rangefeed.go:506  [n2,s2,r28/2:/{Table/32-Max}] WIP MVCCWriteValueOp  /Table/50/"c8082902":1581093238.142249297,0 -> /
I200207 16:33:58.143606 5417 kv/kvnemesis/kvnemesis.go:63
  4 OP  db2.Put(ctx, /Table/50/"c8082902", v-9) // nil
I200207 16:33:58.144028 1382 storage/replica_rangefeed.go:505  [n3,s3,r28/3:/{Table/32-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.144082 1382 storage/replica_rangefeed.go:506  [n3,s3,r28/3:/{Table/32-Max}] WIP MVCCWriteValueOp  /Table/50/"c8082902":1581093238.142249297,0 -> /
I200207 16:33:58.145724 5416 kv/kvnemesis/kvnemesis.go:63
  3 OP  db1.AdminSplit(ctx, /Table/50/"06af801b") // nil
I200207 16:33:58.149481 5455 storage/replica_command.go:396  [n2,s2,r30/2:/{Table/50/"06…-Max}] initiating a split of this range at key /Table/50/"105bb8ed" [r32] (manual)
I200207 16:33:58.151783 1058 storage/replica_rangefeed.go:505  [n2,s2,r3/3:/System/{NodeLive…-tsd}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.151816 1058 storage/replica_rangefeed.go:506  [n2,s2,r3/3:/System/{NodeLive…-tsd}] WIP MVCCWriteValueOp  /System/"range-idgen":1581093238.150501059,0 -> /
I200207 16:33:58.152068 1395 storage/replica_rangefeed.go:505  [n3,s3,r3/4:/System/{NodeLive…-tsd}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.152092 1395 storage/replica_rangefeed.go:506  [n3,s3,r3/4:/System/{NodeLive…-tsd}] WIP MVCCWriteValueOp  /System/"range-idgen":1581093238.150501059,0 -> /
I200207 16:33:58.152263 1875 storage/replica_rangefeed.go:505  [n4,s4,r3/2:/System/{NodeLive…-tsd}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.152287 1875 storage/replica_rangefeed.go:506  [n4,s4,r3/2:/System/{NodeLive…-tsd}] WIP MVCCWriteValueOp  /System/"range-idgen":1581093238.150501059,0 -> /
I200207 16:33:58.156619 1401 storage/replica_rangefeed.go:505  [n3,s3,r1/3:/{Min-System/NodeL…}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.156658 1401 storage/replica_rangefeed.go:506  [n3,s3,r1/3:/{Min-System/NodeL…}] WIP MVCCCommitIntentOp /Meta2/Table/50/"06af801b":1581093238.119357861,0 -> / 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.156677 1401 storage/replica_rangefeed.go:506  [n3,s3,r1/3:/{Min-System/NodeL…}] WIP MVCCCommitIntentOp /Meta2/Max:1581093238.119357861,0 -> / 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.157018 76 storage/replica_rangefeed.go:505  [n1,s1,r1/1:/{Min-System/NodeL…}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.157048 76 storage/replica_rangefeed.go:506  [n1,s1,r1/1:/{Min-System/NodeL…}] WIP MVCCCommitIntentOp /Meta2/Table/50/"06af801b":1581093238.119357861,0 -> / 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.157068 76 storage/replica_rangefeed.go:506  [n1,s1,r1/1:/{Min-System/NodeL…}] WIP MVCCCommitIntentOp /Meta2/Max:1581093238.119357861,0 -> / 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.157388 1877 storage/replica_rangefeed.go:505  [n4,s4,r1/2:/{Min-System/NodeL…}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.157424 1877 storage/replica_rangefeed.go:506  [n4,s4,r1/2:/{Min-System/NodeL…}] WIP MVCCCommitIntentOp /Meta2/Table/50/"06af801b":1581093238.119357861,0 -> / 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.157443 1877 storage/replica_rangefeed.go:506  [n4,s4,r1/2:/{Min-System/NodeL…}] WIP MVCCCommitIntentOp /Meta2/Max:1581093238.119357861,0 -> / 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.159431 1392 storage/replica_rangefeed.go:505  [n3,s3,r9/2:/Table/1{3-4}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.162283 1392 storage/replica_rangefeed.go:506  [n3,s3,r9/2:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.119358Z/527639635997687810/0:1581093238.119357861,0 -> / 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.159738 79 storage/replica_rangefeed.go:564  WIP handleLogicalOpLogRaftMuLocked /Table/50/"472df5f4":1581093238.134363957,0 -> /BYTES/v-6 
I200207 16:33:58.160162 1879 storage/replica_rangefeed.go:505  [n4,s4,r9/3:/Table/1{3-4}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.160673 1079 storage/replica_rangefeed.go:505  [n2,s2,r30/2:/{Table/50/"06…-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.160980 1412 storage/replica_rangefeed.go:505  [n3,s3,r30/3:/{Table/50/"06…-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.162077 1411 storage/replica_rangefeed.go:505  [n3,s3,r1/3:/{Min-System/NodeL…}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.162880 1411 storage/replica_rangefeed.go:506  [n3,s3,r1/3:/{Min-System/NodeL…}] WIP MVCCWriteIntentOp /Local/Range/Table/50/"06af801b"/RangeDescriptor 1581093238.149503121,0 1581093238.149503121,0 b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.163115 120 storage/replica_rangefeed.go:505  [n1,s1,r1/1:/{Min-System/NodeL…}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.163241 120 storage/replica_rangefeed.go:506  [n1,s1,r1/1:/{Min-System/NodeL…}] WIP MVCCWriteIntentOp /Local/Range/Table/50/"06af801b"/RangeDescriptor 1581093238.149503121,0 1581093238.149503121,0 b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.163315 120 storage/replica_rangefeed.go:506  [n1,s1,r1/1:/{Min-System/NodeL…}] WIP MVCCWriteIntentOp /Local/Range/Table/50/"06af801b"/RangeDescriptor 1581093238.149503121,0 1581093238.149503121,0 b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.163170 1878 storage/replica_rangefeed.go:505  [n4,s4,r1/2:/{Min-System/NodeL…}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.163440 1878 storage/replica_rangefeed.go:506  [n4,s4,r1/2:/{Min-System/NodeL…}] WIP MVCCWriteIntentOp /Local/Range/Table/50/"06af801b"/RangeDescriptor 1581093238.149503121,0 1581093238.149503121,0 b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.163475 1878 storage/replica_rangefeed.go:506  [n4,s4,r1/2:/{Min-System/NodeL…}] WIP MVCCWriteIntentOp /Local/Range/Table/50/"06af801b"/RangeDescriptor 1581093238.149503121,0 1581093238.149503121,0 b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.162469 79 storage/rangefeed/processor.go:443  WIP MVCCWriteValueOp  /Table/50/"472df5f4":1581093238.134363957,0 -> /BYTES/v-6
I200207 16:33:58.163572 79 storage/rangefeed/processor.go:443  WIP MVCCWriteIntentOp /Table/50/"681f3c98" 1581093238.124590463,0 1581093238.124590463,0 b77d2434-a69d-43f5-8faa-672414f69bae
I200207 16:33:58.163669 5176 storage/rangefeed/processor.go:603  [n1,s1,r28/1:/{Table/32-Max},rangefeed] publishValue /Table/50/"472df5f4":1581093238.134363957,0 -> /BYTES/v-6
I200207 16:33:58.163730 5139 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"472df5f4" 1581093238.134363957,0 -> /BYTES/v-6 (prev /)
I200207 16:33:58.162543 1080 storage/replica_rangefeed.go:505  [n2,s2,r9/4:/Table/1{3-4}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.163790 1080 storage/replica_rangefeed.go:506  [n2,s2,r9/4:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.119358Z/527639635997687810/0:1581093238.119357861,0 -> / 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.163819 1080 storage/replica_rangefeed.go:506  [n2,s2,r9/4:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.119358Z/527639635997687810/2/1:1581093238.119357861,0 -> / 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.163847 1080 storage/replica_rangefeed.go:506  [n2,s2,r9/4:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.119358Z/527639635997687810/3/1:1581093238.119357861,0 -> / 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.162381 1392 storage/replica_rangefeed.go:506  [n3,s3,r9/2:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.119358Z/527639635997687810/2/1:1581093238.119357861,0 -> / 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.162673 1079 storage/replica_rangefeed.go:506  [n2,s2,r30/2:/{Table/50/"06…-Max}] WIP MVCCWriteIntentOp /Table/50/"c8082902" 1581093238.142249297,1 1581093238.139260809,0 1cc91982-dee8-4d12-89b9-f885de4fbc1d
I200207 16:33:58.162788 1412 storage/replica_rangefeed.go:506  [n3,s3,r30/3:/{Table/50/"06…-Max}] WIP MVCCWriteIntentOp /Table/50/"c8082902" 1581093238.142249297,1 1581093238.139260809,0 1cc91982-dee8-4d12-89b9-f885de4fbc1d
I200207 16:33:58.162579 1879 storage/replica_rangefeed.go:506  [n4,s4,r9/3:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.119358Z/527639635997687810/0:1581093238.119357861,0 -> / 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.163915 1080 storage/replica_rangefeed.go:506  [n2,s2,r9/4:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.119358Z/527639635997687810/4/1:1581093238.119357861,0 -> / 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.164007 1411 storage/replica_rangefeed.go:506  [n3,s3,r1/3:/{Min-System/NodeL…}] WIP MVCCWriteIntentOp /Local/Range/Table/50/"06af801b"/RangeDescriptor 1581093238.149503121,0 1581093238.149503121,0 b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.164021 1392 storage/replica_rangefeed.go:506  [n3,s3,r9/2:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.119358Z/527639635997687810/3/1:1581093238.119357861,0 -> / 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.165976 1086 storage/replica_rangefeed.go:505  [n2,s2,r30/2:/{Table/50/"06…-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.166590 1086 storage/replica_rangefeed.go:506  [n2,s2,r30/2:/{Table/50/"06…-Max}] WIP MVCCUpdateIntentOp 1581093238.142249297,1 1cc91982-dee8-4d12-89b9-f885de4fbc1d
I200207 16:33:58.166241 1879 storage/replica_rangefeed.go:506  [n4,s4,r9/3:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.119358Z/527639635997687810/2/1:1581093238.119357861,0 -> / 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.167399 1879 storage/replica_rangefeed.go:506  [n4,s4,r9/3:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.119358Z/527639635997687810/3/1:1581093238.119357861,0 -> / 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.166283 1080 storage/replica_rangefeed.go:506  [n2,s2,r9/4:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.119358Z/527639635997687810/5/1:1581093238.119357861,0 -> / 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.166518 1392 storage/replica_rangefeed.go:506  [n3,s3,r9/2:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.119358Z/527639635997687810/4/1:1581093238.119357861,0 -> / 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.167010 5413 kv/kvnemesis/kvnemesis.go:63
  0 OP  db1.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  0 OP    txn.Get(ctx, /Table/50/"c2f94d6d") // (nil, nil)
  0 OP    txn.Put(ctx, /Table/50/"c8082902", v-7) // nil
  0 OP    txn.Put(ctx, /Table/50/"c8082902", v-8) // nil
  0 OP    return nil
  0 OP  }) // nil
I200207 16:33:58.167756 5413 kv/kvnemesis/applier.go:88  Applier txn ec6b7da9-edf2-4dd8-9773-6a19f29d9163
I200207 16:33:58.168113 5413 kv/kvnemesis/kvnemesis.go:63
  0 OP  db1.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  0 OP    return nil
  0 OP  }) // nil
I200207 16:33:58.167211 1415 storage/replica_rangefeed.go:505  [n3,s3,r30/3:/{Table/50/"06…-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.169258 1415 storage/replica_rangefeed.go:506  [n3,s3,r30/3:/{Table/50/"06…-Max}] WIP MVCCUpdateIntentOp 1581093238.142249297,1 1cc91982-dee8-4d12-89b9-f885de4fbc1d
I200207 16:33:58.167505 1879 storage/replica_rangefeed.go:506  [n4,s4,r9/3:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.119358Z/527639635997687810/4/1:1581093238.119357861,0 -> / 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.167586 1080 storage/replica_rangefeed.go:506  [n2,s2,r9/4:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.119358Z/527639635997687810/6/1:1581093238.119357861,0 -> / 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.167657 1392 storage/replica_rangefeed.go:506  [n3,s3,r9/2:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.119358Z/527639635997687810/5/1:1581093238.119357861,0 -> / 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.168817 80 storage/rangefeed/processor.go:443  WIP MVCCAbortIntentOp b77d2434-a69d-43f5-8faa-672414f69bae
I200207 16:33:58.169622 80 storage/rangefeed/processor.go:443  WIP MVCCAbortIntentOp b77d2434-a69d-43f5-8faa-672414f69bae
I200207 16:33:58.169423 1879 storage/replica_rangefeed.go:506  [n4,s4,r9/3:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.119358Z/527639635997687810/5/1:1581093238.119357861,0 -> / 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.169746 1879 storage/replica_rangefeed.go:506  [n4,s4,r9/3:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.119358Z/527639635997687810/6/1:1581093238.119357861,0 -> / 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.169811 80 storage/replica_rangefeed.go:564  WIP handleLogicalOpLogRaftMuLocked /Table/50/"c8082902":1581093238.142249297,0 -> /BYTES/v-9 
I200207 16:33:58.169562 1392 storage/replica_rangefeed.go:506  [n3,s3,r9/2:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.119358Z/527639635997687810/6/1:1581093238.119357861,0 -> / 444a7dab-a283-4583-873a-e0a00cdefd81
I200207 16:33:58.169854 80 storage/rangefeed/processor.go:443  WIP MVCCWriteValueOp  /Table/50/"c8082902":1581093238.142249297,0 -> /BYTES/v-9
I200207 16:33:58.170065 5176 storage/rangefeed/processor.go:603  [n1,s1,r28/1:/{Table/32-Max},rangefeed] publishValue /Table/50/"c8082902":1581093238.142249297,0 -> /BYTES/v-9
I200207 16:33:58.170144 5139 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"c8082902" 1581093238.142249297,0 -> /BYTES/v-9 (prev /)
I200207 16:33:58.170763 1312 storage/replica_rangefeed.go:505  [n3,s3,r9/2:/Table/1{3-4}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.170806 1312 storage/replica_rangefeed.go:506  [n3,s3,r9/2:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/50/"06af801b"/RangeDescriptor 1581093238.149503121,0 1581093238.149503121,0 b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.170834 1312 storage/replica_rangefeed.go:506  [n3,s3,r9/2:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/50/"06af801b"/RangeDescriptor 1581093238.149503121,0 1581093238.149503121,0 b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.170861 1312 storage/replica_rangefeed.go:506  [n3,s3,r9/2:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/50/"06af801b"/RangeDescriptor 1581093238.149503121,0 1581093238.149503121,0 b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.170887 1312 storage/replica_rangefeed.go:506  [n3,s3,r9/2:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/50/"06af801b"/RangeDescriptor 1581093238.149503121,0 1581093238.149503121,0 b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.170912 1312 storage/replica_rangefeed.go:506  [n3,s3,r9/2:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/50/"06af801b"/RangeDescriptor 1581093238.149503121,0 1581093238.149503121,0 b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.170955 1312 storage/replica_rangefeed.go:506  [n3,s3,r9/2:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/50/"06af801b"/RangeDescriptor 1581093238.149503121,0 1581093238.149503121,0 b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.171096 1892 storage/replica_rangefeed.go:505  [n4,s4,r9/3:/Table/1{3-4}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.171262 1892 storage/replica_rangefeed.go:506  [n4,s4,r9/3:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/50/"06af801b"/RangeDescriptor 1581093238.149503121,0 1581093238.149503121,0 b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.171407 1892 storage/replica_rangefeed.go:506  [n4,s4,r9/3:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/50/"06af801b"/RangeDescriptor 1581093238.149503121,0 1581093238.149503121,0 b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.171447 1892 storage/replica_rangefeed.go:506  [n4,s4,r9/3:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/50/"06af801b"/RangeDescriptor 1581093238.149503121,0 1581093238.149503121,0 b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.171594 1892 storage/replica_rangefeed.go:506  [n4,s4,r9/3:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/50/"06af801b"/RangeDescriptor 1581093238.149503121,0 1581093238.149503121,0 b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.171635 1892 storage/replica_rangefeed.go:506  [n4,s4,r9/3:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/50/"06af801b"/RangeDescriptor 1581093238.149503121,0 1581093238.149503121,0 b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.171804 1892 storage/replica_rangefeed.go:506  [n4,s4,r9/3:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/50/"06af801b"/RangeDescriptor 1581093238.149503121,0 1581093238.149503121,0 b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.171594 1085 storage/replica_rangefeed.go:505  [n2,s2,r9/4:/Table/1{3-4}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.172015 1085 storage/replica_rangefeed.go:506  [n2,s2,r9/4:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/50/"06af801b"/RangeDescriptor 1581093238.149503121,0 1581093238.149503121,0 b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.172045 1085 storage/replica_rangefeed.go:506  [n2,s2,r9/4:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/50/"06af801b"/RangeDescriptor 1581093238.149503121,0 1581093238.149503121,0 b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.172073 1085 storage/replica_rangefeed.go:506  [n2,s2,r9/4:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/50/"06af801b"/RangeDescriptor 1581093238.149503121,0 1581093238.149503121,0 b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.172095 1085 storage/replica_rangefeed.go:506  [n2,s2,r9/4:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/50/"06af801b"/RangeDescriptor 1581093238.149503121,0 1581093238.149503121,0 b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.172121 1085 storage/replica_rangefeed.go:506  [n2,s2,r9/4:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/50/"06af801b"/RangeDescriptor 1581093238.149503121,0 1581093238.149503121,0 b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.172147 1085 storage/replica_rangefeed.go:506  [n2,s2,r9/4:/Table/1{3-4}] WIP MVCCWriteIntentOp /Local/Range/Table/50/"06af801b"/RangeDescriptor 1581093238.149503121,0 1581093238.149503121,0 b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.173358 1089 storage/replica_rangefeed.go:505  [n2,s2,r30/2:/{Table/50/"06…-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.173404 1089 storage/replica_rangefeed.go:506  [n2,s2,r30/2:/{Table/50/"06…-Max}] WIP MVCCCommitIntentOp /Table/50/"c8082902":1581093238.142249297,1 -> / 1cc91982-dee8-4d12-89b9-f885de4fbc1d
I200207 16:33:58.173428 1089 storage/replica_rangefeed.go:505  [n2,s2,r30/2:/{Table/50/"06…-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.173457 1089 storage/replica_rangefeed.go:506  [n2,s2,r30/2:/{Table/50/"06…-Max}] WIP MVCCWriteValueOp  /Table/50/"b6c11210":1581093238.168988514,0 -> /
I200207 16:33:58.173778 5413 kv/kvnemesis/kvnemesis.go:63
  0 OP  db2.Put(ctx, /Table/50/"b6c11210", v-10) // nil
I200207 16:33:58.173839 5413 kv/kvnemesis/applier.go:88  Applier txn fb8762a8-4483-4d21-b4c2-f1708b77365a
I200207 16:33:58.174259 5176 storage/rangefeed/registry.go:535  remove /Table/5{0-1} 1581093237.992869126,0 retry rangefeed (REASON_RANGE_SPLIT)
I200207 16:33:58.174356 5174 kv/dist_sender_rangefeed.go:281  [n1] RangeFeedError: retry rangefeed (REASON_RANGE_SPLIT)
I200207 16:33:58.174422 5174 kv/dist_sender_rangefeed.go:153  [n1] RangeFeed dial /Table/5{0-1} 1581093237.992869126,0 returned 1581093238.059657892,0 retry rangefeed (REASON_RANGE_SPLIT)
I200207 16:33:58.174548 5393 kv/dist_sender_rangefeed.go:151  [n1] RangeFeed dial /Table/5{0/"06af801b"-1} 1581093238.059657892,0
I200207 16:33:58.175237 5392 kv/dist_sender_rangefeed.go:151  [n1] RangeFeed dial /Table/50{-/"06af801b"} 1581093238.059657892,0
I200207 16:33:58.175582 5732 storage/rangefeed/registry.go:451  register /Table/50{-/"06af801b"} 1581093238.059657892,0
I200207 16:33:58.175951 5395 storage/rangefeed/registry.go:451  register /Table/5{0/"06af801b"-1} 1581093238.059657892,0
I200207 16:33:58.177387 1307 storage/replica_rangefeed.go:505  [n3,s3,r30/3:/{Table/50/"06…-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.178399 1307 storage/replica_rangefeed.go:506  [n3,s3,r30/3:/{Table/50/"06…-Max}] WIP MVCCCommitIntentOp /Table/50/"c8082902":1581093238.142249297,1 -> / 1cc91982-dee8-4d12-89b9-f885de4fbc1d
I200207 16:33:58.178469 1307 storage/replica_rangefeed.go:505  [n3,s3,r30/3:/{Table/50/"06…-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.178890 1307 storage/replica_rangefeed.go:506  [n3,s3,r30/3:/{Table/50/"06…-Max}] WIP MVCCWriteValueOp  /Table/50/"b6c11210":1581093238.168988514,0 -> /
I200207 16:33:58.179654 5139 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"25567fd6" 1581093238.070282014,0 -> /BYTES/v-0 (prev /)
I200207 16:33:58.179733 5139 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"25567fd6" 1581093238.092918251,0 -> /BYTES/v-1 (prev /BYTES/v-0)
I200207 16:33:58.179808 5139 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"25567fd6" 1581093238.103499287,0 -> /BYTES/v-2 (prev /BYTES/v-1)
I200207 16:33:58.179869 5139 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"472df5f4" 1581093238.134363957,0 -> /BYTES/v-6 (prev /)
I200207 16:33:58.179935 5139 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"c8082902" 1581093238.142249297,0 -> /BYTES/v-9 (prev /)
I200207 16:33:58.187629 5416 kv/kvnemesis/kvnemesis.go:63
  3 OP  db0.AdminSplit(ctx, /Table/50/"105bb8ed") // nil
I200207 16:33:58.187715 5416 kv/kvnemesis/applier.go:88  Applier txn 368aa35c-e12d-4c4a-bf33-93b195dc6b10
I200207 16:33:58.188745 5416 kv/kvnemesis/kvnemesis.go:63
  3 OP  db0.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  3 OP    txn.Get(ctx, /Table/50/"c2f94d6d") // (nil, nil)
  3 OP    b := &Batch{}
  3 OP    txn.CommitInBatch(ctx, b) // nil
  3 OP    return nil
  3 OP  }) // nil
I200207 16:33:58.189053 5417 kv/kvnemesis/kvnemesis.go:63
  4 OP  db3.Get(ctx, /Table/50/"c8082902") // ("v-8", nil)
I200207 16:33:58.189080 5413 kv/kvnemesis/kvnemesis.go:63
  0 OP  db3.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  0 OP    txn.Get(ctx, /Table/50/"472df5f4") // ("v-6", nil)
  0 OP    return errors.New("rollback")
  0 OP  }) // rollback
I200207 16:33:58.189175 5413 kv/kvnemesis/applier.go:88  Applier txn 9a96d41b-8843-4c7a-8adc-bd274cb1b2a0
I200207 16:33:58.192913 1107 storage/replica_rangefeed.go:505  [n2,s2,r32/2:/{Table/50/"10…-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.192976 1107 storage/replica_rangefeed.go:506  [n2,s2,r32/2:/{Table/50/"10…-Max}] WIP MVCCWriteValueOp  /Table/50/"317ffd5d":1581093238.188813359,0 -> /
I200207 16:33:58.192998 1107 storage/replica_rangefeed.go:505  [n2,s2,r32/2:/{Table/50/"10…-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.193021 1107 storage/replica_rangefeed.go:506  [n2,s2,r32/2:/{Table/50/"10…-Max}] WIP MVCCWriteIntentOp /Table/50/"4493fa8c" 1581093238.189158698,0 1581093238.189158698,0 9a96d41b-8843-4c7a-8adc-bd274cb1b2a0
I200207 16:33:58.193216 5416 kv/kvnemesis/kvnemesis.go:63
  3 OP  db1.Put(ctx, /Table/50/"317ffd5d", v-11) // nil
I200207 16:33:58.193697 1310 storage/replica_rangefeed.go:505  [n3,s3,r32/3:/{Table/50/"10…-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.193769 1310 storage/replica_rangefeed.go:506  [n3,s3,r32/3:/{Table/50/"10…-Max}] WIP MVCCWriteValueOp  /Table/50/"317ffd5d":1581093238.188813359,0 -> /
I200207 16:33:58.193806 1310 storage/replica_rangefeed.go:505  [n3,s3,r32/3:/{Table/50/"10…-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.193847 1310 storage/replica_rangefeed.go:506  [n3,s3,r32/3:/{Table/50/"10…-Max}] WIP MVCCWriteIntentOp /Table/50/"4493fa8c" 1581093238.189158698,0 1581093238.189158698,0 9a96d41b-8843-4c7a-8adc-bd274cb1b2a0
I200207 16:33:58.193871 1409 storage/replica_rangefeed.go:505  [n3,s3,r1/3:/{Min-System/NodeL…}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.194059 1409 storage/replica_rangefeed.go:506  [n3,s3,r1/3:/{Min-System/NodeL…}] WIP MVCCCommitIntentOp /Meta2/Table/50/"105bb8ed":1581093238.149503121,0 -> / b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.194103 1409 storage/replica_rangefeed.go:506  [n3,s3,r1/3:/{Min-System/NodeL…}] WIP MVCCCommitIntentOp /Meta2/Max:1581093238.149503121,0 -> / b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.194619 117 storage/replica_rangefeed.go:505  [n1,s1,r1/1:/{Min-System/NodeL…}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.195030 117 storage/replica_rangefeed.go:506  [n1,s1,r1/1:/{Min-System/NodeL…}] WIP MVCCCommitIntentOp /Meta2/Table/50/"105bb8ed":1581093238.149503121,0 -> / b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.195206 117 storage/replica_rangefeed.go:506  [n1,s1,r1/1:/{Min-System/NodeL…}] WIP MVCCCommitIntentOp /Meta2/Max:1581093238.149503121,0 -> / b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.195241 1369 storage/replica_rangefeed.go:505  [n3,s3,r9/2:/Table/1{3-4}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.195438 1369 storage/replica_rangefeed.go:506  [n3,s3,r9/2:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.149503Z/527639636083146754/0:1581093238.149503121,0 -> / b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.195511 1369 storage/replica_rangefeed.go:506  [n3,s3,r9/2:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.149503Z/527639636083146754/2/1:1581093238.149503121,0 -> / b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.195570 1369 storage/replica_rangefeed.go:506  [n3,s3,r9/2:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.149503Z/527639636083146754/3/1:1581093238.149503121,0 -> / b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.195660 1369 storage/replica_rangefeed.go:506  [n3,s3,r9/2:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.149503Z/527639636083146754/4/1:1581093238.149503121,0 -> / b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.195719 1369 storage/replica_rangefeed.go:506  [n3,s3,r9/2:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.149503Z/527639636083146754/5/1:1581093238.149503121,0 -> / b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.194643 1891 storage/replica_rangefeed.go:505  [n4,s4,r1/2:/{Min-System/NodeL…}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.195817 1369 storage/replica_rangefeed.go:506  [n3,s3,r9/2:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.149503Z/527639636083146754/6/1:1581093238.149503121,0 -> / b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.195596 1105 storage/replica_rangefeed.go:505  [n2,s2,r9/4:/Table/1{3-4}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.195987 1105 storage/replica_rangefeed.go:506  [n2,s2,r9/4:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.149503Z/527639636083146754/0:1581093238.149503121,0 -> / b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.196022 1105 storage/replica_rangefeed.go:506  [n2,s2,r9/4:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.149503Z/527639636083146754/2/1:1581093238.149503121,0 -> / b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.196046 1105 storage/replica_rangefeed.go:506  [n2,s2,r9/4:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.149503Z/527639636083146754/3/1:1581093238.149503121,0 -> / b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.196064 1105 storage/replica_rangefeed.go:506  [n2,s2,r9/4:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.149503Z/527639636083146754/4/1:1581093238.149503121,0 -> / b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.196083 1105 storage/replica_rangefeed.go:506  [n2,s2,r9/4:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.149503Z/527639636083146754/5/1:1581093238.149503121,0 -> / b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.196129 1105 storage/replica_rangefeed.go:506  [n2,s2,r9/4:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.149503Z/527639636083146754/6/1:1581093238.149503121,0 -> / b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.195480 1893 storage/replica_rangefeed.go:505  [n4,s4,r9/3:/Table/1{3-4}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.196214 1893 storage/replica_rangefeed.go:506  [n4,s4,r9/3:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.149503Z/527639636083146754/0:1581093238.149503121,0 -> / b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.196241 1893 storage/replica_rangefeed.go:506  [n4,s4,r9/3:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.149503Z/527639636083146754/2/1:1581093238.149503121,0 -> / b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.196262 1893 storage/replica_rangefeed.go:506  [n4,s4,r9/3:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.149503Z/527639636083146754/3/1:1581093238.149503121,0 -> / b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.196284 1893 storage/replica_rangefeed.go:506  [n4,s4,r9/3:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.149503Z/527639636083146754/4/1:1581093238.149503121,0 -> / b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.196303 1893 storage/replica_rangefeed.go:506  [n4,s4,r9/3:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.149503Z/527639636083146754/5/1:1581093238.149503121,0 -> / b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.196322 1893 storage/replica_rangefeed.go:506  [n4,s4,r9/3:/Table/1{3-4}] WIP MVCCCommitIntentOp /Table/13/1/2020-02-07T16:33:58.149503Z/527639636083146754/6/1:1581093238.149503121,0 -> / b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.195881 1891 storage/replica_rangefeed.go:506  [n4,s4,r1/2:/{Min-System/NodeL…}] WIP MVCCCommitIntentOp /Meta2/Table/50/"105bb8ed":1581093238.149503121,0 -> / b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.196399 1891 storage/replica_rangefeed.go:506  [n4,s4,r1/2:/{Min-System/NodeL…}] WIP MVCCCommitIntentOp /Meta2/Max:1581093238.149503121,0 -> / b302dc3f-6356-4ed7-aa67-124ff5d04de0
I200207 16:33:58.197936 5416 kv/kvnemesis/kvnemesis.go:63
  3 OP  db0.AdminMerge(ctx, /Table/50/"105bb8ed") // storage/replica_command.go:776: merge failed: cannot merge final range
I200207 16:33:58.198025 5416 kv/kvnemesis/applier.go:88  Applier txn 22651cb7-1aee-470f-94f6-e71bb3a784c6
I200207 16:33:58.199171 5415 kv/kvnemesis/kvnemesis.go:63
  2 OP  db3.AdminSplit(ctx, /Table/50/"105bb8ed") // nil
I200207 16:33:58.201343 1110 storage/replica_rangefeed.go:505  [n2,s2,r32/2:/{Table/50/"10…-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.201374 1110 storage/replica_rangefeed.go:506  [n2,s2,r32/2:/{Table/50/"10…-Max}] WIP MVCCWriteValueOp  /Table/50/"b6c11210":1581093238.191042087,0 -> /
I200207 16:33:58.203552 5417 kv/kvnemesis/kvnemesis.go:63
  4 OP  db2.Put(ctx, /Table/50/"b6c11210", v-12) // nil
I200207 16:33:58.203608 1381 storage/replica_rangefeed.go:505  [n3,s3,r32/3:/{Table/50/"10…-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.203641 1381 storage/replica_rangefeed.go:506  [n3,s3,r32/3:/{Table/50/"10…-Max}] WIP MVCCWriteValueOp  /Table/50/"b6c11210":1581093238.191042087,0 -> /
I200207 16:33:58.204551 1112 storage/replica_rangefeed.go:505  [n2,s2,r32/2:/{Table/50/"10…-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.204621 1112 storage/replica_rangefeed.go:506  [n2,s2,r32/2:/{Table/50/"10…-Max}] WIP MVCCWriteIntentOp /Table/50/"b6c11210" 1581093238.198008960,0 1581093238.198008960,0 22651cb7-1aee-470f-94f6-e71bb3a784c6
I200207 16:33:58.204698 1112 storage/replica_rangefeed.go:505  [n2,s2,r32/2:/{Table/50/"10…-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.204914 1112 storage/replica_rangefeed.go:506  [n2,s2,r32/2:/{Table/50/"10…-Max}] WIP MVCCWriteIntentOp /Table/50/"b6c11210" 1581093238.198008960,0 1581093238.198008960,0 22651cb7-1aee-470f-94f6-e71bb3a784c6
I200207 16:33:58.205416 1386 storage/replica_rangefeed.go:505  [n3,s3,r32/3:/{Table/50/"10…-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.205444 1386 storage/replica_rangefeed.go:506  [n3,s3,r32/3:/{Table/50/"10…-Max}] WIP MVCCWriteIntentOp /Table/50/"b6c11210" 1581093238.198008960,0 1581093238.198008960,0 22651cb7-1aee-470f-94f6-e71bb3a784c6
I200207 16:33:58.205460 1386 storage/replica_rangefeed.go:505  [n3,s3,r32/3:/{Table/50/"10…-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.205479 1386 storage/replica_rangefeed.go:506  [n3,s3,r32/3:/{Table/50/"10…-Max}] WIP MVCCWriteIntentOp /Table/50/"b6c11210" 1581093238.198008960,0 1581093238.198008960,0 22651cb7-1aee-470f-94f6-e71bb3a784c6
I200207 16:33:58.207967 1114 storage/replica_rangefeed.go:505  [n2,s2,r32/2:/{Table/50/"10…-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.207998 1114 storage/replica_rangefeed.go:506  [n2,s2,r32/2:/{Table/50/"10…-Max}] WIP MVCCWriteIntentOp /Table/50/"b6c11210" 1581093238.198008960,0 1581093238.198008960,0 22651cb7-1aee-470f-94f6-e71bb3a784c6
I200207 16:33:58.208792 1410 storage/replica_rangefeed.go:505  [n3,s3,r32/3:/{Table/50/"10…-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.208824 1410 storage/replica_rangefeed.go:506  [n3,s3,r32/3:/{Table/50/"10…-Max}] WIP MVCCWriteIntentOp /Table/50/"b6c11210" 1581093238.198008960,0 1581093238.198008960,0 22651cb7-1aee-470f-94f6-e71bb3a784c6
I200207 16:33:58.209710 5416 kv/kvnemesis/kvnemesis.go:63
  3 OP  db1.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  3 OP    txn.Put(ctx, /Table/50/"b6c11210", v-16) // nil
  3 OP    txn.Put(ctx, /Table/50/"5417c5c4", v-17) // nil
  3 OP    txn.Put(ctx, /Table/50/"a577349f", v-18) // nil
  3 OP    return nil
  3 OP  }) // nil
I200207 16:33:58.210519 1097 storage/replica_rangefeed.go:505  [n2,s2,r32/2:/{Table/50/"10…-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.210581 1097 storage/replica_rangefeed.go:506  [n2,s2,r32/2:/{Table/50/"10…-Max}] WIP MVCCCommitIntentOp /Table/50/"5417c5c4":1581093238.198008960,0 -> / 22651cb7-1aee-470f-94f6-e71bb3a784c6
I200207 16:33:58.210623 1097 storage/replica_rangefeed.go:506  [n2,s2,r32/2:/{Table/50/"10…-Max}] WIP MVCCCommitIntentOp /Table/50/"a577349f":1581093238.198008960,0 -> / 22651cb7-1aee-470f-94f6-e71bb3a784c6
I200207 16:33:58.210653 1097 storage/replica_rangefeed.go:506  [n2,s2,r32/2:/{Table/50/"10…-Max}] WIP MVCCCommitIntentOp /Table/50/"b6c11210":1581093238.198008960,0 -> / 22651cb7-1aee-470f-94f6-e71bb3a784c6
I200207 16:33:58.211024 1382 storage/replica_rangefeed.go:505  [n3,s3,r32/3:/{Table/50/"10…-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.211090 1382 storage/replica_rangefeed.go:506  [n3,s3,r32/3:/{Table/50/"10…-Max}] WIP MVCCCommitIntentOp /Table/50/"5417c5c4":1581093238.198008960,0 -> / 22651cb7-1aee-470f-94f6-e71bb3a784c6
I200207 16:33:58.211127 1382 storage/replica_rangefeed.go:506  [n3,s3,r32/3:/{Table/50/"10…-Max}] WIP MVCCCommitIntentOp /Table/50/"a577349f":1581093238.198008960,0 -> / 22651cb7-1aee-470f-94f6-e71bb3a784c6
I200207 16:33:58.211161 1382 storage/replica_rangefeed.go:506  [n3,s3,r32/3:/{Table/50/"10…-Max}] WIP MVCCCommitIntentOp /Table/50/"b6c11210":1581093238.198008960,0 -> / 22651cb7-1aee-470f-94f6-e71bb3a784c6
I200207 16:33:58.223818 5414 kv/kvnemesis/kvnemesis.go:63
  1 OP  db2.AdminSplit(ctx, /Table/50/"06af801b") // nil
I200207 16:33:58.241037 1064 storage/replica_rangefeed.go:505  [n2,s2,r32/2:/{Table/50/"10…-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.241105 1064 storage/replica_rangefeed.go:506  [n2,s2,r32/2:/{Table/50/"10…-Max}] WIP MVCCWriteIntentOp /Table/50/"4493fa8c" 1581093238.198008960,1 1581093238.189158698,0 9a96d41b-8843-4c7a-8adc-bd274cb1b2a0
I200207 16:33:58.241161 1064 storage/replica_rangefeed.go:506  [n2,s2,r32/2:/{Table/50/"10…-Max}] WIP MVCCWriteIntentOp /Table/50/"4493fa8c" 1581093238.198008960,1 1581093238.189158698,0 9a96d41b-8843-4c7a-8adc-bd274cb1b2a0
I200207 16:33:58.241872 1395 storage/replica_rangefeed.go:505  [n3,s3,r32/3:/{Table/50/"10…-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.241945 1395 storage/replica_rangefeed.go:506  [n3,s3,r32/3:/{Table/50/"10…-Max}] WIP MVCCWriteIntentOp /Table/50/"4493fa8c" 1581093238.198008960,1 1581093238.189158698,0 9a96d41b-8843-4c7a-8adc-bd274cb1b2a0
I200207 16:33:58.241979 1395 storage/replica_rangefeed.go:506  [n3,s3,r32/3:/{Table/50/"10…-Max}] WIP MVCCWriteIntentOp /Table/50/"4493fa8c" 1581093238.198008960,1 1581093238.189158698,0 9a96d41b-8843-4c7a-8adc-bd274cb1b2a0
I200207 16:33:58.242373 5413 kv/kvnemesis/kvnemesis.go:63
  0 OP  db3.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
  0 OP    txn.Put(ctx, /Table/50/"4493fa8c", v-13) // nil
  0 OP    b := &Batch{}
  0 OP    b.Put(ctx, /Table/50/"efda1d8b", v-14) // nil
  0 OP    b.Put(ctx, /Table/50/"b6c11210", v-15) // nil
  0 OP    txn.CommitInBatch(ctx, b) // nil
  0 OP    return nil
  0 OP  }) // nil
I200207 16:33:58.242418 14 kv/kvnemesis/watcher.go:131  watcher waiting for 1581093238.242324520,0
I200207 16:33:58.248634 1065 storage/replica_rangefeed.go:505  [n2,s2,r32/2:/{Table/50/"10…-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.248697 1065 storage/replica_rangefeed.go:506  [n2,s2,r32/2:/{Table/50/"10…-Max}] WIP MVCCCommitIntentOp /Table/50/"4493fa8c":1581093238.198008960,1 -> / 9a96d41b-8843-4c7a-8adc-bd274cb1b2a0
I200207 16:33:58.248771 1065 storage/replica_rangefeed.go:506  [n2,s2,r32/2:/{Table/50/"10…-Max}] WIP MVCCCommitIntentOp /Table/50/"b6c11210":1581093238.198008960,1 -> / 9a96d41b-8843-4c7a-8adc-bd274cb1b2a0
I200207 16:33:58.248845 1065 storage/replica_rangefeed.go:506  [n2,s2,r32/2:/{Table/50/"10…-Max}] WIP MVCCCommitIntentOp /Table/50/"efda1d8b":1581093238.198008960,1 -> / 9a96d41b-8843-4c7a-8adc-bd274cb1b2a0
I200207 16:33:58.249432 1396 storage/replica_rangefeed.go:505  [n3,s3,r32/3:/{Table/50/"10…-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.249506 1396 storage/replica_rangefeed.go:506  [n3,s3,r32/3:/{Table/50/"10…-Max}] WIP MVCCCommitIntentOp /Table/50/"4493fa8c":1581093238.198008960,1 -> / 9a96d41b-8843-4c7a-8adc-bd274cb1b2a0
I200207 16:33:58.249575 1396 storage/replica_rangefeed.go:506  [n3,s3,r32/3:/{Table/50/"10…-Max}] WIP MVCCCommitIntentOp /Table/50/"b6c11210":1581093238.198008960,1 -> / 9a96d41b-8843-4c7a-8adc-bd274cb1b2a0
I200207 16:33:58.249630 1396 storage/replica_rangefeed.go:506  [n3,s3,r32/3:/{Table/50/"10…-Max}] WIP MVCCCommitIntentOp /Table/50/"efda1d8b":1581093238.198008960,1 -> / 9a96d41b-8843-4c7a-8adc-bd274cb1b2a0
I200207 16:33:58.264113 1897 storage/replica_rangefeed.go:505  [n4,s4,r6/2:/Table/{SystemCon…-11}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.264185 1897 storage/replica_rangefeed.go:506  [n4,s4,r6/2:/Table/{SystemCon…-11}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.243025842,0 1581093238.243025842,0 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.264554 130 storage/replica_rangefeed.go:505  [n1,s1,r6/1:/Table/{SystemCon…-11}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.264639 130 storage/replica_rangefeed.go:506  [n1,s1,r6/1:/Table/{SystemCon…-11}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.243025842,0 1581093238.243025842,0 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.264942 1399 storage/replica_rangefeed.go:505  [n3,s3,r6/3:/Table/{SystemCon…-11}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.264977 1399 storage/replica_rangefeed.go:506  [n3,s3,r6/3:/Table/{SystemCon…-11}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.243025842,0 1581093238.243025842,0 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.273871 134 storage/replica_rangefeed.go:505  [n1,s1,r8/1:/Table/1{2-3}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.274691 1374 storage/replica_rangefeed.go:505  [n3,s3,r8/3:/Table/1{2-3}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.274776 1374 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.243025842,0 1581093238.243025842,0 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.274823 1374 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.243025842,0 1581093238.243025842,0 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.274861 1374 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.243025842,0 1581093238.243025842,0 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.274914 1374 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.243025842,0 1581093238.243025842,0 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.274949 1374 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.243025842,0 1581093238.243025842,0 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.279747 1069 storage/replica_rangefeed.go:505  [n2,s2,r8/2:/Table/1{2-3}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.279807 1069 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.243025842,0 1581093238.243025842,0 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.279838 1069 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.243025842,0 1581093238.243025842,0 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.279866 1069 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.243025842,0 1581093238.243025842,0 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.279893 1069 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.243025842,0 1581093238.243025842,0 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.279912 1069 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.243025842,0 1581093238.243025842,0 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.282489 134 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.243025842,0 1581093238.243025842,0 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.282536 134 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.243025842,0 1581093238.243025842,0 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.282568 134 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.243025842,0 1581093238.243025842,0 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.282595 134 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.243025842,0 1581093238.243025842,0 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.282621 134 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093238.243025842,0 1581093238.243025842,0 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.284263 1909 storage/replica_rangefeed.go:505  [n4,s4,r6/2:/Table/{SystemCon…-11}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.284337 1909 storage/replica_rangefeed.go:506  [n4,s4,r6/2:/Table/{SystemCon…-11}] WIP MVCCCommitIntentOp /Table/6/1/"kv.closed_timestamp.target_duration"/0:1581093238.243025842,0 -> / 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.284599 142 storage/replica_rangefeed.go:505  [n1,s1,r6/1:/Table/{SystemCon…-11}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.284647 142 storage/replica_rangefeed.go:506  [n1,s1,r6/1:/Table/{SystemCon…-11}] WIP MVCCCommitIntentOp /Table/6/1/"kv.closed_timestamp.target_duration"/0:1581093238.243025842,0 -> / 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.284956 1402 storage/replica_rangefeed.go:505  [n3,s3,r6/3:/Table/{SystemCon…-11}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.284991 1402 storage/replica_rangefeed.go:506  [n3,s3,r6/3:/Table/{SystemCon…-11}] WIP MVCCCommitIntentOp /Table/6/1/"kv.closed_timestamp.target_duration"/0:1581093238.243025842,0 -> / 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.285272 4931 sql/event_log.go:132  [n1,client=127.0.0.1:36458,user=root] Event: "set_cluster_setting", target: 0, info: {SettingName:kv.closed_timestamp.target_duration Value:00:00:00.001 User:root}
I200207 16:33:58.293997 135 storage/replica_rangefeed.go:505  [n1,s1,r8/1:/Table/1{2-3}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.294121 135 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.243026Z/"\xc1\x12\xfen+\xdcGx\x83\x96\xa1\x8f\x9e\x8b\x1eU"/0:1581093238.243025842,0 -> / 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.294210 135 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.243026Z/"\xc1\x12\xfen+\xdcGx\x83\x96\xa1\x8f\x9e\x8b\x1eU"/2/1:1581093238.243025842,0 -> / 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.294327 135 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.243026Z/"\xc1\x12\xfen+\xdcGx\x83\x96\xa1\x8f\x9e\x8b\x1eU"/3/1:1581093238.243025842,0 -> / 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.294448 135 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.243026Z/"\xc1\x12\xfen+\xdcGx\x83\x96\xa1\x8f\x9e\x8b\x1eU"/4/1:1581093238.243025842,0 -> / 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.294540 135 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.243026Z/"\xc1\x12\xfen+\xdcGx\x83\x96\xa1\x8f\x9e\x8b\x1eU"/5/1:1581093238.243025842,0 -> / 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.295089 1397 storage/replica_rangefeed.go:505  [n3,s3,r8/3:/Table/1{2-3}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.295138 1397 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.243026Z/"\xc1\x12\xfen+\xdcGx\x83\x96\xa1\x8f\x9e\x8b\x1eU"/0:1581093238.243025842,0 -> / 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.295163 1070 storage/replica_rangefeed.go:505  [n2,s2,r8/2:/Table/1{2-3}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.295181 1397 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.243026Z/"\xc1\x12\xfen+\xdcGx\x83\x96\xa1\x8f\x9e\x8b\x1eU"/2/1:1581093238.243025842,0 -> / 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.295199 1070 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.243026Z/"\xc1\x12\xfen+\xdcGx\x83\x96\xa1\x8f\x9e\x8b\x1eU"/0:1581093238.243025842,0 -> / 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.295212 1397 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.243026Z/"\xc1\x12\xfen+\xdcGx\x83\x96\xa1\x8f\x9e\x8b\x1eU"/3/1:1581093238.243025842,0 -> / 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.295224 1070 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.243026Z/"\xc1\x12\xfen+\xdcGx\x83\x96\xa1\x8f\x9e\x8b\x1eU"/2/1:1581093238.243025842,0 -> / 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.295255 1397 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.243026Z/"\xc1\x12\xfen+\xdcGx\x83\x96\xa1\x8f\x9e\x8b\x1eU"/4/1:1581093238.243025842,0 -> / 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.295290 1397 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.243026Z/"\xc1\x12\xfen+\xdcGx\x83\x96\xa1\x8f\x9e\x8b\x1eU"/5/1:1581093238.243025842,0 -> / 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.295262 1070 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.243026Z/"\xc1\x12\xfen+\xdcGx\x83\x96\xa1\x8f\x9e\x8b\x1eU"/3/1:1581093238.243025842,0 -> / 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.295393 1070 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.243026Z/"\xc1\x12\xfen+\xdcGx\x83\x96\xa1\x8f\x9e\x8b\x1eU"/4/1:1581093238.243025842,0 -> / 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.295421 1070 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCCommitIntentOp /Table/12/1/2020-02-07T16:33:58.243026Z/"\xc1\x12\xfen+\xdcGx\x83\x96\xa1\x8f\x9e\x8b\x1eU"/5/1:1581093238.243025842,0 -> / 34c5ecfa-9997-48a9-ac4f-c2dfcb8f86c4
I200207 16:33:58.565958 1911 storage/replica_rangefeed.go:505  [n4,s4,r2/3:/System/NodeLiveness{-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.566036 1911 storage/replica_rangefeed.go:506  [n4,s4,r2/3:/System/NodeLiveness{-Max}] WIP MVCCWriteValueOp  /System/NodeLiveness/1:1581093238.563088842,0 -> /
I200207 16:33:58.566779 1080 storage/replica_rangefeed.go:505  [n2,s2,r2/2:/System/NodeLiveness{-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.566820 1080 storage/replica_rangefeed.go:506  [n2,s2,r2/2:/System/NodeLiveness{-Max}] WIP MVCCWriteValueOp  /System/NodeLiveness/1:1581093238.563088842,0 -> /
I200207 16:33:58.566896 156 storage/replica_rangefeed.go:505  [n1,s1,r2/1:/System/NodeLiveness{-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.566942 156 storage/replica_rangefeed.go:506  [n1,s1,r2/1:/System/NodeLiveness{-Max}] WIP MVCCWriteValueOp  /System/NodeLiveness/1:1581093238.563088842,0 -> /
I200207 16:33:58.984882 1892 storage/replica_rangefeed.go:505  [n4,s4,r2/3:/System/NodeLiveness{-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.995881 1892 storage/replica_rangefeed.go:506  [n4,s4,r2/3:/System/NodeLiveness{-Max}] WIP MVCCWriteValueOp  /System/NodeLiveness/2:1581093238.980880117,2 -> /
I200207 16:33:58.985791 77 storage/replica_rangefeed.go:505  [n1,s1,r2/1:/System/NodeLiveness{-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.996220 77 storage/replica_rangefeed.go:506  [n1,s1,r2/1:/System/NodeLiveness{-Max}] WIP MVCCWriteValueOp  /System/NodeLiveness/2:1581093238.980880117,2 -> /
I200207 16:33:58.986076 1111 storage/replica_rangefeed.go:505  [n2,s2,r2/2:/System/NodeLiveness{-Max}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:58.996335 1111 storage/replica_rangefeed.go:506  [n2,s2,r2/2:/System/NodeLiveness{-Max}] WIP MVCCWriteValueOp  /System/NodeLiveness/2:1581093238.980880117,2 -> /
I200207 16:33:59.039659 117 storage/rangefeed/processor.go:443  WIP MVCCWriteIntentOp /Table/50/"c8082902" 1581093238.142249297,1 1581093238.139260809,0 1cc91982-dee8-4d12-89b9-f885de4fbc1d
I200207 16:33:59.039710 117 storage/rangefeed/processor.go:443  WIP MVCCUpdateIntentOp 1581093238.142249297,1 1cc91982-dee8-4d12-89b9-f885de4fbc1d
I200207 16:33:59.039819 117 storage/replica_rangefeed.go:564  WIP handleLogicalOpLogRaftMuLocked /Table/50/"c8082902":1581093238.142249297,1 -> /BYTES/v-8 
I200207 16:33:59.039837 117 storage/rangefeed/processor.go:443  WIP MVCCCommitIntentOp /Table/50/"c8082902":1581093238.142249297,1 -> /BYTES/v-8 1cc91982-dee8-4d12-89b9-f885de4fbc1d
I200207 16:33:59.039875 117 storage/replica_rangefeed.go:564  WIP handleLogicalOpLogRaftMuLocked /Table/50/"b6c11210":1581093238.168988514,0 -> /BYTES/v-10 
I200207 16:33:59.039889 117 storage/rangefeed/processor.go:443  WIP MVCCWriteValueOp  /Table/50/"b6c11210":1581093238.168988514,0 -> /BYTES/v-10
I200207 16:33:59.040419 5395 storage/rangefeed/processor.go:603  [n1,s1,r30/1:/Table/50/"{06af801…-105bb8e…},rangefeed] publishValue /Table/50/"c8082902":1581093238.142249297,1 -> /BYTES/v-8
I200207 16:33:59.040452 5395 storage/rangefeed/processor.go:603  [n1,s1,r30/1:/Table/50/"{06af801…-105bb8e…},rangefeed] publishValue /Table/50/"b6c11210":1581093238.168988514,0 -> /BYTES/v-10
I200207 16:33:59.040617 5139 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"c8082902" 1581093238.142249297,1 -> /BYTES/v-8 (prev /BYTES/v-9)
I200207 16:33:59.040643 5139 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"b6c11210" 1581093238.168988514,0 -> /BYTES/v-10 (prev /)
I200207 16:33:59.040665 5395 storage/rangefeed/registry.go:535  remove /Table/5{0/"06af801b"-1} 1581093238.059657892,0 retry rangefeed (REASON_RANGE_SPLIT)
I200207 16:33:59.040709 5393 kv/dist_sender_rangefeed.go:281  [n1] RangeFeedError: retry rangefeed (REASON_RANGE_SPLIT)
I200207 16:33:59.040732 5393 kv/dist_sender_rangefeed.go:153  [n1] RangeFeed dial /Table/5{0/"06af801b"-1} 1581093238.059657892,0 returned 1581093238.059657892,0 retry rangefeed (REASON_RANGE_SPLIT)
I200207 16:33:59.040795 6873 kv/dist_sender_rangefeed.go:151  [n1] RangeFeed dial /Table/5{0/"105bb8ed"-1} 1581093238.059657892,0
I200207 16:33:59.041115 6875 storage/rangefeed/registry.go:451  register /Table/5{0/"105bb8ed"-1} 1581093238.059657892,0
I200207 16:33:59.041561 6872 kv/dist_sender_rangefeed.go:151  [n1] RangeFeed dial /Table/50/"{06af801b"-105bb8ed"} 1581093238.059657892,0
I200207 16:33:59.041910 6842 storage/rangefeed/registry.go:451  register /Table/50/"{06af801b"-105bb8ed"} 1581093238.059657892,0
I200207 16:33:59.044176 5139 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"25567fd6" 1581093238.070282014,0 -> /BYTES/v-0 (prev /)
I200207 16:33:59.044215 5139 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"25567fd6" 1581093238.092918251,0 -> /BYTES/v-1 (prev /BYTES/v-0)
I200207 16:33:59.044242 5139 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"25567fd6" 1581093238.103499287,0 -> /BYTES/v-2 (prev /BYTES/v-1)
I200207 16:33:59.044290 5139 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"472df5f4" 1581093238.134363957,0 -> /BYTES/v-6 (prev /)
I200207 16:33:59.044336 5139 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"b6c11210" 1581093238.168988514,0 -> /BYTES/v-10 (prev /)
I200207 16:33:59.044362 5139 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"c8082902" 1581093238.142249297,0 -> /BYTES/v-9 (prev /)
I200207 16:33:59.044381 5139 kv/kvnemesis/watcher.go:162  rangefeed Put /Table/50/"c8082902" 1581093238.142249297,1 -> /BYTES/v-8 (prev /BYTES/v-9)
I200207 16:33:59.044418 5139 kv/kvnemesis/watcher.go:197  watcher reached frontier 1581093239.037443588,0 lagging by 6.962905ms
I200207 16:33:59.044429 5139 kv/kvnemesis/watcher.go:203  watcher notifying 1581093238.242324520,0
I200207 16:33:59.062941 1898 storage/replica_rangefeed.go:505  [n4,s4,r6/2:/Table/{SystemCon…-11}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:59.062991 1898 storage/replica_rangefeed.go:506  [n4,s4,r6/2:/Table/{SystemCon…-11}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093239.057129085,2 1581093239.044924979,0 6b13bc6c-c5b9-4104-9242-eda790ad2386
I200207 16:33:59.063500 1384 storage/replica_rangefeed.go:505  [n3,s3,r6/3:/Table/{SystemCon…-11}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:59.063679 1384 storage/replica_rangefeed.go:506  [n3,s3,r6/3:/Table/{SystemCon…-11}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093239.057129085,2 1581093239.044924979,0 6b13bc6c-c5b9-4104-9242-eda790ad2386
I200207 16:33:59.063558 129 storage/replica_rangefeed.go:505  [n1,s1,r6/1:/Table/{SystemCon…-11}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:59.063956 129 storage/replica_rangefeed.go:506  [n1,s1,r6/1:/Table/{SystemCon…-11}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093239.057129085,2 1581093239.044924979,0 6b13bc6c-c5b9-4104-9242-eda790ad2386
I200207 16:33:59.072295 123 storage/replica_rangefeed.go:505  [n1,s1,r8/1:/Table/1{2-3}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:59.072340 123 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093239.069962797,2 1581093239.044924979,0 6b13bc6c-c5b9-4104-9242-eda790ad2386
I200207 16:33:59.072370 123 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093239.069962797,2 1581093239.044924979,0 6b13bc6c-c5b9-4104-9242-eda790ad2386
I200207 16:33:59.072394 123 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093239.069962797,2 1581093239.044924979,0 6b13bc6c-c5b9-4104-9242-eda790ad2386
I200207 16:33:59.072415 123 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093239.069962797,2 1581093239.044924979,0 6b13bc6c-c5b9-4104-9242-eda790ad2386
I200207 16:33:59.072437 123 storage/replica_rangefeed.go:506  [n1,s1,r8/1:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093239.069962797,2 1581093239.044924979,0 6b13bc6c-c5b9-4104-9242-eda790ad2386
I200207 16:33:59.073280 1381 storage/replica_rangefeed.go:505  [n3,s3,r8/3:/Table/1{2-3}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:59.073341 1381 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093239.069962797,2 1581093239.044924979,0 6b13bc6c-c5b9-4104-9242-eda790ad2386
I200207 16:33:59.073393 1381 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093239.069962797,2 1581093239.044924979,0 6b13bc6c-c5b9-4104-9242-eda790ad2386
I200207 16:33:59.073430 1381 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093239.069962797,2 1581093239.044924979,0 6b13bc6c-c5b9-4104-9242-eda790ad2386
I200207 16:33:59.073451 1381 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093239.069962797,2 1581093239.044924979,0 6b13bc6c-c5b9-4104-9242-eda790ad2386
I200207 16:33:59.073469 1381 storage/replica_rangefeed.go:506  [n3,s3,r8/3:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093239.069962797,2 1581093239.044924979,0 6b13bc6c-c5b9-4104-9242-eda790ad2386
I200207 16:33:59.074802 1113 storage/replica_rangefeed.go:505  [n2,s2,r8/2:/Table/1{2-3}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:59.075153 1113 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093239.069962797,2 1581093239.044924979,0 6b13bc6c-c5b9-4104-9242-eda790ad2386
I200207 16:33:59.075198 1113 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093239.069962797,2 1581093239.044924979,0 6b13bc6c-c5b9-4104-9242-eda790ad2386
I200207 16:33:59.075240 1113 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093239.069962797,2 1581093239.044924979,0 6b13bc6c-c5b9-4104-9242-eda790ad2386
I200207 16:33:59.075298 1113 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093239.069962797,2 1581093239.044924979,0 6b13bc6c-c5b9-4104-9242-eda790ad2386
I200207 16:33:59.075680 1113 storage/replica_rangefeed.go:506  [n2,s2,r8/2:/Table/1{2-3}] WIP MVCCWriteIntentOp /Table/SystemConfigSpan/Start 1581093239.069962797,2 1581093239.044924979,0 6b13bc6c-c5b9-4104-9242-eda790ad2386
I200207 16:33:59.081602 1905 storage/replica_rangefeed.go:505  [n4,s4,r6/2:/Table/{SystemCon…-11}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:59.081677 1905 storage/replica_rangefeed.go:506  [n4,s4,r6/2:/Table/{SystemCon…-11}] WIP MVCCCommitIntentOp /Table/6/1/"kv.closed_timestamp.target_duration"/0:1581093239.069962797,2 -> / 6b13bc6c-c5b9-4104-9242-eda790ad2386
I200207 16:33:59.082553 1386 storage/replica_rangefeed.go:505  [n3,s3,r6/3:/Table/{SystemCon…-11}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:59.083404 1386 storage/replica_rangefeed.go:506  [n3,s3,r6/3:/Table/{SystemCon…-11}] WIP MVCCCommitIntentOp /Table/6/1/"kv.closed_timestamp.target_duration"/0:1581093239.069962797,2 -> / 6b13bc6c-c5b9-4104-9242-eda790ad2386
I200207 16:33:59.082581 130 storage/replica_rangefeed.go:505  [n1,s1,r6/1:/Table/{SystemCon…-11}] handleLogicalOpLogRaftMuLocked no processor
I200207 16:33:59.083608 130 storage/replica_rangefeed.go:506  [n1,s1,r6/1:/Table/{SystemCon…-11}] WIP MVCCCommitIntentOp /Table/6/1/"kv.closed_timestamp.target_duration"/0:1581093239.069962797,2 -> / 6b13bc6c-c5b9-4104-9242-eda790ad2386
I200207 16:33:59.082703 4931 sql/event_log.go:132  [n1,client=127.0.0.1:36458,user=root] Event: "set_cluster_setting", target: 0, info: {SettingName:kv.closed_timestamp.target_duration Value:DEFAULT User:root}
I200207 16:33:59.086189 6842 storage/rangefeed/registry.go:500  remove /Table/50/"{06af801b"-105bb8ed"} 1581093238.059657892,0
I200207 16:33:59.086277 6875 storage/rangefeed/registry.go:500  remove /Table/5{0/"105bb8ed"-1} 1581093238.059657892,0
I200207 16:33:59.086332 5732 storage/rangefeed/registry.go:500  remove /Table/50{-/"06af801b"} 1581093238.059657892,0
I200207 16:33:59.086398 6872 kv/dist_sender_rangefeed.go:281  [n1] RangeFeedError: context canceled
I200207 16:33:59.086436 6872 kv/dist_sender_rangefeed.go:153  [n1] RangeFeed dial /Table/50/"{06af801b"-105bb8ed"} 1581093238.059657892,0 returned 1581093239.076685918,0 context canceled
I200207 16:33:59.086489 6873 kv/dist_sender_rangefeed.go:153  [n1] RangeFeed dial /Table/5{0/"105bb8ed"-1} 1581093238.059657892,0 returned 1581093239.037443588,0 context canceled
I200207 16:33:59.086529 5392 kv/dist_sender_rangefeed.go:281  [n1] RangeFeedError: context canceled
I200207 16:33:59.086578 5392 kv/dist_sender_rangefeed.go:153  [n1] RangeFeed dial /Table/50{-/"06af801b"} 1581093238.059657892,0 returned 1581093239.076685918,0 context canceled
I200207 16:33:59.087133 14 kv/kvnemesis/kvnemesis.go:86  reproduction steps:
g := ctxgroup.WithContext(ctx)
g.GoCtx(func(ctx context.Context) error {
  db1.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    txn.Get(ctx, /Table/50/"917c48c6") // (nil, nil)
    return errors.New("rollback")
  }) // rollback
  db3.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    txn.Get(ctx, /Table/50/"231d22e7") // (nil, nil)
    txn.Get(ctx, /Table/50/"25567fd6") // ("v-2", nil)
    txn.Put(ctx, /Table/50/"25567fd6", v-5) // nil
    return errors.New("rollback")
  }) // rollback
  db0.Put(ctx, /Table/50/"472df5f4", v-6) // nil
  db1.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    txn.Get(ctx, /Table/50/"c2f94d6d") // (nil, nil)
    txn.Put(ctx, /Table/50/"c8082902", v-7) // nil
    txn.Put(ctx, /Table/50/"c8082902", v-8) // nil
    return nil
  }) // nil
  db1.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    return nil
  }) // nil
  db2.Put(ctx, /Table/50/"b6c11210", v-10) // nil
  db3.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    txn.Get(ctx, /Table/50/"472df5f4") // ("v-6", nil)
    return errors.New("rollback")
  }) // rollback
  db3.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    txn.Put(ctx, /Table/50/"4493fa8c", v-13) // nil
    b := &Batch{}
    b.Put(ctx, /Table/50/"efda1d8b", v-14) // nil
    b.Put(ctx, /Table/50/"b6c11210", v-15) // nil
    txn.CommitInBatch(ctx, b) // nil
    return nil
  }) // nil
  return nil
})
g.GoCtx(func(ctx context.Context) error {
  db2.AdminSplit(ctx, /Table/50/"06af801b") // nil
  return nil
})
g.GoCtx(func(ctx context.Context) error {
  db3.AdminSplit(ctx, /Table/50/"105bb8ed") // nil
  return nil
})
g.GoCtx(func(ctx context.Context) error {
  db0.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    return nil
  }) // nil
  db1.AdminSplit(ctx, /Table/50/"06af801b") // nil
  db0.AdminSplit(ctx, /Table/50/"105bb8ed") // nil
  db0.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    txn.Get(ctx, /Table/50/"c2f94d6d") // (nil, nil)
    b := &Batch{}
    txn.CommitInBatch(ctx, b) // nil
    return nil
  }) // nil
  db1.Put(ctx, /Table/50/"317ffd5d", v-11) // nil
  db0.AdminMerge(ctx, /Table/50/"105bb8ed") // storage/replica_command.go:776: merge failed: cannot merge final range
  db1.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    txn.Put(ctx, /Table/50/"b6c11210", v-16) // nil
    txn.Put(ctx, /Table/50/"5417c5c4", v-17) // nil
    txn.Put(ctx, /Table/50/"a577349f", v-18) // nil
    return nil
  }) // nil
  return nil
})
g.GoCtx(func(ctx context.Context) error {
  db0.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    txn.Put(ctx, /Table/50/"25567fd6", v-0) // nil
    txn.Get(ctx, /Table/50/"dd91893a") // (nil, nil)
    txn.Get(ctx, /Table/50/"00fbca0e") // (nil, nil)
    b := &Batch{}
    b.Get(ctx, /Table/50/"0d087981") // (nil, nil)
    txn.CommitInBatch(ctx, b) // nil
    return nil
  }) // nil
  db2.Get(ctx, /Table/50/"184059ab") // (nil, nil)
  db3.Get(ctx, /Table/50/"4915918b") // (nil, nil)
  db0.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    txn.Get(ctx, /Table/50/"32825014") // (nil, nil)
    txn.Get(ctx, /Table/50/"f16eda4e") // (nil, nil)
    txn.Get(ctx, /Table/50/"e9b876b3") // (nil, nil)
    b := &Batch{}
    b.Get(ctx, /Table/50/"25567fd6") // ("v-0", nil)
    b.Get(ctx, /Table/50/"ead4a2f3") // (nil, nil)
    b.Put(ctx, /Table/50/"25567fd6", v-1) // nil
    txn.CommitInBatch(ctx, b) // nil
    return nil
  }) // nil
  db1.AdminMerge(ctx, /Table/50/"105bb8ed") // storage/replica_command.go:776: merge failed: cannot merge final range
  db2.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    txn.Put(ctx, /Table/50/"25567fd6", v-2) // nil
    return nil
  }) // nil
  db3.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    txn.Get(ctx, /Table/50/"25567fd6") // ("v-2", nil)
    txn.Get(ctx, /Table/50/"1a68b653") // (nil, nil)
    return nil
  }) // nil
  db0.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    return errors.New("rollback")
  }) // rollback
  db1.AdminMerge(ctx, /Table/50/"06af801b") // storage/replica_command.go:776: merge failed: cannot merge final range
  db2.Txn(ctx, func(ctx context.Context, txn *client.Txn) error {
    txn.Put(ctx, /Table/50/"681f3c98", v-3) // nil
    txn.Put(ctx, /Table/50/"c2f94d6d", v-4) // nil
    txn.Get(ctx, /Table/50/"681f3c98") // ("v-3", nil)
    return errors.New("rollback")
  }) // rollback
  db2.Put(ctx, /Table/50/"c8082902", v-9) // nil
  db3.Get(ctx, /Table/50/"c8082902") // ("v-8", nil)
  db2.Put(ctx, /Table/50/"b6c11210", v-12) // nil
  return nil
})
g.Wait()
I200207 16:33:59.087237 14 kv/kvnemesis/kvnemesis.go:87  kvs (recorded from rangefeed):
  /Table/50/"25567fd6" 1581093238.103499287,0 -> /BYTES/v-2
  /Table/50/"25567fd6" 1581093238.092918251,0 -> /BYTES/v-1
  /Table/50/"25567fd6" 1581093238.070282014,0 -> /BYTES/v-0
  /Table/50/"472df5f4" 1581093238.134363957,0 -> /BYTES/v-6
  /Table/50/"b6c11210" 1581093238.168988514,0 -> /BYTES/v-10
  /Table/50/"c8082902" 1581093238.142249297,1 -> /BYTES/v-8
  /Table/50/"c8082902" 1581093238.142249297,0 -> /BYTES/v-9
I200207 16:33:59.088980 14 kv/kvnemesis/kvnemesis.go:98  kvs (scan of latest values according to crdb):
  /Table/50/"25567fd6" 1581093238.103499287,0 -> /BYTES/v-2
  /Table/50/"317ffd5d" 1581093238.188813359,0 -> /BYTES/v-11
  /Table/50/"4493fa8c" 1581093238.198008960,1 -> /BYTES/v-13
  /Table/50/"472df5f4" 1581093238.134363957,0 -> /BYTES/v-6
  /Table/50/"5417c5c4" 1581093238.198008960,0 -> /BYTES/v-17
  /Table/50/"a577349f" 1581093238.198008960,0 -> /BYTES/v-18
  /Table/50/"b6c11210" 1581093238.198008960,1 -> /BYTES/v-15
  /Table/50/"c8082902" 1581093238.142249297,1 -> /BYTES/v-8
  /Table/50/"efda1d8b" 1581093238.198008960,1 -> /BYTES/v-14
I200207 16:33:59.089354 14 util/stop/stopper.go:539  quiescing
I200207 16:33:59.089414 6939 util/stop/stopper.go:539  quiescing
I200207 16:33:59.089558 6936 util/stop/stopper.go:539  quiescing
I200207 16:33:59.089683 6937 util/stop/stopper.go:539  quiescing
W200207 16:33:59.089782 6830 storage/intentresolver/intent_resolver.go:614  failed to cleanup transaction intents: failed to resolve intents: node unavailable; try another peer
I200207 16:33:59.089815 6938 util/stop/stopper.go:539  quiescing
W200207 16:33:59.089974 1359 storage/raft_transport.go:637  [n1] while processing outgoing Raft queue to node 2: rpc error: code = Canceled desc = grpc: the client connection is closing:
I200207 16:33:59.093735 1934 storage/queue.go:524  [n4,s4] rate limited in gossip update (split): node unavailable; try another peer
I200207 16:33:59.093815 1934 storage/queue.go:524  [n4,s4] rate limited in gossip update (merge): node unavailable; try another peer
W200207 16:33:59.094114 1757 gossip/gossip.go:1518  [n4] no incoming or outgoing connections
W200207 16:33:59.094187 2027 storage/raft_transport.go:637  [n4] while processing outgoing Raft queue to node 1: rpc error: code = Canceled desc = grpc: the client connection is closing:
W200207 16:33:59.094248 2129 storage/raft_transport.go:637  [n4] while processing outgoing Raft queue to node 1: rpc error: code = Canceled desc = grpc: the client connection is closing:
W200207 16:33:59.094234 2190 storage/raft_transport.go:637  [n4] while processing outgoing Raft queue to node 3: rpc error: code = Canceled desc = grpc: the client connection is closing:
W200207 16:33:59.094357 2377 storage/raft_transport.go:637  [n4] while processing outgoing Raft queue to node 2: rpc error: code = Canceled desc = grpc: the client connection is closing:
W200207 16:33:59.094561 1777 storage/raft_transport.go:637  [n1] while processing outgoing Raft queue to node 3: rpc error: code = Canceled desc = grpc: the client connection is closing:
W200207 16:33:59.094567 1472 storage/raft_transport.go:637  [n1] while processing outgoing Raft queue to node 3: rpc error: code = Canceled desc = grpc: the client connection is closing:
W200207 16:33:59.094654 2041 storage/raft_transport.go:637  [n1] while processing outgoing Raft queue to node 4: rpc error: code = Canceled desc = grpc: the client connection is closing:
W200207 16:33:59.094708 2615 storage/raft_transport.go:637  [n1] while processing outgoing Raft queue to node 4: rpc error: code = Canceled desc = grpc: the client connection is closing:
W200207 16:33:59.094750 1776 storage/raft_transport.go:637  [n1] while processing outgoing Raft queue to node 2: rpc error: code = Canceled desc = grpc: the client connection is closing:
W200207 16:33:59.094993 1339 storage/raft_transport.go:637  [n2] while processing outgoing Raft queue to node 1: EOF:
W200207 16:33:59.095027 1496 storage/raft_transport.go:637  [n2] while processing outgoing Raft queue to node 1: EOF:
W200207 16:33:59.095073 1610 storage/raft_transport.go:637  [n3] while processing outgoing Raft queue to node 1: EOF:
W200207 16:33:59.095086 1272 gossip/gossip.go:1518  [n3] no incoming or outgoing connections
W200207 16:33:59.095180 2256 storage/raft_transport.go:637  [n3] while processing outgoing Raft queue to node 4: EOF:
W200207 16:33:59.095190 1613 storage/raft_transport.go:637  [n3] while processing outgoing Raft queue to node 1: EOF:
W200207 16:33:59.095254 2371 storage/raft_transport.go:637  [n2] while processing outgoing Raft queue to node 4: EOF:
W200207 16:33:59.095422 2431 storage/raft_transport.go:637  [n2] while processing outgoing Raft queue to node 4: EOF:
W200207 16:33:59.095574 2478 storage/raft_transport.go:637  [n4] while processing outgoing Raft queue to node 3: rpc error: code = Canceled desc = grpc: the client connection is closing:
W200207 16:33:59.097872 2640 storage/raft_transport.go:637  [n4] while processing outgoing Raft queue to node 2: rpc error: code = Canceled desc = grpc: the client connection is closing:
W200207 16:33:59.098258 911 gossip/gossip.go:1518  [n2] no incoming or outgoing connections
W200207 16:33:59.098542 5781 storage/intentresolver/intent_resolver.go:820  failed to gc transaction record: could not GC completed transaction anchored at /Table/50/"4493fa8c": node unavailable; try another peer
W200207 16:33:59.099081 2294 storage/raft_transport.go:637  [n2] while processing outgoing Raft queue to node 3: rpc error: code = Canceled desc = grpc: the client connection is closing:
W200207 16:33:59.099194 1767 storage/raft_transport.go:637  [n2] while processing outgoing Raft queue to node 3: rpc error: code = Canceled desc = grpc: the client connection is closing:
W200207 16:33:59.099832 3551 storage/raft_transport.go:637  [n3] while processing outgoing Raft queue to node 2: EOF:
W200207 16:33:59.100238 2013 storage/raft_transport.go:637  [n3] while processing outgoing Raft queue to node 2: EOF:
W200207 16:33:59.100375 2589 storage/raft_transport.go:637  [n3] while processing outgoing Raft queue to node 4: EOF:
W200207 16:33:59.102548 1422 storage/store.go:1538  [n3,s3,r1/3:/{Min-System/NodeL…}] could not gossip first range descriptor: node unavailable; try another peer
--- FAIL: TestKVNemesisMultiNode (5.29s)
    kvnemesis_test.go:79: failure:
        error with attached stack trace:
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.(*validator).checkCommittedTxn
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/validator.go:272
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.(*validator).checkAtomic
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/validator.go:231
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.(*validator).processOp
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/validator.go:119
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.Validate
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/validator.go:69
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.RunNemesis
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/kvnemesis.go:83
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.TestKVNemesisMultiNode
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/kvnemesis_test.go:75
            testing.tRunner
            	/usr/local/go/src/testing/testing.go:909
            runtime.goexit
            	/usr/local/go/src/runtime/asm_amd64.s:1357
          - error with embedded safe details: %s: %s
            -- arg 1: 
            -- arg 2: /Table/50/"317ffd5d":missing->v-11
          - committed put missing write: /Table/50/"317ffd5d":missing->v-11
    kvnemesis_test.go:79: failure:
        error with attached stack trace:
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.(*validator).checkCommittedTxn
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/validator.go:272
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.(*validator).checkAtomic
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/validator.go:231
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.(*validator).processOp
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/validator.go:119
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.Validate
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/validator.go:69
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.RunNemesis
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/kvnemesis.go:83
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.TestKVNemesisMultiNode
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/kvnemesis_test.go:75
            testing.tRunner
            	/usr/local/go/src/testing/testing.go:909
            runtime.goexit
            	/usr/local/go/src/runtime/asm_amd64.s:1357
          - error with embedded safe details: %s: %s
            -- arg 1: 
            -- arg 2: /Table/50/"b6c11210":missing->v-12
          - committed put missing write: /Table/50/"b6c11210":missing->v-12
    kvnemesis_test.go:79: failure:
        error with attached stack trace:
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.(*validator).checkCommittedTxn
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/validator.go:272
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.(*validator).checkAtomic
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/validator.go:231
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.(*validator).processOp
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/validator.go:217
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.Validate
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/validator.go:69
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.RunNemesis
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/kvnemesis.go:83
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.TestKVNemesisMultiNode
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/kvnemesis_test.go:75
            testing.tRunner
            	/usr/local/go/src/testing/testing.go:909
            runtime.goexit
            	/usr/local/go/src/runtime/asm_amd64.s:1357
          - error with embedded safe details: %s: %s
            -- arg 1: 
            -- arg 2: /Table/50/"5417c5c4":missing->v-17 /Table/50/"a577349f":missing->v-18 /Table/50/"b6c11210":missing->v-16
          - committed txn missing write: /Table/50/"5417c5c4":missing->v-17 /Table/50/"a577349f":missing->v-18 /Table/50/"b6c11210":missing->v-16
    kvnemesis_test.go:79: failure:
        error with attached stack trace:
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.(*validator).checkCommittedTxn
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/validator.go:272
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.(*validator).checkAtomic
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/validator.go:231
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.(*validator).processOp
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/validator.go:217
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.Validate
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/validator.go:69
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.RunNemesis
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/kvnemesis.go:83
            github.com/cockroachdb/cockroach/pkg/kv/kvnemesis.TestKVNemesisMultiNode
            	/go/src/github.com/cockroachdb/cockroach/pkg/kv/kvnemesis/kvnemesis_test.go:75
            testing.tRunner
            	/usr/local/go/src/testing/testing.go:909
            runtime.goexit
            	/usr/local/go/src/runtime/asm_amd64.s:1357
          - error with embedded safe details: %s: %s
            -- arg 1: 
            -- arg 2: /Table/50/"4493fa8c":missing->v-13 /Table/50/"b6c11210":missing->v-15 /Table/50/"efda1d8b":missing->v-14
          - committed txn missing write: /Table/50/"4493fa8c":missing->v-13 /Table/50/"b6c11210":missing->v-15 /Table/50/"efda1d8b":missing->v-14
FAIL

ERROR: exit status 1


context canceled

Error:  exit status 1


44518 runs completed, 0 failures, over 43m8s

context canceled

FAIL

make: *** [roachprod-stress] Error 1

@danhhz danhhz mentioned this pull request Feb 7, 2020
Closed
@danhhz danhhz force-pushed the kvnemesis_multinode branch from cb47fcd to e43a9eb Compare February 7, 2020 21:18
@danhhz danhhz force-pushed the kvnemesis_multinode branch from e43a9eb to 31b2f5d Compare February 21, 2020 18:16
@danhhz
kvnemesis: add a multi-node variant of the test
ecfb9a3 
To make this more interesting, at the same time, add support for the
ChangeReplicas command.

Release note: None
@danhhz danhhz force-pushed the kvnemesis_multinode branch from 31b2f5d to ecfb9a3 Compare February 21, 2020 19:25
@danhhz
Copy link
Contributor Author

danhhz commented Feb 21, 2020

As discussed offline, disabling splits and merges in the multinode test until #44878 is fixed. I stressed this on 25 nodes for over an hour without a flake. TFTR!

193067 runs completed, 0 failures, over 1h10m30s

bors r=nvanbenschoten

@craig
Copy link
Contributor

craig bot commented Feb 21, 2020

Build failed

@danhhz
Copy link
Contributor Author

danhhz commented Feb 21, 2020

flake is #44985

bors r=nvanbenschoten

@craig
Copy link
Contributor

craig bot commented Feb 21, 2020

Build succeeded

@craig craig bot merged commit 2627ba3 into cockroachdb:master Feb 21, 2020
@danhhz danhhz deleted the kvnemesis_multinode branch February 21, 2020 21:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nvanbenschoten nvanbenschoten nvanbenschoten approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@danhhz @cockroach-teamcity @nvanbenschoten