Simple StoreFinder and new capacity gossiping #187
Conversation
This should be sufficient for the beta stages. When more stores are added, then switch to gossiping capacity through groups, so that only the top N will be gossiped. This will lower the gossip and storage load on all machines.
Store now tracks gossip keys of store capacity, and scans the capacity gossip for attribute matches when they need to make allocation decisions. A simple, linear implementation that has a straightforward way to implement garbage collection of capacity keys, but faster methods can be employed later. It is likely sufficient as long as it is sufficient for all stores to gossip capacity, but will need to be reevaluated when capacity is gossiped through groups.
| @@ -0,0 +1,88 @@ | |||
| // Copyright 2014 The Cockroach Authors. | |||
There was a problem hiding this comment.
All of this could be done in storage/store.go, I just felt that file was getting pretty long. Willing to change back at request!
There was a problem hiding this comment.
Why not combine the code in this file which adds methods to the Store struct into a StoreFinder struct and then embed that in Store? I dislike having methods defined in separate files for the same struct.
Currently "StoreFinder" is a func type. We could consider just renaming that FindStoreFunc
There was a problem hiding this comment.
Or don't embed it...just keep it as a constituent object. That's fine too.
There was a problem hiding this comment.
Are you suggesting making the below methods defined on the finderContext or new StoreFinder struct instead of methods defined directly on Store?
There was a problem hiding this comment.
Yeah, my instinct would be to create a StoreFinder struct and add these methods to it. Then just create an instance of StoreFinder when creating a new Store. Embedding might make sense too, but I don't think it matters in this context.
There was a problem hiding this comment.
Alright, I tried the embedding method. Looks reasonable -- PTAL
|
LGTM |
Changes integer base on capacity gossip keys and adds TODO for possible speedup.
In response to Spencer's PR comments, rather than increasing the length of storage/store by defining store finder methods on Store itself.
|
LGTM |
Simple StoreFinder and new capacity gossiping
cluster-ui: sessions redux functionality
…h-assert-quorum Test: Replace t.error/fatal with assert/request in [/quorum]
As discussed, capacity gossip is now gossiped per store, not by group, at least for the beta release.
Store now tracks gossip keys of store capacity, and scans the capacity gossip
for attribute matches when they need to make allocation decisions. A simple,
linear implementation that has a straightforward way to implement garbage
collection of capacity keys, but faster methods can be employed later. It
is might be sufficient as long as it is sufficient for all stores to gossip
capacity, but will need to be reevaluated when capacity is gossiped through
groups