-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
WIP on KV projection pushdown #92294
Conversation
0615019
to
f100b72
Compare
f100b72
to
6093c36
Compare
pkg/roachpb/api.proto
Outdated
// This "any type" field is a serialized descpb.IndexFetchSpec that can be | ||
// used to initialize a columnar scan for MVCC. We use this generic type to | ||
// resolve some painful circular dependency issues. | ||
// TODO: better comment. | ||
google.protobuf.Any index_fetch_spec = 29; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Have you considered making this bytes or rather string? I think that a lot of the time the index_fetch_spec is going to be the same. I imagine nobody is every going to use the protobuf directly but rather will build some other sort of decoder struct from it. I imagine we'll want to cache those things. The string itself can be a hash key into the cache. I think it'd be nice to avoid decoding and parsing the specs every time. Ideally we'd define some type like type EncodedFetchSpec string
and use a cast type.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
No, I haven't really considered this, but thanks for taking a look.
I'm currently trying to make the prototype work without too much thought to this kind of questions - at the moment, I'm struggling with all the intricacies of advancing the iterator in getAndAdvance
, especially in the presence of multiple versions of the same keys. That code is hard to comprehend :/
50a81a3
to
5a48615
Compare
Multi-tenant is still slower than single tenant which is because other fetchers (e.g. index and lookup joins) don't yet use this:
|
969c3d1
to
814f2b1
Compare
77431e5
to
3977214
Compare
5337deb
to
6bab717
Compare
This commit introduces `NextKVer` interface which only consists of a single method for returning KVs from somewhere. The interface is then used by the `cFetcher`. Currently, there is only one implementation (`row.KVFetcher`), but soon we'll introduce another one to push the projections into the KV layer. This required moving `MVCCDecodingStrategy` type into `storage` package as well. Note that I chose to not introduce other methods that `cFetcher` is calling on `row.KVFetcher` since they won't be necessary for the pushdown work. Also since (at least currently) the `cFetcher` doesn't use the `spanID` value, we omit it from the interface method signature (it's only used by the lookup joins which aren't yet vectorized). Additionally, this commit moves a single comment to the appropriate place. Release note: None
This reverts commit 134f90b7ce578f63a915e08e757d64f7a255e128.
This reverts commit a2d9008fea22b80f126a54da3b06614369d5fba6.
This reverts commit cf810a18c3f1e6852818125b521fd299e580db5a.
This will allow us to import the index fetch spec into `roachpb` which might be needed (TBD) for the KV projection pushdown work. This required changing the custom cast types used in the spec from type aliases defined in `descpb` to one level lower ones from `catid`. Epic: None Release note: None
…etchpb into roachpb
026a4a8
to
a0693f8
Compare
Epic: CRDB-14837
Informs: #82323
Informs: #87610