-
Notifications
You must be signed in to change notification settings - Fork 3.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
ccl/sqlproxyccl: add postgres interceptors for message forwarding #76006
Conversation
02fb0cf
to
e02a13d
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do you know if the FrontendInterceptor could also be used to intercept CancelRequests? (not needed now, but wondering for the future)
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @andy-kimball, @jaylim-crl, and @jeffswenson)
pkg/ccl/sqlproxyccl/interceptor/interceptor_test.go, line 23 at r1 (raw file):
"github.com/stretchr/testify/require" )
can this file also include a benchmark test?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is looking great. I like the overall structure and really appreciate the fact this is a standalone PR.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TFTRs!
do you know if the FrontendInterceptor could also be used to intercept CancelRequests? (not needed now, but wondering for the future)
As discussed over Slack, I originally thought that CancelRequest messages are just like any other messages, but it appears that it's part of the startup message, so no. These interceptors are meant to be used post-auth phase, where all messages must include 5 bytes for the headers (byte for type + int for length). In the future, if we'd like to combine both phases, we could modify the interceptor to support this.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @andy-kimball, @jeffswenson, and @rafiss)
pkg/ccl/sqlproxyccl/interceptor/interceptor.go, line 56 at r1 (raw file):
Previously, JeffSwenson (Jeff Swenson) wrote…
Does the interceptor return any recoverable errors? If all of the errors are final, I think we should remove .closed field. Tracking the .closed state adds some complexity to the code and I think it is reasonable to document the caller needs to throw away the interceptor if there is a read/write error.
The interceptor does not return any recoverable errors unless the caller wrapped src
or dst
to return custom errors. I've thought about this further, and I think I'll remove this. We'll just defer all of this logic to the caller. If the caller wraps any of those connections with custom errors, those errors should be handled accordingly. If this becomes an issue, we'll add this back.
pkg/ccl/sqlproxyccl/interceptor/interceptor.go, line 116 at r1 (raw file):
Previously, JeffSwenson (Jeff Swenson) wrote…
On 32 bit machines this cast can overflow. Most of the cases would be caught by the size<0 check below, but ~0+3, ~0+2, ~1, and ~0 would result in a weird state.
I'm not sure if it's worth handling this case given that we're already doing this on the DB side today:
cockroach/pkg/sql/pgwire/pgwirebase/encoding.go
Lines 154 to 156 in f1fcb83
size := int(binary.BigEndian.Uint32(b.tmp[:])) | |
// size includes itself. | |
size -= 4 |
I'm thinking to leave this as-is unless you strongly feel about this.
pkg/ccl/sqlproxyccl/interceptor/interceptor.go, line 157 at r1 (raw file):
Previously, JeffSwenson (Jeff Swenson) wrote…
I recommend changing the API to return msg []byte instead of body []byte. That allows us to simplify the implementation of chunkReader.Next and removes a few instances of needing to add/subtract the header size.
Good point. I can do that.
pkg/ccl/sqlproxyccl/interceptor/interceptor_test.go, line 23 at r1 (raw file):
Previously, rafiss (Rafi Shamim) wrote…
can this file also include a benchmark test?
Yes, I'll add one based on the benchmarks that I've done back then: https://docs.google.com/spreadsheets/d/1ui7Rkw9zIsEctOIFT8GFoO9aHpvBCJbBlsxg6JN_CmE/edit#gid=517332031.
a1ac855
to
a983a19
Compare
Informs cockroachdb#76000. Follow up to cockroachdb#76006. This commit adds a benchmark test for the connection copying logic. ``` go version go1.17.3 linux/amd64 --- goos: linux goarch: amd64 pkg: github.com/cockroachdb/cockroach/pkg/ccl/sqlproxyccl/interceptor cpu: AMD Ryzen 9 5950X 16-Core Processor BenchmarkConnectionCopy/msgCount=10000/msgLen=170/io.Copy-32 385 3538410 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=170/io.CopyN-32 15 75036404 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=170/pgproto3-32 26 42251150 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=170/chunkreader-32 18 65000047 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=170/interceptor-32 33 39125341 ns/op BenchmarkConnectionCopy/msgCount=500/msgLen=320/io.Copy-32 3204 406888 ns/op BenchmarkConnectionCopy/msgCount=500/msgLen=320/io.CopyN-32 410 3123298 ns/op BenchmarkConnectionCopy/msgCount=500/msgLen=320/pgproto3-32 577 2387535 ns/op BenchmarkConnectionCopy/msgCount=500/msgLen=320/chunkreader-32 469 3565173 ns/op BenchmarkConnectionCopy/msgCount=500/msgLen=320/interceptor-32 652 2015079 ns/op BenchmarkConnectionCopy/msgCount=9000/msgLen=2900/io.Copy-32 49 23330567 ns/op BenchmarkConnectionCopy/msgCount=9000/msgLen=2900/io.CopyN-32 18 72003323 ns/op BenchmarkConnectionCopy/msgCount=9000/msgLen=2900/pgproto3-32 15 82500818 ns/op BenchmarkConnectionCopy/msgCount=9000/msgLen=2900/chunkreader-32 18 79832494 ns/op BenchmarkConnectionCopy/msgCount=9000/msgLen=2900/interceptor-32 20 54727023 ns/op BenchmarkConnectionCopy/msgCount=5000/msgLen=30000/io.Copy-32 12 98640876 ns/op BenchmarkConnectionCopy/msgCount=5000/msgLen=30000/io.CopyN-32 10 110690053 ns/op BenchmarkConnectionCopy/msgCount=5000/msgLen=30000/pgproto3-32 6 177894915 ns/op BenchmarkConnectionCopy/msgCount=5000/msgLen=30000/chunkreader-32 9 129588686 ns/op BenchmarkConnectionCopy/msgCount=5000/msgLen=30000/interceptor-32 9 112610362 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=10/io.Copy-32 591 2274817 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=10/io.CopyN-32 25 47465099 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=10/pgproto3-32 58 23077900 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=10/chunkreader-32 38 31459201 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=10/interceptor-32 64 17616468 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=15/io.Copy-32 531 2336896 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=15/io.CopyN-32 30 45135200 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=15/pgproto3-32 51 22100293 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=15/chunkreader-32 49 28931167 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=15/interceptor-32 66 15189020 ns/op ``` Release note: None
Informs cockroachdb#76000. Follow up to cockroachdb#76006. This commit adds a benchmark test for the connection copying logic. ``` go version go1.17.3 linux/amd64 --- goos: linux goarch: amd64 pkg: github.com/cockroachdb/cockroach/pkg/ccl/sqlproxyccl/interceptor cpu: AMD Ryzen 9 5950X 16-Core Processor BenchmarkConnectionCopy/msgCount=10000/msgLen=170/io.Copy-32 385 3538410 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=170/io.CopyN-32 15 75036404 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=170/pgproto3-32 26 42251150 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=170/chunkreader-32 18 65000047 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=170/interceptor-32 33 39125341 ns/op BenchmarkConnectionCopy/msgCount=500/msgLen=320/io.Copy-32 3204 406888 ns/op BenchmarkConnectionCopy/msgCount=500/msgLen=320/io.CopyN-32 410 3123298 ns/op BenchmarkConnectionCopy/msgCount=500/msgLen=320/pgproto3-32 577 2387535 ns/op BenchmarkConnectionCopy/msgCount=500/msgLen=320/chunkreader-32 469 3565173 ns/op BenchmarkConnectionCopy/msgCount=500/msgLen=320/interceptor-32 652 2015079 ns/op BenchmarkConnectionCopy/msgCount=9000/msgLen=2900/io.Copy-32 49 23330567 ns/op BenchmarkConnectionCopy/msgCount=9000/msgLen=2900/io.CopyN-32 18 72003323 ns/op BenchmarkConnectionCopy/msgCount=9000/msgLen=2900/pgproto3-32 15 82500818 ns/op BenchmarkConnectionCopy/msgCount=9000/msgLen=2900/chunkreader-32 18 79832494 ns/op BenchmarkConnectionCopy/msgCount=9000/msgLen=2900/interceptor-32 20 54727023 ns/op BenchmarkConnectionCopy/msgCount=5000/msgLen=30000/io.Copy-32 12 98640876 ns/op BenchmarkConnectionCopy/msgCount=5000/msgLen=30000/io.CopyN-32 10 110690053 ns/op BenchmarkConnectionCopy/msgCount=5000/msgLen=30000/pgproto3-32 6 177894915 ns/op BenchmarkConnectionCopy/msgCount=5000/msgLen=30000/chunkreader-32 9 129588686 ns/op BenchmarkConnectionCopy/msgCount=5000/msgLen=30000/interceptor-32 9 112610362 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=10/io.Copy-32 591 2274817 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=10/io.CopyN-32 25 47465099 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=10/pgproto3-32 58 23077900 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=10/chunkreader-32 38 31459201 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=10/interceptor-32 64 17616468 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=15/io.Copy-32 531 2336896 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=15/io.CopyN-32 30 45135200 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=15/pgproto3-32 51 22100293 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=15/chunkreader-32 49 28931167 ns/op BenchmarkConnectionCopy/msgCount=10000/msgLen=15/interceptor-32 66 15189020 ns/op ``` Release note: None
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TFTRs! The PR is ready for another round of review.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @andy-kimball, @jeffswenson, and @rafiss)
pkg/ccl/sqlproxyccl/interceptor/interceptor.go, line 56 at r1 (raw file):
Previously, jaylim-crl (Jay Lim) wrote…
The interceptor does not return any recoverable errors unless the caller wrapped
src
ordst
to return custom errors. I've thought about this further, and I think I'll remove this. We'll just defer all of this logic to the caller. If the caller wraps any of those connections with custom errors, those errors should be handled accordingly. If this becomes an issue, we'll add this back.
I'm going to take that back. I'd rather be on the safe side, and keep closed
here for correctness purposes. Once we're sure that everything has been stabilized, we can remove this. Similarly, if we decide in the future that this adds a lot of complexity to the caller, we can remove this.
pkg/ccl/sqlproxyccl/interceptor/interceptor.go, line 116 at r1 (raw file):
Previously, jaylim-crl (Jay Lim) wrote…
I'm not sure if it's worth handling this case given that we're already doing this on the DB side today:
cockroach/pkg/sql/pgwire/pgwirebase/encoding.go
Lines 154 to 156 in f1fcb83
size := int(binary.BigEndian.Uint32(b.tmp[:])) // size includes itself. size -= 4 I'm thinking to leave this as-is unless you strongly feel about this.
As discussed, we'll leave it as it (i.e. cast int on uint32) given that we're already doing that elsewhere. That said, I've moved the subtraction to below, and checked on size < 4
instead.
pkg/ccl/sqlproxyccl/interceptor/interceptor.go, line 157 at r1 (raw file):
Previously, jaylim-crl (Jay Lim) wrote…
Good point. I can do that.
Done.
pkg/ccl/sqlproxyccl/interceptor/interceptor_test.go, line 23 at r1 (raw file):
Previously, jaylim-crl (Jay Lim) wrote…
Yes, I'll add one based on the benchmarks that I've done back then: https://docs.google.com/spreadsheets/d/1ui7Rkw9zIsEctOIFT8GFoO9aHpvBCJbBlsxg6JN_CmE/edit#gid=517332031.
I added that in a different PR: #76429.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: complete! 1 of 0 LGTMs obtained (waiting on @jaylim-crl, @jeffswenson, and @rafiss)
pkg/ccl/sqlproxyccl/interceptor/interceptor.go, line 48 at r2 (raw file):
// buf stores bytes which have been read, but have not been processed yet. buf []byte // readPos and writePos indicates the read and write pointers for bytes in
It'd be good to document a bit more detail on this buffer and pointers so that it's not necessary to read the code to discover the answers:
- Does a message always start at the beginning of a buffer (i.e. 1st 5 bytes are always the message header)? Or is it a circular buffer?
- Can the buffer contain multiple messages?
- Is
readPos
always <=writePos
?
Also, the comment says, "which have been read, but have not been processed yet", but if I call ReadMsg
, that does "process" bytes in the buffer, correct?
pkg/ccl/sqlproxyccl/interceptor/interceptor.go, line 75 at r2 (raw file):
// ensureNextNBytes blocks on IO reads until the buffer has at least n bytes. func (p *pgInterceptor) ensureNextNBytes(n int) error {
NIT: I typically order exported methods above un-exported methods, roughly ordering from most important at the top to least important at the bottom, since humans typically read in that order (top-to-bottom).
pkg/ccl/sqlproxyccl/interceptor/interceptor.go, line 168 at r2 (raw file):
return nil, err } msgSizeBytes := pgHeaderSizeBytes + size
NIT: It is a bit strange that you're returning the size of the message minus the header, and then immediately adding back the header size. You're even calling the combined size msgSizeBytes
. Maybe it'd be better to make PeekMsg
return the unmodified message size to begin with?
pkg/ccl/sqlproxyccl/interceptor/interceptor.go, line 188 at r2 (raw file):
// Copy bytes which have already been read. toCopy := msgSizeBytes if p.ReadSize() < msgSizeBytes {
NIT: This condition will always be true
, since msgSizeBytes > len(p.buf)
, and there's no way that p.ReadSize()
could be >=
that value.
pkg/ccl/sqlproxyccl/interceptor/interceptor.go, line 340 at r2 (raw file):
} // FrontendInterceptor is a client interceptor for the Postgres frontend
NIT: Why not break one or more of these other classes into separate files?
pkg/ccl/sqlproxyccl/interceptor/interceptor.go, line 343 at r2 (raw file):
// protocol. type FrontendInterceptor struct { p *pgInterceptor
NIT: Should you instead use type FrontendInterceptor pgInterceptor
here, to avoid allocating an extra object? Or embed the pgInterceptor
struct rather than the pointer? Is there the possibility of adding more fields to this struct, or some other consideration? Not a big deal, though, since we won't create many of these.
a47401e
to
a213b2a
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TFTRs!
Reviewable status: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @andy-kimball, @jeffswenson, and @rafiss)
pkg/ccl/sqlproxyccl/interceptor/interceptor.go, line 48 at r2 (raw file):
- No, a message can start in the middle of the buffer. This would happen if multiple messages were read into the buffer at once through the
io.ReadAtLeast
call. It is not a circular buffer for simplicity - we'll just read/write until the end of the buffer. All the public APIs handle this case. - Yes.
- Yes, that is correct. If
readPos == writePos
, we have nothing in the buffer.writePos
advances whenever we read into the buffer, whereasreadPos
advances whenever we read from the buffer to process the message; the latter is only possible throughReadMsg
andForwardMsg
.
Also, the comment says, "which have been read, but have not been processed yet", but if I call ReadMsg, that does "process" bytes in the buffer, correct?
Yes, that is also correct.
I've updated the comment with more information.
pkg/ccl/sqlproxyccl/interceptor/interceptor.go, line 75 at r2 (raw file):
Previously, andy-kimball (Andy Kimball) wrote…
NIT: I typically order exported methods above un-exported methods, roughly ordering from most important at the top to least important at the bottom, since humans typically read in that order (top-to-bottom).
Done. Makes sense.
pkg/ccl/sqlproxyccl/interceptor/interceptor.go, line 168 at r2 (raw file):
Previously, andy-kimball (Andy Kimball) wrote…
NIT: It is a bit strange that you're returning the size of the message minus the header, and then immediately adding back the header size. You're even calling the combined size
msgSizeBytes
. Maybe it'd be better to makePeekMsg
return the unmodified message size to begin with?
There are two types of sizes here:
- message size: 1 byte (type) + 4 bytes (length) + # of body bytes
- body size: # of body bytes
It's strange that the Postgres protocol defined the length int as the # of body bytes + 4 bytes (which is inclusive of itself). PeekMsg
returns the body size here; note that we subtracted 4 bytes (since it included itself), and not the header size (5 bytes). If we returned the unmodified size (which is defined as 4 bytes + # of body bytes), we'd still have to add 1 for the type, and define that as message size here.
For now, I'll keep it as-is, i.e. PeekMsg
returns the body size. I'll think further about this, and if we decide to change the definition of PeekMsg
, I'll do that in a follow up PR. In that case, we'd still have to add 1 to the unmodified length for PeekMsg
to return the message size.
pkg/ccl/sqlproxyccl/interceptor/interceptor.go, line 188 at r2 (raw file):
Previously, andy-kimball (Andy Kimball) wrote…
NIT: This condition will always be
true
, sincemsgSizeBytes > len(p.buf)
, and there's no way thatp.ReadSize()
could be>=
that value.
Done. Good catch. In the initial version, we always allocate regardless, but now that we only allocate if the message doesn't fit, this condition is redundant.
pkg/ccl/sqlproxyccl/interceptor/interceptor.go, line 340 at r2 (raw file):
Previously, andy-kimball (Andy Kimball) wrote…
NIT: Why not break one or more of these other classes into separate files?
Done.
pkg/ccl/sqlproxyccl/interceptor/interceptor.go, line 343 at r2 (raw file):
Previously, andy-kimball (Andy Kimball) wrote…
NIT: Should you instead use
type FrontendInterceptor pgInterceptor
here, to avoid allocating an extra object? Or embed thepgInterceptor
struct rather than the pointer? Is there the possibility of adding more fields to this struct, or some other consideration? Not a big deal, though, since we won't create many of these.
No, we won't be adding more fields to this struct. The APIs that we have here should suffice. I moved it to use pgInterceptor
as you suggested.
Informs cockroachdb#76000. This commit implements postgres interceptors, namely FrontendInterceptor and BackendInterceptor, as described in the sqlproxy connection migration RFC. These interceptors will be used as building blocks for the forwarder component that we will be adding in a later PR. Since the forwarder component has not been added, a simple proxy test (i.e. TestSimpleProxy) has been added to illustrate how the frontend and backend interceptors can be used within the proxy. Release note: None
a213b2a
to
f39ee92
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @andy-kimball, @jaylim-crl, and @rafiss)
pkg/ccl/sqlproxyccl/interceptor/interceptor.go, line 168 at r2 (raw file):
Previously, jaylim-crl (Jay Lim) wrote…
There are two types of sizes here:
- message size: 1 byte (type) + 4 bytes (length) + # of body bytes
- body size: # of body bytes
It's strange that the Postgres protocol defined the length int as the # of body bytes + 4 bytes (which is inclusive of itself).
PeekMsg
returns the body size here; note that we subtracted 4 bytes (since it included itself), and not the header size (5 bytes). If we returned the unmodified size (which is defined as 4 bytes + # of body bytes), we'd still have to add 1 for the type, and define that as message size here.For now, I'll keep it as-is, i.e.
PeekMsg
returns the body size. I'll think further about this, and if we decide to change the definition ofPeekMsg
, I'll do that in a follow up PR. In that case, we'd still have to add 1 to the unmodified length forPeekMsg
to return the message size.
I'm okay with deferring this decision to a future PR, but I think the size returned should be bodySize + headerSize. All of the consumers of PeekMsg in this PR immediately add the header size. It also provides a nice symmetry in the API. That way the size returned by PeekMsg is equal to the length of the byte slice returned by ReadMsg().
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TFTRs!
Reviewable status: complete! 0 of 0 LGTMs obtained (and 1 stale) (waiting on @andy-kimball, @jeffswenson, and @rafiss)
pkg/ccl/sqlproxyccl/interceptor/interceptor.go, line 168 at r2 (raw file):
Previously, JeffSwenson (Jeff Swenson) wrote…
I'm okay with deferring this decision to a future PR, but I think the size returned should be bodySize + headerSize. All of the consumers of PeekMsg in this PR immediately add the header size. It also provides a nice symmetry in the API. That way the size returned by PeekMsg is equal to the length of the byte slice returned by ReadMsg().
Sounds like a plan. I added that as an item to #76000.
bors r=andy-kimball,JeffSwenson |
Build succeeded: |
…y size Informs cockroachdb#76000. Follow-up to cockroachdb#76006. Previously, PeekMsg was returning the body size (excluding header size), which is a bit awkward from an API point of view because most callers of PeekMsg immediately adds the header size to the returned size previously. This commit cleans the API design up by making PeekMsg return the message size instead, i.e. header inclusive. At the same time, returning the message size makes it consistent with the ReadMsg API since that returns the entire message. Release note: None
…y size Informs cockroachdb#76000. Follow-up to cockroachdb#76006. Previously, PeekMsg was returning the body size (excluding header size), which is a bit awkward from an API point of view because most callers of PeekMsg immediately adds the header size to the returned size previously. This commit cleans the API design up by making PeekMsg return the message size instead, i.e. header inclusive. At the same time, returning the message size makes it consistent with the ReadMsg API since that returns the entire message. Release note: None
…76591 75809: [CRDB-12226] server, ui: display circuit breakers in problem ranges and range status r=Santamaura a=Santamaura This PR adds changes to the reports/problemranges and reports/range pages. Ranges with replicas that have a circuit breaker will show up as problem ranges and the circuit breaker error will show up as a row on the status page. Release note (ui change): display circuit breakers in problems ranges and range status Problem Ranges page: ![Screen Shot 2022-02-08 at 4 57 51 PM](https://user-images.githubusercontent.com/17861665/153082648-6c03d195-e395-456a-be00-55ad24863752.png) Range status page: ![Screen Shot 2022-02-08 at 4 57 34 PM](https://user-images.githubusercontent.com/17861665/153082705-cbbe5507-e81d-49d7-a3f7-21b4c84226c2.png) 76278: Add cluster version as feature gate for block properties r=dt,erikgrinaker,jbowens a=nicktrav Two commits here - the first adds a new cluster version, the second makes use of the cluster version as a feature gate (and update various call sites all over the place). --- pkg/clusterversion: add new version as feature gate for block properties Prior to this change, the block properties SSTable-level feature was enabled in a single cluster version. This introduced a subtle race in that for the period in which each node is being updated to `PebbleFormatBlockPropertyCollector`, there is a brief period where not all nodes are at the same cluster version, and thus store versions. If nodes at the newer version write SSTables that make use of block properties, and these tables are consumed by nodes that are yet to be updated, the older nodes could panic when attempting to read the tables with a format they do not yet understand. While this race is academic, given that a) there are now subsequent cluster versions that act as barriers during the migration, and b) block properties were disabled in 1a8fb57, this patch addresses the race by adding a second cluster version. `EnablePebbleFormatVersionBlockProperties` acts as a barrier and a feature gate. A guarantee of the migration framework is that any node at this newer version is part of a cluster that has already run the necessary migrations for the older version, and thus ratcheted the format major version in the store, and thus enabled the block properties feature, across all nodes. Add additional documentation in `pebble.go` that details how to make use of the two-version pattern for future table-level version changes. --- pkg/storage: make MakeIngestionWriterOptions version aware With the introduction of block properties, and soon, range keys, which introduce backward-incompatible changes at the SSTable level, all nodes in a cluster must all have a sufficient store version in order to avoid runtime incompatibilities. Update `storage.MakeIngestionWriterOptions` to add a `context.Context` and `cluster.Settings` as parameters, which allows for determining whether a given cluster version is active (via `(clusterversion.Handle).IsActive()`). This allows gating the enabling / disabling of block properties (and soon, range keys), on all nodes being at a sufficient cluster version. Update various call-sites to pass in the `context.Context` and `cluster.Settings`. --- 76348: ui: downsample SQL transaction metrics using MAX r=maryliag a=dhartunian Previously, we were using the default downsampling behavior of the timeseries query engine for "Open SQL Transactions" and "Active SQL Statements" on the metrics page in DB console. This led to confusion when zooming in on transaction spikes since the spike would get larger as the zoom got tighter. This PR changes the aggregation function to use MAX to prevent this confusion. Resolves: #71827 Release note (ui change): Open SQL Transactions and Active SQL Transactions are downsampled using MAX instead of AVG and will more accurately reflect narrow spikes in transaction counts when looking and downsampled data. 76414: spanconfig: teach the KVAccessor about system span configurations r=arulajmani a=arulajmani First 3 commits are from #76219, this one's quite small -- mostly just tests. ---- This patch teaches the KVAccessor to update and get system span configurations. Release note: None 76538: ui: Use liveness info to populate decommissioned node lists r=zachlite a=zachlite Previously, the decommissioned node lists considered node status entries to determine decommissioning and decommissioned status. This changed in #56529, resulting in empty lists. Now, the node's liveness entry is considered and these lists are correctly populated. Release note (bug fix): The list of recently decommissioned nodes and the historical list of decommissioned nodes correctly display decommissioned nodes. 76544: builtins: add rehome_row to DistSQLBlocklist r=mgartner,otan a=rafiss fixes #76153 This builtin always needs to run on the gateway node. Release note: None 76546: build: display pebble git SHA in GitHub messages r=rickystewart a=nicktrav Use the short from of the Git SHA from the go.mod-style version in DEPS.bzl as the Pebble commit. This ensures that failure messages created by Team City link to a GitHub page that renders correctly. Release note: None 76550: gen/genbzl: general improvements r=ajwerner a=ajwerner This change does a few things: * It reworks the queries in terms of eachother in-memory. This is better than the previous iteration whereby it'd generate the results and then rely on the output of that query. Instead, we just build up bigger query expressions and pass them to bazel using the --query_file flag. * It avoids exploring the pkg/ui directory (and the pkg/gen directory) because those can cause problems. The pkg/ui directory ends up bringing in npm, which hurts. * It stops rewriting the files before executing the queries. It no longer needs to rewrite them up front because they aren't referenced by later queries. * It removes the excluded target which was problematic because those files weren't properly visible. Fixes #76521 Fixes #76503 Release note: None 76562: ccl/sqlproxyccl: update PeekMsg to return message size instead of body size r=JeffSwenson a=jaylim-crl Informs #76000. Follow-up to #76006. Previously, PeekMsg was returning the body size (excluding header size), which is a bit awkward from an API point of view because most callers of PeekMsg immediately adds the header size to the returned size previously. This commit cleans the API design up by making PeekMsg return the message size instead, i.e. header inclusive. At the same time, returning the message size makes it consistent with the ReadMsg API since that returns the entire message. Release note: None 76591: bazel: update shebang line in `sql-gen.sh` r=rail a=rickystewart Release note: None Co-authored-by: Santamaura <[email protected]> Co-authored-by: Nick Travers <[email protected]> Co-authored-by: David Hartunian <[email protected]> Co-authored-by: arulajmani <[email protected]> Co-authored-by: Zach Lite <[email protected]> Co-authored-by: Rafi Shamim <[email protected]> Co-authored-by: Andrew Werner <[email protected]> Co-authored-by: Jay <[email protected]> Co-authored-by: Ricky Stewart <[email protected]>
…y size Informs cockroachdb#76000. Follow-up to cockroachdb#76006. Previously, PeekMsg was returning the body size (excluding header size), which is a bit awkward from an API point of view because most callers of PeekMsg immediately adds the header size to the returned size previously. This commit cleans the API design up by making PeekMsg return the message size instead, i.e. header inclusive. At the same time, returning the message size makes it consistent with the ReadMsg API since that returns the entire message. Release note: None
Informs #76000.
This commit implements postgres interceptors, namely FrontendInterceptor and
BackendInterceptor, as described in the sqlproxy connection migration RFC.
These interceptors will be used as building blocks for the forwarder component
that we will be adding in a later PR. Since the forwarder component has not
been added, a simple proxy test (i.e. TestSimpleProxy) has been added to
illustrate how the frontend and backend interceptors can be used within the
proxy.
Release note: None