-
Notifications
You must be signed in to change notification settings - Fork 3.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
sql: implement SHOW [ALL] CLUSTER SETTINGS FOR TENANT #77742
Conversation
ae5de1c
to
d89ab30
Compare
|
281cb73
to
a1889a8
Compare
0c61e21
to
688a3a2
Compare
d59b53a
to
becdf78
Compare
Updated to reflect #77935. |
becdf78
to
39da854
Compare
RFAL |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
just had small comments
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @knz and @RaduBerinde)
-- commits, line 22 at r15:
does this need a release note?
pkg/sql/delegate/show_all_cluster_settings.go, line 69 at r15 (raw file):
if !d.evalCtx.Codec.ForSystemTenant() { return nil, pgerror.Newf(pgcode.InsufficientPrivilege, "SHOW CLUSTER SETTINGS FOR TENANT can only be called by system operators")
nit: is the term "on the system tenant" more familiar than "by system operatosr"
pkg/sql/delegate/show_all_cluster_settings.go, line 84 at r15 (raw file):
isvalid AS ( SELECT IF(tenant_id=0,
i think this would be more readable with:
CASE
WHEN tenant_id=0 THEN ...
WHEN tenant_id=1 THEN ...
WHEN st.id IS NULL THEN ...
ELSE 0
END
pkg/sql/delegate/show_all_cluster_settings.go, line 85 at r15 (raw file):
SELECT IF(tenant_id=0, crdb_internal.force_error('22023', 'tenant ID must be non-zero'),
why do this validation / error creation in SQL rather than go code?
pkg/sql/delegate/show_all_cluster_settings.go, line 98 at r15 (raw file):
crdb_internal.decode_cluster_setting(allsettings.variable, -- NB: careful not to coalesce with allsettings.value directly! -- This is the value fo the system tenant and is not relevant to other tenants.
nit: "fo"
pkg/sql/delegate/show_all_cluster_settings.go, line 107 at r15 (raw file):
allsettings.type, ` + publicCol + ` IF(tenantspecific.value IS NOT NULL, 'per-tenant-override',
i think CASE
would be more readable here too
pkg/sql/delegate/show_all_cluster_settings.go, line 112 at r15 (raw file):
allsettings.description FROM (SELECT * FROM system.crdb_internal.cluster_settings ` + publicFilter + `) AS allsettings
since there already are CTEs in this query, i wonder if this would be more understandable if CTEs were used for allsettings
and tenantspecific
instead of subqueries?
pkg/sql/delegate/show_all_cluster_settings.go, line 113 at r15 (raw file):
FROM (SELECT * FROM system.crdb_internal.cluster_settings ` + publicFilter + `) AS allsettings LEFT JOIN (SELECT * FROM system.tenant_settings t, tenant_id WHERE t.tenant_id = tenant_id.tenant_id) AS tenantspecific ON
here and the above lines, would be a bit safer and easier to read this if the columns were named explicitly instead of using *
pkg/sql/sem/builtins/builtins.go, line 4632 at r14 (raw file):
setting, ok := rawSetting.(settings.NonMaskedSetting) if !ok { return nil, errors.AssertionFailedf("setting '%s' is masked", name)
should this be a user-friendly error?
pkg/sql/sem/builtins/builtins.go, line 4664 at r14 (raw file):
) if !ok { return nil, errors.Newf("unknown cluster setting '%s'", name)
could this have a pgcode
?
pkg/sql/sem/builtins/builtins.go, line 4668 at r14 (raw file):
setting, ok := rawSetting.(settings.NonMaskedSetting) if !ok { return nil, errors.AssertionFailedf("setting '%s' is masked", name)
should this be a user-friendly error?
pkg/sql/sem/builtins/builtins.go, line 4676 at r14 (raw file):
return tree.NewDString(repr), nil }, Info: "Returns the encoded default value of the given cluster setting.",
remove "default" and change "encoded" to "decoded" (or reword as you like)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @RaduBerinde and @rafiss)
Previously, rafiss (Rafi Shamim) wrote…
does this need a release note?
I don't believe so, since the tenant config stuff is not visible to end-users?
Let me know what you think.
pkg/sql/delegate/show_all_cluster_settings.go, line 69 at r15 (raw file):
Previously, rafiss (Rafi Shamim) wrote…
nit: is the term "on the system tenant" more familiar than "by system operatosr"
I reused the same words as was originally implemented by adam in sql/tenant_settings.go for consistency.
pkg/sql/delegate/show_all_cluster_settings.go, line 84 at r15 (raw file):
Previously, rafiss (Rafi Shamim) wrote…
i think this would be more readable with:
CASE WHEN tenant_id=0 THEN ... WHEN tenant_id=1 THEN ... WHEN st.id IS NULL THEN ... ELSE 0 END
Good call! Done.
pkg/sql/delegate/show_all_cluster_settings.go, line 85 at r15 (raw file):
Previously, rafiss (Rafi Shamim) wrote…
why do this validation / error creation in SQL rather than go code?
Added a comment to explain.
pkg/sql/delegate/show_all_cluster_settings.go, line 107 at r15 (raw file):
Previously, rafiss (Rafi Shamim) wrote…
i think
CASE
would be more readable here too
Done.
pkg/sql/delegate/show_all_cluster_settings.go, line 112 at r15 (raw file):
Previously, rafiss (Rafi Shamim) wrote…
since there already are CTEs in this query, i wonder if this would be more understandable if CTEs were used for
allsettings
andtenantspecific
instead of subqueries?
Good idea. Done.
pkg/sql/delegate/show_all_cluster_settings.go, line 113 at r15 (raw file):
Previously, rafiss (Rafi Shamim) wrote…
here and the above lines, would be a bit safer and easier to read this if the columns were named explicitly instead of using
*
Good idea. Done.
pkg/sql/sem/builtins/builtins.go, line 4632 at r14 (raw file):
Previously, rafiss (Rafi Shamim) wrote…
should this be a user-friendly error?
The user-friendly error is just above: "unknown cluster setting". Masked settings should already been invisible to the Lookup function. If we get here and we still have a masked setting, this means that Lookup did not do its job properly, so that's an assertion failure.
pkg/sql/sem/builtins/builtins.go, line 4664 at r14 (raw file):
Previously, rafiss (Rafi Shamim) wrote…
could this have a
pgcode
?
I'm not opposed, but then I'd like to use a separate PR for this: the base case in sql/set_cluster_setting.go
doesn't use a pgcode yet either, and we'd want both to be consistent.
pkg/sql/sem/builtins/builtins.go, line 4668 at r14 (raw file):
Previously, rafiss (Rafi Shamim) wrote…
should this be a user-friendly error?
see comment above.
pkg/sql/sem/builtins/builtins.go, line 4676 at r14 (raw file):
Previously, rafiss (Rafi Shamim) wrote…
remove "default" and change "encoded" to "decoded" (or reword as you like)
Done.
a544d8d
to
2425e8d
Compare
… decode_cluster_setting This is a requirement to implementing SHOW CLUSTER SETTINGS FOR TENANT. Release justification: low risk, high benefit changes to existing functionality Release note: None
2425e8d
to
629f295
Compare
Release justification: low risk, high benefit changes to existing functionality Release note: None
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm!
Reviewable status: complete! 0 of 0 LGTMs obtained (waiting on @knz and @RaduBerinde)
Previously, knz (kena) wrote…
I don't believe so, since the tenant config stuff is not visible to end-users?
Let me know what you think.
sgtm!
pkg/sql/delegate/show_all_cluster_settings.go, line 85 at r15 (raw file):
Previously, knz (kena) wrote…
Added a comment to explain.
thanks
pkg/sql/sem/builtins/builtins.go, line 4632 at r14 (raw file):
Previously, knz (kena) wrote…
The user-friendly error is just above: "unknown cluster setting". Masked settings should already been invisible to the Lookup function. If we get here and we still have a masked setting, this means that Lookup did not do its job properly, so that's an assertion failure.
i see, thanks
thanks! bors r=rafiss |
Build succeeded: |
74005: server: hash tenantID into ClusterID used in SQL exec r=knz a=dt Fixes #74480. Prior to this patch, in a multi-tenant deployment the ClusterID value visible to tenans was the same as the ClusterID for the storage cluster. This original design was defective because it prevented the backup/restore subsystem from distinguishing backups made for different tenants. In this commit, we change the ClusterID for tenants to become a new UUID that is the storage cluster's ClusterID with a hash of the tenant's ID added to the lower 64 bits. This makes it so that two tenants on the same storage cluster will observe different values when calling ClusterID(). Release note (bug fix): Separate tenants now use unique values for the internal field ClusterID. Previously separate tenants that were hosted on the same storage cluster would be assigned the same ClusterID. Jira issue: CRDB-14750 77794: sql: implement SHOW CLUSTER SETTING FOR TENANT r=rafiss a=knz All commits but the last 2 from #77742. (Reviewers: only the last 2 commits belong to this PR.) Fixes #77471 Release justification: fixes to high-priority bugs 79503: backupccl: avoid over-shrinking memory monitor r=adityamaru a=stevendanna This change updates DecryptFile to optionally use a memory monitor and adjusts the relevant calling functions. Previously, the readManifest function that used DecryptFile could result in shrinking too many bytes from the bound account: ``` ERROR: a panic has occurred! root: no bytes in account to release, current 29571, free 30851 (1) attached stack trace -- stack trace: | runtime.gopanic | GOROOT/src/runtime/panic.go:1038 | [...repeated from below...] Wraps: (2) attached stack trace -- stack trace: | github.com/cockroachdb/cockroach/pkg/util/log/logcrash.ReportOrPanic | github.com/cockroachdb/cockroach/pkg/util/log/logcrash/crash_reporting.go:374 | github.com/cockroachdb/cockroach/pkg/util/mon.(*BoundAccount).Shrink | github.com/cockroachdb/cockroach/pkg/util/mon/bytes_usage.go:709 | github.com/cockroachdb/cockroach/pkg/ccl/backupccl.(*restoreResumer).doResume.func1 | github.com/cockroachdb/cockroach/pkg/ccl/backupccl/pkg/ccl/backupccl/restore_job.go:1244 ``` This was the result of us freeing `cap(descBytes)` despite the fact that we never accounted for the fact that `descBytes` had been re-assigned after the decryption without accounting for changes in the capacity used by the old vs new buffer. That is, we called mem.Grow with the capcity of the old buffer, but mem.Shrink with the capacity of the new buffer. We generally expect that the size of encrypted data to be larger than the size of the plaintext data, so in most cases, the existing code would work without error because it was freeing an amount smaller than the original allocation. However, while the plaintext data is smaller than the encrypted data, that doesn't mean that the _capacity of the slice holding the plaintext data_ is smaller than the _capacity of the slice holding the encrypted data_. The slice holding the encrypted data was created with mon.ReadAll which has slice growth strategy of doubling the size until it reaches 8MB. The slice holding the plaintext data was created by ioutil.ReadAll that defers to appends slice growth strategy, which differs from that used in mon.ReadAll. In the current implementations, for byte sizes around 30k, the capacity of the buffer create by append's strategy is larger than that created by mon.ReadAll despite the len of the buffer being smaller: before decrypt: len(descBytes) = 27898, cap(descBytes) = 32768 after decrypt: len(descBytes) = 27862, cap(descBytes) = 40960 We could have fixed this by simply adjusting the memory account by the difference. Instead, I've opted to thread the memory monitor into DecryptFile to better account for the fact that we technically do hold 2 copies of the data during this decryption. Fixes #79488 Release note: None 79532: ci: fix some janky ci scripts that prevent posting issues to github r=mari-crl a=rickystewart In a couple places in `82e0b121c715c59cebbb8d53e29edf7952d6913f` I accidentally did `$exit_status=$?` instead of `exit_status=$?`, which is not a proper variable assignment. This was causing these jobs to fail before they could post issues to GitHub. Closes #79403. Release note: None Co-authored-by: David Taylor <[email protected]> Co-authored-by: Raphael 'kena' Poss <[email protected]> Co-authored-by: Steven Danna <[email protected]> Co-authored-by: Ricky Stewart <[email protected]>
All commits but the last 2 from #77740.
(Reviewers: only the last 2 commits belong to this PR.)
Informs #77471
Release justification: low risk, high benefit changes to existing functionality