Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tenantcapabilities: restructure interfaces #100217

Merged

Conversation

ajwerner
Copy link
Contributor

This commit is mostly just a refactoring of what we have. The previous iteration had some unfortunate coupling between the protobufs and the top-level package. Having the protobuf generated code not be a leaf hurts.

Another pain point was the lack of strong typing when interacting with capabilities. Given this is go, somewhere there's a need to do some type assertion when dealing sets of variant instance types.

The biggest change is that the TenantCapabilities concept is no longer wrapped in an interface implemented by the protobuf. Instead, the Capability implementations now live inside the tenantcapabilities package, and can be looked up with FromName and FromID, without any access to concrete capabilities. These capabilies then know how to look up values in a strongly typed way from the concrete capabilities.

Epic: none

Release note: None

@cockroach-teamcity
Copy link
Member

This change is Reviewable

@ajwerner ajwerner force-pushed the ajwerner/rework-tenant-capabilities branch 3 times, most recently from 1a91748 to 0172db8 Compare March 30, 2023 22:14
Copy link
Collaborator

@arulajmani arulajmani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a great simplification, I love where this ended up -- nice parting gift indeed. Thanks for taking the time to refactor this stuff.

Reviewed 51 of 53 files at r1, 8 of 8 files at r2, 1 of 1 files at r3, 2 of 2 files at r4, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @ajwerner)


-- commits line 6 at r4:

Having the protobuf generated code not be a leaf hurts.

Heavy +1.


pkg/multitenant/tenantcapabilities/value.go line 71 at r3 (raw file):

type spanConfigBoundsValue struct {
	b **tenantcapabilitiespb.SpanConfigBounds

nit: could you add a comment explaining why a pointer to a pointer (as opposed to just a pointer) here?


pkg/multitenant/tenantcapabilities/value.go line 74 at r3 (raw file):

}

func (s *spanConfigBoundsValue) Get() *spanconfigbounds.Bounds {

I like the fact that we're returning spanconfigbounds.Bounds right here, instead of having to convert it in the spanconfig package. I had found myself wrestling with the package structure here initially, so I'm a solid 💯 on this (and the reorganization you've done here, in general).


pkg/multitenant/tenantcapabilities/values.go line 20 at r4 (raw file):

// MustGetValueByID will get the value for the capability corresponding to
// the requested ID. If the ID is not valid, this function will panic.
func MustGetValueByID(t *tenantcapabilitiespb.TenantCapabilities, id ID) Value {

Instead of both this and GetValueByID, consider just having the latter and letting that thing panic if need be. Looks like we're only ever using the version that panics in production anyway, so not having two options here will be one less question future visitors will have to ask when they see this code.


pkg/spanconfig/spanconfigbounds/span_config_bounds.go line 26 at r3 (raw file):

// policy to interact with SpanConfigs.
type Bounds struct {
	b *tenantcapabilitiespb.SpanConfigBounds

Slightly curious why you had this wrapping before but decided to change it.


pkg/spanconfig/spanconfigstore/bounds_reader.go line 11 at r4 (raw file):

// licenses/APL.txt.

package spanconfigstore

Was the package move motivated by not wanting to import spanconfigbounds into KVSubscriber? Or was there more to it?


pkg/multitenant/tenantcapabilities/values_test.go line 20 at r4 (raw file):

)

func TestCapabilityGetSet(t *testing.T) {

This test wasn't great, but do you think it's worth keeping around a TestCapabilitiesGetSet test albeit more useful?

Mostly in the context of span config bounds -- should we add something that uses Set to alter span config bounds and then makes sure the underlying proto is indeed modified using a Get? Your call on whether it's worth having.

@ajwerner ajwerner force-pushed the ajwerner/rework-tenant-capabilities branch from 0172db8 to 1b504a5 Compare March 30, 2023 23:12
Copy link
Contributor Author

@ajwerner ajwerner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TFTR!

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @arulajmani)


-- commits line 6 at r4:

Previously, arulajmani (Arul Ajmani) wrote…

Having the protobuf generated code not be a leaf hurts.

Heavy +1.

Done.


pkg/multitenant/tenantcapabilities/value.go line 71 at r3 (raw file):

Previously, arulajmani (Arul Ajmani) wrote…

nit: could you add a comment explaining why a pointer to a pointer (as opposed to just a pointer) here?

Done.


pkg/multitenant/tenantcapabilities/values.go line 20 at r4 (raw file):

Previously, arulajmani (Arul Ajmani) wrote…

Instead of both this and GetValueByID, consider just having the latter and letting that thing panic if need be. Looks like we're only ever using the version that panics in production anyway, so not having two options here will be one less question future visitors will have to ask when they see this code.

"in production" 🤔 we haven't shipped this code. I don't love the idea of panics that can happen with a malformed number. I'm inclined to not do that.


pkg/spanconfig/spanconfigbounds/span_config_bounds.go line 26 at r3 (raw file):

Previously, arulajmani (Arul Ajmani) wrote…

Slightly curious why you had this wrapping before but decided to change it.

🤷 I wasn't seeing the value in the wrapping.


pkg/spanconfig/spanconfigstore/bounds_reader.go line 11 at r4 (raw file):

Previously, arulajmani (Arul Ajmani) wrote…

Was the package move motivated by not wanting to import spanconfigbounds into KVSubscriber? Or was there more to it?

It was to avoid a cycle. This interface depends on tenantcapabilities and tenantcapabilities now depends on spanconfigbounds so this couldn't live in spanconfigbounds.


pkg/multitenant/tenantcapabilities/values_test.go line 20 at r4 (raw file):

Previously, arulajmani (Arul Ajmani) wrote…

This test wasn't great, but do you think it's worth keeping around a TestCapabilitiesGetSet test albeit more useful?

Mostly in the context of span config bounds -- should we add something that uses Set to alter span config bounds and then makes sure the underlying proto is indeed modified using a Get? Your call on whether it's worth having.

I'll do it in the follow-up PR which integrates SpanConfigBounds.

@ajwerner ajwerner force-pushed the ajwerner/rework-tenant-capabilities branch 2 times, most recently from 88d596d to f466673 Compare March 31, 2023 02:14
@ajwerner ajwerner marked this pull request as ready for review March 31, 2023 02:14
@ajwerner ajwerner requested review from a team as code owners March 31, 2023 02:14
@ajwerner ajwerner requested a review from a team March 31, 2023 02:14
@ajwerner ajwerner added the backport-23.1.x Flags PRs that need to be backported to 23.1 label Mar 31, 2023
@ajwerner
Copy link
Contributor Author

TFTR!

bors r+

@craig
Copy link
Contributor

craig bot commented Mar 31, 2023

Build failed (retrying...):

@craig
Copy link
Contributor

craig bot commented Mar 31, 2023

Build failed (retrying...):

@ajwerner
Copy link
Contributor Author

bors r-

@craig
Copy link
Contributor

craig bot commented Mar 31, 2023

Canceled.

@ajwerner ajwerner force-pushed the ajwerner/rework-tenant-capabilities branch from f466673 to ae0ee1b Compare March 31, 2023 04:13
This commit is mostly just a refactoring of what we have. The previous
iteration had some unfortunate coupling between the protobufs and the
top-level package. Having the protobuf generated code not be a leaf hurts.

Another pain point was the lack of strong typing when interacting with
capabilities. Given this is go, somewhere there's a need to do some
type assertion when dealing sets of variant instance types.

The biggest change is that the `TenantCapabilities` concept is no longer
wrapped in an interface implemented by the protobuf. Instead, the `Capability`
implementations now live inside the `tenantcapabilities` package, and can
be looked up with `FromName` and `FromID`, without any access to concrete
capabilities. These capabilies then know how to look up values in a strongly
typed way from the concrete capabilities.

Epic: none

Release note: None
@ajwerner ajwerner force-pushed the ajwerner/rework-tenant-capabilities branch from ae0ee1b to 8e75cfa Compare March 31, 2023 04:18
Copy link
Contributor Author

@ajwerner ajwerner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @arulajmani)


pkg/multitenant/tenantcapabilities/values_test.go line 20 at r4 (raw file):

Previously, ajwerner wrote…

I'll do it in the follow-up PR which integrates SpanConfigBounds.

I added it back

@ajwerner
Copy link
Contributor Author

bors r+

@craig
Copy link
Contributor

craig bot commented Mar 31, 2023

Build failed (retrying...):

@craig
Copy link
Contributor

craig bot commented Mar 31, 2023

Build succeeded:

@craig craig bot merged commit 9eae556 into cockroachdb:master Mar 31, 2023
@blathers-crl
Copy link

blathers-crl bot commented Mar 31, 2023

Encountered an error creating backports. Some common things that can go wrong:

  1. The backport branch might have already existed.
  2. There was a merge conflict.
  3. The backport branch contained merge commits.

You might need to create your backport manually using the backport tool.


error setting reviewers, but backport branch blathers/backport-release-23.1-100217 is ready: POST https://api.github.com/repos/cockroachdb/cockroach/pulls/100272/requested_reviewers: 422 Reviews may only be requested from collaborators. One or more of the teams you specified is not a collaborator of the cockroachdb/cockroach repository. []

Backport to branch 23.1.x failed. See errors above.


🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

Copy link
Contributor

@knz knz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewed 41 of 53 files at r1, 5 of 8 files at r2, 2 of 2 files at r4, 5 of 6 files at r5, 7 of 7 files at r6, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained


pkg/multitenant/tenantcapabilities/tenantcapabilitiestestutils/testutils.go line 79 at r6 (raw file):

				return roachpb.TenantID{}, nil, err
			}
			c.Value(&caps).Set(spanconfigbounds.New(&v))

I'm missing a default case here.

ajwerner added a commit to ajwerner/cockroach that referenced this pull request Mar 31, 2023
The test should fail if we ever add a new type of capability and use it in the
data driven test but don't update the test to handle it.

Epic: none

Follow-up from cockroachdb#100217 (review)

Release note: None
Copy link
Contributor Author

@ajwerner ajwerner left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained


pkg/multitenant/tenantcapabilities/tenantcapabilitiestestutils/testutils.go line 79 at r6 (raw file):

Previously, knz (Raphael 'kena' Poss) wrote…

I'm missing a default case here.

#100294

craig bot pushed a commit that referenced this pull request Mar 31, 2023
99312: sqlsmith: add DEFAULT expressions to newly added columns r=mgartner a=mgartner

Sqlsmith now builds `ALTER TABLE .. ADD COLUMN .. DEFAULT` statements
with default expressions that have different types than the column type.
This is allowed if the default expression type can be assignment-casted
to the column's type.

Fixes #98133

Release note: None


99348: testutils: move default test tenant message r=rharding6373 a=herkolategan

In order to reduce logging noise but still inform test authors of the default test tenant, the message has been moved to where there is a `testing.TB` interface.

Epic: CRDB-18499

99835: opt/execbuilder: add panic catching to buildRoutinePlanGenerator r=mgartner a=mgartner

This commit adds a panic catcher to callback functions created in
execbuilder and invoked during evaluation of UDFs and correlated
subqueries. It matches the panic catcher logic in `buildApplyJoin`.

Fixes #98786

Release note: None


100267: roachtest: own autoupgrade to TestEng r=renatolabs a=tbg

Discussed in #99479.

Epic: none
Release note: None


100286: roachtest: prevent aws roachtest panic r=rail a=msbutler

After #99723 merged as a bandaid for #98783, the aws roachtest nightly began to panic because of a different roachtest papercut #96655. Specifically, because roachtest filters which tests run on which cloud within the evaluation of the test closure, tests meant to run on gce will still get registered in an AWS run. During the registration of the gce test
`restore/tpce/400GB/gce/nodes=4/cpus=8/lowmem` _on aws_, the aws test harness panics because the aws roachprod implementation does not have a low memory cpu configuration. This patch prevents this panic and should be reverted once the pr #99402 merges.

Epic: None

Release note: None

100294: tenantcapabilitiestestutils: add a missing default case r=ajwerner a=ajwerner

The test should fail if we ever add a new type of capability and use it in the data driven test but don't update the test to handle it.

Epic: none

Follow-up from #100217 (review)

Release note: None

100296: rpc: correctly check for nil before cast r=ajwerner a=andrewbaptist

As part of the fix of #99104, a cast without a nil check was introduced. This PR addresses that by only casting if it is known to be not nil.

Epic: none
Fixes: #100275
Release note: None

Co-authored-by: Marcus Gartner <[email protected]>
Co-authored-by: Herko Lategan <[email protected]>
Co-authored-by: Tobias Grieger <[email protected]>
Co-authored-by: Michael Butler <[email protected]>
Co-authored-by: ajwerner <[email protected]>
Co-authored-by: Andrew Baptist <[email protected]>
knz pushed a commit that referenced this pull request Apr 4, 2023
The test should fail if we ever add a new type of capability and use it in the
data driven test but don't update the test to handle it.

Epic: none

Follow-up from #100217 (review)

Release note: None
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-23.1.x Flags PRs that need to be backported to 23.1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants