Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kvserver: fix TestStoreRangeSplitAndMergeWithGlobalReads #122710

Merged
merged 5 commits into from
Apr 22, 2024

Conversation

andrewbaptist
Copy link
Collaborator

Directly set the span config for the range under test rather than setting the ZoneConfig and waiting for it to propagate. In addition to simplifying the test it also makes it run faster.

Fixes: #119230

Epic: none

Release note: None

@cockroach-teamcity
Copy link
Member

This change is Reviewable

@andrewbaptist andrewbaptist force-pushed the 2024-04-19-fix-global-reads branch from 9164a4d to bfc308e Compare April 19, 2024 20:57
@andrewbaptist andrewbaptist force-pushed the 2024-04-19-fix-global-reads branch from bfc308e to 07fa6ce Compare April 22, 2024 13:32
@andrewbaptist andrewbaptist marked this pull request as ready for review April 22, 2024 13:33
@andrewbaptist andrewbaptist requested review from a team as code owners April 22, 2024 13:33
Copy link
Collaborator

@arulajmani arulajmani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great!

Reviewed 1 of 1 files at r1, 3 of 3 files at r2, 2 of 2 files at r3, 1 of 1 files at r4, 1 of 1 files at r5, all commit messages.
Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @andrewbaptist)


-- commits line 2 at r1:
IIUC, we're not reverting the entire commit here, are we? We're only reverting the changes to this particular test?

It seems like that from the diff. Could you clarify this in the title?


pkg/spanconfig/testing_knobs.go line 159 at r3 (raw file):

// of DefaultZoneConfigOverride, DefaultSystemZoneConfigOverride,
// OverrideFallbackConf, ConfReaderInterceptor, UseSystemConfigSpanForQueues,
// SpanConfigUpdateInterceptor or SetSpanConfigInterceptor. By using this class

nit: given this is go, let's replace all mentions of class with struct.


pkg/spanconfig/testing_knobs.go line 162 at r3 (raw file):

// as an alternative to the others it mainly avoids any need to depend on
// rangefeed timing to propagate the config change.
type WrappingKVSubscriber struct {

Let's move this into the spanconfigtestutils package. Separately, should we rename this to WrappedKVSubscriber?


pkg/spanconfig/testing_knobs.go line 163 at r4 (raw file):

// rangefeed timing to propagate the config change.
type WrappingKVSubscriber struct {
	Wrapped KVSubscriber

Let's make these private fields and introduce a constructor to initialize the struct instead.


pkg/kv/kvserver/client_split_test.go line 4058 at r5 (raw file):

	ctx := context.Background()
	s := serverutils.StartServerOnly(t, base.TestServerArgs{
		DisableSQLServer: true,

Nice 💯

@andrewbaptist andrewbaptist force-pushed the 2024-04-19-fix-global-reads branch from 07fa6ce to ef7f2a1 Compare April 22, 2024 17:08
Copy link
Collaborator Author

@andrewbaptist andrewbaptist left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! Also thanks for the feedback on the changes!

Reviewable status: :shipit: complete! 0 of 0 LGTMs obtained (waiting on @arulajmani)


-- commits line 2 at r1:

Previously, arulajmani (Arul Ajmani) wrote…

IIUC, we're not reverting the entire commit here, are we? We're only reverting the changes to this particular test?

It seems like that from the diff. Could you clarify this in the title?

Done


pkg/spanconfig/testing_knobs.go line 159 at r3 (raw file):

Previously, arulajmani (Arul Ajmani) wrote…

nit: given this is go, let's replace all mentions of class with struct.

Done


pkg/spanconfig/testing_knobs.go line 162 at r3 (raw file):

Previously, arulajmani (Arul Ajmani) wrote…

Let's move this into the spanconfigtestutils package. Separately, should we rename this to WrappedKVSubscriber?

Moved - I didn't like where it was before, so this is much nicer.


pkg/spanconfig/testing_knobs.go line 163 at r4 (raw file):

Previously, arulajmani (Arul Ajmani) wrote…

Let's make these private fields and introduce a constructor to initialize the struct instead.

Changed. I also made AddOverride a method rather than passing in construction. This helps a bit with adding multiple and does not' require configOverride to be public either. I'm not 100% sure its great since it somewhat implies that you can add an override after construction, but other than a comment I'm not sure the best way to do this. Let me know if you think this could be structured differently.


pkg/kv/kvserver/client_split_test.go line 4058 at r5 (raw file):

Previously, arulajmani (Arul Ajmani) wrote…

Nice 💯

Thanks! I'm hoping we can keep rolling this out to more of the tests. It takes 20-25% off the time to run most of the tests so could be a big win in total. I'm tempted to just try and apply it to "all" the files and see which break and then keep any of the ones that don't. Maybe some friday when I have extra time.

Copy link
Collaborator

@arulajmani arulajmani left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm: :shipit:

Reviewed 5 of 5 files at r6, 3 of 3 files at r7, 3 of 3 files at r8, 3 of 3 files at r9, 2 of 2 files at r10, all commit messages.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @andrewbaptist)


-- commits line 29 at r8:
nit: test struct WrappedKVSubscriber now


pkg/spanconfig/testing_knobs.go line 163 at r4 (raw file):

Previously, andrewbaptist (Andrew Baptist) wrote…

Changed. I also made AddOverride a method rather than passing in construction. This helps a bit with adding multiple and does not' require configOverride to be public either. I'm not 100% sure its great since it somewhat implies that you can add an override after construction, but other than a comment I'm not sure the best way to do this. Let me know if you think this could be structured differently.

I can imagine cases where setting an override (or updating it) after construction could be useful. Previously, we would have to set zone configs and wait for them to reconcile; now, we can just override. I'm pro this change.


pkg/kv/kvserver/client_split_test.go line 4058 at r5 (raw file):

Previously, andrewbaptist (Andrew Baptist) wrote…

Thanks! I'm hoping we can keep rolling this out to more of the tests. It takes 20-25% off the time to run most of the tests so could be a big win in total. I'm tempted to just try and apply it to "all" the files and see which break and then keep any of the ones that don't. Maybe some friday when I have extra time.

Yeah, this is neat -- now we have this testing knob to override specific span configs, there's really no need to go through the whole reconciliation process. Maybe one Friday ;)

The original attempt to fix this test introduced timing issues which
made this test flakey. This commit reverts all the changes to
TestStoreRangeSplitAndMergeWithGlobalReads.

Epic: none
Fixes: cockroachdb#119230

Release note: None
This commit allows the testing StoreKVSubscriberOverride to be made more
flexible. Specifically it passes the original to the test to allow
wrapping it rather than just creating a brand new one. The main benefit
of this change is to allow overriding of a single span rather than some
of the other ways of injecting spans in.

Epic: none

Release note: None
This commit adds a utility test struct WrappedKVSubscriber which can
wrap another KVSubscriber directly and provide targetted overrides. This
approach simplifies the KV testing of altered span configs without
resorting to modifying them through SQL.

Epic: none

Release note: None
Directly set the span config for the range under test rather than
setting the ZoneConfig and waiting for it to propagate. In addition to
simplifying the test it also makes it run faster.

Fixes: cockroachdb#119230

Epic: none

Release note: None
Previously this test required ZoneConfig changes and waited for them to
propagate using rangefeeds. We no longer have that dependency as the
span config is set directly so we can remove any need for SQL and no
longer update the settings for rangefeeds.

Epic: none

Release note: None
@andrewbaptist andrewbaptist force-pushed the 2024-04-19-fix-global-reads branch from ef7f2a1 to d7c87e6 Compare April 22, 2024 20:55
@andrewbaptist
Copy link
Collaborator Author

TFTR! I fixed the commit message.

bors r=arulajmani

@craig craig bot merged commit e215cf4 into cockroachdb:master Apr 22, 2024
21 of 22 checks passed
@andrewbaptist andrewbaptist deleted the 2024-04-19-fix-global-reads branch April 23, 2024 12:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

kv/kvserver: TestStoreRangeSplitAndMergeWithGlobalReads failed
3 participants