Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

kvserver: add TestFlowControlSendQueueRangeSplitMerge test #136258

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

kvoli
Copy link
Collaborator

@kvoli kvoli commented Nov 26, 2024

First commit from #136330.


Add a new rac2 flow control integration test,
TestFlowControlSendQueueRangeSplitMerge.

This test takes the following steps:

-- We will exhaust the tokens across all streams while admission is blocked on
-- n3, using a single 4 MiB (deduction, the write itself is small) write. Then,
-- we will write a 1 MiB put to the range, split it, write a 1 MiB put to the
-- RHS range, merge the ranges, and write a 1 MiB put to the merged range. We
-- expect that at each stage where a send queue develops n1->s3, the send queue
-- will be flushed by the range merge and range split range operations.

Part of: #132614
Release note: None

Copy link

blathers-crl bot commented Nov 26, 2024

Your pull request contains more than 1000 changes. It is strongly encouraged to split big PRs into smaller chunks.

🦉 Hoot! I am a Blathers, a bot for CockroachDB. My owner is dev-inf.

@kvoli kvoli self-assigned this Nov 26, 2024
@cockroach-teamcity
Copy link
Member

This change is Reviewable

@kvoli kvoli force-pushed the 241126.rac2-send-queue-test-split-etc branch from c45a091 to fc7bd4d Compare November 27, 2024 20:06
@kvoli kvoli marked this pull request as ready for review November 27, 2024 20:27
@kvoli kvoli requested review from a team as code owners November 27, 2024 20:27
@kvoli
Copy link
Collaborator Author

kvoli commented Nov 27, 2024

Should be good for review, ignoring the first commit (I won't merge this until after we merge the split-out fixes from #136077).

ptal @sumeerbhola.

Copy link
Collaborator

@sumeerbhola sumeerbhola left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:lgtm:

Reviewed 2 of 2 files at r2.
Reviewable status: :shipit: complete! 1 of 0 LGTMs obtained (waiting on @kvoli)


pkg/kv/kvserver/flow_control_integration_test.go line 5606 at r2 (raw file):

	h.comment(`(Sending 1 MiB put request to post-split RHS range)`)
	h.put(ctx, k, 1, admissionpb.NormalPri)

should this be writing to right.StartKey since that is the start of the RHS. k seems to be the LHS.

Perhaps we should write to both LHS and RHS, to make sure everything is working as expected. The SubsumeRequest force-flush will flush on the RHS, and the txn commit (whose txn record is on the LHS) should cause a force-flush on the LHS.

- ReplicatedEvalResult.DoTimelyApplicationToAllReplicas is added and set
  in splitTriggerHelper, MergeTrigger, Migrate, Subsume.
- The previous setting to true is gated on cluster version
  V25_1_AddRangeForceFlushKey.
- This causes ReplicaState.ForceFlushIndex to be set, and
  RangeForceFlushKey (a replicated range-id local key) to be written when
  applying the corresponding batch to the state machine. The index is set
  to the index of the entry being applied, and is monotonically
  increasing.
- replica_rac2.Processor and rac2.RangeController have
  existing ForceFlushIndexChangedLocked methods that are called whenever
  the Replica sees a change in the force-flush-index.

Fixes cockroachdb#135601

Epic: CRDB-37515

Release note: None
Add a new rac2 flow control integration test,
`TestFlowControlSendQueueRangeSplitMerge`.

This test takes the following steps:

```sql
-- We will exhaust the tokens across all streams while admission is blocked on
-- n3, using a single 4 MiB (deduction, the write itself is small) write. Then,
-- we will write a 1 MiB put to the range, split it, write a 1 MiB put to the
-- RHS range, merge the ranges, and write a 1 MiB put to the merged range. We
-- expect that at each stage where a send queue develops n1->s3, the send queue
-- will be flushed by the range merge and range split range operations.
```

Part of: cockroachdb#132614
Release note: None
@kvoli kvoli force-pushed the 241126.rac2-send-queue-test-split-etc branch from 227bdb1 to f25fb10 Compare December 2, 2024 22:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants