ddl: fix dist ddl reorg table range endKey

[jiyfhust]

What problem does this PR solve?

Issue Number: close #45652
Problem Summary:

What is changed and how it works?

As the issue describe:

function getTableRange:
https://github.com/pingcap/tidb/blob/cf7ae4c3b917f81e43cbb2beab44abf841af52cc/ddl/reorg.go#L669C2-L669C2
return [startHandleKey, endHandleKey];
the reorg will use the range above to backfill index data.

Then splitTableRanges will split [startHandleKey, endHandleKey] into small ranges.
https://github.com/pingcap/tidb/blob/cf7ae4c3b917f81e43cbb2beab44abf841af52cc/ddl/scheduler.go#L267C8-L267C8
the small ranges is like [a, b], [b.next, c],[c.next, d].

here use a function getRangeEndKey:
https://github.com/pingcap/tidb/blob/cf7ae4c3b917f81e43cbb2beab44abf841af52cc/ddl/backfilling.go#L562C17-L562C17
change small ranges to [a, b.next), [b.next, c.next), [c.next, d.next).
and it use a "endInclude" to mark case when it is [c.next d].

This pr:

1. fix the getTableRange, return [startHandleKey, endHandleKey.next)
2. split rangs into [a, b.next), [b.next, c.next), [c.next, d.next)
3. don't use getRangeEndKey to change the endKey
4. remove "endInclude"

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No code

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

[ti-chi-bot]

Hi @jiyfhust. Thanks for your PR.

I'm waiting for a pingcap member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[tiprow]

Hi @jiyfhust. Thanks for your PR.

PRs from untrusted users cannot be marked as trusted with /ok-to-test in this repo meaning untrusted PR authors can never trigger tests themselves. Collaborators can still trigger tests on the PR using /test all.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[hawkingrei]

/ok-to-test

[codecov]

Codecov Report

Merging #45679 (11a54e3) into master (e3b95fb) will increase coverage by 0.0045%.
The diff coverage is 85.7142%.

Additional details and impacted files

@@               Coverage Diff                @@
##             master     #45679        +/-   ##
================================================
+ Coverage   73.3543%   73.3589%   +0.0045%     
================================================
  Files          1272       1275         +3     
  Lines        393070     393723       +653     
================================================
+ Hits         288334     288831       +497     
- Misses        86358      86483       +125     
- Partials      18378      18409        +31     

Flag	Coverage Δ
integration	78.1388% <ø> (?)
unit	73.3507% <85.7142%> (-0.0037%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	54.0444% <ø> (ø)
parser	85.0359% <ø> (ø)
br	52.0258% <ø> (-0.0209%)	⬇️

[jiyfhust]

/cc @Defined2014 @zimulala

[zimulala]

Could you briefly describe the fix in the PR description?

[jiyfhust]

Could you briefly describe the fix in the PR description?

/cc @zimulala

[jiyfhust]

/test unit-test

[tiprow]

@jiyfhust: The specified target(s) for /test were not found.
The following commands are available to trigger required jobs:

• /test tiprow_fast_test

Use /test all to run all jobs.

In response to this:

/test unit-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[jiyfhust]

Sorry, just ignore it.

The code annotation mislead me.
https://github.com/tikv/client-go/blob/cbfd64018c2194af29694547bfcad1972cb24512/internal/locate/region_cache.go#L993

// LoadRegionsInKeyRange lists regions in [start_key,end_key].
func (c *RegionCache) LoadRegionsInKeyRange(bo *retry.Backoffer, startKey, endKey []byte) (regions []*Region, err error) {
	var batchRegions []*Region
	for {
		batchRegions, err = c.Bat
....
....

it is not [start_key,end_key], but actually [start_key,end_key)，so it is not a "tikv/client-go" bug which i said regionCache above.

There is no problems now.

[purelind]

/retest

[tangenta]

I just found that index ID is not included :(

t.IndexPrefix() is the key like t[tableID]_i. If we use t.IndexPrefix().PrefixNext() here, endKey will be t[tableID]_j?

As a result, we will scan the index records from all indexes.

@jiyfhust Let me fix this issue in another PR, which will include some tests. Could you revert 47f36bd? Rest LGTM.

[jiyfhust]

@jiyfhust Let me fix this issue in another PR, which will include some tests. Could you revert 47f36bd? Rest LGTM.

@tangenta done.

[jiyfhust]

/retest-required

[zimulala]

LGTM

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tangenta, zimulala

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [tangenta,zimulala]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2023-08-07 05:52:56.874414952 +0000 UTC m=+683060.816763482: ☑️ agreed by tangenta.
• 2023-08-07 06:54:01.99353469 +0000 UTC m=+686725.935883221: ☑️ agreed by zimulala.