*: support paging protocol on unistore

[tiancaiamao]

What problem does this PR solve?

Issue Number: close #35243 #35242

Problem Summary:

I'm try to support coprocessor paging protocol on unistore, and found those issues:

• There are several kinds of coprocessor request, including DAG and others. admin check table/index analyze are not using DAG so they don't support paging
• Find bug In dynamic partition mode distsql request key ranges is not sorted #35242 that for partition table on dynamic pruning mode, the key ranges is not sorted. So rebuilding cop tasks and calculate remain key ranges get wrong result.
• cop cache for paging is not well supported.
• When enable/disable paging, the explain result (related to IndexLookup) is different, so some test cases need to be updated.
• QueryFeedback is broken when coprocessor paging is enabled.
• mocktikv doesn't support paging, and some test cases are still using mocktikv, so if we enable paging by default, those test cases needs update
• Read from INFORMATION_SCHEMA.CLUSTER_XXX table use coprocessor request to query tidb instances, and tidb coprocessor doesn't support paging

What is changed and how it works?

You can ignore all the xxx_test.go files and just review cop_handler.go / mpp_exec.go
The actual changes are there.

In brief, I update the paging in tablescan and indexscan's next() function to keep the paging key range update to date.
And in the out most loop, check if the chunk pages are enough to break the next() looping.

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)

I make session variable tidb_enable_paging default value to On and run all the unit tests to check everything go work well.

• No code

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

[ti-chi-bot]

[REVIEW NOTIFICATION]

This pull request has been approved by:

• XuHuaiyu
• wshwsh12

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

[sre-bot]

Code Coverage Details: https://codecov.io/github/pingcap/tidb/commit/827ef7b9b3182a8cfb4214c9becfdec070eb93e5

[tiancaiamao]

/run-unit-test

[tiancaiamao]

PTAL @XuHuaiyu @wshwsh12

[tiancaiamao]

/run-unit-test
/run-realtikv-test

[tiancaiamao]

/run-unit-test

[hawkingrei]

/run-unit-test

[tiancaiamao]

/run-unit-test

[tiancaiamao]

PTAL @XuHuaiyu @wshwsh12

[ti-chi-bot]

@tiancaiamao: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[wshwsh12]

LGTM

[XuHuaiyu]

It seems that ReqTypeAnalyze will not use paging. We do not need to set enable_paging = false for the analyze tests.

[tiancaiamao]

paging is a property of the kv request,
kv request can be one of the following kinds:

// ReqTypes.
const (
	ReqTypeSelect   = 101
	ReqTypeIndex    = 102
	ReqTypeDAG      = 103
	ReqTypeAnalyze  = 104
	ReqTypeChecksum = 105

	ReqSubTypeBasic      = 0
	ReqSubTypeDesc       = 10000
	ReqSubTypeGroupBy    = 10001
	ReqSubTypeTopN       = 10002
	ReqSubTypeSignature  = 10003
	ReqSubTypeAnalyzeIdx = 10004
	ReqSubTypeAnalyzeCol = 10005
)

As you can see, there are ReqTypeDAG and ReqTypeAnalyze, and also ReqTypeChecksum for admin check table ... for now, we just support ReqTypeDAG, so all other cases req.Paging = false is set.

A possible change is that we can move the paging property to coprocessor.Request.
That change would make it clear that paging is a property of ReqTypeDAG (or coprocessor.Request to be more specific), not a property kv.Request.
The drawback of the change is, if one day, we need paging protocol also for Analyze/Checksum, we need to add the property to kv.Request...

[tiancaiamao]

/merge

[ti-chi-bot]

This pull request has been accepted and is ready to merge.

Commit hash: eae342b

[sre-bot]

TiDB MergeCI notify

✅ Well Done! New fixed [1] after this pr merged.

CI Name	Result	Duration	Compare with Parent commit
idc-jenkins-ci-tidb/integration-common-test	🔴 failed 2, success 9, total 11	19 min	Existing failure
idc-jenkins-ci-tidb/common-test	✅ all 12 tests passed	9 min 37 sec	Fixed
idc-jenkins-ci/integration-cdc-test	🟢 all 35 tests passed	25 min	Existing passed
idc-jenkins-ci-tidb/integration-ddl-test	🟢 all 6 tests passed	6 min 16 sec	Existing passed
idc-jenkins-ci-tidb/sqllogic-test-2	🟢 all 28 tests passed	5 min 24 sec	Existing passed
idc-jenkins-ci-tidb/tics-test	🟢 all 1 tests passed	5 min 14 sec	Existing passed
idc-jenkins-ci-tidb/sqllogic-test-1	🟢 all 26 tests passed	5 min 1 sec	Existing passed
idc-jenkins-ci-tidb/integration-compatibility-test	🟢 all 1 tests passed	3 min 16 sec	Existing passed
idc-jenkins-ci-tidb/mybatis-test	🟢 all 1 tests passed	3 min 4 sec	Existing passed
idc-jenkins-ci-tidb/plugin-test	🟢 build success, plugin test success	4min	Existing passed