encoding: skip utf8 charset validation in some cases

[tangenta]

What problem does this PR solve?

Issue Number: close #31014

Problem Summary:

I suspect that the utf8 validation in common code path is the main reason to cause performance regression, even if a faster version utf8.Valid() is used.

What is changed and how it works?

This PR tries to reduce the string validation as less as possible.

Check List

Tests

• Unit test
• Integration test
• Manual test (add detailed scripts or steps below)
• No code

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

None

[ti-chi-bot]

[REVIEW NOTIFICATION]

This pull request has been approved by:

• Defined2014
• xiongjiwei

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

[sre-bot]

Code Coverage Details: https://codecov.io/github/pingcap/tidb/commit/eea4fd7ef1647df44df1508cf1db787be772c06e

[bb7133]

1. Please add a test case for this function.
2. The comment is still confusing to me at least, what is the case when "utf-8 is treated as binary encoding"?

[tangenta]

1. I think the logic here is simple enough...?
2. Binary encoding is a noop encoding. Different from utf-8 encoding, it means all the methods are trivial, including Transform(), IsValid() and others. The cost should be O(1) for these operations. I am trying to avoid string validation(before parsing or writing result to client) to see if there is any improvement.

[bb7133]

Binary encoding is a noop encoding. Different from utf-8 encoding, it means all the methods are trivial, including Transform(), IsValid() and others. The cost should be O(1) for these operations. I am trying to avoid string validation(before parsing or writing result to client) to see if there is any improvement.

Thanks! I think this can be added to the comment in some way...

@Yui-Song Could you try to validate the result using your benchmark?

[Yui-Song]

Thanks! I think this can be added to the comment in some way...

@Yui-Song Could you try to validate the result using your benchmark?

I have told @tangenta how to validate it easily with Benchbot, a platform provided by QA team to do benchmarks. All the benchmarks the Perf Team run daily could be run with it.

[bb7133]

@Yui-Song For the benchmark today, could you share the profiling files for both 7555536 and this commit?

[tangenta]

/run-build comment=true

[Defined2014]

Could you add some performance result? @tangenta

[tangenta]

@bb7133 @Defined2014 I have compared the performance between e3c56b7(the commit before #30288) and 4f8a041(this PR). Here is the result:

bench_start_time        bench_type      thread      tps_tpm     commit_hash
2021-12-28 15:39:24     shenma          200         5639        4f8a0413d39
2021-12-28 14:34:23     shenma          200         5570        e3c56b75eae

I think the performance regression have been solved by this PR.

[xiongjiwei]

/merge

[ti-chi-bot]

This pull request has been accepted and is ready to merge.

Commit hash: 71e4389

[tangenta]

/run-check_dev_2