statistics: enable the auto-analysis priority queue

[Rustin170506]

What problem does this PR solve?

Issue Number: ref #50132

Problem Summary:

What changed and how does it work?

I have enabled the auto-analysis priority queue by default. Also from this PR, the priority queue starts respecting the auto-analyze time window.

Check List

Tests

• Unit test
• Integration test
• Manual test
• No need to test

  • I checked and no code files have been changed.

Side effects

• Performance regression: Consumes more CPU
• Performance regression: Consumes more Memory
• Breaking backward compatibility

Documentation

• Affects user behaviors
• Contains syntax changes
• Contains variable changes
• Contains experimental features
• Changes MySQL compatibility

Release note

Please refer to Release Notes Language Style Guide to write a quality release note.

None

[codecov]

Codecov Report

Merging #51537 (d187139) into master (f94a6ba) will increase coverage by 2.0573%.
Report is 3 commits behind head on master.
The diff coverage is 48.9361%.

Additional details and impacted files

@@               Coverage Diff                @@
##             master     #51537        +/-   ##
================================================
+ Coverage   70.7917%   72.8491%   +2.0573%     
================================================
  Files          1467       1467                
  Lines        436397     436623       +226     
================================================
+ Hits         308933     318076      +9143     
+ Misses       108129      98577      -9552     
- Partials      19335      19970       +635     

Flag	Coverage Δ
integration	48.8877% <0.0000%> (?)
unit	70.4988% <48.9361%> (-0.0992%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Components	Coverage Δ
dumpling	53.9957% <ø> (ø)
parser	∅ <ø> (∅)
br	51.5373% <ø> (+5.6426%)	⬆️

[Rustin170506]

🔢 Self-check (PR reviewed by myself and ready for feedback.)

[Rustin170506]

Tested locally:

1. start the TiDB cluster
2. check the TiDB version

mysql> select tidb_version();
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| tidb_version()                                                                                                                                                                                                                                                                       |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
| Release Version: v8.0.0-alpha-424-gd18713949c
Edition: Community
Git Commit Hash: d18713949ccbf81ab1b38efa48ea1ca9ee6c98d5
Git Branch: rustin-patch-enable-pq
UTC Build Time: 2024-03-07 06:19:15
GoVersion: go1.21.7
Race Enabled: false
Check Table Before Drop: false
Store: tikv |
+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+
1 row in set (0.00 sec)

3. Run the below python script

import pymysql
import random

# Connect to TiDB
conn = pymysql.connect(host='localhost', port=4000, user='root', passwd='', db='test')
cursor = conn.cursor()

# Create partitioned table
cursor.execute("""
    CREATE TABLE users (
      id INT NOT NULL,
      name VARCHAR(50) NOT NULL,
      age INT NOT NULL,
      country VARCHAR(2) NOT NULL
    )
    PARTITION BY RANGE (age) (
      PARTITION p0 VALUES LESS THAN (20),
      PARTITION p1 VALUES LESS THAN (40),
      PARTITION p2 VALUES LESS THAN (60)
    );
""")

# Generate and insert data
for i in range(2000):
  id = i
  name = 'User {}'.format(i)
  age = random.randint(15, 55)
  country = random.choice(['US', 'CN', 'IN'])

  cursor.execute("INSERT INTO users VALUES (%s, %s, %s, %s)", (id, name, age, country))

conn.commit()

print("Partitioned table created and data inserted successfully!")

conn.close()

4. Check logs:

[2024/03/07 14:27:58.017 +08:00] [INFO] [refresher.go:101] ["Auto analyze triggered"] [category=stats] [job="DynamicPartitionedTableAnalysisJob:\n\tAnalyzeType: analyzeDynamicPartition\n\tPartitions: p1, p2\n\tPartitionIndexes: map[]\n\tSchema: test\n\tGlobal Table: users\n\tGlobal TableID: 104\n\tTableStatsVer: 2\n\tChangePercentage: 1.00\n\tTableSize: 14052.00\n\tLastAnalysisDuration: 30m0s\n\tWeight: 1.3791\n"]
[2024/03/07 14:27:58.049 +08:00] [INFO] [save.go:200] ["incrementally update modifyCount"] [category=stats] [tableID=106] [curModifyCnt=3898] [results.BaseModifyCnt=3898] [modifyCount=0]
[2024/03/07 14:27:58.049 +08:00] [INFO] [save.go:222] ["directly update count"] [category=stats] [tableID=106] [results.Count=3898] [count=3898]
[2024/03/07 14:27:58.169 +08:00] [INFO] [analyze.go:745] ["analyze table `test`.`users` has finished"] [partition=p1] ["job info"="auto analyze table all columns with 256 buckets, 500 topn, 1 samplerate"] ["start time"=2024/03/07 14:27:58.027 +08:00] ["end time"=2024/03/07 14:27:58.168 +08:00] [cost=140.721792ms] ["sample rate reason"="use min(1, 110000/3898) as the sample-rate=1"]
[2024/03/07 14:27:58.170 +08:00] [INFO] [save.go:200] ["incrementally update modifyCount"] [category=stats] [tableID=107] [curModifyCnt=3128] [results.BaseModifyCnt=3128] [modifyCount=0]
[2024/03/07 14:27:58.170 +08:00] [INFO] [save.go:222] ["directly update count"] [category=stats] [tableID=107] [results.Count=3128] [count=3128]
[2024/03/07 14:27:58.264 +08:00] [INFO] [analyze.go:745] ["analyze table `test`.`users` has finished"] [partition=p2] ["job info"="auto analyze table all columns with 256 buckets, 500 topn, 1 samplerate"] ["start time"=2024/03/07 14:27:58.048 +08:00] ["end time"=2024/03/07 14:27:58.262 +08:00] [cost=214.074167ms] ["sample rate reason"="use min(1, 110000/3128) as the sample-rate=1"]

[ti-chi-bot]

[LGTM Timeline notifier]

Timeline:

• 2024-03-07 07:11:51.483623196 +0000 UTC m=+323938.505869584: ☑️ agreed by qw4990.
• 2024-03-07 07:27:51.043259212 +0000 UTC m=+324898.065505599: ☑️ agreed by AilinKid.

[easonn7]

/approve

[ti-chi-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: AilinKid, easonn7, qw4990

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [AilinKid,qw4990]
• pkg/sessionctx/variable/OWNERS [easonn7]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ti-chi-bot]

@hi-rustin: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
idc-jenkins-ci-tidb/check_dev_2	d187139	link	unknown	/test check-dev2

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[Rustin170506]

/retest