[WIP]: add v8.5.0 release notes

releases/release-8.5.0.md

@@ -0,0 +1,279 @@
+---
+title: TiDB 8.5.0 Release Notes
+summary: Learn about the new features, compatibility changes, improvements, and bug fixes in TiDB 8.5.0.
+---
+
+# TiDB 8.5.0 Release Notes
+
+<EmailSubscriptionWrapper />
+
+Release date: xx xx, 2024
+
+TiDB version: 8.5.0
+
+Quick access: [Quick start](https://docs.pingcap.com/tidb/v8.5/quick-start-with-tidb) | [Production deployment](https://docs.pingcap.com/tidb/v8.5/production-deployment-using-tiup)
+
+TiDB 8.5.0 is a Long-Term Support Release (LTS).
+
+Compared with the previous LTS 8.1.0, 8.5.0 includes new features, improvements, and bug fixes released in [8.2.0-DMR](/releases/release-8.2.0.md), [8.3.0-DMR](/releases/release-8.3.0.md), and [8.4.0-DMR](/releases/release-8.4.0.md). When you upgrade from 8.1.x to 8.5.0, you can download the [TiDB Release Notes PDF](https://download.pingcap.org/tidb-v8.1-to-v8.5-en-release-notes.pdf) to view all release notes between the two LTS versions. The following table lists some highlights from 8.1.0 to 8.5.0:
+
+<!--Highlights table: ToDo-->
+
+## Feature details
+
+### Scalability
+
+* The Schema cache memory limit feature is now Generally Available (GA), reducing memory usage in large-scale data scenarios.  [#50959](https://github.com/pingcap/tidb/issues/50959) @[tiancaiamao](https://github.com/tiancaiamao) @[wjhuang2016](https://github.com/wjhuang2016) @[gmhdbjd](https://github.com/gmhdbjd) @[tangenta](https://github.com/tangenta) tw@hfxsd <!--1976-->
+
+    In some SaaS scenarios, when the number of tables reaches hundreds of thousands or even millions, the schema meta can consume significant memory. Enabling this feature allows the system to use the LRU algorithm to cache and evict the corresponding schema meta information, effectively reducing memory usage. Starting from v8.4, this feature is enabled by default with a default value of 512MiB, and users can adjust it as needed through the variable [tidb_schema_cache_size](/system-variables#tidb_schema_cache_size-new-in-v800).
+
+    For more information, see [Documentation](link).
+
+* Use the Active PD Follower feature to enhance the scalability of PD's Region information query service (General Availability) [#7431](https://github.com/tikv/pd/issues/7431) @[okJiang](https://github.com/okJiang)
+
+    In a TiDB cluster with a large number of Regions, the PD leader might experience high CPU load due to the increased overhead of handling heartbeats and scheduling tasks. If the cluster has many TiDB instances, and there is a high concurrency of requests for Region information, the CPU pressure on the PD leader increases further and might cause PD services to become unavailable.
+
+    To ensure high availability and also enhance the scalability of PD's Region information query service. You can enable the Active PD Follower feature by setting the system variable [`pd_enable_follower_handle_region`](/system-variables.md#pd_enable_follower_handle_region-new-in-v760) to `ON`. After this feature is enabled, TiDB evenly distributes Region information requests to all PD servers, and PD followers can also handle Region requests, thereby reducing the CPU pressure on the PD leader.
+
+    For more information, see [documentation](/tune-region-performance.md#use-the-active-pd-follower-feature-to-enhance-the-scalability-of-pds-region-information-query-service)
+
+### Performance
+
+* Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** <!--1234-->
+
+    Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well.
+
+    For more information, see [Documentation](link).
+
+* The database and table creation acceleration feature is now Generally Available (GA), significantly reducing the time required for data migration and cluster initialization. [#50052](https://github.com/pingcap/tidb/issues/50052) @[D3Hunter](https://github.com/D3Hunter) @[gmhdbjd](https://github.com/gmhdbjd) tw@Oreoxmt <!--1977-->
+
+    In v8.0.0, the system variable [tidb_enable_fast_create_table](/system-variables#tidb_enable_fast_create_table-new-in-v800) was introduced to improve the performance of batch database and table creation. During data migration and cluster initialization, it enables the rapid creation of tables at the million-scale level, significantly reducing the time required.
+
+    For more information, see [Documentation](link).
+
+### Reliability
+
+* Enabling rate limiter can protect PD from being crash under a large number of sudden requests and improve the stability of PD [#5739](https://github.com/tikv/pd/issues/5739) @[rleungx](https://github.com/rleungx)
+
+    You can adjust the rate limiter configuration through pd-ctl. 
+
+    For more information, see [Documentation](/stable/pd-control.md).
+
+### Availability
+
+* Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** <!--1234-->
+
+    Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well.
+
+    For more information, see [Documentation](link).
+
+### SQL
+
+* Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** <!--1234-->
+
+    Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well.
+
+    For more information, see [Documentation](link).
+
+* The Foreign Key feature is now Generally Available (GA) [#36982](https://github.com/pingcap/tidb/issues/36982) @[YangKeao](https://github.com/YangKeao) @[crazycs520](https://github.com/crazycs520) tw@lilin90 <!--1894-->
+
+    The Foreign Key feature is now GA, enabling the use of foreign key constraints to enhance data consistency and integrity. Users can easily create foreign key constraints between tables, with support for cascading updates and deletions, making data management more convenient. This feature provides better support for applications with complex data constraints.
+
+    For more information, see [Documentation](link).
+
+* Introduction of ADMIN ALTER DDL JOBS syntax to support online modification of DDL job variables [#57229](https://github.com/pingcap/tidb/issues/57229) @[fzzf678](https://github.com/fzzf678) @[tangenta](https://github.com/tangenta) tw@hfxsd <!--2016--> 
+
+    Starting from v8.3, it is now possible to set the variables [tidb_ddl_reorg_batch_size](/system-variables#tidb_ddl_reorg_batch_size)  and [tidb_ddl_reorg_worker_cnt](/system-variables#tidb_ddl_reorg_worker_cnt) at the session level. As a result, setting these two variables globally no longer affects all running DDL jobs. To modify the values of these variables, the DDL job must first be canceled, the variables adjusted, and then resubmit the job.
+
+    Since v8.5, the ADMIN ALTER DDL JOBS syntax has been introduced, allowing online adjustment of variable values for specific DDL jobs. This enables flexible balancing of resource consumption and performance, with changes limited to an individual job, making the impact more controllable. For example:
+
+    - `ADMIN ALTER DDL JOBS job_id THREAD = 8;` — Adjusts the `tidb_ddl_reorg_worker_cnt` for the specified DDL task.
+    - `ADMIN ALTER DDL JOBS job_id BATCH_SIZE = 256;` — Adjusts the `tidb_ddl_reorg_batch_size` for the specified DDL task.
+    - `ADMIN ALTER DDL JOBS job_id MAX_WRITE_SPEED = '200MiB';` — Adjusts the write traffic size for index data on each TiKV node.
+
+  For more information, see [Documentation](link).
+
+### DB operations
+
+* Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** <!--1234-->
+
+    Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well.
+
+    For more information, see [Documentation](link).
+
+### Observability
+
+* Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** <!--1234-->
+
+    Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well.
+
+    For more information, see [Documentation](link).
+
+### Security
+
+* BR supports client-side encryption of log backup data (GA) [#56433](https://github.com/pingcap/tidb/issues/56433) @[Tristan1900](https://github.com/Tristan1900) tw@qiancai <!--1998-->
+
+    TiDB v8.4.0 introduced an experimental feature to encrypt, on the client side, log backup data. Starting from v8.5.0, this feature is now Generally Avaialble. Before uploading log backup data to your backup storage, you can encrypt the backup data to ensure its security via one of the following methods:
+
+    - Encrypt using a custom fixed key
+    - Encrypt using a master key stored on a local disk
+    - Encrypt using a master key managed by a Key Management Service (KMS)
+
+  For more information, see [documentation](/br/br-pitr-manual.md#encrypt-the-log-backup-data).
+
+### Data migration
+
+* Placeholder for feature summary [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link) **tw@xxx** <!--1234-->
+
+    Provide a concise overview of what the feature is, the value it offers to users, and include a brief sentence on how to use it effectively. If there are any particularly important aspects of this feature, be sure to mention them as well.
+
+    For more information, see [Documentation](link).
+
+## Compatibility changes
+
+> **Note:**
+>
+> This section provides compatibility changes you need to know when you upgrade from v8.1.0 to the current version (v8.2.0). If you are upgrading from v8.0.0 or earlier versions to the current version, you might also need to check the compatibility changes introduced in intermediate versions.
+
+### Behavior changes
+
+### MySQL compatibility
+
+### System variables
+
+| Variable name | Change type | Description |
+|--------|------------------------------|------|
+|tidb_ddl_reorg_max_write_speed  | Newly added |Used to control the speed at which TiDB writes index data to a single TiKV node. For example, setting the value to 200 MiB limits the maximum write speed to 200 MiB/s.  |
+|  |  |  |
+|  |  |  |
+
+### Configuration parameters
+
+| Configuration file or component | Configuration parameter | Change type | Description |
+| -------- | -------- | -------- | -------- |
+|  |  |  |  |
+|  |  |  |  |
+|  |  |  |  |
+
+### System tables
+
+### Other changes
+
+## Offline package changes
+
+## Removed features
+
+## Deprecated features
+
+The following features are planned for deprecation in future versions:
+
+* TiDB introduces the system variable [`tidb_enable_auto_analyze_priority_queue`](/system-variables.md#tidb_enable_auto_analyze_priority_queue-new-in-v800), which controls whether priority queues are enabled to optimize the ordering of tasks that automatically collect statistics. In future releases, the priority queue will be the only way to order tasks for automatically collecting statistics, so this system variable will be deprecated.
+* TiDB introduces the system variable [`tidb_enable_async_merge_global_stats`](/system-variables.md#tidb_enable_async_merge_global_stats-new-in-v750) in v7.5.0. You can use it to set TiDB to use asynchronous merging of partition statistics to avoid OOM issues. In future releases, partition statistics will be merged asynchronously, so this system variable will be deprecated.
+* It is planned to redesign [the automatic evolution of execution plan bindings](/sql-plan-management.md#baseline-evolution) in subsequent releases, and the related variables and behavior will change.
+* In v8.0.0, TiDB introduces the [`tidb_enable_parallel_hashagg_spill`](/system-variables.md#tidb_enable_parallel_hashagg_spill-new-in-v800) system variable to control whether TiDB supports disk spill for the concurrent HashAgg algorithm. In future versions, the [`tidb_enable_parallel_hashagg_spill`](/system-variables.md#tidb_enable_parallel_hashagg_spill-new-in-v800) system variable will be deprecated.
+* The TiDB Lightning parameter [`conflict.max-record-rows`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task) is planned for deprecation in a future release and will be subsequently removed. This parameter will be replaced by [`conflict.threshold`](/tidb-lightning/tidb-lightning-configuration.md#tidb-lightning-task), which means that the maximum number of conflicting records is consistent with the maximum number of conflicting records that can be tolerated in a single import task.
+* Starting from v6.3.0, partitioned tables use [dynamic pruning mode](/partitioned-table.md#dynamic-pruning-mode) by default. Compared with static pruning mode, dynamic pruning mode supports features such as IndexJoin and plan cache with better performance. Therefore, static pruning mode will be deprecated.
+
+## Improvements
+
++ TiDB
+
+    - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+    - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+
++ TiKV
+
+    - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+    - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+
++ PD
+
+    - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+    - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+
++ TiFlash
+
+    - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+    - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+
++ Tools
+
+    + Backup & Restore (BR)
+
+        - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+        - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+
+    + TiCDC
+
+        - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+        - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+
+    + TiDB Data Migration (DM)
+
+        - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+        - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+
+    + TiDB Lightning
+
+        - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+        - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+
+    + TiUP
+
+        - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+        - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+
+## Bug fixes
+
++ TiDB
+
+    - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+    - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+
++ TiKV
+
+    - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+    - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+
++ PD
+
+    - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+    - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+
++ TiFlash
+
+    - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+    - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+
++ Tools
+
+    + Backup & Restore (BR)
+
+        - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+        - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+
+    + TiCDC
+
+        - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+        - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+
+    + TiDB Data Migration (DM)
+
+        - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+        - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+
+    + TiDB Lightning
+
+        - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+        - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+
+    + TiUP
+
+        - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+        - note [#Issue-number](issue-link) @[Contributor-GitHub-ID](id-link)
+
+## Contributors
+
+We would like to thank the following contributors from the TiDB community:
+
+- [Contributor-GitHub-ID](id-link)