From 79dde519dc18835e325daec93b52fabfc8159a1a Mon Sep 17 00:00:00 2001 From: ti-srebot <66930949+ti-srebot@users.noreply.github.com> Date: Tue, 14 Sep 2021 19:16:46 +0800 Subject: [PATCH 01/15] releases: add TiDB 4.0.15-19-16-44 release notes * releases: add TiDB 4.0.15-19-16-44 release notes * Update releases/release-4.0.15-19-16-44.md --- releases/release-4.0.15-19-16-44.md | 143 ++++++++++++++++++++++++++++ 1 file changed, 143 insertions(+) create mode 100644 releases/release-4.0.15-19-16-44.md diff --git a/releases/release-4.0.15-19-16-44.md b/releases/release-4.0.15-19-16-44.md new file mode 100644 index 0000000000000..04d54bfdb875a --- /dev/null +++ b/releases/release-4.0.15-19-16-44.md @@ -0,0 +1,143 @@ +--- +title: TiDB 4.0.15 Release Notes +category: Releases +--- + + + +# TiDB 4.0.15 Release Notes + +Release Date: September 14, 2021 + +TiDB version: 4.0.15 + +## __unsorted + ++ PingCAP/TiDB + + - executor: fix unexpected behavior when casting invalid string to date [#27935](https://github.com/pingcap/tidb/pull/27935) + - fix expression rewrite makes between expr infers wrong collation. [#27851](https://github.com/pingcap/tidb/pull/27851) + - make `group_concat` function consider the collation [#27835](https://github.com/pingcap/tidb/pull/27835) + - Fix bug that count disctinct on multi-columns return wrong result when new collation is on. [#27830](https://github.com/pingcap/tidb/pull/27830) + - fix wrong selection push down when having above agg [#27741](https://github.com/pingcap/tidb/pull/27741) + - expression: fix extract bug when argument is a negative duration [#27369](https://github.com/pingcap/tidb/pull/27369) + - planner: add missing column for Apply convert to Join [#27282](https://github.com/pingcap/tidb/pull/27282) + - The undocumented `/debug/sub-optimal-plan` HTTP API has been removed. [#27264](https://github.com/pingcap/tidb/pull/27264) + - fix range building for binary literal [#26455](https://github.com/pingcap/tidb/pull/26455) + - Revert #21045 to avoid #24326. [#26240](https://github.com/pingcap/tidb/pull/26240) + + ++ TiKV/TiKV + + - ```release-note [#10917](https://github.com/tikv/tikv/pull/10917) + - RaftStore Snapshot GC fix: fix the issue that snapshot GC missed GC snapshot files when there's one snapshot file failed to be GC-ed. [#10871](https://github.com/tikv/tikv/pull/10871) + - - TiKV coprocessor slow log will only consider time spent on processing the request. +- Drop log instead of blocking threads when slogger thread is overloaded and queue is filled up. [#10863](https://github.com/tikv/tikv/pull/10863) + - ```release-note [#10781](https://github.com/tikv/tikv/pull/10781) + - Support changing CDC configs dynamically [#10684](https://github.com/tikv/tikv/pull/10684) + - Reduce resolved ts message size to save network bandwidth. [#10677](https://github.com/tikv/tikv/pull/10677) + - Check stale file information from encryption file dict [#10598](https://github.com/tikv/tikv/pull/10598) + - fix frequently reconnecting pd client [#9818](https://github.com/tikv/tikv/pull/9818) + + ++ PingCAP/TiFlash + + - Please add a release note, or a 'None' if it is not needed. [#3018](https://github.com/pingcap/tics/pull/3018) + + ++ PD + + - None. [#3974](https://github.com/tikv/pd/pull/3974) + - Improved the performance of synchronizing Region information between PDs. [#3932](https://github.com/tikv/pd/pull/3932) + + ++ Tools + + + PingCAP/BR + + - ```release-note [#1433](https://github.com/pingcap/br/pull/1433) + - ```release-note [#1432](https://github.com/pingcap/br/pull/1432) + - Restore many small tables would be faster now. [#1429](https://github.com/pingcap/br/pull/1429) + - Merged the rebase auto id operation into create table. [#1424](https://github.com/pingcap/br/pull/1424) + - Expression index and index depending on virtual generated columns are now valid. Previously these indices are broken when importing through Lightning local or importer backend. [#1418](https://github.com/pingcap/br/pull/1418) + - fix the bug that the average speed isn't accurate in backup and restore [#1410](https://github.com/pingcap/br/pull/1410) + - [#1346](https://github.com/pingcap/br/pull/1346) + + + + PingCAP/Dumpling + + - fix pending on show table status in some mysql version [#332](https://github.com/pingcap/dumpling/pull/332) + - Support for backing up MySQL compatible databases that don't support START TRANSACTION ... WITH CONSISTENT SNAPSHOT +- Support for backing up MySQL compatible databases that don't support SHOW CREATE TABLE [#329](https://github.com/pingcap/dumpling/pull/329) + + + + PingCAP/TiCDC + + - Fix json encoding could panic when processing a string type value in some cases. [#2781](https://github.com/pingcap/ticdc/pull/2781) + - Fix a bug that multiple processors could write the same table when this table is re-scheduling [#2727](https://github.com/pingcap/ticdc/pull/2727) + - Fix OOM when TiCDC captures too many regions [#2723](https://github.com/pingcap/ticdc/pull/2723) + - Fix gRPC keepalive error when memory pressure is high. [#2718](https://github.com/pingcap/ticdc/pull/2718) + - Optimize memory management when unified sorter is using memory to sort. [#2710](https://github.com/pingcap/ticdc/pull/2710) + - Please add a release note. +If you don't think this PR needs a release note then fill it with `None`. [#2681](https://github.com/pingcap/ticdc/pull/2681) + - Fix a bug that causes TiCDC to panic on an unsigned tinyint [#2654](https://github.com/pingcap/ticdc/pull/2654) + - fix memory leak which may happen in create new changefeed. [#2623](https://github.com/pingcap/ticdc/pull/2623) + - Fix open protocol, don't output an empty value when there is no change in one transaction. [#2619](https://github.com/pingcap/ticdc/pull/2619) + - Fixed a bug in DDL handling when the owner restarts. [#2609](https://github.com/pingcap/ticdc/pull/2609) + - Prohibit operating TiCDC clusters across major and minor versions [#2601](https://github.com/pingcap/ticdc/pull/2601) + - Fix a bug in metadata management [#2557](https://github.com/pingcap/ticdc/pull/2557) + - Add a global gRPC connection pool and share gRPC connections among kv clients. [#2531](https://github.com/pingcap/ticdc/pull/2531) + - `None`. [#2519](https://github.com/pingcap/ticdc/pull/2519) + - Fix a bug that multiple processors could write the same table when this table is re-scheduling [#2495](https://github.com/pingcap/ticdc/pull/2495) + - Optimize workerpool for fewer goroutines when concurrency is high. [#2486](https://github.com/pingcap/ticdc/pull/2486) + - Fix a bug that owner could meet ErrSchemaStorageTableMiss error and reset a changefeed by accident. [#2457](https://github.com/pingcap/ticdc/pull/2457) + - fix the bug that changefeed cannot be removed if meet GcTTL Exceeded Error [#2455](https://github.com/pingcap/ticdc/pull/2455) + - fix outdated capture info may appear in capture list command [#2447](https://github.com/pingcap/ticdc/pull/2447) + - fix a bug where synchronizing large tables to cdclog failed. [#2444](https://github.com/pingcap/ticdc/pull/2444) + - Reduce goroutine usage when a table's region transfer away from a TiKV node [#2376](https://github.com/pingcap/ticdc/pull/2376) + - Remove file sorter. [#2325](https://github.com/pingcap/ticdc/pull/2325) + - Cleanup changefeed metrics when changefeed is removed. +- Cleanup processor metrics when processor exits. [#2313](https://github.com/pingcap/ticdc/pull/2313) + - puller,mounter,processor: always pull the old value internally [#2304](https://github.com/pingcap/ticdc/pull/2304) + - Fix minor runtime panic risk [#2298](https://github.com/pingcap/ticdc/pull/2298) + - Fix potential DDL loss when owner crashes while executing DDL [#2291](https://github.com/pingcap/ticdc/pull/2291) + - Don't resolve lock immediately after a region is initialized. [#2264](https://github.com/pingcap/ticdc/pull/2264) + - Fix deadlock [#2017](https://github.com/pingcap/ticdc/pull/2017) + + +## Improvements + ++ PingCAP/TiDB + + - Trigger auto-analyze based on histogram row count [#26706](https://github.com/pingcap/tidb/pull/26706) + + ++ TiKV/TiKV + + - - separate read write ready to reduce read latency [#10619](https://github.com/tikv/tikv/pull/10619) + + +## Bug Fixes + ++ PingCAP/TiDB + + - Fix wrong charset and collation for case when function [#26671](https://github.com/pingcap/tidb/pull/26671) + - Fix the issue that greatest(datetime) union null returns empty string [#26564](https://github.com/pingcap/tidb/pull/26564) + - Fix the issue that sometimes fails to send requests if there are tombstone stores. [#25849](https://github.com/pingcap/tidb/pull/25849) + + ++ PingCAP/TiFlash + + - Fix the potential issue of data inconsistency after crashes when deployed on multi-disks [#2774](https://github.com/pingcap/tics/pull/2774) + - Fix the bug that filters unexpected data if the query contains "constant" "<"/"<="/">"/">=" "column" [#2634](https://github.com/pingcap/tics/pull/2634) + - Fix inaccurate store size on tiflash metric under heavy write scenario [#2596](https://github.com/pingcap/tics/pull/2596) + - Fix the bug that TiFlash can not restore data under some situations when deployed on multi disks [#2433](https://github.com/pingcap/tics/pull/2433) + - Fix the potential issue that TiFlash cannot GC the delta data after running for a long time [#2335](https://github.com/pingcap/tics/pull/2335) + + ++ PD + + - Fix the bug that PD would not fix down-peer in time. [#4081](https://github.com/tikv/pd/pull/4081) + - Fix the bug that PD may panic during scaling out TiKV. [#3909](https://github.com/tikv/pd/pull/3909) + + From c50fc0f29f4ff2d5f64679e15065f373a9390865 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Wed, 15 Sep 2021 10:46:24 +0800 Subject: [PATCH 02/15] add toc --- TOC.md | 3 +- ...e-4.0.15-19-16-44.md => release-4.0.15.md} | 268 ++++++++---------- releases/release-notes.md | 1 + 3 files changed, 128 insertions(+), 144 deletions(-) rename releases/{release-4.0.15-19-16-44.md => release-4.0.15.md} (79%) diff --git a/TOC.md b/TOC.md index 9640c57529fc5..54b08d641e4ab 100644 --- a/TOC.md +++ b/TOC.md @@ -557,7 +557,7 @@ + [All Releases](/releases/release-notes.md) + v5.2 + [5.2.1](/releases/release-5.2.1.md) - + [5.2.0](/releases/release-5.2.0.md) + + [5.2.0](/releases/release-5.2.0.md) + v5.1 + [5.1.1](/releases/release-5.1.1.md) + [5.1.0](/releases/release-5.1.0.md) @@ -568,6 +568,7 @@ + [5.0 GA](/releases/release-5.0.0.md) + [5.0.0-rc](/releases/release-5.0.0-rc.md) + v4.0 + + [4.0.15](/releases/release-4.0.15.md) + [4.0.14](/releases/release-4.0.14.md) + [4.0.13](/releases/release-4.0.13.md) + [4.0.12](/releases/release-4.0.12.md) diff --git a/releases/release-4.0.15-19-16-44.md b/releases/release-4.0.15.md similarity index 79% rename from releases/release-4.0.15-19-16-44.md rename to releases/release-4.0.15.md index 04d54bfdb875a..2157ffe6ed0f5 100644 --- a/releases/release-4.0.15-19-16-44.md +++ b/releases/release-4.0.15.md @@ -1,143 +1,125 @@ ---- -title: TiDB 4.0.15 Release Notes -category: Releases ---- - - - -# TiDB 4.0.15 Release Notes - -Release Date: September 14, 2021 - -TiDB version: 4.0.15 - -## __unsorted - -+ PingCAP/TiDB - - - executor: fix unexpected behavior when casting invalid string to date [#27935](https://github.com/pingcap/tidb/pull/27935) - - fix expression rewrite makes between expr infers wrong collation. [#27851](https://github.com/pingcap/tidb/pull/27851) - - make `group_concat` function consider the collation [#27835](https://github.com/pingcap/tidb/pull/27835) - - Fix bug that count disctinct on multi-columns return wrong result when new collation is on. [#27830](https://github.com/pingcap/tidb/pull/27830) - - fix wrong selection push down when having above agg [#27741](https://github.com/pingcap/tidb/pull/27741) - - expression: fix extract bug when argument is a negative duration [#27369](https://github.com/pingcap/tidb/pull/27369) - - planner: add missing column for Apply convert to Join [#27282](https://github.com/pingcap/tidb/pull/27282) - - The undocumented `/debug/sub-optimal-plan` HTTP API has been removed. [#27264](https://github.com/pingcap/tidb/pull/27264) - - fix range building for binary literal [#26455](https://github.com/pingcap/tidb/pull/26455) - - Revert #21045 to avoid #24326. [#26240](https://github.com/pingcap/tidb/pull/26240) - - -+ TiKV/TiKV - - - ```release-note [#10917](https://github.com/tikv/tikv/pull/10917) - - RaftStore Snapshot GC fix: fix the issue that snapshot GC missed GC snapshot files when there's one snapshot file failed to be GC-ed. [#10871](https://github.com/tikv/tikv/pull/10871) - - - TiKV coprocessor slow log will only consider time spent on processing the request. -- Drop log instead of blocking threads when slogger thread is overloaded and queue is filled up. [#10863](https://github.com/tikv/tikv/pull/10863) - - ```release-note [#10781](https://github.com/tikv/tikv/pull/10781) - - Support changing CDC configs dynamically [#10684](https://github.com/tikv/tikv/pull/10684) - - Reduce resolved ts message size to save network bandwidth. [#10677](https://github.com/tikv/tikv/pull/10677) - - Check stale file information from encryption file dict [#10598](https://github.com/tikv/tikv/pull/10598) - - fix frequently reconnecting pd client [#9818](https://github.com/tikv/tikv/pull/9818) - - -+ PingCAP/TiFlash - - - Please add a release note, or a 'None' if it is not needed. [#3018](https://github.com/pingcap/tics/pull/3018) - - -+ PD - - - None. [#3974](https://github.com/tikv/pd/pull/3974) - - Improved the performance of synchronizing Region information between PDs. [#3932](https://github.com/tikv/pd/pull/3932) - - -+ Tools - - + PingCAP/BR - - - ```release-note [#1433](https://github.com/pingcap/br/pull/1433) - - ```release-note [#1432](https://github.com/pingcap/br/pull/1432) - - Restore many small tables would be faster now. [#1429](https://github.com/pingcap/br/pull/1429) - - Merged the rebase auto id operation into create table. [#1424](https://github.com/pingcap/br/pull/1424) - - Expression index and index depending on virtual generated columns are now valid. Previously these indices are broken when importing through Lightning local or importer backend. [#1418](https://github.com/pingcap/br/pull/1418) - - fix the bug that the average speed isn't accurate in backup and restore [#1410](https://github.com/pingcap/br/pull/1410) - - [#1346](https://github.com/pingcap/br/pull/1346) - - - + PingCAP/Dumpling - - - fix pending on show table status in some mysql version [#332](https://github.com/pingcap/dumpling/pull/332) - - Support for backing up MySQL compatible databases that don't support START TRANSACTION ... WITH CONSISTENT SNAPSHOT -- Support for backing up MySQL compatible databases that don't support SHOW CREATE TABLE [#329](https://github.com/pingcap/dumpling/pull/329) - - - + PingCAP/TiCDC - - - Fix json encoding could panic when processing a string type value in some cases. [#2781](https://github.com/pingcap/ticdc/pull/2781) - - Fix a bug that multiple processors could write the same table when this table is re-scheduling [#2727](https://github.com/pingcap/ticdc/pull/2727) - - Fix OOM when TiCDC captures too many regions [#2723](https://github.com/pingcap/ticdc/pull/2723) - - Fix gRPC keepalive error when memory pressure is high. [#2718](https://github.com/pingcap/ticdc/pull/2718) - - Optimize memory management when unified sorter is using memory to sort. [#2710](https://github.com/pingcap/ticdc/pull/2710) - - Please add a release note. -If you don't think this PR needs a release note then fill it with `None`. [#2681](https://github.com/pingcap/ticdc/pull/2681) - - Fix a bug that causes TiCDC to panic on an unsigned tinyint [#2654](https://github.com/pingcap/ticdc/pull/2654) - - fix memory leak which may happen in create new changefeed. [#2623](https://github.com/pingcap/ticdc/pull/2623) - - Fix open protocol, don't output an empty value when there is no change in one transaction. [#2619](https://github.com/pingcap/ticdc/pull/2619) - - Fixed a bug in DDL handling when the owner restarts. [#2609](https://github.com/pingcap/ticdc/pull/2609) - - Prohibit operating TiCDC clusters across major and minor versions [#2601](https://github.com/pingcap/ticdc/pull/2601) - - Fix a bug in metadata management [#2557](https://github.com/pingcap/ticdc/pull/2557) - - Add a global gRPC connection pool and share gRPC connections among kv clients. [#2531](https://github.com/pingcap/ticdc/pull/2531) - - `None`. [#2519](https://github.com/pingcap/ticdc/pull/2519) - - Fix a bug that multiple processors could write the same table when this table is re-scheduling [#2495](https://github.com/pingcap/ticdc/pull/2495) - - Optimize workerpool for fewer goroutines when concurrency is high. [#2486](https://github.com/pingcap/ticdc/pull/2486) - - Fix a bug that owner could meet ErrSchemaStorageTableMiss error and reset a changefeed by accident. [#2457](https://github.com/pingcap/ticdc/pull/2457) - - fix the bug that changefeed cannot be removed if meet GcTTL Exceeded Error [#2455](https://github.com/pingcap/ticdc/pull/2455) - - fix outdated capture info may appear in capture list command [#2447](https://github.com/pingcap/ticdc/pull/2447) - - fix a bug where synchronizing large tables to cdclog failed. [#2444](https://github.com/pingcap/ticdc/pull/2444) - - Reduce goroutine usage when a table's region transfer away from a TiKV node [#2376](https://github.com/pingcap/ticdc/pull/2376) - - Remove file sorter. [#2325](https://github.com/pingcap/ticdc/pull/2325) - - Cleanup changefeed metrics when changefeed is removed. -- Cleanup processor metrics when processor exits. [#2313](https://github.com/pingcap/ticdc/pull/2313) - - puller,mounter,processor: always pull the old value internally [#2304](https://github.com/pingcap/ticdc/pull/2304) - - Fix minor runtime panic risk [#2298](https://github.com/pingcap/ticdc/pull/2298) - - Fix potential DDL loss when owner crashes while executing DDL [#2291](https://github.com/pingcap/ticdc/pull/2291) - - Don't resolve lock immediately after a region is initialized. [#2264](https://github.com/pingcap/ticdc/pull/2264) - - Fix deadlock [#2017](https://github.com/pingcap/ticdc/pull/2017) - - -## Improvements - -+ PingCAP/TiDB - - - Trigger auto-analyze based on histogram row count [#26706](https://github.com/pingcap/tidb/pull/26706) - - -+ TiKV/TiKV - - - - separate read write ready to reduce read latency [#10619](https://github.com/tikv/tikv/pull/10619) - - -## Bug Fixes - -+ PingCAP/TiDB - - - Fix wrong charset and collation for case when function [#26671](https://github.com/pingcap/tidb/pull/26671) - - Fix the issue that greatest(datetime) union null returns empty string [#26564](https://github.com/pingcap/tidb/pull/26564) - - Fix the issue that sometimes fails to send requests if there are tombstone stores. [#25849](https://github.com/pingcap/tidb/pull/25849) - - -+ PingCAP/TiFlash - - - Fix the potential issue of data inconsistency after crashes when deployed on multi-disks [#2774](https://github.com/pingcap/tics/pull/2774) - - Fix the bug that filters unexpected data if the query contains "constant" "<"/"<="/">"/">=" "column" [#2634](https://github.com/pingcap/tics/pull/2634) - - Fix inaccurate store size on tiflash metric under heavy write scenario [#2596](https://github.com/pingcap/tics/pull/2596) - - Fix the bug that TiFlash can not restore data under some situations when deployed on multi disks [#2433](https://github.com/pingcap/tics/pull/2433) - - Fix the potential issue that TiFlash cannot GC the delta data after running for a long time [#2335](https://github.com/pingcap/tics/pull/2335) - - -+ PD - - - Fix the bug that PD would not fix down-peer in time. [#4081](https://github.com/tikv/pd/pull/4081) - - Fix the bug that PD may panic during scaling out TiKV. [#3909](https://github.com/tikv/pd/pull/3909) - - +--- +title: TiDB 4.0.15 Release Notes +category: Releases +--- + +# TiDB 4.0.15 Release Notes + +Release Date: September 27, 2021 + +TiDB version: 4.0.15 + +## Improvements + ++ TiDB + + - Trigger auto-analyze based on histogram row count [#26706](https://github.com/pingcap/tidb/pull/26706) + ++ TiKV + + - separate read write ready to reduce read latency [#10619](https://github.com/tikv/tikv/pull/10619) + +## Bug Fixes + ++ TiDB + + - Fix wrong charset and collation for case when function [#26671](https://github.com/pingcap/tidb/pull/26671) + - Fix the issue that greatest(datetime) union null returns empty string [#26564](https://github.com/pingcap/tidb/pull/26564) + - Fix the issue that sometimes fails to send requests if there are tombstone stores. [#25849](https://github.com/pingcap/tidb/pull/25849) + ++ TiFlash + + - Fix the potential issue of data inconsistency after crashes when deployed on multi-disks [#2774](https://github.com/pingcap/tics/pull/2774) + - Fix the bug that filters unexpected data if the query contains "constant" "<"/"<="/">"/">=" "column" [#2634](https://github.com/pingcap/tics/pull/2634) + - Fix inaccurate store size on tiflash metric under heavy write scenario [#2596](https://github.com/pingcap/tics/pull/2596) + - Fix the bug that TiFlash can not restore data under some situations when deployed on multi disks [#2433](https://github.com/pingcap/tics/pull/2433) + - Fix the potential issue that TiFlash cannot GC the delta data after running for a long time [#2335](https://github.com/pingcap/tics/pull/2335) + ++ PD + + - Fix the bug that PD would not fix down-peer in time. [#4081](https://github.com/tikv/pd/pull/4081) + - Fix the bug that PD may panic during scaling out TiKV. [#3909](https://github.com/tikv/pd/pull/3909) + +## 以下 note 未分类。请将以下 note 进行分类 (Feature enhancements, Improvements, Bug fixes, Compatibility Changes 四类)，并移动到上面对应的标题下。如果某条 note 为多余的，请删除。如果漏抓取了 note，请手动补充 + ++ TiDB + + - executor: fix unexpected behavior when casting invalid string to date [#27935](https://github.com/pingcap/tidb/pull/27935) + - fix expression rewrite makes between expr infers wrong collation. [#27851](https://github.com/pingcap/tidb/pull/27851) + - make `group_concat` function consider the collation [#27835](https://github.com/pingcap/tidb/pull/27835) + - Fix bug that count disctinct on multi-columns return wrong result when new collation is on. [#27830](https://github.com/pingcap/tidb/pull/27830) + - fix wrong selection push down when having above agg [#27741](https://github.com/pingcap/tidb/pull/27741) + - expression: fix extract bug when argument is a negative duration [#27369](https://github.com/pingcap/tidb/pull/27369) + - planner: add missing column for Apply convert to Join [#27282](https://github.com/pingcap/tidb/pull/27282) + - The undocumented `/debug/sub-optimal-plan` HTTP API has been removed. [#27264](https://github.com/pingcap/tidb/pull/27264) + - fix range building for binary literal [#26455](https://github.com/pingcap/tidb/pull/26455) + - Revert #21045 to avoid #24326. [#26240](https://github.com/pingcap/tidb/pull/26240) + ++ TiKV + + - Fix the issue that br reports file already exists error when TDE enabled during restoration. [#10917](https://github.com/tikv/tikv/pull/10917) + - RaftStore Snapshot GC fix: fix the issue that snapshot GC missed GC snapshot files when there's one snapshot file failed to be GC-ed. [#10871](https://github.com/tikv/tikv/pull/10871) + - TiKV coprocessor slow log will only consider time spent on processing the request. + - Drop log instead of blocking threads when slogger thread is overloaded and queue is filled up. [#10863](https://github.com/tikv/tikv/pull/10863) + - Fix delete stale region too frequently. [#10781](https://github.com/tikv/tikv/pull/10781) + - Support changing CDC configs dynamically [#10684](https://github.com/tikv/tikv/pull/10684) + - Reduce resolved ts message size to save network bandwidth. [#10677](https://github.com/tikv/tikv/pull/10677) + - Check stale file information from encryption file dict [#10598](https://github.com/tikv/tikv/pull/10598) + - fix frequently reconnecting pd client [#9818](https://github.com/tikv/tikv/pull/9818) + ++ TiFlash + + - function result name should contain collator info [#3018](https://github.com/pingcap/tics/pull/3018) + ++ PD + + - Improved the performance of synchronizing Region information between PDs. [#3932](https://github.com/tikv/pd/pull/3932) + ++ Tools + + + Backup & Restore (BR) + + - Retry pd request and TiKV IO timeout error [#1433](https://github.com/pingcap/br/pull/1433) + - Too many empty region after restoration of many small tables [#1374](https://github.com/pingcap/br/issues/1374) + - Restore many small tables would be faster now. [#1429](https://github.com/pingcap/br/pull/1429) + - Merged the rebase auto id operation into create table. [#1424](https://github.com/pingcap/br/pull/1424) + - Expression index and index depending on virtual generated columns are now valid. Previously these indices are broken when importing through Lightning local or importer backend. [#1418](https://github.com/pingcap/br/pull/1418) + - fix the bug that the average speed isn't accurate in backup and restore [#1410](https://github.com/pingcap/br/pull/1410) + - Remove unnecessary speed and concurrent limitation [#1346](https://github.com/pingcap/br/pull/1346) + + + Dumpling + + - fix pending on show table status in some mysql version [#332](https://github.com/pingcap/dumpling/pull/332) + - Support for backing up MySQL compatible databases that don't support START TRANSACTION ... WITH CONSISTENT SNAPSHOT + - Support for backing up MySQL compatible databases that don't support SHOW CREATE TABLE [#329](https://github.com/pingcap/dumpling/pull/329) + + + TiCDC + + - Fix json encoding could panic when processing a string type value in some cases. [#2781](https://github.com/pingcap/ticdc/pull/2781) + - Fix a bug that multiple processors could write the same table when this table is re-scheduling [#2727](https://github.com/pingcap/ticdc/pull/2727) + - Fix OOM when TiCDC captures too many regions [#2723](https://github.com/pingcap/ticdc/pull/2723) + - Fix gRPC keepalive error when memory pressure is high. [#2718](https://github.com/pingcap/ticdc/pull/2718) + - Optimize memory management when unified sorter is using memory to sort. [#2710](https://github.com/pingcap/ticdc/pull/2710) + - add prometheus metrics for sink execution ddl [#2681](https://github.com/pingcap/ticdc/pull/2681) + - Fix a bug that causes TiCDC to panic on an unsigned tinyint [#2654](https://github.com/pingcap/ticdc/pull/2654) + - fix memory leak which may happen in create new changefeed. [#2623](https://github.com/pingcap/ticdc/pull/2623) + - Fix open protocol, don't output an empty value when there is no change in one transaction. [#2619](https://github.com/pingcap/ticdc/pull/2619) + - Fixed a bug in DDL handling when the owner restarts. [#2609](https://github.com/pingcap/ticdc/pull/2609) + - Prohibit operating TiCDC clusters across major and minor versions [#2601](https://github.com/pingcap/ticdc/pull/2601) + - Fix a bug in metadata management [#2557](https://github.com/pingcap/ticdc/pull/2557) + - Add a global gRPC connection pool and share gRPC connections among kv clients. [#2531](https://github.com/pingcap/ticdc/pull/2531) + - Fix a bug that multiple processors could write the same table when this table is re-scheduling [#2495](https://github.com/pingcap/ticdc/pull/2495) + - Optimize workerpool for fewer goroutines when concurrency is high. [#2486](https://github.com/pingcap/ticdc/pull/2486) + - Fix a bug that owner could meet ErrSchemaStorageTableMiss error and reset a changefeed by accident. [#2457](https://github.com/pingcap/ticdc/pull/2457) + - fix the bug that changefeed cannot be removed if meet GcTTL Exceeded Error [#2455](https://github.com/pingcap/ticdc/pull/2455) + - fix outdated capture info may appear in capture list command [#2447](https://github.com/pingcap/ticdc/pull/2447) + - fix a bug where synchronizing large tables to cdclog failed. [#2444](https://github.com/pingcap/ticdc/pull/2444) + - Reduce goroutine usage when a table's region transfer away from a TiKV node [#2376](https://github.com/pingcap/ticdc/pull/2376) + - Remove file sorter. [#2325](https://github.com/pingcap/ticdc/pull/2325) + - Cleanup changefeed metrics when changefeed is removed. + - Cleanup processor metrics when processor exits. [#2313](https://github.com/pingcap/ticdc/pull/2313) + - puller,mounter,processor: always pull the old value internally [#2304](https://github.com/pingcap/ticdc/pull/2304) + - Fix minor runtime panic risk [#2298](https://github.com/pingcap/ticdc/pull/2298) + - Fix potential DDL loss when owner crashes while executing DDL [#2291](https://github.com/pingcap/ticdc/pull/2291) + - Don't resolve lock immediately after a region is initialized. [#2264](https://github.com/pingcap/ticdc/pull/2264) + - Fix deadlock [#2017](https://github.com/pingcap/ticdc/pull/2017) diff --git a/releases/release-notes.md b/releases/release-notes.md index 7082bf502c881..ee2b2ffe1c76f 100644 --- a/releases/release-notes.md +++ b/releases/release-notes.md @@ -25,6 +25,7 @@ aliases: ['/docs/dev/releases/release-notes/','/docs/dev/releases/rn/'] ## 4.0 +- [4.0.15](/releases/release-4.0.15.md) - [4.0.14](/releases/release-4.0.14.md) - [4.0.13](/releases/release-4.0.13.md) - [4.0.12](/releases/release-4.0.12.md) From e4950d0254bdd20da9e374693d362176546e5bfc Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Wed, 22 Sep 2021 14:45:17 +0800 Subject: [PATCH 03/15] add notes from comments --- releases/release-4.0.15.md | 152 +++++++++++++++++++------------------ 1 file changed, 79 insertions(+), 73 deletions(-) diff --git a/releases/release-4.0.15.md b/releases/release-4.0.15.md index 2157ffe6ed0f5..4736f3250d3ea 100644 --- a/releases/release-4.0.15.md +++ b/releases/release-4.0.15.md @@ -9,6 +9,10 @@ Release Date: September 27, 2021 TiDB version: 4.0.15 +## Compatibility changes + +- Revert [#21045](https://github.com/pingcap/tidb/pull/21045) to avoid show session variables is very slow on first run. [#24326](https://github.com/pingcap/tidb/issues/24326).[26240] + ## Improvements + TiDB @@ -18,108 +22,110 @@ TiDB version: 4.0.15 + TiKV - separate read write ready to reduce read latency [#10619](https://github.com/tikv/tikv/pull/10619) + - TiKV coprocessor slow log will only consider time spent on processing the request. [#10863](https://github.com/tikv/tikv/pull/10863) + - Drop log instead of blocking threads when slogger thread is overloaded and queue is filled up. [#10863](https://github.com/tikv/tikv/pull/10863) + - Support changing CDC configs dynamically [#10684](https://github.com/tikv/tikv/pull/10684) + - Reduce resolved ts message size to save network bandwidth. [#10677](https://github.com/tikv/tikv/pull/10677) -## Bug Fixes ++ PD -+ TiDB + - Improved the performance of synchronizing Region information between PDs. [#3932](https://github.com/tikv/pd/pull/3932) - - Fix wrong charset and collation for case when function [#26671](https://github.com/pingcap/tidb/pull/26671) - - Fix the issue that greatest(datetime) union null returns empty string [#26564](https://github.com/pingcap/tidb/pull/26564) - - Fix the issue that sometimes fails to send requests if there are tombstone stores. [#25849](https://github.com/pingcap/tidb/pull/25849) ++ Tools -+ TiFlash + + Backup & Restore (BR) - - Fix the potential issue of data inconsistency after crashes when deployed on multi-disks [#2774](https://github.com/pingcap/tics/pull/2774) - - Fix the bug that filters unexpected data if the query contains "constant" "<"/"<="/">"/">=" "column" [#2634](https://github.com/pingcap/tics/pull/2634) - - Fix inaccurate store size on tiflash metric under heavy write scenario [#2596](https://github.com/pingcap/tics/pull/2596) - - Fix the bug that TiFlash can not restore data under some situations when deployed on multi disks [#2433](https://github.com/pingcap/tics/pull/2433) - - Fix the potential issue that TiFlash cannot GC the delta data after running for a long time [#2335](https://github.com/pingcap/tics/pull/2335) + - Make split and scatter regions exeuction concurrently, improve the restore speed from 2h to 30 min in our testing [#1429](https://github.com/pingcap/br/pull/1429) + - Retry on PD request error or TiKV IO timeout error [#1433](https://github.com/pingcap/br/pull/1433) + - Reduce empty regions when restoration of many small tables [#1374](https://github.com/pingcap/br/issues/1374) [#1432](https://github.com/pingcap/br/pull/1432) + - Rebase auto id operation while create table, Save the separate rebase ddl operations to speed up restore [#1424](https://github.com/pingcap/br/pull/1424) -+ PD + + Dumpling - - Fix the bug that PD would not fix down-peer in time. [#4081](https://github.com/tikv/pd/pull/4081) - - Fix the bug that PD may panic during scaling out TiKV. [#3909](https://github.com/tikv/pd/pull/3909) + - Filter skipped database before traversing the tables, save soma traverse operations [#337](https://github.com/pingcap/dumpling/pull/337) + - Use `show full tables` to traverse the tables because `show table status` can't work in some mysql version [#332](https://github.com/pingcap/dumpling/pull/332) + - Support export MySQL compatible databases that don't support `START TRANSACTION ... WITH CONSISTENT SNAPSHOT` or `SHOW CREATE TABLE` [#329](https://github.com/pingcap/dumpling/pull/329) + - Refine dumpling warn log to avoid misunderstand that dump is failed [#340](https://github.com/pingcap/dumpling/pull/340) + + + TiDB Lightning -## 以下 note 未分类。请将以下 note 进行分类 (Feature enhancements, Improvements, Bug fixes, Compatibility Changes 四类)，并移动到上面对应的标题下。如果某条 note 为多余的，请删除。如果漏抓取了 note，请手动补充 + - Support import into table that has expression index or index depending on virtual generated columns [#1418](https://github.com/pingcap/br/pull/1418) + + + TiCDC + + - Solve the new collation and TiCDC compatibility issue: TiCDC always pull the old value from TiKV internally [#2304](https://github.com/pingcap/ticdc/pull/2304) + - Reduce goroutine usage when a table's region transfer away from a TiKV node [#2376](https://github.com/pingcap/ticdc/pull/2376) + - Optimize workerpool for fewer goroutines when concurrency is high [#2486](https://github.com/pingcap/ticdc/pull/2486) + - Make DDL asynchronous execution to avoid affecting other changefeeds [#2471](https://github.com/pingcap/ticdc/pull/2471) + - Add a global gRPC connection pool and share gRPC connections among kv clients [#2531](https://github.com/pingcap/ticdc/pull/2531) + - Fail-fast for unrecoverable DML errors [2315](https://github.com/pingcap/ticdc/pull/2315) + - Optimize memory management when unified sorter is using memory to sort [#2710](https://github.com/pingcap/ticdc/pull/2710) + - add prometheus metrics for sink execution ddl [#2681](https://github.com/pingcap/ticdc/pull/2681) + - Prohibit operating TiCDC clusters across major and minor versions [#2601](https://github.com/pingcap/ticdc/pull/2601) + - Remove file sorter whichi is taked place by unify sorter [#2325](https://github.com/pingcap/ticdc/pull/2325) + - Cleanup changefeed metrics when changefeed is removed, and processor metrics when processor exits [#2313](https://github.com/pingcap/ticdc/pull/2313) + - Optimize the resolve lock algorithm after a region is initialized [#2264](https://github.com/pingcap/ticdc/pull/2264) + +## Bug Fixes + TiDB - - executor: fix unexpected behavior when casting invalid string to date [#27935](https://github.com/pingcap/tidb/pull/27935) - - fix expression rewrite makes between expr infers wrong collation. [#27851](https://github.com/pingcap/tidb/pull/27851) - - make `group_concat` function consider the collation [#27835](https://github.com/pingcap/tidb/pull/27835) - - Fix bug that count disctinct on multi-columns return wrong result when new collation is on. [#27830](https://github.com/pingcap/tidb/pull/27830) - - fix wrong selection push down when having above agg [#27741](https://github.com/pingcap/tidb/pull/27741) - - expression: fix extract bug when argument is a negative duration [#27369](https://github.com/pingcap/tidb/pull/27369) - - planner: add missing column for Apply convert to Join [#27282](https://github.com/pingcap/tidb/pull/27282) - - The undocumented `/debug/sub-optimal-plan` HTTP API has been removed. [#27264](https://github.com/pingcap/tidb/pull/27264) - - fix range building for binary literal [#26455](https://github.com/pingcap/tidb/pull/26455) - - Revert #21045 to avoid #24326. [#26240](https://github.com/pingcap/tidb/pull/26240) + - Fix wrong charset and collation for case when function [#26671](https://github.com/pingcap/tidb/pull/26671) + - Fix the issue that greatest(datetime) union null returns empty string [#26564](https://github.com/pingcap/tidb/pull/26564) + - Fix the issue that sometimes fails to send requests if there are tombstone stores. [#25849](https://github.com/pingcap/tidb/pull/25849) + - Fix range building for binary literal.[26455](https://github.com/pingcap/tidb/pull/26455) + - Fix "index out of range" error when a SQL contains both group by and union.[26553](https://github.com/pingcap/tidb/pull/26553) + - executor: fix unexpected behavior when casting invalid string to date.[27935](https://github.com/pingcap/tidb/pull/27935) + - planner: add missing column for Apply convert to Join.[27282](https://github.com/pingcap/tidb/pull/27282) + - Fix bug that count disctinct on multi-columns return wrong result when new collation is on.[27830](https://github.com/pingcap/tidb/pull/27830) + - expression: fix extract bug when argument is a negative duration.[27369](https://github.com/pingcap/tidb/pull/27369) + - Make `group_concat` function consider the collation.[27835](https://github.com/pingcap/tidb/pull/27835) + - Fix expression rewrite makes between expr infers wrong collation.[27851](https://github.com/pingcap/tidb/pull/27851) + - Fix wrong selection push down when having above agg.[27741](https://github.com/pingcap/tidb/pull/27741) + - Remove the undocumented `/debug/sub-optimal-plan` HTTP API. [#27264](https://github.com/pingcap/tidb/pull/27264) + TiKV - Fix the issue that br reports file already exists error when TDE enabled during restoration. [#10917](https://github.com/tikv/tikv/pull/10917) - RaftStore Snapshot GC fix: fix the issue that snapshot GC missed GC snapshot files when there's one snapshot file failed to be GC-ed. [#10871](https://github.com/tikv/tikv/pull/10871) - - TiKV coprocessor slow log will only consider time spent on processing the request. - - Drop log instead of blocking threads when slogger thread is overloaded and queue is filled up. [#10863](https://github.com/tikv/tikv/pull/10863) - Fix delete stale region too frequently. [#10781](https://github.com/tikv/tikv/pull/10781) - - Support changing CDC configs dynamically [#10684](https://github.com/tikv/tikv/pull/10684) - - Reduce resolved ts message size to save network bandwidth. [#10677](https://github.com/tikv/tikv/pull/10677) - - Check stale file information from encryption file dict [#10598](https://github.com/tikv/tikv/pull/10598) - fix frequently reconnecting pd client [#9818](https://github.com/tikv/tikv/pull/9818) + - Check stale file information from encryption file dict [#10598](https://github.com/tikv/tikv/pull/10598) -+ TiFlash ++ PD - - function result name should contain collator info [#3018](https://github.com/pingcap/tics/pull/3018) + - Fix the bug that PD would not fix down-peer in time. [#4081](https://github.com/tikv/pd/pull/4081) + - Fix the bug that PD may panic during scaling out TiKV. [#3909](https://github.com/tikv/pd/pull/3909) -+ PD ++ TiFlash - - Improved the performance of synchronizing Region information between PDs. [#3932](https://github.com/tikv/pd/pull/3932) + - Fix the potential issue of data inconsistency after crashes when deployed on multi-disks + - Fix a bug of incorrect results that occurs when queries contain filters like `CONSTANT` `<` | `<=` | `>` | `>=` `COLUMN` + - Fix the inaccurate store size on tiflash metric under heavy write scenario + - Fix a bug that TiFlash can not restore data under some situations when deployed on multi disks + - Fix the potential issue that TiFlash cannot GC the delta data after running for a long time + Tools + Backup & Restore (BR) - - Retry pd request and TiKV IO timeout error [#1433](https://github.com/pingcap/br/pull/1433) - - Too many empty region after restoration of many small tables [#1374](https://github.com/pingcap/br/issues/1374) - - Restore many small tables would be faster now. [#1429](https://github.com/pingcap/br/pull/1429) - - Merged the rebase auto id operation into create table. [#1424](https://github.com/pingcap/br/pull/1424) - - Expression index and index depending on virtual generated columns are now valid. Previously these indices are broken when importing through Lightning local or importer backend. [#1418](https://github.com/pingcap/br/pull/1418) - - fix the bug that the average speed isn't accurate in backup and restore [#1410](https://github.com/pingcap/br/pull/1410) - - Remove unnecessary speed and concurrent limitation [#1346](https://github.com/pingcap/br/pull/1346) - - + Dumpling - - - fix pending on show table status in some mysql version [#332](https://github.com/pingcap/dumpling/pull/332) - - Support for backing up MySQL compatible databases that don't support START TRANSACTION ... WITH CONSISTENT SNAPSHOT - - Support for backing up MySQL compatible databases that don't support SHOW CREATE TABLE [#329](https://github.com/pingcap/dumpling/pull/329) + - Fix the bug that the average speed isn't accurate in backup and restore [#1410](https://github.com/pingcap/br/pull/1410) + TiCDC - - Fix json encoding could panic when processing a string type value in some cases. [#2781](https://github.com/pingcap/ticdc/pull/2781) - - Fix a bug that multiple processors could write the same table when this table is re-scheduling [#2727](https://github.com/pingcap/ticdc/pull/2727) - - Fix OOM when TiCDC captures too many regions [#2723](https://github.com/pingcap/ticdc/pull/2723) - - Fix gRPC keepalive error when memory pressure is high. [#2718](https://github.com/pingcap/ticdc/pull/2718) - - Optimize memory management when unified sorter is using memory to sort. [#2710](https://github.com/pingcap/ticdc/pull/2710) - - add prometheus metrics for sink execution ddl [#2681](https://github.com/pingcap/ticdc/pull/2681) + - Fix a bug that owner could meet ErrSchemaStorageTableMiss error and reset a changefeed by accident [#2457](https://github.com/pingcap/ticdc/pull/2457) + - Fix the bug that changefeed cannot be removed if meet GcTTL Exceeded Error [#2455](https://github.com/pingcap/ticdc/pull/2455) + - Fix outdated capture info may appear in capture list command [#2447](https://github.com/pingcap/ticdc/pull/2447) + - Fix deadlock in cdc processor [#2017](https://github.com/pingcap/ticdc/pull/2017) + - Fix a bug that multiple processors could write the same table when this table is re-scheduling [#2495](https://github.com/pingcap/ticdc/pull/2495)[#2727](https://github.com/pingcap/ticdc/pull/2727) + - Fix a bug in metadata management: EtcdWorker snapshot isolation [#2557](https://github.com/pingcap/ticdc/pull/2557) + - Fix ddl sink error don't stop changefeed [#2556](https://github.com/pingcap/ticdc/pull/2556) + - Fix open protocol issue: output an empty value when there is no change in one transaction [#2619](https://github.com/pingcap/ticdc/pull/2619) - Fix a bug that causes TiCDC to panic on an unsigned tinyint [#2654](https://github.com/pingcap/ticdc/pull/2654) - - fix memory leak which may happen in create new changefeed. [#2623](https://github.com/pingcap/ticdc/pull/2623) - - Fix open protocol, don't output an empty value when there is no change in one transaction. [#2619](https://github.com/pingcap/ticdc/pull/2619) - - Fixed a bug in DDL handling when the owner restarts. [#2609](https://github.com/pingcap/ticdc/pull/2609) - - Prohibit operating TiCDC clusters across major and minor versions [#2601](https://github.com/pingcap/ticdc/pull/2601) - - Fix a bug in metadata management [#2557](https://github.com/pingcap/ticdc/pull/2557) - - Add a global gRPC connection pool and share gRPC connections among kv clients. [#2531](https://github.com/pingcap/ticdc/pull/2531) - - Fix a bug that multiple processors could write the same table when this table is re-scheduling [#2495](https://github.com/pingcap/ticdc/pull/2495) - - Optimize workerpool for fewer goroutines when concurrency is high. [#2486](https://github.com/pingcap/ticdc/pull/2486) - - Fix a bug that owner could meet ErrSchemaStorageTableMiss error and reset a changefeed by accident. [#2457](https://github.com/pingcap/ticdc/pull/2457) - - fix the bug that changefeed cannot be removed if meet GcTTL Exceeded Error [#2455](https://github.com/pingcap/ticdc/pull/2455) - - fix outdated capture info may appear in capture list command [#2447](https://github.com/pingcap/ticdc/pull/2447) - - fix a bug where synchronizing large tables to cdclog failed. [#2444](https://github.com/pingcap/ticdc/pull/2444) - - Reduce goroutine usage when a table's region transfer away from a TiKV node [#2376](https://github.com/pingcap/ticdc/pull/2376) - - Remove file sorter. [#2325](https://github.com/pingcap/ticdc/pull/2325) - - Cleanup changefeed metrics when changefeed is removed. - - Cleanup processor metrics when processor exits. [#2313](https://github.com/pingcap/ticdc/pull/2313) - - puller,mounter,processor: always pull the old value internally [#2304](https://github.com/pingcap/ticdc/pull/2304) - - Fix minor runtime panic risk [#2298](https://github.com/pingcap/ticdc/pull/2298) + - Fix gRPC keepalive error when memory pressure is high [#2718](https://github.com/pingcap/ticdc/pull/2718) + - Fix OOM when TiCDC captures too many regions [#2723](https://github.com/pingcap/ticdc/pull/2723) + - Fix json encoding could panic when processing a string type value in some cases [#2781](https://github.com/pingcap/ticdc/pull/2781) + - fix memory leak which may happen in create new changefeed [#2623](https://github.com/pingcap/ticdc/pull/2623) + - Fixed a bug in DDL handling when the owner restarts [#2609](https://github.com/pingcap/ticdc/pull/2609) - Fix potential DDL loss when owner crashes while executing DDL [#2291](https://github.com/pingcap/ticdc/pull/2291) - - Don't resolve lock immediately after a region is initialized. [#2264](https://github.com/pingcap/ticdc/pull/2264) - - Fix deadlock [#2017](https://github.com/pingcap/ticdc/pull/2017) + - Fix runtime panic bug [#2298](https://github.com/pingcap/ticdc/pull/2298) From 5dafa940b2e4c6d7af39c27105dba724223c5f51 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Wed, 22 Sep 2021 14:46:17 +0800 Subject: [PATCH 04/15] Update release-4.0.15.md --- releases/release-4.0.15.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/releases/release-4.0.15.md b/releases/release-4.0.15.md index 4736f3250d3ea..642c3cbec5219 100644 --- a/releases/release-4.0.15.md +++ b/releases/release-4.0.15.md @@ -11,7 +11,9 @@ TiDB version: 4.0.15 ## Compatibility changes -- Revert [#21045](https://github.com/pingcap/tidb/pull/21045) to avoid show session variables is very slow on first run. [#24326](https://github.com/pingcap/tidb/issues/24326).[26240] ++ TiDB + + - Revert [#21045](https://github.com/pingcap/tidb/pull/21045) to avoid show session variables is very slow on first run. [#24326](https://github.com/pingcap/tidb/issues/24326).[26240] ## Improvements From 6fb6fecdc846ea361831f1fa88bece2fea39f62b Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Wed, 22 Sep 2021 15:49:08 +0800 Subject: [PATCH 05/15] revise < 44 --- releases/release-4.0.15.md | 26 +++++++++++++++----------- 1 file changed, 15 insertions(+), 11 deletions(-) diff --git a/releases/release-4.0.15.md b/releases/release-4.0.15.md index 642c3cbec5219..99196a8f92a62 100644 --- a/releases/release-4.0.15.md +++ b/releases/release-4.0.15.md @@ -1,6 +1,5 @@ --- title: TiDB 4.0.15 Release Notes -category: Releases --- # TiDB 4.0.15 Release Notes @@ -13,31 +12,36 @@ TiDB version: 4.0.15 + TiDB - - Revert [#21045](https://github.com/pingcap/tidb/pull/21045) to avoid show session variables is very slow on first run. [#24326](https://github.com/pingcap/tidb/issues/24326).[26240] + - Fix the issue that executing `SHOW VARIABLES` in a new session is slow. This fix reverts some changes made in [#21045](https://github.com/pingcap/tidb/pull/21045) and might cause compatibility issues. [#24326](https://github.com/pingcap/tidb/issues/24326) + +## Feature enhancement + ++ TiKV + + - Support changing TiCDC configurations dynamically [#10645](https://github.com/tikv/tikv/issues/10645) ## Improvements + TiDB - - Trigger auto-analyze based on histogram row count [#26706](https://github.com/pingcap/tidb/pull/26706) + - Trigger auto-analyze based on the histogram row count [#26706](https://github.com/pingcap/tidb/pull/26706) + TiKV - - separate read write ready to reduce read latency [#10619](https://github.com/tikv/tikv/pull/10619) - - TiKV coprocessor slow log will only consider time spent on processing the request. [#10863](https://github.com/tikv/tikv/pull/10863) - - Drop log instead of blocking threads when slogger thread is overloaded and queue is filled up. [#10863](https://github.com/tikv/tikv/pull/10863) - - Support changing CDC configs dynamically [#10684](https://github.com/tikv/tikv/pull/10684) - - Reduce resolved ts message size to save network bandwidth. [#10677](https://github.com/tikv/tikv/pull/10677) + - Handle read ready and write ready separately to reduce read latency [#10475](https://github.com/tikv/tikv/issues/10475) + - The slow log of TiKV coprocessor only considers the time spent on processing requests. [#1084](https://github.com/tikv/tikv/issues/10841) + - Drop log instead of blocking threads when the slogger thread is overloaded and the queue is filled up [#10841](https://github.com/tikv/tikv/issues/10841) + - Reduce the size of Resolved TS messages to save network bandwidth [#2448](https://github.com/pingcap/ticdc/issues/2448) + PD - - Improved the performance of synchronizing Region information between PDs. [#3932](https://github.com/tikv/pd/pull/3932) + - Improve the performance of synchronizing Region information between PDs [#3932](https://github.com/tikv/pd/pull/3932) + Tools + Backup & Restore (BR) - - Make split and scatter regions exeuction concurrently, improve the restore speed from 2h to 30 min in our testing [#1429](https://github.com/pingcap/br/pull/1429) + - Split and scatter Regions concurrently to improve restore speed [#1429](https://github.com/pingcap/br/pull/1429) - Retry on PD request error or TiKV IO timeout error [#1433](https://github.com/pingcap/br/pull/1433) - Reduce empty regions when restoration of many small tables [#1374](https://github.com/pingcap/br/issues/1374) [#1432](https://github.com/pingcap/br/pull/1432) - Rebase auto id operation while create table, Save the separate rebase ddl operations to speed up restore [#1424](https://github.com/pingcap/br/pull/1424) @@ -58,7 +62,7 @@ TiDB version: 4.0.15 - Solve the new collation and TiCDC compatibility issue: TiCDC always pull the old value from TiKV internally [#2304](https://github.com/pingcap/ticdc/pull/2304) - Reduce goroutine usage when a table's region transfer away from a TiKV node [#2376](https://github.com/pingcap/ticdc/pull/2376) - Optimize workerpool for fewer goroutines when concurrency is high [#2486](https://github.com/pingcap/ticdc/pull/2486) - - Make DDL asynchronous execution to avoid affecting other changefeeds [#2471](https://github.com/pingcap/ticdc/pull/2471) + - Make DDL asynchronous execution to avoid affecting other changefeeds [#2471](https://github.com/pingcap/ticdc/pull/2471) - Add a global gRPC connection pool and share gRPC connections among kv clients [#2531](https://github.com/pingcap/ticdc/pull/2531) - Fail-fast for unrecoverable DML errors [2315](https://github.com/pingcap/ticdc/pull/2315) - Optimize memory management when unified sorter is using memory to sort [#2710](https://github.com/pingcap/ticdc/pull/2710) From d0b13265bf7bee5db1b3ebcac184063556b104ec Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Wed, 22 Sep 2021 20:43:19 +0800 Subject: [PATCH 06/15] revise < 97 --- releases/release-4.0.15.md | 40 +++++++++++++++++++------------------- 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/releases/release-4.0.15.md b/releases/release-4.0.15.md index 99196a8f92a62..5749f34274681 100644 --- a/releases/release-4.0.15.md +++ b/releases/release-4.0.15.md @@ -42,35 +42,35 @@ TiDB version: 4.0.15 + Backup & Restore (BR) - Split and scatter Regions concurrently to improve restore speed [#1429](https://github.com/pingcap/br/pull/1429) - - Retry on PD request error or TiKV IO timeout error [#1433](https://github.com/pingcap/br/pull/1433) - - Reduce empty regions when restoration of many small tables [#1374](https://github.com/pingcap/br/issues/1374) [#1432](https://github.com/pingcap/br/pull/1432) - - Rebase auto id operation while create table, Save the separate rebase ddl operations to speed up restore [#1424](https://github.com/pingcap/br/pull/1424) + - Retry BR tasks when encountering the PD request error or the TiKV I/O timeout error [#1433](https://github.com/pingcap/br/pull/1433) + - Reduce empty Regions when restoring many small tables to avoid affecting cluster operation after the restore [#1374](https://github.com/pingcap/br/issues/1374) [#1432](https://github.com/pingcap/br/pull/1432) + - Perform the `rebase auto id` operation while creating tables, which saves the separate `rebase auto id` DDL operations to speed up restore [#1424](https://github.com/pingcap/br/pull/1424) + Dumpling - - Filter skipped database before traversing the tables, save soma traverse operations [#337](https://github.com/pingcap/dumpling/pull/337) - - Use `show full tables` to traverse the tables because `show table status` can't work in some mysql version [#332](https://github.com/pingcap/dumpling/pull/332) - - Support export MySQL compatible databases that don't support `START TRANSACTION ... WITH CONSISTENT SNAPSHOT` or `SHOW CREATE TABLE` [#329](https://github.com/pingcap/dumpling/pull/329) - - Refine dumpling warn log to avoid misunderstand that dump is failed [#340](https://github.com/pingcap/dumpling/pull/340) + - Filter the skipped databases before getting the table information to improve the filtering efficiency of `SHOW TABLE STATUS` [#337](https://github.com/pingcap/dumpling/pull/337) + - Use `SHOW FULL TABLES` to get table information for tables to be exported, because `SHOW TABLE STATUS` cannot work properly in some MySQL versions [#332](https://github.com/pingcap/dumpling/pull/332) + - Support backing up MySQL-compatible databases that do not support the `START TRANSACTION ... WITH CONSISTENT SNAPSHOT` or the `SHOW CREATE TABLE` syntax [#309](https://github.com/pingcap/dumpling/issues/309) + - Refine the Dumpling warning log to avoid the misleading information that a dump fails [#340](https://github.com/pingcap/dumpling/pull/340) + TiDB Lightning - - Support import into table that has expression index or index depending on virtual generated columns [#1418](https://github.com/pingcap/br/pull/1418) + - Support importing data into tables that have expression index or the index that depends on virtual generated columns [#1418](https://github.com/pingcap/br/pull/1418) + TiCDC - - Solve the new collation and TiCDC compatibility issue: TiCDC always pull the old value from TiKV internally [#2304](https://github.com/pingcap/ticdc/pull/2304) - - Reduce goroutine usage when a table's region transfer away from a TiKV node [#2376](https://github.com/pingcap/ticdc/pull/2376) - - Optimize workerpool for fewer goroutines when concurrency is high [#2486](https://github.com/pingcap/ticdc/pull/2486) - - Make DDL asynchronous execution to avoid affecting other changefeeds [#2471](https://github.com/pingcap/ticdc/pull/2471) - - Add a global gRPC connection pool and share gRPC connections among kv clients [#2531](https://github.com/pingcap/ticdc/pull/2531) - - Fail-fast for unrecoverable DML errors [2315](https://github.com/pingcap/ticdc/pull/2315) - - Optimize memory management when unified sorter is using memory to sort [#2710](https://github.com/pingcap/ticdc/pull/2710) - - add prometheus metrics for sink execution ddl [#2681](https://github.com/pingcap/ticdc/pull/2681) - - Prohibit operating TiCDC clusters across major and minor versions [#2601](https://github.com/pingcap/ticdc/pull/2601) - - Remove file sorter whichi is taked place by unify sorter [#2325](https://github.com/pingcap/ticdc/pull/2325) - - Cleanup changefeed metrics when changefeed is removed, and processor metrics when processor exits [#2313](https://github.com/pingcap/ticdc/pull/2313) - - Optimize the resolve lock algorithm after a region is initialized [#2264](https://github.com/pingcap/ticdc/pull/2264) + - TiCDC always pulls old values from TiKV internally [#2304](https://github.com/pingcap/ticdc/pull/2304) + - Reduce the goroutine usage when a table's Regions are all transferred away from a TiKV node [#2284](https://github.com/pingcap/ticdc/issues/2284) + - Optimize workerpool for fewer goroutines when concurrency is high [#2211](https://github.com/pingcap/ticdc/issues/2211) + - Execute DDL statements asynchronously to avoid affecting other changefeeds [#2471](https://github.com/pingcap/ticdc/pull/2471) + - Add a global gRPC connection pool and share gRPC connections among KV clients [#2531](https://github.com/pingcap/ticdc/pull/2531) + - Fail fast for unrecoverable DML errors [2315](https://github.com/pingcap/ticdc/pull/2315) + - Optimize memory management when unified sorter is using memory to sort [#2553](https://github.com/pingcap/ticdc/issues/2553) + - Add Prometheus metrics for DDL executions [#2681](https://github.com/pingcap/ticdc/pull/2681) + - Prohibit operating TiCDC clusters across major or minor versions [#2601](https://github.com/pingcap/ticdc/pull/2601) + - Remove `file sorter` [#2325](https://github.com/pingcap/ticdc/pull/2325) + - Clean up changefeed metrics when a changefeed is removed, and clean up processor metrics when a processor exits [#2313](https://github.com/pingcap/ticdc/pull/2313) + - Optimize the lock-resolving algorithm after a Region is initialized [#2264](https://github.com/pingcap/ticdc/pull/2264) ## Bug Fixes From 7d1072504517446dfde63c8c93074280203c53fe Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Thu, 23 Sep 2021 16:49:05 +0800 Subject: [PATCH 07/15] revise > end --- releases/release-4.0.15.md | 84 +++++++++++++++++++------------------- 1 file changed, 42 insertions(+), 42 deletions(-) diff --git a/releases/release-4.0.15.md b/releases/release-4.0.15.md index 5749f34274681..ff4c28a433a9d 100644 --- a/releases/release-4.0.15.md +++ b/releases/release-4.0.15.md @@ -72,66 +72,66 @@ TiDB version: 4.0.15 - Clean up changefeed metrics when a changefeed is removed, and clean up processor metrics when a processor exits [#2313](https://github.com/pingcap/ticdc/pull/2313) - Optimize the lock-resolving algorithm after a Region is initialized [#2264](https://github.com/pingcap/ticdc/pull/2264) -## Bug Fixes +## Bug fixes + TiDB - - Fix wrong charset and collation for case when function [#26671](https://github.com/pingcap/tidb/pull/26671) - - Fix the issue that greatest(datetime) union null returns empty string [#26564](https://github.com/pingcap/tidb/pull/26564) - - Fix the issue that sometimes fails to send requests if there are tombstone stores. [#25849](https://github.com/pingcap/tidb/pull/25849) - - Fix range building for binary literal.[26455](https://github.com/pingcap/tidb/pull/26455) - - Fix "index out of range" error when a SQL contains both group by and union.[26553](https://github.com/pingcap/tidb/pull/26553) - - executor: fix unexpected behavior when casting invalid string to date.[27935](https://github.com/pingcap/tidb/pull/27935) - - planner: add missing column for Apply convert to Join.[27282](https://github.com/pingcap/tidb/pull/27282) - - Fix bug that count disctinct on multi-columns return wrong result when new collation is on.[27830](https://github.com/pingcap/tidb/pull/27830) - - expression: fix extract bug when argument is a negative duration.[27369](https://github.com/pingcap/tidb/pull/27369) - - Make `group_concat` function consider the collation.[27835](https://github.com/pingcap/tidb/pull/27835) - - Fix expression rewrite makes between expr infers wrong collation.[27851](https://github.com/pingcap/tidb/pull/27851) - - Fix wrong selection push down when having above agg.[27741](https://github.com/pingcap/tidb/pull/27741) - - Remove the undocumented `/debug/sub-optimal-plan` HTTP API. [#27264](https://github.com/pingcap/tidb/pull/27264) + - Fix a bug that collation is incorrectly set for binary literals when building range [26455](https://github.com/pingcap/tidb/pull/26455) + - Fix the issue of wrong character set and collation for the `case when` expression [#26671](https://github.com/pingcap/tidb/pull/26671) + - Fix the "index out of range" error that occurs when a query includes both `GROUP BY` and `UNION` [26553](https://github.com/pingcap/tidb/pull/26553) + - Fix the issue that TiDB might fail to send requests if TiKV has tombstone stores [#25849](https://github.com/pingcap/tidb/pull/25849) + - Fix the issue of unexpected behavior when casting the invalid string to `DATE` [#27935](https://github.com/pingcap/tidb/pull/27935) + - Fix the issue that column information is missed when converting the `Apply` operator to `Join` [#27282](https://github.com/pingcap/tidb/pull/27282) + - Fix a bug that the `count distinct` result on multiple columns is wrong when the new collation is enabled [#27830](https://github.com/pingcap/tidb/pull/27830) + - Fix the result wrong that occurs when the argument of the `extract` function is a negative duration [#27236](https://github.com/pingcap/tidb/issues/27236) + - Fix the wrong execution results that occur when the column in the `group_concat` function has a non-bin collation [#27429](https://github.com/pingcap/tidb/issues/27429) + - Fix the wrong execution results that occur when the collations around the `between` expression are different [#27146](https://github.com/pingcap/tidb/issues/27146) + - Fix the issue that `greatest(datetime) union null` returns empty string [#26564](https://github.com/pingcap/tidb/pull/26564) + - Fix the issue that the `having` clause might not work correctly [#26496](https://github.com/pingcap/tidb/issues/26496) + - Remove the undocumented `/debug/sub-optimal-plan` HTTP API [#27264](https://github.com/pingcap/tidb/pull/27264) + TiKV - - Fix the issue that br reports file already exists error when TDE enabled during restoration. [#10917](https://github.com/tikv/tikv/pull/10917) - - RaftStore Snapshot GC fix: fix the issue that snapshot GC missed GC snapshot files when there's one snapshot file failed to be GC-ed. [#10871](https://github.com/tikv/tikv/pull/10871) - - Fix delete stale region too frequently. [#10781](https://github.com/tikv/tikv/pull/10781) - - fix frequently reconnecting pd client [#9818](https://github.com/tikv/tikv/pull/9818) - - Check stale file information from encryption file dict [#10598](https://github.com/tikv/tikv/pull/10598) + - Fix the issue that BR reports the "file already exists" error when TDE is enabled during data restore [#10917](https://github.com/tikv/tikv/pull/10917) + ?- Fix the issue that snapshot GC might miss GC snapshot files when there is a snapshot file failed to be garbage-collected [#10813](https://github.com/tikv/tikv/issues/10813) + - Fix the issue that TiKV deletes stale Regions too frequently [#10781](https://github.com/tikv/tikv/pull/10781) + - Fix the issue that TiKV frequently reconnects the PD client [#9818](https://github.com/tikv/tikv/pull/9818) + - Check stale file information from the encryption file dictionary [#10598](https://github.com/tikv/tikv/pull/10598) + PD - - Fix the bug that PD would not fix down-peer in time. [#4081](https://github.com/tikv/pd/pull/4081) - - Fix the bug that PD may panic during scaling out TiKV. [#3909](https://github.com/tikv/pd/pull/3909) + - Fix the issue that PD does not fix the down peers in time [#4077](https://github.com/tikv/pd/issues/4077) + - Fix a bug that PD might panic when scaling out TiKV [#3868](https://github.com/tikv/pd/issues/3868) + TiFlash - - Fix the potential issue of data inconsistency after crashes when deployed on multi-disks + - Fix the potential issue of data inconsistency that occurs when TiFlash is deployed on multiple disks - Fix a bug of incorrect results that occurs when queries contain filters like `CONSTANT` `<` | `<=` | `>` | `>=` `COLUMN` - - Fix the inaccurate store size on tiflash metric under heavy write scenario - - Fix a bug that TiFlash can not restore data under some situations when deployed on multi disks - - Fix the potential issue that TiFlash cannot GC the delta data after running for a long time + - Fix the issue that the store size in metrics is inaccurate under heavy writing + - Fix a potential bug that TiFlash cannot restore data when deployed on multiple disks + - Fix the potential issue that TiFlash cannot garbage-collect the delta data after running for a long time + Tools + Backup & Restore (BR) - - Fix the bug that the average speed isn't accurate in backup and restore [#1410](https://github.com/pingcap/br/pull/1410) + - Fix a bug that the average speed is inaccurate in backup and restore [#1410](https://github.com/pingcap/br/pull/1410) + TiCDC - - Fix a bug that owner could meet ErrSchemaStorageTableMiss error and reset a changefeed by accident [#2457](https://github.com/pingcap/ticdc/pull/2457) - - Fix the bug that changefeed cannot be removed if meet GcTTL Exceeded Error [#2455](https://github.com/pingcap/ticdc/pull/2455) - - Fix outdated capture info may appear in capture list command [#2447](https://github.com/pingcap/ticdc/pull/2447) - - Fix deadlock in cdc processor [#2017](https://github.com/pingcap/ticdc/pull/2017) - - Fix a bug that multiple processors could write the same table when this table is re-scheduling [#2495](https://github.com/pingcap/ticdc/pull/2495)[#2727](https://github.com/pingcap/ticdc/pull/2727) - - Fix a bug in metadata management: EtcdWorker snapshot isolation [#2557](https://github.com/pingcap/ticdc/pull/2557) - - Fix ddl sink error don't stop changefeed [#2556](https://github.com/pingcap/ticdc/pull/2556) - - Fix open protocol issue: output an empty value when there is no change in one transaction [#2619](https://github.com/pingcap/ticdc/pull/2619) - - Fix a bug that causes TiCDC to panic on an unsigned tinyint [#2654](https://github.com/pingcap/ticdc/pull/2654) - - Fix gRPC keepalive error when memory pressure is high [#2718](https://github.com/pingcap/ticdc/pull/2718) - - Fix OOM when TiCDC captures too many regions [#2723](https://github.com/pingcap/ticdc/pull/2723) - - Fix json encoding could panic when processing a string type value in some cases [#2781](https://github.com/pingcap/ticdc/pull/2781) - - fix memory leak which may happen in create new changefeed [#2623](https://github.com/pingcap/ticdc/pull/2623) - - Fixed a bug in DDL handling when the owner restarts [#2609](https://github.com/pingcap/ticdc/pull/2609) - - Fix potential DDL loss when owner crashes while executing DDL [#2291](https://github.com/pingcap/ticdc/pull/2291) - - Fix runtime panic bug [#2298](https://github.com/pingcap/ticdc/pull/2298) + - Fix the `ErrSchemaStorageTableMiss` error that occurs when the DDL Job duplication is encountered in the integrated test [#2422](https://github.com/pingcap/ticdc/issues/2422) + - Fix the bug that a changefeed cannot be removed if the `ErrGCTTLExceeded` error occurs [#2391](https://github.com/pingcap/ticdc/issues/2391) + - Fix the issue that outdated capture might appear in the output of the `capture list` command [#2388](https://github.com/pingcap/ticdc/issues/2388) + - Fix the deadlock issue in the TiCDC processor [#2017](https://github.com/pingcap/ticdc/pull/2017) + - Fix a data inconsistency issue that occurs because multiple processors might write data to the same table when this table is being re-scheduled [#2495](https://github.com/pingcap/ticdc/pull/2495)[#2727](https://github.com/pingcap/ticdc/pull/2727) + - Fix a bug that the `EtcdWorker` snapshot isolation is violated in metadata management [#2557](https://github.com/pingcap/ticdc/pull/2557) + - Fix the issue that the changefeed cannot be stopped due to the DDL sink error [#2556](https://github.com/pingcap/ticdc/pull/2556) + - Fix the issue of TiCDC Open Protocol: TiCDC outputs an empty value when there is no change in a transaction [#2619](https://github.com/pingcap/ticdc/pull/2619) + - Fix a bug that causes TiCDC to panic on the unsigned `TINYINT` type [#2648](https://github.com/pingcap/ticdc/issues/2648) + - Decrease the gRPC window size to avoid the OOM that occurs when TiCDC captures too many Regions [#2202](https://github.com/pingcap/ticdc/issues/2202) + - Fix the OOM issue that occurs when TiCDC captures too many Regions [#2723](https://github.com/pingcap/ticdc/pull/2723) + - Fix the issue of process panic that occurs when encoding the data types such as `mysql.TypeString, mysql.TypeVarString, mysql.TypeVarchar` into JSON [#2758](https://github.com/pingcap/ticdc/issues/2758) + - Fix the a memory leak issue that might occur when creating a new changefeed [#2623](https://github.com/pingcap/ticdc/pull/2623) + - Fix a bug that DDL handling fails when a changefeed starts at the finish TS of a schema change [#2603](https://github.com/pingcap/ticdc/issues/2603) + - Fix the issue of potential DDL loss when the owner crashes when executing DDL statements [#1260](https://github.com/pingcap/ticdc/issues/1260) + - Fix the issue of insecure concurrent access to the map in `SinkManager` [#2298](https://github.com/pingcap/ticdc/pull/2298) From ef0e7433668ac159c047656adbf7f80e475366e8 Mon Sep 17 00:00:00 2001 From: qiancai Date: Fri, 24 Sep 2021 16:04:46 +0800 Subject: [PATCH 08/15] Update release-4.0.15.md --- releases/release-4.0.15.md | 34 +++++++++++++++++----------------- 1 file changed, 17 insertions(+), 17 deletions(-) diff --git a/releases/release-4.0.15.md b/releases/release-4.0.15.md index ff4c28a433a9d..ff3529d03b155 100644 --- a/releases/release-4.0.15.md +++ b/releases/release-4.0.15.md @@ -12,7 +12,7 @@ TiDB version: 4.0.15 + TiDB - - Fix the issue that executing `SHOW VARIABLES` in a new session is slow. This fix reverts some changes made in [#21045](https://github.com/pingcap/tidb/pull/21045) and might cause compatibility issues. [#24326](https://github.com/pingcap/tidb/issues/24326) + - Fix the issue that executing `SHOW VARIABLES` in a new session is slow. This fix reverts some changes made in [#18517](https://github.com/pingcap/tidb/issues/18517) and might cause compatibility issues. [#18517](https://github.com/pingcap/tidb/issues/18517) ## Feature enhancement @@ -24,7 +24,7 @@ TiDB version: 4.0.15 + TiDB - - Trigger auto-analyze based on the histogram row count [#26706](https://github.com/pingcap/tidb/pull/26706) + - Trigger auto-analyze based on the histogram row count [#24237](https://github.com/pingcap/tidb/issues/24237) + TiKV @@ -55,7 +55,7 @@ TiDB version: 4.0.15 + TiDB Lightning - - Support importing data into tables that have expression index or the index that depends on virtual generated columns [#1418](https://github.com/pingcap/br/pull/1418) + - Support importing data into tables that have expression index or the index that depends on virtual generated columns [#1404](https://github.com/pingcap/br/issues/1404) + TiCDC @@ -69,34 +69,34 @@ TiDB version: 4.0.15 - Add Prometheus metrics for DDL executions [#2681](https://github.com/pingcap/ticdc/pull/2681) - Prohibit operating TiCDC clusters across major or minor versions [#2601](https://github.com/pingcap/ticdc/pull/2601) - Remove `file sorter` [#2325](https://github.com/pingcap/ticdc/pull/2325) - - Clean up changefeed metrics when a changefeed is removed, and clean up processor metrics when a processor exits [#2313](https://github.com/pingcap/ticdc/pull/2313) - - Optimize the lock-resolving algorithm after a Region is initialized [#2264](https://github.com/pingcap/ticdc/pull/2264) + - Clean up changefeed metrics when a changefeed is removed, and clean up processor metrics when a processor exits [#2156](https://github.com/pingcap/ticdc/issues/2156) + - Optimize the lock-resolving algorithm after a Region is initialized [#2188](https://github.com/pingcap/ticdc/issues/2188) ## Bug fixes + TiDB - Fix a bug that collation is incorrectly set for binary literals when building range [26455](https://github.com/pingcap/tidb/pull/26455) - - Fix the issue of wrong character set and collation for the `case when` expression [#26671](https://github.com/pingcap/tidb/pull/26671) + - Fix the issue of wrong character set and collation for the `case when` expression [#26662](https://github.com/pingcap/tidb/issues/26662) - Fix the "index out of range" error that occurs when a query includes both `GROUP BY` and `UNION` [26553](https://github.com/pingcap/tidb/pull/26553) - Fix the issue that TiDB might fail to send requests if TiKV has tombstone stores [#25849](https://github.com/pingcap/tidb/pull/25849) - - Fix the issue of unexpected behavior when casting the invalid string to `DATE` [#27935](https://github.com/pingcap/tidb/pull/27935) - - Fix the issue that column information is missed when converting the `Apply` operator to `Join` [#27282](https://github.com/pingcap/tidb/pull/27282) - - Fix a bug that the `count distinct` result on multiple columns is wrong when the new collation is enabled [#27830](https://github.com/pingcap/tidb/pull/27830) + - Fix the issue of unexpected behavior when casting the invalid string to `DATE` [#26762](https://github.com/pingcap/tidb/issues/26762) + - Fix the issue that column information is missed when converting the `Apply` operator to `Join` [#27233](https://github.com/pingcap/tidb/issues/27233) + - Fix a bug that the `count distinct` result on multiple columns is wrong when the new collation is enabled [#27091](https://github.com/pingcap/tidb/issues/27091) - Fix the result wrong that occurs when the argument of the `extract` function is a negative duration [#27236](https://github.com/pingcap/tidb/issues/27236) - Fix the wrong execution results that occur when the column in the `group_concat` function has a non-bin collation [#27429](https://github.com/pingcap/tidb/issues/27429) - Fix the wrong execution results that occur when the collations around the `between` expression are different [#27146](https://github.com/pingcap/tidb/issues/27146) - - Fix the issue that `greatest(datetime) union null` returns empty string [#26564](https://github.com/pingcap/tidb/pull/26564) + - Fix the issue that `greatest(datetime) union null` returns empty string [#26532](https://github.com/pingcap/tidb/issues/26532) - Fix the issue that the `having` clause might not work correctly [#26496](https://github.com/pingcap/tidb/issues/26496) - Remove the undocumented `/debug/sub-optimal-plan` HTTP API [#27264](https://github.com/pingcap/tidb/pull/27264) + TiKV - - Fix the issue that BR reports the "file already exists" error when TDE is enabled during data restore [#10917](https://github.com/tikv/tikv/pull/10917) + - Fix the issue that BR reports the "file already exists" error when TDE is enabled during data restore [#1179](https://github.com/pingcap/br/issues/1179) ?- Fix the issue that snapshot GC might miss GC snapshot files when there is a snapshot file failed to be garbage-collected [#10813](https://github.com/tikv/tikv/issues/10813) - - Fix the issue that TiKV deletes stale Regions too frequently [#10781](https://github.com/tikv/tikv/pull/10781) - - Fix the issue that TiKV frequently reconnects the PD client [#9818](https://github.com/tikv/tikv/pull/9818) - - Check stale file information from the encryption file dictionary [#10598](https://github.com/tikv/tikv/pull/10598) + - Fix the issue that TiKV deletes stale Regions too frequently [#10680](https://github.com/tikv/tikv/issues/10680) + - Fix the issue that TiKV frequently reconnects the PD client [#9690](https://github.com/tikv/tikv/issues/9690) + - Check stale file information from the encryption file dictionary [#9115](https://github.com/tikv/tikv/issues/9115) + PD @@ -115,7 +115,7 @@ TiDB version: 4.0.15 + Backup & Restore (BR) - - Fix a bug that the average speed is inaccurate in backup and restore [#1410](https://github.com/pingcap/br/pull/1410) + - Fix a bug that the average speed is inaccurate in backup and restore [#1405](https://github.com/pingcap/br/issues/1405) + TiCDC @@ -126,12 +126,12 @@ TiDB version: 4.0.15 - Fix a data inconsistency issue that occurs because multiple processors might write data to the same table when this table is being re-scheduled [#2495](https://github.com/pingcap/ticdc/pull/2495)[#2727](https://github.com/pingcap/ticdc/pull/2727) - Fix a bug that the `EtcdWorker` snapshot isolation is violated in metadata management [#2557](https://github.com/pingcap/ticdc/pull/2557) - Fix the issue that the changefeed cannot be stopped due to the DDL sink error [#2556](https://github.com/pingcap/ticdc/pull/2556) - - Fix the issue of TiCDC Open Protocol: TiCDC outputs an empty value when there is no change in a transaction [#2619](https://github.com/pingcap/ticdc/pull/2619) + - Fix the issue of TiCDC Open Protocol: TiCDC outputs an empty value when there is no change in a transaction [#2612](https://github.com/pingcap/ticdc/issues/2612) - Fix a bug that causes TiCDC to panic on the unsigned `TINYINT` type [#2648](https://github.com/pingcap/ticdc/issues/2648) - Decrease the gRPC window size to avoid the OOM that occurs when TiCDC captures too many Regions [#2202](https://github.com/pingcap/ticdc/issues/2202) - Fix the OOM issue that occurs when TiCDC captures too many Regions [#2723](https://github.com/pingcap/ticdc/pull/2723) - Fix the issue of process panic that occurs when encoding the data types such as `mysql.TypeString, mysql.TypeVarString, mysql.TypeVarchar` into JSON [#2758](https://github.com/pingcap/ticdc/issues/2758) - - Fix the a memory leak issue that might occur when creating a new changefeed [#2623](https://github.com/pingcap/ticdc/pull/2623) + - Fix the a memory leak issue that might occur when creating a new changefeed [#2389](https://github.com/pingcap/ticdc/issues/2389) - Fix a bug that DDL handling fails when a changefeed starts at the finish TS of a schema change [#2603](https://github.com/pingcap/ticdc/issues/2603) - Fix the issue of potential DDL loss when the owner crashes when executing DDL statements [#1260](https://github.com/pingcap/ticdc/issues/1260) - Fix the issue of insecure concurrent access to the map in `SinkManager` [#2298](https://github.com/pingcap/ticdc/pull/2298) From 6f94b4738250134a2ae6ce6c170c50517745b161 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Fri, 24 Sep 2021 20:24:47 +0800 Subject: [PATCH 09/15] more pr to issue --- releases/release-4.0.15.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/releases/release-4.0.15.md b/releases/release-4.0.15.md index ff3529d03b155..9b03d3359d89d 100644 --- a/releases/release-4.0.15.md +++ b/releases/release-4.0.15.md @@ -12,7 +12,7 @@ TiDB version: 4.0.15 + TiDB - - Fix the issue that executing `SHOW VARIABLES` in a new session is slow. This fix reverts some changes made in [#18517](https://github.com/pingcap/tidb/issues/18517) and might cause compatibility issues. [#18517](https://github.com/pingcap/tidb/issues/18517) + - Fix the issue that executing `SHOW VARIABLES` in a new session is slow. This fix reverts some changes made in [#21045](https://github.com/pingcap/tidb/pull/21045) and might cause compatibility issues. [#24326](https://github.com/pingcap/tidb/issues/24326) ## Feature enhancement @@ -41,15 +41,15 @@ TiDB version: 4.0.15 + Backup & Restore (BR) - - Split and scatter Regions concurrently to improve restore speed [#1429](https://github.com/pingcap/br/pull/1429) - - Retry BR tasks when encountering the PD request error or the TiKV I/O timeout error [#1433](https://github.com/pingcap/br/pull/1433) - - Reduce empty Regions when restoring many small tables to avoid affecting cluster operation after the restore [#1374](https://github.com/pingcap/br/issues/1374) [#1432](https://github.com/pingcap/br/pull/1432) + - Split and scatter Regions concurrently to improve restore speed [#1363](https://github.com/pingcap/br/pull/1363) + - Retry BR tasks when encountering the PD request error or the TiKV I/O timeout error [#27787](https://github.com/pingcap/tidb/issues/27787) + - Reduce empty Regions when restoring many small tables to avoid affecting cluster operation after the restore [#1374](https://github.com/pingcap/br/issues/1374) - Perform the `rebase auto id` operation while creating tables, which saves the separate `rebase auto id` DDL operations to speed up restore [#1424](https://github.com/pingcap/br/pull/1424) + Dumpling - Filter the skipped databases before getting the table information to improve the filtering efficiency of `SHOW TABLE STATUS` [#337](https://github.com/pingcap/dumpling/pull/337) - - Use `SHOW FULL TABLES` to get table information for tables to be exported, because `SHOW TABLE STATUS` cannot work properly in some MySQL versions [#332](https://github.com/pingcap/dumpling/pull/332) + - Use `SHOW FULL TABLES` to get table information for tables to be exported, because `SHOW TABLE STATUS` cannot work properly in some MySQL versions [#322](https://github.com/pingcap/dumpling/issues/322) - Support backing up MySQL-compatible databases that do not support the `START TRANSACTION ... WITH CONSISTENT SNAPSHOT` or the `SHOW CREATE TABLE` syntax [#309](https://github.com/pingcap/dumpling/issues/309) - Refine the Dumpling warning log to avoid the misleading information that a dump fails [#340](https://github.com/pingcap/dumpling/pull/340) @@ -59,14 +59,14 @@ TiDB version: 4.0.15 + TiCDC - - TiCDC always pulls old values from TiKV internally [#2304](https://github.com/pingcap/ticdc/pull/2304) + - TiCDC always pulls old values from TiKV internally [#2397](https://github.com/pingcap/ticdc/pull/2397) - Reduce the goroutine usage when a table's Regions are all transferred away from a TiKV node [#2284](https://github.com/pingcap/ticdc/issues/2284) - Optimize workerpool for fewer goroutines when concurrency is high [#2211](https://github.com/pingcap/ticdc/issues/2211) - - Execute DDL statements asynchronously to avoid affecting other changefeeds [#2471](https://github.com/pingcap/ticdc/pull/2471) + - Execute DDL statements asynchronously to avoid affecting other changefeeds [#2295](https://github.com/pingcap/ticdc/issues/2295) - Add a global gRPC connection pool and share gRPC connections among KV clients [#2531](https://github.com/pingcap/ticdc/pull/2531) - - Fail fast for unrecoverable DML errors [2315](https://github.com/pingcap/ticdc/pull/2315) + - Fail fast for unrecoverable DML errors [#1724](https://github.com/pingcap/ticdc/issues/1724) - Optimize memory management when unified sorter is using memory to sort [#2553](https://github.com/pingcap/ticdc/issues/2553) - - Add Prometheus metrics for DDL executions [#2681](https://github.com/pingcap/ticdc/pull/2681) + - Add Prometheus metrics for DDL executions [#2595](https://github.com/pingcap/ticdc/issues/2595) [#2669](https://github.com/pingcap/ticdc/issues/2669) - Prohibit operating TiCDC clusters across major or minor versions [#2601](https://github.com/pingcap/ticdc/pull/2601) - Remove `file sorter` [#2325](https://github.com/pingcap/ticdc/pull/2325) - Clean up changefeed metrics when a changefeed is removed, and clean up processor metrics when a processor exits [#2156](https://github.com/pingcap/ticdc/issues/2156) @@ -79,7 +79,7 @@ TiDB version: 4.0.15 - Fix a bug that collation is incorrectly set for binary literals when building range [26455](https://github.com/pingcap/tidb/pull/26455) - Fix the issue of wrong character set and collation for the `case when` expression [#26662](https://github.com/pingcap/tidb/issues/26662) - Fix the "index out of range" error that occurs when a query includes both `GROUP BY` and `UNION` [26553](https://github.com/pingcap/tidb/pull/26553) - - Fix the issue that TiDB might fail to send requests if TiKV has tombstone stores [#25849](https://github.com/pingcap/tidb/pull/25849) + - Fix the issue that TiDB might fail to send requests if TiKV has tombstone stores [#23676](https://github.com/pingcap/tidb/issues/23676) [#24648](https://github.com/pingcap/tidb/issues/24648) - Fix the issue of unexpected behavior when casting the invalid string to `DATE` [#26762](https://github.com/pingcap/tidb/issues/26762) - Fix the issue that column information is missed when converting the `Apply` operator to `Join` [#27233](https://github.com/pingcap/tidb/issues/27233) - Fix a bug that the `count distinct` result on multiple columns is wrong when the new collation is enabled [#27091](https://github.com/pingcap/tidb/issues/27091) @@ -125,11 +125,11 @@ TiDB version: 4.0.15 - Fix the deadlock issue in the TiCDC processor [#2017](https://github.com/pingcap/ticdc/pull/2017) - Fix a data inconsistency issue that occurs because multiple processors might write data to the same table when this table is being re-scheduled [#2495](https://github.com/pingcap/ticdc/pull/2495)[#2727](https://github.com/pingcap/ticdc/pull/2727) - Fix a bug that the `EtcdWorker` snapshot isolation is violated in metadata management [#2557](https://github.com/pingcap/ticdc/pull/2557) - - Fix the issue that the changefeed cannot be stopped due to the DDL sink error [#2556](https://github.com/pingcap/ticdc/pull/2556) + - Fix the issue that the changefeed cannot be stopped due to the DDL sink error [#2552](https://github.com/pingcap/ticdc/issues/2552) - Fix the issue of TiCDC Open Protocol: TiCDC outputs an empty value when there is no change in a transaction [#2612](https://github.com/pingcap/ticdc/issues/2612) - Fix a bug that causes TiCDC to panic on the unsigned `TINYINT` type [#2648](https://github.com/pingcap/ticdc/issues/2648) - Decrease the gRPC window size to avoid the OOM that occurs when TiCDC captures too many Regions [#2202](https://github.com/pingcap/ticdc/issues/2202) - - Fix the OOM issue that occurs when TiCDC captures too many Regions [#2723](https://github.com/pingcap/ticdc/pull/2723) + - Fix the OOM issue that occurs when TiCDC captures too many Regions [#2673](https://github.com/pingcap/ticdc/issues/2673) - Fix the issue of process panic that occurs when encoding the data types such as `mysql.TypeString, mysql.TypeVarString, mysql.TypeVarchar` into JSON [#2758](https://github.com/pingcap/ticdc/issues/2758) - Fix the a memory leak issue that might occur when creating a new changefeed [#2389](https://github.com/pingcap/ticdc/issues/2389) - Fix a bug that DDL handling fails when a changefeed starts at the finish TS of a schema change [#2603](https://github.com/pingcap/ticdc/issues/2603) From 863109ce47ebe7b4591f24f6cd78d9eddf7821cd Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Fri, 24 Sep 2021 20:28:07 +0800 Subject: [PATCH 10/15] Update releases/release-4.0.15.md --- releases/release-4.0.15.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-4.0.15.md b/releases/release-4.0.15.md index 9b03d3359d89d..f39931ba19115 100644 --- a/releases/release-4.0.15.md +++ b/releases/release-4.0.15.md @@ -123,7 +123,7 @@ TiDB version: 4.0.15 - Fix the bug that a changefeed cannot be removed if the `ErrGCTTLExceeded` error occurs [#2391](https://github.com/pingcap/ticdc/issues/2391) - Fix the issue that outdated capture might appear in the output of the `capture list` command [#2388](https://github.com/pingcap/ticdc/issues/2388) - Fix the deadlock issue in the TiCDC processor [#2017](https://github.com/pingcap/ticdc/pull/2017) - - Fix a data inconsistency issue that occurs because multiple processors might write data to the same table when this table is being re-scheduled [#2495](https://github.com/pingcap/ticdc/pull/2495)[#2727](https://github.com/pingcap/ticdc/pull/2727) + - Fix a data inconsistency issue that occurs because multiple processors might write data to the same table when this table is being re-scheduled [#2230](https://github.com/pingcap/ticdc/issues/2230) - Fix a bug that the `EtcdWorker` snapshot isolation is violated in metadata management [#2557](https://github.com/pingcap/ticdc/pull/2557) - Fix the issue that the changefeed cannot be stopped due to the DDL sink error [#2552](https://github.com/pingcap/ticdc/issues/2552) - Fix the issue of TiCDC Open Protocol: TiCDC outputs an empty value when there is no change in a transaction [#2612](https://github.com/pingcap/ticdc/issues/2612) From d7c2f78a4a9eb44db33228a38172d80e3b2b5fe0 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Fri, 24 Sep 2021 20:29:07 +0800 Subject: [PATCH 11/15] Update releases/release-4.0.15.md --- releases/release-4.0.15.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-4.0.15.md b/releases/release-4.0.15.md index f39931ba19115..bdf0e577fa835 100644 --- a/releases/release-4.0.15.md +++ b/releases/release-4.0.15.md @@ -76,7 +76,7 @@ TiDB version: 4.0.15 + TiDB - - Fix a bug that collation is incorrectly set for binary literals when building range [26455](https://github.com/pingcap/tidb/pull/26455) + - Fix a bug that collation is incorrectly set for binary literals when building range [#23672](https://github.com/pingcap/tidb/issues/23672) - Fix the issue of wrong character set and collation for the `case when` expression [#26662](https://github.com/pingcap/tidb/issues/26662) - Fix the "index out of range" error that occurs when a query includes both `GROUP BY` and `UNION` [26553](https://github.com/pingcap/tidb/pull/26553) - Fix the issue that TiDB might fail to send requests if TiKV has tombstone stores [#23676](https://github.com/pingcap/tidb/issues/23676) [#24648](https://github.com/pingcap/tidb/issues/24648) From 9319d019503251dd9e965ab810e15268a16b351e Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Fri, 24 Sep 2021 20:30:14 +0800 Subject: [PATCH 12/15] Update release-4.0.15.md --- releases/release-4.0.15.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-4.0.15.md b/releases/release-4.0.15.md index bdf0e577fa835..6f226ab4fe14e 100644 --- a/releases/release-4.0.15.md +++ b/releases/release-4.0.15.md @@ -78,7 +78,7 @@ TiDB version: 4.0.15 - Fix a bug that collation is incorrectly set for binary literals when building range [#23672](https://github.com/pingcap/tidb/issues/23672) - Fix the issue of wrong character set and collation for the `case when` expression [#26662](https://github.com/pingcap/tidb/issues/26662) - - Fix the "index out of range" error that occurs when a query includes both `GROUP BY` and `UNION` [26553](https://github.com/pingcap/tidb/pull/26553) + - Fix the "index out of range" error that occurs when a query includes both `GROUP BY` and `UNION` [#26553](https://github.com/pingcap/tidb/pull/26553) - Fix the issue that TiDB might fail to send requests if TiKV has tombstone stores [#23676](https://github.com/pingcap/tidb/issues/23676) [#24648](https://github.com/pingcap/tidb/issues/24648) - Fix the issue of unexpected behavior when casting the invalid string to `DATE` [#26762](https://github.com/pingcap/tidb/issues/26762) - Fix the issue that column information is missed when converting the `Apply` operator to `Join` [#27233](https://github.com/pingcap/tidb/issues/27233) From 033d492645105d8c9251ca0a6eb3989197198b3c Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Sun, 26 Sep 2021 11:51:22 +0800 Subject: [PATCH 13/15] fix --- releases/release-4.0.15.md | 16 ++++++++-------- 1 file changed, 8 insertions(+), 8 deletions(-) diff --git a/releases/release-4.0.15.md b/releases/release-4.0.15.md index 6f226ab4fe14e..d0383bf362e7d 100644 --- a/releases/release-4.0.15.md +++ b/releases/release-4.0.15.md @@ -43,8 +43,8 @@ TiDB version: 4.0.15 - Split and scatter Regions concurrently to improve restore speed [#1363](https://github.com/pingcap/br/pull/1363) - Retry BR tasks when encountering the PD request error or the TiKV I/O timeout error [#27787](https://github.com/pingcap/tidb/issues/27787) - - Reduce empty Regions when restoring many small tables to avoid affecting cluster operation after the restore [#1374](https://github.com/pingcap/br/issues/1374) - - Perform the `rebase auto id` operation while creating tables, which saves the separate `rebase auto id` DDL operations to speed up restore [#1424](https://github.com/pingcap/br/pull/1424) + - Reduce empty Regions when restoring many small tables to avoid affecting cluster operations after the restore [#1374](https://github.com/pingcap/br/issues/1374) + - Perform the `rebase auto id` operation while creating tables, which saves the separate `rebase auto id` DDL operation and speeds up restore [#1424](https://github.com/pingcap/br/pull/1424) + Dumpling @@ -59,13 +59,13 @@ TiDB version: 4.0.15 + TiCDC - - TiCDC always pulls old values from TiKV internally [#2397](https://github.com/pingcap/ticdc/pull/2397) + - Always pulls old values from TiKV internally to improve usability [#2397](https://github.com/pingcap/ticdc/pull/2397) - Reduce the goroutine usage when a table's Regions are all transferred away from a TiKV node [#2284](https://github.com/pingcap/ticdc/issues/2284) - Optimize workerpool for fewer goroutines when concurrency is high [#2211](https://github.com/pingcap/ticdc/issues/2211) - Execute DDL statements asynchronously to avoid affecting other changefeeds [#2295](https://github.com/pingcap/ticdc/issues/2295) - Add a global gRPC connection pool and share gRPC connections among KV clients [#2531](https://github.com/pingcap/ticdc/pull/2531) - Fail fast for unrecoverable DML errors [#1724](https://github.com/pingcap/ticdc/issues/1724) - - Optimize memory management when unified sorter is using memory to sort [#2553](https://github.com/pingcap/ticdc/issues/2553) + - Optimize memory management when the Unified Sorter is using memory to sort data [#2553](https://github.com/pingcap/ticdc/issues/2553) - Add Prometheus metrics for DDL executions [#2595](https://github.com/pingcap/ticdc/issues/2595) [#2669](https://github.com/pingcap/ticdc/issues/2669) - Prohibit operating TiCDC clusters across major or minor versions [#2601](https://github.com/pingcap/ticdc/pull/2601) - Remove `file sorter` [#2325](https://github.com/pingcap/ticdc/pull/2325) @@ -76,7 +76,7 @@ TiDB version: 4.0.15 + TiDB - - Fix a bug that collation is incorrectly set for binary literals when building range [#23672](https://github.com/pingcap/tidb/issues/23672) + - Fix a bug that collation is incorrectly set for binary literals when building ranges [#23672](https://github.com/pingcap/tidb/issues/23672) - Fix the issue of wrong character set and collation for the `case when` expression [#26662](https://github.com/pingcap/tidb/issues/26662) - Fix the "index out of range" error that occurs when a query includes both `GROUP BY` and `UNION` [#26553](https://github.com/pingcap/tidb/pull/26553) - Fix the issue that TiDB might fail to send requests if TiKV has tombstone stores [#23676](https://github.com/pingcap/tidb/issues/23676) [#24648](https://github.com/pingcap/tidb/issues/24648) @@ -106,7 +106,7 @@ TiDB version: 4.0.15 + TiFlash - Fix the potential issue of data inconsistency that occurs when TiFlash is deployed on multiple disks - - Fix a bug of incorrect results that occurs when queries contain filters like `CONSTANT` `<` | `<=` | `>` | `>=` `COLUMN` + - Fix a bug of incorrect results that occurs when queries contain filters like `CONSTANT`, `<`, `<=`, `>`, `>=`, or `COLUMN` - Fix the issue that the store size in metrics is inaccurate under heavy writing - Fix a potential bug that TiFlash cannot restore data when deployed on multiple disks - Fix the potential issue that TiFlash cannot garbage-collect the delta data after running for a long time @@ -115,12 +115,12 @@ TiDB version: 4.0.15 + Backup & Restore (BR) - - Fix a bug that the average speed is inaccurate in backup and restore [#1405](https://github.com/pingcap/br/issues/1405) + - Fix a bug that the average speed is inaccurately calculated for backup and restore [#1405](https://github.com/pingcap/br/issues/1405) + TiCDC - Fix the `ErrSchemaStorageTableMiss` error that occurs when the DDL Job duplication is encountered in the integrated test [#2422](https://github.com/pingcap/ticdc/issues/2422) - - Fix the bug that a changefeed cannot be removed if the `ErrGCTTLExceeded` error occurs [#2391](https://github.com/pingcap/ticdc/issues/2391) + - Fix a bug that a changefeed cannot be removed if the `ErrGCTTLExceeded` error occurs [#2391](https://github.com/pingcap/ticdc/issues/2391) - Fix the issue that outdated capture might appear in the output of the `capture list` command [#2388](https://github.com/pingcap/ticdc/issues/2388) - Fix the deadlock issue in the TiCDC processor [#2017](https://github.com/pingcap/ticdc/pull/2017) - Fix a data inconsistency issue that occurs because multiple processors might write data to the same table when this table is being re-scheduled [#2230](https://github.com/pingcap/ticdc/issues/2230) From d887255010629f8402624505f86dd65b3fbd0ee4 Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Sun, 26 Sep 2021 14:19:50 +0800 Subject: [PATCH 14/15] add compatibility changes --- releases/release-4.0.15.md | 22 ++++++++++++---------- 1 file changed, 12 insertions(+), 10 deletions(-) diff --git a/releases/release-4.0.15.md b/releases/release-4.0.15.md index d0383bf362e7d..e818400a3efdb 100644 --- a/releases/release-4.0.15.md +++ b/releases/release-4.0.15.md @@ -13,6 +13,16 @@ TiDB version: 4.0.15 + TiDB - Fix the issue that executing `SHOW VARIABLES` in a new session is slow. This fix reverts some changes made in [#21045](https://github.com/pingcap/tidb/pull/21045) and might cause compatibility issues. [#24326](https://github.com/pingcap/tidb/issues/24326) + + The following bug fixes change execution results, which might cause upgrade incompatibilities: + - Fix the issue that `greatest(datetime) union null` returns empty string [#26532](https://github.com/pingcap/tidb/issues/26532) + - Fix the issue that the `having` clause might not work correctly [#26496](https://github.com/pingcap/tidb/issues/26496) + - Fix the wrong execution results that occur when the collations around the `between` expression are different [#27146](https://github.com/pingcap/tidb/issues/27146) + - Fix the result wrong that occurs when the argument of the `extract` function is a negative duration [#27236](https://github.com/pingcap/tidb/issues/27236) + - Fix the wrong execution results that occur when the column in the `group_concat` function has a non-bin collation [#27429](https://github.com/pingcap/tidb/issues/27429) + - Fix the issue of wrong character set and collation for the `case when` expression [#26662](https://github.com/pingcap/tidb/issues/26662) + - Fix the issue that column information is missed when converting the `Apply` operator to `Join` [#27233](https://github.com/pingcap/tidb/issues/27233) + - Fix the issue of unexpected behavior when casting the invalid string to `DATE` [#26762](https://github.com/pingcap/tidb/issues/26762) + - Fix a bug that the `count distinct` result on multiple columns is wrong when the new collation is enabled [#27091](https://github.com/pingcap/tidb/issues/27091) ## Feature enhancement @@ -77,23 +87,15 @@ TiDB version: 4.0.15 + TiDB - Fix a bug that collation is incorrectly set for binary literals when building ranges [#23672](https://github.com/pingcap/tidb/issues/23672) - - Fix the issue of wrong character set and collation for the `case when` expression [#26662](https://github.com/pingcap/tidb/issues/26662) + - Fix the "index out of range" error that occurs when a query includes both `GROUP BY` and `UNION` [#26553](https://github.com/pingcap/tidb/pull/26553) - Fix the issue that TiDB might fail to send requests if TiKV has tombstone stores [#23676](https://github.com/pingcap/tidb/issues/23676) [#24648](https://github.com/pingcap/tidb/issues/24648) - - Fix the issue of unexpected behavior when casting the invalid string to `DATE` [#26762](https://github.com/pingcap/tidb/issues/26762) - - Fix the issue that column information is missed when converting the `Apply` operator to `Join` [#27233](https://github.com/pingcap/tidb/issues/27233) - - Fix a bug that the `count distinct` result on multiple columns is wrong when the new collation is enabled [#27091](https://github.com/pingcap/tidb/issues/27091) - - Fix the result wrong that occurs when the argument of the `extract` function is a negative duration [#27236](https://github.com/pingcap/tidb/issues/27236) - - Fix the wrong execution results that occur when the column in the `group_concat` function has a non-bin collation [#27429](https://github.com/pingcap/tidb/issues/27429) - - Fix the wrong execution results that occur when the collations around the `between` expression are different [#27146](https://github.com/pingcap/tidb/issues/27146) - - Fix the issue that `greatest(datetime) union null` returns empty string [#26532](https://github.com/pingcap/tidb/issues/26532) - - Fix the issue that the `having` clause might not work correctly [#26496](https://github.com/pingcap/tidb/issues/26496) - Remove the undocumented `/debug/sub-optimal-plan` HTTP API [#27264](https://github.com/pingcap/tidb/pull/27264) + TiKV - Fix the issue that BR reports the "file already exists" error when TDE is enabled during data restore [#1179](https://github.com/pingcap/br/issues/1179) - ?- Fix the issue that snapshot GC might miss GC snapshot files when there is a snapshot file failed to be garbage-collected [#10813](https://github.com/tikv/tikv/issues/10813) + - Fix the potential disk full issue caused by corrupted snapshot files [#10813](https://github.com/tikv/tikv/issues/10813) - Fix the issue that TiKV deletes stale Regions too frequently [#10680](https://github.com/tikv/tikv/issues/10680) - Fix the issue that TiKV frequently reconnects the PD client [#9690](https://github.com/tikv/tikv/issues/9690) - Check stale file information from the encryption file dictionary [#9115](https://github.com/tikv/tikv/issues/9115) From 5b06466c86a259f5400b4f01f02ac911cbde0c8b Mon Sep 17 00:00:00 2001 From: TomShawn <41534398+TomShawn@users.noreply.github.com> Date: Mon, 27 Sep 2021 09:55:07 +0800 Subject: [PATCH 15/15] Update releases/release-4.0.15.md Co-authored-by: Enwei --- releases/release-4.0.15.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/releases/release-4.0.15.md b/releases/release-4.0.15.md index e818400a3efdb..6d2b718b75166 100644 --- a/releases/release-4.0.15.md +++ b/releases/release-4.0.15.md @@ -39,7 +39,7 @@ TiDB version: 4.0.15 + TiKV - Handle read ready and write ready separately to reduce read latency [#10475](https://github.com/tikv/tikv/issues/10475) - - The slow log of TiKV coprocessor only considers the time spent on processing requests. [#1084](https://github.com/tikv/tikv/issues/10841) + - The slow log of TiKV coprocessor only considers the time spent on processing requests. [#10841](https://github.com/tikv/tikv/issues/10841) - Drop log instead of blocking threads when the slogger thread is overloaded and the queue is filled up [#10841](https://github.com/tikv/tikv/issues/10841) - Reduce the size of Resolved TS messages to save network bandwidth [#2448](https://github.com/pingcap/ticdc/issues/2448)