Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ticdc: update troubleshoot for ddl skip #6331

Merged
merged 3 commits into from
May 26, 2021

Conversation

amyangfei
Copy link
Contributor

@amyangfei amyangfei commented May 25, 2021

First-time contributors' checklist

What is changed, added or deleted? (Required)

Which TiDB version(s) do your changes apply to? (Required)

  • master (the latest development version)
  • v5.0 (TiDB 5.0 versions) Add troubleshooting info for ddl skip
  • v4.0 (TiDB 4.0 versions) Add troubleshooting info for ddl skip
  • v3.1 (TiDB 3.1 versions)
  • v3.0 (TiDB 3.0 versions)
  • v2.1 (TiDB 2.1 versions)

What is the related PR or file link(s)?

ref: https://github.com/pingcap/ticdc/pull/1835

Add more instructions about how to find start-ts when cdc log doesn't contain it.

Do your changes match any of the following descriptions?

  • Delete files
  • Change aliases
  • Need modification after applied to another branch
  • Might cause conflicts after applied to another branch

@ti-chi-bot ti-chi-bot requested a review from TomShawn May 25, 2021 08:03
@ti-chi-bot ti-chi-bot added missing-translation-status This PR does not have translation status info. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels May 25, 2021
@TomShawn TomShawn requested a review from zier-one May 25, 2021 08:11
@TomShawn TomShawn added needs-cherry-pick-release-4.0 sig/docs Indicates that the Issue or PR belongs to the docs SIG. area/migrate Indicates that the Issue or PR belongs to the area of TiDB migration tools. translation/doing This PR’s assignee is translating this PR. labels May 25, 2021
@ti-chi-bot ti-chi-bot removed the missing-translation-status This PR does not have translation status info. label May 25, 2021
@@ -311,9 +311,9 @@ TiCDC 对大事务(大小超过 5 GB)提供部分支持,根据场景不同
## 当 changefeed 的下游为类 MySQL 数据库时,TiCDC 执行了一个耗时较长的 DDL 语句,阻塞了所有其他 changefeed,应该怎样处理?

1. 首先暂停执行耗时较长的 DDL 的 changefeed。此时可以观察到,这个 changefeed 暂停后,其他的 changefeed 不再阻塞了。
2. 在 TiCDC log 中搜寻 `apply job` 字段,确认耗时较长的 DDL 的 `StartTs`。
2. 在 TiCDC log 中搜寻 `apply job` 字段,确认耗时较长的 DDL 的 `start-ts`。如果旧版本的 TiCDC 没有打印该 `start-ts`,可以查询 TiDB DDL history,找到该条 DDL 语句的 `binlog.TableInfo.update_timestamp` 字段,该字段即为所需的 `start-ts`。
Copy link
Contributor Author

@amyangfei amyangfei May 25, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
2. 在 TiCDC log 中搜寻 `apply job` 字段,确认耗时较长的 DDL 的 `start-ts`如果旧版本的 TiCDC 没有打印该 `start-ts`,可以查询 TiDB DDL history,找到该条 DDL 语句的 `binlog.TableInfo.update_timestamp` 字段,该字段即为所需的 `start-ts`
2. 在 TiCDC log 中搜寻 `apply job` 字段,确认耗时较长的 DDL 的 `start-ts`如果旧版本(<= v4.0.13 或 <= v5.0.1)的 TiCDC 没有打印该 `start-ts`,可以查询 TiDB DDL history,找到该条 DDL 语句的 `binlog.TableInfo.update_timestamp` 字段,该字段即为所需的 `start-ts`

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PTAL @TomShawn

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dev 版是最新开发版,应该不存在这个“旧版本”的历史问题。要不在 dev 中去掉“如果旧版本的 TiCDC 没有打印”这一句,在 release-x.0 文档中加上?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

可以

@amyangfei
Copy link
Contributor Author

ptal @TomShawn

@TomShawn TomShawn added the requires-version-specific-changes After cherry-picked, the cherry-picked PR requires further changes. label May 26, 2021
Copy link
Contributor

@TomShawn TomShawn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ti-chi-bot
Copy link
Member

[REVIEW NOTIFICATION]

This pull request has been approved by:

  • TomShawn

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by writing /lgtm in a comment.
Reviewer can cancel approval by writing /lgtm cancel in a comment.

@ti-chi-bot ti-chi-bot added the status/LGT1 Indicates that a PR has LGTM 1. label May 26, 2021
@TomShawn TomShawn added the require-LGT1 Indicates that the PR requires an LGTM. label May 26, 2021
@TomShawn
Copy link
Contributor

/merge

@ti-chi-bot
Copy link
Member

This pull request has been accepted and is ready to merge.

Commit hash: 46517fa

@ti-chi-bot ti-chi-bot added the status/can-merge Indicates a PR has been approved by a committer. label May 26, 2021
@ti-chi-bot ti-chi-bot merged commit 3172bfe into pingcap:master May 26, 2021
@amyangfei amyangfei deleted the fix-cdc-skip-ddl branch May 26, 2021 03:12
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #6334.

@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request created: #6335.

@Joyinqin
Copy link
Contributor

/translation done
/remove-translation doing

@ti-chi-bot ti-chi-bot added translation/done This PR has been translated from English into Chinese and updated to pingcap/docs-cn in a PR. and removed translation/doing This PR’s assignee is translating this PR. labels May 26, 2021
@ti-chi-bot
Copy link
Member

In response to a cherrypick label: new pull request could not be created: failed to create pull request against pingcap/docs-cn#release-4.0 from head ti-chi-bot:cherry-pick-6331-to-release-4.0: status code 422 not one of [201], body: {"message":"Validation Failed","errors":[{"resource":"PullRequest","code":"custom","message":"A pull request already exists for ti-chi-bot:cherry-pick-6331-to-release-4.0."}],"documentation_url":"https://docs.github.com/rest/reference/pulls#create-a-pull-request"}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/migrate Indicates that the Issue or PR belongs to the area of TiDB migration tools. require-LGT1 Indicates that the PR requires an LGTM. requires-version-specific-changes After cherry-picked, the cherry-picked PR requires further changes. sig/docs Indicates that the Issue or PR belongs to the docs SIG. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. status/can-merge Indicates a PR has been approved by a committer. status/LGT1 Indicates that a PR has LGTM 1. translation/done This PR has been translated from English into Chinese and updated to pingcap/docs-cn in a PR.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants