Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

changefeedccl: periodic pts record updates #13064

Closed
cockroach-teamcity opened this issue Feb 18, 2022 · 1 comment
Closed

changefeedccl: periodic pts record updates #13064

cockroach-teamcity opened this issue Feb 18, 2022 · 1 comment

Comments

@cockroach-teamcity
Copy link
Member

cockroach-teamcity commented Feb 18, 2022

Exalate commented:

cockroachdb/cockroach#76605 --- Release note (enterprise change): changefeeds running on tables with a low gcttl will function more reliably due to protected timestamps being maintained for the changefeed targets at the resolved timestamp of the changefeed. The frequency at which the protected timestamp is updated to the resolved timestamp can be configured through the changefeed.protect_timestamp_interval cluster setting. If the changefeed lags too far behind such that storage of old data becomes an issue, cancelling the changefeed will release the protected timestamps and allow garbage collection to resume. If protect_data_from_gc_on_pause is unset, pausing the changefeed will release the existing protected timestamp record.

Jira Issue: DOC-2684

@exalate-issue-sync
Copy link

Abbey Russell (amruss) commented:
Before we would only lay a protected timestamp if you paused with gc or during initial scan, now we by default protect changefeed data from being garbage collected

Every 10 minutes it will move the protected timestamp up to the current high water mark

The only way for changefeeds to not protect data is to pause the changefeed without protect_from_gc_on_pause, to cancel the changefeed, or for the changefeed to fail without on_error=pause

We recommend monitoring of storage and of running changefeeds.. now if a changefeed is not advancing and is retrying, it will (without limit) accumulate garbage while it retries to run

  • Later work: retry limits

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants