Updates to known limitations for cdc #12503
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
This change is |
|
Files changed: _includes/v21.1/known-limitations/cdc.md |
|
✔️ Netlify Preview 🔨 Explore the source changes: 38c6050 🔍 Inspect the deploy log: https://app.netlify.com/sites/cockroachdb-docs/deploys/61b0c88b4c8c910007431e5e 😎 Browse the preview: https://deploy-preview-12503--cockroachdb-docs.netlify.app |
rmloveland
approved these changes
Dec 7, 2021
| - Changefeed target options are limited to tables. [Tracking GitHub Issue](https://github.com/cockroachdb/cockroach/issues/73435) | ||
| - Using a [cloud storage sink](create-changefeed.html#cloud-storage-sink) only works with `JSON` and emits [newline-delimited json](http://ndjson.org) files. [Tracking GitHub Issue](https://github.com/cockroachdb/cockroach/issues/73432) | ||
| - Webhook sinks only support HTTPS. Use the [`insecure_tls_skip_verify`](create-changefeed.html#tls-skip-verify) parameter when testing to disable certificate verification; however, this still requires HTTPS and certificates. [Tracking GitHub Issue](https://github.com/cockroachdb/cockroach/issues/73431) | ||
| - Currently, webhook sinks only have support for emitting `JSON`. [Tracking GitHub Issue](https://github.com/cockroachdb/cockroach/issues/73432) |
|
|
||
| - It is necessary to [enable rangefeeds](#enable-rangefeeds) for changefeeds to work. | ||
| - Changefeeds do not share internal buffers, so each running changefeed will increase total memory usage. To watch multiple tables, we recommend creating a changefeed with a comma-separated list of tables. | ||
| - Many DDL queries (including [`TRUNCATE`](truncate.html) and [`DROP TABLE`](drop-table.html)) will cause errors on a changefeed watching the affected tables. You will need to [start a new changefeed](create-changefeed.html#start-a-new-changefeed-where-another-ended). |
There was a problem hiding this comment.
Do we have an issue to document all the DDL that will cause errors? As a user I'd be interested in that.
| - It is necessary to [enable rangefeeds](#enable-rangefeeds) for changefeeds to work. | ||
| - Changefeeds do not share internal buffers, so each running changefeed will increase total memory usage. To watch multiple tables, we recommend creating a changefeed with a comma-separated list of tables. | ||
| - Many DDL queries (including [`TRUNCATE`](truncate.html) and [`DROP TABLE`](drop-table.html)) will cause errors on a changefeed watching the affected tables. You will need to [start a new changefeed](create-changefeed.html#start-a-new-changefeed-where-another-ended). | ||
| - Partial or intermittent sink unavailability may impact changefeed stability; however, [ordering guarantees](stream-data-out-of-cockroachdb-using-changefeeds.html#ordering-guarantees) will still hold for as long as a changefeed [remains active](stream-data-out-of-cockroachdb-using-changefeeds.html#monitor-a-changefeed). |
There was a problem hiding this comment.
Does "impact stability" mean it would cause the change feed to become inactive? or something else? sorry if dumb question
| - Changefeeds do not share internal buffers, so each running changefeed will increase total memory usage. To watch multiple tables, we recommend creating a changefeed with a comma-separated list of tables. | ||
| - Many DDL queries (including [`TRUNCATE`](truncate.html) and [`DROP TABLE`](drop-table.html)) will cause errors on a changefeed watching the affected tables. You will need to [start a new changefeed](create-changefeed.html#start-a-new-changefeed-where-another-ended). | ||
| - Partial or intermittent sink unavailability may impact changefeed stability; however, [ordering guarantees](stream-data-out-of-cockroachdb-using-changefeeds.html#ordering-guarantees) will still hold for as long as a changefeed [remains active](stream-data-out-of-cockroachdb-using-changefeeds.html#monitor-a-changefeed). | ||
| - When an [`IMPORT INTO`](import-into.html) statement is run, changefeed jobs targeting that table will fail. |
There was a problem hiding this comment.
Does this state persist? E.g. if I have a change feed targeting table X and I IMPORT INTO X, will any future change feeds targeting the table fail? or do we mean that any current change feed jobs will fail? (I guess it's the latter after writing this?)
Suggest saying "any current jobs targeting" if the latter
| - It is necessary to [enable rangefeeds](#enable-rangefeeds) for changefeeds to work. | ||
| - Changefeeds do not share internal buffers, so each running changefeed will increase total memory usage. To watch multiple tables, we recommend creating a changefeed with a comma-separated list of tables. | ||
| - Many DDL queries (including [`TRUNCATE`](truncate.html) and [`DROP TABLE`](drop-table.html)) will cause errors on a changefeed watching the affected tables. You will need to [start a new changefeed](create-changefeed.html#start-a-new-changefeed-where-another-ended). | ||
| - Partial or intermittent sink unavailability may impact changefeed stability; however, [ordering guarantees](stream-data-out-of-cockroachdb-using-changefeeds.html#ordering-guarantees) will still hold for as long as a changefeed [remains active](stream-data-out-of-cockroachdb-using-changefeeds.html#monitor-a-changefeed). |
There was a problem hiding this comment.
same q re: does unstable mean become inactive?
| - Changefeeds do not share internal buffers, so each running changefeed will increase total memory usage. To watch multiple tables, we recommend creating a changefeed with a comma-separated list of tables. | ||
| - Many DDL queries (including [`TRUNCATE`](truncate.html) and [`DROP TABLE`](drop-table.html)) will cause errors on a changefeed watching the affected tables. You will need to [start a new changefeed](create-changefeed.html#start-a-new-changefeed-where-another-ended). | ||
| - Partial or intermittent sink unavailability may impact changefeed stability; however, [ordering guarantees](stream-data-out-of-cockroachdb-using-changefeeds.html#ordering-guarantees) will still hold for as long as a changefeed [remains active](stream-data-out-of-cockroachdb-using-changefeeds.html#monitor-a-changefeed). | ||
| - When an [`IMPORT INTO`](import-into.html) statement is run, changefeed jobs targeting that table will fail. |
There was a problem hiding this comment.
same re: do we mean any current jobs?
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Closes DOC-1112
Before, cdc known limitations did not have tracking GH issues for the limitations — with the help of @amruss we have updated these with relevant issues. Also moved items that are general considerations into their own section, which helps to define the difference between limitation & more general considerations/guidelines when using changefeeds.