Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
changefeedccl: Bump default per changefeed limit
Inrease default changefeed limit to 512MiB. The limit was previously reduced from 1GiB due to the concerns of changefeed impact on foreground latency. That impact in turn was due how changefeed allocated and management memory. The issue leading to the reduction in this setting default have been addressed (via improvememnts in memory management, leading to the changefeed impact reduction on the Go runtime, as well as addition of CPU pacing). The 128MiB default limit is too low for file based sinks. By default, those sinks buffer 16MB of data, and often times this is combined with compression. If the data is highly compressible (5-10x, which is not uncommon), changefeds wind up using the entirety of memory budget, before reaching their target file size, resulting in small files being produced. This PR splits a difference between original, 1GiB setting, and the new 128MiB setting by increasing default setting to 1/2GiB. Epic: None Release note (enterprise change): Bump up default `changefeed.memory.per_changefeed_limit` setting to 1/2GiB. This should result in changefeeds being able to produce larger files.
- Loading branch information