Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[FEATURE] Skipping index and materialized view refresh synchronization #93

Open
dai-chen opened this issue Oct 23, 2023 · 1 comment
Open
Labels
enhancement New feature or request

Comments

@dai-chen
Copy link
Collaborator

Is your feature request related to a problem?

Currently there is no coordination between skipping index and materialized view auto refresh. In the case of materialized view with filtering condition that can be accelerated by skipping index, the initial backlog processing should be fine. However, when MV refresh catches up, each micro batch of skipping index refresh on same source file list is not guaranteed to finish before MV. If hybrid scan is not enabled, skipping index may tell MV there is no file matched the filtering condition for current micro batch.

What solution would you like?

  1. Add basic IT to make sure MV refresh can be accelerated by skipping index
  2. Look into code to verify if the issue above may happen
  3. Quick solution would be enabling hybrid scan mode in this case
@dai-chen
Copy link
Collaborator Author

dai-chen commented Mar 7, 2024

This may be resolved in #218.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Development

No branches or pull requests

1 participant