Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle ALTER Index Queries. #2554

Merged

Conversation

vmmusings
Copy link
Member

@vmmusings vmmusings commented Mar 13, 2024

Description

  • ALTER INDEX syntax updates.
  • Handle ALTER INDEX Queries with auto_refresh=false in sql plugin as it requires stopping spark job.
  • Sending rest of the ALTER INDEX queries are streaming queries in separate job to spark instance.
  • Refactored existing drop operation.
  • Introduced rolling back in flint index operations.

Issues Resolved

[List any issues this PR will resolve]

Check List

  • New functionality includes testing.
    • All tests pass, including unit test, integration test and doctest
  • New functionality has been documented.
    • New functionality has javadoc added
    • New functionality has user manual doc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link

codecov bot commented Mar 13, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 95.53%. Comparing base (a84c3ef) to head (4319e3f).

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #2554      +/-   ##
============================================
+ Coverage     95.47%   95.53%   +0.05%     
- Complexity     5046     5101      +55     
============================================
  Files           486      488       +2     
  Lines         14087    14261     +174     
  Branches        943      958      +15     
============================================
+ Hits          13450    13624     +174     
  Misses          614      614              
  Partials         23       23              
Flag Coverage Δ
sql-engine 95.53% <100.00%> (+0.05%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@vmmusings vmmusings force-pushed the index-metadata-refactor branch 2 times, most recently from dc770a0 to a19cf56 Compare March 13, 2024 20:09
seankao-az
seankao-az previously approved these changes Mar 13, 2024
@vmmusings vmmusings force-pushed the index-metadata-refactor branch 3 times, most recently from 7606bf7 to 2cc0dc3 Compare March 14, 2024 01:15
@vmmusings vmmusings force-pushed the index-metadata-refactor branch from 0d90565 to d90d80b Compare March 18, 2024 15:43
@vmmusings vmmusings force-pushed the index-metadata-refactor branch from d90d80b to 43a4bf6 Compare March 18, 2024 16:45
@dai-chen dai-chen mentioned this pull request Mar 18, 2024
6 tasks
@vmmusings vmmusings force-pushed the index-metadata-refactor branch from 43a4bf6 to da7e70f Compare March 18, 2024 19:20
@vmmusings vmmusings force-pushed the index-metadata-refactor branch 2 times, most recently from 43441ea to 31ce5cf Compare March 18, 2024 20:12
@dai-chen dai-chen mentioned this pull request Mar 18, 2024
6 tasks
@vmmusings vmmusings force-pushed the index-metadata-refactor branch from 31ce5cf to 15be8b1 Compare March 18, 2024 20:30
@dai-chen
Copy link
Collaborator

This PR also incorporated latest Flint SQL grammar changes below, so there is no need to publish separate PR. @rupal-bq @seankao-az

  1. ALTER statement: Add grammar files for alter index opensearch-spark#279
  2. BLOOM_FILTER skip type: Add BloomFilter skipping index SQL support opensearch-spark#283
  3. ANALYZE statement: Implement analyze skipping index statement opensearch-spark#284

dai-chen
dai-chen previously approved these changes Mar 18, 2024
@vmmusings vmmusings force-pushed the index-metadata-refactor branch 4 times, most recently from bf374fb to 67ef580 Compare March 18, 2024 21:47
@vmmusings vmmusings force-pushed the index-metadata-refactor branch from 67ef580 to 4319e3f Compare March 18, 2024 22:02
@vmmusings vmmusings merged commit d11a268 into opensearch-project:main Mar 18, 2024
27 of 28 checks passed
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Navigate to the root of your repository
cd $(git rev-parse --show-toplevel)
# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/sql/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/sql/backport-2.x
# Create a new branch
git switch --create backport/backport-2554-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 d11a26865dea8649fa4611084db08230ab9e7382
# Push it to GitHub
git push --set-upstream origin backport/backport-2554-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/sql/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-2554-to-2.x.

vmmusings added a commit that referenced this pull request Mar 18, 2024
Signed-off-by: Vamsi Manohar <[email protected]>
(cherry picked from commit d11a268)
vmmusings added a commit that referenced this pull request Mar 19, 2024
Signed-off-by: Vamsi Manohar <[email protected]>
(cherry picked from commit d11a268)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x enhancement New feature or request v2.13.0 Issues targeting release v2.13.0
Projects
Status: No status
Development

Successfully merging this pull request may close these issues.

5 participants