Add support for index create block #4607

RS146BIJAY · 2022-09-27T12:22:47Z

Signed-off-by: Rishav Sagar [email protected]

Description

We want a way to dynamically trigger a setting to apply index create block. This block should only prevent index creation, while write should continue to happen on the existing index.

Issues Resolved

#4603

Check List

New functionality includes testing.
- All tests pass
New functionality has been documented.
- New functionality has javadoc added
Commits are signed per the DCO using --signoff
Commit changes are listed out in CHANGELOG.md file (See: Changelog)

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

github-actions · 2022-09-27T12:46:31Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/3469/
CommitID: 07407c7

github-actions · 2022-09-27T16:12:27Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/3473/
CommitID: ae3cdf4

github-actions · 2022-09-27T19:10:43Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/3493/
CommitID: 4bd7473

github-actions · 2022-09-27T19:52:22Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/3497/
CommitID: 6dee99c

github-actions · 2022-09-27T21:34:43Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/3501/
CommitID: ac2fa38

github-actions · 2022-09-28T05:12:40Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/3508/
CommitID: a9dfed9

codecov-commenter · 2022-09-28T05:14:26Z

Codecov Report

Merging #4607 (b32763d) into main (be64e82) will decrease coverage by 0.07%.
The diff coverage is 56.52%.

@@             Coverage Diff              @@
##               main    #4607      +/-   ##
============================================
- Coverage     71.07%   70.99%   -0.08%     
+ Complexity    58687    58685       -2     
============================================
  Files          4763     4764       +1     
  Lines        279874   279895      +21     
  Branches      40408    40412       +4     
============================================
- Hits         198907   198705     -202     
- Misses        64658    65033     +375     
+ Partials      16309    16157     -152

Impacted Files	Coverage Δ
...earch/cluster/block/IndexCreateBlockException.java	`0.00% <0.00%> (ø)`
...rg/opensearch/common/settings/ClusterSettings.java	`91.89% <ø> (ø)`
...action/admin/cluster/settings/SettingsUpdater.java	`91.17% <20.00%> (-5.65%)`	⬇️
...va/org/opensearch/cluster/block/ClusterBlocks.java	`83.91% <33.33%> (-0.78%)`	⬇️
.../main/java/org/opensearch/OpenSearchException.java	`92.38% <100.00%> (-1.23%)`	⬇️
...min/indices/create/TransportCreateIndexAction.java	`96.15% <100.00%> (+0.69%)`	⬆️
...rg/opensearch/cluster/block/ClusterBlockLevel.java	`100.00% <100.00%> (ø)`
...java/org/opensearch/cluster/metadata/Metadata.java	`88.27% <100.00%> (-0.61%)`	⬇️
...adonly/AddIndexBlockClusterStateUpdateRequest.java	`0.00% <0.00%> (-75.00%)`	⬇️
...readonly/TransportVerifyShardIndexBlockAction.java	`9.75% <0.00%> (-73.18%)`	⬇️
... and 518 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

server/src/internalClusterTest/java/org/opensearch/blocks/CreateIndexBlockIT.java

server/src/main/java/org/opensearch/cluster/block/ClusterBlocks.java

github-actions · 2022-10-07T07:33:18Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/4108/
CommitID: 9b5b322

github-actions · 2022-10-07T07:39:55Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/4110/
CommitID: 18c9749

server/src/main/java/org/opensearch/OpenSearchException.java

pranikum · 2022-10-09T02:18:05Z

server/src/main/java/org/opensearch/cluster/block/ClusterBlocks.java

@@ -204,6 +204,13 @@ public ClusterBlockException globalBlockedException(ClusterBlockLevel level) {
        return new ClusterBlockException(global(level));
    }

+    public IndexCreateBlockException createIndexBlockedException(ClusterBlockLevel level) {
+        if (globalBlocked(level) == false) {


just use if (!globalBlocked(level))

CHANGELOG.md

sachinpkale · 2022-10-14T06:06:08Z

server/src/main/java/org/opensearch/cluster/block/ClusterBlockLevel.java

@@ -43,7 +43,8 @@ public enum ClusterBlockLevel {
    READ,
    WRITE,
    METADATA_READ,
-    METADATA_WRITE;
+    METADATA_WRITE,
+    CREATE_NEW_INDEX;


nit: Why not just CREATE_INDEX?

Will address this in the next revision.

server/src/main/java/org/opensearch/cluster/metadata/Metadata.java

server/src/main/java/org/opensearch/action/admin/indices/create/TransportCreateIndexAction.java

sachinpkale · 2022-10-14T06:22:58Z

Write PR title using imperative mood.

github-actions · 2022-10-20T09:33:27Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/5040/
CommitID: d304a45

Bukhtawar · 2022-10-20T09:56:27Z

server/src/main/java/org/opensearch/OpenSearchException.java

+            org.opensearch.cluster.block.IndexCreateBlockException.class,
+            org.opensearch.cluster.block.IndexCreateBlockException::new,
+            165,
+            UNKNOWN_VERSION_ADDED


Why UNKNOWN_VERSION_ADDED and not 3.0.0

Will address this in next revision. I have not associated this exception with any ES version similar to other blocks: https://github.com/opensearch-project/OpenSearch/blob/main/server/src/main/java/org/opensearch/OpenSearchException.java#L1045

github-actions · 2022-10-26T15:39:06Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/5682/
CommitID: 9f99dca
Please examine the workflow log, locate, and copy-paste the failure below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2022-10-26T16:38:39Z

Gradle Check (Jenkins) Run Completed with:

RESULT: SUCCESS ✅
URL: https://build.ci.opensearch.org/job/gradle-check/5685/
CommitID: 2730f28

Signed-off-by: Rishav Sagar <[email protected]>

github-actions · 2023-01-05T12:13:47Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      1 org.opensearch.indices.replication.SegmentReplicationIT.testCancelPrimaryAllocation

URL: https://build.ci.opensearch.org/job/gradle-check/8861/
CommitID: 54cf77a
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

github-actions · 2023-01-05T12:21:33Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/8874/
CommitID: 129aa76
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-01-05T12:43:18Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/8875/
CommitID: 1b155d7
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

…ption Signed-off-by: Rishav Sagar <[email protected]>

github-actions · 2023-01-05T13:08:29Z

Gradle Check (Jenkins) Run Completed with:

RESULT: FAILURE ❌
URL: https://build.ci.opensearch.org/job/gradle-check/8878/
CommitID: 052ad75
Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green.
Is the failure a flaky test unrelated to your change?

github-actions · 2023-01-05T13:27:28Z

Gradle Check (Jenkins) Run Completed with:

RESULT: UNSTABLE ❕
TEST FAILURES:

      1 org.opensearch.indices.replication.SegmentReplicationIT.testRestartPrimary

URL: https://build.ci.opensearch.org/job/gradle-check/8881/
CommitID: bd99a2e
Please review all flaky tests that succeeded after retry and create an issue if one does not already exist to track the flaky failure.

opensearch-trigger-bot · 2023-01-11T05:58:56Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-4607-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 c21c899892240db1b8a29f7237257c4c0c81fa99
# Push it to GitHub
git push --set-upstream origin backport/backport-4607-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-4607-to-2.x.

opensearch-trigger-bot · 2023-01-11T05:59:10Z

The backport to 2.5 failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.5 2.5
# Navigate to the new working tree
pushd ../.worktrees/backport-2.5
# Create a new branch
git switch --create backport/backport-4607-to-2.5
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 c21c899892240db1b8a29f7237257c4c0c81fa99
# Push it to GitHub
git push --set-upstream origin backport/backport-4607-to-2.5
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.5

Then, create a pull request where the base branch is 2.5 and the compare/head branch is backport/backport-4607-to-2.5.

andrross · 2023-01-20T23:23:30Z

@RS146BIJAY Are you going to backport this to 2.x?

RS146BIJAY · 2023-01-21T05:24:52Z

@andrross this will go in 2.6 release. Yes I will back port it for 2.x branch.

opensearch-trigger-bot · 2023-01-26T17:23:23Z

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 128

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add ../.worktrees/backport-2.x 2.x
# Navigate to the new working tree
pushd ../.worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-4607-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 c21c899892240db1b8a29f7237257c4c0c81fa99
# Push it to GitHub
git push --set-upstream origin backport/backport-4607-to-2.x
# Go back to the original working tree
popd
# Delete the working tree
git worktree remove ../.worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-4607-to-2.x.

andrross · 2023-01-26T17:23:28Z

@RS146BIJAY Please ensure this PR along with #5852 get backported

RS146BIJAY · 2023-01-26T18:34:40Z

Sure will backport both these PR on 2.x branch on priority.

RS146BIJAY force-pushed the disk-guardrails branch from 07407c7 to ae3cdf4 Compare September 27, 2022 15:49

RS146BIJAY force-pushed the disk-guardrails branch from ae3cdf4 to 4bd7473 Compare September 27, 2022 18:46

RS146BIJAY force-pushed the disk-guardrails branch from 4bd7473 to 6dee99c Compare September 27, 2022 19:28

RS146BIJAY force-pushed the disk-guardrails branch from 6dee99c to ac2fa38 Compare September 27, 2022 21:08

RS146BIJAY force-pushed the disk-guardrails branch from ac2fa38 to a9dfed9 Compare September 28, 2022 04:46

RS146BIJAY marked this pull request as ready for review September 28, 2022 06:48

RS146BIJAY requested review from a team and reta as code owners September 28, 2022 06:48

Gaganjuneja reviewed Oct 3, 2022

View reviewed changes

server/src/internalClusterTest/java/org/opensearch/blocks/CreateIndexBlockIT.java Outdated Show resolved Hide resolved

server/src/main/java/org/opensearch/cluster/block/ClusterBlocks.java Outdated Show resolved Hide resolved

RS146BIJAY force-pushed the disk-guardrails branch from a9dfed9 to 9b5b322 Compare October 7, 2022 07:08

pranikum reviewed Oct 7, 2022

View reviewed changes

server/src/main/java/org/opensearch/OpenSearchException.java Outdated Show resolved Hide resolved

pranikum reviewed Oct 9, 2022

View reviewed changes

sachinpkale reviewed Oct 14, 2022

View reviewed changes

RS146BIJAY changed the title ~~Adding support for index create block in OpenSearch~~ Add support for index create block Oct 20, 2022

Bukhtawar reviewed Oct 20, 2022

View reviewed changes

RS146BIJAY force-pushed the disk-guardrails branch from d304a45 to 9f99dca Compare October 26, 2022 15:11

pranikum approved these changes Oct 31, 2022

View reviewed changes

Rishav Sagar added 3 commits January 5, 2023 17:10

Refactoring cluster block name and modifying PR title in CHANGELOG.md

a4dfb5f

Signed-off-by: Rishav Sagar <[email protected]>

Changing the versionAdded feild for INDEX_CREATE_BLOCK_EXCEPTION

9bff0c9

Signed-off-by: Rishav Sagar <[email protected]>

Fixing failing unit test cases

b8c723a

Signed-off-by: Rishav Sagar <[email protected]>

RS146BIJAY force-pushed the disk-guardrails branch from b32763d to 54cf77a Compare January 5, 2023 11:43

RS146BIJAY force-pushed the disk-guardrails branch 2 times, most recently from 129aa76 to 1b155d7 Compare January 5, 2023 12:20

RS146BIJAY force-pushed the disk-guardrails branch from 1b155d7 to 052ad75 Compare January 5, 2023 12:35

Refactoring variable name and changing id for index create block exce…

bd99a2e

…ption Signed-off-by: Rishav Sagar <[email protected]>

RS146BIJAY force-pushed the disk-guardrails branch from 052ad75 to bd99a2e Compare January 5, 2023 12:58

Bukhtawar merged commit c21c899 into opensearch-project:main Jan 5, 2023

gbbafna added backport 2.x Backport to 2.x branch backport 2.5 labels Jan 11, 2023

gbbafna removed backport 2.x Backport to 2.x branch backport 2.5 labels Jan 11, 2023

andrross mentioned this pull request Jan 19, 2023

[Backport 2.x] Adding index create block when all nodes have breached high disk watermark #5943

Closed

andrross added backport 2.x Backport to 2.x branch v2.6.0 'Issues and PRs related to version v2.6.0' labels Jan 26, 2023

RS146BIJAY mentioned this pull request Jan 30, 2023

[Backport 2.x] Add support for index create block #6068

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add support for index create block #4607

Add support for index create block #4607

RS146BIJAY commented Sep 27, 2022 •

edited

Loading

github-actions bot commented Sep 27, 2022

github-actions bot commented Sep 27, 2022

github-actions bot commented Sep 27, 2022

github-actions bot commented Sep 27, 2022

github-actions bot commented Sep 27, 2022

github-actions bot commented Sep 28, 2022

codecov-commenter commented Sep 28, 2022 •

edited

Loading

github-actions bot commented Oct 7, 2022

github-actions bot commented Oct 7, 2022

pranikum Oct 9, 2022

sachinpkale Oct 14, 2022

RS146BIJAY Oct 17, 2022

sachinpkale commented Oct 14, 2022

github-actions bot commented Oct 20, 2022

Bukhtawar Oct 20, 2022

RS146BIJAY Oct 26, 2022

github-actions bot commented Oct 26, 2022

github-actions bot commented Oct 26, 2022

github-actions bot commented Jan 5, 2023

github-actions bot commented Jan 5, 2023

github-actions bot commented Jan 5, 2023

github-actions bot commented Jan 5, 2023

github-actions bot commented Jan 5, 2023

opensearch-trigger-bot bot commented Jan 11, 2023

opensearch-trigger-bot bot commented Jan 11, 2023

andrross commented Jan 20, 2023

RS146BIJAY commented Jan 21, 2023

opensearch-trigger-bot bot commented Jan 26, 2023

andrross commented Jan 26, 2023

RS146BIJAY commented Jan 26, 2023

Add support for index create block #4607

Add support for index create block #4607

Conversation

RS146BIJAY commented Sep 27, 2022 • edited Loading

Description

Issues Resolved

Check List

github-actions bot commented Sep 27, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 27, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 27, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 27, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 27, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Sep 28, 2022

Gradle Check (Jenkins) Run Completed with:

codecov-commenter commented Sep 28, 2022 • edited Loading

Codecov Report

github-actions bot commented Oct 7, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 7, 2022

Gradle Check (Jenkins) Run Completed with:

pranikum Oct 9, 2022

Choose a reason for hiding this comment

sachinpkale Oct 14, 2022

Choose a reason for hiding this comment

RS146BIJAY Oct 17, 2022

Choose a reason for hiding this comment

sachinpkale commented Oct 14, 2022

github-actions bot commented Oct 20, 2022

Gradle Check (Jenkins) Run Completed with:

Bukhtawar Oct 20, 2022

Choose a reason for hiding this comment

RS146BIJAY Oct 26, 2022

Choose a reason for hiding this comment

github-actions bot commented Oct 26, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Oct 26, 2022

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jan 5, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jan 5, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jan 5, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jan 5, 2023

Gradle Check (Jenkins) Run Completed with:

github-actions bot commented Jan 5, 2023

Gradle Check (Jenkins) Run Completed with:

opensearch-trigger-bot bot commented Jan 11, 2023

opensearch-trigger-bot bot commented Jan 11, 2023

andrross commented Jan 20, 2023

RS146BIJAY commented Jan 21, 2023

opensearch-trigger-bot bot commented Jan 26, 2023

andrross commented Jan 26, 2023

RS146BIJAY commented Jan 26, 2023

RS146BIJAY commented Sep 27, 2022 •

edited

Loading

codecov-commenter commented Sep 28, 2022 •

edited

Loading