Add documentation for new bloom filter settings #6449
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mgodwan
requested review from
hdhalter,
kolchfa-aws,
Naarcha-AWS,
vagimeli,
AMoo-Miki,
natebower and
dlvenable
as code owners
Signed-off-by: mgodwan <[email protected]>
4 tasks
hdhalter
added
4 - Doc review
vagimeli
approved these changes
Feb 20, 2024
_install-and-configure/configuring-opensearch/index-settings.md
Outdated
Show resolved
Hide resolved
_install-and-configure/configuring-opensearch/index-settings.md
Outdated
Show resolved
Hide resolved
|
Thanks for reviewing this PR so quickly! @vagimeli |
Signed-off-by: mgodwan <[email protected]>
|
Thanks @vagimeli for the review. I've addressed the comments based on your suggestions. |
natebower
reviewed
Feb 20, 2024
_install-and-configure/configuring-opensearch/index-settings.md
Outdated
Show resolved
Hide resolved
| @@ -182,6 +182,10 @@ OpenSearch supports the following dynamic index-level index settings: | |||
|
|
|||
| - `index.final_pipeline` (String): The final ingest node pipeline for the index. If the final pipeline is set and the pipeline does not exist, then index requests fail. The pipeline name `_none` specifies that the index does not have an ingest pipeline. | |||
|
|
|||
| - `index.optimize_doc_id_lookup.fuzzy_set.enabled` (Boolean): This setting controls whether `fuzzy_set` should be enabled for optimizing document ID lookups in indexing or searching calls by using an additional data structure, in this case, the Bloom filter data structure. Enabling this setting improves performance for upsert and search operations that rely on document ID by creating a new data structure (Bloom filter). The Bloom filter allows for the handling of negative cases (that is, IDs being absent in the existing index) through faster off-heap look-ups. Default is `false`. This setting can only be used if the feature flag `opensearch.experimental.optimize_doc_id_lookup.fuzzy_set.enabled` is set to `true`. | |||
There was a problem hiding this comment.
"in order to optimize" instead of "for optimizing"? "lookups in index or search calls"?
| @@ -182,6 +182,10 @@ OpenSearch supports the following dynamic index-level index settings: | |||
|
|
|||
| - `index.final_pipeline` (String): The final ingest node pipeline for the index. If the final pipeline is set and the pipeline does not exist, then index requests fail. The pipeline name `_none` specifies that the index does not have an ingest pipeline. | |||
|
|
|||
| - `index.optimize_doc_id_lookup.fuzzy_set.enabled` (Boolean): This setting controls whether `fuzzy_set` should be enabled for optimizing document ID lookups in indexing or searching calls by using an additional data structure, in this case, the Bloom filter data structure. Enabling this setting improves performance for upsert and search operations that rely on document ID by creating a new data structure (Bloom filter). The Bloom filter allows for the handling of negative cases (that is, IDs being absent in the existing index) through faster off-heap look-ups. Default is `false`. This setting can only be used if the feature flag `opensearch.experimental.optimize_doc_id_lookup.fuzzy_set.enabled` is set to `true`. | |||
|
|
|||
| - `index.optimize_doc_id_lookup.fuzzy_set.false_positive_probability` (Double): Set the false-positive probability for the underlying `fuzzy_set` (that is, the Bloom filter). A lower false-positive probability ensures higher throughput improvement for `UPSERT` or `GET` operations. Allowed values range between `0.01` and `0.50`. Default is `0.20`. This setting can only be used if the feature flag `opensearch.experimental.optimize_doc_id_lookup.fuzzy_set.enabled` is set to `true`. | |||
_install-and-configure/configuring-opensearch/index-settings.md
Outdated
Show resolved
Hide resolved
Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Melissa Vagi <[email protected]>
Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Melissa Vagi <[email protected]>
Signed-off-by: Melissa Vagi <[email protected]> Signed-off-by: Melissa Vagi <[email protected]>
oeyh
pushed a commit
to oeyh/documentation-website
that referenced
this pull request
Mar 14, 2024
) * Add documentation for new bloom filter settings Signed-off-by: mgodwan <[email protected]> * Address PR comments Signed-off-by: mgodwan <[email protected]> * Update _install-and-configure/configuring-opensearch/index-settings.md Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Melissa Vagi <[email protected]> * Update _install-and-configure/configuring-opensearch/index-settings.md Co-authored-by: Nathan Bower <[email protected]> Signed-off-by: Melissa Vagi <[email protected]> * Update index-settings.md Signed-off-by: Melissa Vagi <[email protected]> Signed-off-by: Melissa Vagi <[email protected]> --------- Signed-off-by: mgodwan <[email protected]> Signed-off-by: Melissa Vagi <[email protected]> Co-authored-by: Melissa Vagi <[email protected]> Co-authored-by: Nathan Bower <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Add documentation for new bloom filter settings
Issues Resolved
Closes #6434
Checklist
For more information on following Developer Certificate of Origin and signing off your commits, please check here.