From ded24153401d9447bb12d628a256c7ad62c962e3 Mon Sep 17 00:00:00 2001 From: zhichao-aws Date: Tue, 16 Jul 2024 22:38:04 +0800 Subject: [PATCH 01/36] Improve wording for the 2 search mode of neural sparse documentation (#7718) * improve wording for ns Signed-off-by: zhichao-aws * Apply suggestions from code review Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: zhichao-aws Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --- _search-plugins/neural-sparse-search.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/_search-plugins/neural-sparse-search.md b/_search-plugins/neural-sparse-search.md index b2b4fc33d6..8aa2ff7dbf 100644 --- a/_search-plugins/neural-sparse-search.md +++ b/_search-plugins/neural-sparse-search.md @@ -16,8 +16,8 @@ Introduced 2.11 When selecting a model, choose one of the following options: -- Use a sparse encoding model at both ingestion time and search time (high performance, relatively high latency). -- Use a sparse encoding model at ingestion time and a tokenizer at search time for relatively low performance and low latency. The tokenism doesn't conduct model inference, so you can deploy and invoke a tokenizer using the ML Commons Model API for a more consistent experience. +- Use a sparse encoding model at both ingestion time and search time for better search relevance at the expense of relatively high latency. +- Use a sparse encoding model at ingestion time and a tokenizer at search time for lower search latency at the expense of relatively lower search relevance. Tokenization doesn't involve model inference, so you can deploy and invoke a tokenizer using the ML Commons Model API for a more streamlined experience. **PREREQUISITE**
Before using neural sparse search, make sure to set up a [pretrained sparse embedding model]({{site.url}}{{site.baseurl}}/ml-commons-plugin/pretrained-models/#sparse-encoding-models) or your own sparse embedding model. For more information, see [Choosing a model]({{site.url}}{{site.baseurl}}/ml-commons-plugin/integrating-ml-models/#choosing-a-model). From da41ad8322b5e0979430325ddeffa7e754510308 Mon Sep 17 00:00:00 2001 From: Landon Lengyel Date: Tue, 16 Jul 2024 12:55:08 -0600 Subject: [PATCH 02/36] Update nodes-stats.md (#7721) Fixing typo Signed-off-by: Landon Lengyel --- _api-reference/nodes-apis/nodes-stats.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_api-reference/nodes-apis/nodes-stats.md b/_api-reference/nodes-apis/nodes-stats.md index 145b3d5b24..ca6810b961 100644 --- a/_api-reference/nodes-apis/nodes-stats.md +++ b/_api-reference/nodes-apis/nodes-stats.md @@ -44,7 +44,7 @@ thread_pool | Statistics about each thread pool for the node. fs | File system statistics, such as read/write statistics, data path, and free disk space. transport | Transport layer statistics about send/receive in cluster communication. http | Statistics about the HTTP layer. -breaker | Statistics about the field data circuit breakers. +breakers | Statistics about the field data circuit breakers. script | Statistics about scripts, such as compilations and cache evictions. discovery | Statistics about cluster states. ingest | Statistics about ingest pipelines. From b6478f593377089e44efebf320d258b4b4f3e823 Mon Sep 17 00:00:00 2001 From: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Date: Tue, 16 Jul 2024 16:37:57 -0400 Subject: [PATCH 03/36] Update PR comment workflow to use pull request target (#7723) Signed-off-by: Fanit Kolchina --- .github/workflows/pr_checklist.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/pr_checklist.yml b/.github/workflows/pr_checklist.yml index accce0f882..4130f5e2bd 100644 --- a/.github/workflows/pr_checklist.yml +++ b/.github/workflows/pr_checklist.yml @@ -1,7 +1,7 @@ name: PR Checklist on: - pull_request: + pull_request_target: types: [opened] permissions: From e3ee238aadac34a42b6880dede93c01c0a50dcda Mon Sep 17 00:00:00 2001 From: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Date: Tue, 16 Jul 2024 17:09:39 -0400 Subject: [PATCH 04/36] Add 1.3.18 to version history (#7726) Signed-off-by: Fanit Kolchina --- _about/version-history.md | 1 + 1 file changed, 1 insertion(+) diff --git a/_about/version-history.md b/_about/version-history.md index 0d6d844951..09f331b235 100644 --- a/_about/version-history.md +++ b/_about/version-history.md @@ -30,6 +30,7 @@ OpenSearch version | Release highlights | Release date [2.0.1](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.0.1.md) | Includes bug fixes and maintenance updates for Alerting and Anomaly Detection. | 16 June 2022 [2.0.0](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.0.0.md) | Includes document-level monitors for alerting, OpenSearch Notifications plugins, and Geo Map Tiles in OpenSearch Dashboards. Also adds support for Lucene 9 and bug fixes for all OpenSearch plugins. For a full list of release highlights, see the Release Notes. | 26 May 2022 [2.0.0-rc1](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-2.0.0-rc1.md) | The Release Candidate for 2.0.0. This version allows you to preview the upcoming 2.0.0 release before the GA release. The preview release adds document-level alerting, support for Lucene 9, and the ability to use term lookup queries in document level security. | 03 May 2022 +[1.3.18](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-1.3.18.md) | Includes maintenance updates for OpenSearch security. | 16 July 2024 [1.3.17](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-1.3.17.md) | Includes maintenance updates for OpenSearch security and OpenSearch Dashboards security. | 06 June 2024 [1.3.16](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-1.3.16.md) | Includes bug fixes and maintenance updates for OpenSearch security, index management, performance analyzer, and reporting. | 23 April 2024 [1.3.15](https://github.com/opensearch-project/opensearch-build/blob/main/release-notes/opensearch-release-notes-1.3.15.md) | Includes bug fixes and maintenance updates for cross-cluster replication, SQL, OpenSearch Dashboards reporting, and alerting. | 05 March 2024 From e05b4004a77c0b5208a339e6abef98121e43b690 Mon Sep 17 00:00:00 2001 From: Heather Halter Date: Wed, 17 Jul 2024 11:41:44 -0700 Subject: [PATCH 05/36] Updates PPL description (#7637) * Update index.md Updated description of PPL. Signed-off-by: Heather Halter * Update index.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update search-plugins/sql/ppl/index/ Signed-off-by: Heather Halter * Update _search-plugins/sql/ppl/index.md Co-authored-by: Nathan Bower Signed-off-by: Heather Halter --------- Signed-off-by: Heather Halter Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Nathan Bower --- _search-plugins/sql/ppl/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_search-plugins/sql/ppl/index.md b/_search-plugins/sql/ppl/index.md index 602255d126..bda67aba36 100644 --- a/_search-plugins/sql/ppl/index.md +++ b/_search-plugins/sql/ppl/index.md @@ -18,7 +18,7 @@ redirect_from: # PPL -Piped Processing Language (PPL) is a query language that focuses on processing data in a sequential, step-by-step manner. PPL uses the pipe (`|`) operator to combine commands to find and retrieve data. It is the primary language used with observability in OpenSearch and supports multi-data queries. +Piped Processing Language (PPL) is a query language that focuses on processing data in a sequential, step-by-step manner. PPL uses the pipe (`|`) operator to combine commands to find and retrieve data. It is particularly well suited for analyzing observability data, such as logs, metrics, and traces, due to its ability to handle semi-structured data efficiently. ## PPL syntax From d2da95d9cbec2d325df083bca817880af32d3180 Mon Sep 17 00:00:00 2001 From: Miki Date: Wed, 17 Jul 2024 12:41:35 -0700 Subject: [PATCH 06/36] Redirect to `latest` when the latest version is picked from the version selector (#7759) Signed-off-by: Miki --- assets/js/_version-selector.js | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/assets/js/_version-selector.js b/assets/js/_version-selector.js index e5efcaaf2c..23730e67f0 100644 --- a/assets/js/_version-selector.js +++ b/assets/js/_version-selector.js @@ -192,7 +192,10 @@ class VersionSelector extends HTMLElement { frag.querySelector('#selected').textContent = `${PREFIX}${this.getAttribute('selected')}`; const pathName = location.pathname.replace(/\/docs(\/((latest|\d+\.\d+)\/?)?)?/, ''); - const versionsDOMNodes = DOC_VERSIONS.map((v, idx) => `${PREFIX}${v}`); + const versionsDOMNodes = DOC_VERSIONS.map((v, idx) => v === DOC_VERSION_LATEST + ? `${PREFIX}${v}` + : `${PREFIX}${v}`, + ); if (Array.isArray(DOC_VERSIONS_ARCHIVED) && DOC_VERSIONS_ARCHIVED.length) { versionsDOMNodes.push( `Show archived`, From d8f1b7b95d3d90f130b6b590ee7526ca0c8a186a Mon Sep 17 00:00:00 2001 From: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Date: Wed, 17 Jul 2024 16:56:02 -0400 Subject: [PATCH 07/36] Fix breadcrumbs by excluding collection index pages from parent relationship (#7758) Signed-off-by: Fanit Kolchina --- _aggregations/bucket/adjacency-matrix.md | 1 - _aggregations/bucket/date-histogram.md | 1 - _aggregations/bucket/date-range.md | 1 - _aggregations/bucket/diversified-sampler.md | 1 - _aggregations/bucket/filter.md | 1 - _aggregations/bucket/filters.md | 1 - _aggregations/bucket/geo-distance.md | 1 - _aggregations/bucket/geohash-grid.md | 1 - _aggregations/bucket/geohex-grid.md | 1 - _aggregations/bucket/geotile-grid.md | 1 - _aggregations/bucket/global.md | 1 - _aggregations/bucket/histogram.md | 1 - _aggregations/bucket/ip-range.md | 1 - _aggregations/bucket/missing.md | 1 - _aggregations/bucket/multi-terms.md | 1 - _aggregations/bucket/nested.md | 1 - _aggregations/bucket/range.md | 1 - _aggregations/bucket/reverse-nested.md | 1 - _aggregations/bucket/sampler.md | 1 - _aggregations/bucket/significant-terms.md | 1 - _aggregations/bucket/significant-text.md | 1 - _aggregations/bucket/terms.md | 1 - _aggregations/index.md | 2 +- _aggregations/metric/average.md | 1 - _aggregations/metric/cardinality.md | 1 - _aggregations/metric/extended-stats.md | 1 - _aggregations/metric/geobounds.md | 1 - _aggregations/metric/geocentroid.md | 1 - _aggregations/metric/matrix-stats.md | 1 - _aggregations/metric/maximum.md | 1 - _aggregations/metric/median-absolute-deviation.md | 1 - _aggregations/metric/minimum.md | 1 - _aggregations/metric/percentile-ranks.md | 1 - _aggregations/metric/percentile.md | 1 - _aggregations/metric/scripted-metric.md | 1 - _aggregations/metric/stats.md | 1 - _aggregations/metric/sum.md | 1 - _aggregations/metric/top-hits.md | 1 - _aggregations/metric/value-count.md | 1 - _aggregations/metric/weighted-avg.md | 1 - _query-dsl/compound/bool.md | 1 - _query-dsl/compound/boosting.md | 1 - _query-dsl/compound/constant-score.md | 1 - _query-dsl/compound/disjunction-max.md | 1 - _query-dsl/compound/function-score.md | 1 - _query-dsl/compound/hybrid.md | 1 - _query-dsl/full-text/intervals.md | 1 - _query-dsl/full-text/match-bool-prefix.md | 1 - _query-dsl/full-text/match-phrase-prefix.md | 1 - _query-dsl/full-text/match-phrase.md | 1 - _query-dsl/full-text/match.md | 1 - _query-dsl/full-text/multi-match.md | 1 - _query-dsl/full-text/query-string.md | 2 -- _query-dsl/full-text/simple-query-string.md | 1 - _query-dsl/geo-and-xy/geo-bounding-box.md | 1 - _query-dsl/geo-and-xy/geodistance.md | 1 - _query-dsl/geo-and-xy/geopolygon.md | 1 - _query-dsl/geo-and-xy/xy.md | 2 -- _query-dsl/specialized/neural-sparse.md | 1 - _query-dsl/specialized/neural.md | 1 - _query-dsl/specialized/script-score.md | 1 - _query-dsl/term/exists.md | 1 - _query-dsl/term/fuzzy.md | 1 - _query-dsl/term/ids.md | 1 - _query-dsl/term/prefix.md | 1 - _query-dsl/term/range.md | 1 - _query-dsl/term/regexp.md | 1 - _query-dsl/term/term.md | 1 - _query-dsl/term/terms-set.md | 1 - _query-dsl/term/terms.md | 1 - _query-dsl/term/wildcard.md | 1 - _search-plugins/knn/api.md | 1 - _search-plugins/knn/approximate-knn.md | 1 - _search-plugins/knn/filter-search-knn.md | 1 - _search-plugins/knn/jni-libraries.md | 1 - _search-plugins/knn/knn-index.md | 1 - _search-plugins/knn/knn-score-script.md | 1 - _search-plugins/knn/knn-vector-quantization.md | 1 - _search-plugins/knn/nested-search-knn.md | 1 - _search-plugins/knn/painless-functions.md | 1 - _search-plugins/knn/performance-tuning.md | 1 - _search-plugins/knn/radial-search-knn.md | 1 - _search-plugins/knn/settings.md | 1 - 83 files changed, 1 insertion(+), 85 deletions(-) diff --git a/_aggregations/bucket/adjacency-matrix.md b/_aggregations/bucket/adjacency-matrix.md index fd521f8510..cf62295763 100644 --- a/_aggregations/bucket/adjacency-matrix.md +++ b/_aggregations/bucket/adjacency-matrix.md @@ -2,7 +2,6 @@ layout: default title: Adjacency matrix parent: Bucket aggregations -grand_parent: Aggregations nav_order: 10 redirect_from: - /query-dsl/aggregations/bucket/adjacency-matrix/ diff --git a/_aggregations/bucket/date-histogram.md b/_aggregations/bucket/date-histogram.md index e308104e16..db3fe6884e 100644 --- a/_aggregations/bucket/date-histogram.md +++ b/_aggregations/bucket/date-histogram.md @@ -2,7 +2,6 @@ layout: default title: Date histogram parent: Bucket aggregations -grand_parent: Aggregations nav_order: 20 redirect_from: - /query-dsl/aggregations/bucket/date-histogram/ diff --git a/_aggregations/bucket/date-range.md b/_aggregations/bucket/date-range.md index c7d66d729d..d2498a53da 100644 --- a/_aggregations/bucket/date-range.md +++ b/_aggregations/bucket/date-range.md @@ -2,7 +2,6 @@ layout: default title: Date range parent: Bucket aggregations -grand_parent: Aggregations nav_order: 30 redirect_from: - /query-dsl/aggregations/bucket/date-range/ diff --git a/_aggregations/bucket/diversified-sampler.md b/_aggregations/bucket/diversified-sampler.md index 7249ac3555..a62410cc8c 100644 --- a/_aggregations/bucket/diversified-sampler.md +++ b/_aggregations/bucket/diversified-sampler.md @@ -2,7 +2,6 @@ layout: default title: Diversified sampler parent: Bucket aggregations -grand_parent: Aggregations nav_order: 40 redirect_from: - /query-dsl/aggregations/bucket/diversified-sampler/ diff --git a/_aggregations/bucket/filter.md b/_aggregations/bucket/filter.md index 0768ea1148..58624c222c 100644 --- a/_aggregations/bucket/filter.md +++ b/_aggregations/bucket/filter.md @@ -2,7 +2,6 @@ layout: default title: Filter parent: Bucket aggregations -grand_parent: Aggregations nav_order: 50 redirect_from: - /query-dsl/aggregations/bucket/filter/ diff --git a/_aggregations/bucket/filters.md b/_aggregations/bucket/filters.md index b3977da7c1..2e9270b30d 100644 --- a/_aggregations/bucket/filters.md +++ b/_aggregations/bucket/filters.md @@ -2,7 +2,6 @@ layout: default title: Filters parent: Bucket aggregations -grand_parent: Aggregations nav_order: 60 redirect_from: - /query-dsl/aggregations/bucket/filters/ diff --git a/_aggregations/bucket/geo-distance.md b/_aggregations/bucket/geo-distance.md index a111015ac1..7b8e660630 100644 --- a/_aggregations/bucket/geo-distance.md +++ b/_aggregations/bucket/geo-distance.md @@ -2,7 +2,6 @@ layout: default title: Geodistance parent: Bucket aggregations -grand_parent: Aggregations nav_order: 70 redirect_from: - /query-dsl/aggregations/bucket/geo-distance/ diff --git a/_aggregations/bucket/geohash-grid.md b/_aggregations/bucket/geohash-grid.md index 13f89799ba..3969ea9a13 100644 --- a/_aggregations/bucket/geohash-grid.md +++ b/_aggregations/bucket/geohash-grid.md @@ -2,7 +2,6 @@ layout: default title: Geohash grid parent: Bucket aggregations -grand_parent: Aggregations nav_order: 80 redirect_from: - /query-dsl/aggregations/bucket/geohash-grid/ diff --git a/_aggregations/bucket/geohex-grid.md b/_aggregations/bucket/geohex-grid.md index 03fd45e369..eef2ed0b21 100644 --- a/_aggregations/bucket/geohex-grid.md +++ b/_aggregations/bucket/geohex-grid.md @@ -2,7 +2,6 @@ layout: default title: Geohex grid parent: Bucket aggregations -grand_parent: Aggregations nav_order: 85 redirect_from: - /opensearch/geohexgrid-agg/ diff --git a/_aggregations/bucket/geotile-grid.md b/_aggregations/bucket/geotile-grid.md index dd0c4f8a1f..e8e80451f5 100644 --- a/_aggregations/bucket/geotile-grid.md +++ b/_aggregations/bucket/geotile-grid.md @@ -2,7 +2,6 @@ layout: default title: Geotile grid parent: Bucket aggregations -grand_parent: Aggregations nav_order: 87 redirect_from: - /query-dsl/aggregations/bucket/geotile-grid/ diff --git a/_aggregations/bucket/global.md b/_aggregations/bucket/global.md index bfd516b8a3..483c28a69e 100644 --- a/_aggregations/bucket/global.md +++ b/_aggregations/bucket/global.md @@ -2,7 +2,6 @@ layout: default title: Global parent: Bucket aggregations -grand_parent: Aggregations nav_order: 90 redirect_from: - /query-dsl/aggregations/bucket/global/ diff --git a/_aggregations/bucket/histogram.md b/_aggregations/bucket/histogram.md index 0d9f2bb964..b97755bcae 100644 --- a/_aggregations/bucket/histogram.md +++ b/_aggregations/bucket/histogram.md @@ -2,7 +2,6 @@ layout: default title: Histogram parent: Bucket aggregations -grand_parent: Aggregations nav_order: 100 redirect_from: - /query-dsl/aggregations/bucket/histogram/ diff --git a/_aggregations/bucket/ip-range.md b/_aggregations/bucket/ip-range.md index 897827d412..bef4eb8f1f 100644 --- a/_aggregations/bucket/ip-range.md +++ b/_aggregations/bucket/ip-range.md @@ -2,7 +2,6 @@ layout: default title: IP range parent: Bucket aggregations -grand_parent: Aggregations nav_order: 110 redirect_from: - /query-dsl/aggregations/bucket/ip-range/ diff --git a/_aggregations/bucket/missing.md b/_aggregations/bucket/missing.md index 547076859d..e9bd5981a1 100644 --- a/_aggregations/bucket/missing.md +++ b/_aggregations/bucket/missing.md @@ -2,7 +2,6 @@ layout: default title: Missing parent: Bucket aggregations -grand_parent: Aggregations nav_order: 120 redirect_from: - /query-dsl/aggregations/bucket/missing/ diff --git a/_aggregations/bucket/multi-terms.md b/_aggregations/bucket/multi-terms.md index 62a4d264e0..8c99a450fe 100644 --- a/_aggregations/bucket/multi-terms.md +++ b/_aggregations/bucket/multi-terms.md @@ -2,7 +2,6 @@ layout: default title: Multi-terms parent: Bucket aggregations -grand_parent: Aggregations nav_order: 130 redirect_from: - /query-dsl/aggregations/bucket/multi-terms/ diff --git a/_aggregations/bucket/nested.md b/_aggregations/bucket/nested.md index 94a0f4416a..89c44c6457 100644 --- a/_aggregations/bucket/nested.md +++ b/_aggregations/bucket/nested.md @@ -2,7 +2,6 @@ layout: default title: Nested parent: Bucket aggregations -grand_parent: Aggregations nav_order: 140 redirect_from: - /query-dsl/aggregations/bucket/nested/ diff --git a/_aggregations/bucket/range.md b/_aggregations/bucket/range.md index f4e19f188d..7b17b64ed4 100644 --- a/_aggregations/bucket/range.md +++ b/_aggregations/bucket/range.md @@ -2,7 +2,6 @@ layout: default title: Range parent: Bucket aggregations -grand_parent: Aggregations nav_order: 150 redirect_from: - /query-dsl/aggregations/bucket/date-range/ diff --git a/_aggregations/bucket/reverse-nested.md b/_aggregations/bucket/reverse-nested.md index bfd04986fa..3757ec7ab7 100644 --- a/_aggregations/bucket/reverse-nested.md +++ b/_aggregations/bucket/reverse-nested.md @@ -2,7 +2,6 @@ layout: default title: Reverse nested parent: Bucket aggregations -grand_parent: Aggregations nav_order: 160 redirect_from: - /query-dsl/aggregations/bucket/reverse-nested/ diff --git a/_aggregations/bucket/sampler.md b/_aggregations/bucket/sampler.md index 5411052d45..7d8d71e04a 100644 --- a/_aggregations/bucket/sampler.md +++ b/_aggregations/bucket/sampler.md @@ -2,7 +2,6 @@ layout: default title: Sampler parent: Bucket aggregations -grand_parent: Aggregations nav_order: 170 redirect_from: - /query-dsl/aggregations/bucket/diversified-sampler/ diff --git a/_aggregations/bucket/significant-terms.md b/_aggregations/bucket/significant-terms.md index 34a4354a73..c255379dd8 100644 --- a/_aggregations/bucket/significant-terms.md +++ b/_aggregations/bucket/significant-terms.md @@ -2,7 +2,6 @@ layout: default title: Significant terms parent: Bucket aggregations -grand_parent: Aggregations nav_order: 180 redirect_from: - /query-dsl/aggregations/bucket/significant-terms/ diff --git a/_aggregations/bucket/significant-text.md b/_aggregations/bucket/significant-text.md index 6f1c7ebeca..b30b3aba95 100644 --- a/_aggregations/bucket/significant-text.md +++ b/_aggregations/bucket/significant-text.md @@ -2,7 +2,6 @@ layout: default title: Significant text parent: Bucket aggregations -grand_parent: Aggregations nav_order: 190 redirect_from: - /query-dsl/aggregations/bucket/significant-text/ diff --git a/_aggregations/bucket/terms.md b/_aggregations/bucket/terms.md index 5d05c328d4..b36214e3f6 100644 --- a/_aggregations/bucket/terms.md +++ b/_aggregations/bucket/terms.md @@ -2,7 +2,6 @@ layout: default title: Terms parent: Bucket aggregations -grand_parent: Aggregations nav_order: 200 redirect_from: - /query-dsl/aggregations/bucket/terms/ diff --git a/_aggregations/index.md b/_aggregations/index.md index 385c7a09d8..a1c457be02 100644 --- a/_aggregations/index.md +++ b/_aggregations/index.md @@ -1,7 +1,7 @@ --- layout: default title: Aggregations -has_children: true +has_children: false nav_order: 5 nav_exclude: true permalink: /aggregations/ diff --git a/_aggregations/metric/average.md b/_aggregations/metric/average.md index 428f1e76b6..9ad0c582fe 100644 --- a/_aggregations/metric/average.md +++ b/_aggregations/metric/average.md @@ -2,7 +2,6 @@ layout: default title: Average parent: Metric aggregations -grand_parent: Aggregations nav_order: 10 redirect_from: - /query-dsl/aggregations/metric/average/ diff --git a/_aggregations/metric/cardinality.md b/_aggregations/metric/cardinality.md index c40dbb4497..e03a561adb 100644 --- a/_aggregations/metric/cardinality.md +++ b/_aggregations/metric/cardinality.md @@ -2,7 +2,6 @@ layout: default title: Cardinality parent: Metric aggregations -grand_parent: Aggregations nav_order: 20 redirect_from: - /query-dsl/aggregations/metric/cardinality/ diff --git a/_aggregations/metric/extended-stats.md b/_aggregations/metric/extended-stats.md index 633407dab0..467fa348b7 100644 --- a/_aggregations/metric/extended-stats.md +++ b/_aggregations/metric/extended-stats.md @@ -2,7 +2,6 @@ layout: default title: Extended stats parent: Metric aggregations -grand_parent: Aggregations nav_order: 30 redirect_from: - /query-dsl/aggregations/metric/extended-stats/ diff --git a/_aggregations/metric/geobounds.md b/_aggregations/metric/geobounds.md index 27b7646ca5..9489c6b18e 100644 --- a/_aggregations/metric/geobounds.md +++ b/_aggregations/metric/geobounds.md @@ -2,7 +2,6 @@ layout: default title: Geobounds parent: Metric aggregations -grand_parent: Aggregations nav_order: 40 redirect_from: - /query-dsl/aggregations/metric/geobounds/ diff --git a/_aggregations/metric/geocentroid.md b/_aggregations/metric/geocentroid.md index 711f49862a..14a2d179bb 100644 --- a/_aggregations/metric/geocentroid.md +++ b/_aggregations/metric/geocentroid.md @@ -2,7 +2,6 @@ layout: default title: Geocentroid parent: Metric aggregations -grand_parent: Aggregations nav_order: 45 --- diff --git a/_aggregations/metric/matrix-stats.md b/_aggregations/metric/matrix-stats.md index 475e0caa24..188f8745fb 100644 --- a/_aggregations/metric/matrix-stats.md +++ b/_aggregations/metric/matrix-stats.md @@ -2,7 +2,6 @@ layout: default title: Matrix stats parent: Metric aggregations -grand_parent: Aggregations nav_order: 50 redirect_from: - /query-dsl/aggregations/metric/matrix-stats/ diff --git a/_aggregations/metric/maximum.md b/_aggregations/metric/maximum.md index 63b4d62a7b..1a1aaff607 100644 --- a/_aggregations/metric/maximum.md +++ b/_aggregations/metric/maximum.md @@ -2,7 +2,6 @@ layout: default title: Maximum parent: Metric aggregations -grand_parent: Aggregations nav_order: 60 redirect_from: - /query-dsl/aggregations/metric/maximum/ diff --git a/_aggregations/metric/median-absolute-deviation.md b/_aggregations/metric/median-absolute-deviation.md index 7332d7eb2f..a882475158 100644 --- a/_aggregations/metric/median-absolute-deviation.md +++ b/_aggregations/metric/median-absolute-deviation.md @@ -2,7 +2,6 @@ layout: default title: Median absolute deviation parent: Metric aggregations -grand_parent: Aggregations nav_order: 65 redirect_from: - /query-dsl/aggregations/metric/median-absolute-deviation/ diff --git a/_aggregations/metric/minimum.md b/_aggregations/metric/minimum.md index dd17c854a9..9455c71fea 100644 --- a/_aggregations/metric/minimum.md +++ b/_aggregations/metric/minimum.md @@ -2,7 +2,6 @@ layout: default title: Minimum parent: Metric aggregations -grand_parent: Aggregations nav_order: 70 redirect_from: - /query-dsl/aggregations/metric/minimum/ diff --git a/_aggregations/metric/percentile-ranks.md b/_aggregations/metric/percentile-ranks.md index 33ccb3d291..660cb01bd1 100644 --- a/_aggregations/metric/percentile-ranks.md +++ b/_aggregations/metric/percentile-ranks.md @@ -2,7 +2,6 @@ layout: default title: Percentile ranks parent: Metric aggregations -grand_parent: Aggregations nav_order: 80 redirect_from: - /query-dsl/aggregations/metric/percentile-ranks/ diff --git a/_aggregations/metric/percentile.md b/_aggregations/metric/percentile.md index c68b0e0ec7..0f241306d1 100644 --- a/_aggregations/metric/percentile.md +++ b/_aggregations/metric/percentile.md @@ -2,7 +2,6 @@ layout: default title: Percentile parent: Metric aggregations -grand_parent: Aggregations nav_order: 90 redirect_from: - /query-dsl/aggregations/metric/percentile/ diff --git a/_aggregations/metric/scripted-metric.md b/_aggregations/metric/scripted-metric.md index d1807efbc0..4247f9aa0e 100644 --- a/_aggregations/metric/scripted-metric.md +++ b/_aggregations/metric/scripted-metric.md @@ -2,7 +2,6 @@ layout: default title: Scripted metric parent: Metric aggregations -grand_parent: Aggregations nav_order: 100 redirect_from: - /query-dsl/aggregations/metric/scripted-metric/ diff --git a/_aggregations/metric/stats.md b/_aggregations/metric/stats.md index 0a54831522..d8ba5963e0 100644 --- a/_aggregations/metric/stats.md +++ b/_aggregations/metric/stats.md @@ -2,7 +2,6 @@ layout: default title: Stats parent: Metric aggregations -grand_parent: Aggregations nav_order: 110 redirect_from: - /query-dsl/aggregations/metric/stats/ diff --git a/_aggregations/metric/sum.md b/_aggregations/metric/sum.md index 0320de63fc..2e0b32cb3d 100644 --- a/_aggregations/metric/sum.md +++ b/_aggregations/metric/sum.md @@ -2,7 +2,6 @@ layout: default title: Sum parent: Metric aggregations -grand_parent: Aggregations nav_order: 120 redirect_from: - /query-dsl/aggregations/metric/sum/ diff --git a/_aggregations/metric/top-hits.md b/_aggregations/metric/top-hits.md index b6752300b2..cead3f77f2 100644 --- a/_aggregations/metric/top-hits.md +++ b/_aggregations/metric/top-hits.md @@ -2,7 +2,6 @@ layout: default title: Top hits parent: Metric aggregations -grand_parent: Aggregations nav_order: 130 redirect_from: - /query-dsl/aggregations/metric/top-hits/ diff --git a/_aggregations/metric/value-count.md b/_aggregations/metric/value-count.md index dfddaf9417..596fb5d806 100644 --- a/_aggregations/metric/value-count.md +++ b/_aggregations/metric/value-count.md @@ -2,7 +2,6 @@ layout: default title: Value count parent: Metric aggregations -grand_parent: Aggregations nav_order: 140 redirect_from: - /query-dsl/aggregations/metric/value-count/ diff --git a/_aggregations/metric/weighted-avg.md b/_aggregations/metric/weighted-avg.md index 268f78bfdc..6f67939d6e 100644 --- a/_aggregations/metric/weighted-avg.md +++ b/_aggregations/metric/weighted-avg.md @@ -2,7 +2,6 @@ layout: default title: Weighted average parent: Metric aggregations -grand_parent: Aggregations nav_order: 150 --- diff --git a/_query-dsl/compound/bool.md b/_query-dsl/compound/bool.md index 12caea2f8d..4479094214 100644 --- a/_query-dsl/compound/bool.md +++ b/_query-dsl/compound/bool.md @@ -2,7 +2,6 @@ layout: default title: Boolean parent: Compound queries -grand_parent: Query DSL nav_order: 10 redirect_from: - /opensearch/query-dsl/compound/bool/ diff --git a/_query-dsl/compound/boosting.md b/_query-dsl/compound/boosting.md index 7aa9c6c035..ede7eb3d51 100644 --- a/_query-dsl/compound/boosting.md +++ b/_query-dsl/compound/boosting.md @@ -2,7 +2,6 @@ layout: default title: Boosting parent: Compound queries -grand_parent: Query DSL nav_order: 30 redirect_from: - /query-dsl/query-dsl/compound/boosting/ diff --git a/_query-dsl/compound/constant-score.md b/_query-dsl/compound/constant-score.md index ed12e33ec0..bb5af3800a 100644 --- a/_query-dsl/compound/constant-score.md +++ b/_query-dsl/compound/constant-score.md @@ -2,7 +2,6 @@ layout: default title: Constant score parent: Compound queries -grand_parent: Query DSL nav_order: 40 redirect_from: - /query-dsl/query-dsl/compound/constant-score/ diff --git a/_query-dsl/compound/disjunction-max.md b/_query-dsl/compound/disjunction-max.md index 8dd9e41d2c..09a7e0e729 100644 --- a/_query-dsl/compound/disjunction-max.md +++ b/_query-dsl/compound/disjunction-max.md @@ -2,7 +2,6 @@ layout: default title: Disjunction max parent: Compound queries -grand_parent: Query DSL nav_order: 50 redirect_from: - /query-dsl/query-dsl/compound/disjunction-max/ diff --git a/_query-dsl/compound/function-score.md b/_query-dsl/compound/function-score.md index 98568e0965..b28a6abed6 100644 --- a/_query-dsl/compound/function-score.md +++ b/_query-dsl/compound/function-score.md @@ -2,7 +2,6 @@ layout: default title: Function score parent: Compound queries -grand_parent: Query DSL nav_order: 60 has_math: true redirect_from: diff --git a/_query-dsl/compound/hybrid.md b/_query-dsl/compound/hybrid.md index 22b3a17fc1..69ce89ce17 100644 --- a/_query-dsl/compound/hybrid.md +++ b/_query-dsl/compound/hybrid.md @@ -2,7 +2,6 @@ layout: default title: Hybrid parent: Compound queries -grand_parent: Query DSL nav_order: 70 --- diff --git a/_query-dsl/full-text/intervals.md b/_query-dsl/full-text/intervals.md index 082f8fbe46..a31f7434b3 100644 --- a/_query-dsl/full-text/intervals.md +++ b/_query-dsl/full-text/intervals.md @@ -3,7 +3,6 @@ layout: default title: Intervals nav_order: 80 parent: Full-text queries -grand_parent: Query DSL --- # Intervals query diff --git a/_query-dsl/full-text/match-bool-prefix.md b/_query-dsl/full-text/match-bool-prefix.md index 3a0d304ce4..3964dc5ee8 100644 --- a/_query-dsl/full-text/match-bool-prefix.md +++ b/_query-dsl/full-text/match-bool-prefix.md @@ -2,7 +2,6 @@ layout: default title: Match Boolean prefix parent: Full-text queries -grand_parent: Query DSL nav_order: 20 --- diff --git a/_query-dsl/full-text/match-phrase-prefix.md b/_query-dsl/full-text/match-phrase-prefix.md index 354dd35c61..9e05f034d7 100644 --- a/_query-dsl/full-text/match-phrase-prefix.md +++ b/_query-dsl/full-text/match-phrase-prefix.md @@ -2,7 +2,6 @@ layout: default title: Match phrase prefix parent: Full-text queries -grand_parent: Query DSL nav_order: 40 --- diff --git a/_query-dsl/full-text/match-phrase.md b/_query-dsl/full-text/match-phrase.md index 18dd6a858c..747c4814d9 100644 --- a/_query-dsl/full-text/match-phrase.md +++ b/_query-dsl/full-text/match-phrase.md @@ -2,7 +2,6 @@ layout: default title: Match phrase parent: Full-text queries -grand_parent: Query DSL nav_order: 30 --- diff --git a/_query-dsl/full-text/match.md b/_query-dsl/full-text/match.md index 746a4cf5b6..b4db30ec1f 100644 --- a/_query-dsl/full-text/match.md +++ b/_query-dsl/full-text/match.md @@ -2,7 +2,6 @@ layout: default title: Match parent: Full-text queries -grand_parent: Query DSL nav_order: 10 --- diff --git a/_query-dsl/full-text/multi-match.md b/_query-dsl/full-text/multi-match.md index 7450b74721..ab1496fdd3 100644 --- a/_query-dsl/full-text/multi-match.md +++ b/_query-dsl/full-text/multi-match.md @@ -2,7 +2,6 @@ layout: default title: Multi-match parent: Full-text queries -grand_parent: Query DSL nav_order: 50 --- diff --git a/_query-dsl/full-text/query-string.md b/_query-dsl/full-text/query-string.md index 12609e29c0..47180e3f6d 100644 --- a/_query-dsl/full-text/query-string.md +++ b/_query-dsl/full-text/query-string.md @@ -2,9 +2,7 @@ layout: default title: Query string parent: Full-text queries -grand_parent: Query DSL nav_order: 60 - redirect_from: - /opensearch/query-dsl/full-text/query-string/ - /query-dsl/query-dsl/full-text/query-string/ diff --git a/_query-dsl/full-text/simple-query-string.md b/_query-dsl/full-text/simple-query-string.md index fbf37f588d..58780cfdb4 100644 --- a/_query-dsl/full-text/simple-query-string.md +++ b/_query-dsl/full-text/simple-query-string.md @@ -2,7 +2,6 @@ layout: default title: Simple query string parent: Full-text queries -grand_parent: Query DSL nav_order: 70 --- diff --git a/_query-dsl/geo-and-xy/geo-bounding-box.md b/_query-dsl/geo-and-xy/geo-bounding-box.md index df697e2ce5..1112a4278e 100644 --- a/_query-dsl/geo-and-xy/geo-bounding-box.md +++ b/_query-dsl/geo-and-xy/geo-bounding-box.md @@ -2,7 +2,6 @@ layout: default title: Geo-bounding box parent: Geographic and xy queries -grand_parent: Query DSL nav_order: 10 redirect_from: - /opensearch/query-dsl/geo-and-xy/geo-bounding-box/ diff --git a/_query-dsl/geo-and-xy/geodistance.md b/_query-dsl/geo-and-xy/geodistance.md index 7a36b0c933..b272cad81e 100644 --- a/_query-dsl/geo-and-xy/geodistance.md +++ b/_query-dsl/geo-and-xy/geodistance.md @@ -2,7 +2,6 @@ layout: default title: Geodistance parent: Geographic and xy queries -grand_parent: Query DSL nav_order: 20 --- diff --git a/_query-dsl/geo-and-xy/geopolygon.md b/_query-dsl/geo-and-xy/geopolygon.md index c53b1379cf..980a0c5a63 100644 --- a/_query-dsl/geo-and-xy/geopolygon.md +++ b/_query-dsl/geo-and-xy/geopolygon.md @@ -2,7 +2,6 @@ layout: default title: Geopolygon parent: Geographic and xy queries -grand_parent: Query DSL nav_order: 30 --- diff --git a/_query-dsl/geo-and-xy/xy.md b/_query-dsl/geo-and-xy/xy.md index 88a22448c3..d0ed61c050 100644 --- a/_query-dsl/geo-and-xy/xy.md +++ b/_query-dsl/geo-and-xy/xy.md @@ -2,9 +2,7 @@ layout: default title: xy parent: Geographic and xy queries -grand_parent: Query DSL nav_order: 50 - redirect_from: - /opensearch/query-dsl/geo-and-xy/xy/ - /query-dsl/query-dsl/geo-and-xy/xy/ diff --git a/_query-dsl/specialized/neural-sparse.md b/_query-dsl/specialized/neural-sparse.md index 47f77fa95d..8de3eaf693 100644 --- a/_query-dsl/specialized/neural-sparse.md +++ b/_query-dsl/specialized/neural-sparse.md @@ -2,7 +2,6 @@ layout: default title: Neural sparse parent: Specialized queries -grand_parent: Query DSL nav_order: 55 --- diff --git a/_query-dsl/specialized/neural.md b/_query-dsl/specialized/neural.md index fea949cf52..14b930cdb6 100644 --- a/_query-dsl/specialized/neural.md +++ b/_query-dsl/specialized/neural.md @@ -2,7 +2,6 @@ layout: default title: Neural parent: Specialized queries -grand_parent: Query DSL nav_order: 50 --- diff --git a/_query-dsl/specialized/script-score.md b/_query-dsl/specialized/script-score.md index d09158f20a..f65c266cc5 100644 --- a/_query-dsl/specialized/script-score.md +++ b/_query-dsl/specialized/script-score.md @@ -2,7 +2,6 @@ layout: default title: Script score parent: Specialized queries -grand_parent: Query DSL nav_order: 60 --- diff --git a/_query-dsl/term/exists.md b/_query-dsl/term/exists.md index 1d52744c91..95573a36f8 100644 --- a/_query-dsl/term/exists.md +++ b/_query-dsl/term/exists.md @@ -2,7 +2,6 @@ layout: default title: Exists parent: Term-level queries -grand_parent: Query DSL nav_order: 10 --- diff --git a/_query-dsl/term/fuzzy.md b/_query-dsl/term/fuzzy.md index 9afa85ea93..bf2bd43bba 100644 --- a/_query-dsl/term/fuzzy.md +++ b/_query-dsl/term/fuzzy.md @@ -2,7 +2,6 @@ layout: default title: Fuzzy parent: Term-level queries -grand_parent: Query DSL nav_order: 20 --- diff --git a/_query-dsl/term/ids.md b/_query-dsl/term/ids.md index 0c3b5393fb..f895745c97 100644 --- a/_query-dsl/term/ids.md +++ b/_query-dsl/term/ids.md @@ -2,7 +2,6 @@ layout: default title: IDs parent: Term-level queries -grand_parent: Query DSL nav_order: 30 --- diff --git a/_query-dsl/term/prefix.md b/_query-dsl/term/prefix.md index eda5307d14..2a429c9f0e 100644 --- a/_query-dsl/term/prefix.md +++ b/_query-dsl/term/prefix.md @@ -2,7 +2,6 @@ layout: default title: Prefix parent: Term-level queries -grand_parent: Query DSL nav_order: 40 --- diff --git a/_query-dsl/term/range.md b/_query-dsl/term/range.md index 8a8f53c480..ceb264db76 100644 --- a/_query-dsl/term/range.md +++ b/_query-dsl/term/range.md @@ -2,7 +2,6 @@ layout: default title: Range parent: Term-level queries -grand_parent: Query DSL nav_order: 50 --- diff --git a/_query-dsl/term/regexp.md b/_query-dsl/term/regexp.md index 65d6953516..4a038729c0 100644 --- a/_query-dsl/term/regexp.md +++ b/_query-dsl/term/regexp.md @@ -2,7 +2,6 @@ layout: default title: Regexp parent: Term-level queries -grand_parent: Query DSL nav_order: 60 --- diff --git a/_query-dsl/term/term.md b/_query-dsl/term/term.md index c1c296b9a0..a33146f6aa 100644 --- a/_query-dsl/term/term.md +++ b/_query-dsl/term/term.md @@ -2,7 +2,6 @@ layout: default title: Term parent: Term-level queries -grand_parent: Query DSL nav_order: 70 --- diff --git a/_query-dsl/term/terms-set.md b/_query-dsl/term/terms-set.md index ea0251ddff..6652d15979 100644 --- a/_query-dsl/term/terms-set.md +++ b/_query-dsl/term/terms-set.md @@ -2,7 +2,6 @@ layout: default title: Terms set parent: Term-level queries -grand_parent: Query DSL nav_order: 90 --- diff --git a/_query-dsl/term/terms.md b/_query-dsl/term/terms.md index fd15126255..42c74c0436 100644 --- a/_query-dsl/term/terms.md +++ b/_query-dsl/term/terms.md @@ -2,7 +2,6 @@ layout: default title: Terms parent: Term-level queries -grand_parent: Query DSL nav_order: 80 --- diff --git a/_query-dsl/term/wildcard.md b/_query-dsl/term/wildcard.md index 0652581941..b2d7238758 100644 --- a/_query-dsl/term/wildcard.md +++ b/_query-dsl/term/wildcard.md @@ -2,7 +2,6 @@ layout: default title: Wildcard parent: Term-level queries -grand_parent: Query DSL nav_order: 100 --- diff --git a/_search-plugins/knn/api.md b/_search-plugins/knn/api.md index 23678063a8..c7314f7ae2 100644 --- a/_search-plugins/knn/api.md +++ b/_search-plugins/knn/api.md @@ -3,7 +3,6 @@ layout: default title: k-NN plugin API nav_order: 30 parent: k-NN search -grand_parent: Search methods has_children: false --- diff --git a/_search-plugins/knn/approximate-knn.md b/_search-plugins/knn/approximate-knn.md index c0a9557728..144365166f 100644 --- a/_search-plugins/knn/approximate-knn.md +++ b/_search-plugins/knn/approximate-knn.md @@ -3,7 +3,6 @@ layout: default title: Approximate k-NN search nav_order: 15 parent: k-NN search -grand_parent: Search methods has_children: false has_math: true --- diff --git a/_search-plugins/knn/filter-search-knn.md b/_search-plugins/knn/filter-search-knn.md index 309cf6850e..2f0c4aa072 100644 --- a/_search-plugins/knn/filter-search-knn.md +++ b/_search-plugins/knn/filter-search-knn.md @@ -3,7 +3,6 @@ layout: default title: k-NN search with filters nav_order: 20 parent: k-NN search -grand_parent: Search methods has_children: false has_math: true --- diff --git a/_search-plugins/knn/jni-libraries.md b/_search-plugins/knn/jni-libraries.md index 59a5b7a1e2..4dbdb2da56 100644 --- a/_search-plugins/knn/jni-libraries.md +++ b/_search-plugins/knn/jni-libraries.md @@ -3,7 +3,6 @@ layout: default title: JNI libraries nav_order: 35 parent: k-NN search -grand_parent: Search methods has_children: false redirect_from: - /search-plugins/knn/jni-library/ diff --git a/_search-plugins/knn/knn-index.md b/_search-plugins/knn/knn-index.md index ab24a0c097..ed8b9217f5 100644 --- a/_search-plugins/knn/knn-index.md +++ b/_search-plugins/knn/knn-index.md @@ -3,7 +3,6 @@ layout: default title: k-NN index nav_order: 5 parent: k-NN search -grand_parent: Search methods has_children: false --- diff --git a/_search-plugins/knn/knn-score-script.md b/_search-plugins/knn/knn-score-script.md index cc79e90850..1696bd4cad 100644 --- a/_search-plugins/knn/knn-score-script.md +++ b/_search-plugins/knn/knn-score-script.md @@ -3,7 +3,6 @@ layout: default title: Exact k-NN with scoring script nav_order: 10 parent: k-NN search -grand_parent: Search methods has_children: false has_math: true --- diff --git a/_search-plugins/knn/knn-vector-quantization.md b/_search-plugins/knn/knn-vector-quantization.md index 549437f346..fe4833ee47 100644 --- a/_search-plugins/knn/knn-vector-quantization.md +++ b/_search-plugins/knn/knn-vector-quantization.md @@ -3,7 +3,6 @@ layout: default title: k-NN vector quantization nav_order: 27 parent: k-NN search -grand_parent: Search methods has_children: false has_math: true --- diff --git a/_search-plugins/knn/nested-search-knn.md b/_search-plugins/knn/nested-search-knn.md index bdc1045387..d947ebc6e6 100644 --- a/_search-plugins/knn/nested-search-knn.md +++ b/_search-plugins/knn/nested-search-knn.md @@ -3,7 +3,6 @@ layout: default title: k-NN search with nested fields nav_order: 21 parent: k-NN search -grand_parent: Search methods has_children: false has_math: true --- diff --git a/_search-plugins/knn/painless-functions.md b/_search-plugins/knn/painless-functions.md index 1f27cc29a6..09eb989702 100644 --- a/_search-plugins/knn/painless-functions.md +++ b/_search-plugins/knn/painless-functions.md @@ -3,7 +3,6 @@ layout: default title: k-NN Painless extensions nav_order: 25 parent: k-NN search -grand_parent: Search methods has_children: false has_math: true --- diff --git a/_search-plugins/knn/performance-tuning.md b/_search-plugins/knn/performance-tuning.md index 24d92bd67d..123b1daef1 100644 --- a/_search-plugins/knn/performance-tuning.md +++ b/_search-plugins/knn/performance-tuning.md @@ -2,7 +2,6 @@ layout: default title: Performance tuning parent: k-NN search -grand_parent: Search methods nav_order: 45 --- diff --git a/_search-plugins/knn/radial-search-knn.md b/_search-plugins/knn/radial-search-knn.md index 48aaac034d..1a4a223294 100644 --- a/_search-plugins/knn/radial-search-knn.md +++ b/_search-plugins/knn/radial-search-knn.md @@ -3,7 +3,6 @@ layout: default title: Radial search nav_order: 28 parent: k-NN search -grand_parent: Search methods has_children: false has_math: true --- diff --git a/_search-plugins/knn/settings.md b/_search-plugins/knn/settings.md index 4d84cc80bb..1b9aa3608c 100644 --- a/_search-plugins/knn/settings.md +++ b/_search-plugins/knn/settings.md @@ -2,7 +2,6 @@ layout: default title: Settings parent: k-NN search -grand_parent: Search methods nav_order: 40 --- From f1cb58fd0c5ad8565e929e162157e857b5d42c06 Mon Sep 17 00:00:00 2001 From: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Date: Wed, 17 Jul 2024 16:01:56 -0500 Subject: [PATCH 08/36] Remove redundant source (#7755) Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --- _api-reference/document-apis/reindex.md | 1 - 1 file changed, 1 deletion(-) diff --git a/_api-reference/document-apis/reindex.md b/_api-reference/document-apis/reindex.md index 48f14923f5..c2afa347e1 100644 --- a/_api-reference/document-apis/reindex.md +++ b/_api-reference/document-apis/reindex.md @@ -79,7 +79,6 @@ version_type | The indexing operation's version type. Valid values are `internal op_type | Whether to copy over documents that are missing in the destination index. Valid values are `create` (ignore documents with the same ID from the source index) and `index` (copy everything from the source index). pipeline | Which ingest pipeline to utilize during the reindex. script | A script that OpenSearch uses to apply transformations to the data during the reindex operation. -source | The actual script that OpenSearch runs. lang | The scripting language. Valid options are `painless`, `expression`, `mustache`, and `java`. ## Response From 7bfee4011d0a82644e3e5868717f44eac82ea6b4 Mon Sep 17 00:00:00 2001 From: Michael Oviedo Date: Wed, 17 Jul 2024 15:15:25 -0700 Subject: [PATCH 09/36] Update compare.md (#7765) The `--results-number-align` option for the compare API is actually spelled `--results-numbers-align`. This change updates the spelling for that option. Signed-off-by: Michael Oviedo --- _benchmark/reference/commands/compare.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_benchmark/reference/commands/compare.md b/_benchmark/reference/commands/compare.md index a9eb8bcc10..35bafe0704 100644 --- a/_benchmark/reference/commands/compare.md +++ b/_benchmark/reference/commands/compare.md @@ -130,7 +130,7 @@ You can use the following options to customize the results of your test comparis - `--baseline`: The baseline TestExecution ID used to compare the contender TestExecution. - `--contender`: The TestExecution ID for the contender being compared to the baseline. - `--results-format`: Defines the output format for the command line results, either `markdown` or `csv`. Default is `markdown`. -- `--results-number-align`: Defines the column number alignment for when the `compare` command outputs results. Default is `right`. +- `--results-numbers-align`: Defines the column number alignment for when the `compare` command outputs results. Default is `right`. - `--results-file`: When provided a file path, writes the compare results to the file indicated in the path. - `--show-in-results`: Determines whether or not to include the comparison in the results file. From 3f37837c631d6f99c14f823965e3fdcedfa67dc8 Mon Sep 17 00:00:00 2001 From: gaobinlong Date: Thu, 18 Jul 2024 23:37:15 +0800 Subject: [PATCH 10/36] Document CreateAnomalyDetectorTool (#7742) * Document CreateAnomalyDetectorTool Signed-off-by: gaobinlong * Fix format issue Signed-off-by: gaobinlong * Fix link Signed-off-by: gaobinlong * Update create-anomaly-detector.md Signed-off-by: Melissa Vagi * Update _ml-commons-plugin/agents-tools/tools/index.md Signed-off-by: Melissa Vagi * Update _ml-commons-plugin/agents-tools/tools/index.md Signed-off-by: Melissa Vagi * Update _ml-commons-plugin/agents-tools/tools/create-anomaly-detector.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _ml-commons-plugin/agents-tools/tools/create-anomaly-detector.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _ml-commons-plugin/agents-tools/tools/create-anomaly-detector.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _ml-commons-plugin/agents-tools/tools/create-anomaly-detector.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _ml-commons-plugin/agents-tools/tools/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi --------- Signed-off-by: gaobinlong Signed-off-by: Melissa Vagi Co-authored-by: Melissa Vagi Co-authored-by: Nathan Bower --- .../tools/create-anomaly-detector.md | 169 ++++++++++++++++++ .../agents-tools/tools/index.md | 3 +- 2 files changed, 171 insertions(+), 1 deletion(-) create mode 100644 _ml-commons-plugin/agents-tools/tools/create-anomaly-detector.md diff --git a/_ml-commons-plugin/agents-tools/tools/create-anomaly-detector.md b/_ml-commons-plugin/agents-tools/tools/create-anomaly-detector.md new file mode 100644 index 0000000000..b6fae8131c --- /dev/null +++ b/_ml-commons-plugin/agents-tools/tools/create-anomaly-detector.md @@ -0,0 +1,169 @@ +--- +layout: default +title: CreateAnomalyDetectorTool +has_children: false +has_toc: false +nav_order: 70 +parent: Tools +grand_parent: Agents and tools +--- + + +# CreateAnomalyDetectorTool +**Introduced 2.16** +{: .label .label-purple } + + +This is an experimental feature and is not recommended for use in a production environment. For updates on the progress of the feature or if you want to leave feedback, see the associated [GitHub issue](https://github.com/opensearch-project/skills/issues/337). +{: .warning} + +The `CreateAnomalyDetectorTool` helps create anomaly detectors based on your provided index. This tool retrieves index mappings and enables a large language model (LLM) to recommend category fields, aggregation fields, and their corresponding aggregation methods, which are required by the Create Anomaly Detector API. + +For comprehensive information about anomaly detectors, see [Anomaly detection]({{site.url}}{{site.baseurl}}/observing-your-data/ad/index/). +{: .tip} + +## Step 1: Register a flow agent that runs the CreateAnomalyDetectorTool + +A flow agent runs a sequence of tools in order, returning the output of the last tool. To create a flow agent, send the following register agent request: + +```json +POST /_plugins/_ml/agents/_register +{ + "name": "Test_Agent_For_Create_Anomaly_Detector_Tool", + "type": "flow", + "description": "this is a test agent for the CreateAnomalyDetectorTool", + "memory": { + "type": "demo" + }, + "tools": [ + { + "type": "CreateAnomalyDetectorTool", + "name": "DemoCreateAnomalyDetectorTool", + "parameters": { + "model_id": "" + } + } + ] +} +``` +{% include copy-curl.html %} + +OpenSearch responds with an agent ID, for example, as follows: + +```json +{ + "agent_id": "EuJYYo0B9RaBCvhuy1q8" +} +``` +{% include copy-curl.html %} + +## Step 2: Run the agent + +Run the agent by sending the following request: + +```json +POST /_plugins/_ml/agents/EuJYYo0B9RaBCvhuy1q8/_execute +{ + "parameters": { + "index": "sample_weblogs_test" + } +} +``` +{% include copy-curl.html %} + +OpenSearch responds with a JSON string containing all of the recommended parameters for creating an anomaly detector, such as the string shown in the following example repsonse: + +```json +{ + "inference_results": [ + { + "output": [ + { + "name": "response", + "result":"""{"index":"sample_weblogs_test","categoryField":"ip.keyword","aggregationField":"bytes,response,responseLatency","aggregationMethod":"sum,avg,avg","dateFields":"utc_time,timestamp"}""" + } + ] + } + ] +} +``` +{% include copy-curl.html %} + +You can then create an anomaly detector containing the recommended parameters by sending a request similar to the following: + +```json +POST _plugins/_anomaly_detection/detectors +{ + "name": "test-detector", + "description": "Test detector", + "time_field": "timestamp", + "indices": [ + "sample_weblogs_test" + ], + "feature_attributes": [ + { + "feature_name": "feature_bytes", + "feature_enabled": true, + "aggregation_query": { + "agg1": { + "sum": { + "field": "bytes" + } + } + } + }, + { + "feature_name": "feature_response", + "feature_enabled": true, + "aggregation_query": { + "agg2": { + "avg": { + "field": "response" + } + } + } + }, + { + "feature_name": "feature_responseLatency", + "feature_enabled": true, + "aggregation_query": { + "agg3": { + "avg": { + "field": "responseLatency" + } + } + } + } + ], + "detection_interval": { + "period": { + "interval": 1, + "unit": "Minutes" + } + }, + "window_delay": { + "period": { + "interval": 1, + "unit": "Minutes" + } + } +} +``` +{% include copy-curl.html %} + +## Register parameters + +The following table lists the available tool parameters for agent registration. + +Parameter | Type | Required/Optional | Description +:--- | :--- | :--- | :--- +`model_id` | String | Required | The LLM model ID used for suggesting required Create Anomaly Detector API parameters. +`model_type` | String | Optional | The model type. Valid values are `CLAUDE` (Anthropic Claude models) and `OPENAI` (OpenAI models). + +## Execute parameters + +The following table lists the available tool parameters for running the agent. + +Parameter | Type | Required/Optional | Description +:--- | :--- | :--- | :--- +`index` | String | Required | The index name. Supports wildcards (for example, `weblogs-*`). If wildcards are used, then the tool fetches mappings from the first resolved index and sends them to the LLM. diff --git a/_ml-commons-plugin/agents-tools/tools/index.md b/_ml-commons-plugin/agents-tools/tools/index.md index fba47b63da..bc71122949 100644 --- a/_ml-commons-plugin/agents-tools/tools/index.md +++ b/_ml-commons-plugin/agents-tools/tools/index.md @@ -35,6 +35,7 @@ Each tool takes a list of parameters specific to that tool. In the preceding exa |[`CatIndexTool`]({{site.url}}{{site.baseurl}}/ml-commons-plugin/agents-tools/tools/cat-index-tool/) |Retrieves index information for the OpenSearch cluster. | |[`ConnectorTool`]({{site.url}}{{site.baseurl}}/ml-commons-plugin/agents-tools/tools/connector-tool/) | Uses a [connector]({{site.url}}{{site.baseurl}}/ml-commons-plugin/remote-models/connectors/) to call any REST API function. | |[`IndexMappingTool`]({{site.url}}{{site.baseurl}}/ml-commons-plugin/agents-tools/tools/index-mapping-tool/) |Retrieves index mapping and setting information for an index. | +|[`CreateAnomalyDetectorTool`]({{site.url}}{{site.baseurl}}/ml-commons-plugin/agents-tools/tools/create-anomaly-detector/) | Enables an LLM to suggest required parameters for creating an anomaly detector. | |[`MLModelTool`]({{site.url}}{{site.baseurl}}/ml-commons-plugin/agents-tools/tools/ml-model-tool/) |Runs machine learning models. | |[`NeuralSparseSearchTool`]({{site.url}}{{site.baseurl}}/ml-commons-plugin/agents-tools/tools/neural-sparse-tool/) | Performs sparse vector retrieval. | |[`PPLTool`]({{site.url}}{{site.baseurl}}/ml-commons-plugin/agents-tools/tools/ppl-tool/) |Translates natural language into a Piped Processing Language (PPL) query. | @@ -49,4 +50,4 @@ Each tool takes a list of parameters specific to that tool. In the preceding exa ## Developer information -The agents and tools framework is flexible and extensible. You can find the list of tools provided by OpenSearch in the [Tools library](https://github.com/opensearch-project/skills/tree/main/src/main/java/org/opensearch/agent/tools). For a different use case, you can build your own tool by implementing the [_Tool_ interface](https://github.com/opensearch-project/ml-commons/blob/2.x/spi/src/main/java/org/opensearch/ml/common/spi/tools/Tool.java). \ No newline at end of file +The agents and tools framework offers flexibility and extensibility. See the [tools library](https://github.com/opensearch-project/skills/tree/main/src/main/java/org/opensearch/agent/tools) for OpenSearch-provided tools. Implement the [**Tool** interface](https://github.com/opensearch-project/ml-commons/blob/2.x/spi/src/main/java/org/opensearch/ml/common/spi/tools/Tool.java) to build custom tools for different use cases. From 0c3dacee582816b9ab0c7c98f927597b28677a79 Mon Sep 17 00:00:00 2001 From: gaobinlong Date: Thu, 18 Jul 2024 23:55:36 +0800 Subject: [PATCH 11/36] Add strict_allow_templates option for the dynamic mapping parameter (#7745) * Add strict_allow_templates option for the dynamic mapping parameter Signed-off-by: gaobinlong * Fix typo Signed-off-by: gaobinlong * Fix header Signed-off-by: gaobinlong * Update dynamic.md Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/supported-field-types/object.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/supported-field-types/object.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/supported-field-types/nested.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update dynamic.md Make changes to address editorial review comments Signed-off-by: Melissa Vagi --------- Signed-off-by: gaobinlong Signed-off-by: Melissa Vagi Co-authored-by: Melissa Vagi Co-authored-by: Nathan Bower --- _field-types/dynamic.md | 342 +++++++++++++++++++ _field-types/supported-field-types/nested.md | 2 +- _field-types/supported-field-types/object.md | 3 +- 3 files changed, 345 insertions(+), 2 deletions(-) create mode 100644 _field-types/dynamic.md diff --git a/_field-types/dynamic.md b/_field-types/dynamic.md new file mode 100644 index 0000000000..59f59bfe3d --- /dev/null +++ b/_field-types/dynamic.md @@ -0,0 +1,342 @@ +--- +layout: default +title: Dynamic parameter +nav_order: 10 +redirect_from: + - /opensearch/dynamic/ +--- + +# Dynamic parameter + +The `dynamic` parameter specifies whether newly detected fields can be added dynamically to a mapping. It accepts the parameters listed in the following table. + +Parameter | Description +:--- | :--- +`true` | Specfies that new fields can be added dynamically to the mapping. Default is `true`. +`false` | Specifies that new fields cannot be added dynamically to the mapping. If a new field is detected, then it is not indexed or searchable but can be retrieved from the `_source` field. +`strict` | Throws an exception. The indexing operation fails when new fields are detected. +`strict_allow_templates` | Adds new fields if they match predefined dynamic templates in the mapping. + +--- + +## Example: Create an index with `dynamic` set to `true` + +1. Create an index with `dynamic` set to `true` by sending the following request: + +```json +PUT testindex1 +{ + "mappings": { + "dynamic": true + } +} +``` +{% include copy-curl.html %} + +2. Index a document with an object field `patient` containing two string fields by sending the following request: + +```json +PUT testindex1/_doc/1 +{ + "patient": { + "name" : "John Doe", + "id" : "123456" + } +} +``` +{% include copy-curl.html %} + +3. Confirm the mapping works as expected by sending the following request: + +```json +GET testindex1/_mapping +``` +{% include copy-curl.html %} + +The object field `patient` and two subfields `name` and `id` are added to the mapping, as shown in the following response: + +```json +{ + "testindex1": { + "mappings": { + "dynamic": "true", + "properties": { + "patient": { + "properties": { + "id": { + "type": "text", + "fields": { + "keyword": { + "type": "keyword", + "ignore_above": 256 + } + } + }, + "name": { + "type": "text", + "fields": { + "keyword": { + "type": "keyword", + "ignore_above": 256 + } + } + } + } + } + } + } + } +} +``` + +--- + +## Example: Create an index with `dynamic` set to `false` + +1. Create an index with explicit mappings and `dynamic` set to `false` by sending the following request: + +```json +PUT testindex1 +{ + "mappings": { + "dynamic": false, + "properties": { + "patient": { + "properties": { + "id": { + "type": "keyword" + }, + "name": { + "type": "keyword" + } + } + } + } + } +} +``` +{% include copy-curl.html %} + +2. Index a document with an object field `patient` containing two string fields and additional unmapped fields by sending the following request: + +```json +PUT testindex1/_doc/1 +{ + "patient": { + "name" : "John Doe", + "id" : "123456" + }, + "room": "room1", + "floor": "1" +} +``` +{% include copy-curl.html %} + +3. Confirm the mapping works as expected by sending the following request: + +```json +GET testindex1/_mapping +``` +{% include copy-curl.html %} + +The following response shows that the new fields `room` and `floor` were not added to the mapping, which remained unchanged: + +```json +{ + "testindex1": { + "mappings": { + "dynamic": "false", + "properties": { + "patient": { + "properties": { + "id": { + "type": "keyword" + }, + "name": { + "type": "keyword" + } + } + } + } + } + } +} +``` + +4. Get the unmapped fields `room` and `floor` from the document by sending the following request: + +```json +PUT testindex1/_doc/1 +{ + "patient": { + "name" : "John Doe", + "id" : "123456" + }, + "room": "room1", + "floor": "1" +} +``` + +The following request searches for the fields `room` and `floor`: + +```json +POST testindex1/_search +{ + "query": { + "term": { + "room": "room1" + } + } +} +``` + +The response returns no results: + +```json +{ + "took": 3, + "timed_out": false, + "_shards": { + "total": 1, + "successful": 1, + "skipped": 0, + "failed": 0 + }, + "hits": { + "total": { + "value": 0, + "relation": "eq" + }, + "max_score": null, + "hits": [] + } +} +``` + +--- + +## Example: Create an index with `dynamic` set to `strict` + +1. Create an index with explicit mappings and `dynamic` set to `strict` by sending the following request: + +```json +PUT testindex1 +{ + "mappings": { + "dynamic": strict, + "properties": { + "patient": { + "properties": { + "id": { + "type": "keyword" + }, + "name": { + "type": "keyword" + } + } + } + } + } +} +``` +{% include copy-curl.html %} + +2. Index a document with an object field `patient` containing two string fields and additional unmapped fields by sending the following request: + +```json +PUT testindex1/_doc/1 +{ + "patient": { + "name" : "John Doe", + "id" : "123456" + }, + "room": "room1", + "floor": "1" +} +``` +{% include copy-curl.html %} + +Note that an exception is thrown, as shown in the following response: + +```json +{ + "error": { + "root_cause": [ + { + "type": "strict_dynamic_mapping_exception", + "reason": "mapping set to strict, dynamic introduction of [room] within [_doc] is not allowed" + } + ], + "type": "strict_dynamic_mapping_exception", + "reason": "mapping set to strict, dynamic introduction of [room] within [_doc] is not allowed" + }, + "status": 400 +} +``` + +--- + +## Example: Create an index with `dynamic` set to `strict_allow_templates` + +1. Create an index with predefined dynamic templates and `dynamic` set to `strict_allow_templates` by sending the following request: + +```json +PUT testindex1 +{ + "mappings": { + "dynamic": "strict_allow_templates", + "dynamic_templates": [ + { + "strings": { + "match": "room*", + "match_mapping_type": "string", + "mapping": { + "type": "keyword" + } + } + } + ], + "properties": { + "patient": { + "properties": { + "id": { + "type": "keyword" + }, + "name": { + "type": "keyword" + } + } + } + } + } +} +``` +{% include copy-curl.html %} + +2. Index a document with an object field `patient` containing two string fields and a new field `room` that matches one of the dynamic templates by sending the following request: + +```json +PUT testindex1/_doc/1 +{ + "patient": { + "name" : "John Doe", + "id" : "123456" + }, + "room": "room1" +} +``` +{% include copy-curl.html %} + +Indexing succeeds because the new field `room` matches the dynamic templates. However, indexing fails for the new field `floor` because it does not match one of the dynamic templates and is not explicitly mapped, as shown in the following response: + +```json +PUT testindex1/_doc/1 +{ + "patient": { + "name" : "John Doe", + "id" : "123456" + }, + "room": "room1", + "floor": "1" +} +``` diff --git a/_field-types/supported-field-types/nested.md b/_field-types/supported-field-types/nested.md index d61ccd53df..90d09177d1 100644 --- a/_field-types/supported-field-types/nested.md +++ b/_field-types/supported-field-types/nested.md @@ -308,7 +308,7 @@ The following table lists the parameters accepted by object field types. All par Parameter | Description :--- | :--- -[`dynamic`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/object#the-dynamic-parameter) | Specifies whether new fields can be dynamically added to this object. Valid values are `true`, `false`, and `strict`. Default is `true`. +[`dynamic`]({{site.url}}{{site.baseurl}}/opensearch/supported-field-types/object#the-dynamic-parameter) | Specifies whether new fields can be dynamically added to the object. Valid values are `true`, `false`, `strict`, and `strict_allow_templates`. Default is `true`. `include_in_parent` | A Boolean value that specifies whether all fields in the child nested object should also be added to the parent document in flattened form. Default is `false`. `include_in_root` | A Boolean value that specifies whether all fields in the child nested object should also be added to the root document in flattened form. Default is `false`. `properties` | Fields of this object, which can be of any supported type. New properties can be dynamically added to this object if `dynamic` is set to `true`. diff --git a/_field-types/supported-field-types/object.md b/_field-types/supported-field-types/object.md index 372a5c46d9..db539a9608 100644 --- a/_field-types/supported-field-types/object.md +++ b/_field-types/supported-field-types/object.md @@ -73,7 +73,7 @@ The following table lists the parameters accepted by object field types. All par Parameter | Description :--- | :--- -[`dynamic`](#the-dynamic-parameter) | Specifies whether new fields can be dynamically added to this object. Valid values are `true`, `false`, and `strict`. Default is `true`. +[`dynamic`](#the-dynamic-parameter) | Specifies whether new fields can be dynamically added to the object. Valid values are `true`, `false`, `strict`, and `strict_allow_templates`. Default is `true`. `enabled` | A Boolean value that specifies whether the JSON contents of the object should be parsed. If `enabled` is set to `false`, the object's contents are not indexed or searchable, but they are still retrievable from the _source field. Default is `true`. `properties` | Fields of this object, which can be of any supported type. New properties can be dynamically added to this object if `dynamic` is set to `true`. @@ -149,6 +149,7 @@ Value | Description `true` | New fields can be added to the mapping dynamically. This is the default. `false` | New fields cannot be added to the mapping dynamically. If a new field is detected, it is not indexed or searchable. However, it is still retrievable from the _source field. `strict` | When new fields are added to the mapping dynamically, an exception is thrown. To add a new field to an object, you have to add it to the mapping first. +`strict_allow_templates` | If the newly detected fields match any of the predefined dynamic templates in the mapping, then they are added to the mapping; if they do not match any of them, then an exception is thrown. Inner objects inherit the `dynamic` parameter value from their parent unless they declare their own `dynamic` parameter value. {: .note } From 4477614f4fdc980fab0808b210107e6972e057b4 Mon Sep 17 00:00:00 2001 From: AWSHurneyt Date: Thu, 18 Jul 2024 15:30:21 -0700 Subject: [PATCH 12/36] Update per-cluster-metrics-monitors.md (#7769) Fixed typo in example. Signed-off-by: AWSHurneyt --- _observing-your-data/alerting/per-cluster-metrics-monitors.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_observing-your-data/alerting/per-cluster-metrics-monitors.md b/_observing-your-data/alerting/per-cluster-metrics-monitors.md index baea9c626b..bcaa03cc0c 100644 --- a/_observing-your-data/alerting/per-cluster-metrics-monitors.md +++ b/_observing-your-data/alerting/per-cluster-metrics-monitors.md @@ -91,7 +91,7 @@ The `script` parameter points the `source` to the Painless script `for (cluster "path": "_cluster/health/", "path_params": "", "url": "http://localhost:9200/_cluster/health/", - "cluster": ["cluster-1", "cluster-2"] + "clusters": ["cluster-1", "cluster-2"] } } ], From 2d55f1cf63203e32e12b3015605b4f0bb7ad7b98 Mon Sep 17 00:00:00 2001 From: Heather Halter Date: Thu, 18 Jul 2024 15:50:25 -0700 Subject: [PATCH 13/36] Update kafka.md (#7774) Fixed capitalization issue. Signed-off-by: Heather Halter --- _data-prepper/pipelines/configuration/sources/kafka.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_data-prepper/pipelines/configuration/sources/kafka.md b/_data-prepper/pipelines/configuration/sources/kafka.md index 4df72cfdd6..e8452a93c3 100644 --- a/_data-prepper/pipelines/configuration/sources/kafka.md +++ b/_data-prepper/pipelines/configuration/sources/kafka.md @@ -120,7 +120,7 @@ Use the following options when setting SSL encryption. Option | Required | Type | Description :--- | :--- | :--- | :--- `type` | No | String | The encryption type. Use `none` to disable encryption. Default is `ssl`. -`Insecure` | No | Boolean | A Boolean flag used to turn off SSL certificate verification. If set to `true`, certificate authority (CA) certificate verification is turned off and insecure HTTP requests are sent. Default is `false`. +`insecure` | No | Boolean | A Boolean flag used to turn off SSL certificate verification. If set to `true`, certificate authority (CA) certificate verification is turned off and insecure HTTP requests are sent. Default is `false`. #### AWS From f63f8b95f6f3dfd5043e8a5152123665fc7e98c0 Mon Sep 17 00:00:00 2001 From: gaobinlong Date: Fri, 19 Jul 2024 21:05:02 +0800 Subject: [PATCH 14/36] Fix ISM error prevention setting key is not correct (#7777) Signed-off-by: gaobinlong --- _im-plugin/ism/api.md | 6 +++--- _im-plugin/ism/error-prevention/api.md | 4 ++-- 2 files changed, 5 insertions(+), 5 deletions(-) diff --git a/_im-plugin/ism/api.md b/_im-plugin/ism/api.md index 441f737e6f..e0fbb904cd 100644 --- a/_im-plugin/ism/api.md +++ b/_im-plugin/ism/api.md @@ -553,13 +553,13 @@ Introduced 2.4 ISM allows you to run an action automatically. However, running an action can fail for a variety of reasons. You can use error prevention validation to test an action in order to rule out failures. -To enable error prevention validation, set the `plugins.index_state_management.validation_service.enabled` setting to `true`: +To enable error prevention validation, set the `plugins.index_state_management.action_validation.enabled` setting to `true`: ```bash PUT _cluster/settings { "persistent":{ - "plugins.index_state_management.validation_action.enabled": true + "plugins.index_state_management.action_validation.enabled": true } } ``` @@ -692,4 +692,4 @@ GET _plugins/_ism/explain/test-000001 }, "total_managed_indices" : 1 } -``` \ No newline at end of file +``` diff --git a/_im-plugin/ism/error-prevention/api.md b/_im-plugin/ism/error-prevention/api.md index a273d25cfb..c03a62d868 100644 --- a/_im-plugin/ism/error-prevention/api.md +++ b/_im-plugin/ism/error-prevention/api.md @@ -12,7 +12,7 @@ The ISM Error Prevention API allows you to enable Index State Management (ISM) e ## Enable error prevention validation -You can configure error prevention validation by setting the `plugins.index_state_management.validation_service.enabled` parameter. +You can configure error prevention validation by setting the `plugins.index_state_management.action_validation.enabled` parameter. #### Example request @@ -20,7 +20,7 @@ You can configure error prevention validation by setting the `plugins.index_stat PUT _cluster/settings { "persistent":{ - "plugins.index_state_management.validation_action.enabled": true + "plugins.index_state_management.action_validation.enabled": true } } ``` From 27c41222955655669469b1319cff9daaa7212956 Mon Sep 17 00:00:00 2001 From: David Venable Date: Fri, 19 Jul 2024 12:16:25 -0500 Subject: [PATCH 15/36] Data Prepper documentation updates: autogeneration campaign (#7707) Updates Data Prepper documentation with some missing fields. Adds support for autogeneration of processors by naming to match the processor and including the autogenerated comment. Signed-off-by: David Venable Signed-off-by: David Venable Signed-off-by: Melissa Vagi Co-authored-by: Melissa Vagi Co-authored-by: Heather Halter --- .../processors/convert_entry_type.md | 16 ++++++++++++++-- .../processors/{parse-ion.md => parse_ion.md} | 12 +++++++++++- .../processors/{parse-json.md => parse_json.md} | 11 +++++++++++ .../processors/{parse-xml.md => parse_xml.md} | 11 ++++++++++- .../configuration/processors/write_json.md | 11 +++++++++-- .../pipelines/configuration/sources/s3.md | 2 +- 6 files changed, 56 insertions(+), 7 deletions(-) rename _data-prepper/pipelines/configuration/processors/{parse-ion.md => parse_ion.md} (61%) rename _data-prepper/pipelines/configuration/processors/{parse-json.md => parse_json.md} (70%) rename _data-prepper/pipelines/configuration/processors/{parse-xml.md => parse_xml.md} (70%) diff --git a/_data-prepper/pipelines/configuration/processors/convert_entry_type.md b/_data-prepper/pipelines/configuration/processors/convert_entry_type.md index 2fc9fdb9bd..c2c46260ed 100644 --- a/_data-prepper/pipelines/configuration/processors/convert_entry_type.md +++ b/_data-prepper/pipelines/configuration/processors/convert_entry_type.md @@ -14,10 +14,22 @@ The `convert_entry_type` processor converts a value type associated with the spe You can configure the `convert_entry_type` processor with the following options. + + | Option | Required | Description | | :--- | :--- | :--- | -| `key`| Yes | Keys whose value needs to be converted to a different type. | -| `type` | No | Target type for the key-value pair. Possible values are `integer`, `double`, `string`, and `Boolean`. Default value is `integer`. | +| `key`| Yes | Key whose value needs to be converted to a different type. | +| `keys`| Yes | Keys whose value needs to be converted to a different type. | +| `type` | No | Target type for the key-value pair. Possible values are `integer`, `long`, `double`, `big_decimal`, `string`, and `boolean`. Default value is `integer`. | +| `null_values` | No | String representation of what constitutes a `null` value. If the field value equals one of these strings, then the value is considered `null` and is converted to `null`. | +| `scale` | No | Modifies the scale of the `big_decimal` when converting to a `big_decimal`. The default value is `0`. | +| `tags_on_failure` | No | A list of tags to be added to the event metadata when the event fails to convert. | +| `convert_when` | No | Specifies a condition using a [Data Prepper expression]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/expression-syntax/) for performing the `convert_entry_type` operation. If specified, the `convert_entry_type` operation runs only when the expression evaluates to `true`. | ## Usage diff --git a/_data-prepper/pipelines/configuration/processors/parse-ion.md b/_data-prepper/pipelines/configuration/processors/parse_ion.md similarity index 61% rename from _data-prepper/pipelines/configuration/processors/parse-ion.md rename to _data-prepper/pipelines/configuration/processors/parse_ion.md index 0edd446c42..8360eaa296 100644 --- a/_data-prepper/pipelines/configuration/processors/parse-ion.md +++ b/_data-prepper/pipelines/configuration/processors/parse_ion.md @@ -14,12 +14,22 @@ The `parse_ion` processor parses [Amazon Ion](https://amazon-ion.github.io/ion-d You can configure the `parse_ion` processor with the following options. + + | Option | Required | Type | Description | | :--- | :--- | :--- | :--- | | `source` | No | String | The field in the `event` that is parsed. Default value is `message`. | | `destination` | No | String | The destination field of the parsed JSON. Defaults to the root of the `event`. Cannot be `""`, `/`, or any white-space-only `string` because these are not valid `event` fields. | | `pointer` | No | String | A JSON pointer to the field to be parsed. There is no `pointer` by default, meaning that the entire `source` is parsed. The `pointer` can access JSON array indexes as well. If the JSON pointer is invalid, then the entire `source` data is parsed into the outgoing `event`. If the key that is pointed to already exists in the `event` and the `destination` is the root, then the pointer uses the entire path of the key. | -| `tags_on_failure` | No | String | A list of strings that specify the tags to be set in the event that the processors fails or an unknown exception occurs while parsing. +| `parse_when` | No | String | Specifies under which conditions the processor should perform parsing. Default is no condition. Accepts a Data Prepper expression string following the [Expression syntax]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/expression-syntax/). | +| `overwrite_if_destination_exists` | No | Boolean | Overwrites the destination if set to `true`. Set to `false` to prevent changing a destination value that exists. Defaults is `true`. | +| `delete_source` | No | Boolean | If set to `true`, then the source field is deleted. Defaults is `false`. | +| `tags_on_failure` | No | String | A list of strings specifying the tags to be set in the event that the processor fails or an unknown exception occurs during parsing. ## Usage diff --git a/_data-prepper/pipelines/configuration/processors/parse-json.md b/_data-prepper/pipelines/configuration/processors/parse_json.md similarity index 70% rename from _data-prepper/pipelines/configuration/processors/parse-json.md rename to _data-prepper/pipelines/configuration/processors/parse_json.md index 2cbce4782e..894d5dba42 100644 --- a/_data-prepper/pipelines/configuration/processors/parse-json.md +++ b/_data-prepper/pipelines/configuration/processors/parse_json.md @@ -15,11 +15,22 @@ The `parse_json` processor parses JSON data for an event, including any nested f You can configure the `parse_json` processor with the following options. + + | Option | Required | Type | Description | | :--- | :--- | :--- | :--- | | `source` | No | String | The field in the `event` that will be parsed. Default value is `message`. | | `destination` | No | String | The destination field of the parsed JSON. Defaults to the root of the `event`. Cannot be `""`, `/`, or any white-space-only `string` because these are not valid `event` fields. | | `pointer` | No | String | A JSON pointer to the field to be parsed. There is no `pointer` by default, meaning the entire `source` is parsed. The `pointer` can access JSON array indexes as well. If the JSON pointer is invalid then the entire `source` data is parsed into the outgoing `event`. If the key that is pointed to already exists in the `event` and the `destination` is the root, then the pointer uses the entire path of the key. | +| `parse_when` | No | String | Specifies under which conditions the processor should perform parsing. Default is no condition. Accepts a Data Prepper expression string following the [Expression syntax]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/expression-syntax/). | +| `overwrite_if_destination_exists` | No | Boolean | Overwrites the destination if set to `true`. Set to `false` to prevent changing a destination value that exists. Defaults to `true`. | +| `delete_source` | No | Boolean | If set to `true` then this will delete the source field. Defaults to `false`. | +| `tags_on_failure` | No | String | A list of strings specifying the tags to be set in the event that the processor fails or an unknown exception occurs during parsing. ## Usage diff --git a/_data-prepper/pipelines/configuration/processors/parse-xml.md b/_data-prepper/pipelines/configuration/processors/parse_xml.md similarity index 70% rename from _data-prepper/pipelines/configuration/processors/parse-xml.md rename to _data-prepper/pipelines/configuration/processors/parse_xml.md index 861705da2b..c8c9f3eebf 100644 --- a/_data-prepper/pipelines/configuration/processors/parse-xml.md +++ b/_data-prepper/pipelines/configuration/processors/parse_xml.md @@ -14,13 +14,22 @@ The `parse_xml` processor parses XML data for an event. You can configure the `parse_xml` processor with the following options. + + | Option | Required | Type | Description | | :--- | :--- | :--- | :--- | | `source` | No | String | Specifies which `event` field to parse. | | `destination` | No | String | The destination field of the parsed XML. Defaults to the root of the `event`. Cannot be `""`, `/`, or any white-space-only string because these are not valid `event` fields. | | `pointer` | No | String | A JSON pointer to the field to be parsed. The value is null by default, meaning that the entire `source` is parsed. The `pointer` can access JSON array indexes as well. If the JSON pointer is invalid, then the entire `source` data is parsed into the outgoing `event` object. If the key that is pointed to already exists in the `event` object and the `destination` is the root, then the pointer uses the entire path of the key. | | `parse_when` | No | String | Specifies under what conditions the processor should perform parsing. Default is no condition. Accepts a Data Prepper expression string following the [Data Prepper Expression Syntax]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/expression-syntax/). | -| `tags_on_failure` | No | String | A list of strings that specify the tags to be set if the processor fails or an unknown exception occurs while parsing. +| `overwrite_if_destination_exists` | No | Boolean | Overwrites the destination if set to `true`. Set to `false` to prevent changing a destination value that exists. Defaults to `true`. | +| `delete_source` | No | Boolean | If set to `true` then this will delete the source field. Defaults to `false`. | +| `tags_on_failure` | No | String | A list of strings specifying the tags to be set in the event that the processor fails or an unknown exception occurs during parsing. ## Usage diff --git a/_data-prepper/pipelines/configuration/processors/write_json.md b/_data-prepper/pipelines/configuration/processors/write_json.md index 8f1e6851da..20414b4672 100644 --- a/_data-prepper/pipelines/configuration/processors/write_json.md +++ b/_data-prepper/pipelines/configuration/processors/write_json.md @@ -11,8 +11,15 @@ nav_order: 56 The `write_json` processor converts an object in an event into a JSON string. You can customize the processor to choose the source and target field names. -Option | Description | Example -:--- | :--- | :--- + + +Option | Description | Example +:--- | :--- | :--- source | Mandatory field that specifies the name of the field in the event containing the message or object to be parsed. | If `source` is set to `"message"` and the input is `{"message": {"key1":"value1", "key2":{"key3":"value3"}}}`, then the `write_json` processor outputs the event as `"{\"key1\":\"value1\",\"key2\":{\"key3\":\"value3\"}}"`. target | An optional field that specifies the name of the field in which the resulting JSON string should be stored. If `target` is not specified, then the `source` field is used. | `key1` diff --git a/_data-prepper/pipelines/configuration/sources/s3.md b/_data-prepper/pipelines/configuration/sources/s3.md index 5a7d9986e5..7b1599f838 100644 --- a/_data-prepper/pipelines/configuration/sources/s3.md +++ b/_data-prepper/pipelines/configuration/sources/s3.md @@ -138,7 +138,7 @@ The `codec` determines how the `s3` source parses each Amazon S3 object. For inc ### `newline` codec -The `newline` codec parses each single line as a single log event. This is ideal for most application logs because each event parses per single line. It can also be suitable for S3 objects that have individual JSON objects on each line, which matches well when used with the [parse_json]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/configuration/processors/parse-json/) processor to parse each line. +The `newline` codec parses each single line as a single log event. This is ideal for most application logs because each event parses per single line. It can also be suitable for S3 objects that have individual JSON objects on each line, which matches well when used with the [parse_json]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/configuration/processors/parse_json/) processor to parse each line. Use the following options to configure the `newline` codec. From 78414ee92fb963dfc7603b78de156e626db35839 Mon Sep 17 00:00:00 2001 From: David Venable Date: Fri, 19 Jul 2024 12:57:24 -0500 Subject: [PATCH 16/36] Adds documentation for the Data Prepper delay processor. (#7708) Adds documentation for the delay processor. Signed-off-by: David Venable Signed-off-by: Melissa Vagi Co-authored-by: Melissa Vagi --- .../configuration/processors/delay.md | 27 +++++++++++++++++++ .../{delete-entries.md => delete_entries.md} | 2 +- .../configuration/processors/mutate-event.md | 2 +- 3 files changed, 29 insertions(+), 2 deletions(-) create mode 100644 _data-prepper/pipelines/configuration/processors/delay.md rename _data-prepper/pipelines/configuration/processors/{delete-entries.md => delete_entries.md} (99%) diff --git a/_data-prepper/pipelines/configuration/processors/delay.md b/_data-prepper/pipelines/configuration/processors/delay.md new file mode 100644 index 0000000000..c4e9d8e973 --- /dev/null +++ b/_data-prepper/pipelines/configuration/processors/delay.md @@ -0,0 +1,27 @@ +--- +layout: default +title: delay +parent: Processors +grand_parent: Pipelines +nav_order: 41 +--- + +# delay + +This processor will add a delay into the processor chain. Typically, you should use this only for testing, experimenting, and debugging. + +## Configuration + +Option | Required | Type | Description +:--- | :--- | :--- | :--- +`for` | No | Duration | The duration of time to delay. Defaults to `1s`. + +## Usage + +The following example shows using the `delay` processor to delay for 2 seconds. + +```yaml +processor: + - delay: + for: 2s +``` diff --git a/_data-prepper/pipelines/configuration/processors/delete-entries.md b/_data-prepper/pipelines/configuration/processors/delete_entries.md similarity index 99% rename from _data-prepper/pipelines/configuration/processors/delete-entries.md rename to _data-prepper/pipelines/configuration/processors/delete_entries.md index 33c54a0b29..c9a93a1f3e 100644 --- a/_data-prepper/pipelines/configuration/processors/delete-entries.md +++ b/_data-prepper/pipelines/configuration/processors/delete_entries.md @@ -3,7 +3,7 @@ layout: default title: delete_entries parent: Processors grand_parent: Pipelines -nav_order: 41 +nav_order: 43 --- # delete_entries diff --git a/_data-prepper/pipelines/configuration/processors/mutate-event.md b/_data-prepper/pipelines/configuration/processors/mutate-event.md index 9b3b2afb33..1afb34a970 100644 --- a/_data-prepper/pipelines/configuration/processors/mutate-event.md +++ b/_data-prepper/pipelines/configuration/processors/mutate-event.md @@ -13,7 +13,7 @@ Mutate event processors allow you to modify events in Data Prepper. The followin * [add_entries]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/configuration/processors/add-entries/) allows you to add entries to an event. * [convert_entry_type]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/configuration/processors/convert_entry_type/) allows you to convert value types in an event. * [copy_values]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/configuration/processors/copy-values/) allows you to copy values within an event. -* [delete_entries]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/configuration/processors/delete-entries/) allows you to delete entries from an event. +* [delete_entries]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/configuration/processors/delete_entries/) allows you to delete entries from an event. * [list_to_map]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/configuration/processors/list-to-map) allows you to convert list of objects from an event where each object contains a `key` field into a map of target keys. * `map_to_list` allows you to convert a map of objects from an event, where each object contains a `key` field, into a list of target keys. * [rename_keys]({{site.url}}{{site.baseurl}}/data-prepper/pipelines/configuration/processors/rename-keys/) allows you to rename keys in an event. From 9dc4d29294550675c1b3a93e9074da07d9455d14 Mon Sep 17 00:00:00 2001 From: Heather Halter Date: Fri, 19 Jul 2024 10:58:35 -0700 Subject: [PATCH 17/36] Update index.md (#7779) Community feedback Signed-off-by: Heather Halter --- _install-and-configure/install-opensearch/index.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_install-and-configure/install-opensearch/index.md b/_install-and-configure/install-opensearch/index.md index 1afe12f6a5..541321bcdd 100644 --- a/_install-and-configure/install-opensearch/index.md +++ b/_install-and-configure/install-opensearch/index.md @@ -121,5 +121,5 @@ Property | Description `opensearch.xcontent.string.length.max=` | By default, OpenSearch does not impose any limits on the maximum length of the JSON/YAML/CBOR/Smile string fields. To protect your cluster against potential distributed denial-of-service (DDoS) or memory issues, you can set the `opensearch.xcontent.string.length.max` system property to a reasonable limit (the maximum is 2,147,483,647), for example, `-Dopensearch.xcontent.string.length.max=5000000`. | `opensearch.xcontent.fast_double_writer=[true|false]` | By default, OpenSearch serializes floating-point numbers using the default implementation provided by the Java Runtime Environment. Set this value to `true` to use the Schubfach algorithm, which is faster but may lead to small differences in precision. Default is `false`. | `opensearch.xcontent.name.length.max=` | By default, OpenSearch does not impose any limits on the maximum length of the JSON/YAML/CBOR/Smile field names. To protect your cluster against potential DDoS or memory issues, you can set the `opensearch.xcontent.name.length.max` system property to a reasonable limit (the maximum is 2,147,483,647), for example, `-Dopensearch.xcontent.name.length.max=50000`. | -`opensearch.xcontent.depth.max=` | By default, OpenSearch does not impose any limits on the maximum nesting depth for JSON/YAML/CBOR/Smile documents. To protect your cluster against potential DDoS or memory issues, you can set the `opensearch.xcontent.depth.max` system property to a reasonable limit (the maximum is 2,147,483,647), for example, `-Dopensearch.xcontent.name.length.max=1000`. | +`opensearch.xcontent.depth.max=` | By default, OpenSearch does not impose any limits on the maximum nesting depth for JSON/YAML/CBOR/Smile documents. To protect your cluster against potential DDoS or memory issues, you can set the `opensearch.xcontent.depth.max` system property to a reasonable limit (the maximum is 2,147,483,647), for example, `-Dopensearch.xcontent.depth.max=1000`. | `opensearch.xcontent.codepoint.max=` | By default, OpenSearch imposes a limit of `52428800` on the maximum size of the YAML documents (in code points). To protect your cluster against potential DDoS or memory issues, you can change the `opensearch.xcontent.codepoint.max` system property to a reasonable limit (the maximum is 2,147,483,647). For example, `-Dopensearch.xcontent.codepoint.max=5000000`. | From 55ce5f81983370e2f1f1dc6a2e5dc7da8b9129f5 Mon Sep 17 00:00:00 2001 From: Stavros Macrakis <134456002+smacrakis@users.noreply.github.com> Date: Mon, 22 Jul 2024 12:01:41 -0400 Subject: [PATCH 18/36] add acronym for reference (#7786) Signed-off-by: Stavros Macrakis <134456002+smacrakis@users.noreply.github.com> --- _search-plugins/cross-cluster-search.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_search-plugins/cross-cluster-search.md b/_search-plugins/cross-cluster-search.md index 947097e8b3..7d3ff72efb 100644 --- a/_search-plugins/cross-cluster-search.md +++ b/_search-plugins/cross-cluster-search.md @@ -9,7 +9,7 @@ redirect_from: # Cross-cluster search -You can use the cross-cluster search feature in OpenSearch to search and analyze data across multiple clusters, enabling you to gain insights from distributed data sources. Cross-cluster search is available by default with the Security plugin, but you need to configure each cluster to allow remote connections from other clusters. This involves setting up remote cluster connections and configuring access permissions. +You can use cross-cluster search (CCS) in OpenSearch to search and analyze data across multiple clusters, enabling you to gain insights from distributed data sources. Cross-cluster search is available by default with the Security plugin, but you need to configure each cluster to allow remote connections from other clusters. This involves setting up remote cluster connections and configuring access permissions. --- From 84e533cc9386f063603f4facf3c95fc3d66e51b5 Mon Sep 17 00:00:00 2001 From: zhichao-aws Date: Tue, 23 Jul 2024 00:14:03 +0800 Subject: [PATCH 19/36] add doc for nested_path (#7741) Signed-off-by: zhichao-aws --- _ml-commons-plugin/agents-tools/tools/neural-sparse-tool.md | 1 + _ml-commons-plugin/agents-tools/tools/rag-tool.md | 1 + _ml-commons-plugin/agents-tools/tools/vector-db-tool.md | 1 + 3 files changed, 3 insertions(+) diff --git a/_ml-commons-plugin/agents-tools/tools/neural-sparse-tool.md b/_ml-commons-plugin/agents-tools/tools/neural-sparse-tool.md index 9fee4dcbd2..9014c585c8 100644 --- a/_ml-commons-plugin/agents-tools/tools/neural-sparse-tool.md +++ b/_ml-commons-plugin/agents-tools/tools/neural-sparse-tool.md @@ -212,6 +212,7 @@ Parameter | Type | Required/Optional | Description `name` | String | Optional | The tool name. Useful when an LLM needs to select an appropriate tool for a task. `description` | String | Optional | A description of the tool. Useful when an LLM needs to select an appropriate tool for a task. `doc_size` | Integer | Optional | The number of documents to fetch. Default is `2`. +`nested_path` | String | Optional | The path to the nested object for the nested query. Only used for nested fields. Default is `null`. ## Execute parameters diff --git a/_ml-commons-plugin/agents-tools/tools/rag-tool.md b/_ml-commons-plugin/agents-tools/tools/rag-tool.md index 1f6fafe49a..c88c2d047b 100644 --- a/_ml-commons-plugin/agents-tools/tools/rag-tool.md +++ b/_ml-commons-plugin/agents-tools/tools/rag-tool.md @@ -136,6 +136,7 @@ Parameter | Type | Required/Optional | Description `prompt` | String | Optional | The prompt to provide to the LLM. `k` | Integer | Optional | The number of nearest neighbors to search for when performing neural search. Default is 10. `enable_Content_Generation` | Boolean | Optional | If `true`, returns results generated by an LLM. If `false`, returns results directly without LLM-assisted content generation. Default is `true`. +`nested_path` | String | Optional | The path to the nested object for the nested query. Only used for nested fields. Default is `null`. ## Execute parameters diff --git a/_ml-commons-plugin/agents-tools/tools/vector-db-tool.md b/_ml-commons-plugin/agents-tools/tools/vector-db-tool.md index 9093541cbb..70d7e19321 100644 --- a/_ml-commons-plugin/agents-tools/tools/vector-db-tool.md +++ b/_ml-commons-plugin/agents-tools/tools/vector-db-tool.md @@ -225,6 +225,7 @@ Parameter | Type | Required/Optional | Description `input` | String | Required for flow agent | Runtime input sourced from flow agent parameters. If using a large language model (LLM), this field is populated with the LLM response. `doc_size` | Integer | Optional | The number of documents to fetch. Default is `2`. `k` | Integer | Optional | The number of nearest neighbors to search for when performing neural search. Default is `10`. +`nested_path` | String | Optional | The path to the nested object for the nested query. Only used for nested fields. Default is `null`. ## Execute parameters From fd629cac17a1ef18ad0729bb51b971515c5e5422 Mon Sep 17 00:00:00 2001 From: Daniel Widdis Date: Mon, 22 Jul 2024 09:17:38 -0700 Subject: [PATCH 20/36] Document new Split and Sort SearchResponseProcessors (#7767) * Add documentation for Sort SearchRequestProcessor Signed-off-by: Daniel Widdis * Add documentation for Split SearchRequestProcessor Signed-off-by: Daniel Widdis * Doc review Signed-off-by: Fanit Kolchina * Update _ingest-pipelines/processors/split.md Co-authored-by: Nathan Bower Signed-off-by: Daniel Widdis * Update _search-plugins/search-pipelines/sort-processor.md Co-authored-by: Nathan Bower Signed-off-by: Daniel Widdis * Update _search-plugins/search-pipelines/split-processor.md Co-authored-by: Nathan Bower Signed-off-by: Daniel Widdis * Update _search-plugins/search-pipelines/split-processor.md Co-authored-by: Nathan Bower Signed-off-by: Daniel Widdis * Update _search-plugins/search-pipelines/split-processor.md Co-authored-by: Nathan Bower Signed-off-by: Daniel Widdis * Update _search-plugins/search-pipelines/split-processor.md Co-authored-by: Nathan Bower Signed-off-by: Daniel Widdis --------- Signed-off-by: Daniel Widdis Signed-off-by: Fanit Kolchina Co-authored-by: Fanit Kolchina Co-authored-by: Nathan Bower --- _ingest-pipelines/processors/split.md | 2 +- .../search-pipelines/search-processors.md | 7 +- .../search-pipelines/sort-processor.md | 251 ++++++++++++++++++ .../search-pipelines/split-processor.md | 234 ++++++++++++++++ 4 files changed, 491 insertions(+), 3 deletions(-) create mode 100644 _search-plugins/search-pipelines/sort-processor.md create mode 100644 _search-plugins/search-pipelines/split-processor.md diff --git a/_ingest-pipelines/processors/split.md b/_ingest-pipelines/processors/split.md index c424ef671c..cdb0cfe3de 100644 --- a/_ingest-pipelines/processors/split.md +++ b/_ingest-pipelines/processors/split.md @@ -30,7 +30,7 @@ Parameter | Required/Optional | Description :--- | :--- | :--- `field` | Required | The field containing the string to be split. `separator` | Required | The delimiter used to split the string. This can be a regular expression pattern. -`preserve_field` | Optional | If set to `true`, preserves empty trailing fields (for example, `''`) in the resulting array. If set to `false`, empty trailing fields are removed from the resulting array. Default is `false`. +`preserve_trailing` | Optional | If set to `true`, preserves empty trailing fields (for example, `''`) in the resulting array. If set to `false`, then empty trailing fields are removed from the resulting array. Default is `false`. `target_field` | Optional | The field where the array of substrings is stored. If not specified, then the field is updated in-place. `ignore_missing` | Optional | Specifies whether the processor should ignore documents that do not contain the specified field. If set to `true`, then the processor ignores missing values in the field and leaves the `target_field` unchanged. Default is `false`. `description` | Optional | A brief description of the processor. diff --git a/_search-plugins/search-pipelines/search-processors.md b/_search-plugins/search-pipelines/search-processors.md index 4630ab950c..ad515cc541 100644 --- a/_search-plugins/search-pipelines/search-processors.md +++ b/_search-plugins/search-pipelines/search-processors.md @@ -37,13 +37,16 @@ The following table lists all supported search response processors. Processor | Description | Earliest available version :--- | :--- | :--- +[`collapse`]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/collapse-processor/)| Deduplicates search hits based on a field value, similarly to `collapse` in a search request. | 2.12 [`personalize_search_ranking`]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/personalize-search-ranking/) | Uses [Amazon Personalize](https://aws.amazon.com/personalize/) to rerank search results (requires setting up the Amazon Personalize service). | 2.9 -[`retrieval_augmented_generation`]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/rag-processor/) | Used for retrieval-augmented generation (RAG) in [conversational search]({{site.url}}{{site.baseurl}}/search-plugins/conversational-search/). | 2.10 (generally available in 2.12) [`rename_field`]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/rename-field-processor/)| Renames an existing field. | 2.8 [`rerank`]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/rerank-processor/)| Reranks search results using a cross-encoder model. | 2.12 -[`collapse`]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/collapse-processor/)| Deduplicates search hits based on a field value, similarly to `collapse` in a search request. | 2.12 +[`retrieval_augmented_generation`]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/rag-processor/) | Used for retrieval-augmented generation (RAG) in [conversational search]({{site.url}}{{site.baseurl}}/search-plugins/conversational-search/). | 2.10 (generally available in 2.12) +[`sort`]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/sort-processor/)| Sorts an array of items in either ascending or descending order. | 2.16 +[`split`]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/split-processor/)| Splits a string field into an array of substrings based on a specified delimiter. | 2.16 [`truncate_hits`]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/truncate-hits-processor/)| Discards search hits after a specified target count is reached. Can undo the effect of the `oversample` request processor. | 2.12 + ## Search phase results processors A search phase results processor runs between search phases at the coordinating node level. It intercepts the results retrieved from one search phase and transforms them before passing them to the next search phase. diff --git a/_search-plugins/search-pipelines/sort-processor.md b/_search-plugins/search-pipelines/sort-processor.md new file mode 100644 index 0000000000..dde05c1b3a --- /dev/null +++ b/_search-plugins/search-pipelines/sort-processor.md @@ -0,0 +1,251 @@ +--- +layout: default +title: Sort +nav_order: 32 +has_children: false +parent: Search processors +grand_parent: Search pipelines +--- + +# Sort processor + +The `sort` processor sorts an array of items in either ascending or descending order. Numeric arrays are sorted numerically, while string or mixed arrays (strings and numbers) are sorted lexicographically. The processor throws an error if the input is not an array. + +## Request fields + +The following table lists all available request fields. + +Field | Data type | Description +:--- | :--- | :--- +`field` | String | The field to be sorted. Must be an array. Required. +`order` | String | The sort order to apply. Accepts `asc` for ascending or `desc` for descending. Default is `asc`. +`target_field` | String | The name of the field in which the sorted array is stored. If not specified, then the sorted array is stored in the same field as the original array (the `field` variable). +`tag` | String | The processor's identifier. +`description` | String | A description of the processor. +`ignore_failure` | Boolean | If `true`, then OpenSearch [ignores any failure]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/creating-search-pipeline/#ignoring-processor-failures) of this processor and continues to run the remaining processors in the search pipeline. Optional. Default is `false`. + +## Example + +The following example demonstrates using a search pipeline with a `sort` processor. + +### Setup + +Create an index named `my_index` and index a document with the field `message` that contains an array of strings: + +```json +POST /my_index/_doc/1 +{ + "message": ["one", "two", "three", "four"], + "visibility": "public" +} +``` +{% include copy-curl.html %} + +### Creating a search pipeline + +Create a search pipeline with a `sort` response processor that sorts the `message` field and stores the sorted results in the `sorted_message` field: + +```json +PUT /_search/pipeline/my_pipeline +{ + "response_processors": [ + { + "sort": { + "field": "message", + "target_field": "sorted_message" + } + } + ] +} +``` +{% include copy-curl.html %} + +### Using a search pipeline + +Search for documents in `my_index` without a search pipeline: + +```json +GET /my_index/_search +``` +{% include copy-curl.html %} + +The response contains the field `message`: + +
+ + Response + + {: .text-delta} + +```json +{ + "took": 1, + "timed_out": false, + "_shards": { + "total": 1, + "successful": 1, + "skipped": 0, + "failed": 0 + }, + "hits": { + "total": { + "value": 1, + "relation": "eq" + }, + "max_score": 1, + "hits": [ + { + "_index": "my_index", + "_id": "1", + "_score": 1, + "_source": { + "message": [ + "one", + "two", + "three", + "four" + ], + "visibility": "public" + } + } + ] + } +} +``` +
+ +To search with a pipeline, specify the pipeline name in the `search_pipeline` query parameter: + +```json +GET /my_index/_search?search_pipeline=my_pipeline +``` +{% include copy-curl.html %} + +The `sorted_message` field contains the strings from the `message` field sorted alphabetically: + +
+ + Response + + {: .text-delta} + +```json +{ + "took": 3, + "timed_out": false, + "_shards": { + "total": 1, + "successful": 1, + "skipped": 0, + "failed": 0 + }, + "hits": { + "total": { + "value": 1, + "relation": "eq" + }, + "max_score": 1, + "hits": [ + { + "_index": "my_index", + "_id": "1", + "_score": 1, + "_source": { + "visibility": "public", + "sorted_message": [ + "four", + "one", + "three", + "two" + ], + "message": [ + "one", + "two", + "three", + "four" + ] + } + } + ] + } +} +``` +
+ +You can also use the `fields` option to search for specific fields in a document: + +```json +POST /my_index/_search?pretty&search_pipeline=my_pipeline +{ + "fields": ["visibility", "message"] +} +``` +{% include copy-curl.html %} + +In the response, the `message` field is sorted and the results are stored in the `sorted_message` field: + +
+ + Response + + {: .text-delta} + +```json +{ + "took": 2, + "timed_out": false, + "_shards": { + "total": 1, + "successful": 1, + "skipped": 0, + "failed": 0 + }, + "hits": { + "total": { + "value": 1, + "relation": "eq" + }, + "max_score": 1, + "hits": [ + { + "_index": "my_index", + "_id": "1", + "_score": 1, + "_source": { + "visibility": "public", + "sorted_message": [ + "four", + "one", + "three", + "two" + ], + "message": [ + "one", + "two", + "three", + "four" + ] + }, + "fields": { + "visibility": [ + "public" + ], + "sorted_message": [ + "four", + "one", + "three", + "two" + ], + "message": [ + "one", + "two", + "three", + "four" + ] + } + } + ] + } +} +``` +
\ No newline at end of file diff --git a/_search-plugins/search-pipelines/split-processor.md b/_search-plugins/search-pipelines/split-processor.md new file mode 100644 index 0000000000..6830f81ec3 --- /dev/null +++ b/_search-plugins/search-pipelines/split-processor.md @@ -0,0 +1,234 @@ +--- +layout: default +title: Split +nav_order: 33 +has_children: false +parent: Search processors +grand_parent: Search pipelines +--- + +# Split processor + +The `split` processor splits a string field into an array of substrings based on a specified delimiter. + +## Request fields + +The following table lists all available request fields. + +Field | Data type | Description +:--- | :--- | :--- +`field` | String | The field containing the string to be split. Required. +`separator` | String | The delimiter used to split the string. Specify either a single separator character or a regular expression pattern. Required. +`preserve_trailing` | Boolean | If set to `true`, preserves empty trailing fields (for example, `''`) in the resulting array. If set to `false`, then empty trailing fields are removed from the resulting array. Default is `false`. +`target_field` | String | The field in which the array of substrings is stored. If not specified, then the field is updated in place. +`tag` | String | The processor's identifier. +`description` | String | A description of the processor. +`ignore_failure` | Boolean | If `true`, then OpenSearch [ignores any failure]({{site.url}}{{site.baseurl}}/search-plugins/search-pipelines/creating-search-pipeline/#ignoring-processor-failures) of this processor and continues to run the remaining processors in the search pipeline. Optional. Default is `false`. + +## Example + +The following example demonstrates using a search pipeline with a `split` processor. + +### Setup + +Create an index named `my_index` and index a document containing the field `message`: + +```json +POST /my_index/_doc/1 +{ + "message": "ingest, search, visualize, and analyze data", + "visibility": "public" +} +``` +{% include copy-curl.html %} + +### Creating a search pipeline + +The following request creates a search pipeline with a `split` response processor that splits the `message` field and stores the results in the `split_message` field: + +```json +PUT /_search/pipeline/my_pipeline +{ + "response_processors": [ + { + "split": { + "field": "message", + "separator": ", ", + "target_field": "split_message" + } + } + ] +} +``` +{% include copy-curl.html %} + +### Using a search pipeline + +Search for documents in `my_index` without a search pipeline: + +```json +GET /my_index/_search +``` +{% include copy-curl.html %} + +The response contains the field `message`: + +
+ + Response + + {: .text-delta} +```json +{ + "took": 3, + "timed_out": false, + "_shards": { + "total": 1, + "successful": 1, + "skipped": 0, + "failed": 0 + }, + "hits": { + "total": { + "value": 1, + "relation": "eq" + }, + "max_score": 1, + "hits": [ + { + "_index": "my_index", + "_id": "1", + "_score": 1, + "_source": { + "message": "ingest, search, visualize, and analyze data", + "visibility": "public" + } + } + ] + } +} +``` +
+ +To search with a pipeline, specify the pipeline name in the `search_pipeline` query parameter: + +```json +GET /my_index/_search?search_pipeline=my_pipeline +``` +{% include copy-curl.html %} + +The `message` field is split and the results are stored in the `split_message` field: + +
+ + Response + + {: .text-delta} + +```json +{ + "took": 6, + "timed_out": false, + "_shards": { + "total": 1, + "successful": 1, + "skipped": 0, + "failed": 0 + }, + "hits": { + "total": { + "value": 1, + "relation": "eq" + }, + "max_score": 1, + "hits": [ + { + "_index": "my_index", + "_id": "1", + "_score": 1, + "_source": { + "visibility": "public", + "message": "ingest, search, visualize, and analyze data", + "split_message": [ + "ingest", + "search", + "visualize", + "and analyze data" + ] + } + } + ] + } +} +``` +
+ +You can also use the `fields` option to search for specific fields in a document: + +```json +POST /my_index/_search?pretty&search_pipeline=my_pipeline +{ + "fields": ["visibility", "message"] +} +``` +{% include copy-curl.html %} + +In the response, the `message` field is split and the results are stored in the `split_message` field: + +
+ + Response + + {: .text-delta} + +```json +{ + "took": 7, + "timed_out": false, + "_shards": { + "total": 1, + "successful": 1, + "skipped": 0, + "failed": 0 + }, + "hits": { + "total": { + "value": 1, + "relation": "eq" + }, + "max_score": 1, + "hits": [ + { + "_index": "my_index", + "_id": "1", + "_score": 1, + "_source": { + "visibility": "public", + "message": "ingest, search, visualize, and analyze data", + "split_message": [ + "ingest", + "search", + "visualize", + "and analyze data" + ] + }, + "fields": { + "visibility": [ + "public" + ], + "message": [ + "ingest, search, visualize, and analyze data" + ], + "split_message": [ + "ingest", + "search", + "visualize", + "and analyze data" + ] + } + } + ] + } +} +``` +
\ No newline at end of file From 17341992c553eddb7b6f560a5dbac6559d2a9237 Mon Sep 17 00:00:00 2001 From: Daniel Widdis Date: Mon, 22 Jul 2024 09:19:24 -0700 Subject: [PATCH 21/36] Add documentation for Deprovision Workflow API allow_delete parameter (#7639) * Add documentation for Deprovision Workflow API allow_delete parameter Signed-off-by: Daniel Widdis * Add new steps and missing delete search pipeline doc Signed-off-by: Daniel Widdis * Revert changes to workflow steps. Users can't use these new step types Signed-off-by: Daniel Widdis * Update _automating-configurations/api/deprovision-workflow.md Co-authored-by: Nathan Bower Signed-off-by: Daniel Widdis * Update _automating-configurations/api/deprovision-workflow.md Co-authored-by: Nathan Bower Signed-off-by: Daniel Widdis * Update _automating-configurations/api/deprovision-workflow.md Co-authored-by: Nathan Bower Signed-off-by: Daniel Widdis * Remove redundant use of workflow, accept other edits Signed-off-by: Daniel Widdis --------- Signed-off-by: Daniel Widdis Co-authored-by: Nathan Bower --- .gitignore | 1 + .../api/deprovision-workflow.md | 13 +++++++++- .../deleting-search-pipeline.md | 26 +++++++++++++++++++ 3 files changed, 39 insertions(+), 1 deletion(-) create mode 100644 _search-plugins/search-pipelines/deleting-search-pipeline.md diff --git a/.gitignore b/.gitignore index 446d1deda6..da3cf9d144 100644 --- a/.gitignore +++ b/.gitignore @@ -6,3 +6,4 @@ Gemfile.lock .idea *.iml .jekyll-cache +.project diff --git a/_automating-configurations/api/deprovision-workflow.md b/_automating-configurations/api/deprovision-workflow.md index e9219536ce..98c944a9d4 100644 --- a/_automating-configurations/api/deprovision-workflow.md +++ b/_automating-configurations/api/deprovision-workflow.md @@ -9,7 +9,9 @@ nav_order: 70 When you no longer need a workflow, you can deprovision its resources. Most workflow steps that create a resource have corresponding workflow steps to reverse that action. To retrieve all resources currently created for a workflow, call the [Get Workflow Status API]({{site.url}}{{site.baseurl}}/automating-configurations/api/get-workflow-status/). When you call the Deprovision Workflow API, resources included in the `resources_created` field of the Get Workflow Status API response will be removed using a workflow step corresponding to the one that provisioned them. -The workflow executes the provisioning workflow steps in reverse order. If failures occur because of resource dependencies, such as preventing deletion of a registered model if it is still deployed, the workflow attempts retries. +The workflow executes the provisioning steps in reverse order. If a failure occurs because of a resource dependency, such as trying to delete a registered model that is still deployed, then the workflow retries the failing step as long as at least one resource was deleted. + +To prevent data loss, resources created using the `create_index`, `create_search_pipeline`, and `create_ingest_pipeline` steps require the resource ID to be included in the `allow_delete` parameter. ## Path and HTTP methods @@ -24,6 +26,7 @@ The following table lists the available path parameters. | Parameter | Data type | Description | | :--- | :--- | :--- | | `workflow_id` | String | The ID of the workflow to be deprovisioned. Required. | +| `allow-delete` | String | A comma-separated list of resource IDs to be deprovisioned. Required if deleting resources of type `index_name` or `pipeline_id`. | ### Example request @@ -53,6 +56,14 @@ If deprovisioning did not completely remove all resources, OpenSearch responds w In some cases, the failure happens because of another dependent resource that took some time to be removed. In this case, you can attempt to send the same request again. {: .tip} +If deprovisioning required the `allow_delete` parameter, then OpenSearch responds with a `403 (FORBIDDEN)` status and identifies the resources that were not deprovisioned: + +```json +{ + "error": "These resources require the allow_delete parameter to deprovision: [index_name my-index]." +} +``` + To obtain a more detailed deprovisioning status than is provided by the summary in the error response, query the [Get Workflow Status API]({{site.url}}{{site.baseurl}}/automating-configurations/api/get-workflow-status/). On success, the workflow returns to a `NOT_STARTED` state. If some resources have not yet been removed, they are provided in the response. \ No newline at end of file diff --git a/_search-plugins/search-pipelines/deleting-search-pipeline.md b/_search-plugins/search-pipelines/deleting-search-pipeline.md new file mode 100644 index 0000000000..3f113f7688 --- /dev/null +++ b/_search-plugins/search-pipelines/deleting-search-pipeline.md @@ -0,0 +1,26 @@ +--- +layout: default +title: Deleting search pipelines +nav_order: 30 +has_children: false +parent: Search pipelines +grand_parent: Search +--- + +# Deleting search pipelines + +Use the following request to delete a pipeline. + +To delete a specific search pipeline, pass the pipeline ID as a parameter: + +```json +DELETE /_search/pipeline/ +``` +{% include copy-curl.html %} + +To delete all search pipelines in a cluster, use the wildcard character (`*`): + +```json +DELETE /_search/pipeline/* +``` +{% include copy-curl.html %} From f3fe8f910fa50f914dfafe90ed3e6fc8c0a87b79 Mon Sep 17 00:00:00 2001 From: Tejas Shah Date: Mon, 22 Jul 2024 10:51:53 -0700 Subject: [PATCH 22/36] Adds Documentation for dynamic query parameters for kNN search request (#7761) * Adds documentation for dynamic query parameters Signed-off-by: Tejas Shah * Update _search-plugins/knn/approximate-knn.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _search-plugins/knn/approximate-knn.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Tejas Shah * Update _search-plugins/knn/approximate-knn.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Tejas Shah * Update _search-plugins/knn/approximate-knn.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Tejas Shah * Update _search-plugins/knn/approximate-knn.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Tejas Shah * Update _search-plugins/knn/approximate-knn.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Tejas Shah * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Apply suggestions from code review Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: Tejas Shah Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower --- _search-plugins/knn/approximate-knn.md | 51 +++++++++++++++++++++++++- 1 file changed, 49 insertions(+), 2 deletions(-) diff --git a/_search-plugins/knn/approximate-knn.md b/_search-plugins/knn/approximate-knn.md index 144365166f..fa1b4096c7 100644 --- a/_search-plugins/knn/approximate-knn.md +++ b/_search-plugins/knn/approximate-knn.md @@ -141,7 +141,7 @@ The following table provides examples of the number of results returned by vario 10 | 1 | 1 | 4 | 4 | 1 10 | 10 | 1 | 4 | 10 | 10 10 | 1 | 2 | 4 | 8 | 2 - + The number of results returned by Faiss/NMSLIB differs from the number of results returned by Lucene only when `k` is smaller than `size`. If `k` and `size` are equal, all engines return the same number of results. Starting in OpenSearch 2.14, you can use `k`, `min_score`, or `max_distance` for [radial search]({{site.url}}{{site.baseurl}}/search-plugins/knn/radial-search-knn/). @@ -253,7 +253,54 @@ POST _bulk ... ``` -After data is ingested, it can be search just like any other `knn_vector` field! +After data is ingested, it can be searched in the same way as any other `knn_vector` field. + +### Additional query parameters + +Starting with version 2.16, you can provide `method_parameters` in a search request: + +```json +GET my-knn-index-1/_search +{ + "size": 2, + "query": { + "knn": { + "my_vector2": { + "vector": [2, 3, 5, 6], + "k": 2, + "method_parameters" : { + "ef_search": 100 + } + } + } + } +} +``` +{% include copy-curl.html %} + +These parameters are dependent on the combination of engine and method used to create the index. The following sections provide information about the supported `method_parameters`. + +#### `ef_search` + +You can provide the `ef_search` parameter when searching an index created using the `hnsw` method. The `ef_search` parameter specifies the number of vectors to examine in order to find the top k nearest neighbors. Higher `ef_search` values improve recall at the cost of increased search latency. The value must be positive. + +The following table provides information about the `ef_search` parameter for the supported engines. + +Engine | Radial query support | Notes +:--- | :--- | :--- +`nmslib` | No | If `ef_search` is present in a query, it overrides the `index.knn.algo_param.ef_search` index setting. +`faiss` | Yes | If `ef_search` is present in a query, it overrides the `index.knn.algo_param.ef_search` index setting. +`lucene` | No | When creating a search query, you must specify `k`. If you provide both `k` and `ef_search`, then the larger value is passed to the engine. If `ef_search` is larger than `k`, you can provide the `size` parameter to limit the final number of results to `k`. + +#### `nprobes` + +You can provide the `nprobes` parameter when searching an index created using the `ivf` method. The `nprobes` parameter specifies the number of `nprobes` clusters to examine in order to find the top k nearest neighbors. Higher `nprobes` values improve recall at the cost of increased search latency. The value must be positive. + +The following table provides information about the `nprobes` parameter for the supported engines. + +Engine | Notes +:--- | :--- +`faiss` | If `nprobes` is present in a query, it overrides the value provided when creating the index. ### Using approximate k-NN with filters From b2abf250a4d0d3316e2960bd8910f1d49f79319c Mon Sep 17 00:00:00 2001 From: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Date: Mon, 22 Jul 2024 16:32:05 -0500 Subject: [PATCH 23/36] Add Rollover API (#7685) * Add Rollover API. Signed-off-by: Archer * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Make rollover match template. Signed-off-by: Archer * Apply suggestions from code review Co-authored-by: Melissa Vagi Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update _api-reference/index-apis/rollover.md Co-authored-by: Melissa Vagi Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update _api-reference/index-apis/rollover.md Co-authored-by: Melissa Vagi Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower Co-authored-by: Melissa Vagi Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update _api-reference/index-apis/rollover.md Co-authored-by: Nathan Bower Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: Archer Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Melissa Vagi Co-authored-by: Nathan Bower --- _api-reference/index-apis/rollover.md | 195 ++++++++++++++++++++++++++ 1 file changed, 195 insertions(+) create mode 100644 _api-reference/index-apis/rollover.md diff --git a/_api-reference/index-apis/rollover.md b/_api-reference/index-apis/rollover.md new file mode 100644 index 0000000000..722dfe196c --- /dev/null +++ b/_api-reference/index-apis/rollover.md @@ -0,0 +1,195 @@ +--- +layout: default +title: Rollover Index +parent: Index APIs +nav_order: 63 +--- + +# Rollover Index +Introduced 1.0 +{: .label .label-purple } + +The Rollover Index API creates a new index for a data stream or index alias based on the `wait_for_active_shards` setting. + +## Path and HTTP methods + +```json +POST //_rollover/ +POST //_rollover/ +``` + +## Rollover types + +You can roll over a data stream, an index alias with one index, or an index alias with a write index. + +### Data stream + +When you perform a rollover operation on a data stream, the API generates a fresh write index for that stream. Simultaneously, the stream's preceding write index transforms into a regular backing index. Additionally, the rollover process increments the generation count of the data stream. Data stream rollovers do not support specifying index settings in the request body. + +### Index alias with one index + +When initiating a rollover on an index alias associated with a single index, the API generates a new index and disassociates the original index from the alias. + +### Index alias with a write index + +When an index alias references multiple indexes, one index must be designated as the write index. During a rollover, the API creates a new write index with its `is_write_index` property set to `true` while updating the previous write index by setting its `is_write_index property` to `false.` + +## Incrementing index names for an alias + +During the index alias rollover process, if you don't specify a custom name and the current index's name ends with a hyphen followed by a number (for example, `my-index-000001` or `my-index-3`), then the rollover operation will automatically increment that number for the new index's name. For instance, rolling over `my-index-000001` will generate `my-index-000002`. The numeric portion is always padded with leading zeros to ensure a consistent length of six characters. + +## Using date math with index rollovers + +When using an index alias for time-series data, you can leverage [date math](https://opensearch.org/docs/latest/field-types/supported-field-types/date/) in the index name to track the rollover date. For example, you can create an alias pointing to `my-index-{now/d}-000001`. If you create an alias on June 11, 2029, then the index name would be `my-index-2029.06.11-000001`. For a rollover on June 12, 2029, the new index would be named `my-index-2029.06.12-000002`. See [Roll over an index alias with a write index](#rolling-over-an-index-alias-with-a-write-index) for a practical example. + +## Path parameters + +The Rollover Index API supports the parameters listed in the following table. + +Parameter | Type | Description +:--- | :--- | :--- +`` | String | The name of the data stream or index alias to roll over. Required. | +`` | String | The name of the index to create. Supports date math. Data streams do not support this parameter. If the name of the alias's current write index does not end with `-` and a number, such as `my-index-000001` or `my-index-2`, then the parameter is required. + +## Query parameters + +The following table lists the supported query parameters. + +Parameter | Type | Description +:--- | :--- | :--- +`cluster_manager_timeout` | Time | The amount of time to wait for a connection to the cluster manager node. Default is `30s`. +`timeout` | Time | The amount of time to wait for a response. Default is `30s`. +`wait_for_active_shards` | String | The number of active shards that must be available before OpenSearch processes the request. Default is `1` (only the primary shard). You can also set to `all` or a positive integer. Values greater than `1` require replicas. For example, if you specify a value of `3`, then the index must have two replicas distributed across two additional nodes in order for the operation to succeed. + +## Request body + +The following request body parameters are supported. + +### `alias` + +The `alias` parameter specifies the alias name as the key. It is required when the `template` option exists in the request body. The object body contains the following optional parameters. + + +Parameter | Type | Description +:--- | :--- | :--- +`filter` | Query DSL object | The query that limits the number of documents that the alias can access. +`index_routing` | String | The value that routes indexing operations to a specific shard. When specified, overwrites the `routing` value for indexing operations. +`is_hidden` | Boolean | Hides or unhides the alias. When `true`, the alias is hidden. Default is `false`. Indexes for the alias must have matching values for this setting. +`is_write_index` | Boolean | Specifies the write index. When `true`, the index is the write index for the alias. Default is `false`. +`routing` | String | The value used to route index and search operations to a specific shard. +`search_routing` | String | Routes search operations to a specific shard. When specified, it overwrites `routing` for search operations. + +### `mappings` + +The `mappings` parameter specifies the index field mappings. It is optional. See [Mappings and field types](https://opensearch.org/docs/latest/field-types/) for more information. + +### `conditions` + +The `conditions` parameter is an optional object defining criteria for triggering the rollover. When provided, OpenSearch only rolls over if the current index satisfies one or more specified conditions. If omitted, then the rollover occurs unconditionally without prerequisites. + +The object body supports the following parameters. + +Parameter | Type | Description +:--- | :--- | :--- +| `max_age` | Time units | Triggers a rollover after the maximum elapsed time since index creation is reached. The elapsed time is always calculated since the index creation time, even if the index origination date is configured to a custom date, such as when using the `index.lifecycle.parse_origination_date` or `index.lifecycle.origination_date` settings. Optional. | +`max_docs` | Integer | Triggers a rollover after the specified maximum number of documents, excluding documents added since the last refresh and documents in replica shards. Optional. +`max_size` | Byte units | Triggers a rollover when the index reaches a specified size, calculated as the total size of all primary shards. Replicas are not counted. Use the `_cat indices` API and check the `pri.store.size` value to see the current index size. Optional. +`max_primary_shard_size` | Byte units | Triggers a rollover when the largest primary shard in the index reaches a certain size. This is the maximum size of the primary shards in the index. As with `max_size`, replicas are ignored. To see the current shard size, use the `_cat shards` API. The `store` value shows the size of each shard, and `prirep` indicates whether a shard is a primary (`p`) or a replica (`r`). Optional. + +### `settings` + +The `settings` parameter specifies the index configuration options. See [Index settings](https://opensearch.org/docs/latest/install-and-configure/configuring-opensearch/index-settings/) for more information. + +## Example requests + +The following examples illustrate using the Rollover Index API. A rollover occurs when one or more of the specified conditions are met: + +- The index was created 5 or more days ago. +- The index contains 500 or more documents. +- The index's largest primary shard is 100 GB or larger. + +### Rolling over a data stream + +The following request rolls over the data stream if the current write index meets any of the specified conditions: + +```json +POST my-data-stream/_rollover +{ + "conditions": { + "max_age": "5d", + "max_docs": 500, + "max_primary_shard_size": "100gb" + } +} +``` +{% include copy-curl.html %} + +### Rolling over an index alias with a write index + +The following request creates a date-time index and sets it as the write index for `my-alias`: + +```json +PUT +PUT %3Cmy-index-%7Bnow%2Fd%7D-000001%3E +{ + "aliases": { + "my-alias": { + "is_write_index": true + } + } +} +``` +{% include copy-curl.html %} + +The next request performs a rollover using the alias: + +```json +POST my-alias/_rollover +{ + "conditions": { + "max_age": "5d", + "max_docs": 500, + "max_primary_shard_size": "100gb" + } +} +``` +{% include copy-curl.html %} + +### Specifying settings during a rollover + +In most cases, you can use an index template to automatically configure the indexes created during a rollover operation. However, when rolling over an index alias, you can use the Rollover Index API to introduce additional index settings or override the settings defined in the template by sending the following request: + +```json +POST my-alias/_rollover +{ + "settings": { + "index.number_of_shards": 4 + } +} +``` +{% include copy-curl.html %} + + +## Example response + +OpenSearch returns the following response confirming that all conditions except `max_primary_shard_size` were met: + +```json +{ + "acknowledged": true, + "shards_acknowledged": true, + "old_index": ".ds-my-data-stream-2029.06.11-000001", + "new_index": ".ds-my-data-stream-2029.06.12-000002", + "rolled_over": true, + "dry_run": false, + "conditions": { + "[max_age: 5d]": true, + "[max_docs: 500]": true, + "[max_primary_shard_size: 100gb]": false + } +} +``` + + + + From 1d2e4447cd518791d2900d59132db198d83eaae6 Mon Sep 17 00:00:00 2001 From: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Date: Mon, 22 Jul 2024 16:32:22 -0500 Subject: [PATCH 24/36] Fix liquid syntax errors. (#7785) * Fix liquid syntax errors. Signed-off-by: Archer * Update render-template.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update _api-reference/render-template.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update _api-reference/render-template.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: Archer Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --- _api-reference/render-template.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/_api-reference/render-template.md b/_api-reference/render-template.md index 16bada0290..409fde5e4a 100644 --- a/_api-reference/render-template.md +++ b/_api-reference/render-template.md @@ -44,7 +44,7 @@ Both of the following request examples use the search template with the template "source": { "query": { "match": { - "play_name": "{{play_name}}" + "play_name": "{% raw %}{{play_name}}{% endraw %}" } } }, @@ -76,11 +76,11 @@ If you don't want to use a saved template, or want to test a template before sav ``` { "source": { - "from": "{{from}}{{^from}}10{{/from}}", - "size": "{{size}}{{^size}}10{{/size}}", + "from": "{% raw %}{{from}}{{^from}}0{{/from}}{% endraw %}", + "size": "{% raw %}{{size}}{{^size}}10{{/size}}{% endraw %}", "query": { "match": { - "play_name": "{{play_name}}" + "play_name": "{% raw %}{{play_name}}{% endraw %}" } } }, From 3977152d4d04f6bd2c6520b94a90cf5202541ab9 Mon Sep 17 00:00:00 2001 From: Stavros Macrakis <134456002+smacrakis@users.noreply.github.com> Date: Tue, 23 Jul 2024 11:48:12 -0400 Subject: [PATCH 25/36] Explain ISM + link (#7787) * Explain ISM + link Signed-off-by: Stavros Macrakis <134456002+smacrakis@users.noreply.github.com> * Update _im-plugin/refresh-analyzer.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: Stavros Macrakis <134456002+smacrakis@users.noreply.github.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --- _im-plugin/refresh-analyzer.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_im-plugin/refresh-analyzer.md b/_im-plugin/refresh-analyzer.md index 2e50f06dc0..bff54b739f 100644 --- a/_im-plugin/refresh-analyzer.md +++ b/_im-plugin/refresh-analyzer.md @@ -10,7 +10,7 @@ redirect_from: # Refresh search analyzer -With ISM installed, you can refresh search analyzers in real time with the following API: +You can refresh search analyzers in real time using the following API. This requires the [Index State Management]({{site.url}}{{site.baseurl}}/im-plugin/ism/index/) (ISM) plugin to be installed. For more information, see [Installing plugins]({{site.url}}{{site.baseurl}}/install-and-configure/plugins/). ```json POST /_plugins/_refresh_search_analyzers/ From eb08f04b89d30f7d26f4fd8b0a47965fb52d0bd1 Mon Sep 17 00:00:00 2001 From: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Date: Tue, 23 Jul 2024 12:36:26 -0400 Subject: [PATCH 26/36] Unify and correct geoshape GeoJSON and WKT examples (#7801) * Unify and correct geoshape GeoJSON and WKT examples Signed-off-by: Fanit Kolchina * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: Fanit Kolchina Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower --- .../supported-field-types/geo-shape.md | 72 +++++++++++-------- 1 file changed, 41 insertions(+), 31 deletions(-) diff --git a/_field-types/supported-field-types/geo-shape.md b/_field-types/supported-field-types/geo-shape.md index cbf63551df..b7b06a0d04 100644 --- a/_field-types/supported-field-types/geo-shape.md +++ b/_field-types/supported-field-types/geo-shape.md @@ -68,7 +68,7 @@ PUT testindex/_doc/1 { "location" : { "type" : "point", - "coordinates" : [74.00, 40.71] + "coordinates" : [74.0060, 40.7128] } } ``` @@ -126,10 +126,12 @@ PUT testindex/_doc/3 "location" : { "type" : "polygon", "coordinates" : [ - [[74.0060, 40.7128], - [71.0589, 42.3601], - [73.7562, 42.6526], - [74.0060, 40.7128]] + [ + [74.0060, 40.7128], + [73.7562, 42.6526], + [71.0589, 42.3601], + [74.0060, 40.7128] + ] ] } } @@ -159,15 +161,18 @@ PUT testindex/_doc/4 "location" : { "type" : "polygon", "coordinates" : [ - [[74.0060, 40.7128], - [71.0589, 42.3601], - [73.7562, 42.6526], - [74.0060, 40.7128]], - - [[72.6734,41.7658], - [72.6506, 41.5623], - [73.0515, 41.5582], - [72.6734, 41.7658]] + [ + [74.0060, 40.7128], + [73.7562, 42.6526], + [71.0589, 42.3601], + [74.0060, 40.7128] + ], + [ + [72.6734,41.7658], + [73.0515, 41.5582], + [72.6506, 41.5623], + [72.6734, 41.7658] + ] ] } } @@ -179,12 +184,12 @@ Index a polygon (triangle) with a triangular hole in WKT format: ```json PUT testindex/_doc/4 { - "location" : "POLYGON ((40.7128 74.0060, 42.3601 71.0589, 42.6526 73.7562, 40.7128 74.0060), (41.7658 72.6734, 41.5623 72.6506, 41.5582 73.0515, 41.7658 72.6734))" + "location" : "POLYGON ((74.0060 40.7128, 71.0589 42.3601, 73.7562 42.6526, 74.0060 40.7128), (72.6734 41.7658, 72.6506 41.5623, 73.0515 41.5582, 72.6734 41.7658))" } ``` {% include copy-curl.html %} -In OpenSearch, you can specify a polygon by listing its vertices clockwise or counterclockwise. This works well for polygons that do not cross the date line (are narrower than 180°). However, a polygon that crosses the date line (is wider than 180°) might be ambiguous because WKT does not impose a specific order on vertices. Thus, you must specify polygons that cross the date line by listing their vertices counterclockwise. +You can specify a polygon in OpenSearch by listing its vertices in clockwise or counterclockwise order. This works well for polygons that do not cross the date line (that are narrower than 180°). However, a polygon that crosses the date line (is wider than 180°) might be ambiguous because WKT does not impose a specific order on vertices. Thus, you must specify polygons that cross the date line by listing their vertices in counterclockwise order. You can define an [`orientation`](#parameters) parameter to specify the vertex traversal order at mapping time: @@ -295,23 +300,28 @@ PUT testindex/_doc/4 "type" : "multipolygon", "coordinates" : [ [ - [[74.0060, 40.7128], - [71.0589, 42.3601], - [73.7562, 42.6526], - [74.0060, 40.7128]], - - [[72.6734,41.7658], - [72.6506, 41.5623], - [73.0515, 41.5582], - [72.6734, 41.7658]] + [ + [74.0060, 40.7128], + [73.7562, 42.6526], + [71.0589, 42.3601], + [74.0060, 40.7128] + ], + [ + [73.0515, 41.5582], + [72.6506, 41.5623], + [72.6734, 41.7658], + [73.0515, 41.5582] + ] ], [ - [[73.9776, 40.7614], - [73.9554, 40.7827], - [73.9631, 40.7812], - [73.9776, 40.7614]] + [ + [73.9146, 40.8252], + [73.8871, 41.0389], + [73.6853, 40.9747], + [73.9146, 40.8252] ] ] + ] } } ``` @@ -322,7 +332,7 @@ Index a multipolygon in WKT format: ```json PUT testindex/_doc/4 { - "location" : "MULTIPOLYGON (((40.7128 74.0060, 42.3601 71.0589, 42.6526 73.7562, 40.7128 74.0060), (41.7658 72.6734, 41.5623 72.6506, 41.5582 73.0515, 41.7658 72.6734)), ((73.9776 40.7614, 73.9554 40.7827, 73.9631 40.7812, 73.9776 40.7614)))" + "location" : "MULTIPOLYGON (((74.0060 40.7128, 71.0589 42.3601, 73.7562 42.6526, 74.0060 40.7128), (72.6734 41.7658, 72.6506 41.5623, 73.0515 41.5582, 72.6734 41.7658)), ((73.9146 40.8252, 73.6853 40.9747, 73.8871 41.0389, 73.9146 40.8252)))" } ``` {% include copy-curl.html %} @@ -400,5 +410,5 @@ Parameter | Description :--- | :--- `coerce` | A Boolean value that specifies whether to automatically close unclosed linear rings. Default is `false`. `ignore_malformed` | A Boolean value that specifies to ignore malformed GeoJSON or WKT geoshapes and not to throw an exception. Default is `false` (throw an exception when geoshapes are malformed). -`ignore_z_value` | Specific to points with three coordinates. If `ignore_z_value` is `true`, the third coordinate is not indexed but is still stored in the _source field. If `ignore_z_value` is `false`, an exception is thrown. Default is `true`. +`ignore_z_value` | Specific to points with three coordinates. If `ignore_z_value` is `true`, then the third coordinate is not indexed but is still stored in the `_source` field. If `ignore_z_value` is `false`, then an exception is thrown. Default is `true`. `orientation` | Specifies the traversal order of the vertices in the geoshape's list of coordinates. `orientation` takes the following values:
1. RIGHT: counterclockwise. Specify RIGHT orientation by using one of the following strings (uppercase or lowercase): `right`, `counterclockwise`, `ccw`.
2. LEFT: clockwise. Specify LEFT orientation by using one of the following strings (uppercase or lowercase): `left`, `clockwise`, `cw`. This value can be overridden by individual documents.
Default is `RIGHT`. From 50eed6b420ea74df74621aad704f9538db115983 Mon Sep 17 00:00:00 2001 From: Frank Dattalo <73919354+fddattal@users.noreply.github.com> Date: Tue, 23 Jul 2024 10:14:14 -0700 Subject: [PATCH 27/36] Documentation Updates for plugins.query.datasources.enabled SQL Setting (#7794) * Documentation Updates for plugins.query.datasources.enabled SQL Setting This setting allows users to toggle the data source code paths in the SQL plugin. Ref: https://github.com/opensearch-project/sql/pull/2811/files Signed-off-by: Frank Dattalo * Update _search-plugins/sql/settings.md Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: Frank Dattalo Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower --- _search-plugins/sql/settings.md | 1 + 1 file changed, 1 insertion(+) diff --git a/_search-plugins/sql/settings.md b/_search-plugins/sql/settings.md index d4aaac7f40..4842f98449 100644 --- a/_search-plugins/sql/settings.md +++ b/_search-plugins/sql/settings.md @@ -78,6 +78,7 @@ Setting | Default | Description `plugins.sql.cursor.keep_alive` | 1 minute | Configures how long the cursor context is kept open. Cursor contexts are resource-intensive, so we recommend a low value. `plugins.query.memory_limit` | 85% | Configures the heap memory usage limit for the circuit breaker of the query engine. `plugins.query.size_limit` | 200 | Sets the default size of index that the query engine fetches from OpenSearch. +`plugins.query.datasources.enabled` | true | Change to `false` to disable support for data sources in the plugin. ## Spark connector settings From 06ef161814c7bc428d1b05dc4b15f6570ff0b9fe Mon Sep 17 00:00:00 2001 From: gaobinlong Date: Wed, 24 Jul 2024 02:07:39 +0800 Subject: [PATCH 28/36] Add the documentation of create or update alias API (#7641) * Add the documentation of create or update alias API Signed-off-by: gaobinlong * Fix typo Signed-off-by: gaobinlong * Refine the wording Signed-off-by: gaobinlong * Update update-alias.md * Fix typo Signed-off-by: gaobinlong * Add some clarification Signed-off-by: gaobinlong * Update update-alias.md * Update update-alias.md * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update _api-reference/index-apis/update-alias.md Signed-off-by: Nathan Bower --------- Signed-off-by: gaobinlong Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: Nathan Bower Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Nathan Bower --- _api-reference/index-apis/update-alias.md | 83 +++++++++++++++++++++++ 1 file changed, 83 insertions(+) create mode 100644 _api-reference/index-apis/update-alias.md diff --git a/_api-reference/index-apis/update-alias.md b/_api-reference/index-apis/update-alias.md new file mode 100644 index 0000000000..cac05ceedb --- /dev/null +++ b/_api-reference/index-apis/update-alias.md @@ -0,0 +1,83 @@ +--- +layout: default +title: Create or Update Alias +parent: Index APIs +nav_order: 5 +--- + +# Create or Update Alias +**Introduced 1.0** +{: .label .label-purple } + +The Create or Update Alias API adds a data stream or index to an alias or updates the settings for an existing alias. For more alias API operations, see [Index aliases]({{site.url}}{{site.baseurl}}/opensearch/index-alias/). + +The Create or Update Alias API is distinct from the [Alias API]({{site.url}}{{site.baseurl}}/opensearch/rest-api/alias/), which supports the addition and removal of aliases and the removal of alias indexes. In contrast, the following API only supports adding or updating an alias without updating the index itself. Each API also uses different request body parameters. +{: .note} + +## Path and HTTP methods + +``` +POST //_alias/ +PUT //_alias/ +POST /_alias/ +PUT /_alias/ +POST //_aliases/ +PUT //_aliases/ +POST /_aliases/ +PUT /_aliases/ +PUT //_alias +PUT //_aliases +PUT /_alias +``` + +## Path parameters + +| Parameter | Type | Description | +:--- | :--- | :--- +| `target` | String | A comma-delimited list of data streams and indexes. Wildcard expressions (`*`) are supported. To target all data streams and indexes in a cluster, use `_all` or `*`. Optional. | +| `alias-name` | String | The alias name to be created or updated. Optional. | + +## Query parameters + +All query parameters are optional. + +Parameter | Type | Description +:--- | :--- | :--- +`cluster_manager_timeout` | Time | The amount of time to wait for a response from the cluster manager node. Default is `30s`. +`timeout` | Time | The amount of time to wait for a response from the cluster. Default is `30s`. + +## Request body + +In the request body, you can specify the index name, the alias name, and the settings for the alias. All fields are optional. + +Field | Type | Description +:--- | :--- | :--- | :--- +`index` | String | A comma-delimited list of data streams or indexes that you want to associate with the alias. If this field is set, it will override the index name specified in the URL path. +`alias` | String | The name of the alias. If this field is set, it will override the alias name specified in the URL path. +`is_write_index` | Boolean | Specifies whether the index should be a write index. An alias can only have one write index at a time. If a write request is submitted to an alias that links to multiple indexes, then OpenSearch runs the request only on the write index. +`routing` | String | Assigns a custom value to a shard for specific operations. +`index_routing` | String | Assigns a custom value to a shard only for index operations. +`search_routing` | String | Assigns a custom value to a shard only for search operations. +`filter` | Object | A filter to use with the alias so that the alias points to a filtered part of the index. + +## Example request + +The following example request adds a sample alias with a custom routing value: + +```json +POST sample-index/_alias/sample-alias +{ + "routing":"test" +} +``` +{% include copy-curl.html %} + +## Example response + +```json +{ + "acknowledged": true +} +``` + +For more alias API operations, see [Index aliases]({{site.url}}{{site.baseurl}}/opensearch/index-alias/). From 5a788e98efc48e65e818610b14b0c513dca1c5eb Mon Sep 17 00:00:00 2001 From: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Date: Wed, 24 Jul 2024 11:36:11 -0500 Subject: [PATCH 29/36] Add Recovery API (#7653) * Add Recover Index API Signed-off-by: Archer * Add examples Signed-off-by: Archer * Fix link and parameter Signed-off-by: Archer * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: gaobinlong Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update recover.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Heather Halter Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update _api-reference/index-apis/recover.md Signed-off-by: Heather Halter * Update _api-reference/index-apis/recover.md Signed-off-by: Heather Halter * Update recover.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update _api-reference/index-apis/recover.md Co-authored-by: Nathan Bower Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: Archer Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: Heather Halter Co-authored-by: gaobinlong Co-authored-by: Heather Halter Co-authored-by: Nathan Bower --- _api-reference/index-apis/recover.md | 291 +++++++++++++++++++++++++++ 1 file changed, 291 insertions(+) create mode 100644 _api-reference/index-apis/recover.md diff --git a/_api-reference/index-apis/recover.md b/_api-reference/index-apis/recover.md new file mode 100644 index 0000000000..dc2df1e5a2 --- /dev/null +++ b/_api-reference/index-apis/recover.md @@ -0,0 +1,291 @@ +--- +layout: default +title: Recovery +parent: Index APIs +nav_order: 40 +--- + +# Recovery API +Introduced 1.0 +{: .label .label-purple } + +The Recovery API provides information about any completed or ongoing shard recoveries for one or more indexes. If a data stream is listed, the API returns information about that data stream's backing indexes. + +Shard recovery involves creating a shard copy to restore a primary shard from a snapshot or to synchronize a replica shard. After the shard recovery process completes, the recovered shard becomes available for use in search and index operations. + +Shard recovery occurs automatically in the following scenarios: + +- Node startup, known as a local store recovery +- Replication of a primary shard +- Relocation of a shard to a different node within the same cluster +- Restoration of a snapshot +- Clone, shrink, or split operations + +The Recovery API reports solely on completed recoveries for shard copies presently stored in the cluster. It reports only the most recent recovery for each shard copy and does not include historical information about previous recoveries or information about recoveries of shard copies that no longer exist. Consequently, if a shard copy completes a recovery and is subsequently relocated to a different node, then the information about the original recovery is not displayed in the Recovery API. + + +## Path and HTTP methods + +```json +GET /_recovery +GET //recovery/ +``` + +## Path parameters + +Parameter | Data type | Description +:--- | :--- +`index-name` | String | A comma-separated list of indexes, data streams, or index aliases to which the operation is applied. Supports wildcard expressions (`*`). Use `_all` or `*` to specify all indexes and data streams in a cluster. | + + +## Query parameters + +All of the following query parameters are optional. + +Parameter | Data type | Description +:--- | :--- | :--- +`active_only` | Boolean | When `true`, the response only includes active shard recoveries. Default is `false`. +`detailed` | Boolean | When `true`, provides detailed information about shard recoveries. Default is `false`. +`index` | String | A comma-separated list or wildcard expression of index names used to limit the request. + +## Response fields + +The API responds with the following information about the recovery shard. + +Parameter | Data type | Description +:--- | :--- | :--- +`id` | Integer | The ID of the shard. +`type` | String | The recovery source for the shard. Returned values include:
- `EMPTY_STORE`: An empty store. Indicates a new primary shard or the forced allocation of an empty primary shard using the Cluster Reroute API.
- `EXISTING_STORE`: The store of an existing primary shard. Indicates that the recovery is related to node startup or the allocation of an existing primary shard.
- `LOCAL_SHARDS`: Shards belonging to another index on the same node. Indicates that the recovery is related to a clone, shrink, or split operation.
- `PEER`: A primary shard on another node. Indicates that the recovery is related to shard replication.
- `SNAPSHOT`: A snapshot. Indicates that the recovery is related to a snapshot restore operation. +`STAGE` | String | The recovery stage. Returned values can include:
- `INIT`: Recovery has not started.
- `INDEX`: Reading index metadata and copying bytes from the source to the destination.
- `VERIFY_INDEX`: Verifying the integrity of the index.
- `TRANSLOG`: Replaying the transaction log.
- `FINALIZE`: Cleanup.
- `DONE`: Complete. +`primary` | Boolean | When `true`, the shard is a primary shard. +`start_time` | String | The timestamp indicating when the recovery started. +`stop_time` | String | The timestamp indicating when the recovery completed. +`total_time_in_millis` | String | The total amount of time taken to recover a shard, in milliseconds. +`source` | Object | The recovery source. This can include a description of the repository (if the recovery is from a snapshot) or a description of the source node. +`target` | Object | The destination node. +`index` | Object | Statistics about the physical index recovery. +`translog` | Object | Statistics about the translog recovery. + `start` | Object | Statistics about the amount of time taken to open and start the index. + +## Example requests + +The following examples demonstrate how to recover information using the Recovery API. + +### Recover information from several or all indexes + +The following example request returns recovery information about several indexes in a [human-readable format](https://opensearch.org/docs/latest/api-reference/common-parameters/#human-readable-output): + +```json +GET index1,index2/_recovery?human +``` +{% include copy-curl.html %} + +The following example request returns recovery information about all indexes in a human-readable format: + +```json +GET /_recovery?human +``` +{% include copy-curl.html %} + +### Recover detailed information + +The following example request returns detailed recovery information: + +```json +GET _recovery?human&detailed=true +``` +{% include copy-curl.html %} + +## Example response + +The following response returns detailed recovery information about an index named `shakespeare`: + +```json +{ + "shakespeare": { + "shards": [ + { + "id": 0, + "type": "EXISTING_STORE", + "stage": "DONE", + "primary": true, + "start_time": "2024-07-01T18:06:47.415Z", + "start_time_in_millis": 1719857207415, + "stop_time": "2024-07-01T18:06:47.538Z", + "stop_time_in_millis": 1719857207538, + "total_time": "123ms", + "total_time_in_millis": 123, + "source": { + "bootstrap_new_history_uuid": false + }, + "target": { + "id": "uerS7REgRQCbBF3ImY8wOQ", + "host": "172.18.0.3", + "transport_address": "172.18.0.3:9300", + "ip": "172.18.0.3", + "name": "opensearch-node2" + }, + "index": { + "size": { + "total": "17.8mb", + "total_in_bytes": 18708764, + "reused": "17.8mb", + "reused_in_bytes": 18708764, + "recovered": "0b", + "recovered_in_bytes": 0, + "percent": "100.0%" + }, + "files": { + "total": 7, + "reused": 7, + "recovered": 0, + "percent": "100.0%", + "details": [ + { + "name": "_1.cfs", + "length": "9.8mb", + "length_in_bytes": 10325945, + "reused": true, + "recovered": "0b", + "recovered_in_bytes": 0 + }, + { + "name": "_0.cfe", + "length": "479b", + "length_in_bytes": 479, + "reused": true, + "recovered": "0b", + "recovered_in_bytes": 0 + }, + { + "name": "_0.si", + "length": "333b", + "length_in_bytes": 333, + "reused": true, + "recovered": "0b", + "recovered_in_bytes": 0 + }, + { + "name": "_1.cfe", + "length": "479b", + "length_in_bytes": 479, + "reused": true, + "recovered": "0b", + "recovered_in_bytes": 0 + }, + { + "name": "_1.si", + "length": "333b", + "length_in_bytes": 333, + "reused": true, + "recovered": "0b", + "recovered_in_bytes": 0 + }, + { + "name": "_0.cfs", + "length": "7.9mb", + "length_in_bytes": 8380790, + "reused": true, + "recovered": "0b", + "recovered_in_bytes": 0 + }, + { + "name": "segments_3", + "length": "405b", + "length_in_bytes": 405, + "reused": true, + "recovered": "0b", + "recovered_in_bytes": 0 + } + ] + }, + "total_time": "6ms", + "total_time_in_millis": 6, + "source_throttle_time": "-1", + "source_throttle_time_in_millis": 0, + "target_throttle_time": "-1", + "target_throttle_time_in_millis": 0 + }, + "translog": { + "recovered": 0, + "total": 0, + "percent": "100.0%", + "total_on_start": 0, + "total_time": "113ms", + "total_time_in_millis": 113 + }, + "verify_index": { + "check_index_time": "0s", + "check_index_time_in_millis": 0, + "total_time": "0s", + "total_time_in_millis": 0 + } + }, + { + "id": 0, + "type": "PEER", + "stage": "DONE", + "primary": false, + "start_time": "2024-07-01T18:06:47.693Z", + "start_time_in_millis": 1719857207693, + "stop_time": "2024-07-01T18:06:47.744Z", + "stop_time_in_millis": 1719857207744, + "total_time": "50ms", + "total_time_in_millis": 50, + "source": { + "id": "uerS7REgRQCbBF3ImY8wOQ", + "host": "172.18.0.3", + "transport_address": "172.18.0.3:9300", + "ip": "172.18.0.3", + "name": "opensearch-node2" + }, + "target": { + "id": "HFYKietmTO6Ud9COgP0k9Q", + "host": "172.18.0.2", + "transport_address": "172.18.0.2:9300", + "ip": "172.18.0.2", + "name": "opensearch-node1" + }, + "index": { + "size": { + "total": "0b", + "total_in_bytes": 0, + "reused": "0b", + "reused_in_bytes": 0, + "recovered": "0b", + "recovered_in_bytes": 0, + "percent": "0.0%" + }, + "files": { + "total": 0, + "reused": 0, + "recovered": 0, + "percent": "0.0%", + "details": [] + }, + "total_time": "1ms", + "total_time_in_millis": 1, + "source_throttle_time": "-1", + "source_throttle_time_in_millis": 0, + "target_throttle_time": "-1", + "target_throttle_time_in_millis": 0 + }, + "translog": { + "recovered": 0, + "total": 0, + "percent": "100.0%", + "total_on_start": -1, + "total_time": "42ms", + "total_time_in_millis": 42 + }, + "verify_index": { + "check_index_time": "0s", + "check_index_time_in_millis": 0, + "total_time": "0s", + "total_time_in_millis": 0 + } + } + ] + } +} +``` From 506b10121166e145a742368c4f12535451c72569 Mon Sep 17 00:00:00 2001 From: AntonEliatra Date: Wed, 24 Jul 2024 17:45:59 +0100 Subject: [PATCH 30/36] Additing error details and escape method to nodes_dn #7681 (#7703) * Additing error details and escape method to nodes_dn #7681 Signed-off-by: AntonEliatra * Update security-settings.md Signed-off-by: AntonEliatra * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: AntonEliatra Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --- .../configuring-opensearch/security-settings.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/_install-and-configure/configuring-opensearch/security-settings.md b/_install-and-configure/configuring-opensearch/security-settings.md index 244d601449..ffdad36cb3 100644 --- a/_install-and-configure/configuring-opensearch/security-settings.md +++ b/_install-and-configure/configuring-opensearch/security-settings.md @@ -15,7 +15,7 @@ The following sections describe security-related settings in `opensearch.yml`. T The Security plugin supports the following common settings: -- `plugins.security.nodes_dn` (Static): Specifies a list of distinguished names (DNs) that denote the other nodes in the cluster. This setting supports wildcards and regular expressions. The list of DNs are also read from the security index **in addition** to the YAML configuration when `plugins.security.nodes_dn_dynamic_config_enabled` is `true`. +- `plugins.security.nodes_dn` (Static): Specifies a list of distinguished names (DNs) that denote the other nodes in the cluster. This setting supports wildcards and regular expressions. The list of DNs are also read from the security index **in addition** to the YAML configuration when `plugins.security.nodes_dn_dynamic_config_enabled` is `true`. If this setting is not configured correctly, the cluster will fail to form as the nodes will not be able to trust each other and will result in the following error: `Transport client authentication no longer supported`. - `plugins.security.nodes_dn_dynamic_config_enabled` (Static): Relevant for `cross_cluster` use cases where there is a need to manage the allow listed `nodes_dn` without having to restart the nodes every time a new `cross_cluster` remote is configured. Setting `nodes_dn_dynamic_config_enabled` to `true` enables **super-admin callable** Distinguished Names APIs, which provide means to update or retrieve `nodes_dn` dynamically. This setting only has effect if `plugins.security.cert.intercluster_request_evaluator_class` is not set. Default is `false`. @@ -357,6 +357,7 @@ The Security plugin supports the following transport layer security settings: plugins.security.nodes_dn: - "CN=*.example.com, OU=SSL, O=Test, L=Test, C=DE" - "CN=node.other.com, OU=SSL, O=Test, L=Test, C=DE" + - "CN=node.example.com, OU=SSL\, Inc., L=Test, C=DE" # escape additional comma with `\` plugins.security.authcz.admin_dn: - CN=kirk,OU=client,O=client,L=test, C=de plugins.security.roles_mapping_resolution: MAPPING_ONLY From d36fcd376aaf0a253f828b2c3a88ddb8f6c19843 Mon Sep 17 00:00:00 2001 From: leanneeliatra <131779422+leanneeliatra@users.noreply.github.com> Date: Wed, 24 Jul 2024 17:58:39 +0100 Subject: [PATCH 31/36] 20240417 Adding OpenSearch demo configuration mac instructions (#7381) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * adding OpenSearch demo configuration mac instructions Signed-off-by: leanne.laceybyrne@eliatra.com * [MDS] Add security analytics, alerting, feature anaywhere in the multiple data source document (#7328) * Add security analy Signed-off-by: yujin-emma * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update multi-data-sources.md Signed-off-by: Yu Jin <112784385+yujin-emma@users.noreply.github.com> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update multi-data-sources.md Signed-off-by: Yu Jin <112784385+yujin-emma@users.noreply.github.com> * Update _dashboards/management/multi-data-sources.md Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Melissa Vagi Signed-off-by: Yu Jin <112784385+yujin-emma@users.noreply.github.com> * Update multi-data-sources.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Signed-off-by: Melissa Vagi * Update multi-data-sources.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Signed-off-by: Melissa Vagi --------- Signed-off-by: yujin-emma Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: Yu Jin <112784385+yujin-emma@users.noreply.github.com> Signed-off-by: Melissa Vagi Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Melissa Vagi Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Added documentation for managed identity support in repository-azure plugin (#7068) * Added documentation for managed identity support in repository-azure plugins Signed-off-by: Chengwu Shi * fixed syntax Signed-off-by: Chengwu Shi * fixed style error Signed-off-by: Chengwu Shi * remove sudo, and added 1 more point when configuring key or sas token Signed-off-by: Chengwu Shi * Update section Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update snapshot-restore.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * improve readability and clarity Signed-off-by: Chengwu Shi * improved naming Signed-off-by: Chengwu Shi * Update _tuning-your-cluster/availability-and-recovery/snapshots/snapshot-restore.md Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: chengwushi-netapp <153049940+chengwushi-netapp@users.noreply.github.com> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update snapshot-restore.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * fixed syntax based on requested changes Signed-off-by: Chengwu Shi * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: Chengwu Shi Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: chengwushi-netapp <153049940+chengwushi-netapp@users.noreply.github.com> Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Updating the documentation for `obfuscate` processor and fixing a issue in `opensearch` sink documentation. (#7251) * Updating the documentation for processor to accomodate change in DataPrepper 2.8 and fixing a issue in sink documentation. Signed-off-by: Utkarsh Agarwal * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: Utkarsh Agarwal Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Utkarsh Agarwal Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * Added documentation for new default workflow templates (#7346) * add new default workflow templates Signed-off-by: Amit Galitzky * Apply suggestions from code review Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: Amit Galitzky Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Remote state publication (#7364) * Remote state publication Signed-off-by: Sooraj Sinha * Address comments Signed-off-by: Sooraj Sinha * Simplify the remote publication description Signed-off-by: Sooraj Sinha * Update remote-cluster-state.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update _tuning-your-cluster/availability-and-recovery/remote-store/remote-cluster-state.md Co-authored-by: Nathan Bower Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: Sooraj Sinha Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Add documentation of derived fields (#7329) * Add documentation of derived fields Signed-off-by: Rishabh Maurya * improve the documentation Signed-off-by: Rishabh Maurya * Fix style-job comments Signed-off-by: Rishabh Maurya * documentation for object derived field Signed-off-by: Rishabh Maurya * Update one of the search request and date format documentation Signed-off-by: Rishabh Maurya * Doc review Signed-off-by: Fanit Kolchina * Tech review comments Signed-off-by: Fanit Kolchina * One more passive voice Signed-off-by: Fanit Kolchina * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _field-types/supported-field-types/derived.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _field-types/supported-field-types/derived.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _field-types/supported-field-types/derived.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * More editorial comments Signed-off-by: Fanit Kolchina --------- Signed-off-by: Rishabh Maurya Signed-off-by: Fanit Kolchina Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Fanit Kolchina Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Add doc for neural-sparse-query-two-phase-processor. (#7306) * Add doc for neural-sparse-query-two-phase-processor. Signed-off-by: conggguan * Make some edits for the comments. Signed-off-by: conggguan * Fix some typo and style-job. Signed-off-by: conggguan * Update neural-sparse-query-two-phase-processor.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update _search-plugins/search-pipelines/neural-sparse-query-two-phase-processor.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: conggguan Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Add documentation related to removal of source and recovery source in k-NN performance tuning section (#7359) * Add documentation related to removal of source and recovery source in k-NN performance tuning section Signed-off-by: Navneet Verma * Update formatting. Add Doc review. Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: Navneet Verma Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Update derived.md (#7393) Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * Update common-filters.md (#7386) Community member pointed out the error in the remove_field field. Signed-off-by: Heather Halter Signed-off-by: leanne.laceybyrne@eliatra.com * Documentation for Indices Request Cache Overview and its settings (#7288) * Create doc for Indices Request Cache Signed-off-by: Kiran Prakash * sentence case Signed-off-by: Kiran Prakash * sentence case Signed-off-by: Kiran Prakash * Create doc for Indices Request Cache Signed-off-by: Kiran Prakash * sentence case Signed-off-by: Kiran Prakash * sentence case Signed-off-by: Kiran Prakash * Doc review Signed-off-by: Fanit Kolchina * Corrects dynamic/static settings Signed-off-by: Fanit Kolchina * Move one of the settings to static Signed-off-by: Fanit Kolchina * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: Kiran Prakash Signed-off-by: Fanit Kolchina Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Fanit Kolchina Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * 20240131 Clarification on backend role concept in access control (#7378) * clarification on backend role concept in access control Signed-off-by: leanne.laceybyrne@eliatra.com * taking out unnesesary changes Signed-off-by: leanne.laceybyrne@eliatra.com * Update _security/access-control/index.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: leanne.laceybyrne@eliatra.com Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * Changed order of Best Practices page and linked it from other relevant pages (#7389) * changed order of best practices page and linked it from other relevant pages Signed-off-by: leanne.laceybyrne@eliatra.com * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: leanne.laceybyrne@eliatra.com Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * AD Enhancements in Version 2.15 (#7388) * Enhancements in Version 2.15 Starting from version 2.15, we have introduced several enhancements: 1. Custom Index Management: * Added support for custom index management. For more details, watch the video on the create detector page and detector detail page anomaly-detection-dashboards-plugin#770. * Custom result indices are now managed as aliases. Consequently, additional security permissions are required for management. 2. New JVM Heap Usage Threshold Setting: * Introduced a new setting, plugins.anomaly_detection.jvm_heap_usage_threshold, to manage the memory circuit breaker threshold. 3. Documentation Improvements: * Added examples for DSL filters to enhance the documentation. 4. Ruby Version Update: * Updated Ruby version in CONTRIBUTING.md from 3.2 to 3.2.4 as version 3.2 is not available. Testing done: * built and viewed the changed website locally Signed-off-by: Kaituo Li * Update _observing-your-data/ad/index.md Co-authored-by: Melissa Vagi Signed-off-by: Kaituo Li * Update _observing-your-data/ad/index.md Co-authored-by: Melissa Vagi Signed-off-by: Kaituo Li * Update _observing-your-data/ad/index.md Co-authored-by: Melissa Vagi Signed-off-by: Kaituo Li * Update _observing-your-data/ad/index.md Co-authored-by: Melissa Vagi Signed-off-by: Kaituo Li * Update _observing-your-data/ad/index.md Co-authored-by: Melissa Vagi Signed-off-by: Kaituo Li * Update _observing-your-data/ad/index.md Co-authored-by: Melissa Vagi Signed-off-by: Kaituo Li * Update _observing-your-data/ad/index.md Co-authored-by: Melissa Vagi Signed-off-by: Kaituo Li * Update _observing-your-data/ad/index.md Co-authored-by: Melissa Vagi Signed-off-by: Kaituo Li * Update _observing-your-data/ad/index.md Co-authored-by: Melissa Vagi Signed-off-by: Kaituo Li * Update _observing-your-data/ad/index.md Co-authored-by: Melissa Vagi Signed-off-by: Kaituo Li * Update _observing-your-data/ad/index.md Co-authored-by: Melissa Vagi Signed-off-by: Kaituo Li * Update _observing-your-data/ad/settings.md Co-authored-by: Melissa Vagi Signed-off-by: Kaituo Li * Update _observing-your-data/ad/settings.md Co-authored-by: Melissa Vagi Signed-off-by: Kaituo Li * Update _observing-your-data/ad/index.md Signed-off-by: Melissa Vagi * Update _observing-your-data/ad/index.md Signed-off-by: Melissa Vagi * Update index.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update _observing-your-data/ad/settings.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi --------- Signed-off-by: Kaituo Li Signed-off-by: Melissa Vagi Co-authored-by: Melissa Vagi Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Add documentation for the new setting of cardinality aggregation dynamic pruning (#7341) * add documentation for the new setting of cardinality aggregation dynamic pruning Signed-off-by: bowenlan-amzn * Update search-settings.md Signed-off-by: bowenlan-amzn * Update _install-and-configure/configuring-opensearch/search-settings.md Signed-off-by: Melissa Vagi * Update search-settings.md Signed-off-by: Melissa Vagi --------- Signed-off-by: bowenlan-amzn Signed-off-by: Melissa Vagi Co-authored-by: Melissa Vagi Signed-off-by: leanne.laceybyrne@eliatra.com * Change introduced version for derived field (#7403) Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * Add documentation for wildcard field type (#7339) * Add documentation for wildcard field type Signed-off-by: Michael Froh * Doc review Signed-off-by: Fanit Kolchina * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * More explanation Signed-off-by: Fanit Kolchina * Update _field-types/supported-field-types/wildcard.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: Michael Froh Signed-off-by: Fanit Kolchina Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Fanit Kolchina Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Add document for top n queries improvements in 2.15 (#7326) * Add document for top n queries by cpu and memory Signed-off-by: Chenyang Ji * update document to fix style checks Signed-off-by: Chenyang Ji * combine exporter and metrics enhancements documents into one PR Signed-off-by: Chenyang Ji * Doc review Signed-off-by: Fanit Kolchina * add document for default exporter pattern Signed-off-by: Chenyang Ji * Consolidating metric types Signed-off-by: Fanit Kolchina * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: Chenyang Ji Signed-off-by: Fanit Kolchina Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Fanit Kolchina Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Add documentation for innerHit on knn nested field (#7404) * Add documentation for innerHit on knn nested field Signed-off-by: Heemin Kim * Doc review Signed-off-by: Fanit Kolchina * Explain excluding source Signed-off-by: Fanit Kolchina --------- Signed-off-by: Heemin Kim Signed-off-by: Fanit Kolchina Co-authored-by: Fanit Kolchina Signed-off-by: leanne.laceybyrne@eliatra.com * Mark docrep to remote migration as GA and modify settings names (#7342) * Mark docrep to remote migration as GA and modify settings names Signed-off-by: Gaurav Bafna * Update _tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md Co-authored-by: Bhumika Saini Signed-off-by: Gaurav Bafna <85113518+gbbafna@users.noreply.github.com> * Update migrating-to-remote.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update _tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _tuning-your-cluster/availability-and-recovery/remote-store/migrating-to-remote.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi --------- Signed-off-by: Gaurav Bafna Signed-off-by: Gaurav Bafna <85113518+gbbafna@users.noreply.github.com> Signed-off-by: Melissa Vagi Co-authored-by: Bhumika Saini Co-authored-by: Melissa Vagi Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Changed VisBuilder status from experimental to GA (#7405) * VisBuilder GA Signed-off-by: Fanit Kolchina * Removed experimental visualization requirement Signed-off-by: Fanit Kolchina * Update screenshots Signed-off-by: Fanit Kolchina --------- Signed-off-by: Fanit Kolchina Signed-off-by: leanne.laceybyrne@eliatra.com * Explicitly add a version question to the doc issue template (#7406) * Separate out the version section in the doc issue Signed-off-by: Fanit Kolchina * typo Signed-off-by: Fanit Kolchina --------- Signed-off-by: Fanit Kolchina Signed-off-by: leanne.laceybyrne@eliatra.com * Add doc for alerting comments (#7360) * alerting comments first documentation rough draft Signed-off-by: Dennis Toepker * formatting and styling changes Signed-off-by: Dennis Toepker * more formatting changes Signed-off-by: Dennis Toepker * more spacing changes Signed-off-by: Dennis Toepker * added not about rbac and added some links Signed-off-by: Dennis Toepker * removing comments history enabled setting Signed-off-by: Dennis Toepker * Update comments.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update _observing-your-data/alerting/api.md Update Update comment API description Co-authored-by: Melissa Vagi Signed-off-by: toepkerd <120457569+toepkerd@users.noreply.github.com> * Update _observing-your-data/alerting/api.md Update Delete comment API description Co-authored-by: Melissa Vagi Signed-off-by: toepkerd <120457569+toepkerd@users.noreply.github.com> * Update _observing-your-data/alerting/settings.md Updated comments_enabled setting description Co-authored-by: Melissa Vagi Signed-off-by: toepkerd <120457569+toepkerd@users.noreply.github.com> * Update _observing-your-data/alerting/settings.md Updated comments_history_max_age setting description Co-authored-by: Melissa Vagi Signed-off-by: toepkerd <120457569+toepkerd@users.noreply.github.com> * Update _observing-your-data/alerting/settings.md Update comments_history_rollover_period setting description Co-authored-by: Melissa Vagi Signed-off-by: toepkerd <120457569+toepkerd@users.noreply.github.com> * Update _observing-your-data/alerting/settings.md Update comments_history_retention_period setting description Co-authored-by: Melissa Vagi Signed-off-by: toepkerd <120457569+toepkerd@users.noreply.github.com> * minor edits and dead link fix attempt Signed-off-by: Dennis Toepker * misc edits Signed-off-by: Dennis Toepker * minor edit Signed-off-by: Dennis Toepker * Update api.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update _observing-your-data/alerting/comments.md Signed-off-by: Melissa Vagi * Update api.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update comments.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update api.md format request and response examples to expand Signed-off-by: Melissa Vagi * Update api.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update api.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update api.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update api.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update api.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update _observing-your-data/alerting/api.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/alerting/api.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/alerting/api.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/alerting/api.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/alerting/api.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/alerting/settings.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/alerting/settings.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/alerting/settings.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/alerting/comments.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/alerting/settings.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/alerting/comments.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/alerting/comments.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/alerting/comments.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update api.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update settings.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update api.md Signed-off-by: Melissa Vagi Added text to address stacked headings issue Signed-off-by: Melissa Vagi * Update api.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update _observing-your-data/alerting/api.md Signed-off-by: Melissa Vagi * Update api.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update _observing-your-data/alerting/api.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Fix broken link Signed-off-by: Melissa Vagi * Fix broken link Signed-off-by: Melissa Vagi --------- Signed-off-by: Dennis Toepker Signed-off-by: Melissa Vagi Signed-off-by: toepkerd <120457569+toepkerd@users.noreply.github.com> Co-authored-by: Dennis Toepker Co-authored-by: Melissa Vagi Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Update the integration page to reflect new integration catalog features (#7324) * update the integration page to reflect the integration catalog and additional capabilities Signed-off-by: YANGDB * update the integration documentation Signed-off-by: YANGDB * Update schema section Signed-off-by: Simeon Widdis * update the metrics analytics documentation Signed-off-by: YANGDB * update the trace analytics documentation Signed-off-by: YANGDB * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update index.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update _integrations/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _integrations/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _integrations/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _integrations/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _integrations/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _integrations/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _integrations/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _integrations/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _integrations/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _integrations/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _integrations/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _integrations/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _integrations/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _integrations/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _integrations/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _integrations/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _integrations/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _integrations/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _integrations/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _integrations/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _integrations/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _integrations/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update index.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update index.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update index.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update index.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update index.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update index.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Rewrite tutorials and update or delete graphics Signed-off-by: Melissa Vagi * Update _integrations/index.md Co-authored-by: Nathan Bower Signed-off-by: YANGDB * Update _integrations/index.md Co-authored-by: Nathan Bower Signed-off-by: YANGDB * Update _integrations/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _integrations/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi * Update _integrations/index.md Signed-off-by: Melissa Vagi --------- Signed-off-by: YANGDB Signed-off-by: Simeon Widdis Signed-off-by: Melissa Vagi Co-authored-by: Simeon Widdis Co-authored-by: Melissa Vagi Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Add connector tool (#7384) * add connector tool Signed-off-by: Yaliang Wu * address comments Signed-off-by: Yaliang Wu * Doc review Signed-off-by: Fanit Kolchina * Update _ml-commons-plugin/agents-tools/tools/connector-tool.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _ml-commons-plugin/agents-tools/tools/connector-tool.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _ml-commons-plugin/agents-tools/tools/connector-tool.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _ml-commons-plugin/agents-tools/tools/connector-tool.md Co-authored-by: Yaliang Wu Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _ml-commons-plugin/agents-tools/tools/connector-tool.md Co-authored-by: Yaliang Wu Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _ml-commons-plugin/remote-models/blueprints.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: Yaliang Wu Signed-off-by: Fanit Kolchina Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Fanit Kolchina Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Alerts in correlations feature documentation (#7410) * alerts in correlations feature documentation Signed-off-by: Riya Saxena * Update correlation-eng.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update alerts.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Copy edit Signed-off-by: Melissa Vagi * Update _security-analytics/api-tools/correlation-eng.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _security-analytics/usage/alerts.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _security-analytics/usage/alerts.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _security-analytics/usage/alerts.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _security-analytics/usage/alerts.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _security-analytics/usage/alerts.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _security-analytics/usage/alerts.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _security-analytics/usage/alerts.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _security-analytics/usage/alerts.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _security-analytics/usage/alerts.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _security-analytics/usage/alerts.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _security-analytics/usage/alerts.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _security-analytics/usage/alerts.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _security-analytics/usage/alerts.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _security-analytics/usage/alerts.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _security-analytics/usage/alerts.md Signed-off-by: Melissa Vagi * Update _security-analytics/usage/alerts.md Signed-off-by: Melissa Vagi * Update _security-analytics/usage/alerts.md Signed-off-by: Melissa Vagi --------- Signed-off-by: Riya Saxena Signed-off-by: Melissa Vagi Co-authored-by: Melissa Vagi Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Trace analytics update (#7362) * update the integration page to reflect the integration catalog and additional capabilities Signed-off-by: YANGDB * update the integration documentation Signed-off-by: YANGDB * Update schema section Signed-off-by: Simeon Widdis * update the metrics analytics documentation Signed-off-by: YANGDB * update the trace analytics documentation Signed-off-by: YANGDB * Update _observing-your-data/trace/ta-dashboards.md Signed-off-by: Melissa Vagi * Update _observing-your-data/trace/ta-dashboards.md Signed-off-by: Melissa Vagi * Update _observing-your-data/trace/ta-dashboards.md Signed-off-by: Melissa Vagi * Update _observing-your-data/trace/ta-dashboards.md Signed-off-by: Melissa Vagi * Update _observing-your-data/trace/ta-dashboards.md Signed-off-by: Melissa Vagi * Update _observing-your-data/trace/ta-dashboards.md Signed-off-by: Melissa Vagi * Update _observing-your-data/trace/ta-dashboards.md Signed-off-by: Melissa Vagi * Update _observing-your-data/trace/ta-dashboards.md Signed-off-by: Melissa Vagi * Update _observing-your-data/trace/ta-dashboards.md Signed-off-by: Melissa Vagi * Update _observing-your-data/trace/ta-dashboards.md Signed-off-by: Melissa Vagi * Update ta-dashboards.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update _observing-your-data/trace/ta-dashboards.md Signed-off-by: Melissa Vagi * Update _observing-your-data/trace/ta-dashboards.md Signed-off-by: Melissa Vagi * update service correlation index naming convention Signed-off-by: YANGDB * Update ta-dashboards.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update ta-dashboards.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update ta-dashboards.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update ta-dashboards.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update ta-dashboards.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update ta-dashboards.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update ta-dashboards.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update _observing-your-data/trace/ta-dashboards.md Signed-off-by: Melissa Vagi * Update _observing-your-data/trace/ta-dashboards.md Signed-off-by: Melissa Vagi * Update _observing-your-data/trace/ta-dashboards.md Co-authored-by: Nathan Bower Signed-off-by: YANGDB * Update _observing-your-data/trace/ta-dashboards.md Co-authored-by: Nathan Bower Signed-off-by: YANGDB * Update _observing-your-data/trace/ta-dashboards.md Co-authored-by: Nathan Bower Signed-off-by: YANGDB * Update _observing-your-data/trace/ta-dashboards.md Co-authored-by: Nathan Bower Signed-off-by: YANGDB * Update _observing-your-data/trace/ta-dashboards.md Co-authored-by: Nathan Bower Signed-off-by: YANGDB * Update _observing-your-data/trace/ta-dashboards.md Co-authored-by: Nathan Bower Signed-off-by: YANGDB * Update _observing-your-data/trace/ta-dashboards.md Co-authored-by: Nathan Bower Signed-off-by: YANGDB * Update _observing-your-data/trace/ta-dashboards.md Co-authored-by: Nathan Bower Signed-off-by: YANGDB * Update _observing-your-data/trace/ta-dashboards.md Co-authored-by: Nathan Bower Signed-off-by: YANGDB * Update _observing-your-data/trace/ta-dashboards.md Co-authored-by: Nathan Bower Signed-off-by: YANGDB * Update _observing-your-data/trace/ta-dashboards.md Co-authored-by: Nathan Bower Signed-off-by: YANGDB * Update _observing-your-data/trace/ta-dashboards.md Co-authored-by: Nathan Bower Signed-off-by: YANGDB * Update _observing-your-data/trace/ta-dashboards.md Co-authored-by: Nathan Bower Signed-off-by: YANGDB * Update _observing-your-data/trace/ta-dashboards.md Co-authored-by: Nathan Bower Signed-off-by: YANGDB * Update _observing-your-data/trace/ta-dashboards.md Co-authored-by: Nathan Bower Signed-off-by: YANGDB * Update _observing-your-data/trace/ta-dashboards.md Co-authored-by: Nathan Bower Signed-off-by: YANGDB * Update _observing-your-data/trace/ta-dashboards.md Co-authored-by: Nathan Bower Signed-off-by: YANGDB * Update _observing-your-data/trace/ta-dashboards.md Co-authored-by: Nathan Bower Signed-off-by: YANGDB * Update _observing-your-data/trace/ta-dashboards.md Co-authored-by: Nathan Bower Signed-off-by: YANGDB * Update _observing-your-data/trace/ta-dashboards.md Co-authored-by: Nathan Bower Signed-off-by: YANGDB * Update _observing-your-data/trace/ta-dashboards.md Co-authored-by: Nathan Bower Signed-off-by: YANGDB * Update _observing-your-data/trace/ta-dashboards.md Signed-off-by: Melissa Vagi * Update _observing-your-data/trace/ta-dashboards.md Signed-off-by: Melissa Vagi * Update _observing-your-data/trace/ta-dashboards.md Signed-off-by: Melissa Vagi * Update _observing-your-data/trace/ta-dashboards.md Signed-off-by: Melissa Vagi * Update _observing-your-data/trace/ta-dashboards.md Signed-off-by: Melissa Vagi * Update _observing-your-data/trace/ta-dashboards.md Signed-off-by: Melissa Vagi * Update _observing-your-data/trace/ta-dashboards.md Signed-off-by: Melissa Vagi * Update _observing-your-data/trace/ta-dashboards.md Signed-off-by: Melissa Vagi * Update _observing-your-data/trace/ta-dashboards.md Signed-off-by: Melissa Vagi * Update _observing-your-data/trace/ta-dashboards.md Signed-off-by: Melissa Vagi --------- Signed-off-by: YANGDB Signed-off-by: Simeon Widdis Signed-off-by: Melissa Vagi Co-authored-by: Simeon Widdis Co-authored-by: Melissa Vagi Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Add documentations for batch ingestion feature (#7408) * Add documents for batch ingestion Signed-off-by: Liyun Xiu * Revert change on other rows of bulk Signed-off-by: Liyun Xiu * Apply suggestions from code review Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _ml-commons-plugin/remote-models/batch-ingestion.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Apply suggestions from code review Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: Liyun Xiu Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Moved batch-enabled processors to a separate section (#7415) Signed-off-by: Fanit Kolchina Signed-off-by: leanne.laceybyrne@eliatra.com * Add link to tutorial in OS Dashboards Assistant documentation (#7416) Signed-off-by: Fanit Kolchina Signed-off-by: leanne.laceybyrne@eliatra.com * Point to the spaces section of approximate k-NN from radial search (#7417) Signed-off-by: Fanit Kolchina Signed-off-by: leanne.laceybyrne@eliatra.com * Update documentation of ml inference processors to support for local models (#7368) * ml inference processor support for local models Signed-off-by: Bhavana Ramaram * address comments Signed-off-by: Bhavana Ramaram * Doc review Signed-off-by: Fanit Kolchina * tech review comments Signed-off-by: Fanit Kolchina * Update _ingest-pipelines/processors/ml-inference.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: Bhavana Ramaram Signed-off-by: Fanit Kolchina Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Fanit Kolchina Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Remove a link that caused link checker fail and update link checker (#7418) Signed-off-by: Fanit Kolchina Signed-off-by: leanne.laceybyrne@eliatra.com * Add remote guardrails model support (#7377) * remote guardrails model support Signed-off-by: Jing Zhang * address comments Signed-off-by: Jing Zhang * Add guardrail information Signed-off-by: Fanit Kolchina * Rename response accept parameter Signed-off-by: Fanit Kolchina * typo fix Signed-off-by: Fanit Kolchina * Fix links Signed-off-by: Fanit Kolchina * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _ml-commons-plugin/remote-models/guardrails.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: Jing Zhang Signed-off-by: Fanit Kolchina Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Fanit Kolchina Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Update the metrics analytics documentation (#7352) * update the metrics analytics documentation Signed-off-by: YANGDB * update the metrics analytics documentation with remote data-sources Signed-off-by: YANGDB * Update metricsanalytics.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update metricsanalytics.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update metricsanalytics.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Doc review Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi * Update _observing-your-data/metricsanalytics.md Signed-off-by: Melissa Vagi --------- Signed-off-by: YANGDB Signed-off-by: Melissa Vagi Co-authored-by: Melissa Vagi Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Update key value processor documentation with new config options (#7413) * Update key value processor documentation with new config options Signed-off-by: Kondaka * Update _data-prepper/pipelines/configuration/processors/key-value.md Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/processors/key-value.md Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/processors/key-value.md Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/processors/key-value.md Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/processors/key-value.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/processors/key-value.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi --------- Signed-off-by: Kondaka Signed-off-by: Melissa Vagi Co-authored-by: Melissa Vagi Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Add version to the warning headers (#7459) Signed-off-by: Fanit Kolchina Signed-off-by: leanne.laceybyrne@eliatra.com * Adds the max_request_length documentation for appropriate Data Prepper sources (#7441) * Adds the max_request_length documentation. Also, updates and corrects the http source documentation to use the correct name. Signed-off-by: David Venable * Update _data-prepper/pipelines/configuration/sources/http.md Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sources/otel-metrics-source.md Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sources/otel-trace-source.md Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sources/http.md Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sources/http.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sources/otel-logs-source.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sources/otel-metrics-source.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sources/otel-trace-source.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi --------- Signed-off-by: David Venable Signed-off-by: Melissa Vagi Co-authored-by: Melissa Vagi Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * adding explanation to jwks validation #7367 #20240522 (#7398) * adding explanation to jwks validation #7367 #20240522 Signed-off-by: AntonEliatra * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: AntonEliatra Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * 20240322 Adding absolute path for config.yml (#7380) * adding absolute path Signed-off-by: leanne.laceybyrne@eliatra.com * adding clarification for mac Signed-off-by: leanne.laceybyrne@eliatra.com * changes commited on incorrect branch - undoing Signed-off-by: leanne.laceybyrne@eliatra.com * Update _security/configuration/configuration.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: leanne.laceybyrne@eliatra.com Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Heather Halter Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * Rename metricsanalytics to prometheusmetrics (#7482) Signed-off-by: Fanit Kolchina Co-authored-by: Melissa Vagi Signed-off-by: leanne.laceybyrne@eliatra.com * Add a video requirement to a PR template (#7488) * Add a video requirement to a PR template Signed-off-by: Fanit Kolchina * Update .github/PULL_REQUEST_TEMPLATE.md Co-authored-by: Melissa Vagi Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: Fanit Kolchina Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Melissa Vagi Signed-off-by: leanne.laceybyrne@eliatra.com * Add 2.15 version (#7483) Signed-off-by: Fanit Kolchina Signed-off-by: leanne.laceybyrne@eliatra.com * Add 2.15 to version history (#7484) Signed-off-by: Fanit Kolchina Signed-off-by: leanne.laceybyrne@eliatra.com * Add release notes 2.15 (#7486) Signed-off-by: Fanit Kolchina Signed-off-by: leanne.laceybyrne@eliatra.com * Add new s3 sink documentation for Data Prepper 2.8 (#7163) * Add new s3 sink documentation for Data Prepper 2.8 Signed-off-by: Taylor Gray * Apply suggestions from code review Co-authored-by: Melissa Vagi Signed-off-by: Taylor Gray * Update s3.md Clean up formatting. Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Signed-off-by: Melissa Vagi * Update s3.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/pipelines/configuration/sinks/s3.md Signed-off-by: Melissa Vagi --------- Signed-off-by: Taylor Gray Signed-off-by: Taylor Gray Signed-off-by: Melissa Vagi Co-authored-by: Melissa Vagi Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Adding information running script in OpenSearch (#7397) * adding one liner to explain script can be defined with source or id Signed-off-by: leanne.laceybyrne@eliatra.com * Update _api-reference/document-apis/bulk.md Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: leanneeliatra <131779422+leanneeliatra@users.noreply.github.com> --------- Signed-off-by: leanne.laceybyrne@eliatra.com Signed-off-by: leanneeliatra <131779422+leanneeliatra@users.noreply.github.com> Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * Fix monitoring predict requests in ML settings (#7512) Signed-off-by: Fanit Kolchina Signed-off-by: leanne.laceybyrne@eliatra.com * Add oidc docker example with keycloak #1566 (#7372) * adding oidc docker example with keycloak #1566 Signed-off-by: AntonEliatra * Update openid-connect.md Signed-off-by: AntonEliatra * Apply suggestions from code review Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: AntonEliatra * Update openid-connect.md Signed-off-by: AntonEliatra * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: AntonEliatra Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Add General Guidelines for concurrent segment search (#7402) * Add General Guidelines for concurrent segment search Signed-off-by: Jay Deng * Apply suggestions from code review Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: Jay Deng Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * Fixes Update compare-search-results.md (#7534) Has parent = true; updated to false so it doesn't look like it has children in the nav pane Signed-off-by: Heather Halter Signed-off-by: leanne.laceybyrne@eliatra.com * Drop repeated key/word in inverted index (#7538) "the" appears twice as a key in the inverted index; keys should only appear once. The correct key-value pair is kept, with a value of "1, 2" as "the" appears in both documents. Signed-off-by: Don Fox Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * Remove Vale from date processor (#7556) Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * Add selective download feature to Data Prepper sources section (#6247) * Add feature to this section Signed-off-by: Melissa Vagi * add content Signed-off-by: Melissa Vagi * Copy edits Signed-off-by: Melissa Vagi * Update selective-download.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Address tech review comments Signed-off-by: Melissa Vagi * Address tech review comments Signed-off-by: Melissa Vagi * Update _data-prepper/common-use-cases/s3-logs.md Co-authored-by: David Venable Signed-off-by: Melissa Vagi * Update _data-prepper/common-use-cases/s3-logs.md Co-authored-by: David Venable Signed-off-by: Melissa Vagi * Update _data-prepper/common-use-cases/s3-logs.md Co-authored-by: David Venable Signed-off-by: Melissa Vagi * Update _data-prepper/common-use-cases/s3-logs.md Co-authored-by: David Venable Signed-off-by: Melissa Vagi * Update s3-logs.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update s3-logs.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update s3-logs.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update _data-prepper/common-use-cases/s3-logs.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/common-use-cases/s3-logs.md Signed-off-by: Melissa Vagi * Update _data-prepper/common-use-cases/s3-logs.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/common-use-cases/s3-logs.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/common-use-cases/s3-logs.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/common-use-cases/s3-logs.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/common-use-cases/s3-logs.md Signed-off-by: Melissa Vagi --------- Signed-off-by: Melissa Vagi Co-authored-by: David Venable Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Update prometheusmetrics.md (#7561) Fixed a broken image link on https://opensearch.org/docs/latest/observing-your-data/prometheusmetrics/ Signed-off-by: Heather Halter Signed-off-by: leanne.laceybyrne@eliatra.com * Update Benchmark index to reflect TOC (#7560) * Update Benchmark index to reflect TOC Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Heather Halter Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Heather Halter Signed-off-by: leanne.laceybyrne@eliatra.com * Fix broken link to wildcard field type in index.md (#7565) Link to wildcard field type documentation is incorrect Signed-off-by: Thomas Wing Signed-off-by: leanne.laceybyrne@eliatra.com * Add Render Template API (#7219) * Add render/template API Signed-off-by: Archer * Add examples Signed-off-by: Archer * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Heather Halter Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Add example from `source`. Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: Archer Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Heather Halter Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Update docs for new clause count setting (#7391) * Update docs for new clause count setting Signed-off-by: Harsha Vamsi Kalluri * Update to 2.16 Signed-off-by: Harsha Vamsi Kalluri * Update _install-and-configure/configuring-opensearch/index-settings.md Signed-off-by: Melissa Vagi * Update _install-and-configure/configuring-opensearch/index-settings.md Signed-off-by: Melissa Vagi * Update _install-and-configure/configuring-opensearch/index-settings.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _install-and-configure/configuring-opensearch/index-settings.md Signed-off-by: Melissa Vagi --------- Signed-off-by: Harsha Vamsi Kalluri Signed-off-by: Melissa Vagi Co-authored-by: Melissa Vagi Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Document new ingest and search pipeline allowlist settings (#7414) * Document new ingest and search pipeline allowlist settings Signed-off-by: Andrew Ross * Update _ingest-pipelines/processors/index-processors.md Signed-off-by: Melissa Vagi * Update _ingest-pipelines/processors/index-processors.md Signed-off-by: Melissa Vagi * Update _search-plugins/search-pipelines/search-processors.md Signed-off-by: Melissa Vagi * Update _ingest-pipelines/processors/index-processors.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _search-plugins/search-pipelines/search-processors.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _search-plugins/search-pipelines/search-processors.md Signed-off-by: Melissa Vagi --------- Signed-off-by: Andrew Ross Signed-off-by: Melissa Vagi Co-authored-by: Melissa Vagi Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Add documentation for using rerank and normalization processors together (#7513) * Add documentation for using rerank and normalization processors together Signed-off-by: Fanit Kolchina * Update _search-plugins/search-relevance/reranking-search-results.md Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: Fanit Kolchina Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Update syntax.md -- where-filter (#7593) * Update syntax.md -- where-filter The argument to the source= command should be named for what it does, not its type, and the doc should say what it does, not how it is evaluated. Signed-off-by: Stavros Macrakis <134456002+smacrakis@users.noreply.github.com> * Update _search-plugins/sql/ppl/syntax.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Stavros Macrakis <134456002+smacrakis@users.noreply.github.com> --------- Signed-off-by: Stavros Macrakis <134456002+smacrakis@users.noreply.github.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * Update syntax.md (#7595) * Update syntax.md Rewrite the PPL syntax section. It doesn't seem useful to say that PPL "currently" supports "only" one search command, as though we have plans to support multiple search commands (with what semantics?). Just say what it does. I also removed the incorrect space in [{ : .note}], which caused that text to appear in the formatted output. I have no idea what that command is supposed to do, but I notice another one later in the file that doesn't have the space, and doesn't appear in the formatted output, so I'm assuming that [{: .note}] is correct. Signed-off-by: Stavros Macrakis <134456002+smacrakis@users.noreply.github.com> * Update _search-plugins/sql/ppl/syntax.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Stavros Macrakis <134456002+smacrakis@users.noreply.github.com> --------- Signed-off-by: Stavros Macrakis <134456002+smacrakis@users.noreply.github.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * Update-jwt-docs (#7236) * Update-jwt-docs Signed-off-by: leedonggyu * Update _security/authentication-backends/jwt.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _security/authentication-backends/jwt.md Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: leedonggyu Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Add more information to Python docs (#7506) * addresources Signed-off-by: Heather Halter * fixedlinks Signed-off-by: Heather Halter * Update _clients/python-low-level.md Signed-off-by: Melissa Vagi --------- Signed-off-by: Heather Halter Signed-off-by: Melissa Vagi Co-authored-by: Melissa Vagi Signed-off-by: leanne.laceybyrne@eliatra.com * Correct step 2 of custom local models (#7600) * Update custom-local-models.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Correct response function Signed-off-by: Fanit Kolchina * More changes Signed-off-by: Fanit Kolchina * Fix rag cluster setting Signed-off-by: Fanit Kolchina --------- Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Fanit Kolchina Signed-off-by: leanne.laceybyrne@eliatra.com * Update options and add more examples for add_entries processor (#7412) * Update options and add more examples Signed-off-by: Hai Yan * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * A few small edits * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: Hai Yan Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Update multiple data sources and add TSVB and Vega viz types (#7229) * Update multiple data sources and add TSVB and Vega viz types Signed-off-by: Melissa Vagi * add text Signed-off-by: Melissa Vagi * Add tutorials Signed-off-by: Melissa Vagi * Add tutorials Signed-off-by: Melissa Vagi * Update _dashboards/visualize/vega.md Co-authored-by: Huy Nguyen <73027756+huyaboo@users.noreply.github.com> Signed-off-by: Melissa Vagi * Update tsvb.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update vega.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update vega.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update tsvb.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update vega.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Address SME feedback Signed-off-by: Melissa Vagi * Address SME feedback Signed-off-by: Melissa Vagi * Address SME feedback Signed-off-by: Melissa Vagi * Address SME feedback Signed-off-by: Melissa Vagi * Update _dashboards/visualize/tsvb.md Co-authored-by: Heather Halter Signed-off-by: Melissa Vagi * Update _dashboards/visualize/tsvb.md Co-authored-by: Heather Halter Signed-off-by: Melissa Vagi * Update _dashboards/visualize/vega.md Co-authored-by: Heather Halter Signed-off-by: Melissa Vagi * Update _dashboards/visualize/vega.md Co-authored-by: Heather Halter Signed-off-by: Melissa Vagi * Update _dashboards/visualize/vega.md Co-authored-by: Heather Halter Signed-off-by: Melissa Vagi * Update _dashboards/visualize/tsvb.md Signed-off-by: Melissa Vagi * Update _dashboards/visualize/tsvb.md Signed-off-by: Melissa Vagi * Update _dashboards/visualize/tsvb.md Signed-off-by: Melissa Vagi * Update _dashboards/visualize/tsvb.md Signed-off-by: Melissa Vagi * Update _dashboards/visualize/tsvb.md Signed-off-by: Melissa Vagi * Update _dashboards/visualize/geojson-regionmaps.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/visualize/selfhost-maps-server.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/visualize/vega.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/visualize/visbuilder.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/visualize/vega.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/visualize/vega.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/visualize/vega.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Signed-off-by: Melissa Vagi * Update _dashboards/management/multi-data-sources.md Signed-off-by: Melissa Vagi * Update _dashboards/visualize/selfhost-maps-server.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/visualize/tsvb.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/visualize/tsvb.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/visualize/tsvb.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/visualize/tsvb.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/visualize/tsvb.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/visualize/tsvb.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/visualize/tsvb.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/visualize/tsvb.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/visualize/tsvb.md Signed-off-by: Melissa Vagi * Update _dashboards/visualize/vega.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/visualize/vega.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _dashboards/visualize/vega.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi --------- Signed-off-by: Melissa Vagi Co-authored-by: Huy Nguyen <73027756+huyaboo@users.noreply.github.com> Co-authored-by: Heather Halter Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Add geodistance query documentation (#7607) * Add geodistance query documentation Signed-off-by: Fanit Kolchina * Add optional sentence Signed-off-by: Fanit Kolchina * Fix links Signed-off-by: Fanit Kolchina * More links Signed-off-by: Fanit Kolchina * Fix a link Signed-off-by: Fanit Kolchina * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Editorial comment Signed-off-by: Fanit Kolchina --------- Signed-off-by: Fanit Kolchina Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Fix: expand_wildcards in _refresh API. (#7620) Signed-off-by: dblock Signed-off-by: leanne.laceybyrne@eliatra.com * Added documentation for cat?sort and format. (#7619) * Added documentation for cat?sort and format. Signed-off-by: dblock * Apply suggestions from code review Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: dblock Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * Add deriving metrics from logs use case to Data Prepper (#6248) * Add use case to Data Prepper Signed-off-by: Melissa Vagi * Add content Signed-off-by: Melissa Vagi * Copy edits Signed-off-by: Melissa Vagi * Update metrics-logs.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update _data-prepper/common-use-cases/metrics-logs.md Co-authored-by: David Venable Signed-off-by: Melissa Vagi * Update _data-prepper/common-use-cases/metrics-logs.md Co-authored-by: David Venable Signed-off-by: Melissa Vagi * Update _data-prepper/common-use-cases/metrics-logs.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/common-use-cases/metrics-logs.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/common-use-cases/metrics-logs.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _data-prepper/common-use-cases/metrics-logs.md Signed-off-by: Melissa Vagi * Update metrics-logs.md Signed-off-by: Melissa Vagi Signed-off-by: Melissa Vagi * Update metrics-logs.md Signed-off-by: Melissa Vagi * Update _data-prepper/common-use-cases/metrics-logs.md Signed-off-by: Melissa Vagi --------- Signed-off-by: Melissa Vagi Co-authored-by: David Venable Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * [DOC] Clarify Interaction of JVM Memory Settings via OPENSEARCH_JAVA_OPTS with Default Values (#7564) * [DOC] add jvm Memory Settings note (#7564) Signed-off-by: c-neto * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: c-neto Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * Add documentation for the UBI plugin (#7284) * gathering potential documentation attempts Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * considering the dashboard tutorial Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * place holder for js data structure usage Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * data-structures placeholder Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Updating index links Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * adding old doc to be merged Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Starting to link things together Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * fix broken link Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * respond to vale Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * more vale violations Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * name files consistently with docs site and fix links. Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * vale Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Minor tweaks. Moved Ubi under SEARCH. Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * add label for versining of spec and OS version Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * try to sort out vale error Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Converting mermaid diagrams to png's Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Updating query_id mermaid code Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Better way to ignore the mermaid scripts in the md files Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * description updates Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * schema.md updating (still need to update the mermaid diagram) Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * schema updates Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * updates Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Rebuilding main Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * merging in images Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Updating UBI spec number Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Use released version Signed-off-by: Eric Pugh * Update _search-plugins/index.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Update _search-plugins/ubi/index.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Update _search-plugins/ubi/index.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Addressing PR feedback Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Update _search-plugins/ubi/data-structures.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Update _search-plugins/ubi/data-structures.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Update _search-plugins/ubi/data-structures.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Update _search-plugins/ubi/data-structures.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Adding dsl intro Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Adding intro sentence Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Title adjust Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Addressing PR feedback Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Addressing pr feedback Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Addressing PR feedback Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Fixing vale errors Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Finishing initial pr feedback Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Next round of PR feedback Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Describing chorus workbench link Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Adding captions for result tables Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * PR clean up Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Heather Halter Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Adding a few more suggestions Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Heather Halter Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Heather Halter Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Heather Halter Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: Heather Halter Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: Heather Halter Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: Heather Halter Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: Heather Halter Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: Heather Halter Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: Heather Halter Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: Heather Halter Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * updating query filter for laptos Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: Heather Halter Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: Heather Halter Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: Heather Halter Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * PR feedback Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Update _search-plugins/ubi/ubi-dashboard-tutorial.md Co-authored-by: Heather Halter Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * PR feedback Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Heather Halter Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> * Apply suggestions from code review Edits to all files to comply with OpenSearch standards; nav_order updates Signed-off-by: Heather Halter * Apply suggestions from code review Missed a file - more commits to the sql-query topic Signed-off-by: Heather Halter * Update _search-plugins/ubi/sql-queries.md Signed-off-by: Heather Halter * Apply suggestions from code review Cleaned up the sample query topic Signed-off-by: Heather Halter * Apply suggestions from code review Signed-off-by: Heather Halter * Update _search-plugins/ubi/dsl-queries.md Signed-off-by: Heather Halter * Update _search-plugins/ubi/dsl-queries.md Signed-off-by: Heather Halter * Apply suggestions from code review Accepted editorial suggestions. Co-authored-by: Nathan Bower Signed-off-by: Heather Halter * Update index.md Reformatted table info Signed-off-by: Heather Halter * Update _search-plugins/ubi/dsl-queries.md Signed-off-by: Heather Halter * Apply suggestions from code review Signed-off-by: Heather Halter * Update index.md Signed-off-by: Heather Halter * Update schemas.md Signed-off-by: Heather Halter * Update index.md Added a missing link and fixed the table. Signed-off-by: Heather Halter * Update index.md Changed the bold to italics Signed-off-by: Heather Halter * Update ubi-dashboard-tutorial.md Removed unnecessary note tag. Signed-off-by: Heather Halter * Update schemas.md Inserted comma Signed-off-by: Heather Halter * Update sql-queries.md Signed-off-by: Heather Halter * Apply suggestions from code review There were some hidden comments that I found in this file. Co-authored-by: Nathan Bower Signed-off-by: Heather Halter * Update _search-plugins/ubi/sql-queries.md Signed-off-by: Heather Halter * Update _search-plugins/ubi/sql-queries.md Signed-off-by: Heather Halter * Apply suggestions from code review Signed-off-by: Heather Halter * Update _search-plugins/ubi/schemas.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update schemas.md Removed links on 'object_id' and 'query_id' Signed-off-by: Heather Halter * Update sql-queries.md removed a note tag and fixed line 326 Signed-off-by: Heather Halter * Update sql-queries.md One more table heading Signed-off-by: Heather Halter --------- Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> Signed-off-by: Eric Pugh Signed-off-by: Heather Halter Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Eric Pugh Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Heather Halter Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Update cat-nodes.md (#7626) 'Local' option is deprecated and no has any purpose. See Issue #7625 Signed-off-by: Landon Lengyel Signed-off-by: leanne.laceybyrne@eliatra.com * Fix Key value table (#7636) Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * Add reranking search results with MS Marco cross-encoder tutorial (#7634) * Add reranking search results with MS Marco cross-encoder tutorial Signed-off-by: Fanit Kolchina * Update _ml-commons-plugin/tutorials/reranking-cross-encoder.md Co-authored-by: Melissa Vagi Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Apply suggestions from code review Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _ml-commons-plugin/tutorials/reranking-cross-encoder.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: Fanit Kolchina Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Melissa Vagi Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Add fingerprint processor (#7631) * Add fingerprint processor Signed-off-by: gaobinlong * Completed doc review Signed-off-by: Melissa Vagi * Update _ingest-pipelines/processors/fingerprint.md Signed-off-by: Melissa Vagi * Update _ingest-pipelines/processors/fingerprint.md Signed-off-by: Melissa Vagi * Update nav order Signed-off-by: Melissa Vagi * Update _ingest-pipelines/processors/fingerprint.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _ingest-pipelines/processors/fingerprint.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _ingest-pipelines/processors/fingerprint.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _ingest-pipelines/processors/fingerprint.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _ingest-pipelines/processors/fingerprint.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _ingest-pipelines/processors/index-processors.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi --------- Signed-off-by: gaobinlong Signed-off-by: Melissa Vagi Co-authored-by: Melissa Vagi Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Removed incorrect ignore_malformed query parameter. (#7652) Signed-off-by: dblock Signed-off-by: leanne.laceybyrne@eliatra.com * Add geo-centroid and weighted average aggregations documentation (#7613) * Add geo-centroid and weighted avaerage aggregations documentation Signed-off-by: Melissa Vagi * Add geocentroid content and examples Signed-off-by: Melissa Vagi * Add weighted average content and examples Signed-off-by: Melissa Vagi * Update _aggregations/metric/geocentroid.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _aggregations/metric/geocentroid.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _aggregations/metric/geocentroid.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _aggregations/metric/geocentroid.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _aggregations/metric/geocentroid.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _aggregations/metric/geocentroid.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _aggregations/metric/weighted-avg.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _aggregations/metric/geocentroid.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _aggregations/metric/weighted-avg.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _aggregations/metric/weighted-avg.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _aggregations/metric/weighted-avg.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _aggregations/metric/weighted-avg.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _aggregations/metric/geocentroid.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _aggregations/metric/weighted-avg.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _aggregations/metric/weighted-avg.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _aggregations/metric/weighted-avg.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _aggregations/metric/weighted-avg.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _aggregations/metric/geocentroid.md Signed-off-by: Melissa Vagi * Update _aggregations/metric/geocentroid.md Signed-off-by: Melissa Vagi --------- Signed-off-by: Melissa Vagi Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Setting-envars-docs #3582 (#7400) * setting-envars-docs #3582 Signed-off-by: AntonEliatra * Update index.md Signed-off-by: AntonEliatra * Apply suggestions from code review Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: AntonEliatra * Update index.md Signed-off-by: AntonEliatra --------- Signed-off-by: AntonEliatra Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * mention both Dashboards and endpoint (#7638) * mention both Dashboards and endpoint Old text said that Dashboards are a prerequisite for using PPL, and mentioned only the Query Workbench, not the _ppl endpoint. Is it really true that Dashboards are a prerequisite? Or is it just the SQL plugin that is a prerequisite? Signed-off-by: Stavros Macrakis <134456002+smacrakis@users.noreply.github.com> * Update index.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: Stavros Macrakis <134456002+smacrakis@users.noreply.github.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * Document '_name' field in 'function_score' query's function definition (#7340) * Document '_name' field in 'function_score' query function definition Signed-off-by: Łukasz Rynek * Ensure real request JSON payload Signed-off-by: Łukasz Rynek * Ensure real response JSON payload + finish the paragraph Signed-off-by: Łukasz Rynek * Add missing copy-curl tag Signed-off-by: Łukasz Rynek * Add missing article Signed-off-by: Łukasz Rynek * Apply suggestions from code review Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Łukasz Rynek <36886649+lrynek@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: Łukasz Rynek <36886649+lrynek@users.noreply.github.com> --------- Signed-off-by: Łukasz Rynek Signed-off-by: Łukasz Rynek <36886649+lrynek@users.noreply.github.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Fix: the value of include_defaults is a boolean. (#7657) * Fix: the value of include_defaults is a boolean. Signed-off-by: dblock * Apply suggestions from code review Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: dblock Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * Update detector-visualization integration documentation to specify real-time AD results only (#7663) * Update doc to specify real-time AD results only Signed-off-by: Tyler Ohlsen * Update _observing-your-data/ad/dashboards-anomaly-detection.md Signed-off-by: Melissa Vagi * Update _observing-your-data/ad/dashboards-anomaly-detection.md Signed-off-by: Melissa Vagi --------- Signed-off-by: Tyler Ohlsen Signed-off-by: Melissa Vagi Co-authored-by: Melissa Vagi Signed-off-by: leanne.laceybyrne@eliatra.com * Fixes table in Data Prepper write_json processor (#7518) * fixtable Signed-off-by: Heather Halter * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update write_json.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: Heather Halter Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * Quote all alphabetic defaults. (#7660) * Quote all default is true/false. Signed-off-by: dblock * Fixed non-boolean defaults. Signed-off-by: dblock * Replaced cluster_manager node by cluster manager node. Signed-off-by: dblock * Replaced master node by cluster manager node. Signed-off-by: dblock * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update _tuning-your-cluster/availability-and-recovery/snapshots/sm-api.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: dblock Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * adding basic_auth config to ldap #907 (#7671) * adding basic_auth config to ldap #907 Signed-off-by: AntonEliatra * Update ldap.md Signed-off-by: AntonEliatra * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: AntonEliatra Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * Remove Point in Time from Vale terms (#7679) Signed-off-by: Fanit Kolchina Signed-off-by: leanne.laceybyrne@eliatra.com * Adding DLS with write permission recommendation #1273 (#7668) * Adding DLS with write permission recommendation #1273 Signed-off-by: AntonEliatra * Update _security/access-control/document-level-security.md Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: AntonEliatra --------- Signed-off-by: AntonEliatra Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * Add some example results to make functionality clearer (#7686) There doesn't seem to be a detailed spec for these functions. For example, what are the arguments of substring? First and last positions? (no) First position and length? (yes) Is position 0-origin or 1-origin? (1-origin) Does it accept counting position from the end with negative arguments? (yes) I've added an example result which at least clarifies the first 3 questions. Signed-off-by: Stavros Macrakis <134456002+smacrakis@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * Update functions.md (#7688) Several of the functions mentioned in the SQL/PPL Functions page (https://opensearch.org/docs/latest/search-plugins/sql/functions/) are not in fact implemented in PPL. Signed-off-by: Stavros Macrakis <134456002+smacrakis@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * Update _security/configuration/tls.md (#7691) Removed a link to a section that referenced itself. Signed-off-by: Heather Halter Signed-off-by: leanne.laceybyrne@eliatra.com * Add geopolygon query (#7665) * Add geopolygon query Signed-off-by: Fanit Kolchina * Update _query-dsl/geo-and-xy/geopolygon.md Co-authored-by: Melissa Vagi Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Add link to index file Signed-off-by: Fanit Kolchina * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: Fanit Kolchina Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Melissa Vagi Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Add PR checklist workflow (#7699) * Add PR checklist workflow Signed-off-by: Fanit Kolchina * Assign to user instead of owner Signed-off-by: Fanit Kolchina * Testing Signed-off-by: Fanit Kolchina * Remove test Signed-off-by: Fanit Kolchina * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * If a PR is submitted by a doc team member, assign that member Signed-off-by: Fanit Kolchina * Remove test Signed-off-by: Fanit Kolchina --------- Signed-off-by: Fanit Kolchina Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Add vector database page (#6238) * Add vector database page Signed-off-by: Fanit Kolchina * Revise wording Signed-off-by: Fanit Kolchina * Add k-NN example and address feedback Signed-off-by: Fanit Kolchina * Update _search-plugins/vector-search.md Co-authored-by: Melissa Vagi Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _search-plugins/vector-search.md Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Link fix Signed-off-by: Fanit Kolchina --------- Signed-off-by: Fanit Kolchina Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Heather Halter Co-authored-by: Melissa Vagi Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Update CONTRIBUTING.md (#7702) * Update CONTRIBUTING.md Tweaked the wording in the troubleshooting section. Signed-off-by: Heather Halter * Update CONTRIBUTING.md Co-authored-by: Nathan Bower Signed-off-by: Heather Halter * Update CONTRIBUTING.md Co-authored-by: Nathan Bower Signed-off-by: Heather Halter --------- Signed-off-by: Heather Halter Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Add new update_fields parameter to update workflow API (#7632) * Add new update_fields parameter to update workflow API Signed-off-by: Daniel Widdis * Update _automating-configurations/api/create-workflow.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Daniel Widdis * Update _automating-configurations/api/create-workflow.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Daniel Widdis * Update _automating-configurations/api/create-workflow.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Daniel Widdis * Fixes from doc review Signed-off-by: Daniel Widdis * Update _automating-configurations/api/create-workflow.md Co-authored-by: Nathan Bower Signed-off-by: Daniel Widdis --------- Signed-off-by: Daniel Widdis Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Correct k-NN settings and add more (#7693) * Correct k-NN settings and add more Signed-off-by: Fanit Kolchina * Add heading Signed-off-by: Fanit Kolchina * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: Fanit Kolchina Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Add permission to write on PRs to the PR checklist workflow (#7711) Signed-off-by: Fanit Kolchina Signed-off-by: leanne.laceybyrne@eliatra.com * Remove model requirement from hybrid search documentation (#7511) * Remove model requirement from hybrid search documentation Signed-off-by: Fanit Kolchina * Review comment Signed-off-by: Fanit Kolchina * Revised sentence Signed-off-by: Fanit Kolchina * Update _search-plugins/hybrid-search.md Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: Fanit Kolchina Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Adds IntelliJ's *.iml to the .gitignore. (#7705) Signed-off-by: David Venable Signed-off-by: leanne.laceybyrne@eliatra.com * Fix format issue for the split ingest processor documentation (#7695) * Fix format issue for the split ingest processor documentation Signed-off-by: gaobinlong * Update _ingest-pipelines/processors/split.md Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: Heather Halter * Update _ingest-pipelines/processors/split.md Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: Heather Halter * Apply suggestions from code review Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: Heather Halter * Update _ingest-pipelines/processors/split.md Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: Heather Halter --------- Signed-off-by: gaobinlong Signed-off-by: Heather Halter Co-authored-by: Heather Halter Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * Update _security/configuration/demo-configuration.md Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: leanneeliatra <131779422+leanneeliatra@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * Improve wording for the 2 search mode of neural sparse documentation (#7718) * improve wording for ns Signed-off-by: zhichao-aws * Apply suggestions from code review Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: zhichao-aws Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * Update nodes-stats.md (#7721) Fixing typo Signed-off-by: Landon Lengyel Signed-off-by: leanne.laceybyrne@eliatra.com * Update PR comment workflow to use pull request target (#7723) Signed-off-by: Fanit Kolchina Signed-off-by: leanne.laceybyrne@eliatra.com * Add 1.3.18 to version history (#7726) Signed-off-by: Fanit Kolchina Signed-off-by: leanne.laceybyrne@eliatra.com * Updates PPL description (#7637) * Update index.md Updated description of PPL. Signed-off-by: Heather Halter * Update index.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update search-plugins/sql/ppl/index/ Signed-off-by: Heather Halter * Update _search-plugins/sql/ppl/index.md Co-authored-by: Nathan Bower Signed-off-by: Heather Halter --------- Signed-off-by: Heather Halter Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Redirect to `latest` when the latest version is picked from the version selector (#7759) Signed-off-by: Miki Signed-off-by: leanne.laceybyrne@eliatra.com * Fix breadcrumbs by excluding collection index pages from parent relationship (#7758) Signed-off-by: Fanit Kolchina Signed-off-by: leanne.laceybyrne@eliatra.com * Remove redundant source (#7755) Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * Update compare.md (#7765) The `--results-number-align` option for the compare API is actually spelled `--results-numbers-align`. This change updates the spelling for that option. Signed-off-by: Michael Oviedo Signed-off-by: leanne.laceybyrne@eliatra.com * Document CreateAnomalyDetectorTool (#7742) * Document CreateAnomalyDetectorTool Signed-off-by: gaobinlong * Fix format issue Signed-off-by: gaobinlong * Fix link Signed-off-by: gaobinlong * Update create-anomaly-detector.md Signed-off-by: Melissa Vagi * Update _ml-commons-plugin/agents-tools/tools/index.md Signed-off-by: Melissa Vagi * Update _ml-commons-plugin/agents-tools/tools/index.md Signed-off-by: Melissa Vagi * Update _ml-commons-plugin/agents-tools/tools/create-anomaly-detector.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _ml-commons-plugin/agents-tools/tools/create-anomaly-detector.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _ml-commons-plugin/agents-tools/tools/create-anomaly-detector.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _ml-commons-plugin/agents-tools/tools/create-anomaly-detector.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _ml-commons-plugin/agents-tools/tools/index.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi --------- Signed-off-by: gaobinlong Signed-off-by: Melissa Vagi Co-authored-by: Melissa Vagi Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Add strict_allow_templates option for the dynamic mapping parameter (#7745) * Add strict_allow_templates option for the dynamic mapping parameter Signed-off-by: gaobinlong * Fix typo Signed-off-by: gaobinlong * Fix header Signed-off-by: gaobinlong * Update dynamic.md Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/supported-field-types/object.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/supported-field-types/object.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/dynamic.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update _field-types/supported-field-types/nested.md Co-authored-by: Nathan Bower Signed-off-by: Melissa Vagi * Update dynamic.md Make changes to address editorial review comments Signed-off-by: Melissa Vagi --------- Signed-off-by: gaobinlong Signed-off-by: Melissa Vagi Co-authored-by: Melissa Vagi Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Update per-cluster-metrics-monitors.md (#7769) Fixed typo in example. Signed-off-by: AWSHurneyt Signed-off-by: leanne.laceybyrne@eliatra.com * Update kafka.md (#7774) Fixed capitalization issue. Signed-off-by: Heather Halter Signed-off-by: leanne.laceybyrne@eliatra.com * Fix ISM error prevention setting key is not correct (#7777) Signed-off-by: gaobinlong Signed-off-by: leanne.laceybyrne@eliatra.com * Data Prepper documentation updates: autogeneration campaign (#7707) Updates Data Prepper documentation with some missing fields. Adds support for autogeneration of processors by naming to match the processor and including the autogenerated comment. Signed-off-by: David Venable Signed-off-by: David Venable Signed-off-by: Melissa Vagi Co-authored-by: Melissa Vagi Co-authored-by: Heather Halter Signed-off-by: leanne.laceybyrne@eliatra.com * Adds documentation for the Data Prepper delay processor. (#7708) Adds documentation for the delay processor. Signed-off-by: David Venable Signed-off-by: Melissa Vagi Co-authored-by: Melissa Vagi Signed-off-by: leanne.laceybyrne@eliatra.com * Update index.md (#7779) Community feedback Signed-off-by: Heather Halter Signed-off-by: leanne.laceybyrne@eliatra.com * add acronym for reference (#7786) Signed-off-by: Stavros Macrakis <134456002+smacrakis@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * add doc for nested_path (#7741) Signed-off-by: zhichao-aws Signed-off-by: leanne.laceybyrne@eliatra.com * Document new Split and Sort SearchResponseProcessors (#7767) * Add documentation for Sort SearchRequestProcessor Signed-off-by: Daniel Widdis * Add documentation for Split SearchRequestProcessor Signed-off-by: Daniel Widdis * Doc review Signed-off-by: Fanit Kolchina * Update _ingest-pipelines/processors/split.md Co-authored-by: Nathan Bower Signed-off-by: Daniel Widdis * Update _search-plugins/search-pipelines/sort-processor.md Co-authored-by: Nathan Bower Signed-off-by: Daniel Widdis * Update _search-plugins/search-pipelines/split-processor.md Co-authored-by: Nathan Bower Signed-off-by: Daniel Widdis * Update _search-plugins/search-pipelines/split-processor.md Co-authored-by: Nathan Bower Signed-off-by: Daniel Widdis * Update _search-plugins/search-pipelines/split-processor.md Co-authored-by: Nathan Bower Signed-off-by: Daniel Widdis * Update _search-plugins/search-pipelines/split-processor.md Co-authored-by: Nathan Bower Signed-off-by: Daniel Widdis --------- Signed-off-by: Daniel Widdis Signed-off-by: Fanit Kolchina Co-authored-by: Fanit Kolchina Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Add documentation for Deprovision Workflow API allow_delete parameter (#7639) * Add documentation for Deprovision Workflow API allow_delete parameter Signed-off-by: Daniel Widdis * Add new steps and missing delete search pipeline doc Signed-off-by: Daniel Widdis * Revert changes to workflow steps. Users can't use these new step types Signed-off-by: Daniel Widdis * Update _automating-configurations/api/deprovision-workflow.md Co-authored-by: Nathan Bower Signed-off-by: Daniel Widdis * Update _automating-configurations/api/deprovision-workflow.md Co-authored-by: Nathan Bower Signed-off-by: Daniel Widdis * Update _automating-configurations/api/deprovision-workflow.md Co-authored-by: Nathan Bower Signed-off-by: Daniel Widdis * Remove redundant use of workflow, accept other edits Signed-off-by: Daniel Widdis --------- Signed-off-by: Daniel Widdis Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Adds Documentation for dynamic query parameters for kNN search request (#7761) * Adds documentation for dynamic query parameters Signed-off-by: Tejas Shah * Update _search-plugins/knn/approximate-knn.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Update _search-plugins/knn/approximate-knn.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Tejas Shah * Update _search-plugins/knn/approximate-knn.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Tejas Shah * Update _search-plugins/knn/approximate-knn.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Tejas Shah * Update _search-plugins/knn/approximate-knn.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Tejas Shah * Update _search-plugins/knn/approximate-knn.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Tejas Shah * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Apply suggestions from code review Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: Tejas Shah Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Add Rollover API (#7685) * Add Rollover API. Signed-off-by: Archer * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Make rollover match template. Signed-off-by: Archer * Apply suggestions from code review Co-authored-by: Melissa Vagi Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update _api-reference/index-apis/rollover.md Co-authored-by: Melissa Vagi Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update _api-reference/index-apis/rollover.md Co-authored-by: Melissa Vagi Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower Co-authored-by: Melissa Vagi Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update _api-reference/index-apis/rollover.md Co-authored-by: Nathan Bower Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: Archer Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Melissa Vagi Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Fix liquid syntax errors. (#7785) * Fix liquid syntax errors. Signed-off-by: Archer * Update render-template.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update _api-reference/render-template.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update _api-reference/render-template.md Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: Archer Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * Explain ISM + link (#7787) * Explain ISM + link Signed-off-by: Stavros Macrakis <134456002+smacrakis@users.noreply.github.com> * Update _im-plugin/refresh-analyzer.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: Stavros Macrakis <134456002+smacrakis@users.noreply.github.com> Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: leanne.laceybyrne@eliatra.com * Unify and correct geoshape GeoJSON and WKT examples (#7801) * Unify and correct geoshape GeoJSON and WKT examples Signed-off-by: Fanit Kolchina * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: Fanit Kolchina Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Documentation Updates for plugins.query.datasources.enabled SQL Setting (#7794) * Documentation Updates for plugins.query.datasources.enabled SQL Setting This setting allows users to toggle the data source code paths in the SQL plugin. Ref: https://github.com/opensearch-project/sql/pull/2811/files Signed-off-by: Frank Dattalo * Update _search-plugins/sql/settings.md Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: Frank Dattalo Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Add the documentation of create or update alias API (#7641) * Add the documentation of create or update alias API Signed-off-by: gaobinlong * Fix typo Signed-off-by: gaobinlong * Refine the wording Signed-off-by: gaobinlong * Update update-alias.md * Fix typo Signed-off-by: gaobinlong * Add some clarification Signed-off-by: gaobinlong * Update update-alias.md * Update update-alias.md * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update _api-reference/index-apis/update-alias.md Signed-off-by: Nathan Bower --------- Signed-off-by: gaobinlong Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: Nathan Bower Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Nathan Bower Signed-off-by: leanne.laceybyrne@eliatra.com * Update to Mac section Signed-off-by: leanne.laceybyrne@eliatra.com * Update to mac section Signed-off-by: leanne.laceybyrne@eliatra.com * changing & to and Signed-off-by: leanne.laceybyrne@eliatra.com --------- Signed-off-by: leanne.laceybyrne@eliatra.com Signed-off-by: yujin-emma Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: Yu Jin <112784385+yujin-emma@users.noreply.github.com> Signed-off-by: Melissa Vagi Signed-off-by: Chengwu Shi Signed-off-by: chengwushi-netapp <153049940+chengwushi-netapp@users.noreply.github.com> Signed-off-by: Utkarsh Agarwal Signed-off-by: Amit Galitzky Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Signed-off-by: Sooraj Sinha Signed-off-by: Rishabh Maurya Signed-off-by: Fanit Kolchina Signed-off-by: conggguan Signed-off-by: Navneet Verma Signed-off-by: Heather Halter Signed-off-by: Kiran Prakash Signed-off-by: Kaituo Li Signed-off-by: bowenlan-amzn Signed-off-by: Michael Froh Signed-off-by: Chenyang Ji Signed-off-by: Heemin Kim Signed-off-by: Gaurav Bafna Signed-off-by: Gaurav Bafna <85113518+gbbafna@users.noreply.github.com> Signed-off-by: Dennis Toepker Signed-off-by: toepkerd <120457569+toepkerd@users.noreply.github.com> Signed-off-by: YANGDB Signed-off-by: Simeon Widdis Signed-off-by: Yaliang Wu Signed-off-by: Riya Saxena Signed-off-by: Liyun Xiu Signed-off-by: Bhavana Ramaram Signed-off-by: Jing Zhang Signed-off-by: Kondaka Signed-off-by: David Venable Signed-off-by: AntonEliatra Signed-off-by: Taylor Gray Signed-off-by: Taylor Gray Signed-off-by: leanneeliatra <131779422+leanneeliatra@users.noreply.github.com> Signed-off-by: Jay Deng Signed-off-by: Don Fox Signed-off-by: Thomas Wing Signed-off-by: Archer Signed-off-by: Harsha Vamsi Kalluri Signed-off-by: Andrew Ross Signed-off-by: Stavros Macrakis <134456002+smacrakis@users.noreply.github.com> Signed-off-by: leedonggyu Signed-off-by: Heather Halter Signed-off-by: Hai Yan Signed-off-by: dblock Signed-off-by: c-neto Signed-off-by: RasonJ <145287540+RasonJ@users.noreply.github.com> Signed-off-by: Eric Pugh Signed-off-by: Landon Lengyel Signed-off-by: gaobinlong Signed-off-by: Łukasz Rynek Signed-off-by: Łukasz Rynek <36886649+lrynek@users.noreply.github.com> Signed-off-by: Tyler Ohlsen Signed-off-by: Daniel Widdis Signed-off-by: zhichao-aws Signed-off-by: Miki Signed-off-by: Michael Oviedo Signed-off-by: AWSHurneyt Signed-off-by: Tejas Shah Signed-off-by: Frank Dattalo Signed-off-by: Nathan Bower Co-authored-by: Yu Jin <112784385+yujin-emma@users.noreply.github.com> Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Melissa Vagi Co-authored-by: Nathan Bower Co-authored-by: chengwushi-netapp <153049940+chengwushi-netapp@users.noreply.github.com> Co-authored-by: Utkarsh Agarwal <126544832+Utkarsh-Aga@users.noreply.github.com> Co-authored-by: Utkarsh Agarwal Co-authored-by: Amit Galitzky Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Sooraj Sinha <81695996+soosinha@users.noreply.github.com> Co-authored-by: Rishabh Maurya Co-authored-by: Fanit Kolchina Co-authored-by: conggguan <157357330+conggguan@users.noreply.github.com> Co-authored-by: Navneet Verma Co-authored-by: Heather Halter Co-authored-by: Kiran Prakash Co-authored-by: Kaituo Li Co-authored-by: bowenlan-amzn Co-authored-by: Michael Froh Co-authored-by: Chenyang Ji Co-authored-by: Heemin Kim Co-authored-by: Gaurav Bafna <85113518+gbbafna@users.noreply.github.com> Co-authored-by: Bhumika Saini Co-authored-by: toepkerd <120457569+toepkerd@users.noreply.github.com> Co-authored-by: Dennis Toepker Co-authored-by: YANGDB Co-authored-by: Simeon Widdis Co-authored-by: Yaliang Wu Co-authored-by: Riya <69919272+riysaxen-amzn@users.noreply.github.com> Co-authored-by: Liyun Xiu Co-authored-by: Bhavana Ramaram Co-authored-by: Jing Zhang Co-authored-by: Krishna Kondaka <41027584+kkondaka@users.noreply.github.com> Co-authored-by: David Venable Co-authored-by: AntonEliatra Co-authored-by: Taylor Gray Co-authored-by: Jay Deng Co-authored-by: Don Fox Co-authored-by: Thomas Wing Co-authored-by: Harsha Vamsi Kalluri Co-authored-by: Andrew Ross Co-authored-by: Stavros Macrakis <134456002+smacrakis@users.noreply.github.com> Co-authored-by: leedonggyu Co-authored-by: Hai Yan <8153134+oeyh@users.noreply.github.com> Co-authored-by: Huy Nguyen <73027756+huyaboo@users.noreply.github.com> Co-authored-by: Daniel (dB.) Doubrovkine Co-authored-by: Carlos Neto Co-authored-by: RasonJ <145287540+RasonJ@users.noreply.github.com> Co-authored-by: Eric Pugh Co-authored-by: Landon Lengyel Co-authored-by: gaobinlong Co-authored-by: Łukasz Rynek <36886649+lrynek@users.noreply.github.com> Co-authored-by: Tyler Ohlsen Co-authored-by: Daniel Widdis Co-authored-by: zhichao-aws Co-authored-by: Miki Co-authored-by: Michael Oviedo Co-authored-by: AWSHurneyt Co-authored-by: Tejas Shah Co-authored-by: Frank Dattalo <73919354+fddattal@users.noreply.github.com> --- _security/configuration/demo-configuration.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/_security/configuration/demo-configuration.md b/_security/configuration/demo-configuration.md index 33794b0d67..be188169ad 100644 --- a/_security/configuration/demo-configuration.md +++ b/_security/configuration/demo-configuration.md @@ -9,7 +9,6 @@ nav_order: 4 Welcome to the OpenSearch Security plugin demo configuration setup guide. This tool provides a quick and easy way to replicate a production environment for testing purposes. The demo configuration includes the setup of security-related components, such as internal users, roles, role mappings, audit configuration, basic authentication, tenants, and allow lists. - The demo configuration tool performs the following tasks: 1. Configures security settings, which are then loaded into the security index. @@ -49,9 +48,10 @@ If you want to disable the Security plugin when using Docker, set the `DISABLE_S - One digit [0--9] - One special character -4. Run `docker-compose up`. +4. Make sure that Docker is running on your local machine +5. Run `docker-compose up` from the file directory where your `docker-compose.yml` file and `.env` file are located. -### TAR (Linux) +### TAR (Linux) and Mac OS For TAR distributions on Linux, download the Linux setup files from the OpenSearch [Download & Get Started](https://opensearch.org/downloads.html) page. Then use the following command to run the demo configuration: From c561f321801e19360ee4f1c7dda04424b4811d13 Mon Sep 17 00:00:00 2001 From: Dan Cristian Cecoi Date: Wed, 24 Jul 2024 18:23:02 +0100 Subject: [PATCH 32/36] Add documentation for configuring the password hashing algorithm and its properties (#7697) * Add documentation for configuring the password hashing algorithms and their properties Signed-off-by: Dan Cecoi * Small change to the warning message Signed-off-by: Dan Cecoi * Modified the warning message and its placement Signed-off-by: Dan Cecoi * modified the bcrypt.rounds explanation Signed-off-by: Dan Cecoi * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: Dan Cecoi Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Dan Cecoi Co-authored-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Nathan Bower --- .../security-settings.md | 35 +++++++++++++++++++ 1 file changed, 35 insertions(+) diff --git a/_install-and-configure/configuring-opensearch/security-settings.md b/_install-and-configure/configuring-opensearch/security-settings.md index ffdad36cb3..b9c375d208 100644 --- a/_install-and-configure/configuring-opensearch/security-settings.md +++ b/_install-and-configure/configuring-opensearch/security-settings.md @@ -122,6 +122,41 @@ The Security plugin supports the following expert-level settings: - `plugins.security.check_snapshot_restore_write_privileges` (Static): Enforces write privilege evaluation when creating snapshots. Default is `true`. +If you change any of the following password hashing properties, you must rehash all internal passwords to ensure compatibility and security. +{: .warning} + +- `plugins.security.password.hashing.algorithm`: (Static): Specifies the password hashing algorithm to use. + + Valid values are: + + - `BCrypt` (Default) + - `PBKDF2` + +- `plugins.security.password.hashing.bcrypt.rounds` (Static): Specifies the number of rounds to use for password hashing with `BCrypt`. Valid values are between `4` and `31`, inclusive. Default is `12`. + +- `plugins.security.password.hashing.bcrypt.minor` (Static): Specifies the minor version of the `BCrypt` algorithm to use for password hashing. + + Valid values are: + + - `A` + - `B` + - `Y` (Default) + +- `plugins.security.password.hashing.pbkdf2.function` (Static): Specifies the pseudo-random function applied to the password. + + Valid values are: + + - `SHA1` + - `SHA224` + - `SHA256` (Default) + - `SHA384` + - `SHA512` + +- `plugins.security.password.hashing.pbkdf2.iterations` (Static): Specifies the number of times that the pseudo-random function is applied to the password. Default is `600,000`. + +- `plugins.security.password.hashing.pbkdf2.length` (Static): Specifies the desired length of the final derived key. Default is `256`. + + ## Audit log settings The Security plugin supports the following audit log settings: From 279237d3e4bebcf16173a3ad03b2ac151d68c2ac Mon Sep 17 00:00:00 2001 From: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Date: Wed, 24 Jul 2024 14:11:51 -0500 Subject: [PATCH 33/36] [Proposal] Update API template (#7709) * [Proposal] Update API template Updates the API template to allow for more dynamic formatting. Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update API_TEMPLATE.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update API_TEMPLATE.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Add H3 level examples. Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Make APIs conform to template Signed-off-by: Archer * Fix cluster API templates Signed-off-by: Archer * Fix link Signed-off-by: Archer --------- Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Signed-off-by: Archer --- API_STYLE_GUIDE.md | 4 +- _api-reference/analyze-apis.md | 2 +- _api-reference/cat/cat-aliases.md | 43 +++--- _api-reference/cat/cat-allocation.md | 43 +++--- _api-reference/cat/cat-cluster_manager.md | 15 +- _api-reference/cat/cat-count.md | 28 ++-- _api-reference/cat/cat-field-data.md | 43 +++--- _api-reference/cat/cat-health.md | 16 +- _api-reference/cat/cat-indices.md | 43 +++--- _api-reference/cat/cat-nodeattrs.md | 17 ++- _api-reference/cat/cat-nodes.md | 17 ++- _api-reference/cat/cat-pending-tasks.md | 17 ++- _api-reference/cat/cat-plugins.md | 17 ++- _api-reference/cat/cat-recovery.md | 43 +++--- _api-reference/cat/cat-repositories.md | 18 ++- _api-reference/cat/cat-segment-replication.md | 16 +- _api-reference/cat/cat-segments.md | 40 ++--- _api-reference/cat/cat-shards.md | 44 +++--- _api-reference/cat/cat-snapshots.md | 17 ++- _api-reference/cat/cat-tasks.md | 18 ++- _api-reference/cat/cat-templates.md | 31 ++-- _api-reference/cat/cat-thread-pool.md | 41 ++--- .../cluster-api/cluster-allocation.md | 25 +-- .../cluster-api/cluster-decommission.md | 4 +- _api-reference/cluster-api/cluster-health.md | 25 +-- .../cluster-api/cluster-settings.md | 42 +++--- _api-reference/cluster-api/cluster-stats.md | 17 ++- _api-reference/document-apis/bulk.md | 2 +- .../document-apis/delete-by-query.md | 2 +- .../document-apis/delete-document.md | 2 +- _api-reference/document-apis/get-documents.md | 2 +- .../document-apis/index-document.md | 2 +- _api-reference/document-apis/multi-get.md | 15 +- _api-reference/document-apis/reindex.md | 2 +- .../document-apis/update-by-query.md | 2 +- .../document-apis/update-document.md | 2 +- _api-reference/index-apis/alias.md | 2 +- .../index-apis/clear-index-cache.md | 12 +- _api-reference/index-apis/clone.md | 20 ++- _api-reference/index-apis/close-index.md | 2 +- _api-reference/index-apis/create-index.md | 2 +- _api-reference/index-apis/delete-index.md | 2 +- _api-reference/index-apis/exists.md | 2 +- _api-reference/index-apis/force-merge.md | 26 +++- _api-reference/index-apis/get-index.md | 2 +- _api-reference/index-apis/get-settings.md | 2 +- _api-reference/index-apis/open-index.md | 2 +- _api-reference/index-apis/split.md | 2 +- _api-reference/index-apis/stats.md | 30 +++- .../nodes-apis/nodes-hot-threads.md | 6 +- _api-reference/nodes-apis/nodes-info.md | 4 +- .../nodes-apis/nodes-reload-secure.md | 4 +- _api-reference/nodes-apis/nodes-stats.md | 4 +- _api-reference/nodes-apis/nodes-usage.md | 4 +- _api-reference/profile.md | 2 +- _api-reference/rank-eval.md | 4 +- .../script-apis/create-stored-script.md | 4 +- _api-reference/script-apis/delete-script.md | 4 +- _api-reference/script-apis/exec-script.md | 4 +- .../script-apis/exec-stored-script.md | 2 +- .../script-apis/get-script-contexts.md | 4 +- .../script-apis/get-script-language.md | 4 +- .../script-apis/get-stored-script.md | 4 +- _api-reference/scroll.md | 2 +- _api-reference/snapshots/create-repository.md | 6 +- _api-reference/snapshots/create-snapshot.md | 4 +- .../snapshots/delete-snapshot-repository.md | 4 +- _api-reference/snapshots/delete-snapshot.md | 4 +- .../snapshots/get-snapshot-repository.md | 4 +- .../snapshots/get-snapshot-status.md | 4 +- _api-reference/snapshots/get-snapshot.md | 4 +- _api-reference/snapshots/restore-snapshot.md | 4 +- .../snapshots/verify-snapshot-repository.md | 4 +- _api-reference/tasks.md | 142 +++--------------- templates/API_TEMPLATE.md | 36 ++++- 75 files changed, 552 insertions(+), 543 deletions(-) diff --git a/API_STYLE_GUIDE.md b/API_STYLE_GUIDE.md index 6dc40df017..a6e0551f17 100644 --- a/API_STYLE_GUIDE.md +++ b/API_STYLE_GUIDE.md @@ -115,7 +115,7 @@ Include a table with these columns: Field | Data type | Description :--- | :--- | :--- -#### Example request +## Example request Provide a sentence that describes what is shown in the example, followed by a cut-and-paste-ready API request in JSON format. Make sure that you test the request yourself in the Dashboards Dev Tools console to make sure it works. See the following examples. @@ -139,7 +139,7 @@ POST _reindex } ``` -#### Example response +## Example response Include a JSON example response to show what the API returns. See the following examples. diff --git a/_api-reference/analyze-apis.md b/_api-reference/analyze-apis.md index 10af71c1ad..ac8e9e249f 100644 --- a/_api-reference/analyze-apis.md +++ b/_api-reference/analyze-apis.md @@ -61,7 +61,7 @@ Field | Data type | Description :--- | :--- | :--- text | String or Array of Strings | Text to analyze. If you provide an array of strings, the text is analyzed as a multi-value field. -#### Example requests +## Example requests [Analyze array of text strings](#analyze-array-of-text-strings) diff --git a/_api-reference/cat/cat-aliases.md b/_api-reference/cat/cat-aliases.md index b0c2d7184e..2d5c5c300a 100644 --- a/_api-reference/cat/cat-aliases.md +++ b/_api-reference/cat/cat-aliases.md @@ -15,26 +15,6 @@ has_children: false The CAT aliases operation lists the mapping of aliases to indexes, plus routing and filtering information. -## Example - -```json -GET _cat/aliases?v -``` -{% include copy-curl.html %} - -To limit the information to a specific alias, add the alias name after your query: - -```json -GET _cat/aliases/?v -``` -{% include copy-curl.html %} - -If you want to get information for more than one alias, separate the alias names with commas: - -```json -GET _cat/aliases/alias1,alias2,alias3 -``` -{% include copy-curl.html %} ## Path and HTTP methods @@ -55,7 +35,28 @@ Parameter | Type | Description local | Boolean | Whether to return information from the local node only instead of from the cluster manager node. Default is `false`. expand_wildcards | Enum | Expands wildcard expressions to concrete indexes. Combine multiple values with commas. Supported values are `all`, `open`, `closed`, `hidden`, and `none`. Default is `open`. -## Response +## Example requests + +```json +GET _cat/aliases?v +``` +{% include copy-curl.html %} + +To limit the information to a specific alias, add the alias name after your query: + +```json +GET _cat/aliases/?v +``` +{% include copy-curl.html %} + +If you want to get information for more than one alias, separate the alias names with commas: + +```json +GET _cat/aliases/alias1,alias2,alias3 +``` +{% include copy-curl.html %} + +## Example response The following response shows that `alias1` refers to a `movies` index and has a configured filter: diff --git a/_api-reference/cat/cat-allocation.md b/_api-reference/cat/cat-allocation.md index 23ebed79ff..085a755dc1 100644 --- a/_api-reference/cat/cat-allocation.md +++ b/_api-reference/cat/cat-allocation.md @@ -14,26 +14,6 @@ has_children: false The CAT allocation operation lists the allocation of disk space for indexes and the number of shards on each node. -## Example - -```json -GET _cat/allocation?v -``` -{% include copy-curl.html %} - -To limit the information to a specific node, add the node name after your query: - -```json -GET _cat/allocation/ -``` -{% include copy-curl.html %} - -If you want to get information for more than one node, separate the node names with commas: - -```json -GET _cat/allocation/node_name_1,node_name_2,node_name_3 -``` -{% include copy-curl.html %} ## Path and HTTP methods @@ -54,7 +34,28 @@ bytes | Byte size | Specify the units for byte size. For example, `7kb` or `6gb` local | Boolean | Whether to return information from the local node only instead of from the cluster manager node. Default is `false`. cluster_manager_timeout | Time | The amount of time to wait for a connection to the cluster manager node. Default is 30 seconds. -## Response +## Example requests + +```json +GET _cat/allocation?v +``` +{% include copy-curl.html %} + +To limit the information to a specific node, add the node name after your query: + +```json +GET _cat/allocation/ +``` +{% include copy-curl.html %} + +If you want to get information for more than one node, separate the node names with commas: + +```json +GET _cat/allocation/node_name_1,node_name_2,node_name_3 +``` +{% include copy-curl.html %} + +## Example response The following response shows that eight shards are allocated to each of the two nodes available: diff --git a/_api-reference/cat/cat-cluster_manager.md b/_api-reference/cat/cat-cluster_manager.md index abf204ce16..d81e334009 100644 --- a/_api-reference/cat/cat-cluster_manager.md +++ b/_api-reference/cat/cat-cluster_manager.md @@ -14,12 +14,6 @@ has_children: false The CAT cluster manager operation lists information that helps identify the elected cluster manager node. -## Example - -``` -GET _cat/cluster_manager?v -``` -{% include copy-curl.html %} ## Path and HTTP methods @@ -37,7 +31,14 @@ Parameter | Type | Description :--- | :--- | :--- cluster_manager_timeout | Time | The amount of time to wait for a connection to the cluster manager node. Default is 30 seconds. -## Response +## Example requests + +``` +GET _cat/cluster_manager?v +``` +{% include copy-curl.html %} + +## Example response ```json id | host | ip | node diff --git a/_api-reference/cat/cat-count.md b/_api-reference/cat/cat-count.md index 34baa04dd4..8d0b4fbad2 100644 --- a/_api-reference/cat/cat-count.md +++ b/_api-reference/cat/cat-count.md @@ -15,7 +15,19 @@ redirect_from: The CAT count operation lists the number of documents in your cluster. -## Example + +## Path and HTTP methods + +``` +GET _cat/count?v +GET _cat/count/?v +``` + +## URL parameters + +All CAT count URL parameters are optional. You can specify any of the [common URL parameters]({{site.url}}{{site.baseurl}}/api-reference/cat/index). + +## Example requests ```json GET _cat/count?v @@ -36,19 +48,7 @@ GET _cat/count/index_or_alias_1,index_or_alias_2,index_or_alias_3 ``` {% include copy-curl.html %} -## Path and HTTP methods - -``` -GET _cat/count?v -GET _cat/count/?v -``` - -## URL parameters - -All CAT count URL parameters are optional. You can specify any of the [common URL parameters]({{site.url}}{{site.baseurl}}/api-reference/cat/index). - - -## Response +## Example response The following response shows the overall document count as 1625: diff --git a/_api-reference/cat/cat-field-data.md b/_api-reference/cat/cat-field-data.md index 6481e5cea1..05c720b952 100644 --- a/_api-reference/cat/cat-field-data.md +++ b/_api-reference/cat/cat-field-data.md @@ -8,13 +8,31 @@ redirect_from: - /opensearch/rest-api/cat/cat-field-data/ --- -# CAT fielddata +# CAT Field Data **Introduced 1.0** {: .label .label-purple } -The CAT fielddata operation lists the memory size used by each field per node. +The CAT Field Data operation lists the memory size used by each field per node. -## Example + +## Path and HTTP methods + +``` +GET _cat/fielddata?v +GET _cat/fielddata/?v +``` + +## URL parameters + +All CAT fielddata URL parameters are optional. + +In addition to the [common URL parameters]({{site.url}}{{site.baseurl}}/api-reference/cat/index), you can specify the following parameter: + +Parameter | Type | Description +:--- | :--- | :--- +bytes | Byte size | Specify the units for byte size. For example, `7kb` or `6gb`. For more information, see [Supported units]({{site.url}}{{site.baseurl}}/opensearch/units/). + +## Example requests ```json GET _cat/fielddata?v @@ -35,24 +53,7 @@ GET _cat/fielddata/field_name_1,field_name_2,field_name_3 ``` {% include copy-curl.html %} -## Path and HTTP methods - -``` -GET _cat/fielddata?v -GET _cat/fielddata/?v -``` - -## URL parameters - -All CAT fielddata URL parameters are optional. - -In addition to the [common URL parameters]({{site.url}}{{site.baseurl}}/api-reference/cat/index), you can specify the following parameter: - -Parameter | Type | Description -:--- | :--- | :--- -bytes | Byte size | Specify the units for byte size. For example, `7kb` or `6gb`. For more information, see [Supported units]({{site.url}}{{site.baseurl}}/opensearch/units/). - -## Response +## Example response The following response shows the memory size for all fields as 284 bytes: diff --git a/_api-reference/cat/cat-health.md b/_api-reference/cat/cat-health.md index 7767cfbc46..1c400916ad 100644 --- a/_api-reference/cat/cat-health.md +++ b/_api-reference/cat/cat-health.md @@ -15,12 +15,6 @@ redirect_from: The CAT health operation lists the status of the cluster, how long the cluster has been up, the number of nodes, and other useful information that helps you analyze the health of your cluster. -## Example - -```json -GET _cat/health?v -``` -{% include copy-curl.html %} ## Path and HTTP methods @@ -38,7 +32,15 @@ Parameter | Type | Description time | Time | Specify the units for time. For example, `5d` or `7h`. For more information, see [Supported units]({{site.url}}{{site.baseurl}}/opensearch/units/). ts | Boolean | If true, returns HH:MM:SS and Unix epoch timestamps. Default is `true`. -## Response +## Example request + +The following example request give cluster health information for the past 5 days: + +```json +GET _cat/health?v&time=5d +``` + +## Example response ```json GET _cat/health?v&time=5d diff --git a/_api-reference/cat/cat-indices.md b/_api-reference/cat/cat-indices.md index fe9556899e..16c57e5791 100644 --- a/_api-reference/cat/cat-indices.md +++ b/_api-reference/cat/cat-indices.md @@ -14,26 +14,6 @@ redirect_from: The CAT indices operation lists information related to indexes, that is, how much disk space they are using, how many shards they have, their health status, and so on. -## Example - -``` -GET _cat/indices?v -``` -{% include copy-curl.html %} - -To limit the information to a specific index, add the index name after your query. - -``` -GET _cat/indices/?v -``` -{% include copy-curl.html %} - -If you want to get information for more than one index, separate the indexes with commas: - -```json -GET _cat/indices/index1,index2,index3 -``` -{% include copy-curl.html %} ## Path and HTTP methods @@ -58,8 +38,29 @@ pri | Boolean | Whether to return information only from the primary shards. Defa time | Time | Specify the units for time. For example, `5d` or `7h`. For more information, see [Supported units]({{site.url}}{{site.baseurl}}/opensearch/units/). expand_wildcards | Enum | Expands wildcard expressions to concrete indexes. Combine multiple values with commas. Supported values are `all`, `open`, `closed`, `hidden`, and `none`. Default is `open`. +## Example requests + +``` +GET _cat/indices?v +``` +{% include copy-curl.html %} + +To limit the information to a specific index, add the index name after your query. + +``` +GET _cat/indices/?v +``` +{% include copy-curl.html %} + +If you want to get information for more than one index, separate the indexes with commas: + +```json +GET _cat/indices/index1,index2,index3 +``` +{% include copy-curl.html %} + -## Response +## Example response ```json health | status | index | uuid | pri | rep | docs.count | docs.deleted | store.size | pri.store.size diff --git a/_api-reference/cat/cat-nodeattrs.md b/_api-reference/cat/cat-nodeattrs.md index 6b4cc6d92e..b09e164698 100644 --- a/_api-reference/cat/cat-nodeattrs.md +++ b/_api-reference/cat/cat-nodeattrs.md @@ -14,12 +14,6 @@ redirect_from: The CAT nodeattrs operation lists the attributes of custom nodes. -## Example - -``` -GET _cat/nodeattrs?v -``` -{% include copy-curl.html %} ## Path and HTTP methods @@ -38,8 +32,17 @@ Parameter | Type | Description local | Boolean | Whether to return information from the local node only instead of from the cluster manager node. Default is `false`. cluster_manager_timeout | Time | The amount of time to wait for a connection to the cluster manager node. Default is 30 seconds. +## Example request + +The following example request returns attributes about custom nodes: + +``` +GET _cat/nodeattrs?v +``` +{% include copy-curl.html %} + -## Response +## Example response ```json node | host | ip | attr | value diff --git a/_api-reference/cat/cat-nodes.md b/_api-reference/cat/cat-nodes.md index 864e5dfdd5..5e7238a0d0 100644 --- a/_api-reference/cat/cat-nodes.md +++ b/_api-reference/cat/cat-nodes.md @@ -16,12 +16,6 @@ The CAT nodes operation lists node-level information, including node roles and l A few important node metrics are `pid`, `name`, `cluster_manager`, `ip`, `port`, `version`, `build`, `jdk`, along with `disk`, `heap`, `ram`, and `file_desc`. -## Example - -``` -GET _cat/nodes?v -``` -{% include copy-curl.html %} ## Path and HTTP methods @@ -43,8 +37,17 @@ cluster_manager_timeout | Time | The amount of time to wait for a connection to time | Time | Specify the units for time. For example, `5d` or `7h`. For more information, see [Supported units]({{site.url}}{{site.baseurl}}/opensearch/units/). include_unloaded_segments | Boolean | Whether to include information from segments not loaded into memory. Default is `false`. +## Example request + +The following example request lists node level information: + +``` +GET _cat/nodes?v +``` +{% include copy-curl.html %} + -## Response +## Example response ```json ip | heap.percent | ram.percent | cpu load_1m | load_5m | load_15m | node.role | node.roles | cluster_manager | name diff --git a/_api-reference/cat/cat-pending-tasks.md b/_api-reference/cat/cat-pending-tasks.md index 748defd06e..ea224670ac 100644 --- a/_api-reference/cat/cat-pending-tasks.md +++ b/_api-reference/cat/cat-pending-tasks.md @@ -15,12 +15,6 @@ redirect_from: The CAT pending tasks operation lists the progress of all pending tasks, including task priority and time in queue. -## Example - -``` -GET _cat/pending_tasks?v -``` -{% include copy-curl.html %} ## Path and HTTP methods @@ -40,7 +34,16 @@ local | Boolean | Whether to return information from the local node only instead cluster_manager_timeout | Time | The amount of time to wait for a connection to the cluster manager node. Default is 30 seconds. time | Time | Specify the units for time. For example, `5d` or `7h`. For more information, see [Supported units]({{site.url}}{{site.baseurl}}/opensearch/units/). -## Response +## Example request + +The following example request lists the progress of all pending node tasks: + +``` +GET _cat/pending_tasks?v +``` +{% include copy-curl.html %} + +## Example response ```json insertOrder | timeInQueue | priority | source diff --git a/_api-reference/cat/cat-plugins.md b/_api-reference/cat/cat-plugins.md index 519c77f27f..358eb70fbf 100644 --- a/_api-reference/cat/cat-plugins.md +++ b/_api-reference/cat/cat-plugins.md @@ -15,12 +15,6 @@ redirect_from: The CAT plugins operation lists the names, components, and versions of the installed plugins. -## Example - -``` -GET _cat/plugins?v -``` -{% include copy-curl.html %} ## Path and HTTP methods @@ -39,7 +33,16 @@ Parameter | Type | Description local | Boolean | Whether to return information from the local node only instead of from the cluster manager node. Default is `false`. cluster_manager_timeout | Time | The amount of time to wait for a connection to the cluster manager node. Default is 30 seconds. -## Response +## Example requests + +The following example request lists all installed plugins: + +``` +GET _cat/plugins?v +``` +{% include copy-curl.html %} + +## Example response ```json name component version diff --git a/_api-reference/cat/cat-recovery.md b/_api-reference/cat/cat-recovery.md index da66aa7272..8f251a94e0 100644 --- a/_api-reference/cat/cat-recovery.md +++ b/_api-reference/cat/cat-recovery.md @@ -15,26 +15,6 @@ redirect_from: The CAT recovery operation lists all completed and ongoing index and shard recoveries. -## Example - -``` -GET _cat/recovery?v -``` -{% include copy-curl.html %} - -To see only the recoveries of a specific index, add the index name after your query. - -``` -GET _cat/recovery/?v -``` -{% include copy-curl.html %} - -If you want to get information for more than one index, separate the indexes with commas: - -```json -GET _cat/recovery/index1,index2,index3 -``` -{% include copy-curl.html %} ## Path and HTTP methods @@ -55,7 +35,28 @@ bytes | Byte size | Specify the units for byte size. For example, `7kb` or `6gb` detailed | Boolean | Whether to include detailed information about shard recoveries. Default is `false`. time | Time | Specify the units for time. For example, `5d` or `7h`. For more information, see [Supported units]({{site.url}}{{site.baseurl}}/opensearch/units/). -## Response +## Example requests + +``` +GET _cat/recovery?v +``` +{% include copy-curl.html %} + +To see only the recoveries of a specific index, add the index name after your query. + +``` +GET _cat/recovery/?v +``` +{% include copy-curl.html %} + +If you want to get information for more than one index, separate the indexes with commas: + +```json +GET _cat/recovery/index1,index2,index3 +``` +{% include copy-curl.html %} + +## Example response ```json index | shard | time | type | stage | source_host | source_node | target_host | target_node | repository | snapshot | files | files_recovered | files_percent | files_total | bytes | bytes_recovered | bytes_percent | bytes_total | translog_ops | translog_ops_recovered | translog_ops_percent diff --git a/_api-reference/cat/cat-repositories.md b/_api-reference/cat/cat-repositories.md index c6d62c9c62..f0fc4bb622 100644 --- a/_api-reference/cat/cat-repositories.md +++ b/_api-reference/cat/cat-repositories.md @@ -15,13 +15,6 @@ redirect_from: The CAT repositories operation lists all snapshot repositories for a cluster. -## Example - -``` -GET _cat/repositories?v -``` -{% include copy-curl.html %} - ## Path and HTTP methods ``` @@ -39,8 +32,17 @@ Parameter | Type | Description local | Boolean | Whether to return information from the local node only instead of from the cluster manager node. Default is `false`. cluster_manager_timeout | Time | The amount of time to wait for a connection to the cluster manager node. Default is 30 seconds. +## Example request + +The following example request lists all snapshot repositories in the cluster: + +``` +GET _cat/repositories?v +``` +{% include copy-curl.html %} + -## Response +## Example response ```json id type diff --git a/_api-reference/cat/cat-segment-replication.md b/_api-reference/cat/cat-segment-replication.md index e22012ea66..5900b97a7c 100644 --- a/_api-reference/cat/cat-segment-replication.md +++ b/_api-reference/cat/cat-segment-replication.md @@ -47,11 +47,11 @@ Parameter | Data type | Description `v` | Boolean | If `true`, the response includes column headings. Defaults to `false`. `s` | String | Specifies to sort the results. For example, `s=shardId:desc` sorts by shardId in descending order. -## Example +## Example requests The following examples illustrate various segment replication responses. -#### Example 1: No active segment replication events +### No active segment replication events The following query requests segment replication metrics with column headings for all indexes: @@ -67,7 +67,7 @@ shardId target_node target_host checkpoints_behind bytes_behind current_lag last [index-1][0] runTask-1 127.0.0.1 0 0b 0s 7ms 0 ``` -#### Example 2: Shard ID specified +### Shard ID specified The following query requests segment replication metrics with column headings for shards with the ID `0` from indexes `index1` and `index2`: @@ -84,7 +84,7 @@ shardId target_node target_host checkpoints_behind bytes_behind current_lag last [index-2][0] runTask-1 127.0.0.1 0 0b 0s 5ms 0 ``` -#### Example 3: Detailed response +### Detailed response The following query requests detailed segment replication metrics with column headings for all indexes: @@ -101,7 +101,7 @@ shardId target_node target_host checkpoints_behind bytes_behind current_lag last [index-2][0] runTask-1 127.0.0.1 0 0b 0s 5ms 0 done 7ms 3 100.0% 3664 100.0% 2023-03-16T13:53:33.466Z 2023-03-16T13:53:33.474Z 3 3 3.5kb 3.5kb 0s 1ms 0s 2ms 2ms ``` -#### Example 4: Sorting the results +### Sorting the results The following query requests segment replication metrics with column headings for all indexes, sorted by shard ID in descending order: @@ -118,7 +118,7 @@ shardId target_node target_host checkpoints_behind bytes_behind current_lag [test6][0] runTask-2 127.0.0.1 0 0b 0s 4ms 0 ``` -#### Example 5: Using a metric alias +### Using a metric alias In a request, you can either use a metric's full name or one of its aliases. The following query is the same as the preceding query, but it uses the alias `s` instead of `shardID` for sorting: @@ -127,9 +127,9 @@ GET /_cat/segment_replication?v&s=s:desc ``` {% include copy-curl.html %} -## Response metrics +## Example response metrics -The following table lists the response metrics that are returned for all requests. When referring to a metric in a query parameter, you can provide either the metric's full name or any of its aliases, as shown in the previous [example](#example-5-using-a-metric-alias). +The following table lists the response metrics that are returned for all requests. When referring to a metric in a query parameter, you can provide either the metric's full name or any of its aliases, as shown in the previous [example](#using-a-metric-alias). Metric | Alias | Description :--- | :--- | :--- diff --git a/_api-reference/cat/cat-segments.md b/_api-reference/cat/cat-segments.md index b860486692..cd9eda38be 100644 --- a/_api-reference/cat/cat-segments.md +++ b/_api-reference/cat/cat-segments.md @@ -15,7 +15,25 @@ redirect_from: The cat segments operation lists Lucene segment-level information for each index. -## Example + +## Path and HTTP methods + +``` +GET _cat/segments +``` + +## URL parameters + +All CAT segments URL parameters are optional. + +In addition to the [common URL parameters]({{site.url}}{{site.baseurl}}/api-reference/cat/index), you can specify the following parameters: + +Parameter | Type | Description +:--- | :--- | :--- +bytes | Byte size | Specify the units for byte size. For example, `7kb` or `6gb`. For more information, see [Supported units]({{site.url}}{{site.baseurl}}/opensearch/units/).. +cluster_manager_timeout | Time | The amount of time to wait for a connection to the cluster manager node. Default is 30 seconds. + +## Example requests ``` GET _cat/segments?v @@ -36,25 +54,7 @@ GET _cat/segments/index1,index2,index3 ``` {% include copy-curl.html %} -## Path and HTTP methods - -``` -GET _cat/segments -``` - -## URL parameters - -All CAT segments URL parameters are optional. - -In addition to the [common URL parameters]({{site.url}}{{site.baseurl}}/api-reference/cat/index), you can specify the following parameters: - -Parameter | Type | Description -:--- | :--- | :--- -bytes | Byte size | Specify the units for byte size. For example, `7kb` or `6gb`. For more information, see [Supported units]({{site.url}}{{site.baseurl}}/opensearch/units/).. -cluster_manager_timeout | Time | The amount of time to wait for a connection to the cluster manager node. Default is 30 seconds. - - -## Response +## Example response ```json index | shard | prirep | ip | segment | generation | docs.count | docs.deleted | size | size.memory | committed | searchable | version | compound diff --git a/_api-reference/cat/cat-shards.md b/_api-reference/cat/cat-shards.md index 9a727b5b11..b07f11aca3 100644 --- a/_api-reference/cat/cat-shards.md +++ b/_api-reference/cat/cat-shards.md @@ -15,26 +15,6 @@ redirect_from: The CAT shards operation lists the state of all primary and replica shards and how they are distributed. -## Example - -``` -GET _cat/shards?v -``` -{% include copy-curl.html %} - -To see only the information about shards of a specific index, add the index name after your query. - -``` -GET _cat/shards/?v -``` -{% include copy-curl.html %} - -If you want to get information for more than one index, separate the indexes with commas: - -``` -GET _cat/shards/index1,index2,index3 -``` -{% include copy-curl.html %} ## Path and HTTP methods @@ -55,8 +35,30 @@ local | Boolean | Whether to return information from the local node only instead cluster_manager_timeout | Time | The amount of time to wait for a connection to the cluster manager node. Default is 30 seconds. time | Time | Specify the units for time. For example, `5d` or `7h`. For more information, see [Supported units]({{site.url}}{{site.baseurl}}/opensearch/units/). +## Example requests + +The following example requests returns information about shards: + +``` +GET _cat/shards?v +``` +{% include copy-curl.html %} + +To see only the information about shards of a specific index, add the index name after your query. + +``` +GET _cat/shards/?v +``` +{% include copy-curl.html %} + +If you want to get information for more than one index, separate the indexes with commas: + +``` +GET _cat/shards/index1,index2,index3 +``` +{% include copy-curl.html %} -## Response +## Example response ```json index | shard | prirep | state | docs | store | ip | | node diff --git a/_api-reference/cat/cat-snapshots.md b/_api-reference/cat/cat-snapshots.md index 82cb5c1b1f..2e1bd514bf 100644 --- a/_api-reference/cat/cat-snapshots.md +++ b/_api-reference/cat/cat-snapshots.md @@ -15,12 +15,6 @@ redirect_from: The CAT snapshots operation lists all snapshots for a repository. -## Example - -``` -GET _cat/snapshots?v -``` -{% include copy-curl.html %} ## Path and HTTP methods @@ -39,8 +33,17 @@ Parameter | Type | Description cluster_manager_timeout | Time | The amount of time to wait for a connection to the cluster manager node. Default is 30 seconds. time | Time | Specify the units for time. For example, `5d` or `7h`. For more information, see [Supported units]({{site.url}}{{site.baseurl}}/opensearch/units/). +## Example request + +The following example request lists all snapshots: + +``` +GET _cat/snapshots?v +``` +{% include copy-curl.html %} + -## Response +## Example response ```json index | shard | prirep | state | docs | store | ip | | node diff --git a/_api-reference/cat/cat-tasks.md b/_api-reference/cat/cat-tasks.md index 4d2a06cced..7a71b592e7 100644 --- a/_api-reference/cat/cat-tasks.md +++ b/_api-reference/cat/cat-tasks.md @@ -15,13 +15,6 @@ redirect_from: The CAT tasks operation lists the progress of all tasks currently running on your cluster. -## Example - -``` -GET _cat/tasks?v -``` -{% include copy-curl.html %} - ## Path and HTTP methods ``` @@ -41,8 +34,17 @@ detailed | Boolean | Returns detailed task information. (Default: false) parent_task_id | String | Returns tasks with a specified parent task ID (node_id:task_number). Keep empty or set to -1 to return all. time | Time | Specify the units for time. For example, `5d` or `7h`. For more information, see [Supported units]({{site.url}}{{site.baseurl}}/opensearch/units/). +## Example request + +The following example request lists all tasks in progress: + +``` +GET _cat/tasks?v +``` +{% include copy-curl.html %} + -## Response +## Example response ```json action | task_id | parent_task_id | type | start_time | timestamp | running_time | ip | node diff --git a/_api-reference/cat/cat-templates.md b/_api-reference/cat/cat-templates.md index d7c7aac90f..ba47ae711d 100644 --- a/_api-reference/cat/cat-templates.md +++ b/_api-reference/cat/cat-templates.md @@ -15,19 +15,6 @@ redirect_from: The CAT templates operation lists the names, patterns, order numbers, and version numbers of index templates. -## Example - -``` -GET _cat/templates?v -``` -{% include copy-curl.html %} - -If you want to get information for a specific template or pattern: - -``` -GET _cat/templates/ -``` -{% include copy-curl.html %} ## Path and HTTP methods @@ -47,8 +34,24 @@ Parameter | Type | Description local | Boolean | Whether to return information from the local node only instead of from the cluster manager node. Default is `false`. cluster_manager_timeout | Time | The amount of time to wait for a connection to the cluster manager node. Default is 30 seconds. +## Example requests + +The following example request returns information about all templates: + +``` +GET _cat/templates?v +``` +{% include copy-curl.html %} + +If you want to get information for a specific template or pattern: + +``` +GET _cat/templates/ +``` +{% include copy-curl.html %} + -## Response +## Example response ``` name | index_patterns order version composed_of diff --git a/_api-reference/cat/cat-thread-pool.md b/_api-reference/cat/cat-thread-pool.md index 491b523092..de24052175 100644 --- a/_api-reference/cat/cat-thread-pool.md +++ b/_api-reference/cat/cat-thread-pool.md @@ -14,7 +14,27 @@ redirect_from: The CAT thread pool operation lists the active, queued, and rejected threads of different thread pools on each node. -## Example + +## Path and HTTP methods + +``` +GET _cat/thread_pool +``` + +## URL parameters + +All CAT thread pool URL parameters are optional. + +In addition to the [common URL parameters]({{site.url}}{{site.baseurl}}/api-reference/cat/index), you can specify the following parameters: + +Parameter | Type | Description +:--- | :--- | :--- +local | Boolean | Whether to return information from the local node only instead of from the cluster manager node. Default is `false`. +cluster_manager_timeout | Time | The amount of time to wait for a connection to the cluster manager node. Default is 30 seconds. + +## Example requests + +The following example request gives information about thread pools on all nodes: ``` GET _cat/thread_pool?v @@ -35,25 +55,8 @@ GET _cat/thread_pool/?v ``` {% include copy-curl.html %} -## Path and HTTP methods - -``` -GET _cat/thread_pool -``` - -## URL parameters - -All CAT thread pool URL parameters are optional. - -In addition to the [common URL parameters]({{site.url}}{{site.baseurl}}/api-reference/cat/index), you can specify the following parameters: - -Parameter | Type | Description -:--- | :--- | :--- -local | Boolean | Whether to return information from the local node only instead of from the cluster manager node. Default is `false`. -cluster_manager_timeout | Time | The amount of time to wait for a connection to the cluster manager node. Default is 30 seconds. - -## Response +## Example response ```json node_name name active queue rejected diff --git a/_api-reference/cluster-api/cluster-allocation.md b/_api-reference/cluster-api/cluster-allocation.md index b1b1c266d6..4ec6e27f2b 100644 --- a/_api-reference/cluster-api/cluster-allocation.md +++ b/_api-reference/cluster-api/cluster-allocation.md @@ -17,17 +17,6 @@ The most basic cluster allocation explain request finds an unassigned shard and If you add some options, you can instead get information on a specific shard, including why OpenSearch assigned it to its current node. -## Example - -```json -GET _cluster/allocation/explain?include_yes_decisions=true -{ - "index": "movies", - "shard": 0, - "primary": true -} -``` -{% include copy-curl.html %} ## Path and HTTP methods @@ -58,8 +47,20 @@ index | String | The name of the shard's index. primary | Boolean | Whether to provide an explanation for the primary shard (true) or its first replica (false), which share the same shard ID. shard | Integer | The shard ID that you want an explanation for. +## Example request + +```json +GET _cluster/allocation/explain?include_yes_decisions=true +{ + "index": "movies", + "shard": 0, + "primary": true +} +``` +{% include copy-curl.html %} + -## Response +## Example response ```json { diff --git a/_api-reference/cluster-api/cluster-decommission.md b/_api-reference/cluster-api/cluster-decommission.md index 867f58eda0..c707e5390a 100644 --- a/_api-reference/cluster-api/cluster-decommission.md +++ b/_api-reference/cluster-api/cluster-decommission.md @@ -54,7 +54,7 @@ DELETE /_cluster/decommission/awareness ``` {% include copy-curl.html %} -#### Response +#### Example response ```json @@ -74,7 +74,7 @@ GET /_cluster/decommission/awareness/zone/_status ``` {% include copy-curl.html %} -#### Response +#### Example response ```json { diff --git a/_api-reference/cluster-api/cluster-health.md b/_api-reference/cluster-api/cluster-health.md index 73c83d5ee6..7081a1a587 100644 --- a/_api-reference/cluster-api/cluster-health.md +++ b/_api-reference/cluster-api/cluster-health.md @@ -17,16 +17,6 @@ The most basic cluster health request returns a simple status of the health of y To get the status of a specific index, provide the index name. -## Example - -This request waits 50 seconds for the cluster to reach the yellow status or better: - -``` -GET _cluster/health?wait_for_status=yellow&timeout=50s -``` -{% include copy-curl.html %} - -If the cluster health becomes yellow or green before 50 seconds elapse, it returns a response immediately. Otherwise it returns a response as soon as it exceeds the timeout. ## Path and HTTP methods @@ -55,7 +45,18 @@ wait_for_no_initializing_shards | Boolean | Whether to wait until there are no i wait_for_status | Enum | Wait until the cluster health reaches the specified status or better. Supported values are `green`, `yellow`, and `red`. weights | JSON object | Assigns weights to attributes within the request body of the PUT request. Weights can be set in any ration, for example, 2:3:5. In a 2:3:5 ratio with three zones, for every 100 requests sent to the cluster, each zone would receive either 20, 30, or 50 search requests in a random order. When assigned a weight of `0`, the zone does not receive any search traffic. -#### Example request +## Example requests + +The following examples show how to use the cluster health API. + +This request waits 50 seconds for the cluster to reach the yellow status or better: + +``` +GET _cluster/health?wait_for_status=yellow&timeout=50s +``` +{% include copy-curl.html %} + +If the cluster health becomes yellow or green before 50 seconds elapse, it returns a response immediately. Otherwise it returns a response as soon as it exceeds the timeout. The following example request retrieves cluster health for all indexes in the cluster: @@ -64,7 +65,7 @@ GET _cluster/health ``` {% include copy-curl.html %} -#### Example response +## Example response The response contains cluster health information: diff --git a/_api-reference/cluster-api/cluster-settings.md b/_api-reference/cluster-api/cluster-settings.md index 3538339001..ec682ecdbd 100644 --- a/_api-reference/cluster-api/cluster-settings.md +++ b/_api-reference/cluster-api/cluster-settings.md @@ -32,25 +32,6 @@ include_defaults (GET only) | Boolean | Whether to include default settings as p cluster_manager_timeout | Time unit | The amount of time to wait for a response from the cluster manager node. Default is `30 seconds`. timeout (PUT only) | Time unit | The amount of time to wait for a response from the cluster. Default is `30 seconds`. - -#### Example request - -```json -GET _cluster/settings?include_defaults=true -``` -{% include copy-curl.html %} - -#### Example response - -```json -PUT _cluster/settings -{ - "persistent":{ - "action.auto_create_index": false - } -} -``` - ## Request fields The GET operation has no request body options. All cluster setting field parameters are optional. @@ -60,9 +41,24 @@ Not all cluster settings can be updated using the cluster settings API. You will For a listing of all cluster settings, see [Configuring OpenSearch]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/index/). -#### Example request -For a PUT operation, the request body must contain `transient` or `persistent`, along with the setting you want to update: +## Example requests + +The following example request show how to use the cluster settings API. + +### Check default cluster settings + +The following example request checks for default cluster settings: + +```json +GET _cluster/settings?include_defaults=true +``` +{% include copy-curl.html %} + +### Update cluster setting + +The following example updates the `cluster.max_shards_per_node` setting. For a PUT operation, the request body must contain `transient` or `persistent`, along with the setting you want to update: + ```json PUT _cluster/settings @@ -76,7 +72,9 @@ PUT _cluster/settings For more information about transient settings, persistent settings, and precedence, see [OpenSearch configuration]({{site.url}}{{site.baseurl}}/install-and-configure/configuring-opensearch/). -#### Example response +## Example response + +The following example response shows that the persistent cluster setting, `max_shard_per_node`, has been updated: ```json { diff --git a/_api-reference/cluster-api/cluster-stats.md b/_api-reference/cluster-api/cluster-stats.md index 8f8b585a6a..fb0ade2c6b 100644 --- a/_api-reference/cluster-api/cluster-stats.md +++ b/_api-reference/cluster-api/cluster-stats.md @@ -15,12 +15,6 @@ redirect_from: The cluster stats API operation returns statistics about your cluster. -## Example - -```json -GET _cluster/stats/nodes/_cluster_manager -``` -{% include copy-curl.html %} ## Path and HTTP methods @@ -41,7 +35,16 @@ Parameter | Type | Description Although the `master` node is now called `cluster_manager` for version 2.0, we retained the `master` field for backwards compatibility. If you have a node that has either a `master` role or a `cluster_manager` role, the `count` increases for both fields by 1. To see an example node count increase, see the Response sample. {: .note } -## Response +## Example request + +The following example requests returns information about the cluster manager node: + +```json +GET _cluster/stats/nodes/_cluster_manager +``` +{% include copy-curl.html %} + +## Example response ```json { diff --git a/_api-reference/document-apis/bulk.md b/_api-reference/document-apis/bulk.md index a020fc459d..a9833a701f 100644 --- a/_api-reference/document-apis/bulk.md +++ b/_api-reference/document-apis/bulk.md @@ -134,7 +134,7 @@ All actions support the same metadata: `_index`, `_id`, and `_require_alias`. If { "script" : { "source": "ctx._source.title = \"World War Z\"" } } ``` -## Response +## Example response In the response, pay particular attention to the top-level `errors` boolean. If true, you can iterate over the individual actions for more detailed information. diff --git a/_api-reference/document-apis/delete-by-query.md b/_api-reference/document-apis/delete-by-query.md index 6f4104c254..64da909aad 100644 --- a/_api-reference/document-apis/delete-by-query.md +++ b/_api-reference/document-apis/delete-by-query.md @@ -88,7 +88,7 @@ To search your index for specific documents, you must include a [query]({{site.u } ``` -## Response +## Example response ```json { "took": 143, diff --git a/_api-reference/document-apis/delete-document.md b/_api-reference/document-apis/delete-document.md index c3dea2f7e1..ece99a28ca 100644 --- a/_api-reference/document-apis/delete-document.md +++ b/_api-reference/document-apis/delete-document.md @@ -42,7 +42,7 @@ version_type | Enum | Retrieves a specifically typed document. Available options wait_for_active_shards | String | The number of active shards that must be available before OpenSearch processes the delete request. Default is 1 (only the primary shard). Set to `all` or a positive integer. Values greater than 1 require replicas. For example, if you specify a value of 3, the index must have two replicas distributed across two additional nodes for the operation to succeed. | No -## Response +## Example response ```json { "_index": "sample-index1", diff --git a/_api-reference/document-apis/get-documents.md b/_api-reference/document-apis/get-documents.md index 3eaeb507d4..d493df136b 100644 --- a/_api-reference/document-apis/get-documents.md +++ b/_api-reference/document-apis/get-documents.md @@ -49,7 +49,7 @@ version | Integer | The version of the document to return, which must match the version_type | Enum | Retrieves a specifically typed document. Available options are `external` (retrieve the document if the specified version number is greater than the document's current version) and `external_gte` (retrieve the document if the specified version number is greater than or equal to the document's current version). For example, to retrieve version 3 of a document, use `/_doc/1?version=3&version_type=external`. -## Response +## Example response ```json { "_index": "sample-index1", diff --git a/_api-reference/document-apis/index-document.md b/_api-reference/document-apis/index-document.md index d131a2f50e..a506e2d9d8 100644 --- a/_api-reference/document-apis/index-document.md +++ b/_api-reference/document-apis/index-document.md @@ -110,7 +110,7 @@ Your request body must contain the information you want to index. } ``` -## Response +## Example response ```json { "_index": "sample-index", diff --git a/_api-reference/document-apis/multi-get.md b/_api-reference/document-apis/multi-get.md index 2d3246fa58..b267b8f3ac 100644 --- a/_api-reference/document-apis/multi-get.md +++ b/_api-reference/document-apis/multi-get.md @@ -54,7 +54,11 @@ _source.excludes | Array | Specifies which fields to exclude in the query respon ids | Array | IDs of the documents to retrieve. Only allowed when an index is specified in the URL. | No -#### Example without specifying index in URL +## Example requests + +### Specify an index in the request body + +The following example requests does specifies an index in the request body: ```json GET _mget @@ -76,7 +80,9 @@ GET _mget ``` {% include copy-curl.html %} -#### Example of specifying index in URL +### Specify an index the URL + +The following example specifies an index in the URL: ```json GET sample-index1/_mget @@ -95,7 +101,10 @@ GET sample-index1/_mget ``` {% include copy-curl.html %} -#### Example Response +## Example response + +The following example response returns information about multiple documents: + ```json { "docs": [ diff --git a/_api-reference/document-apis/reindex.md b/_api-reference/document-apis/reindex.md index c2afa347e1..8ac1c48be4 100644 --- a/_api-reference/document-apis/reindex.md +++ b/_api-reference/document-apis/reindex.md @@ -81,7 +81,7 @@ pipeline | Which ingest pipeline to utilize during the reindex. script | A script that OpenSearch uses to apply transformations to the data during the reindex operation. lang | The scripting language. Valid options are `painless`, `expression`, `mustache`, and `java`. -## Response +## Example response ```json { "took": 28829, diff --git a/_api-reference/document-apis/update-by-query.md b/_api-reference/document-apis/update-by-query.md index 217ae69550..09b3bd599f 100644 --- a/_api-reference/document-apis/update-by-query.md +++ b/_api-reference/document-apis/update-by-query.md @@ -102,7 +102,7 @@ To update your indexes and documents by query, you must include a [query]({{site } ``` -## Response +## Example response ```json { "took": 21, diff --git a/_api-reference/document-apis/update-document.md b/_api-reference/document-apis/update-document.md index 3da7030fa5..3f951b5adf 100644 --- a/_api-reference/document-apis/update-document.md +++ b/_api-reference/document-apis/update-document.md @@ -195,7 +195,7 @@ After the upsert operation, the document's `first_name` and `last_name` fields a } ``` -## Response +## Example response ```json { "_index": "sample-index1", diff --git a/_api-reference/index-apis/alias.md b/_api-reference/index-apis/alias.md index a38a3798a4..ebd7bdedfd 100644 --- a/_api-reference/index-apis/alias.md +++ b/_api-reference/index-apis/alias.md @@ -75,7 +75,7 @@ routing | String | Used to assign a custom value to a shard for specific operati index_routing | String | Assigns a custom value to a shard only for index operations. | No search_routing | String | Assigns a custom value to a shard only for search operations. | No -## Response +## Example response ```json { diff --git a/_api-reference/index-apis/clear-index-cache.md b/_api-reference/index-apis/clear-index-cache.md index 55a5ce85d5..9bf873301d 100644 --- a/_api-reference/index-apis/clear-index-cache.md +++ b/_api-reference/index-apis/clear-index-cache.md @@ -38,11 +38,11 @@ All query parameters are optional. | query | Boolean | If `true`, clears the query cache. Defaults to `true`. | | request | Boolean | If `true`, clears the request cache. Defaults to `true`. | -#### Example requests +## Example requests The following example requests show multiple clear cache API uses. -##### Clear a specific cache +### Clear a specific cache The following request clears the fields cache only: @@ -69,7 +69,7 @@ POST /my-index/_cache/clear?request=true ``` {% include copy-curl.html %} -#### Clear the cache for specific fields +### Clear the cache for specific fields The following request clears the fields caches of `fielda` and `fieldb`: @@ -78,7 +78,7 @@ POST /my-index/_cache/clear?fields=fielda,fieldb ``` {% include copy-curl.html %} -#### Clear caches for specific data streams or indexes +### Clear caches for specific data streams or indexes The following request clears the cache for two specific indexes: @@ -96,14 +96,14 @@ POST /_cache/clear ``` {% include copy-curl.html %} -#### Clear unused entries from the cache on search-capable nodes +### Clear unused entries from the cache on search-capable nodes ```json POST /*/_cache/clear?file=true ``` {% include copy-curl.html %} -#### Example response +## Example response The `POST /books,hockey/_cache/clear` request returns the following fields: diff --git a/_api-reference/index-apis/clone.md b/_api-reference/index-apis/clone.md index 60228b5894..c1496cbaf8 100644 --- a/_api-reference/index-apis/clone.md +++ b/_api-reference/index-apis/clone.md @@ -66,7 +66,25 @@ task_execution_timeout | Time | The explicit task execution timeout. Only useful The clone index API operation creates a new target index, so you can specify any [index settings]({{site.url}}{{site.baseurl}}/im-plugin/index-settings/) and [aliases]({{site.url}}{{site.baseurl}}/opensearch/index-alias/) to apply to the target index. -## Response +## Example request + +```json +PUT /sample-index1/_clone/cloned-index1 +{ + "settings": { + "index": { + "number_of_shards": 2, + "number_of_replicas": 1 + } + }, + "aliases": { + "sample-alias1": {} + } +} +``` +{% include copy-curl.html %} + +## Example response ```json { diff --git a/_api-reference/index-apis/close-index.md b/_api-reference/index-apis/close-index.md index 7e43198d37..865d17d90a 100644 --- a/_api-reference/index-apis/close-index.md +++ b/_api-reference/index-apis/close-index.md @@ -41,7 +41,7 @@ cluster_manager_timeout | Time | How long to wait for a connection to the cluste timeout | Time | How long to wait for a response from the cluster. Default is `30s`. -## Response +## Example response ```json { "acknowledged": true, diff --git a/_api-reference/index-apis/create-index.md b/_api-reference/index-apis/create-index.md index 53d2dc28f9..ff5d7dbda5 100644 --- a/_api-reference/index-apis/create-index.md +++ b/_api-reference/index-apis/create-index.md @@ -52,7 +52,7 @@ timeout | Time | How long to wait for the request to return. Default is `30s`. As part of your request, you can optionally specify [index settings]({{site.url}}{{site.baseurl}}/im-plugin/index-settings/), [mappings]({{site.url}}{{site.baseurl}}/field-types/index/), and [aliases]({{site.url}}{{site.baseurl}}/opensearch/index-alias/) for your newly created index. -#### Example request +## Example request ```json PUT /sample-index1 diff --git a/_api-reference/index-apis/delete-index.md b/_api-reference/index-apis/delete-index.md index 20e5c51c93..ad00eb7eca 100644 --- a/_api-reference/index-apis/delete-index.md +++ b/_api-reference/index-apis/delete-index.md @@ -38,7 +38,7 @@ cluster_manager_timeout | Time | How long to wait for a connection to the cluste timeout | Time | How long to wait for the response to return. Default is `30s`. -## Response +## Example response ```json { "acknowledged": true diff --git a/_api-reference/index-apis/exists.md b/_api-reference/index-apis/exists.md index 429ac40745..351e2f2088 100644 --- a/_api-reference/index-apis/exists.md +++ b/_api-reference/index-apis/exists.md @@ -40,6 +40,6 @@ ignore_unavailable | Boolean | If true, OpenSearch does not search for missing o local | Boolean | Whether to return information from only the local node instead of from the cluster manager node. Default is `false`. -## Response +## Example response The index exists API operation returns only one of two possible response codes: `200` -- the index exists, and `404` -- the index does not exist. diff --git a/_api-reference/index-apis/force-merge.md b/_api-reference/index-apis/force-merge.md index 6c2a61bef3..ce7501ebe3 100644 --- a/_api-reference/index-apis/force-merge.md +++ b/_api-reference/index-apis/force-merge.md @@ -74,42 +74,56 @@ The following table lists the available query parameters. All query parameters a | `only_expunge_deletes` | Boolean | If `true`, the merge operation only expunges segments containing a certain percentage of deleted documents. The percentage is 10% by default and is configurable in the `index.merge.policy.expunge_deletes_allowed` setting. Prior to OpenSearch 2.12, `only_expunge_deletes` ignored the `index.merge.policy.max_merged_segment` setting. Starting with OpenSearch 2.12, using `only_expunge_deletes` does not produce segments larger than `index.merge.policy.max_merged_segment` (by default, 5 GB). For more information, see [Deleted documents](#deleted-documents). Default is `false`. | | `primary_only` | Boolean | If set to `true`, then the merge operation is performed only on the primary shards of an index. This can be useful when you want to take a snapshot of the index after the merge is complete. Snapshots only copy segments from the primary shards. Merging the primary shards can reduce resource consumption. Default is `false`. | -#### Example request: Force merge a specific index +## Example requests + +The following examples show how to use the Force merge API. + +### Force merge a specific index + +The following example force merges a specific index: ```json POST /testindex1/_forcemerge ``` {% include copy-curl.html %} -#### Example request: Force merge multiple indexes +### Force merge multiple indexes + +The following example force merges multiple indexes: ```json POST /testindex1,testindex2/_forcemerge ``` {% include copy-curl.html %} -#### Example request: Force merge all indexes +### Force merge all indexes + +The following example force merges all indexes: ```json POST /_forcemerge ``` {% include copy-curl.html %} -#### Example request: Force merge a data stream's backing indexes into one segment +### Force merge a data stream's backing indexes into one segment + +The following example force merges a data stream's backing indexes into one segment: ```json POST /.testindex-logs/_forcemerge?max_num_segments=1 ``` {% include copy-curl.html %} -#### Example request: Force merge primary shards +### Force merge primary shards + +The following example force merges an index's primary shards: ```json POST /.testindex-logs/_forcemerge?primary_only=true ``` {% include copy-curl.html %} -#### Example response +## Example response ```json { diff --git a/_api-reference/index-apis/get-index.md b/_api-reference/index-apis/get-index.md index 733110d63a..e2d2d85c65 100644 --- a/_api-reference/index-apis/get-index.md +++ b/_api-reference/index-apis/get-index.md @@ -41,7 +41,7 @@ local | Boolean | Whether to return information from only the local node instead cluster_manager_timeout | Time | How long to wait for a connection to the cluster manager node. Default is `30s`. -## Response +## Example response ```json { "sample-index1": { diff --git a/_api-reference/index-apis/get-settings.md b/_api-reference/index-apis/get-settings.md index c41b25b4f5..94cb4a7c6c 100644 --- a/_api-reference/index-apis/get-settings.md +++ b/_api-reference/index-apis/get-settings.md @@ -45,7 +45,7 @@ ignore_unavailable | Boolean | If true, OpenSearch does not include missing or c local | Boolean | Whether to return information from the local node only instead of the cluster manager node. Default is `false`. cluster_manager_timeout | Time | How long to wait for a connection to the cluster manager node. Default is `30s`. -## Response +## Example response ```json { diff --git a/_api-reference/index-apis/open-index.md b/_api-reference/index-apis/open-index.md index 12381aa8c6..0d8ef62282 100644 --- a/_api-reference/index-apis/open-index.md +++ b/_api-reference/index-apis/open-index.md @@ -43,7 +43,7 @@ wait_for_completion | Boolean | When set to `false`, the request returns immedia task_execution_timeout | Time | The explicit task execution timeout. Only useful when wait_for_completion is set to `false`. Default is `1h`. -## Response +## Example response ```json { "acknowledged": true, diff --git a/_api-reference/index-apis/split.md b/_api-reference/index-apis/split.md index 03b2f742d1..ad13bffbba 100644 --- a/_api-reference/index-apis/split.md +++ b/_api-reference/index-apis/split.md @@ -66,7 +66,7 @@ task_execution_timeout | Time | The explicit task execution timeout. Only useful The split index API operation creates a new target index, so you can specify any [index settings]({{site.url}}{{site.baseurl}}/im-plugin/index-settings/) and [aliases]({{site.url}}{{site.baseurl}}/opensearch/index-alias/) to apply to the target index. -## Response +## Example response ```json { diff --git a/_api-reference/index-apis/stats.md b/_api-reference/index-apis/stats.md index 8ccd624de1..7310298594 100644 --- a/_api-reference/index-apis/stats.md +++ b/_api-reference/index-apis/stats.md @@ -71,49 +71,65 @@ Parameter | Data type | Description `include_segment_file_sizes` | Boolean | Specifies whether to report the aggregated disk usage of each Lucene index file. Only applies to `segments` statistics. Default is `false`. `include_unloaded_segments` | Boolean | Specifies whether to include information from segments that are not loaded into memory. Default is `false`. -#### Example request: One index +## Example requests + +The following example requests show how to use the Index Stats API. + +### One index + +The following example returns index stats for a single index: ```json GET /testindex/_stats ``` {% include copy-curl.html %} -#### Example request: Comma-separated list of indexes +### Comma-separated list of indexes + +The following example returns stats for multiple indexes: ```json GET /testindex1,testindex2/_stats ``` {% include copy-curl.html %} -#### Example request: Wildcard expression +### Wildcard expression + +The following example returns starts about any index that starts with `testindex`: ```json GET /testindex*/_stats ``` {% include copy-curl.html %} -#### Example request: Specific stats +### Specific stats + +The following example returns index stats related to the index and flush operations: ```json GET /testindex/_stats/refresh,flush ``` {% include copy-curl.html %} -#### Example request: Expand wildcards +### Expand wildcards + +The following example expands all wildcards related to index stats: ```json GET /testindex*/_stats?expand_wildcards=open,hidden ``` {% include copy-curl.html %} -#### Example request: Shard-level statistics +### Shard-level statistics + +The following example returns shard level stats about a test index: ```json GET /testindex/_stats?level=shards ``` {% include copy-curl.html %} -#### Example response +## Example response By default, the returned statistics are aggregated in the `primaries` and `total` aggregations. The `primaries` aggregation contains statistics for the primary shards. The `total` aggregation contains statistics for both primary and replica shards. The following is an example Index Stats API response: diff --git a/_api-reference/nodes-apis/nodes-hot-threads.md b/_api-reference/nodes-apis/nodes-hot-threads.md index 3fb6ff65ea..f5e014dd6d 100644 --- a/_api-reference/nodes-apis/nodes-hot-threads.md +++ b/_api-reference/nodes-apis/nodes-hot-threads.md @@ -46,14 +46,14 @@ ignore_idle_threads | Boolean | Don’t show threads that are in known idle st type | String | Supported thread types are `cpu`, `wait`, or `block`. Defaults to `cpu`. timeout | Time | Sets the time limit for node response. Default value is `30s`. -#### Example request +## Example request ```json GET /_nodes/hot_threads ``` {% include copy-curl.html %} -#### Example response +## Example response ```bash ::: {opensearch}{F-ByTQzVQ3GQeYzQJArJGQ}{GxbcLdCATPWggOuQHJAoCw}{127.0.0.1}{127.0.0.1:9300}{dimr}{shard_indexing_pressure_enabled=true} @@ -65,7 +65,7 @@ GET /_nodes/hot_threads org.opensearch.performanceanalyzer.collectors.ScheduledMetricCollectorsExecutor.run(ScheduledMetricCollectorsExecutor.java:100) ``` -## Response +## Example response Unlike the majority of OpenSearch API responses, this response is in a text format. diff --git a/_api-reference/nodes-apis/nodes-info.md b/_api-reference/nodes-apis/nodes-info.md index d7c810410e..a8953505ff 100644 --- a/_api-reference/nodes-apis/nodes-info.md +++ b/_api-reference/nodes-apis/nodes-info.md @@ -79,7 +79,7 @@ Parameter | Type | Description flat_settings| Boolean | Specifies whether to return the `settings` object of the response in flat format. Default is `false`. timeout | Time | Sets the time limit for node response. Default value is `30s`. -#### Example request +## Example request The following query requests the `process` and `transport` metrics from the cluster manager node: @@ -88,7 +88,7 @@ GET /_nodes/cluster_manager:true/process,transport ``` {% include copy-curl.html %} -#### Example response +## Example response The response contains the metric groups specified in the `` request parameter (in this case, `process` and `transport`): diff --git a/_api-reference/nodes-apis/nodes-reload-secure.md b/_api-reference/nodes-apis/nodes-reload-secure.md index 52b2ef67ab..b4be66ddd4 100644 --- a/_api-reference/nodes-apis/nodes-reload-secure.md +++ b/_api-reference/nodes-apis/nodes-reload-secure.md @@ -36,7 +36,7 @@ The request may include an optional object containing the password for the OpenS } ``` -#### Example request +## Example request The following is an example API request: @@ -45,7 +45,7 @@ POST _nodes/reload_secure_settings ``` {% include copy-curl.html %} -#### Example response +## Example response The following is an example response: diff --git a/_api-reference/nodes-apis/nodes-stats.md b/_api-reference/nodes-apis/nodes-stats.md index ca6810b961..2ccea54390 100644 --- a/_api-reference/nodes-apis/nodes-stats.md +++ b/_api-reference/nodes-apis/nodes-stats.md @@ -118,14 +118,14 @@ level | String | Specifies whether statistics for the `indices` metric are aggre timeout | Time | Sets the time limit for node response. Default is `30s`. include_segment_file_sizes | Boolean | If segment statistics are requested, this field specifies to return the aggregated disk usage of every Lucene index file. Default is `false`. -#### Example request +## Example request ```json GET _nodes/stats/ ``` {% include copy-curl.html %} -#### Example response +## Example response Select the arrow to view the example response. diff --git a/_api-reference/nodes-apis/nodes-usage.md b/_api-reference/nodes-apis/nodes-usage.md index 532ddb626b..355b7f8ff2 100644 --- a/_api-reference/nodes-apis/nodes-usage.md +++ b/_api-reference/nodes-apis/nodes-usage.md @@ -38,7 +38,7 @@ Parameter | Type | Description timeout | Time | Sets the time limit for a response from the node. Default is `30s`. cluster_manager_timeout | Time | Sets the time limit for a response from the cluster manager. Default is `30s`. -#### Example request +## Example request The following request returns usage details for all nodes: @@ -47,7 +47,7 @@ GET _nodes/usage ``` {% include copy-curl.html %} -#### Example response +## Example response The following is an example response: diff --git a/_api-reference/profile.md b/_api-reference/profile.md index 94c7857b80..4f8c69db9c 100644 --- a/_api-reference/profile.md +++ b/_api-reference/profile.md @@ -26,7 +26,7 @@ A slice is the unit of work that can be executed by a thread. Each query can be In general, the max/min/avg slice time captures statistics across all slices for a timing type. For example, when profiling aggregations, the `max_slice_time_in_nanos` field in the `aggregations` section shows the maximum time consumed by the aggregation operation and its children across all slices. -#### Example request: Non-concurrent search +## Example request: Non-concurrent search To use the Profile API, include the `profile` parameter set to `true` in the search request sent to the `_search` endpoint: diff --git a/_api-reference/rank-eval.md b/_api-reference/rank-eval.md index 04fd3cf5c0..881ff3a22b 100644 --- a/_api-reference/rank-eval.md +++ b/_api-reference/rank-eval.md @@ -45,7 +45,7 @@ ignore_unlabeled | Defaults to `false`. Unlabeled documents are ignored when set template_id | Template ID. params | Parameters used in the template. -#### Example request +## Example request ````json GET shakespeare/_rank_eval @@ -76,7 +76,7 @@ GET shakespeare/_rank_eval ```` {% include copy-curl.html %} -#### Example response +## Example response ````json { diff --git a/_api-reference/script-apis/create-stored-script.md b/_api-reference/script-apis/create-stored-script.md index 04a73a205a..0a915cd836 100644 --- a/_api-reference/script-apis/create-stored-script.md +++ b/_api-reference/script-apis/create-stored-script.md @@ -47,7 +47,7 @@ All parameters are optional. | lang | String | Scripting language. Required. | | source | String or Object | Required.

For scripts, a string with the contents of the script.

For search templates, an object that defines the search template. Supports the same parameters as the [Search]({{site.url}}{{site.baseurl}}/api-reference/search) API request body. Search templates also support Mustache variables. | -#### Example request +## Example request The sample uses an index called `books` with the following documents: @@ -117,7 +117,7 @@ PUT _scripts/my-first-script See [Execute Painless stored script]({{site.url}}{{site.baseurl}}/api-reference/script-apis/exec-stored-script/) for information about running the script. -#### Example response +## Example response The `PUT _scripts/my-first-script` request returns the following field: diff --git a/_api-reference/script-apis/delete-script.md b/_api-reference/script-apis/delete-script.md index 363b0152df..fe9c272acc 100644 --- a/_api-reference/script-apis/delete-script.md +++ b/_api-reference/script-apis/delete-script.md @@ -26,7 +26,7 @@ Path parameters are optional. | cluster_manager_timeout | Time | Amount of time to wait for a connection to the cluster manager. Optional, defaults to `30s`. | | timeout | Time | The period of time to wait for a response. If a response is not received before the timeout value, the request will be dropped. -#### Example request +## Example request The following request deletes the `my-first-script` script: @@ -35,7 +35,7 @@ DELETE _scripts/my-script ```` {% include copy-curl.html %} -#### Example response +## Example response The `DELETE _scripts/my-first-script` request returns the following field: diff --git a/_api-reference/script-apis/exec-script.md b/_api-reference/script-apis/exec-script.md index 4ecb6a37fc..b6476be980 100644 --- a/_api-reference/script-apis/exec-script.md +++ b/_api-reference/script-apis/exec-script.md @@ -26,7 +26,7 @@ POST /_scripts/painless/_execute | context | A context for the script. Optional. Default is `painless_test`. | | context_setup | Specifies additional parameters for the context. Optional.| -#### Example request +## Example request The following request uses the default `painless_context` for the script: @@ -44,7 +44,7 @@ GET /_scripts/painless/_execute ``` {% include copy-curl.html %} -#### Example response +## Example response The response contains the average of two script parameters: diff --git a/_api-reference/script-apis/exec-stored-script.md b/_api-reference/script-apis/exec-stored-script.md index 7525ec81a4..a7de3b5274 100644 --- a/_api-reference/script-apis/exec-stored-script.md +++ b/_api-reference/script-apis/exec-stored-script.md @@ -21,7 +21,7 @@ OpenSearch provides several ways to run a script; the following sections show ho | script_fields | Object | Fields to include in output. | | script | Object | ID of the script that produces a value for a field. | -#### Example request +## Example request The following request runs the stored script that was created in [Create or update stored script]({{site.url}}{{site.baseurl}}/api-reference/script-apis/create-stored-script/). The script sums the ratings for each book and displays the sum in the `total_ratings` field in the output. diff --git a/_api-reference/script-apis/get-script-contexts.md b/_api-reference/script-apis/get-script-contexts.md index 40a155955f..85421128a1 100644 --- a/_api-reference/script-apis/get-script-contexts.md +++ b/_api-reference/script-apis/get-script-contexts.md @@ -11,14 +11,14 @@ nav_order: 5 Retrieves all contexts for stored scripts. -#### Example request +## Example request ````json GET _script_context ```` {% include copy-curl.html %} -#### Example response +## Example response The `GET _script_context` request returns the following fields: diff --git a/_api-reference/script-apis/get-script-language.md b/_api-reference/script-apis/get-script-language.md index f84b7db4a0..76414d52ea 100644 --- a/_api-reference/script-apis/get-script-language.md +++ b/_api-reference/script-apis/get-script-language.md @@ -11,14 +11,14 @@ nav_order: 6 The get script language API operation retrieves all supported script languages and the contexts in which they may be used. -#### Example request +## Example request ```json GET _script_language ``` {% include copy-curl.html %} -#### Example response +## Example response The `GET _script_language` request returns the available contexts for each language: diff --git a/_api-reference/script-apis/get-stored-script.md b/_api-reference/script-apis/get-stored-script.md index cc681cd0f4..d7987974d3 100644 --- a/_api-reference/script-apis/get-stored-script.md +++ b/_api-reference/script-apis/get-stored-script.md @@ -23,7 +23,7 @@ Retrieves a stored script. :--- | :--- | :--- | cluster_manager_timeout | Time | Amount of time to wait for a connection to the cluster manager. Optional, defaults to `30s`. | -#### Example request +## Example request The following retrieves the `my-first-script` stored script. @@ -32,7 +32,7 @@ GET _scripts/my-first-script ```` {% include copy-curl.html %} -#### Example response +## Example response The `GET _scripts/my-first-script` request returns the following fields: diff --git a/_api-reference/scroll.md b/_api-reference/scroll.md index cee599902d..b940c90d86 100644 --- a/_api-reference/scroll.md +++ b/_api-reference/scroll.md @@ -106,7 +106,7 @@ scroll | Time | Specifies the amount of time the search context is maintained. scroll_id | String | The scroll ID for the search. rest_total_hits_as_int | Boolean | Whether the `hits.total` property is returned as an integer (`true`) or an object (`false`). Default is `false`. -## Response +## Example response ```json { diff --git a/_api-reference/snapshots/create-repository.md b/_api-reference/snapshots/create-repository.md index 54807b85d1..8ee7885ca8 100644 --- a/_api-reference/snapshots/create-repository.md +++ b/_api-reference/snapshots/create-repository.md @@ -48,7 +48,7 @@ Request field | Description `remote_store_index_shallow_copy` | Boolean | Determines whether the snapshot of the remote store indexes are captured as a shallow copy. Default is `false`. `readonly` | Whether the repository is read-only. Useful when migrating from one cluster (`"readonly": false` when registering) to another cluster (`"readonly": true` when registering). Optional. -#### Example request +## Example request The following example registers an `fs` repository using the local directory `/mnt/snapshots` as `location`. @@ -85,7 +85,7 @@ Request field | Description For the `base_path` parameter, do not enter the `s3://` prefix when entering your S3 bucket details. Only the name of the bucket is required. {: .note} -#### Example request +## Example request The following request registers a new S3 repository called `my-opensearch-repo` in an existing bucket called `my-open-search-bucket`. By default, all snapshots are stored in the `my/snapshot/directory`. @@ -101,7 +101,7 @@ PUT /_snapshot/my-opensearch-repo ``` {% include copy-curl.html %} -#### Example response +## Example response Upon success, the following JSON object is returned: diff --git a/_api-reference/snapshots/create-snapshot.md b/_api-reference/snapshots/create-snapshot.md index 6334878d8c..d4c9ef8219 100644 --- a/_api-reference/snapshots/create-snapshot.md +++ b/_api-reference/snapshots/create-snapshot.md @@ -46,7 +46,7 @@ Field | Data type | Description `include_global_state` | Boolean | Whether to include cluster state in the snapshot. Default is `true`. `partial` | Boolean | Whether to allow partial snapshots. Default is `false`, which fails the entire snapshot if one or more shards fails to stor -#### Example requests +## Example requests ##### Request without a body @@ -72,7 +72,7 @@ PUT _snapshot/my-s3-repository/2 ``` {% include copy-curl.html %} -#### Example responses +## Example responses Upon success, the response content depends on whether you include the `wait_for_completion` query parameter. diff --git a/_api-reference/snapshots/delete-snapshot-repository.md b/_api-reference/snapshots/delete-snapshot-repository.md index 385205a5df..1fadc21207 100644 --- a/_api-reference/snapshots/delete-snapshot-repository.md +++ b/_api-reference/snapshots/delete-snapshot-repository.md @@ -21,7 +21,7 @@ Parameter | Data type | Description :--- | :--- | :--- repository | String | Repository to delete. | -#### Example request +## Example request The following request deletes the `my-opensearch-repo` repository: @@ -30,7 +30,7 @@ DELETE _snapshot/my-opensearch-repo ```` {% include copy-curl.html %} -#### Example response +## Example response Upon success, the response returns the following JSON object: diff --git a/_api-reference/snapshots/delete-snapshot.md b/_api-reference/snapshots/delete-snapshot.md index e4232c20ec..d231adf74a 100644 --- a/_api-reference/snapshots/delete-snapshot.md +++ b/_api-reference/snapshots/delete-snapshot.md @@ -24,7 +24,7 @@ Parameter | Data type | Description repository | String | Repostory that contains the snapshot. | snapshot | String | Snapshot to delete. | -#### Example request +## Example request The following request deletes a snapshot called `my-first-snapshot` from the `my-opensearch-repo` repository: @@ -33,7 +33,7 @@ DELETE _snapshot/my-opensearch-repo/my-first-snapshot ``` {% include copy-curl.html %} -#### Example response +## Example response Upon success, the response returns the following JSON object: diff --git a/_api-reference/snapshots/get-snapshot-repository.md b/_api-reference/snapshots/get-snapshot-repository.md index 501d0785dd..6617106059 100644 --- a/_api-reference/snapshots/get-snapshot-repository.md +++ b/_api-reference/snapshots/get-snapshot-repository.md @@ -29,7 +29,7 @@ You can also get details about a snapshot during and after snapshot creation. Se | local | Boolean | Whether to get information from the local node. Optional, defaults to `false`.| | cluster_manager_timeout | Time | Amount of time to wait for a connection to the cluster manager node. Optional, defaults to 30 seconds. | -#### Example request +## Example request The following request retrieves information for the `my-opensearch-repo` repository: @@ -38,7 +38,7 @@ GET /_snapshot/my-opensearch-repo ```` {% include copy-curl.html %} -#### Example response +## Example response Upon success, the response returns repositry information. This sample is for an `s3` repository type. diff --git a/_api-reference/snapshots/get-snapshot-status.md b/_api-reference/snapshots/get-snapshot-status.md index 6f8320d0b0..c7f919bcb3 100644 --- a/_api-reference/snapshots/get-snapshot-status.md +++ b/_api-reference/snapshots/get-snapshot-status.md @@ -42,7 +42,7 @@ Using the API to return state for other than currently running snapshots can be :--- | :--- | :--- | ignore_unavailable | Boolean | How to handles requests for unavailable snapshots. If `false`, the request returns an error for unavailable snapshots. If `true`, the request ignores unavailable snapshots, such as those that are corrupted or temporarily cannot be returned. Defaults to `false`.| -#### Example request +## Example request The following request returns the status of `my-first-snapshot` in the `my-opensearch-repo` repository. Unavailable snapshots are ignored. @@ -54,7 +54,7 @@ GET _snapshot/my-opensearch-repo/my-first-snapshot/_status ```` {% include copy-curl.html %} -#### Example response +## Example response The example that follows corresponds to the request above in the [Example request](#example-request) section. diff --git a/_api-reference/snapshots/get-snapshot.md b/_api-reference/snapshots/get-snapshot.md index da44c1f23d..ac55c0370f 100644 --- a/_api-reference/snapshots/get-snapshot.md +++ b/_api-reference/snapshots/get-snapshot.md @@ -25,7 +25,7 @@ Retrieves information about a snapshot. | verbose | Boolean | Whether to show all, or just basic snapshot information. If `true`, returns all information. If `false`, omits information like start/end times, failures, and shards. Optional, defaults to `true`.| | ignore_unavailable | Boolean | How to handle snapshots that are unavailable (corrupted or otherwise temporarily can't be returned). If `true` and the snapshot is unavailable, the request does not return the snapshot. If `false` and the snapshot is unavailable, the request returns an error. Optional, defaults to `false`.| -#### Example request +## Example request The following request retrieves information for the `my-first-snapshot` located in the `my-opensearch-repo` repository: @@ -34,7 +34,7 @@ GET _snapshot/my-opensearch-repo/my-first-snapshot ```` {% include copy-curl.html %} -#### Example response +## Example response Upon success, the response returns snapshot information: diff --git a/_api-reference/snapshots/restore-snapshot.md b/_api-reference/snapshots/restore-snapshot.md index 7b82f72256..cdb9948c28 100644 --- a/_api-reference/snapshots/restore-snapshot.md +++ b/_api-reference/snapshots/restore-snapshot.md @@ -57,7 +57,7 @@ All request body parameters are optional. * Ingest pipelines * Index lifecycle policies -#### Example request +## Example request The following request restores the `opendistro-reports-definitions` index from `my-first-snapshot`. The `rename_pattern` and `rename_replacement` combination causes the index to be renamed to `opendistro-reports-definitions_restored` because duplicate open index names in a cluster are not allowed. @@ -73,7 +73,7 @@ POST /_snapshot/my-opensearch-repo/my-first-snapshot/_restore } ```` -#### Example response +## Example response Upon success, the response returns the following JSON object: diff --git a/_api-reference/snapshots/verify-snapshot-repository.md b/_api-reference/snapshots/verify-snapshot-repository.md index 12fada3303..e5e6337196 100644 --- a/_api-reference/snapshots/verify-snapshot-repository.md +++ b/_api-reference/snapshots/verify-snapshot-repository.md @@ -32,7 +32,7 @@ Path parameters are optional. | cluster_manager_timeout | Time | Amount of time to wait for a connection to the cluster manager node. Optional, defaults to `30s`. | | timeout | Time | The period of time to wait for a response. If a response is not received before the timeout value, the request fails and returns an error. Defaults to `30s`. | -#### Example request +## Example request The following request verifies that the my-opensearch-repo is functional: @@ -40,7 +40,7 @@ The following request verifies that the my-opensearch-repo is functional: POST /_snapshot/my-opensearch-repo/_verify?timeout=0s&cluster_manager_timeout=50s ```` -#### Example response +## Example response The example that follows corresponds to the request above in the [Example request](#example-request) section. diff --git a/_api-reference/tasks.md b/_api-reference/tasks.md index 5c3a41fd34..e4ca0b6049 100644 --- a/_api-reference/tasks.md +++ b/_api-reference/tasks.md @@ -28,59 +28,6 @@ GET _tasks/ Note that if a task finishes running, it won't be returned as part of your request. For an example of a task that takes a little longer to finish, you can run the [`_reindex`]({{site.url}}{{site.baseurl}}/opensearch/reindex-data) API operation on a larger document, and then run `tasks`. -#### Example response -```json -{ - "nodes": { - "Mgqdm0r9SEGClWxp_RbnaQ": { - "name": "opensearch-node1", - "transport_address": "172.18.0.3:9300", - "host": "172.18.0.3", - "ip": "172.18.0.3:9300", - "roles": [ - "data", - "ingest", - "master", - "remote_cluster_client" - ], - "tasks": { - "Mgqdm0r9SEGClWxp_RbnaQ:17416": { - "node": "Mgqdm0r9SEGClWxp_RbnaQ", - "id": 17416, - "type": "transport", - "action": "cluster:monitor/tasks/lists", - "start_time_in_millis": 1613599752458, - "running_time_in_nanos": 994000, - "cancellable": false, - "headers": {} - } - }, - "Mgqdm0r9SEGClWxp_RbnaQ:17413": { - "node": "Mgqdm0r9SEGClWxp_RbnaQ", - "id": 17413, - "type": "transport", - "action": "indices:data/write/bulk", - "start_time_in_millis": 1613599752286, - "running_time_in_nanos": 172846500, - "cancellable": false, - "parent_task_id": "Mgqdm0r9SEGClWxp_RbnaQ:17366", - "headers": {} - }, - "Mgqdm0r9SEGClWxp_RbnaQ:17366": { - "node": "Mgqdm0r9SEGClWxp_RbnaQ", - "id": 17366, - "type": "transport", - "action": "indices:data/write/reindex", - "start_time_in_millis": 1613599750929, - "running_time_in_nanos": 1529733100, - "cancellable": true, - "headers": {} - } - } - } - } -} -``` You can also use the following parameters with your query. @@ -97,14 +44,29 @@ Parameter | Data type | Description | For example, this request returns tasks currently running on a node named `opensearch-node1`: -#### Example request +## Example requests + +### Return information about running tasks + +The following request returns tasks currently running on a node named `opensearch-node1`: ```json GET /_tasks?nodes=opensearch-node1 ``` {% include copy-curl.html %} -#### Example response +### Return information about active search tasks + +The following request returns detailed information about active search tasks: + +```bash +curl -XGET "localhost:9200/_tasks?actions=*search&detailed +``` +{% include copy.html %} + +## Example response + +The following example response shows information about running tasks: ```json { @@ -148,76 +110,6 @@ GET /_tasks?nodes=opensearch-node1 } ``` -The following request returns detailed information about active search tasks: - -#### Example request - -```bash -curl -XGET "localhost:9200/_tasks?actions=*search&detailed -``` -{% include copy.html %} - -#### Example response - -```json -{ - "nodes" : { - "CRqNwnEeRXOjeTSYYktw-A" : { - "name" : "runTask-0", - "transport_address" : "127.0.0.1:9300", - "host" : "127.0.0.1", - "ip" : "127.0.0.1:9300", - "roles" : [ - "cluster_manager", - "data", - "ingest", - "remote_cluster_client" - ], - "attributes" : { - "testattr" : "test", - "shard_indexing_pressure_enabled" : "true" - }, - "tasks" : { - "CRqNwnEeRXOjeTSYYktw-A:677" : { - "node" : "CRqNwnEeRXOjeTSYYktw-A", - "id" : 677, - "type" : "transport", - "action" : "indices:data/read/search", - "description" : "indices[], search_type[QUERY_THEN_FETCH], source[{\"query\":{\"query_string\":}}]", - "start_time_in_millis" : 1660106254525, - "running_time_in_nanos" : 1354236, - "cancellable" : true, - "cancelled" : false, - "headers" : { }, - "resource_stats" : { - "average" : { - "cpu_time_in_nanos" : 0, - "memory_in_bytes" : 0 - }, - "total" : { - "cpu_time_in_nanos" : 0, - "memory_in_bytes" : 0 - }, - "min" : { - "cpu_time_in_nanos" : 0, - "memory_in_bytes" : 0 - }, - "max" : { - "cpu_time_in_nanos" : 0, - "memory_in_bytes" : 0 - }, - "thread_info" : { - "thread_executions" : 0, - "active_threads" : 0 - } - } - } - } - } - } -} - -``` ### The `resource_stats` object diff --git a/templates/API_TEMPLATE.md b/templates/API_TEMPLATE.md index c4c46fc5ce..02c0f341d9 100644 --- a/templates/API_TEMPLATE.md +++ b/templates/API_TEMPLATE.md @@ -30,13 +30,13 @@ The following table lists the available path parameters. All path parameters are The following table lists the available query parameters. All query parameters are optional. -| Parameter | Data type | Description | +| Parameter | Data type | Description | | :--- | :--- | :--- | | `query_parameter` | String | Example query parameter description. Default is ... | -## Request fields +## Request body fields -The following table lists the available request fields. +The following table lists the available request body fields. | Field | Data type | Description | | :--- | :--- | :--- | @@ -44,7 +44,13 @@ The following table lists the available request fields. | `example_object.required_request_field` | Type | Required request field description. Required. | | `example_object.optional_request_field` | Type | Optional request field description. Optional. Default is ... | -#### Example request +## Example request(s) + +**TIP:** If multiple examples exist for the request, seperate those examples using an `h3` header underneath this section. + +### Request with an example object + +The following example shows an API request with an example object: ```json POST /_example/endpoint/ @@ -57,7 +63,21 @@ POST /_example/endpoint/ ``` {% include copy-curl.html %} -#### Example response +## Request without an example object + +The following example shows an API request without an example object: + +```json +POST /_example/endpoint/ +``` +{% include copy-curl.html %} + + +## Example response + +**TIP:** If multiple response examples exist for the request, seperate those examples using an `h3` header underneath this section, similar to the [Example requests](#example-requests). + +The following example shows an API response:
@@ -76,9 +96,9 @@ POST /_example/endpoint/ ```
-## Response fields +## Response body fields -The following table lists all response fields. +The following table lists all response body fields. | Field | Data type | Description | | :--- | :--- | :--- | @@ -87,3 +107,5 @@ The following table lists all response fields. ## Required permissions If you use the Security plugin, make sure you have the appropriate permissions: `cluster:example/permission/name`. + + From beb3bc225ac83b19e0d4545ea1fb5285b4d062d9 Mon Sep 17 00:00:00 2001 From: zane-neo Date: Thu, 25 Jul 2024 22:33:36 +0800 Subject: [PATCH 34/36] Add disk free space cluster settings (#7799) * Add disk free space cluster settings Signed-off-by: zane-neo * Add value range to disk free space setting Signed-off-by: zane-neo * Update _ml-commons-plugin/cluster-settings.md Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> --------- Signed-off-by: zane-neo Signed-off-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: kolchfa-aws <105444904+kolchfa-aws@users.noreply.github.com> Co-authored-by: Nathan Bower --- _ml-commons-plugin/cluster-settings.md | 17 +++++++++++++++++ 1 file changed, 17 insertions(+) diff --git a/_ml-commons-plugin/cluster-settings.md b/_ml-commons-plugin/cluster-settings.md index 0c1f433bf2..efb13dd73a 100644 --- a/_ml-commons-plugin/cluster-settings.md +++ b/_ml-commons-plugin/cluster-settings.md @@ -256,6 +256,23 @@ plugins.ml_commons.jvm_heap_memory_threshold: 85 - Default value: 85 - Value range: [0, 100] +## Set a disk free space threshold + +Sets a disk circuit breaker that checks disk usage before running an ML task. If the amount of disk free space exceeds the threshold, then OpenSearch triggers a circuit breaker and throws an exception to maintain optimal performance. + +Valid values are in byte units. To disable the circuit breaker, set this value to -1. + +### Setting + +``` +plugins.ml_commons.disk_free_space_threshold: 5G +``` + +### Values + +- Default value: 5G +- Value range: [-1, Long.MAX_VALUE] + ## Exclude node names Use this setting to specify the names of nodes on which you don't want to run ML tasks. The value should be a valid node name or a comma-separated node name list. From b56abe2fce09a7f2b54da00d7b176b4a67035463 Mon Sep 17 00:00:00 2001 From: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Date: Thu, 25 Jul 2024 10:48:36 -0500 Subject: [PATCH 35/36] Fix alias page (#7831) Signed-off-by: Archer --- _api-reference/index-apis/update-alias.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/_api-reference/index-apis/update-alias.md b/_api-reference/index-apis/update-alias.md index cac05ceedb..f32d34025e 100644 --- a/_api-reference/index-apis/update-alias.md +++ b/_api-reference/index-apis/update-alias.md @@ -1,11 +1,12 @@ --- layout: default -title: Create or Update Alias +title: Create or update alias parent: Index APIs nav_order: 5 --- # Create or Update Alias + **Introduced 1.0** {: .label .label-purple } @@ -16,7 +17,7 @@ The Create or Update Alias API is distinct from the [Alias API]({{site.url}}{{si ## Path and HTTP methods -``` +```json POST //_alias/ PUT //_alias/ POST /_alias/ From fdfd53f29f0d6ea1a04761e33b8fc837b25be770 Mon Sep 17 00:00:00 2001 From: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Date: Thu, 25 Jul 2024 12:19:07 -0500 Subject: [PATCH 36/36] Add Segment API (#7768) * Add Segment API Signed-off-by: Archer * Update segment.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Heather Halter Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update _api-reference/index-apis/segment.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update _api-reference/index-apis/segment.md Co-authored-by: Heather Halter Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Update segment.md Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> * Apply suggestions from code review Co-authored-by: Nathan Bower Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> --------- Signed-off-by: Archer Signed-off-by: Naarcha-AWS <97990722+Naarcha-AWS@users.noreply.github.com> Co-authored-by: Heather Halter Co-authored-by: Nathan Bower --- _api-reference/index-apis/segment.md | 122 +++++++++++++++++++++++++++ 1 file changed, 122 insertions(+) create mode 100644 _api-reference/index-apis/segment.md diff --git a/_api-reference/index-apis/segment.md b/_api-reference/index-apis/segment.md new file mode 100644 index 0000000000..a8a7ccaee1 --- /dev/null +++ b/_api-reference/index-apis/segment.md @@ -0,0 +1,122 @@ +--- +layout: default +title: Segment +parent: Index APIs +nav_order: 64 +--- + +# Segment +Introduced 1.0 +{: .label .label-purple } + +The Segment API provides details about the Lucene segments within index shards as well as information about the backing indexes of data streams. + + +## Path and HTTP methods + +```json +GET //_segments +GET /_segments +``` + +## Path parameters + +The following table lists the available path parameters. All path parameters are optional. + +Parameter | Data type | Description +:--- | :--- | :--- +`` | String | A comma-separated list of indexes, data streams, or index aliases to which the operation is applied. Supports wildcard expressions (`*`). Use `_all` or `*` to specify all indexes and data streams in a cluster. | + +## Query parameters + +The Segment API supports the following optional query parameters. + +Parameter | Data type | Description +:--- | :--- | :--- +`allow_no_indices` | Boolean | Whether to ignore wildcards that don't match any indexes. Default is `true`. +`allow_partial_search_results` | Boolean | Whether to return partial results if the request encounters an error or times out. Default is `true`. +`expand_wildcards` | String | Specifies the type of index that wildcard expressions can match. Supports comma-separated values. Valid values are `all` (match any index), `open` (match open, non-hidden indexes), `closed` (match closed, non-hidden indexes), `hidden` (match hidden indexes), and `none` (deny wildcard expressions). Default is `open`. +`ignore_unavailable` | Boolean | When `true`, OpenSearch ignores missing or closed indexes. If `false`, OpenSearch returns an error if the force merge operation encounters missing or closed indexes. Default is `false`. +`verbose` | Boolean | When `true`, provides information about Lucene's memory usage. Default is `false`. + +## Response body fields + +Parameter | Data type | Description + :--- | :--- | :--- +`` | String | The name of the segment used to create internal file names in the shard directory. +`generation` | Integer | The generation number, such as `0`, incremented for each written segment and used to name the segment. +`num_docs` | Integer | The number of documents, obtained from Lucene. Nested documents are counted separately from their parents. Deleted documents, as well as recently indexed documents that are not yet assigned to a segment, are excluded. +`deleted_docs` | Integer | The number of deleted documents, obtained from Lucene, which may not match the actual number of delete operations performed. Recently deleted documents that are not yet assigned to a segment are excluded. Deleted documents are automatically merged when appropriate. OpenSearch will occasionally delete extra documents in order to track recent shard operations. +`size_in_bytes` | Integer | The amount of disk space used by the segment, for example, `50kb`. +`memory_in_bytes` | Integer | The amount of segment data, measured in bytes, that is kept in memory to facilitate efficient search operations, such as `1264`. A value of `-1` indicates that OpenSearch was unable to compute this number. +`committed` | Boolean | When `true`, the segments are synced to disk. Segments synced to disk can survive a hard reboot. If `false`, then uncommitted segment data is stored in the transaction log as well so that changes can be replayed at the next startup. +`search` | Boolean | When `true`, segment search is enabled. When `false`, the segment may have already been written to disk and require a refresh in order to be searchable. +`version` | String | The Lucene version used to write the segment. +`compound` | Boolean | When `true`, indicates that Lucene merged all segment files into one file in order to save any file descriptions. +`attributes` | Object | Shows if high compression was enabled. + +## Example requests + +The following example requests show you how to use the Segment API. + +### Specific data stream or index + +```json +GET /index1/_segments +``` +{% include copy-curl.html %} + +### Several data streams and indexes + +```json +GET /index1,index2/_segments +``` +{% include copy-curl.html %} + +### All data streams and indexes in a cluster + +```json +GET /_segments +``` +{% include copy-curl.html %} + +## Example response + +```json +{ + "_shards": ... + "indices": { + "test": { + "shards": { + "0": [ + { + "routing": { + "state": "STARTED", + "primary": true, + "node": "zDC_RorJQCao9xf9pg3Fvw" + }, + "num_committed_segments": 0, + "num_search_segments": 1, + "segments": { + "_0": { + "generation": 0, + "num_docs": 1, + "deleted_docs": 0, + "size_in_bytes": 3800, + "memory_in_bytes": 1410, + "committed": false, + "search": true, + "version": "7.0.0", + "compound": true, + "attributes": { + } + } + } + } + ] + } + } + } +} +``` +