diff --git a/docs/reference/aggregations/bucket/composite-aggregation.asciidoc b/docs/reference/aggregations/bucket/composite-aggregation.asciidoc index e10e8d6ef6c5b..f8c3a1b898b4e 100644 --- a/docs/reference/aggregations/bucket/composite-aggregation.asciidoc +++ b/docs/reference/aggregations/bucket/composite-aggregation.asciidoc @@ -6,7 +6,7 @@ A multi-bucket aggregation that creates composite buckets from different sources Unlike the other `multi-bucket` aggregation the `composite` aggregation can be used to paginate **all** buckets from a multi-level aggregation efficiently. This aggregation provides a way to stream **all** buckets of a specific aggregation similarly to what -<> does for documents. +<> does for documents. The composite buckets are built from the combinations of the values extracted/created for each document and each combination is considered as diff --git a/docs/reference/how-to/general.asciidoc b/docs/reference/how-to/general.asciidoc index ac6760ee8d2ad..4a9331194d902 100644 --- a/docs/reference/how-to/general.asciidoc +++ b/docs/reference/how-to/general.asciidoc @@ -9,7 +9,7 @@ Elasticsearch is designed as a search engine, which makes it very good at getting back the top documents that match a query. However, it is not as good for workloads that fall into the database domain, such as retrieving all documents that match a particular query. If you need to do this, make sure to -use the <> API. +use the <> API. [discrete] [[maximum-document-size]] diff --git a/docs/reference/index-modules.asciidoc b/docs/reference/index-modules.asciidoc index 5fa68df9d2e64..c8e753823d19f 100644 --- a/docs/reference/index-modules.asciidoc +++ b/docs/reference/index-modules.asciidoc @@ -155,7 +155,7 @@ specific index module: The maximum value of `from + size` for searches to this index. Defaults to `10000`. Search requests take heap memory and time proportional to `from + size` and this limits that memory. See - <> or <> for a more efficient alternative + <> or <> for a more efficient alternative to raising this. `index.max_inner_result_window`:: diff --git a/docs/reference/modules/cross-cluster-search.asciidoc b/docs/reference/modules/cross-cluster-search.asciidoc index 374cda8a283c7..2cfcc746f5749 100644 --- a/docs/reference/modules/cross-cluster-search.asciidoc +++ b/docs/reference/modules/cross-cluster-search.asciidoc @@ -292,7 +292,7 @@ handling network delays: By default, {es} reduces the number of network roundtrips between remote clusters. This reduces the impact of network delays on search speed. However, {es} can't reduce network roundtrips for large search requests, such as those -including a <> or +including a <> or <>. + See <> to learn how this option works. diff --git a/docs/reference/redirects.asciidoc b/docs/reference/redirects.asciidoc index 3181872d4310c..e240e8fe594c2 100644 --- a/docs/reference/redirects.asciidoc +++ b/docs/reference/redirects.asciidoc @@ -111,11 +111,11 @@ See <>. [role="exclude",id="search-request-scroll"] === Scroll parameter for request body search API -See <>. +See <>. [role="exclude",id="search-request-search-after"] === Search after parameter for request body search API -See <>. +See <>. [role="exclude",id="search-request-search-type"] === Search type parameter for request body search API @@ -837,7 +837,7 @@ See <>. [role="exclude",id="scan-scroll"] === Scan and scroll -See <>. +See <>. [role="exclude",id="mapping-dynamic-mapping"] === Dynamic mapping @@ -969,6 +969,16 @@ See <>. See <>. +[role="exclude",id="request-body-search-scroll"] +==== Scroll + +See <>. + +[role="exclude",id="request-body-search-search-after"] +==== Search After + +See <>. + [role="exclude",id="request-body-search-sort"] ==== Sort diff --git a/docs/reference/search/async-search.asciidoc b/docs/reference/search/async-search.asciidoc index 3c788f704bc96..72241e2387848 100644 --- a/docs/reference/search/async-search.asciidoc +++ b/docs/reference/search/async-search.asciidoc @@ -129,7 +129,7 @@ query get skipped. * `ccs_minimize_roundtrips` defaults to `false`, which is also the only supported value -WARNING: Async search does not support <> +WARNING: Async search does not support <> nor search requests that only include the <>. {ccs} is supported only with <> set to `false`. diff --git a/docs/reference/search/clear-scroll-api.asciidoc b/docs/reference/search/clear-scroll-api.asciidoc index 6e5325d658153..6892beebe7372 100644 --- a/docs/reference/search/clear-scroll-api.asciidoc +++ b/docs/reference/search/clear-scroll-api.asciidoc @@ -5,7 +5,7 @@ ++++ Clears the search context and results for a -<>. +<>. //// [source,console] diff --git a/docs/reference/search/request/from-size.asciidoc b/docs/reference/search/paginate-search-results.asciidoc similarity index 85% rename from docs/reference/search/request/from-size.asciidoc rename to docs/reference/search/paginate-search-results.asciidoc index 3848727131b9c..6e23966418ca4 100644 --- a/docs/reference/search/request/from-size.asciidoc +++ b/docs/reference/search/paginate-search-results.asciidoc @@ -1,8 +1,7 @@ -[discrete] [[paginate-search-results]] -=== Paginate search results +== Paginate search results -By default, the search API returns the top 10 matching documents. +By default, the <> returns the top 10 matching documents. To paginate through a larger set of results, you can use the search API's `size` and `from` parameters. The `size` parameter is the number of matching documents @@ -40,10 +39,14 @@ results must then be combined and sorted to ensure that the overall sort order is correct. As an alternative to deep paging, we recommend using -<> or the -<> parameter. +<> or the +<> parameter. WARNING: {es} uses Lucene's internal doc IDs as tie-breakers. These internal doc IDs can be completely different across replicas of the same data. When paginating, you might occasionally see that documents with the same sort values are not ordered consistently. + +include::request/scroll.asciidoc[] + +include::request/search-after.asciidoc[] \ No newline at end of file diff --git a/docs/reference/search/request-body.asciidoc b/docs/reference/search/request-body.asciidoc index 8cc8b8c1c0963..abb68c0ce4c44 100644 --- a/docs/reference/search/request-body.asciidoc +++ b/docs/reference/search/request-body.asciidoc @@ -133,9 +133,25 @@ include::request/rescore.asciidoc[] include::request/script-fields.asciidoc[] -include::request/scroll.asciidoc[] +[[request-body-search-scroll]] +==== Scroll -include::request/search-after.asciidoc[] +See <>. + +[[_clear_scroll_api]] +===== Clear scroll API + +See <>. + +[[sliced-scroll]] +===== Sliced scroll + +See <>. + +[[request-body-search-search-after]] +==== Search After + +See <>. include::request/search-type.asciidoc[] diff --git a/docs/reference/search/request/collapse.asciidoc b/docs/reference/search/request/collapse.asciidoc index 9780116f61ab9..b97c4044ec0c0 100644 --- a/docs/reference/search/request/collapse.asciidoc +++ b/docs/reference/search/request/collapse.asciidoc @@ -116,8 +116,8 @@ The `max_concurrent_group_searches` request parameter can be used to control the maximum number of concurrent searches allowed in this phase. The default is based on the number of data nodes and the default search thread pool size. -WARNING: `collapse` cannot be used in conjunction with <>, -<> or <>. +WARNING: `collapse` cannot be used in conjunction with <>, +<> or <>. [discrete] [[second-level-of-collapsing]] diff --git a/docs/reference/search/request/scroll.asciidoc b/docs/reference/search/request/scroll.asciidoc index 54f2da57c9c2c..dfe2193e47dcc 100644 --- a/docs/reference/search/request/scroll.asciidoc +++ b/docs/reference/search/request/scroll.asciidoc @@ -1,5 +1,6 @@ -[[request-body-search-scroll]] -==== Scroll +[discrete] +[[scroll-search-results]] +=== Scroll search results While a `search` request returns a single ``page'' of results, the `scroll` API can be used to retrieve large numbers of results (or even all results) @@ -101,8 +102,9 @@ GET /_search?scroll=1m -------------------------------------------------- // TEST[setup:twitter] +[discrete] [[scroll-search-context]] -===== Keeping the search context alive +==== Keeping the search context alive A scroll returns all the documents which matched the search at the time of the initial search request. It ignores any subsequent changes to these documents. @@ -148,7 +150,9 @@ You can check how many search contexts are open with the GET /_nodes/stats/indices/search --------------------------------------- -===== Clear scroll API +[discrete] +[[clear-scroll]] +==== Clear scroll Search context are automatically removed when the `scroll` timeout has been exceeded. However keeping scrolls open has a cost, as discussed in the @@ -195,8 +199,9 @@ DELETE /_search/scroll/DXF1ZXJ5QW5kRmV0Y2gBAAAAAAAAAD4WYm9laVYtZndUQlNsdDcwakFMN --------------------------------------- // TEST[catch:missing] -[[sliced-scroll]] -===== Sliced Scroll +[discrete] +[[slice-scroll]] +==== Sliced Scroll For scroll queries that return a lot of documents it is possible to split the scroll in multiple slices which can be consumed independently: diff --git a/docs/reference/search/request/search-after.asciidoc b/docs/reference/search/request/search-after.asciidoc index 3196ac6dae2be..4e02abec363ae 100644 --- a/docs/reference/search/request/search-after.asciidoc +++ b/docs/reference/search/request/search-after.asciidoc @@ -1,9 +1,10 @@ -[[request-body-search-search-after]] -==== Search After +[discrete] +[[search-after]] +=== Search after Pagination of results can be done by using the `from` and `size` but the cost becomes prohibitive when the deep pagination is reached. The `index.max_result_window` which defaults to 10,000 is a safeguard, search requests take heap memory and time proportional to `from + size`. -The <> api is recommended for efficient deep scrolling but scroll contexts are costly and it is not +The <> API is recommended for efficient deep scrolling but scroll contexts are costly and it is not recommended to use it for real time user requests. The `search_after` parameter circumvents this problem by providing a live cursor. The idea is to use the results from the previous page to help the retrieval of the next page. diff --git a/docs/reference/search/request/sort.asciidoc b/docs/reference/search/request/sort.asciidoc index e6b2bd4b895cf..4b7879d3ab2b6 100644 --- a/docs/reference/search/request/sort.asciidoc +++ b/docs/reference/search/request/sort.asciidoc @@ -46,7 +46,7 @@ GET /my-index-000001/_search NOTE: `_doc` has no real use-case besides being the most efficient sort order. So if you don't care about the order in which documents are returned, then you -should sort by `_doc`. This especially helps when <>. +should sort by `_doc`. This especially helps when <>. [discrete] === Sort Values diff --git a/docs/reference/search/scroll-api.asciidoc b/docs/reference/search/scroll-api.asciidoc index 6b16e63993005..6c3f9d86de199 100644 --- a/docs/reference/search/scroll-api.asciidoc +++ b/docs/reference/search/scroll-api.asciidoc @@ -4,7 +4,7 @@ Scroll ++++ -Retrieves the next batch of results for a <>. //// @@ -48,7 +48,7 @@ deprecated:[7.0.0] ==== {api-description-title} You can use the scroll API to retrieve large sets of results from a single -<> request. +<> request. The scroll API requires a scroll ID. To get a scroll ID, submit a <> request that includes an argument for the @@ -63,7 +63,7 @@ next batch of results for the request. You can also use the scroll API to specify a new `scroll` parameter that extends or shortens the retention period for the search context. -See <>. +See <>. IMPORTANT: Results from a scrolling search reflect the state of the index at the time of the initial search request. Subsequent indexing or document changes only @@ -86,7 +86,7 @@ the <>. `scroll`:: (Optional, <>) Period to retain the <> for scrolling. See -<>. +<>. + This value overrides the duration set by the original search API request's `scroll` parameter. @@ -118,7 +118,7 @@ Defaults to `false`. `scroll`:: (Optional, <>) Period to retain the <> for scrolling. See -<>. +<>. + This value overrides the duration set by the original search API request's `scroll` parameter. diff --git a/docs/reference/search/search-your-data.asciidoc b/docs/reference/search/search-your-data.asciidoc index e4f4905a993c5..4ce44071aeab8 100644 --- a/docs/reference/search/search-your-data.asciidoc +++ b/docs/reference/search/search-your-data.asciidoc @@ -292,10 +292,10 @@ GET /*/_search } ---- -include::request/from-size.asciidoc[] include::search-fields.asciidoc[] include::request/collapse.asciidoc[] include::request/highlighting.asciidoc[] +include::paginate-search-results.asciidoc[] include::request/sort.asciidoc[] include::{es-repo-dir}/async-search.asciidoc[] include::{es-repo-dir}/modules/cross-cluster-search.asciidoc[] diff --git a/docs/reference/search/search.asciidoc b/docs/reference/search/search.asciidoc index 71149d27e1804..a32ccf46e3ef9 100644 --- a/docs/reference/search/search.asciidoc +++ b/docs/reference/search/search.asciidoc @@ -95,8 +95,8 @@ results must then be combined and sorted to ensure that the overall order is correct. As an alternative to deep paging, we recommend using -<> or the -<> parameter. +<> or the +<> parameter. -- `ignore_throttled`:: @@ -153,7 +153,7 @@ include::{es-repo-dir}/rest-api/common-parms.asciidoc[tag=routing] `scroll`:: (Optional, <>) Period to retain the <> for scrolling. See -<>. +<>. + By default, this value cannot exceed `1d` (24 hours). You can change this limit using the `search.max_keep_alive` cluster-level setting. @@ -179,8 +179,8 @@ results must then be combined and sorted to ensure that the overall order is correct. As an alternative to deep paging, we recommend using -<> or the -<> parameter. +<> or the +<> parameter. If the <> is specified, this value cannot be `0`. @@ -316,8 +316,8 @@ results must then be combined and sorted to ensure that the overall order is correct. As an alternative to deep paging, we recommend using -<> or the -<> parameter. +<> or the +<> parameter. -- [[request-body-search-query]] @@ -345,8 +345,8 @@ results must then be combined and sorted to ensure that the overall order is correct. As an alternative to deep paging, we recommend using -<> or the -<> parameter. +<> or the +<> parameter. If the <> is specified, this value cannot be `0`. @@ -422,7 +422,7 @@ Identifier for the search and its <>. + You can use this scroll ID with the <> to retrieve the next batch of search results for the request. See -<>. +<>. + This parameter is only returned if the <> is specified in the request. diff --git a/docs/reference/sql/endpoints/rest.asciidoc b/docs/reference/sql/endpoints/rest.asciidoc index 51e5aaa9fbffe..9ea3055321a95 100644 --- a/docs/reference/sql/endpoints/rest.asciidoc +++ b/docs/reference/sql/endpoints/rest.asciidoc @@ -308,7 +308,7 @@ Which looks like: Note that the `columns` object is only part of the first page. You've reached the last page when there is no `cursor` returned -in the results. Like Elasticsearch's <>, +in the results. Like Elasticsearch's <>, SQL may keep state in Elasticsearch to support the cursor. Unlike scroll, receiving the last page is enough to guarantee that the Elasticsearch state is cleared. diff --git a/docs/reference/sql/endpoints/translate.asciidoc b/docs/reference/sql/endpoints/translate.asciidoc index 1312aea8ecaec..fdccbf00956b4 100644 --- a/docs/reference/sql/endpoints/translate.asciidoc +++ b/docs/reference/sql/endpoints/translate.asciidoc @@ -49,7 +49,7 @@ Which returns: -------------------------------------------------- Which is the request that SQL will run to provide the results. -In this case, SQL will use the <> +In this case, SQL will use the <> API. If the result contained an aggregation then SQL would use the normal <> API. diff --git a/rest-api-spec/src/main/resources/rest-api-spec/api/clear_scroll.json b/rest-api-spec/src/main/resources/rest-api-spec/api/clear_scroll.json index f484c94246c7b..b0e50045cd7cd 100644 --- a/rest-api-spec/src/main/resources/rest-api-spec/api/clear_scroll.json +++ b/rest-api-spec/src/main/resources/rest-api-spec/api/clear_scroll.json @@ -1,7 +1,7 @@ { "clear_scroll":{ "documentation":{ - "url":"https://www.elastic.co/guide/en/elasticsearch/reference/master/search-request-body.html#_clear_scroll_api", + "url":"https://www.elastic.co/guide/en/elasticsearch/reference/current/clear-scroll-api.html", "description":"Explicitly clears the search context for a scroll." }, "stability":"stable",