Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rename reader context to search context #56351

Merged
merged 2 commits into from
May 8, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -861,8 +861,8 @@ public void testApiNamingConventions() throws Exception {
"indices.create_data_stream",
"indices.get_data_streams",
"indices.delete_data_stream",
"clear_reader",
"open_reader"
"open_search_context",
"close_search_context",
};
//These API are not required for high-level client feature completeness
String[] notRequiredApi = new String[] {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,24 +1,24 @@
[[reader-context]]
==== Reader Context
[[search-context]]
==== Search Context

By default, a search request executes against the latest point in time readers of the
target indices. Sometimes it's preferred to execute multiple search requests using
the same point in time readers. For example, the combined result of the initial and
subsequent search_after requests is more consistent if they use the same point in time
readers.
readers. This can be done by using a single search context for all search requests.

Point in time readers must be opened in a separate step before being used in subsequent
A search context must be opened in a separate step before being used in subsequent
search requests. The keep_alive parameter tells Elasticsearch how long it should keep
the ``reader_context'' alive, e.g. `?keep_alive=5m`.
the seach context alive, e.g. `?keep_alive=5m`.

[source,console]
--------------------------------------------------
POST /twitter/_open_reader?keep_alive=1m
POST /twitter/_search_context?keep_alive=1m
--------------------------------------------------
// TEST[setup:twitter]

The result from the above request includes a `reader_id`, which should
be passed to the `id` of the `reader` parameter of a search request.
The result from the above request includes a `id`, which should
be passed to the `id` of the `search_context` parameter of a search request.

POST /_search <1>
{
Expand All @@ -28,63 +28,63 @@ POST /_search <1>
"title" : "elasticsearch"
}
},
"reader": {
"search_context": {
"id": "46ToAwEHdHdpdHRlchZSa1dRZDFELVNLbTZkQVJudUtGMFFnAAAWNVAwS09JWTdTRUdibWE1ZC1id0tjQRRURnhPWjNFQmI3bmVOeVZReS1tRAAAAAAAAAAB", <2>
"keep_alive": "1m" <3>
}
}

<1> A search request with `reader.id` must not specify `index`, `routing`,
and `preferences` as these parameters are copied from the `reader.id`.
<2> The `reader.id` parameter tells Elasticsearch to execute the request using
the point-in-time readers from this id.
<3> The `reader.keep_alive` parameter tells Elasticsearch how long it should extend
the time to live of these readers.

IMPORTANT: The open_reader request and each subsequent search request can return
different `reader_id`; thus always use the most recently received `reader_id`
for the next search request.

[[reader-keep-alive]]
===== Keeping the reader context alive
The `keep_alive` parameter (passed to a `open_reader` request and search
request) extends the time to live of open reader contexts. The value
<1> A search request with `search_context` must not specify `index`, `routing`,
and `preferences` as these parameters are copied from the `search_context`.
<2> The `id` parameter tells Elasticsearch to execute the request using
the point-in-time readers from this search context id.
<3> The `keep_alive` parameter tells Elasticsearch how long it should extend
the time to live of the search context

IMPORTANT: The open search context request and each subsequent search request can
return different `id`; thus always use the most recently received `id` for the
next search request.

[[search-context-keep-alive]]
===== Keeping the search context alive
The `keep_alive` parameter, which is passed to a open search context request and
search request, extends the time to live of the search context. The value
(e.g. `1m`, see <<time-units>>) does not need to be long enough to
process all data -- it just needs to be long enough for the next request.

Normally, the background merge process optimizes the index by merging together
smaller segments to create new, bigger segments. Once the smaller segments are
no longer needed they are deleted. However, open reader contexts prevents the
no longer needed they are deleted. However, open search contexts prevents the
old segments from being deleted since they are still in use.

TIP: Keeping older segments alive means that more disk space and file handles
are needed. Ensure that you have configured your nodes to have ample free file
handles. See <<file-descriptors>>.

Additionally, if a segment contains deleted or updated documents then the reader
Additionally, if a segment contains deleted or updated documents then the search
context must keep track of whether each document in the segment was live at the
time of the initial search request. Ensure that your nodes have sufficient heap
space if you have many open reader contexts on an index that is subject to ongoing
space if you have many open search contexts on an index that is subject to ongoing
deletes or updates.

You can check how many reader contexts are open with the
You can check how many search contexts are open with the
<<cluster-nodes-stats,nodes stats API>>:

[source,console]
---------------------------------------
GET /_nodes/stats/indices/search
---------------------------------------

===== Clear reader API
===== Close search context API

Reader contexts are automatically closed when the `reader.keep_alive` has
been elapsed. However keeping readers open has a cost, as discussed in the
<<reader-keep-alive,previous section>> so readers should be explicitly
closed as soon as they are no longer used in search requests.
Search contexts are automatically closed when the `keep_alive` has
been elapsed. However keeping search contexts as a cost, as discussed in the
<<search-context-keep-alive,previous section>> so search context should be
explicitly closed as soon as they are no longer used in search requests.

[source,console]
---------------------------------------
DELETE /_search/reader
DELETE /_search_context
{
"id" : "46ToAwEHdHdpdHRlchZSa1dRZDFELVNLbTZkQVJudUtGMFFnAAAWNVAwS09JWTdTRUdibWE1ZC1id0tjQRRURnhPWjNFQmI3bmVOeVZReS1tRAAAAAAAAAAB"
}
Expand Down
Original file line number Diff line number Diff line change
@@ -1,14 +1,14 @@
{
"clear_reader":{
"close_search_context":{
"documentation":{
"url":"https://www.elastic.co/guide/en/elasticsearch/reference/master/reader-context.html",
"description":"Explicitly close the point in time readers"
"url":"https://www.elastic.co/guide/en/elasticsearch/reference/master/search-context.html",
"description":"Close a search context"
},
"stability":"beta",
"url":{
"paths":[
{
"path":"/_search/reader",
"path":"/_search_context",
"methods":[
"DELETE"
]
Expand All @@ -17,7 +17,7 @@
},
"params":{},
"body":{
"description": "point in time readers to clear"
"description": "a search context to close"
}
}
}
Original file line number Diff line number Diff line change
@@ -1,27 +1,27 @@
{
"open_reader":{
"open_search_context":{
"documentation":{
"url":"https://www.elastic.co/guide/en/elasticsearch/reference/master/reader-context.html",
"description":"Open and keep point-in-time readers that can be used in subsequent search requests"
"url":"https://www.elastic.co/guide/en/elasticsearch/reference/master/search-context.html",
"description":"Open a search context that can be used in subsequent searches"
},
"stability":"beta",
"url":{
"paths":[
{
"path":"/_open_reader",
"path":"/_search_context",
"methods":[
"POST"
]
},
{
"path":"/{index}/_open_reader",
"path":"/{index}/_search_context",
"methods":[
"POST"
],
"parts":{
"index":{
"type":"list",
"description":"A comma-separated list of index names to search; use `_all` or empty string to perform the operation on all indices"
"description":"A comma-separated list of index names to open search context; use `_all` or empty string to perform the operation on all indices"
}
}
}
Expand Down Expand Up @@ -54,7 +54,7 @@
},
"keep_alive": {
"type": "string",
"description": "Specific the time to live for the point-in-time readers"
"description": "Specific the time to live for the search context"
}
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -25,15 +25,15 @@ setup:
index: test

---
"search with reader context":
"search context":
- skip:
version: " - 7.9.99"
reason: "reader context is introduced in 8.0"
reason: "search context is introduced in 8.0"
- do:
open_reader:
open_search_context:
index: test
keep_alive: 1m
- set: {reader_id: reader_id}
keep_alive: 5m
- set: {id: search_context_id}

- do:
search:
Expand All @@ -44,8 +44,8 @@ setup:
match:
foo: bar
sort: [{ age: desc }, { id: desc }]
reader:
id: "$reader_id"
search_context:
id: "$search_context_id"
keep_alive: 1m

- match: {hits.total: 3 }
Expand All @@ -63,7 +63,7 @@ setup:
indices.refresh:
index: test

# search with the retaining readers
# search with the retaining search context
- do:
search:
rest_total_hits_as_int: true
Expand All @@ -74,8 +74,8 @@ setup:
foo: bar
sort: [{ age: desc }, { id: desc }]
search_after: [24, 172]
reader:
id: "$reader_id"
search_context:
id: "$search_context_id"
keep_alive: 1m

- match: {hits.total: 3 }
Expand All @@ -94,8 +94,8 @@ setup:
foo: bar
sort: [ { age: desc }, { id: desc } ]
search_after: [18, 42]
reader:
id: "$reader_id"
search_context:
id: "$search_context_id"
keep_alive: 1m

- match: {hits.total: 3}
Expand All @@ -114,14 +114,14 @@ setup:
foo: bar
sort: [{ age: desc }, { id: desc } ]
search_after: [18, 1]
reader:
id: "$reader_id"
search_context:
id: "$search_context_id"
keep_alive: 1m

- match: {hits.total: 3}
- length: {hits.hits: 0 }

- do:
clear_reader:
close_search_context:
body:
id: "$reader_id"
id: "$search_context_id"
Loading