Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Introduce search context - point in time view of indices #56480

Closed
wants to merge 98 commits into from
Closed
Changes from 1 commit
Commits
Show all changes
98 commits
Select commit Hold shift + click to select a range
3ce9a3b
Cut over from SearchContext to ReaderContext (#51282)
dnhatn Jan 27, 2020
f797bec
Merge branch 'master' into feature/reader-context
dnhatn Jan 29, 2020
d2c651e
Merge branch 'master' into feature/reader-context
dnhatn Feb 5, 2020
e6a5bec
Revert "Cut over from SearchContext to ReaderContext (#51282)"
dnhatn Feb 16, 2020
a45ca7b
Merge branch 'master' into feature/reader-context
dnhatn Feb 16, 2020
ea528f1
Merge branch 'master' into feature/reader-context
dnhatn Feb 19, 2020
8b710d1
Merge branch 'master' into feature/reader-context
dnhatn Feb 21, 2020
fb7386a
Cut over from SearchContext to ReaderContext (#51282)
dnhatn Jan 27, 2020
222d5f3
Merge branch 'master' into feature/reader-context
dnhatn Feb 22, 2020
aade8b6
Merge branch 'master' into feature/reader-context
dnhatn Feb 26, 2020
cdbfb67
Merge branch 'master' into feature/reader-context
dnhatn Feb 28, 2020
f921838
Merge branch 'master' into feature/reader-context
dnhatn Mar 2, 2020
206381e
Move states of search to coordinating node (#52741)
dnhatn Mar 3, 2020
6805c5e
Merge branch 'master' into feature/reader-context
dnhatn Mar 13, 2020
395e2a9
Adjust SearchService after merging from master
dnhatn Mar 13, 2020
b4ffd2e
Merge branch 'master' into feature/reader-context
dnhatn Mar 23, 2020
2bcee88
Merge branch 'master' into feature/reader-context
dnhatn Mar 23, 2020
8913369
Allow searches with specific reader contexts (#53989)
dnhatn Mar 26, 2020
70ce5e5
Merge branch 'master' into feature/reader-context
jimczi Mar 26, 2020
5430ab2
fix checkstyle after backport
jimczi Mar 26, 2020
4713339
Merge branch 'master' into feature/reader-context
dnhatn Apr 8, 2020
3edbda9
Merge branch 'master' into feature/reader-context
dnhatn Apr 12, 2020
3b0760b
Adjust hlrc tests after merge
dnhatn Apr 12, 2020
ff4689b
enable bwc
dnhatn Apr 13, 2020
defa2e7
Restore missing rewrite when create search context
dnhatn Apr 14, 2020
1b538c6
Merge branch 'master' into feature/reader-context
dnhatn Apr 14, 2020
437094b
Merge branch 'master' into feature/reader-context
dnhatn Apr 14, 2020
07f5000
Merge branch 'master' into feature/reader-context
dnhatn Apr 15, 2020
c8b0ccb
Merge branch 'master' into feature/reader-context
dnhatn Apr 15, 2020
be30138
Merge branch 'master' into feature/reader-context
dnhatn Apr 17, 2020
ac2e9ac
Merge branch 'master' into feature/reader-context
dnhatn Apr 20, 2020
aa14abe
Add open reader contexts API (#55265)
dnhatn Apr 21, 2020
f879ed5
Merge branch 'master' into feature/reader-context
dnhatn Apr 21, 2020
15f18b1
Merge branch 'master' into feature/reader-context
dnhatn Apr 26, 2020
51f9542
Adds the ability to acquire readers in IndexShard (#54966)
jimczi Apr 27, 2020
5b3a41b
Mark the reader context as used (#55854)
jimczi Apr 28, 2020
d3fc71e
Merge branch 'master' into feature/reader-context
dnhatn Apr 29, 2020
728db34
Merge branch 'master' into feature/reader-context
dnhatn Apr 29, 2020
8b9878b
Merge branch 'master' into feature/reader-context
dnhatn May 1, 2020
3be26ed
Enable can match for search with reader contexts (#56032)
dnhatn May 4, 2020
1ff72dc
Merge branch 'master' into feature/reader-context
dnhatn May 5, 2020
fb3cf40
Merge branch 'master' into feature/reader-context
dnhatn May 6, 2020
00c5ddc
Merge branch 'master' into feature/reader-context
dnhatn May 7, 2020
c2231fe
Move SearchWithReaderContextIT to internalClusterTest
dnhatn May 7, 2020
c6dac42
Rename reader context to search context (#56351)
dnhatn May 8, 2020
9ceef08
Merge branch 'master' into feature/reader-context
dnhatn May 8, 2020
501c677
Merge branch 'master' into feature/reader-context
dnhatn May 8, 2020
a85c6cf
Merge branch 'master' into feature/reader-context
dnhatn May 8, 2020
de53080
Add total hits info to test assertion
dnhatn May 8, 2020
1c36fef
more on renaming
dnhatn May 8, 2020
3d0b127
Merge branch 'master' into feature/reader-context
dnhatn May 9, 2020
13ef912
fix security listener
dnhatn May 9, 2020
6bc453f
simplify validation of reader context
dnhatn May 9, 2020
63a3595
more on renaming
dnhatn May 9, 2020
454f7fb
bwc: remove scroll on failed authorization
dnhatn May 9, 2020
5971c97
Merge branch 'master' into feature/reader-context
dnhatn May 10, 2020
a60ea8d
Merge branch 'master' into feature/reader-context
dnhatn May 12, 2020
085c4ea
Remove reader context if index gets index midway
dnhatn May 12, 2020
23e8194
Merge branch 'master' into feature/reader-context
dnhatn May 13, 2020
aa8ed55
Merge branch 'master' into feature/reader-context
dnhatn May 14, 2020
8572a05
Merge branch 'master' into feature/reader-context
dnhatn May 25, 2020
852f45d
Merge branch 'master' into feature/reader-context
dnhatn May 26, 2020
d654b26
remove unused code
jimczi May 27, 2020
ce60174
Merge branch 'master' into feature/reader-context
dnhatn May 29, 2020
2d9a281
Merge branch 'master' into feature/reader-context
dnhatn Jun 4, 2020
0d0d647
Use IndexShard from reader context (#57384)
dnhatn Jun 4, 2020
3f0fc80
Merge branch 'master' into feature/reader-context
dnhatn Jun 16, 2020
3ae4ff3
Merge branch 'master' into feature/reader-context
dnhatn Jun 17, 2020
f68dc64
Update docs/reference/search/search_context.asciidoc
jimczi Jun 26, 2020
699c4d4
Update docs/reference/search/search_context.asciidoc
jimczi Jun 26, 2020
0bc08e1
address review
jimczi Jun 26, 2020
2027856
Merge branch 'master' into reader-context
jimczi Jun 26, 2020
7b765d2
ensure that we remove the reader context on failures
jimczi Jun 26, 2020
5be41e0
Merge branch 'master' into feature/reader-context
dnhatn Jun 28, 2020
e61bd20
remove reader from the active list during put
dnhatn Jun 29, 2020
44aca91
Ensure open before acquire searcher
dnhatn Jun 29, 2020
e365093
check wrapper once in search supplier
dnhatn Jun 29, 2020
969ddf4
fix test
dnhatn Jun 29, 2020
ae7f1a4
add doc for can match
dnhatn Jun 29, 2020
b3fe074
can_match constant
dnhatn Jun 29, 2020
05dd518
stop execution on query rewrite failures
jimczi Jun 29, 2020
1c8dd51
Merge branch 'master' into feature/reader-context
dnhatn Jul 1, 2020
aeeee70
add javadocs for RescoreDocIds
dnhatn Jul 1, 2020
96fe51f
add AliasFilter in the search context id and rename SearchContextId i…
jimczi Jul 2, 2020
29e304a
fix doc test
dnhatn Jul 2, 2020
1d9bb74
Add response sample for close api
dnhatn Jul 3, 2020
5290201
support wildcard
dnhatn Jul 3, 2020
21da46f
Merge branch 'master' into reader-context
jimczi Jul 3, 2020
92131bd
Merge branch 'master' into feature/reader-context
dnhatn Jul 3, 2020
4a38f0d
Fix AliasFilter serialize
dnhatn Jul 4, 2020
c77d3ca
fix docs
dnhatn Jul 4, 2020
e2a7fef
explain refresh in search context
dnhatn Jul 5, 2020
c2b02cd
Merge branch 'master' into feature/reader-context
dnhatn Jul 7, 2020
ada43b3
rewording search context
dnhatn Jul 7, 2020
2f867dc
apply doc suggestion
dnhatn Jul 20, 2020
c155098
Merge branch 'master' into feature/reader-context
dnhatn Jul 20, 2020
840ca7c
bump bwc version
dnhatn Jul 22, 2020
e2e5edc
Merge branch 'master' into feature/reader-context
dnhatn Jul 23, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 9 additions & 4 deletions server/src/main/java/org/elasticsearch/search/RescoreDocIds.java
Original file line number Diff line number Diff line change
Expand Up @@ -27,6 +27,11 @@
import java.util.Map;
import java.util.Set;

/**
* Since {@link org.elasticsearch.search.internal.SearchContext} no longer hold the states of search, the top K results
* (i.e., documents that will be rescored by query rescorers) need to be serialized/ deserialized between search phases.
* A {@link RescoreDocIds} encapsulates the top K results for each rescorer by its ordinal index.
*/
dnhatn marked this conversation as resolved.
Show resolved Hide resolved
public final class RescoreDocIds implements Writeable {
dnhatn marked this conversation as resolved.
Show resolved Hide resolved
public static final RescoreDocIds EMPTY = new RescoreDocIds(Map.of());

Expand All @@ -40,12 +45,12 @@ public RescoreDocIds(StreamInput in) throws IOException {
docIds = in.readMap(StreamInput::readVInt, i -> i.readSet(StreamInput::readVInt));
}

public Set<Integer> getId(int index) {
return docIds.get(index);
}

@Override
public void writeTo(StreamOutput out) throws IOException {
out.writeMap(docIds, StreamOutput::writeVInt, (o, v) -> o.writeCollection(v, StreamOutput::writeVInt));
}

public Set<Integer> getId(int index) {
return docIds.get(index);
}
}