-
Notifications
You must be signed in to change notification settings - Fork 24.9k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Search enhancement: pinned queries #44345
Conversation
Pinging @elastic/es-search |
...ck/plugin/search-business-rules/src/main/java/org/apache/lucene/search/CappedScoreQuery.java
Outdated
Show resolved
Hide resolved
...ck/plugin/search-business-rules/src/main/java/org/apache/lucene/search/CappedScoreQuery.java
Outdated
Show resolved
Hide resolved
...ck/plugin/search-business-rules/src/main/java/org/apache/lucene/search/CappedScoreQuery.java
Show resolved
Hide resolved
...ck/plugin/search-business-rules/src/main/java/org/apache/lucene/search/CappedScoreQuery.java
Outdated
Show resolved
Hide resolved
...k/plugin/search-business-rules/src/main/java/org/apache/lucene/search/CappedScoreWeight.java
Show resolved
Hide resolved
...ness-rules/src/main/java/org/elasticsearch/xpack/searchbusinessrules/PinnedQueryBuilder.java
Show resolved
Hide resolved
...ness-rules/src/main/java/org/elasticsearch/xpack/searchbusinessrules/PinnedQueryBuilder.java
Outdated
Show resolved
Hide resolved
...ness-rules/src/main/java/org/elasticsearch/xpack/searchbusinessrules/PinnedQueryBuilder.java
Outdated
Show resolved
Hide resolved
...ness-rules/src/main/java/org/elasticsearch/xpack/searchbusinessrules/PinnedQueryBuilder.java
Outdated
Show resolved
Hide resolved
...ness-rules/src/main/java/org/elasticsearch/xpack/searchbusinessrules/PinnedQueryBuilder.java
Outdated
Show resolved
Hide resolved
...ck/plugin/search-business-rules/src/main/java/org/apache/lucene/search/CappedScoreQuery.java
Show resolved
Hide resolved
classname 'org.elasticsearch.xpack.searchbusinessrules.SearchBusinessRules' | ||
extendedPlugins = ['x-pack-core'] | ||
} | ||
archivesBaseName = 'x-pack-searchbusinessrules' |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
why do we have this?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It's based on a cut-and-paste of vectors' build.gradle with modifications where it seemed appropriate.
...ck/plugin/search-business-rules/src/main/java/org/apache/lucene/search/CappedScoreQuery.java
Outdated
Show resolved
Hide resolved
...ck/plugin/search-business-rules/src/main/java/org/apache/lucene/search/CappedScoreQuery.java
Show resolved
Hide resolved
|
||
@Override | ||
public Weight createWeight(IndexSearcher searcher, ScoreMode scoreMode, float boost) throws IOException { | ||
final Weight innerWeight = searcher.createWeight(query, ScoreMode.COMPLETE, 1f); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
should we pass scoreMode rather than hardcode ScoreMode.COMPLETE, and boost
rather than 1f
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
apologies, this was carried over from ConstantScoreQuery too.
...ck/plugin/search-business-rules/src/main/java/org/apache/lucene/search/CappedScoreQuery.java
Outdated
Show resolved
Hide resolved
x-pack/plugin/search-business-rules/src/main/java/org/apache/lucene/search/CappedScorer.java
Outdated
Show resolved
Hide resolved
...ness-rules/src/main/java/org/elasticsearch/xpack/searchbusinessrules/PinnedQueryBuilder.java
Outdated
Show resolved
Hide resolved
...ness-rules/src/main/java/org/elasticsearch/xpack/searchbusinessrules/PinnedQueryBuilder.java
Outdated
Show resolved
Hide resolved
...ness-rules/src/main/java/org/elasticsearch/xpack/searchbusinessrules/PinnedQueryBuilder.java
Outdated
Show resolved
Hide resolved
...ness-rules/src/main/java/org/elasticsearch/xpack/searchbusinessrules/PinnedQueryBuilder.java
Outdated
Show resolved
Hide resolved
organicAndPinned.add(pinnedQueries.build()); | ||
// Cap the scores of the organic query | ||
organicAndPinned.add(new CappedScoreQuery(organicQuery.toQuery(context), MAX_ORGANIC_SCORE)); | ||
return new DisjunctionMaxQuery(organicAndPinned, 0); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
FYI seeing this made me notice that disjunction max queries are a bit inefficient at retrieving top hits at the moment, even though there are some easy wins, especially when the tie-break multiplier is 0, so I opened https://issues.apache.org/jira/browse/LUCENE-8922
b4276d6
to
b7afe05
Compare
@hub-cap This basic-licensed Query will presumably need an Apache-licensed client-side equivalent for use in HLRC. Much code would be duplicated between client and server-side classes so my assumption is server-side QueryBuilder with the execution logic should extend a client-side QueryBuilder that holds the search criteria. The server would override this method in the client-side class:
Does this seem a reasonable way forward? |
heya @markharwood it is correct that client and server code should diverge, especially in this case. I am not sure about server code extending client code though. We don't want server to depend on HLRC? I think the way forward may be to have two separate classes. |
My concern with duplicating code is the potential to diverge e.g. it then relies on 2 classes agreeing on search criteria choices like:
I guess adding test code helps ensure they don't diverge but it may require more HLRC tests to guarantee that.
To be clear in this case it's a QueryBuilder class - and they currently all exist in package |
heya @markharwood indeed, you should chat with @hub-cap to get more details on this though. Not sure if we plan on having some common library shared between server and client. Probably not in the short term. |
Agreed with @javanna I think it's unlikely we'll get a shared library in the short term. There are already a number of request objects that we duplicated between |
Is there a reason why it couldn't go directly into the |
I was putting it with all the other existing QueryBuilders in server but I can break with that tradition if you like |
@hub-cap would appreciate your ruling on this packaging issue - new Basic-licensed query (a first) and where to put the Apache-licensed QueryBuilder for use in HLRC |
@hub-cap added you as a reviewer to check out the HLRC approach to packaging this basic-licensed query. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I left one comment, I think we should duplicate the request/response like we did for other APIs in the client module and not in server.
server/src/main/java/org/elasticsearch/index/query/PinnedQueryBuilder.java
Outdated
Show resolved
Hide resolved
Following a conversation with Jim we've decided to pull out HLRC support from this PR to allow us to get it into master and will then open a separate discussion to figure out the right approach for HLRC |
de74c31
to
73c479d
Compare
Good to go, @jimczi ? |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
@@ -0,0 +1 @@ | |||
/bin/ |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is this needed ?
@@ -722,5 +722,5 @@ public static GeoShapeQueryBuilder geoDisjointQuery(String name, String indexedS | |||
*/ | |||
public static ExistsQueryBuilder existsQuery(String name) { | |||
return new ExistsQueryBuilder(name); | |||
} | |||
} |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you revert this ?
…promoted above “any organic” search results. This is the first feature in a new module `search-business-rules` which will house licensed (non OSS) logic for rewriting queries according to business rules. The PinnedQueryBuilder class offers a new `pinned` query in the DSL that takes an array of promoted IDs and an “organic” query and ensures the documents with the promoted IDs rank higher than the organic matches. Closes elastic#44074
Search enhancement: - new query type allows selected documents to be promoted above any "organic” search results. This is the first feature in a new module `search-business-rules` which will house licensed (non OSS) logic for rewriting queries according to business rules. The PinnedQueryBuilder class offers a new `pinned` query in the DSL that takes an array of promoted IDs and an “organic” query and ensures the documents with the promoted IDs rank higher than the organic matches. Closes elastic#44074
* Search enhancement: pinned queries (#44345) Search enhancement: - new query type allows selected documents to be promoted above any "organic” search results. This is the first feature in a new module `search-business-rules` which will house licensed (non OSS) logic for rewriting queries according to business rules. The PinnedQueryBuilder class offers a new `pinned` query in the DSL that takes an array of promoted IDs and an “organic” query and ensures the documents with the promoted IDs rank higher than the organic matches. Closes #44074
* Search enhancement: pinned queries Addresses elastic/elasticsearch#44345 * update docs in test * remove params overloads on Ids that already convert to Ids * add note to IdsQuery
* Search enhancement: pinned queries Addresses elastic/elasticsearch#44345 * update docs in test * remove params overloads on Ids that already convert to Ids * add note to IdsQuery
New query type allows selected documents to be promoted above “any organic” search results.
This is the first feature in a new module
search-business-rules
which will house licensed (non OSS) logic for rewriting queries according to business rules.The new PinnedQueryBuilder class offers a new
pinned
query in the DSL that takes an array of promoted IDs and an “organic” query and ensures the documents with the promoted IDs rank higher than those matched purely by the organic query.This first commit contains the core query and tests - docs to follow
Closes #44074