Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Query refactoring: Introduce toQuery() and fromXContent() methods in QueryBuilders and QueryParsers #10580

Closed

Conversation

cbuescher
Copy link
Member

The currently planed refactoring of search queries layed out in #9901 and currently tracked in #10217 requires the current "parse()" methods into two methods, first a "fromXContent(...)" method that allows parsing to an intermediate query representation and second a "Query toQuery(...)" method these parsed query objects that create the actual lucene queries.

This PR is a first step in that direction as it introduces the interface changes necessary for the further refactoring. It introduces the new interface methods while for now keeping the old Builder/Parser API still in place by delegating the new "toQuery()" implementations to the existing "parse()" methods, and by introducing a "catch-all" "fromXContent()" implementation in a BaseQueryParser that returns a temporary QueryBuilder wrapper implementation. This allows us to refactor the existing QueryBuilders step by step while already beeing able to start refactoring queries with nested inner queries.


@Override
protected void doXContent(XContentBuilder builder, Params params) throws IOException {
throw new org.apache.commons.lang3.NotImplementedException("Not Implemented");
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think a regular UnsupportedOperationException would work better instead of the Apache one

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changed that.

@dakrone
Copy link
Member

dakrone commented Apr 14, 2015

Left a minor comment but this looks pretty good to me, @javanna what do you think?

…ilder, one for checking caching. Delegate those two from old method for bwc.
@cbuescher
Copy link
Member Author

Working on BoolQueryBuilder based in this I realized that I also needed to split QueryParseContext#parseInnerQuery() into two parts, will be used to build the inner QueryBuilder in the parse-step and the second to later do some cache checking that is in place there at the moment. This part will only be called once the lucene queries are created.
I added this split already to this PR we will need it for some of the nestes queries. I already encountered problems with existing tests when refactoring the BoolQueryBuilder that can be solved by having the cache-check available in the "toQuery()" step.


public abstract String[] names();

public abstract Query parse(QueryParseContext parseContext) throws IOException, QueryParsingException;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the above two methods don't need to be repeated here as long as this abstract class implements QueryParser

@javanna
Copy link
Member

javanna commented Apr 16, 2015

I really like this change, it allows us to migrate queries gradually without changing everything at the same time. The only concern I have is whether we should merge this into master or start a new feature branch for the query refactoring. I do know that we are afraid of merge conflicts here, but I am also afraid that the solution is not merging temporary code. I am not doubting the quality of this PR, it's just that part of it will go away at the end of different steps of the refactoring and we don't know yet how long it will take to get it done. That said I have the feeling we want to work on this on a feature branch and pay the price of merge conflicts. Should get easier already though with this PR as we don't merge parser and builder anymore.

@cbuescher
Copy link
Member Author

Changes according to your comments. Main thing I did is pulling the "toQuery()" logic to the BaseQueryBuilder and letting the different builders provide the name of their corresponding parsers by a final method. Open to suggestions on this one though.

@@ -68,5 +69,14 @@ public XContentBuilder toXContent(XContentBuilder builder, Params params) throws
return builder;
}

public Query toQuery(QueryParseContext parseContext) throws QueryParsingException, IOException {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let's make it final?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if I understand that. We will have to override that one in the QueryBuilders that we refactor, or am I wrong?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right I got confused with the parser, sorry! Shall we introduce these new methods through a new intermediate base class (BaseQueryBuilderTemp?) instead and have a new one for the migrated builders too? maybe overkill... what do you think?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think in this case it's not so nice since the existing BaseQueryBuilder already contains some implementations (toString, buildAsBytes etc...) that otherwise we just have to move over. I think to check how many extending classes don't have their own refactored "toQuery()" method it is sufficient to temporarily remove it from the base class and see what turns red. At least that works for me.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fine with me

@javanna
Copy link
Member

javanna commented Apr 17, 2015

I like it, left a few more comments.

@cbuescher
Copy link
Member Author

Added changes for your last comments, left out the second BaseQueryBuilder class you suggested for now, but I can still add that if you like. Just mentioned my thought about it in the comments.

@javanna
Copy link
Member

javanna commented Apr 17, 2015

LGTM, this is ready to go on a feature branch as discussed, at least till the first step of the query refactoring is completed (moving over all of the queries to the new fromXContent and toQuery code, but leaving parsing on the data nodes as-is).

cbuescher pushed a commit that referenced this pull request Apr 17, 2015
…lders and QueryParsers

The planed refactoring of search queries layed out in #9901 requires to split the "parse()"
method in QueryParsers into two methods, first a "fromXContent(...)" method that allows parsing
to an intermediate query representation (currently called FooQueryBuilder) and second a
"Query toQuery(...)" method on these intermediate representations that create the actual lucene queries.

This PR is a first step in that direction as it introduces the interface changes necessary for the further
refactoring. It introduces the new interface methods while for now keeping the old Builder/Parsers still
in place by delegating the new "toQuery()" implementations to the existing "parse()" methods, and by
introducing a "catch-all" "fromXContent()" implementation in a BaseQueryParser that returns a temporary
QueryBuilder wrapper implementation. This allows us to refactor the existing QueryBuilders step by step
while already beeing able to start refactoring queries with nested inner queries.

Closes #10580
@cbuescher
Copy link
Member Author

I pushed this PR to a new feature branch here:
https://github.com/elastic/elasticsearch/commits/feature/query-refactoring

@cbuescher cbuescher closed this Apr 17, 2015
mute pushed a commit to mute/elasticsearch that referenced this pull request Jul 29, 2015
…lders and QueryParsers

The planed refactoring of search queries layed out in elastic#9901 requires to split the "parse()"
method in QueryParsers into two methods, first a "fromXContent(...)" method that allows parsing
to an intermediate query representation (currently called FooQueryBuilder) and second a
"Query toQuery(...)" method on these intermediate representations that create the actual lucene queries.

This PR is a first step in that direction as it introduces the interface changes necessary for the further
refactoring. It introduces the new interface methods while for now keeping the old Builder/Parsers still
in place by delegating the new "toQuery()" implementations to the existing "parse()" methods, and by
introducing a "catch-all" "fromXContent()" implementation in a BaseQueryParser that returns a temporary
QueryBuilder wrapper implementation. This allows us to refactor the existing QueryBuilders step by step
while already beeing able to start refactoring queries with nested inner queries.

Closes elastic#10580
@clintongormley clintongormley added :Search/Search Search-related issues that do not fall into other categories and removed :Query Refactoring labels Feb 14, 2018
@cbuescher cbuescher deleted the feature/query-interface-refactoring branch March 20, 2024 20:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Search/Search Search-related issues that do not fall into other categories
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants