Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

NullPointerException with bool nested query #45933

Closed
dvandersluis opened this issue Aug 23, 2019 · 11 comments
Closed

NullPointerException with bool nested query #45933

dvandersluis opened this issue Aug 23, 2019 · 11 comments
Assignees
@jtibshirani
Labels
:Search/Search Search-related issues that do not fall into other categories

Comments

@dvandersluis
Copy link

dvandersluis commented Aug 23, 2019 •
edited
Loading

Elasticsearch version (bin/elasticsearch --version):

OpenJDK 64-Bit Server VM warning: Option UseConcMarkSweepGC was deprecated in version 9.0 and will likely be removed in a future release.
Version: 7.3.1, Build: default/deb/4749ba6/2019-08-19T20:19:25.651794Z, JVM: 12.0.2

Plugins installed: None

JVM version (java -version):

openjdk version "11.0.4" 2019-07-16
OpenJDK Runtime Environment (build 11.0.4+11-post-Ubuntu-1ubuntu218.04.3)
OpenJDK 64-Bit Server VM (build 11.0.4+11-post-Ubuntu-1ubuntu218.04.3, mixed mode, sharing)

OS version (uname -a if on a Unix-like system):

Linux rhino 4.15.0-58-generic #64-Ubuntu SMP Tue Aug 6 11:12:41 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

(Ubuntu 18.04.3 via Cloud66)

Description of the problem including expected versus actual behavior:

I'm experiencing NullPointerExceptions with this query (simplified but still causes the NPE):

{
  "query": {
    "bool": {
      "must": [
        {
          "nested": { 
            "path": "color",
            "query": {
              "match_all": {}
            }
          }
        }
      ],
      "should": [
        {
          "match_all": {}
        }
      ]
    }
  }
}

I believe this may be related to #43497 although my query is not using min_score. The stacktrace looks similar to me.

Stacktrace:

[2019-08-23T20:26:22,426][DEBUG][o.e.a.s.TransportSearchAction] [rhino] [photos_production_20190823024146185][0], node[T2HI2dq_TcKjlRfb5GgMhA], [R], s[STARTED], a[id=bdet3EL_Sk-8s5fEh6w_Pw]: Failed to execute [SearchRequest{searchType=QUERY_THEN_FETCH, indices=[photos_production], indicesOptions=IndicesOptions[ignore_unavailable=false, allow_no_indices=true, expand_wildcards_open=true, expand_wildcards_closed=false, allow_aliases_to_multiple_indices=true, forbid_closed_indices=true, ignore_aliases=false, ignore_throttled=true], types=[], routing='null', preference='null', requestCache=null, scroll=null, maxConcurrentShardRequests=0, batchedReduceSize=512, preFilterShardSize=128, allowPartialSearchResults=true, localClusterAlias=null, getOrCreateAbsoluteStartMillis=-1, ccsMinimizeRoundtrips=true, source={"query":{"bool":{"must":[{"nested":{"query":{"match_all":{"boost":1.0}},"path":"color","ignore_unmapped":false,"score_mode":"avg","boost":1.0}}],"should":[{"match_all":{"boost":1.0}}],"adjust_pure_negative":true,"boost":1.0}}}}]
org.elasticsearch.transport.RemoteTransportException: [rhino][10.132.245.76:9300][indices:data/read/search[phase/query]]
Caused by: org.elasticsearch.search.query.QueryPhaseExecutionException: Query Failed [Failed to execute main query]
	at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:305) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:113) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:335) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:360) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.search.SearchService.lambda$executeQueryPhase$1(SearchService.java:340) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.ActionListener.lambda$map$2(ActionListener.java:145) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:62) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.search.SearchService$2.doRun(SearchService.java:1052) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:44) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:758) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.3.1.jar:7.3.1]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
	at java.lang.Thread.run(Thread.java:835) [?:?]
Caused by: java.lang.NullPointerException
	at org.apache.lucene.search.ReqOptSumScorer.setMinCompetitiveScore(ReqOptSumScorer.java:299) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
	at org.apache.lucene.search.TopScoreDocCollector.updateMinCompetitiveScore(TopScoreDocCollector.java:242) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
	at org.apache.lucene.search.TopScoreDocCollector$SimpleTopScoreDocCollector$1.collect(TopScoreDocCollector.java:78) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
	at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreRange(Weight.java:242) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
	at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:229) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
	at org.elasticsearch.search.internal.CancellableBulkScorer.score(CancellableBulkScorer.java:56) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
	at org.elasticsearch.search.internal.ContextIndexSearcher.searchInternal(ContextIndexSearcher.java:198) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:179) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:443) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
	at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:270) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:113) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:335) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:360) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.search.SearchService.lambda$executeQueryPhase$1(SearchService.java:340) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.ActionListener.lambda$map$2(ActionListener.java:145) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:62) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.search.SearchService$2.doRun(SearchService.java:1052) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:44) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:758) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) ~[elasticsearch-7.3.1.jar:7.3.1]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) ~[?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) ~[?:?]
	at java.lang.Thread.run(Thread.java:835) ~[?:?]
[2019-08-23T20:26:22,428][DEBUG][o.e.a.s.TransportSearchAction] [rhino] All shards failed for phase: [query]
org.elasticsearch.ElasticsearchException$1: null
	at org.elasticsearch.ElasticsearchException.guessRootCauses(ElasticsearchException.java:639) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:137) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:264) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.InitialSearchPhase.onShardFailure(InitialSearchPhase.java:105) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.InitialSearchPhase.access$200(InitialSearchPhase.java:50) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.InitialSearchPhase$2.onFailure(InitialSearchPhase.java:273) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.SearchExecutionStatsCollector.onFailure(SearchExecutionStatsCollector.java:73) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:59) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.SearchTransportService$ConnectionCountingHandler.handleException(SearchTransportService.java:441) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1111) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.transport.TransportService$DirectResponseChannel.processException(TransportService.java:1223) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1197) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.transport.TaskTransportChannel.sendResponse(TaskTransportChannel.java:60) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.support.ChannelActionListener.onFailure(ChannelActionListener.java:56) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.ActionListener$1.onFailure(ActionListener.java:70) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:64) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.search.SearchService$2.doRun(SearchService.java:1052) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:44) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:758) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.3.1.jar:7.3.1]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
	at java.lang.Thread.run(Thread.java:835) [?:?]
Caused by: java.lang.NullPointerException
	at org.apache.lucene.search.ReqOptSumScorer.setMinCompetitiveScore(ReqOptSumScorer.java:299) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
	at org.apache.lucene.search.TopScoreDocCollector.updateMinCompetitiveScore(TopScoreDocCollector.java:242) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
	at org.apache.lucene.search.TopScoreDocCollector$SimpleTopScoreDocCollector$1.collect(TopScoreDocCollector.java:78) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
	at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreRange(Weight.java:242) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
	at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:229) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
	at org.elasticsearch.search.internal.CancellableBulkScorer.score(CancellableBulkScorer.java:56) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
	at org.elasticsearch.search.internal.ContextIndexSearcher.searchInternal(ContextIndexSearcher.java:198) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:179) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:443) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
	at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:270) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:113) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:335) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:360) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.search.SearchService.lambda$executeQueryPhase$1(SearchService.java:340) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.ActionListener.lambda$map$2(ActionListener.java:145) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:62) ~[elasticsearch-7.3.1.jar:7.3.1]
	... 8 more
[2019-08-23T20:26:22,429][WARN ][r.suppressed             ] [rhino] path: /photos_production/_search, params: {pretty=, index=photos_production}
org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:305) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:139) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:264) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.InitialSearchPhase.onShardFailure(InitialSearchPhase.java:105) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.InitialSearchPhase.access$200(InitialSearchPhase.java:50) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.InitialSearchPhase$2.onFailure(InitialSearchPhase.java:273) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.SearchExecutionStatsCollector.onFailure(SearchExecutionStatsCollector.java:73) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:59) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.SearchTransportService$ConnectionCountingHandler.handleException(SearchTransportService.java:441) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1111) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.transport.TransportService$DirectResponseChannel.processException(TransportService.java:1223) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1197) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.transport.TaskTransportChannel.sendResponse(TaskTransportChannel.java:60) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.support.ChannelActionListener.onFailure(ChannelActionListener.java:56) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.ActionListener$1.onFailure(ActionListener.java:70) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:64) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.search.SearchService$2.doRun(SearchService.java:1052) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:44) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:758) [elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-7.3.1.jar:7.3.1]
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
	at java.lang.Thread.run(Thread.java:835) [?:?]
Caused by: org.elasticsearch.ElasticsearchException$1
	at org.elasticsearch.ElasticsearchException.guessRootCauses(ElasticsearchException.java:639) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:137) [elasticsearch-7.3.1.jar:7.3.1]
	... 22 more
Caused by: java.lang.NullPointerException
	at org.apache.lucene.search.ReqOptSumScorer.setMinCompetitiveScore(ReqOptSumScorer.java:299) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
	at org.apache.lucene.search.TopScoreDocCollector.updateMinCompetitiveScore(TopScoreDocCollector.java:242) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
	at org.apache.lucene.search.TopScoreDocCollector$SimpleTopScoreDocCollector$1.collect(TopScoreDocCollector.java:78) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
	at org.apache.lucene.search.Weight$DefaultBulkScorer.scoreRange(Weight.java:242) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
	at org.apache.lucene.search.Weight$DefaultBulkScorer.score(Weight.java:229) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
	at org.elasticsearch.search.internal.CancellableBulkScorer.score(CancellableBulkScorer.java:56) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.apache.lucene.search.BulkScorer.score(BulkScorer.java:39) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
	at org.elasticsearch.search.internal.ContextIndexSearcher.searchInternal(ContextIndexSearcher.java:198) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.search.internal.ContextIndexSearcher.search(ContextIndexSearcher.java:179) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.apache.lucene.search.IndexSearcher.search(IndexSearcher.java:443) ~[lucene-core-8.1.0.jar:8.1.0 dbe5ed0b2f17677ca6c904ebae919363f2d36a0a - ishan - 2019-05-09 19:34:03]
	at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:270) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:113) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:335) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:360) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.search.SearchService.lambda$executeQueryPhase$1(SearchService.java:340) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.ActionListener.lambda$map$2(ActionListener.java:145) ~[elasticsearch-7.3.1.jar:7.3.1]
	at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:62) ~[elasticsearch-7.3.1.jar:7.3.1]
	... 8 more
@dvandersluis dvandersluis changed the title NullPointerException with nested query NullPointerException with bool nested query Aug 23, 2019
@jtibshirani jtibshirani added the :Search/Search Search-related issues that do not fall into other categories label Aug 23, 2019
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-search

@jtibshirani
Copy link
Contributor

Thanks @dvandersluis for this bug report. Would you be able to provide a full reproduction of the issue, including the mappings and documents that were defined? I tried to run the search request you provided on 7.3.1, but was not able to reproduce the NullPointerException.

@dvandersluis
Copy link
Author

Do you have a way I can share them with you? The index is 2.7GB so I'm not exactly sure how I would do that.

I did notice that if I played around with the filters I could filter down the result set to a point where it does work so that does suggest to me that the NPE is dependent on what data is being searched.

@jtibshirani jtibshirani self-assigned this Aug 27, 2019
@nikator
Copy link

nikator commented Aug 28, 2019

We were having the same issue with one of our indexes after upgrade to 7.3.1. So we had to downgrade to 7.1.1 version.

@jtibshirani
Copy link
Contributor

I think I managed to reproduce the error locally by ingesting a larger number of documents. I have an idea of what the issue is and hope to put up a PR soon.

@dvandersluis once the PR is up, it would be really helpful if you tried out the fix on your data to confirm it resolves the error you're seeing (assuming it's not too inconvenient).

@dvandersluis
Copy link
Author

@jtibshirani thanks! I would love to help test it out, if you can email me to help me figure out how to get your patch up working with our data!

@dvandersluis
Copy link
Author

FWIW the index in question has 32927960 docs.

@jtibshirani jtibshirani mentioned this issue Aug 29, 2019
Merged
@jtibshirani
Copy link
Contributor

I merged a fix that will be available in the next 7.3 and 7.4 releases. If you would like to test out the fix now instead of waiting for a release, you can check out the 7.3 branch and build a distribution from source by following the instructions here: https://github.com/elastic/elasticsearch#building-from-source. When testing out a source distribution it's recommended to use an isolated test cluster, since the 7.3 branch contains unreleased code and may have other bugs.

Thanks again @dvandersluis for helping us to track down this bug.

@dvandersluis
Copy link
Author

@jtibshirani thank you! We have our ES cluster managed so I'm not sure if I can build a version from source on it but I'll see what I can do. Any idea when the next 7.3 release will be?

@jtibshirani
Copy link
Contributor

We have our ES cluster managed so I'm not sure if I can build a version from source on it but I'll see what I can do.

Sounds good, I will close issue for now and we can re-open if you notice more errors. We don't communicate release dates, but it should be available fairly soon!

@dvandersluis
Copy link
Author

@jtibshirani we updated to 7.3.2 today and I'm monitoring but it appears to have fixed our issue! Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jtibshirani jtibshirani

Labels
:Search/Search Search-related issues that do not fall into other categories
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@dvandersluis @jtibshirani @nikator @elasticmachine