Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Rollup: split series in a visualization + sorting by average creates an invalid query #34782

Closed
ismael-hasan opened this issue Apr 9, 2019 · 4 comments
Closed

Rollup: split series in a visualization + sorting by average creates an invalid query #34782

ismael-hasan opened this issue Apr 9, 2019 · 4 comments
Labels
Feature:Aggregations Aggregation infrastructure (AggConfig, esaggs, ...) Feature:Rollups Team:Visualizations Visualization editors, elastic-charts and infrastructure

Comments

@ismael-hasan
Copy link

Kibana version: 6.7.0

Elasticsearch version: 6.7.0

Server OS version: Windows 10

Browser version: 73.0.3683.86 (Official Build) (64-bit)

Browser OS version: Windows 10

Original install method (e.g. download page, yum, from source, etc.): Download

Describe the bug:
When having a rollup index with an average aggregation, if we split the aggregation by some term and sort by that average value, it fails.

Steps to reproduce:

  1. Create a rollup job with an average and count metric. It follows an example using monitoring indices:
{
	"id": "bugtest",
	"index_pattern": ".mon*",
	"rollup_index": "bugtestr",
	"cron": "0 * * * * ?",
	"groups": {
		"date_histogram": {
			"interval": "1m",
			"field": "timestamp",
			"delay": "1m",
			"time_zone": "UTC"
		},
		"terms": {
			"fields": [
				"source_node.host"
			]
		}
	},
	"metrics": [{
			"field": "index_stats.primaries.search.query_total",
			"metrics": [
				"avg",
				"value_count"
			]
		}
	],
	"timeout": "20s",
	"page_size": 1000
}
  1. Start the job
  2. Create the rollup index pattern
  3. Create a visualization: Vertical bar, y-axis: the average, x-axis: timestamp, Split the series by the term, Sort by average descending
    At this point, there is an error:
Error in visualization

Rollup search error: An internal server error occurred

Expected behavior:
The visualization does not fail

Screenshots (if relevant):

Errors in browser console (if relevant):

Provide logs and/or server output (if relevant):

The most relevant error is in Elasticsearch logs:

Expand error 

[2019-04-09T12:53:58,304][DEBUG][o.e.a.s.TransportSearchAction] [wR18dbv] [bugtestr][4], node[wR18dbvqTsCpoHwuvF2Ofg], [P], s[STARTED], a[id=05HqimJ8QxqB6-7BszLjsg]: Failed to execute [SearchRequest{searchType=QUERY_THEN_FETCH, indices=[bugtestr], indicesOptions=IndicesOptions[ignore_unavailable=false, allow_no_indices=true, expand_wildcards_open=true, expand_wildcards_closed=false, allow_aliases_to_multiple_indices=true, forbid_closed_indices=true, ignore_aliases=false, ignore_throttled=true], types=[], routing='null', preference='null', requestCache=null, scroll=null, maxConcurrentShardRequests=5, batchedReduceSize=512, preFilterShardSize=128, allowPartialSearchResults=true, localClusterAlias=null, getOrCreateAbsoluteStartMillis=-1, source={"size":0,"query":{"bool":{"must":[{"bool":{"must":[{"match_all":{"boost":1.0}},{"match_all":{"boost":1.0}},{"range":{"timestamp.date_histogram.timestamp":{"from":1554802496196,"to":1554803396196,"include_lower":true,"include_upper":true,"format":"epoch_millis","boost":1.0}}}],"adjust_pure_negative":true,"boost":1.0}}],"filter":[{"term":{"_rollup.id":{"value":"bugtest","boost":1.0}}},{"terms":{"_rollup.version":[1,2],"boost":1.0}}],"adjust_pure_negative":true,"boost":1.0}},"aggregations":{"filter_2":{"filter":{"bool":{"must":[{"term":{"timestamp.date_histogram.time_zone":{"value":"UTC","boost":1.0}}}],"adjust_pure_negative":true,"boost":1.0}},"aggregations":{"2":{"meta":{},"date_histogram":{"field":"timestamp.date_histogram.timestamp","interval":"1m","offset":0,"order":{"_key":"asc"},"keyed":false,"min_doc_count":1},"aggregations":{"3":{"terms":{"field":"source_node.host.terms.value","size":5,"min_doc_count":1,"shard_min_doc_count":0,"show_term_doc_count_error":false,"order":[{"1":"desc"},{"_key":"asc"}]},"aggregations":{"1.value":{"sum":{"field":"index_stats.primaries.search.query_total.avg.value"}},"1._count":{"sum":{"field":"index_stats.primaries.search.query_total.avg._count"}},"3._count":{"sum":{"field":"source_node.host.terms._count"}}}},"2._count":{"sum":{"field":"timestamp.date_histogram._count"}}}}}}}}}]
org.elasticsearch.transport.RemoteTransportException: [wR18dbv][127.0.0.1:9300][indices:data/read/search[phase/query]]
Caused by: org.elasticsearch.search.aggregations.AggregationExecutionException: Invalid aggregator order path [1]. Unknown aggregation [1]
        at org.elasticsearch.search.aggregations.support.AggregationPath.validate(AggregationPath.java:294) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.InternalOrder.validate(InternalOrder.java:98) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.InternalOrder.validate(InternalOrder.java:95) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.terms.TermsAggregator.<init>(TermsAggregator.java:189) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.terms.AbstractStringTermsAggregator.<init>(AbstractStringTermsAggregator.java:43) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.terms.GlobalOrdinalsStringTermsAggregator.<init>(GlobalOrdinalsStringTermsAggregator.java:93) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.terms.TermsAggregatorFactory$ExecutionMode$2.create(TermsAggregatorFactory.java:301) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.terms.TermsAggregatorFactory.doCreateInternal(TermsAggregatorFactory.java:154) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory.createInternal(ValuesSourceAggregatorFactory.java:59) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactory.create(AggregatorFactory.java:216) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactory.asMultiBucketAggregator(AggregatorFactory.java:230) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.terms.TermsAggregatorFactory.doCreateInternal(TermsAggregatorFactory.java:116) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory.createInternal(ValuesSourceAggregatorFactory.java:59) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactory.create(AggregatorFactory.java:216) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactories.createSubAggregators(AggregatorFactories.java:201) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorBase.<init>(AggregatorBase.java:76) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.BucketsAggregator.<init>(BucketsAggregator.java:47) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.histogram.DateHistogramAggregator.<init>(DateHistogramAggregator.java:74) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.histogram.DateHistogramAggregatorFactory.createAggregator(DateHistogramAggregatorFactory.java:80) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.histogram.DateHistogramAggregatorFactory.doCreateInternal(DateHistogramAggregatorFactory.java:74) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.histogram.DateHistogramAggregatorFactory.doCreateInternal(DateHistogramAggregatorFactory.java:38) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory.createInternal(ValuesSourceAggregatorFactory.java:59) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactory.create(AggregatorFactory.java:216) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactory.asMultiBucketAggregator(AggregatorFactory.java:230) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.histogram.DateHistogramAggregatorFactory.doCreateInternal(DateHistogramAggregatorFactory.java:72) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.histogram.DateHistogramAggregatorFactory.doCreateInternal(DateHistogramAggregatorFactory.java:38) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory.createInternal(ValuesSourceAggregatorFactory.java:59) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactory.create(AggregatorFactory.java:216) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactories.createSubAggregators(AggregatorFactories.java:201) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorBase.<init>(AggregatorBase.java:76) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.BucketsAggregator.<init>(BucketsAggregator.java:47) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.filter.FilterAggregator.<init>(FilterAggregator.java:53) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.filter.FilterAggregatorFactory.createInternal(FilterAggregatorFactory.java:72) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactory.create(AggregatorFactory.java:216) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactories.createTopLevelAggregators(AggregatorFactories.java:217) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregationPhase.preProcess(AggregationPhase.java:55) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:112) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.indices.IndicesService.lambda$loadIntoContext$17(IndicesService.java:1250) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.indices.IndicesService.lambda$cacheShardLevelResult$18(IndicesService.java:1306) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.indices.IndicesRequestCache$Loader.load(IndicesRequestCache.java:164) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.indices.IndicesRequestCache$Loader.load(IndicesRequestCache.java:147) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.common.cache.Cache.computeIfAbsent(Cache.java:433) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.indices.IndicesRequestCache.getOrCompute(IndicesRequestCache.java:119) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.indices.IndicesService.cacheShardLevelResult(IndicesService.java:1312) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.indices.IndicesService.loadIntoContext(IndicesService.java:1248) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:347) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:393) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.SearchService.access$100(SearchService.java:125) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:358) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:354) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.SearchService$4.doRun(SearchService.java:1085) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:751) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.7.0.jar:6.7.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:1.8.0_121]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:1.8.0_121]
        at java.lang.Thread.run(Unknown Source) [?:1.8.0_121]
[2019-04-09T12:53:58,337][DEBUG][o.e.a.s.TransportSearchAction] [wR18dbv] All shards failed for phase: [query]
org.elasticsearch.search.aggregations.AggregationExecutionException: Invalid aggregator order path [1]. Unknown aggregation [1]
        at org.elasticsearch.search.aggregations.support.AggregationPath.validate(AggregationPath.java:294) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.InternalOrder.validate(InternalOrder.java:98) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.InternalOrder.validate(InternalOrder.java:95) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.terms.TermsAggregator.<init>(TermsAggregator.java:189) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.terms.AbstractStringTermsAggregator.<init>(AbstractStringTermsAggregator.java:43) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.terms.GlobalOrdinalsStringTermsAggregator.<init>(GlobalOrdinalsStringTermsAggregator.java:93) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.terms.TermsAggregatorFactory$ExecutionMode$2.create(TermsAggregatorFactory.java:301) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.terms.TermsAggregatorFactory.doCreateInternal(TermsAggregatorFactory.java:154) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory.createInternal(ValuesSourceAggregatorFactory.java:59) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactory.create(AggregatorFactory.java:216) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactory.asMultiBucketAggregator(AggregatorFactory.java:230) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.terms.TermsAggregatorFactory.doCreateInternal(TermsAggregatorFactory.java:116) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory.createInternal(ValuesSourceAggregatorFactory.java:59) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactory.create(AggregatorFactory.java:216) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactories.createSubAggregators(AggregatorFactories.java:201) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorBase.<init>(AggregatorBase.java:76) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.BucketsAggregator.<init>(BucketsAggregator.java:47) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.histogram.DateHistogramAggregator.<init>(DateHistogramAggregator.java:74) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.histogram.DateHistogramAggregatorFactory.createAggregator(DateHistogramAggregatorFactory.java:80) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.histogram.DateHistogramAggregatorFactory.doCreateInternal(DateHistogramAggregatorFactory.java:74) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.histogram.DateHistogramAggregatorFactory.doCreateInternal(DateHistogramAggregatorFactory.java:38) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory.createInternal(ValuesSourceAggregatorFactory.java:59) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactory.create(AggregatorFactory.java:216) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactory.asMultiBucketAggregator(AggregatorFactory.java:230) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.histogram.DateHistogramAggregatorFactory.doCreateInternal(DateHistogramAggregatorFactory.java:72) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.histogram.DateHistogramAggregatorFactory.doCreateInternal(DateHistogramAggregatorFactory.java:38) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory.createInternal(ValuesSourceAggregatorFactory.java:59) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactory.create(AggregatorFactory.java:216) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactories.createSubAggregators(AggregatorFactories.java:201) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorBase.<init>(AggregatorBase.java:76) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.BucketsAggregator.<init>(BucketsAggregator.java:47) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.filter.FilterAggregator.<init>(FilterAggregator.java:53) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.filter.FilterAggregatorFactory.createInternal(FilterAggregatorFactory.java:72) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactory.create(AggregatorFactory.java:216) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactories.createTopLevelAggregators(AggregatorFactories.java:217) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregationPhase.preProcess(AggregationPhase.java:55) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:112) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.indices.IndicesService.lambda$loadIntoContext$17(IndicesService.java:1250) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.indices.IndicesService.lambda$cacheShardLevelResult$18(IndicesService.java:1306) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.indices.IndicesRequestCache$Loader.load(IndicesRequestCache.java:164) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.indices.IndicesRequestCache$Loader.load(IndicesRequestCache.java:147) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.common.cache.Cache.computeIfAbsent(Cache.java:433) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.indices.IndicesRequestCache.getOrCompute(IndicesRequestCache.java:119) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.indices.IndicesService.cacheShardLevelResult(IndicesService.java:1312) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.indices.IndicesService.loadIntoContext(IndicesService.java:1248) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:347) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:393) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.SearchService.access$100(SearchService.java:125) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:358) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:354) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.SearchService$4.doRun(SearchService.java:1085) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:751) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.7.0.jar:6.7.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:1.8.0_121]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:1.8.0_121]
        at java.lang.Thread.run(Unknown Source) [?:1.8.0_121]
[2019-04-09T12:53:58,379][WARN ][r.suppressed             ] [wR18dbv] path: /bugtestr*/_rollup_search, params: {rest_total_hits_as_int=true, index=bugtestr*}
java.lang.RuntimeException: all shards failed
        at org.elasticsearch.xpack.rollup.RollupResponseTranslator.lambda$translateResponse$0(RollupResponseTranslator.java:85) ~[?:?]
        at java.util.stream.ReferencePipeline$3$1.accept(Unknown Source) ~[?:1.8.0_121]
        at java.util.Spliterators$ArraySpliterator.forEachRemaining(Unknown Source) ~[?:1.8.0_121]
        at java.util.stream.AbstractPipeline.copyInto(Unknown Source) ~[?:1.8.0_121]
        at java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source) ~[?:1.8.0_121]
        at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source) ~[?:1.8.0_121]
        at java.util.stream.AbstractPipeline.evaluate(Unknown Source) ~[?:1.8.0_121]
        at java.util.stream.ReferencePipeline.collect(Unknown Source) ~[?:1.8.0_121]
        at org.elasticsearch.xpack.rollup.RollupResponseTranslator.translateResponse(RollupResponseTranslator.java:88) ~[?:?]
        at org.elasticsearch.xpack.rollup.action.TransportRollupSearchAction.processResponses(TransportRollupSearchAction.java:136) ~[?:?]
        at org.elasticsearch.xpack.rollup.action.TransportRollupSearchAction.lambda$doExecute$1(TransportRollupSearchAction.java:121) ~[?:?]
        at org.elasticsearch.action.ActionListener$1.onResponse(ActionListener.java:61) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:85) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.action.support.TransportAction$1.onResponse(TransportAction.java:81) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.action.search.TransportMultiSearchAction$1.finish(TransportMultiSearchAction.java:166) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.action.search.TransportMultiSearchAction$1.handleResponse(TransportMultiSearchAction.java:153) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.action.search.TransportMultiSearchAction$1.onFailure(TransportMultiSearchAction.java:146) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.action.support.TransportAction$1.onFailure(TransportAction.java:91) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.action.search.AbstractSearchAsyncAction.raisePhaseFailure(AbstractSearchAsyncAction.java:233) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseFailure(AbstractSearchAsyncAction.java:291) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.action.search.AbstractSearchAsyncAction.executeNextPhase(AbstractSearchAsyncAction.java:133) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.action.search.AbstractSearchAsyncAction.onPhaseDone(AbstractSearchAsyncAction.java:254) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.action.search.InitialSearchPhase.onShardFailure(InitialSearchPhase.java:100) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.action.search.InitialSearchPhase.access$100(InitialSearchPhase.java:48) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.action.search.InitialSearchPhase$2.lambda$onFailure$1(InitialSearchPhase.java:220) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.action.search.InitialSearchPhase.maybeFork(InitialSearchPhase.java:174) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.action.search.InitialSearchPhase.access$000(InitialSearchPhase.java:48) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.action.search.InitialSearchPhase$2.onFailure(InitialSearchPhase.java:220) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.action.search.SearchExecutionStatsCollector.onFailure(SearchExecutionStatsCollector.java:73) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.action.ActionListenerResponseHandler.handleException(ActionListenerResponseHandler.java:59) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.action.search.SearchTransportService$ConnectionCountingHandler.handleException(SearchTransportService.java:463) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.transport.TransportService$ContextRestoreResponseHandler.handleException(TransportService.java:1108) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.transport.TransportService$DirectResponseChannel.processException(TransportService.java:1220) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.transport.TransportService$DirectResponseChannel.sendResponse(TransportService.java:1194) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.transport.TaskTransportChannel.sendResponse(TaskTransportChannel.java:60) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.action.support.ChannelActionListener.onFailure(ChannelActionListener.java:56) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.SearchService$2.onFailure(SearchService.java:366) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:360) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:354) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.SearchService$4.doRun(SearchService.java:1085) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.common.util.concurrent.TimedRunnable.doRun(TimedRunnable.java:41) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingAbstractRunnable.doRun(ThreadContext.java:751) [elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.common.util.concurrent.AbstractRunnable.run(AbstractRunnable.java:37) [elasticsearch-6.7.0.jar:6.7.0]
        at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [?:1.8.0_121]
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [?:1.8.0_121]
        at java.lang.Thread.run(Unknown Source) [?:1.8.0_121]
Caused by: org.elasticsearch.action.search.SearchPhaseExecutionException: all shards failed
        ... 28 more
Caused by: org.elasticsearch.search.aggregations.AggregationExecutionException: Invalid aggregator order path [1]. Unknown aggregation [1]
        at org.elasticsearch.search.aggregations.support.AggregationPath.validate(AggregationPath.java:294) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.InternalOrder.validate(InternalOrder.java:98) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.InternalOrder.validate(InternalOrder.java:95) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.terms.TermsAggregator.<init>(TermsAggregator.java:189) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.terms.AbstractStringTermsAggregator.<init>(AbstractStringTermsAggregator.java:43) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.terms.GlobalOrdinalsStringTermsAggregator.<init>(GlobalOrdinalsStringTermsAggregator.java:93) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.terms.TermsAggregatorFactory$ExecutionMode$2.create(TermsAggregatorFactory.java:301) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.terms.TermsAggregatorFactory.doCreateInternal(TermsAggregatorFactory.java:154) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory.createInternal(ValuesSourceAggregatorFactory.java:59) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactory.create(AggregatorFactory.java:216) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactory.asMultiBucketAggregator(AggregatorFactory.java:230) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.terms.TermsAggregatorFactory.doCreateInternal(TermsAggregatorFactory.java:116) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory.createInternal(ValuesSourceAggregatorFactory.java:59) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactory.create(AggregatorFactory.java:216) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactories.createSubAggregators(AggregatorFactories.java:201) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorBase.<init>(AggregatorBase.java:76) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.BucketsAggregator.<init>(BucketsAggregator.java:47) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.histogram.DateHistogramAggregator.<init>(DateHistogramAggregator.java:74) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.histogram.DateHistogramAggregatorFactory.createAggregator(DateHistogramAggregatorFactory.java:80) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.histogram.DateHistogramAggregatorFactory.doCreateInternal(DateHistogramAggregatorFactory.java:74) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.histogram.DateHistogramAggregatorFactory.doCreateInternal(DateHistogramAggregatorFactory.java:38) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory.createInternal(ValuesSourceAggregatorFactory.java:59) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactory.create(AggregatorFactory.java:216) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactory.asMultiBucketAggregator(AggregatorFactory.java:230) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.histogram.DateHistogramAggregatorFactory.doCreateInternal(DateHistogramAggregatorFactory.java:72) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.histogram.DateHistogramAggregatorFactory.doCreateInternal(DateHistogramAggregatorFactory.java:38) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.support.ValuesSourceAggregatorFactory.createInternal(ValuesSourceAggregatorFactory.java:59) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactory.create(AggregatorFactory.java:216) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactories.createSubAggregators(AggregatorFactories.java:201) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorBase.<init>(AggregatorBase.java:76) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.BucketsAggregator.<init>(BucketsAggregator.java:47) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.filter.FilterAggregator.<init>(FilterAggregator.java:53) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.bucket.filter.FilterAggregatorFactory.createInternal(FilterAggregatorFactory.java:72) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactory.create(AggregatorFactory.java:216) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregatorFactories.createTopLevelAggregators(AggregatorFactories.java:217) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.aggregations.AggregationPhase.preProcess(AggregationPhase.java:55) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.query.QueryPhase.execute(QueryPhase.java:112) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.indices.IndicesService.lambda$loadIntoContext$17(IndicesService.java:1250) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.indices.IndicesService.lambda$cacheShardLevelResult$18(IndicesService.java:1306) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.indices.IndicesRequestCache$Loader.load(IndicesRequestCache.java:164) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.indices.IndicesRequestCache$Loader.load(IndicesRequestCache.java:147) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.common.cache.Cache.computeIfAbsent(Cache.java:433) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.indices.IndicesRequestCache.getOrCompute(IndicesRequestCache.java:119) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.indices.IndicesService.cacheShardLevelResult(IndicesService.java:1312) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.indices.IndicesService.loadIntoContext(IndicesService.java:1248) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.SearchService.loadOrExecuteQueryPhase(SearchService.java:347) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:393) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.SearchService.access$100(SearchService.java:125) ~[elasticsearch-6.7.0.jar:6.7.0]
        at org.elasticsearch.search.SearchService$2.onResponse(SearchService.java:358) ~[elasticsearch-6.7.0.jar:6.7.0]
        ... 9 more

Any additional context:
This seems related to using dots in the aggregation names. Copy-pasting the search from the logs, I can reproduce the same failure via dev tools - note that the aggregation is called 1.value:

Non working aggregation 

GET bugtestr/_search
{
  "size": 0,
  "query": {
    "bool": {
      "must": [
        {
          "bool": {
            "must": [
              {
                "match_all": {
                  "boost": 1
                }
              },
              {
                "match_all": {
                  "boost": 1
                }
              },
              {
                "range": {
                  "timestamp.date_histogram.timestamp": {
                    "from": 1554802496196,
                    "to": 1554803396196,
                    "include_lower": true,
                    "include_upper": true,
                    "format": "epoch_millis",
                    "boost": 1
                  }
                }
              }
            ],
            "adjust_pure_negative": true,
            "boost": 1
          }
        }
      ],
      "filter": [
        {
          "term": {
            "_rollup.id": {
              "value": "bugtest",
              "boost": 1
            }
          }
        },
        {
          "terms": {
            "_rollup.version": [
              1,
              2
            ],
            "boost": 1
          }
        }
      ],
      "adjust_pure_negative": true,
      "boost": 1
    }
  },
  "aggregations": {
    "filter_2": {
      "filter": {
        "bool": {
          "must": [
            {
              "term": {
                "timestamp.date_histogram.time_zone": {
                  "value": "UTC",
                  "boost": 1
                }
              }
            }
          ],
          "adjust_pure_negative": true,
          "boost": 1
        }
      },
      "aggregations": {
        "2": {
          "meta": {},
          "date_histogram": {
            "field": "timestamp.date_histogram.timestamp",
            "interval": "1m",
            "offset": 0,
            "order": {
              "_key": "asc"
            },
            "keyed": false,
            "min_doc_count": 1
          },
          "aggregations": {
            "3": {
              "terms": {
                "field": "source_node.host.terms.value",
                "size": 5,
                "min_doc_count": 1,
                "shard_min_doc_count": 0,
                "show_term_doc_count_error": false,
                "order": [
                  {
                    "1.value": "desc"
                  },
                  {
                    "_key": "asc"
                  }
                ]
              },
              "aggregations": {
                "1.value": {
                  "sum": {
                    "field": "index_stats.primaries.search.query_total.avg.value"
                  }
                },
                "1._count": {
                  "sum": {
                    "field": "index_stats.primaries.search.query_total.avg._count"
                  }
                },
                "3._count": {
                  "sum": {
                    "field": "source_node.host.terms._count"
                  }
                }
              }
            },
            "2._count": {
              "sum": {
                "field": "timestamp.date_histogram._count"
              }
            }
          }
        }
      }
    }
  }
}

If I rename the aggregation from 1.value to nodots - and update the reference to the aggregation in the sort - it works fine:

Working aggregation 

GET bugtestr/_search
{
  "size": 0,
  "query": {
    "bool": {
      "must": [
        {
          "bool": {
            "must": [
              {
                "match_all": {
                  "boost": 1
                }
              },
              {
                "match_all": {
                  "boost": 1
                }
              },
              {
                "range": {
                  "timestamp.date_histogram.timestamp": {
                    "from": 1554802496196,
                    "to": 1554803396196,
                    "include_lower": true,
                    "include_upper": true,
                    "format": "epoch_millis",
                    "boost": 1
                  }
                }
              }
            ],
            "adjust_pure_negative": true,
            "boost": 1
          }
        }
      ],
      "filter": [
        {
          "term": {
            "_rollup.id": {
              "value": "bugtest",
              "boost": 1
            }
          }
        },
        {
          "terms": {
            "_rollup.version": [
              1,
              2
            ],
            "boost": 1
          }
        }
      ],
      "adjust_pure_negative": true,
      "boost": 1
    }
  },
  "aggregations": {
    "filter_2": {
      "filter": {
        "bool": {
          "must": [
            {
              "term": {
                "timestamp.date_histogram.time_zone": {
                  "value": "UTC",
                  "boost": 1
                }
              }
            }
          ],
          "adjust_pure_negative": true,
          "boost": 1
        }
      },
      "aggregations": {
        "2": {
          "meta": {},
          "date_histogram": {
            "field": "timestamp.date_histogram.timestamp",
            "interval": "1m",
            "offset": 0,
            "order": {
              "_key": "asc"
            },
            "keyed": false,
            "min_doc_count": 1
          },
          "aggregations": {
            "3": {
              "terms": {
                "field": "source_node.host.terms.value",
                "size": 5,
                "min_doc_count": 1,
                "shard_min_doc_count": 0,
                "show_term_doc_count_error": false,
                "order": [
                  {
                    "nodots": "desc"
                  },
                  {
                    "_key": "asc"
                  }
                ]
              },
              "aggregations": {
                "nodots": {
                  "sum": {
                    "field": "index_stats.primaries.search.query_total.avg.value"
                  }
                },
                "1._count": {
                  "sum": {
                    "field": "index_stats.primaries.search.query_total.avg._count"
                  }
                },
                "3._count": {
                  "sum": {
                    "field": "source_node.host.terms._count"
                  }
                }
              }
            },
            "2._count": {
              "sum": {
                "field": "timestamp.date_histogram._count"
              }
            }
          }
        }
      }
    }
  }
}

@timroes
Copy link
Contributor

timroes commented Apr 9, 2019

Hi,

thanks a lot for the detailed report. Does that same chart configuration work on a regular (non rolled up) index work? That error sounds like it would intepret the order path now as a path not a unique name, which sounds a bit strange to me, that this error should only happen in the rollup API?

Could you please verify that the same visualization on the non rolledup index doesn't throw that error?

Cheers,
Tim

@timroes timroes added Feature:Aggregations Aggregation infrastructure (AggConfig, esaggs, ...) Team:Visualizations Visualization editors, elastic-charts and infrastructure labels Apr 9, 2019
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-app

@ismael-hasan
Copy link
Author

Hi @timroes ! It did not fail when running it directly against the indices instead of the rolled up index pattern. I just added both visualizations to a dashboard to inspect them; the working one is sending:

{
  "aggs": {
    "2": {
      "date_histogram": {
        "field": "timestamp",
        "interval": "1m",
        "time_zone": "Europe/Berlin",
        "min_doc_count": 1
      },
      "aggs": {
        "3": {
          "terms": {
            "field": "source_node.host",
            "size": 5,
            "order": {
              "1": "desc"
            }
          },
          "aggs": {
            "1": {
              "avg": {
                "field": "index_stats.primaries.search.query_total"
              }
            }
          }
        }
      }
    }
  },
  "size": 0,
  "_source": {
    "excludes": []
  },
  "stored_fields": [
    "*"
  ],
  "script_fields": {},
  "docvalue_fields": [
    {
      "field": "job_stats.data_counts.earliest_record_timestamp",
      "format": "date_time"
    },
    {
      "field": "job_stats.data_counts.latest_record_timestamp",
      "format": "date_time"
    },
    {
      "field": "kibana_stats.timestamp",
      "format": "date_time"
    },
    {
      "field": "source_node.timestamp",
      "format": "date_time"
    },
    {
      "field": "timestamp",
      "format": "date_time"
    }
  ],
  "query": {
    "bool": {
      "must": [
        {
          "match_all": {}
        },
        {
          "match_all": {}
        },
        {
          "range": {
            "timestamp": {
              "gte": 1554811211246,
              "lte": 1554812111246,
              "format": "epoch_millis"
            }
          }
        }
      ],
      "filter": [],
      "should": [],
      "must_not": []
    }
  }
}

and the non working one (the rollup one) is sending:

{
  "aggs": {
    "2": {
      "date_histogram": {
        "field": "timestamp",
        "interval": "1m",
        "time_zone": "UTC",
        "min_doc_count": 1
      },
      "aggs": {
        "3": {
          "terms": {
            "field": "source_node.host",
            "size": 5,
            "order": {
              "1": "desc"
            }
          },
          "aggs": {
            "1": {
              "avg": {
                "field": "index_stats.primaries.search.query_total"
              }
            }
          }
        }
      }
    }
  },
  "size": 0,
  "_source": {
    "excludes": []
  },
  "stored_fields": [
    "*"
  ],
  "script_fields": {},
  "docvalue_fields": [
    {
      "field": "timestamp",
      "format": "date_time"
    }
  ],
  "query": {
    "bool": {
      "must": [
        {
          "match_all": {}
        },
        {
          "match_all": {}
        },
        {
          "range": {
            "timestamp": {
              "gte": 1554811254805,
              "lte": 1554812154805,
              "format": "epoch_millis"
            }
          }
        }
      ],
      "filter": [],
      "should": [],
      "must_not": []
    }
  }
}

The only difference that I see is that the normal one calls to _msearch and the rollup one calls to api/rollup/search.

Checking further, I think it may be related to elastic/elasticsearch#30467 , not to kibana.

@timroes
Copy link
Contributor

timroes commented Apr 9, 2019

Yes I am pretty sure elastic/elasticsearch#30467 is the actual issue here. I would close this, and refer to the ES issue for further tracking the fix of that.

@timroes timroes closed this as completed Apr 9, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Feature:Aggregations Aggregation infrastructure (AggConfig, esaggs, ...) Feature:Rollups Team:Visualizations Visualization editors, elastic-charts and infrastructure
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@timroes @ismael-hasan @elasticmachine