[8.11] [fix] Using max_result_window to set up MVT size request leads to not showing all results (#171344) #171440
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… showing all results (elastic#171344) Closes elastic#170272 The cause of the problem is that Maps is not setting [_mvt](https://www.elastic.co/guide/en/elasticsearch/reference/current/search-vector-tile-api.html) `size` parameter. Lets illustrate the problem with an example. * index.max_result_window is set to 20,000. * There are 15,000 hits matching the query for the tile, but because `size` is not provided, the tile returns the default size value of 10,000 hits. * Tile request sets `track_total_hits` to 20,001. Tile meta returns `"hits.total.relation": "eq"` and `"hits.total.value": 15000` properties, indicating that all possible hits are returned in the tile. * Maps UI does not display "incomplete results" even though 5000 hits are not contained in the tile. * This happens because `hits.total` properties from tile meta is used to indicate "incomplete results" and the returned value indicates that all results are contained in the tile. We know this is not true because the tile actually only returned 10,000 hits and not all 15,000 matching hits. Thus, without setting `size`, the tile is not returning all of the hits indicated by `track_total_hits`. ### test instructions 1. install web logs sample data set 2. In dev tools, run the following ``` POST kibana_sample_data_logs/_doc/ { "geo": { "coordinates": "59,-106.5" }, "@timestamp": "2023-08-01" } PUT /kibana_sample_data_logs/_settings { "index" : { "max_result_window" : 20000 } } ``` 3. Create new map 4. Set time range to `1 year ago to 1 year from now` to display all web logs data 5. add documents layer. Verify point circled in image is displayed. Without changes, this point would not be returned in the tile because the tile would only return the first 10000 matches. <img width="500" alt="Screenshot 2023-11-15 at 11 31 25 AM" src="https://github.com/elastic/kibana/assets/373691/2249d571-24c4-41ad-975e-d44ee6d246ce"> (cherry picked from commit 95a8d6a)
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
💚 Build Succeeded
Metrics [docs]Async chunks
To update your PR or re-run it, just comment with: cc @nreese |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Backport
This will backport the following commits from
mainto8.11:Questions ?
Please refer to the Backport tool documentation