Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Maps] Use ES mvt #114553

Merged
merged 114 commits into from
Oct 25, 2021
Merged

[Maps] Use ES mvt #114553

merged 114 commits into from
Oct 25, 2021

Conversation

thomasneirynck
Copy link
Contributor

@thomasneirynck thomasneirynck commented Oct 11, 2021

Replace Kibana's mvt generation with Elasticsearch ES-mvt generation


Vector tile drawing of Elasticsearch data is going GA in Kibana Maps

This PR removes the Kibana implementation of mapbox vector tiles (" mvt") (https://docs.mapbox.com/vector-tiles/reference/), in favor of the mvt-implementation in Elasticsearch( cf. https://www.elastic.co/guide/en/elasticsearch/reference/current/search-vector-tile-api.html)

Since 7.11, the Kibana implementation of mvt was a beta-feature, and being used for:

  • vector tile scaling option in document layers
  • super-fine grid resolution for cluster layers

With this change to using Elasticsearch, both these capabilities are now going GA.

Increase geo-data volume on screen

This change significantly enhances the amount of geo-data that can be shown on screen.

Displaying more individual documents

Each tile contains up to the max_result_window of features. This is a per-index setting https://www.elastic.co/guide/en/elasticsearch/reference/current/index-modules.html. A single view contain will conist of 4-9 tiles.

image

Displaying clusters at high resolution

The super-fine resolutions displays up to 128x128 cells per tile, creating high-res cluster maps

image


Other changes

The client-side data-mapping is no longer possible for ES document layers

image

This switch is always on by default for scaling with vector tiles. This means the domain-range (ie. min & max) of a value is determined by querying the min-max of the underlying data in Elasticsearch, and not by only getting the min-max of the features on screen.

top-term aggregation is disabled

Top-terms agg is not supported by the vector tile search API of Elasticsearch.

too many features outline

This PR introduces a orange dotted outline of the bounding-box of the features int he tile. This is only used if the number of features exceeds the maximum allowed number of features per tile.

e.g.: zoomed out, cannot show all features at this scale level. Users would need to filter or zoom-in further to see all data.
image

Field formatters are not applied on map labels

When using vector tiles, field formatters of the data view are not applied for labeling on the map.

@nreese nreese removed the WIP Work in progress label Oct 22, 2021
@nreese
Copy link
Contributor

nreese commented Oct 25, 2021

@elasticmachine merge upstream

@nreese nreese marked this pull request as ready for review October 25, 2021 13:56
@nreese nreese requested a review from a team as a code owner October 25, 2021 13:56
@elasticmachine
Copy link
Contributor

Pinging @elastic/kibana-gis (Team:Geo)

Copy link
Contributor Author

@thomasneirynck thomasneirynck left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

scope exceeds original goals. I'm +1 on this change. thx!

Copy link
Contributor

@nreese nreese left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, code review and tested in chrome. This is such a huge improvement. Elasticsearch MVTs have such an amazing performance boost.

Anything that I am still unhappy with is tracked in #116150 and can be fixed at a later time.

Copy link
Member

@jsanz jsanz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested with Chrome and Firefox and I haven't got a single glitch. I will open a separate issue for removing labels in super-fine grid since they don't make any sense with such small squares but other than that, I've tested medium/large datasets with point, lines, and polygon geometry types and they render fast, panning now around is a great experience. Beautiful job everyone!

image

@kibanamachine
Copy link
Contributor

💛 Build succeeded, but was flaky

Test Failures

  • [job] [logs] Default CI Group #7 / dashboard feature controls dashboard feature controls security no dashboard privileges "before all" hook for "doesn't show dashboard navLink"

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
maps 839 837 -2

Public APIs missing comments

Total count of every public API that lacks a comment. Target amount is 0. Run node scripts/build_api_docs --plugin [yourplugin] --stats comments for more detailed information.

id before after diff
maps 201 203 +2

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
maps 2.7MB 2.7MB +3.3KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id before after diff
maps 42.1KB 41.7KB -470.0B
Unknown metric groups

API count

id before after diff
maps 202 204 +2

References to deprecated APIs

id before after diff
maps 1697 1707 +10

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

@nreese nreese merged commit 33fd1bd into master Oct 25, 2021
@kibanamachine kibanamachine added the backport missing Added to PRs automatically when the are determined to be missing a backport. label Oct 27, 2021
@kibanamachine
Copy link
Contributor

Friendly reminder: Looks like this PR hasn’t been backported yet.
To create backports run node scripts/backport --pr 114553 or prevent reminders by adding the backport:skip label.

@thomasneirynck thomasneirynck added the backport:skip This commit does not require backporting label Oct 27, 2021
@kibanamachine kibanamachine removed the backport missing Added to PRs automatically when the are determined to be missing a backport. label Oct 27, 2021
@nreese nreese mentioned this pull request May 6, 2022
@spalger spalger deleted the feature/maps/use_es_mvt branch May 8, 2022 22:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport:skip This commit does not require backporting [Deprecated-Use Team:Presentation]Team:Geo Former Team Label for Geo Team. Now use Team:Presentation release_note:enhancement v8.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants