Vector Tiles

[pjcozzi]

2D maps like Google, Apple, MapBox GL, and OL3 are moving to actual vectors for rendering vector data, instead of baking vector data into a raster tile. This allows, for example, to make labels always face up even if south is up and to fade out labels so they never hit each other.

For now, this roadmap is a collection of resources on this topic.

• Google Maps + WebGL = MapsGL
• Vector tiles for fast custom maps (video)
• Videos from FOSS4G
• Mapbox Vector Tile Specification
  • blog post
  • tools: vector-tile-js and mapnik-vector-tile
  • Client-side: pbf.js to decode tiles (protocol buffers).
• geojson-vt
• MMT-Vector-Tiles

[pksorensen]

Vector tiles for fast custom maps (video) , http://vimeo.com/10622814 Wrong video ? :)

[pjcozzi]

Fixed. Thanks.

[pjcozzi]

Cesium forum discussion

[pjcozzi]

More discussion: https://groups.google.com/forum/#!topic/cesium-dev/QnOxsA0iwa0

[mramato]

Looks like ESRI is adopting Mapbox vector tiles: https://www.mapbox.com/blog/vector-tile-adoption/

[pjcozzi]

Info from FOSS4G NA 2015.

• Video The State of Vector Tiles

[pjcozzi]

From Mapzen: Look Upon Our Squares of Math in Three Dimensions

[pjcozzi]

• State of the Map 2015 - Vector Rendering Panel - video, transcript

[pjcozzi]

http://osgeo-org.1560.x6.nabble.com/Asking-for-new-vector-tiles-community-module-tt5211994.html

[pjcozzi]

• From Esri: http://blogs.esri.com/esri/arcgis/2015/07/20/vector-tiles-preview/

[GatorScott]

Demo of vector tiles at Esri UC is last video on page

http://www10.giscafe.com/blogs/gissanjay/2015/07/20/keynote-from-jack-dangermond-at-the-2015-esri-user-conference/

From: Patrick Cozzi [mailto:[email protected]]
Sent: Monday, July 20, 2015 5:45 PM
To: AnalyticalGraphicsInc/cesium
Subject: Re: [cesium] Vector Tiles (#2132)

• From Esri: http://blogs.esri.com/esri/arcgis/2015/07/20/vector-tiles-preview/

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/2132#issuecomment-123053536.

[bensleveritt]

Is this still being considered by Cesium? It would be an impressive feature to tout.

[pjcozzi]

Yes, but our current focus is on 3D Tiles.

[bingqixuan]

Hello, I want to know when will it come out, I'm very looking forward to it~

[pjcozzi]

Current work-in-progress for 3D Tiles vector tiles is in the vector-tiles branch. The final date is still TBA.

[davidyaha]

@pjcozzi Was this issue "fixed" with #6097?

[pjcozzi]

@davidyaha that added support for the draft vector tile payload in 3D Tiles, but I would still leave this issue open for other potential vector tile specs.

[OmarShehata]

OSM has global vector data, it would be cool to support that in Cesium and eventually serve it from ion:

https://openmaptiles.com/downloads/dataset/osm/#0.23/0/-26

Forum discussion.

[OmarShehata]

Requested again here: #7939

Also perhaps relevant, some thoughts on how TerriaJS loads these vector tiles? #6182 (comment)

[bampakoa]

Is there any related work or PR that we could follow up?

[icemagno]

How can I extend MapboxImageryProvider to load PBF files instead PNG ? I think I can do something to put it to work using this : https://landtechnologies.github.io/Mapbox-vector-tiles-basic-js-renderer/.

As I can see, this renderer can get a PBF from {x},{y},{z} and render it in any canvas (the Cesium canvas??).

https://landtechnologies.github.io/Mapbox-vector-tiles-basic-js-renderer/debug/basic/google

it is just an idea to start. I know almost nothing about how Providers work. I need to know how to test a new Provider without need to compile all Cesium code. Any way to add it as a plugin? Some code to show me how?

What do you say?

[OmarShehata]

@icemagno you might find the GridImageryProvider useful as a reference here. It's the simplest imagery provider I know of (procedural creates a grid as the tiles): https://github.com/AnalyticalGraphicsInc/cesium/blob/master/Source/Scene/GridImageryProvider.js

It should be straightforward to set up the CesiumJS source code so you can add your custom class, see the build guide here: https://github.com/AnalyticalGraphicsInc/cesium/blob/master/Documentation/Contributors/BuildGuide/README.md#build-guide

[icemagno]

Ok. I'll give a try.
BTW: I know you guys have a long expertise to write thousands of lines before build and test but I like to check every step I do in the code.

So... I dont like the "write-build-test" method because I'll spend hours in a lot of builds. Is there some "write-F5-test" way to do this? I mean... can I write my code and immediately check it just refreshing the browser without the need to rebuild every time?

[OmarShehata]

BTW: I know you guys have a long expertise to write thousands of lines before build and test

I don't think that's a good idea no matter how much expertise anyone has 😄

When running CesiumJS from the source as described in the linked guide above, you can make changes directly to the source and see the changes immediately by refreshing the page or re-running the Sandcastle example. This is how most CesiumJS development is done.

If you have any follow up questions on best practices here, let's take this conversation to the forum so as not to derail this GitHub issue.

[davenquinn]

Actually, yeah there are some updates! I just finished the initial public rollout of the cesium-vector-provider module and created a demo website that shows it in action.

I think this will be pretty great implementation that maximizes compatibility with the Mapbox stack, especially once some of the outstanding design deficiencies are addressed. I'm working to ensure that the project can be based directly on the rendering stack within Maplibre GL (see maplibre/maplibre-gl-js#166). It'd be nice if we could eventually drop in a Mapbox stylesheet to Cesium and get a functional basemap.

Contributions to the project are welcome!

[ebogo1]

@davenquinn Thanks for the updates and work on the vector provider module! We started taking a look at your repo and were wondering if you had any ideas about how the code could be included in CesiumJS. The main concern is that maplibre-gl is ~14 MB unpacked, which could be a bit heavy for combined Cesium builds.

Just to throw a couple ideas out there, we could:

• Add a flag to the makeZipFile command that optionally includes maplibre-gl and the vector provider files (false by default)
• Let users manually import maplibre-gl from a path similarly to how TaskProcessor points to .wasm paths

[davenquinn]

Hi @ebogo1, and thanks for looking at this. I agree that the size of Maplibre GL is a bit of a problem here. Right now the cesium-vector-provider module references a full/compiled version of Maplibre, but that is because I haven't figured out the right strategy for referring to things directly from their source code (a large part of the problem here is replicating their wasm compilation pipeline). This is probably simple but the module just hasn't evolved to that point.

Thinking ahead, I suspect that having this component as a standalone module makes sense. Ideally, it would just be a custom layer that can be imported with all dependencies from an external node module (this is the role that cesium-vector-provider seeks to fill). This would allow glue code between Maplibre and Cesium to have a lifecycle independent of Cesium's internals, and would allow (the large majority of) people with no need for Mapbox-compatible vector rendering to omit this heavy dependency.

I don't know if keeping things outside of the Cesium core library is a preferred approach of the team or how it fits with your development workflow, but it's how I'd try to proceed, at least initially.

[sanjeetsuhag]

@davenquinn Given the heavy dependency on maplibre-gl, I agree that it may be better to keep this as a separate module to start. I took a look at the cesium-vector-provider repo and the associated fork of maplibre-gl, and it seems like the heavy lifting is done in the BasicRenderer in maplibre-gl, with cesium-vector-provider mostly being an API to call through to query for images and features. I have a few questions so far:

• In the README, it says "It'd be nice to align the approaches used here with Cesium's first-party 3D Tiles stack". Is this referring to the caching system? I know that there is a caching system implemented on the renderer side, is there a need for one on the provider side?
• I noticed that in some areas there is reference to a light source in the shading, would it be possible to account for the position of the sun in Cesium?
• Regarding performance, is the rendering done in workers on an offscreen canvas?
• What is the state of metadata support right now? Is there any limitation to users being able to pick features?

[davenquinn]

Hi @sanjeetsuhag thanks for the questions.

The current structure of the code (the fork of maplibre-gl with the BasicRenderer sidecar) is mostly due to my having not figured out how to compile and reference the shaders that Mapbox GL uses independently of their build pipeline. Much of the BasicRenderer code may more properly belong in cesium-vector-provider but I think once the approach is solidified, it might be possible to build a small number of necessary hooks into Maplibre and contribute them to the core project. maplibre/maplibre-gl-js#166 demonstrates that the team there is amenable to making some APIs public to serve deep integrations, which would be the ideal long-term approach.

• By "aligning with 3D Tiles" I was thinking mostly about taking cues from where in the Cesium rendering stack 3D tiles are handled. I'm not really that familiar with Cesium's internal APIs and right now everything is done in single tiles.
• I believe rendering is right now being done in worker threads, because offscreen canvas is not supported everywhere
• Metadata support and basic filtering have been carried over from @kikitte's work, but not validated. Making this work is on the roadmap, but basic rendering + labels are I think a somewhat higher priority. The goal would be to have it work somewhat similarly to how Maplibre filtering works within their stack.

Additional points:

1. One big item that might have some influence on the "right" way to hook into Cesium is labeling. Labels clearly must be viewport-aligned and span tile boundaries. Figuring how to integrate these layers with Cesium's renderer might be difficult and is something I just haven't tackled yet.
2. I'm happy to discuss moving or changing the ownership of davenquinn/cesium-vector-provider if this is something that would help centralize specific work on this in the right space.

[davenquinn]

I will also note that one of the potential improvements to rendering will be if we can directly render into Cesium-managed canvas contexts rather than creating our own, serializing as images, and exposing in a tile provider.

This feels like a lot of hoops to jump through and is purely a function of my and previous folks’ unfamiliarity with Cesium’s internals.

[nf-s]

A light-weight alternative to mapbox-gl/maplibre-gl is https://github.com/protomaps/protomaps.js - it supports MVT and a subset of mapbox style spec.

We have been using it in TerriaJS for nearly a year - and it works quite well.

https://github.com/TerriaJS/terriajs/blob/main/lib/Map/ImageryProvider/ProtomapsImageryProvider.ts

[davenquinn]

I will also note that one of the potential improvements to rendering will be if we can directly render into Cesium-managed canvas contexts rather than creating our own, serializing as images, and exposing in a tile provider. This feels like a lot of hoops to jump through and is purely a function of my and previous folks’ unfamiliarity with Cesium’s internals.

…

On May 13, 2022, 11:17 AM -0400, Daven Quinn ***@***.***>, wrote: Hi @sanjeetsuhag thanks for the questions. The current structure of the code (the fork of maplibre-gl with the BasicRenderer sidecar) is mostly due to my having not figured out how to compile and reference the shaders that Mapbox GL uses independently of their build pipeline. Much of the BasicRenderer code may more properly belong in cesium-vector-provider but I think once the approach is solidified, it might be possible to build a small number of necessary hooks into Maplibre and contribute them to the core project. maplibre/maplibre-gl-js#166 demonstrates that the team there is amenable to making some APIs public to serve deep integrations, which would be the ideal long-term approach. • By "aligning with 3D Tiles" I was thinking mostly about taking cues from where in the Cesium rendering stack 3D tiles are handled. I'm not really that familiar with Cesium's internal APIs and right now everything is done in single tiles. • I believe rendering is right now being done in worker threads, because offscreen canvas is not supported everywhere • Metadata support and basic filtering have been carried over from @kikitte's work, but not validated. Making this work is on the roadmap, but basic rendering + labels are I think a somewhat higher priority. The goal would be to have it work somewhat similarly to how Maplibre filtering works within their stack. Additional points: 1. One big item that might have some influence on the "right" way to hook into Cesium is labeling. Labels clearly must be viewport-aligned and span tile boundaries. Figuring how to integrate these layers with Cesium's renderer might be difficult and is something I just haven't tackled yet. 2. I'm happy to discuss moving or changing the ownership of davenquinn/cesium-vector-provider if this is something that would help centralize specific work on this in the right space. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[gberaudo]

For information, MapLibre is building their roadmap and calling for popular vote.
There is a proposition for a minimal build which may align with your needs @davenquinn.
See maplibre/maplibre#159 / https://maplibre.org/news/2023-03-13-second-call-for-bounties/

[puzonevan]

@davenquinn Any updated progress on the module?

[markchagers]

We really look forward to vector tile (PBF) support for cesium. The newer ESRI services use this format, and currently cannot be used in cesiumjs. This is a much needed enhancement.

[DoisKoh]

Fyi we're in a similar boat and our company is having a hard time justifying going all-in with Cesium because of this.