Rewrite map layer

[corradio]

This should speed up performances considerably.
A good strategy seems to use a big canvas (world), and then only draw parts of the canvas which correspond to the active screen. One could pre-draw the (invisible) surroundings to ensure smooth transition. Of course, we should remember what has been drawn in order not to redraw things.
We need a fast algorithm in order to detect mouseovers and clicks. https://github.com/d3/d3/blob/master/API.md#quadtrees-d3-quadtree could be used.

This would improve the drawing speed, but would not reduce the need to transfer, and load in memory, all geojson polygons.
We could generate tiles as GMaps does, but that would probably make mouse hit tests complicated, as we need the coordinates of zones. Moreover, it would complicate the interactivity use cases (change color of hovered zone etc..)

[corradio]

We could also use this Uber library, using WebGL:
https://uber.github.io/deck.gl/#/documentation/layer-catalog/geojson-layer.

Thanks @martindaniel4 for pointing that out!

[jarek]

If you're after performance drawing GeoJSON, I've also seen Mapbox's WebGL renderer do pretty well www.mapbox.com/mapbox-gl-js/api/ - you can easily get it to do things like "here's a URL for a geojson file with shapes of 90k buildings, please draw it" e.g. https://maarouf.me/athena-buildings/

Actually I think in the current version the animated exchange arrows use most of the CPU...

[corradio]

We should probably do some serious investigating but the memory usage definitely comes from loading all the polygons in (SVG) memory..

[alex3165]

I believe mapbox-gl-js is the solution to build high performant vector maps. Also, Mapbox provide all the tooling to build a map, for example they provide Mapbox studio which allow you to completely design the map. The nice thing is that all the rendering of the vector tiles happen on web workers on the client so it doesn't block the UI.

I actually wrote a wrapper on top of mapbox-gl-js for a better integration in a React application and built monumap with it.

The only downside of using Mapbox for an OSS project is the limitation of 50 000 views / months for a free plan.

An alternative solution for not having to pay would be to install a vector tiles server with OSM data and use Tangram as a client library to display the vector tiles.

[corradio]

@alex3165 what about deck.gl?
In principle our map is very simple: we just need to display geojson, colour the interior of polygons and be able to select them.
50K views / month is definitely too little for us (we have >3K DAU).
I'd like to avoid using a tile server because it would add complexity. Do you think it's doable to do on the client while maintaining proper performances?

[alex3165]

I don't know if you can use deck.gl as a standalone library I thought it was meant to be used with react-map-gl which is based on mapbox-gl-js. If you could use it independently it seems like deck.gl already provide a set of layers which might not fit your need but again I don't know enough about deck.gl to state on this point.

I believe there is a way to display a simple vector map as the one used for electricitymap in webgl but I don't think you can go with libraries like mapbox-gl-js or tangram without vector tiles server and I am not yet sure how you can achieve it.

Although I don't think there is much overhead of creating your own vector tiles server with solutions like tileserver-gl. Looking at the readme it seems that they have a dockerfile you can probably just deploy on a box somewhere and you can consume the vector tiles for free with mapbox-gl-js or tangram. I am not sure how you can use mapbox studio to customise your map in this case.

[alex3165]

After further thinking you might be able to use mapbox-gl-js without server and just pass a static geojson of the frontiers, need to be proved, I will check this when I have some time.

[corradio]

Thanks man! I will dig into that.

…

On Thu, Sep 21, 2017 at 11:20 AM, Alexandre ***@***.***> wrote: After further thinking you might be able to use mapbox-gl-js without server and just pass a static geojson of the frontiers, need to be proved, I will check this when I have some time. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <electricitymaps/electricitymaps#670 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ABlEKNe_KjnOtQRNqYYtKWnmRlLLOhtxks5skipbgaJpZM4OxS46> .

[Wykks]

I think deck-gl is the way to go
This example load a geojson, without anything else than deck-gl : https://github.com/uber/deck.gl/tree/master/examples/pure-js/without-map

[alex3165]

Look like you can go with just mapbox-gl without server http://jsfiddle.net/gxfpbeme/ cf @Wykks

[corradio]

Cool! Do we need an access token then? if we only use the client library?

[Wykks]

No, access token is only needed if you use their map service

[corradio]

@alex3165 wrote a prototype here:
https://github.com/alex3165/serverless-world-map-poc
which seems to work very well.

Next steps as I see it:

• Export all of our geometries into a geoJSON file (GeoJSON support #913)
• Double check @alex3165 's poc with those geometries
• Switch to react (Refactor with React #853), or implement @alex3165 's proto without React

@superlopuh I know you had the geoJSON files exported, can you send them here if you still have them?

[HansHyde]

Is this in line with being able to customize geometries (even if grossly simplified - straight lines vs intricate borders along rivers) at some time in the future?

For example - NY is going to need 3 geometries, while with NE-ISO we can take 6 states (NH, CT, RI, MA, ME & VT) and turn them into 1 common geometry. PJM will go from ~12 full/partial state geometries to 3 geometries covering the same ~12 state territory.

[corradio]

@HansHyde yes

[HansHyde]

@corradio - great! I'll keep moving forward on getting NorAm (mainly US states) in order.

[corradio]

From the tests I have done on my old iPad:

• remove the arrows layer did not really change performances
• replacing our current map layer with mapbox-gl-js really improves dragging FPS. I have a prototype running. Will submit a branch soon.
• i have not tried adding the colors yet - hopefully this won't affect performances

The good news is that it doesn't change the code too much. I'll make sure to fit the wind/solar layers and test this thoroughly, but in principle it should help quite a bit.

Kudos to @alex3165 for pushing me in the good direction by giving me a working mapbox-gl example.

[corradio]

Updates:

• Map is working well, but I need to test performances in modern browsers. Seems like Firefox is the only one that's slow.
• The hovering effect has performances problems due to mapbox's slow setFilter, but I'm confident I can solve that using several workarounds
• Wind/solar layer updates are working with same performances as previously
• Arrow layer is a bit trickier. I wanted to add the arrows inside the webgl layer using videos, but turns out the only HTML5 video format that handles transparency is WebM in Chrome (https://developers.google.com/web/updates/2013/07/Alpha-transparency-in-Chrome-video). I might try to draw to canvas, and then use the canvas layer in mapbox (http://bl.ocks.org/lbud/a1f6e07528cc62317309c0da5f00f3a2) but I'm afraid of performance issues because that would mean using requestAnimationFrame.
• I'm afraid webgl isn't supported on old browsers, which means that on old Android phones it might not work :/

[alex3165]

@corradio you can use html markers for the arrows: https://www.mapbox.com/help/custom-markers-gl-js/

[corradio]

@alex3165 that's a very good idea - didn't know they provided such an API.

Another thing, for further reference:
it seems that there's a significant drop in FPS when synchronising the layer positions.
Basically changing the transform of the other hand-made layers upon drag of the map is heavy. I will try to trim that part.

[corradio]

Just tried it. When the map is dragged, the markers are lagging behind on slow machines. It's actually much worse performance than setting a transform on a div. Seems like performance scales linearly with the number of markers.

[corradio]

Using the regular layer with animated GIFs, it seems that replacing with static PNGs removes 70% of the jitter. Seems like animating GIFs is heavy :(
Unfortunately we can't pause the GIFs while dragging..

[corradio]

A good trick I'm doing and that seems to solve the problem is to hide the arrows while dragging 👍

[corradio]

Allright, he's a summary of today's work with potential solutions:

• When disabling the arrow layer, mapbox is much faster on old devices
• The issues arise when adding the arrows layer
• Current dragging is slow because GIFs can't be paused, and they force a lot of unnecessary re-paints
• Using mapbox markers is worse
• Hiding GIFs during drag works, but the UX is not optimal
• One solution is to use the video API in mapbox, but this requires animations without transparency (as it is not supported in all browsers). This requires redesigning the exchange symbols.
• A second solution is to investigate CSS animations (that run on the GPU), that we can pause during drag (if needed). Maybe we can re-use the existing images used to build the GIFs.

[corradio]

PR submitted here #958.
I'm still not sure CSS animations are better than GIFs. We should test that carefully.