Cache map tiles locally server-side

[ivan-aksamentov]

This PR attempts to implement a file-based local server-side cache for map tiles, addressing some of the concerns of #714

Motivation

Together with #826 this PR allows to run local server without internet connection. When reliable connection is available, user may selectively visit regions of interest on the map in order to build up the cache, and then go offline, with these same regions still being available. Because cache is filesystem-based, offline tiles persist between server restarts and browser sessions.

Implementation details

In order to not violate terms of currently used MapBox service, this PR switches to OpenStreetMap provider, which provides free tiles and allows (even encourages) caching.

Instead of fetching tiles directly, leaflet layer now requests tiles from /tiles/{s}/{x}/{y}/{z} (when running server locally, this resolves to localhost). On server side, this request is routed to TileCache service that initially proxies requests to the remote tile provider of choice, currently OpenStreetMap.

The image tiles resulting from this request are sent back to client, but are also asynchronously written to filesystem. If previously seen tile is requested again, TileCache will return corresponding cached file instead of making requests to remote tile provider.
When remote tiles cannot be successfully retrieved, TileCache currently returns status 404. Leaflet in this case renders a grey tile.

Discussion and further improvements

• OpenStreetMap Tile Usage Policy allows caching and proxying, but they forbid setting HTTP user agent. However, requesting tiles with node-fetch always returns status 429, therefore, in this PR we forward User-Agent from the original client (browser) request. We have to check if this behavior is allowed.

• Although individual tile images are small (10-20kb), they come in bulk numbers and overall cache size may reach limits of available disk space. We may implement various cache eviction policies (e.g. LRU) to free up disk space on user's machine.

• Similarly, caching can be implemented client-side. The tiles can be stored in LocalStorage or in IndexedDB. This is more tricky, as it will involve modifying leaflet's layer, but is doable.

• Currently, remote tiles are being requested, stored in a buffer and then sent to client/written to disk. Instead, for performance reasons, we may want to pipe the response data stream from tile provider directly to the response stream. This however means that we need to "fork" the very same stream in order to be able to also pipe it to file stream.

• When a remote tile cannot be fetched, currently we return 404 and leaflet renders its grey tile. We could easily send another PNG image instead, to distinguish "in-progress" grey tiles and "failed" tiles.

• The application can come pre-bundled with tiles up to a certain zoom level.

• We may want to use any of the alternative map styles or design our own. See openmaptiles.org/styles.

[jameshadfield]

I'm not a fan of transpiling the server code as I value the simplicity of being able to run node server.js, although it does mean some code is a little bit more verbose. For background, auspice used to transpile the server code and it's been a much nicer dev experience since we stopped doing this.

[jameshadfield]

Thanks @ivan-aksamentov -- this is really cool & happy to see an implementation of this 😄

I haven't yet had a chance to really test the implementation but wanted to discuss two bigger points:

Tile provider

This PR switches to OpenStreetMap which allows caching by the (auspice) server. This is a big aesthetic change for auspice which we'll need to discuss internally, and test with different datasets to see how this changes our visualisation of demes on a map:

As mentioned in #714, MapBox tiles can be cached by a service worker in the browser, so that approach should work if we choose to keep MapBox. A nice side-effect of this PR would be the ability for implementations of auspice which use a custom server to define different map tile sources -- meaning that nextstrain.org could continue to use MapBox tiles by implementing a custom /tiles handler.

New API call

Background:
The auspice npm package / this github repo provides both a server (e.g. auspice view) and a client (built using auspice build). The auspice client is designed to be used by other projects (such as nextstrain.org), and they may choose to run the auspice server or design their own. This design allows projects to implement their own server handlers for the auspice-client-API calls which allows a great deal of flexibility. See these docs for more info.

This PR would require a major version bump and for all custom servers to implement a new handler for /tiles. (A lot of the code, such as TileCache.js can be exposed by auspice and therefore imported by such servers.) Note that we just released v2 a few weeks ago!

P.S. The actual API address should use getServerAddress() (https://github.com/nextstrain/auspice/blob/master/src/util/globals.js#L143), which by default is/charon. This allows it to be modified by a build-time customisation to auspice (docs here).

---

I'll make sure we discuss these bigger points internally over the coming two weeks & thanks again for the help here!