Enable caching of offline basemaps using service workers

[modulitos]

Some useful resources:

slides from @npeihl's talk at a recent CUGOS meeting that inspired this issue:
http://www.npeihl.com/service-worker-presentation/#0

@npeihl's project which uses service workers:
https://github.com/sjcgis/polarisjs

Here is a useful package that might be useful when setting up the boilerplate for the service worker precaching:
https://github.com/GoogleChrome/sw-precache

More info on service workers:
https://developers.google.com/web/fundamentals/getting-started/primers/service-workers
https://developer.mozilla.org/en-US/docs/Web/API/Service_Worker_API/Using_Service_Workers

• Service Workers require SSL, so we'll need to fix #394 first

[goldpbear]

I think this would be a nice thing to have in place for mywater.world. @Lukeswart -- do you think the static site is basically a prerequisite for this issue, given the SSL requirements?

[jalMogo]

This doesn't look easy to implement. The examples above are only caching tile requests as they go out, but not pre-fetching the tile pyramids.

In short, this would to take a lot of work and extra ongoing effort to maintain our vector tiles:

• store a file on our s3 bucket (eg: vector-tiles.mapseed.org/some-layer/tile-info.json) that contains a list of all the tiles in that layer (eg: vector-tiles.mapseed.org/some-layer/8/1234/5678.pbf). tile-info.json is generated using the result of calling aws s3 ls s3://my-bucket/some-layer/ --recursive.
• the client fetches that tile-info.json file to know the names of all tile files in the layer. This could be consist of several thousand tile files. For each tile, we can test whether it fits within our zoom, x, and y selection by mapping the lat/long of the bounding box to the z/x/y attributes in the tile's file path.
• We might be able to limit the tiles that we need to fetch down to only a few hundred by restricting the zoom to be no more than 14 or so.
• Now, we can fetch all of the tiles within our bounding box and zoom level. Note that this may still require several hundred requests - one for each tile.

Some restrictions:

• we can only provide this offline pre-fetch feature for tiles that we host ourselves, because we'll need the tile-info.json file to know where all of our tiles are
• If we are downloading more than a few layers, we can easily exceed 1000's of requests. Not sure how this will affect throttling or performance on the client.
  • note that we can take a shortcut here, and have the tile-info.json file only go down to zoom level 12 or so. This should keep the requests well under 1000 for each layer.
• To keep the number of tiles to fetch down, we'll have to limit the zoom level to 14 or so.

Note that this should work for raster tiles as well as vector tiles, as long as they both use a consistent z/y/x file hierarchy, which we can map by using the lat/long of the bounding box.

I think a more robust solution is to download all of the vector tiles at once, unzip them on the client, determine the file paths, and manually populate our cache using the file patch and tiles. But unless we limit the vector tiles bundle to just the bounding box (or zoom levels), downloading and unzipping thousands of tiles could stall the client. Also, this would requires more implementation, and not sure if it's worth the trouble.

[goldpbear]

Interesting-- it sounds like there is significantly more complexity to pre-fetching tiles for a given bounding box than we thought, especially if we're fetching data from sources we don't directly control.

Still, it seems like there must be a reasonable solution that simply takes a bounding box, max zoom, and list of layer sources and figures out all the individual tile requests that are necessary.

I found this SO post interesting: https://gis.stackexchange.com/questions/189445/finding-the-map-tilesz-x-y-tile-data-in-a-given-bounding-box-and-zoom-level

It contains a PHP implementation for translating between a lat/lng bounding box and a range of x and y rows/columns that would constitute complete coverage at a given zoom, assuming tiles sizes of 256px/256px. At least that's how I understood it at first read.

The math behind the lat/lng to x/y translation looks a little hairy, but seems established. This page discusses the details: https://wiki.openstreetmap.org/wiki/Slippy_map_tilenames. It also gives a JS sample implementation:

function long2tile(lon,zoom) { 
  return (Math.floor((lon+180)/360*Math.pow(2,zoom))); 
}

function lat2tile(lat,zoom)  { 
  return (Math.floor((1-Math.log(Math.tan(lat*Math.PI/180) + 1/Math.cos(lat*Math.PI/180))/Math.PI)/2 *Math.pow(2,zoom))); 
}

I agree we should discuss the merits of pursuing this too much further, but I would be happy to try and help get this part of the offline system working. But I would also need to read more deeply-- everything noted here is just a first stab, so there very well could be other hangups that we're missing too.