Connections with datashader

[jhamman]

Earlier this week, @jbednar had some interesting questions for me about possible datashader applications for pyramid generation. I wasn't tracking at the time but I've come around to the idea that datashader could play an interesting role here. So I'm opening this issue to give it a second try.

Due to my lack of understanding of how datashader works under the hood, I'm curious which of the following tasks datashader could help with:

1. Projection from one coordinate system to another (e.g. epsg:4326 to epsg:3857)
2. Spatial aggregation of data within an existing coordinate system (most likely epsg:3857)

[jbednar]

What I was thinking of was that Datashader could make use of a previously calculated ndpyramid for its own purposes when displaying overviews, which would provide vastly better performance when zoomed far out. Without such a pyramid, Datashader has to process the entire dataset to give such an overview, and automatically switching over to the appropriate subsampled version could be a big win for interactive usage.

Going the other direction, Datashader can easily generate the spatial pyramid in the first place (point 2), and indeed, for our purposes it's better if it does so, because then everything adds up (literally), i.e. the subsampled ones will accurately represent the higher-resolution versions. Not sure how you are generating them at present, or whether Datashader's downsampling code has advantages or disadvantages there. We definitely don't yet have all the types of interpolation that could be useful in such cases.

I did add a projection function to Datashader (point 1), but it's a trivial 4-line use of NumPy (see https://github.com/holoviz/datashader/blob/master/datashader/utils.py#L402); it's solely there to project things into Bokeh's preferred coordinate system for display. So it's not really a general-purpose system for projections, and probably can't become so given that it's a general purpose non-geo tool.