/hilbert_frame

Experimental spatially-partitioned data structure for use with the Datashader

Primary LanguageJupyter NotebookMIT LicenseMIT

Overview

This repo was an experiment in creating a spatially partitioned data structure for use with the Datashader project. See holoviz/datashader#678 for more background.

It was integrated into Datashader in pyviz/datashader#702 and pyviz/datashader#702

nbviewer Notebooks

  1. Spatial partitioning of Dask DataFrames using Hilbert curves
  2. Hilbert-curve spatial data structure performance on 300 million point census dataset
  3. Faster interactive exploration of 1 billion points

Credits

The hilbert curve implementation is a 2D specialization and numba optimization of the excellent work by @galtay in https://github.com/galtay/hilbert_curve