H3 Population Helium

A global population dataset based on Uber's H3 mapping system and seeded from the open source Kontur Population dataset.

Optimized for easy consumption in Helium mapping projects.

Using the data

Download the latest kontur_population_20211109.csv.zip from this repository and extract it to your working directory.

The CSV file contains no header and two columns:

hex_res8 (string): H3 index at resolution 8 truncated to first 10 chars. For example, the hex 8828309565fffff will be 8828309565. If you need the full H3 index for any hex, simply append fffff.
population (integer): Total human population in this hex

The file is hundreds of megabytes, so it's recommended to import it into a database like Postgres for use. You can find an example of how to import a CSV into Postgres in this StackOverflow answer.

Preparing Kontur Population dataset in PostgreSQL from scratch

Although the Kontur Population dataset uses H3 resolution 8 to segment population data, it doesn't provide the H3 index of each hex. Instead, it provides a set of points that make up each hex.

We need the H3 index of each hex to easily join and compare with data in the Helium ecosystem. Let's calculate the H3 indices ourselves!

Requirements

PostgreSQL 9 or higher with PostGIS extension installed
ogr2ogr CLI (instructions below)

1. Download and extract

Download the latest dataset from this page.

The file will be of the format .gpkg.gz. Extract the .gpkg file to your working directory.

2. Use ogr2ogr to import gpkg into Postgres

You'll need ogr2ogr CLI installed for this.

If you're using Homebrew on macOS, the following command will accomplish this:

brew install gdal

Then, run the following command to import into your Postgres DB. This assumes you're running Postgres on localhost and that your DB name is heliumpop.

ogr2ogr -f PostgreSQL "PG:dbname=heliumpop" kontur_population_20211109.gpkg

This will import the data into a table called kontur_population. NOTE: If the import is stopped partway through, make sure to TRUNCATE this table before restarting the import or you will get duplicate data.

3. Convert geometry to SRID 4326 and find centers

The dataset includes hexagons in the format of PostGIS Geometry objects with SRID 3857 (Spherical Mercator). Since we're looking to get the H3 index of each hex, we need to convert the hex geometry to lat/long (SRID 4326) and find the center.

Open psql or your favorite Postgres client and run the following queries:

alter table kontur_population
add column center geometry(Geometry,4326);

update kontur_population
set center = st_centroid(st_transform(geom,4326));

4. Add H3 index to dataset

We're almost there! Add a new column for the H3 index of each hex and create a unique index to ensure consistency.

alter table kontur_population
add column hex_res8 text;

create unique index on kontur_population (hex_res8);

Clone this repo, then install required dependencies for the next step:

npm install

Copy .env.sample to a new file called .env and add your Postgres database credentials.

Then, run the H3 index update script. This will add H3 indexes, truncated to the first 10 significant characters, to the hex_res8 column of the kontur_population table:

npm run update-h3-index

5. Use or export data

And that's it! You can create the final dataset using the following query and export to CSV using your favorite tool:

select hex_res8, population
from kontur_population

rawrmaan/h3-population-helium