pelias/whosonfirst

Support downloading individual country SQLite databases

orangejulius opened this issue · 4 comments

The Who's on First team now publishes country specific whosonfirst-data-admin-$$-latest.db SQLite databases.

These new files could be used to limit the amount of data needed for regional builds. This is a common request since the planet DB is quite large (27GB as of this writing).

There are at least a few considerations:

  • The XX and XY databases would probably always have to be downloaded, as they include records with either disputed or unknown countries.
  • We would likely need to keep the importPlace configuration parameter and add a countries parameter or something similar. I don't think it's possible to determine what the country code of a specific record or set of records passed to importPlace. So we have to make sure the experience is still good for users who specify invalid combinations of importPlace and country.

Are there any planes on this feature? It would be awesome.

Yes this is definitely happening this quarter but unfortunately isn't the sort of feature that usually attracts sponsorship funding and so it's a lower priority than some other features right now ;(

We've made some recent progress by publishing up-to-date data builds of WOF: https://wof.synergy.io/sqlite/

Data moved to a permanent home at https://geocode.earth/data/whosonfirst

This is done and should work as of #487!

It took an entire infrastructure (now sponsored by Geocode Earth) to generate proper country bundles, as well as some tweaks to the importer code here.

But you can now set the imports.whosonfirst.countryCode property to a 2-letter country code (as described in the readme). If anyone has any issues don't hesitate to let us know.