mapzen/chef-mapzen_odes

Generalized files at different tolerances have the same output

rmglennon opened this issue · 3 comments

rmglennon commented

From the IMPOSM spec, https://imposm.org/docs/imposm/latest/database_schema.html#generalized-tables

the layers gen0 (tolerance 200m) and gen1 (tolerance 50m) should be different, but have the same output for both shapefiles and geojsons.

To reproduce:

  1. Choose an extract. I tested with the Prague pre-made popular extract.
  2. Download either the geojson or the shapefile IMPOSM file.
  3. I opened the files in QGIS and they both had the same geometries and attributes.

A user alerted us about this through email support.

rmglennon commented

I also tested with a fresh, custom extract and see the same problem.

Also looks like issues with other generalized layers.

  • generalized waterareas and landusages only generate dbf, not the rest of the files needed to make a shapefile.
  • generalized waterareas and landusages are not created at all for geojson format.
heffergm commented

So there are two issues raised here, one of which isn't an issue but a symptom of the other, namely:

  • missing geojson for generalized tables is a result of those tables being empty, which results in an empty .dbf export and no .shp, hence no geojson is generated from said non-existent file.

The problem is that the generalized tables, loaded from the extracts we produce using the mapping found in https://github.com/mapzen/chef-mapzen_odes/blob/master/files/default/mapping.json, are empty.

From the imposm3 documentation regarding the mapping:

tolerance is the resolution used for the Douglas-Peucker simplification. It has the same unit as the import -srid, i.e. meters for EPSG:3857 and degrees for EPSG:4326.

I've been running tests this morning to try and get the generalized tables for waterareas and landusages to populate any data without any success. @migurski , and thoughts on what you think the mapping should look like? Do we even want to produce the generalized data? There was never any conscious decision made to do so, they just happened to be included in the default imposm3 mapping.

heffergm commented

As for the roads generalized tables, there are indeed differences in the geojson output depending on what location extract you're looking at between gen0 and gen1. So presumably for Prague, the available data simply happens to pass the threshold in both cases, making them the same. But if you look at SF Bay for example, you'll find differences.