GeoJSON export does not include all features (even after using planet-latest)

[FDenker]

Hi,
I would see this issue as building on #186 . However, I have followed the advice and build a reproducible example that persists even when using the full planet-latest data.

The commands to reproduce the issue can be found below. I would appreciate any suggestions on what to test:

osmium getid -r planet-latest.osm.pbf  r57537 r64424 -v --output test_planet.osm.pbf
osmium export test_planet.osm.pbf -O -o testing_completeness.geojson -c settings.json

The two OSM ids that I chose for this are Cornwall, UK (r57537) and a river in Austria (r64424 ).

Both are found in the planet-latest.osm.pbf file (see python code) :

import esy.osm.pbf
osm_test = esy.osm.pbf.File('/data/shapefiles/osm/test_planet.osm.pbf')
ids = [tag[0] for tag in osm_test if type(tag) == esy.osm.pbf.file.Relation]
57537 in ids and 64424 in ids 
## -> Prints out True

However, no matter how I load in the GeoJSON, the ids are nowhere to be found:

import geopandas as gpd
data_from_file = gpd.read_file("/data/shapefiles/osm/testing_completeness.geojson")
data_from_file[data_from_file['@id'] == 64424]
## Returns an empty GeoDataFrame

I would be very helpful for any suggestion.

Best
Frederic

[joto]

What's in your settings.json? Can you put your test_planet.osm.pbf somewhere that we can try to reproduce this?

[FDenker]

Sorry. Yes I have attached the files as a ZIP File. If you prefer the manual files, the test_planet.osm.pbf can be found on google drive here and I have copied the settings.json below:

{
    "attributes": {
        "type":      true,
        "id":        true,
        "version":   false,
        "changeset": false,
        "timestamp": false,
        "uid":       false,
        "user":      false,
        "way_nodes": false
    },
    "format_options": {
    },
    "linear_tags":  ["id","type"],
    "area_tags":    [],
    "exclude_tags": [],
    "include_tags": []
}

[joto]

The Cornwall relation is broken. Somebody produced a mess here: https://tools.geofabrik.de/osmi/?view=areas&lon=-5.14383&lat=50.40510&zoom=18

DATA PROBLEM: intersection on r57537 (with 78456 nodes): way1_id=67145407 way1_seg_start=(-5.1436135,50.4051048) way1_seg_end=(-5.1435277,50.4050638) way2_id=945889839 way2_seg_start=(-5.1436,50.405066) way2_seg_end=(-5.1434958,50.4050823) intersection=(-5.143549,50.405074)
DATA PROBLEM: intersection on r57537 (with 78456 nodes): way1_id=945889839 way1_seg_start=(-5.1434958,50.4050823) way1_seg_end=(-5.1434063,50.4051242) way2_id=67145407 way2_seg_start=(-5.1434526,50.4050843) way2_seg_end=(-5.1434526,50.4051664) intersection=(-5.1434526,50.4051025)

The river tagging is broken. These kinds of tags should be on a way, not a relation. That's why this doesn't show up.

[lonvia]

The river relation is correct. osmium just doesn't handle these kind of relations. It also doesn't handle route relations.