omniscale/imposm3

support for optional jsonb_tags instead of hstore

RobinKamps opened this issue · 2 comments

RobinKamps commented

currently hstore extension is needed to import tags.
however json columns provide more or less the same functionality.
cockroachdb e.g. has no buioltin hstore functionality nor a hstore-extension, but instead could use jsonb columns for tags.
is there a way to utilize json columns instead of hstore, or is this feature planned?
Thanks in advance

ImreSamu commented

is there a way to utilize json columns instead of hstore,

imho: with a minimal Golang knowledge - it is easy to extend - with new columns types.

  • imposm3/mapping/columns.go

    Lines 20 to 43 in 53bb807

    AvailableColumnTypes = map[string]ColumnType{
    "bool": {"bool", "bool", Bool, nil, nil, false},
    "boolint": {"boolint", "int8", BoolInt, nil, nil, false},
    "id": {"id", "int64", ID, nil, nil, false},
    "string": {"string", "string", String, nil, nil, false},
    "direction": {"direction", "int8", Direction, nil, nil, false},
    "integer": {"integer", "int32", Integer, nil, nil, false},
    "mapping_key": {"mapping_key", "string", KeyName, nil, nil, false},
    "mapping_value": {"mapping_value", "string", ValueName, nil, nil, false},
    "member_id": {"member_id", "int64", nil, nil, RelationMemberID, true},
    "member_role": {"member_role", "string", nil, nil, RelationMemberRole, true},
    "member_type": {"member_type", "int8", nil, nil, RelationMemberType, true},
    "member_index": {"member_index", "int32", nil, nil, RelationMemberIndex, true},
    "geometry": {"geometry", "geometry", Geometry, nil, nil, false},
    "validated_geometry": {"validated_geometry", "validated_geometry", Geometry, nil, nil, false},
    "hstore_tags": {"hstore_tags", "hstore_string", nil, MakeHStoreString, nil, false},
    "wayzorder": {"wayzorder", "int32", nil, MakeWayZOrder, nil, false},
    "pseudoarea": {"pseudoarea", "float32", nil, MakePseudoArea, nil, false},
    "area": {"area", "float32", Area, nil, nil, false},
    "webmerc_area": {"webmerc_area", "float32", WebmercArea, nil, nil, false},
    "zorder": {"zorder", "int32", nil, MakeZOrder, nil, false},
    "enumerate": {"enumerate", "int32", nil, MakeEnumerate, nil, false},
    "string_suffixreplace": {"string_suffixreplace", "string", nil, MakeSuffixReplace, nil, false},

I have created an old - and not updated PR ( in 2016 )

  • for extending column types - as an "addon" with this filedtypes ( json / jsonb mapping included .. ) .. but I have not finished .. and not working with the current codebase .. so need to adapt.
RobinKamps commented

Thank you very much.
It was indeed very easy to add a new column type directly to imposm (not as an addon/user custom type).
After some additional tweaking of the sql dialect - some commands from imposm are not supported by cockroach e.g. "RESTART IDENTITY" or "DropGeometryTable" and a workaround to an annoying json escape problem on cockroach COPY, i finally got a working tag import with cockroach and jsonb instead of hstore.