geojson_intersects does not work for linestrings

When testing geojson_intersects_feature I encountered a segmentation violation

Error

/path/to/imposm/imposm import -connection 'postgres: host=<host> port=5432 dbname=mapdata user=importer password=<pass>' \
-mapping /path/to/mapping.yaml -cachedir /mnt/ebs/imposm/cache -appendcache -diffdir /mnt/ebs/imposm/diff -diff -read /mnt/ebs/imposm/input/washington-191208.osm.pbf

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x8 pc=0x5eda8a]
goroutine 1 [running]:
github.com/omniscale/imposm3/geom/geojson.constructPolygonFeatures(0x0, 0x7, 0xa39140, 0x7, 0xffffffffffffffff, 0x0)
	/home/travis/gopath/src/github.com/omniscale/imposm3/geom/geojson/geojson.go:136 +0x3a
github.com/omniscale/imposm3/geom/geojson.constructPolygonFeatures(0xc000110a10, 0xc0005691e0, 0x1, 0x1, 0xc0005691c0, 0x1)
	/home/travis/gopath/src/github.com/omniscale/imposm3/geom/geojson/geojson.go:148 +0x29c
github.com/omniscale/imposm3/geom/geojson.constructPolygonFeatures(0xc00024e9b0, 0x949b80, 0xc00024e9b0, 0x0, 0x0, 0x4346ca)
	/home/travis/gopath/src/github.com/omniscale/imposm3/geom/geojson/geojson.go:161 +0x5a9
github.com/omniscale/imposm3/geom/geojson.ParseGeoJSON(0xaff620, 0xc000162fe0, 0x0, 0x0, 0xc000162fe0, 0x0, 0x0)
	/home/travis/gopath/src/github.com/omniscale/imposm3/geom/geojson/geojson.go:126 +0xaf
github.com/omniscale/imposm3/mapping.loadFeatures(0xc0001657f0, 0xc, 0x0, 0x0, 0x0, 0x0, 0x0, 0xc0001e98a0, 0x1a, 0xc00015dd40, ...)
	/home/travis/gopath/src/github.com/omniscale/imposm3/mapping/columns_intersection.go:47 +0x21e
github.com/omniscale/imposm3/mapping.MakeIntersectsFeatureField(0xc0001657f0, 0xc, 0xa46e12, 0x1a, 0xa38b6c, 0x6, 0x0, 0xa648c0, 0x0, 0x0, ...)
	/home/travis/gopath/src/github.com/omniscale/imposm3/mapping/columns_intersection.go:71 +0x60
github.com/omniscale/imposm3/mapping.MakeColumnType(0xc000250f00, 0xc00029e9c0, 0x0, 0x0)
	/home/travis/gopath/src/github.com/omniscale/imposm3/mapping/mapping.go:230 +0x243
github.com/omniscale/imposm3/mapping.makeRowBuilder(0xc0002425a0, 0xa3c079, 0xa, 0x1)
	/home/travis/gopath/src/github.com/omniscale/imposm3/mapping/mapping.go:213 +0xd4
github.com/omniscale/imposm3/mapping.(*Mapping).tables(0xc00016e680, 0xa3c079, 0xa, 0xc00028f020, 0x0, 0x0)
	/home/travis/gopath/src/github.com/omniscale/imposm3/mapping/mapping.go:196 +0xf8
github.com/omniscale/imposm3/mapping.(*Mapping).lineStringMatcher(0xc00016e680, 0xaff000, 0xc00028e600, 0x0, 0x0)
	/home/travis/gopath/src/github.com/omniscale/imposm3/mapping/matcher.go:29 +0xc8
github.com/omniscale/imposm3/mapping.(*Mapping).createMatcher(0xc00016e680, 0x0, 0x0)
	/home/travis/gopath/src/github.com/omniscale/imposm3/mapping/mapping.go:150 +0x75
github.com/omniscale/imposm3/mapping.New(0xc000190000, 0x6755, 0x6955, 0x6755, 0x6955, 0x0)
	/home/travis/gopath/src/github.com/omniscale/imposm3/mapping/mapping.go:113 +0xb2
github.com/omniscale/imposm3/mapping.FromFile(0x7fff24746b62, 0x20, 0x8, 0xc000096900, 0xc000111350)
	/home/travis/gopath/src/github.com/omniscale/imposm3/mapping/mapping.go:98 +0x87
github.com/omniscale/imposm3/import_.Import(0x7fff24746ab0, 0xa8, 0x7fff24746b8d, 0x15, 0x7fff24746bb9, 0x14, 0x7fff24746b62, 0x20, 0xf11, 0x0, ...)
	/home/travis/gopath/src/github.com/omniscale/imposm3/import_/import.go:50 +0xf8
main.Main(0xa64b30)
	/home/travis/gopath/src/github.com/omniscale/imposm3/cmd/imposm/main.go:48 +0x4db
main.main()
	/home/travis/gopath/src/github.com/omniscale/imposm3/cmd/imposm/main.go:77 +0x2d

The extract from the config is

tables:
  highway_linestring:
    fields:
    - {name: osm_id, type: id}
    - {name: geometry, type: geometry}
    - {name: country_code, type: geojson_intersects_feature, args: {geojson: /path/to/mapping/borders.json, property: iso1A2}}
    mapping:
      highway: [motorway, motorway_link, trunk, trunk_link, primary, primary_link,
        secondary, secondary_link, tertiary, tertiary_link, unclassified, residential,
        road, living_street, raceway, track, service, path, cycleway, bridleway, footway,
        corridor, pedestrian, steps]
      man_made: [pier]
      public_transport: [platform]
    type: linestring

/path/to/mapping/borders.json is this file from country-coder

I tried trimming borders.json to just the Canada and USA polygons but still got the error. I also tried forcing the right-hand rule with http://mapster.me/right-hand-rule-geojson-fixer/ and still got the error.

I've reduced this to a smaller testcase

imposm import -connection postgis://paunorma@localhost/imposm -mapping config.yml -overwritecache -read w5255262.osm.pbf -write

Where config.yml is

tables:
  highway_linestring:
    fields:
    - {name: osm_id, type: id}
    - {name: geometry, type: geometry}
    - {name: country_code, type: geojson_intersects_feature, args: {geojson: borders.json, property: iso1A2}}
    mapping:
      highway: [residential]
    type: linestring

and borders.json is

{
  "type": "FeatureCollection",
  "features": [
    {
      "type": "Feature",
      "properties": {"iso1A2": "US"},
      "geometry": {
        "type": "Polygon",
        "coordinates": [[[-122.7,48.8],[-122.6,48.8],[-122.6,48.9],[-122.7,48.9],[-122.7,48.8]]]
      }
    }
  ]
}

and w5255262.osm.pbf is obtained by osmium cat w5255262.osm -o w5255262.osm.pbf and w5255262.osm is

<?xml version='1.0' encoding='UTF-8'?>
<osm version="0.6" generator="osmium/1.11.1">
  <bounds minlat="45.54326" minlon="-126.7423" maxlat="49.00708" maxlon="-116.9145"/>
  <node id="37064015" version="3" timestamp="2013-12-11T18:41:15Z" lat="48.8462652" lon="-122.6477865"/>
  <node id="37064016" version="3" timestamp="2013-12-11T18:41:15Z" lat="48.8463666" lon="-122.6478154"/>
  <node id="37064017" version="3" timestamp="2013-12-11T18:41:15Z" lat="48.8474395" lon="-122.6483791"/>
  <node id="2576091539" version="1" timestamp="2013-12-11T18:41:14Z" lat="48.8473304" lon="-122.6479495"/>
  <node id="2576091541" version="1" timestamp="2013-12-11T18:41:14Z" lat="48.8482759" lon="-122.6483664"/>
  <way id="5255262" version="4" timestamp="2017-06-05T19:55:38Z">
    <nd ref="37064015"/>
    <nd ref="37064016"/>
    <nd ref="2576091539"/>
    <nd ref="37064017"/>
    <nd ref="2576091541"/>
    <tag k="highway" v="residential"/>
    <tag k="name" v="Barr Road"/>
    <tag k="tiger:cfcc" v="A41"/>
    <tag k="tiger:county" v="Whatcom, WA"/>
    <tag k="tiger:name_base" v="Barr"/>
    <tag k="tiger:name_type" v="Rd"/>
    <tag k="tiger:reviewed" v="no"/>
  </way>
</osm>

This testcase gives a different segmentation violation

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x419a5f0]

goroutine 68 [running]:
github.com/omniscale/imposm3/geom/geos.(*Geos).IndexQuery.func1(0xc000142840, 0xc00013c990, 0x0, 0xc00030e0f0, 0x203000)
	/Users/paunorma/go/src/github.com/omniscale/imposm3/geom/geos/index.go:65 +0x40
github.com/omniscale/imposm3/geom/geos.(*Geos).IndexQuery(0xc000142840, 0xc00013c990, 0x0, 0x0, 0x0, 0x0)
	/Users/paunorma/go/src/github.com/omniscale/imposm3/geom/geos/index.go:65 +0xee
github.com/omniscale/imposm3/mapping.MakeIntersectsFeatureField.func1(0x0, 0x0, 0xc00023e020, 0xc00023e040, 0x463705f, 0x7, 0x463a24a, 0xb, 0xc00014c100, 0x12, ...)
	/Users/paunorma/go/src/github.com/omniscale/imposm3/mapping/columns_intersection.go:88 +0x77
github.com/omniscale/imposm3/mapping.(*valueBuilder).Value(...)
	/Users/paunorma/go/src/github.com/omniscale/imposm3/mapping/matcher.go:242
github.com/omniscale/imposm3/mapping.(*rowBuilder).MakeRow(0xc000142400, 0xc00023e020, 0xc00023e040, 0x463705f, 0x7, 0x463a24a, 0xb, 0xc00014c100, 0x12, 0x0, ...)
	/Users/paunorma/go/src/github.com/omniscale/imposm3/mapping/matcher.go:270 +0x2e2
github.com/omniscale/imposm3/mapping.(*Match).Row(...)
	/Users/paunorma/go/src/github.com/omniscale/imposm3/mapping/matcher.go:118
github.com/omniscale/imposm3/database/postgis.(*PostGIS).InsertLineString(0xc000198140, 0x50305e, 0xc000146540, 0x0, 0x0, 0xc00031c000, 0xba, 0xba, 0xc000240050, 0x1, ...)
	/Users/paunorma/go/src/github.com/omniscale/imposm3/database/postgis/postgis.go:474 +0x475
github.com/omniscale/imposm3/writer.(*WayWriter).buildAndInsert(0xc0000ab7c0, 0xc0001bcf70, 0xc00029a050, 0xc000240050, 0x1, 0x1, 0x0, 0x0, 0x0, 0x0)
	/Users/paunorma/go/src/github.com/omniscale/imposm3/writer/ways.go:152 +0x760
github.com/omniscale/imposm3/writer.(*WayWriter).loop(0xc0000ab7c0)
	/Users/paunorma/go/src/github.com/omniscale/imposm3/writer/ways.go:101 +0x57a
created by github.com/omniscale/imposm3/writer.(*OsmElemWriter).Start
	/Users/paunorma/go/src/github.com/omniscale/imposm3/writer/writer.go:52 +0x81

Debugging with delve and stepping to

imposm3/mapping/columns_intersection.go

Line 88 in 3b6e6b3

indices := g.IndexQuery(idx, geom.Geom)

tells me that geom is

(dlv) print geom
*github.com/omniscale/imposm3/geom.Geometry {
	Geom: *github.com/omniscale/imposm3/geom/geos.Geom nil,
	Wkb: []uint8 len: 186, cap: 186, [48,49,48,50,48,48,48,48,50,48,49,49,48,102,48,48,48,48,48,53,48,48,48,48,48,48,102,102,98,54,56,51,50,52,56,99,48,97,54,97,99,49,54,50,97,48,100,102,102,99,53,98,100,54,53,55,52,49,53,52,55,100,56,54,...+122 more],}

Trying with and without the geojson_intersects_feature column, the Geom is nil in both cases.

@olt, do you have a working example of geojson_intersects_feature?

Thanks for digging into this. Apparently this feature does not work with linestrings. It expects a GEOS geometry but simple linestrings are directly converted to WKB.

Are GEOS geometries not constructed for linestrings for performance reasons? If so, do you think building them would be an unacceptable performance loss?

ping @olt

Yes, it speeds up the import. Unacceptable? No, but it's a nice performance gain and I actually wanted to extend this in the future for polygons, but the Area column types are also require a GEOS geometry.

Disabling this optimization if there is a mapping that requires GEOS geometries might be a good compromise.