Segfault when importing global file to DB

[four43]

Context

I've separated out my read and write commands into two different steps. I am running in minimal docker images and experienced this panic after ~1.5hrs of processing.

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x5a0231]

goroutine 290 [running]:
github.com/omniscale/imposm3/cache.(*DeltaCoordsCache).CheckCapacity(0xc00012c3c0)
      /root/imposm/gopath/src/github.com/omniscale/imposm3/cache/delta.go:352 +0x111
github.com/omniscale/imposm3/cache.(*DeltaCoordsCache).getBunch(0xc00012c3c0, 0x16a7e1d7)
      /root/imposm/gopath/src/github.com/omniscale/imposm3/cache/delta.go:330 +0x3c5
github.com/omniscale/imposm3/cache.(*DeltaCoordsCache).FillWay(0xc00012c3c0, 0xc004a22780)
      /root/imposm/gopath/src/github.com/omniscale/imposm3/cache/delta.go:180 +0x193
github.com/omniscale/imposm3/writer.(*RelationWriter).loop(0xc00018ae70)
      /root/imposm/gopath/src/github.com/omniscale/imposm3/writer/relations.go:91 +0x31b
created by github.com/omniscale/imposm3/writer.(*OsmElemWriter).Start in goroutine 1
      /root/imposm/gopath/src/github.com/omniscale/imposm3/writer/writer.go:52 +0x3e

Expected Behavior

imposm continues to run till completion, exit 0. This worked fine with the smaller regions of monaco and central USA.

Actual Behavior

Panic, segfault

Possible Fix

Unknown

Steps to Reproduce

1. Using FROM debian:12.8, install released version v0.14.0
2. Download global OSM pdb from S3, aws s3 cp --no-sign-request s3://osm-pds/2024/planet-241202.osm.pbf ./country.pbf
3. Run via import -mapping /config/mapping.yml -write -cachedir /cache -connection postgis://osm:osm@db/osm -srid 4326 -deployproduction -optimize

Context

I'm trying to get some local maps running to turn them into PMTiles

Your Environment

Debian Linux running Docker

Linux [hostname] 5.10.0-28-amd64 #1 SMP Debian 5.10.209-2 (2024-01-31) x86_64 GNU/Linux

sudo docker version
Client: Docker Engine - Community
 Version:           26.0.1
 API version:       1.45
 Go version:        go1.21.9
 Git commit:        d260a54
 Built:             Thu Apr 11 10:53:52 2024
 OS/Arch:           linux/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          26.0.1
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.9
  Git commit:       60b9add
  Built:            Thu Apr 11 10:53:52 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.31
  GitCommit:        e377cd56a71523140ca6ae87e30244719194a521
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Thanks all, great project!

[four43]

Yeah this is not handling invalid data correctly...

panic: proto: Way: illegal tag 0 (wire type 7)

goroutine 195 [running]:
github.com/omniscale/imposm3/cache.(*WaysCache).Iter.func1()
      /root/imposm/gopath/src/github.com/omniscale/imposm3/cache/ways.go:90 +0x17a
created by github.com/omniscale/imposm3/cache.(*WaysCache).Iter in goroutine 1
      /root/imposm/gopath/src/github.com/omniscale/imposm3/cache/ways.go:77 +0x78
import-write-1 exited with code 2