Error message when importing diff
CartONG-staff opened this issue · 3 comments
Context
I created a postgis database from several .pbf files filtered by a geographic area
./imposm import -config /home/debian/imposm/imposm_files/config.json -read /home/debian/imposm/osm_original_files/moldova-latest.osm.pbf -write -limitto /home/debian/imposm/hcr_files/perimeter.geojson -diff -optimize
./imposm import -appendcache -config /home/debian/imposm/imposm_files/config.json -read /home/debian/imposm/osm_original_files/turkey-latest.osm.pbf -write -limitto /home/debian/imposm/hcr_files/perimeter.geojson -diff -optimize
./imposm import -appendcache -config /home/debian/imposm/imposm_files/config.json -read /home/debian/imposm/osm_original_files/papua-new-guinea-latest.osm.pbf -write -limitto /home/debian/imposm/hcr_files/perimeter.geojson -diff -optimize
./imposm import -appendcache -config /home/debian/imposm/imposm_files/config.json -read /home/debian/imposm/osm_original_files/africa-latest.osm.pbf -write -limitto /home/debian/imposm/hcr_files/perimeter.geojson -diff -optimize
./imposm import -appendcache -config /home/debian/imposm/imposm_files/config.json -read /home/debian/imposm/osm_original_files/asia-latest.osm.pbf -write -limitto /home/debian/imposm/hcr_files/perimeter.geojson -diff -optimize
./imposm import -config /home/debian/imposm/imposm_files/config.json -deployproduction
The database was created without errors, I added triggers on the tables to follow the updates and then I launched the run command as a service to launch the updates with these parameters on my config.json
"replication_url": "https://planet.openstreetmap.org/replication/hour/",
"replication_interval": "1h"
/home/debian/imposm/imposm_exec/imposm run -config /home/debian/imposm/imposm_files/config.json -expiretiles-dir /home/debian/imposm/expiretiles/ -limitto /home/debian/imposm/perimeter.geojson
Expected Behavior
Recover diff files and perform updates
Actual Behavior
The diff files are retrieved and processed but I get an error message when importing into postgis and the operation restarts again and again.
Nov 03 20:30:42 ns3112944 imposm[15406]: [2021-11-03T20:30:42Z] 0:00:00 [info] Starting replication from https://planet.openstreetmap.org/replication/hour/ with 1h0m0s interval
Nov 03 20:30:43 ns3112944 imposm[15406]: [2021-11-03T20:30:43Z] 0:00:01 [info] Importing #80103 including changes till 2021-11-01 22:00:00 +0000 UTC (46h30m43s behind)
Nov 03 20:30:43 ns3112944 imposm[15406]: [2021-11-03T20:30:43Z] 0:00:01 [step] Starting: Importing #80103
Nov 03 20:30:43 ns3112944 imposm[15406]: [2021-11-03T20:30:43Z] 0:00:01 [step] Starting: Processing /home/debian/imposm/imposm_diff/000/080/103.osc.gz
Nov 03 20:30:43 ns3112944 imposm[15406]: [2021-11-03T20:30:43Z] 0:00:01 [step] Starting: Parsing changes, updating cache and removing elements
Nov 03 20:30:43 ns3112944 imposm[15406]: [2021-11-03T20:30:43Z] 0:00:01 [step] Finished: Processing /home/debian/imposm/imposm_diff/000/080/103.osc.gz in 38.422606ms
Nov 03 20:30:43 ns3112944 imposm[15406]: [2021-11-03T20:30:43Z] 0:00:01 [step] Finished: Importing #80103 in 39.827829ms
Nov 03 20:30:43 ns3112944 imposm[15406]: [2021-11-03T20:30:43Z] 0:00:01 [error] Importing #80103: delete element osm.Diff{Create:false, Modify:true, Delete:false, Node:(*osm.Node)(0xc00051df80), Way:(*osm.Way)(nil), Rel:(*osm.Relation)(nil)}: deleting -1600844 from "ong_administrative_boundary_polygon": SQL Error: pq: null value in column "osm_id" violates not-null constraint in query DELETE FROM "public"."osm_ong_administrative_boundary_polygon" WHERE "osm_id" = $1 (-1600844)
Nov 03 20:30:43 ns3112944 imposm[15406]: [2021-11-03T20:30:43Z] 0:00:01 [info] Retrying in 2s
Your Environment
- Version used: Imposm3 v0.11.1
- Environment name and version (PostgreSQL 12.8 with PostGIS 3.1:
- Operating System and version:Debian
I did a new test today with the OSM exports of the day and therefore different diff files and I have a similar error, the problem does not come from an error in a diff file.
Hi @CartONG-staff, could you share your config file to help debug it ?
Hi @francois2metz , I found the origin of the error, in fact the error was in the trigger function (the NEW variable was used instead of OLD, which resulted in a null insert in my update monitoring table) which blocked the update transaction. Once the trigger is corrected, the update is done correctly.