Zverik/regional

trim_osc.py incompatible with flat nodes

Istador opened this issue · 0 comments

When using a full planet import, the trim_osc.py script isn't necessary. When not using a full planet import, you need the trim to prevent data out of bounds to be added to the database by updates.

But even when not usind a full planet import, depending on the use case, it might be beneficial to use flat nodes (e.g. when using continents).

The problem with the trim_osc.py when using flat nodes is that the database table PREFIX_nodes doesn't exist. But it is always accessed by the script to check if a node exists in the database or not (so that it is not trimmed if it exists, e.g. for partly out of bounds polygons).

regional/trim_osc.py

Lines 128 to 129 in a618ea4

q1='select id from {0}_nodes where id = ANY(%s);'.format(prefix)
cur.execute(q1, (nodesM,))

Because the database table doesn't exists, the script will always fail with an error:

[2022-05-23 11:45:14] 4708 filtering diff
Traceback (most recent call last):
  File "/home/renderer/src/regional/trim_osc.py", line 129, in <module>
    cur.execute(q1, (nodesM,))
psycopg2.errors.UndefinedTable: relation "planet_osm_nodes" does not exist
LINE 1: select id from planet_osm_nodes where id = ANY(ARRAY[4937988... 

What's needed is an additional parameter for the trim_osc.py script, that let you provide a path to the binary flat nodes file. When provided, the script should no longer try to access the nodes table in the database to check wether a node exists, but instead should look into the flat nodes file.

Note: This might not be an easy change. The flat node file is a very big binary file. But with knowledge of the internal file structure it should be possible to calculate the exact position for any osm_id inside the file and to read out its value directly.