Writting osm data to postgresql sometimes get error
scku208 opened this issue · 1 comments
Context
I use imposm-0.11.1-linux-x86-64/imposm import -config imposm3_config_taiwan.json -write -dbschema-import importtest
to import osm data after reading it.
Somtimes it worked and finished.
Finished: Imposm in 2m52.851072235s
Sometimes not. With different kinds of errors about permission:
pq: could not open file "base/13395/1142219": Permission denied
[fatal] bulk insert into "osm_admin": SQL Error: pq: could not open file "base/13395/2673_fsm": Permission
denied in query COPY "importtest"."osm_admin" ("osm_id", "geometry", "name", "name_en", "int_name", "type", "admin_level", "area") FROM STDIN
0:00:04 [fatal] bulk insert into "osm_admin": SQL Error: pq: could not open file "base/13395/1137580": Permission d
enied in query COPY "importtest"."osm_admin" ("osm_id", "geometry", "name", "name_en", "int_name", "type", "admin_level", "area") FROM STDIN
0:00:00 SQL Error: pq: could not create file "base/13395/1131388": Permission denied in query
CREATE TABLE IF NOT EXISTS "importtest"."osm_transport_points" (
id SERIAL,
"osm_id" BIGINT,
"name" VARCHAR,
"name_en" VARCHAR,
"int_name" VARCHAR,
"type" VARCHAR,
"ref" VARCHAR,
PRIMARY KEY ("osm_id", "id")
);
Expected Behavior
Should finished all the time.
Actual Behavior
Sometimes success, sometimes not.
Possible Fix
I'm using VPN to office computer wihich has docker with postgresql + postgis which mount a NAS share folder by sshfs.
Maybe simplify to a docker db without mount volume or just a localhost db can solve,
but I really need disk space from NAS.
Steps to Reproduce
- run
imposm-0.11.1-linux-x86-64/imposm import -config imposm3_config_taiwan.jso n -write -dbschema-import importtest
multiple times for testing. - sometimes worked, somtimes not.
Not minimum, but I use the osm.pdf file from geofabrik
Context
I will try dockerize another local db without mount volume to test, but office computer cannot use for production without enough disk space.
Your Environment
- Version used: 0.11.1 binary
- Environment name and version (e.g. PostgreSQL x.y.z with PostGIS a.b.c): docker image postgis/postgis:13-master
- PostgreSQL 13.2 (Debian 13.2-1.pgdg100+1) on x86_64-pc-linux-gnu, compiled by gcc (Debian 8.3.0-6) 8.3.0, 64-bit
- POSTGIS="3.2.0dev 3.1.0rc1-56-g52f942583" [EXTENSION] PGSQL="130" GEOS="3.10.0dev-CAPI-1.15.0" PROJ="7.2.1"
LIBXML="2.9.4" LIBJSON="0.12.1" LIBPROTOBUF="1.3.1" WAGYU="0.5.0 (Internal)" TOPOLOGY
- Server type and version: None
- Operating System and version: 20.04.2 LTS (GNU/Linux 5.8.0-55-generic x86_64)
I do some testing (local docker db without mount NAS volumes): 100% success. (7/7)
Finished: Imposm in 53.751675598s
Finished: Imposm in 53.618025753s
Finished: Imposm in 50.150860108s
Finished: Imposm in 23.377249433s
Finished: Imposm in 23.641579415s (with -dbschema-import importtest2)
Finished: Imposm in 24.185745121s (with -dbschema-import importtest2)
Finished: Imposm in 53.676281275s (with -dbschema-import importtest2)
Then, I think it's not a bug from imposm3 but mount NAS folder with postgresql or sshfs etc.
If any suggestion will be great, and thank you for the masterpiece - imposm3.