Generating .json from planet.osm.pbf results in process being 'killed'.
Opened this issue · 1 comments
When I try to generate a .json file based on either unfiltered or filtered by boundaries/admin_level planet.osm.pbf the process gets 'killed' after a while. I'll leave more info below and attached in the image.
I'm using WSL2 and the steps I'm doing are just:
1 - Get planet.osm.pbf file(in my case I got it from the torrent version);
2 - Filter pbf file by either boundary or admin_level, which would result in a considerably smaller file. Additionally I'm also filtering some other info out. The commands I'm using are these:
osmium tags-filter planet-latest.osm.pbf r/admin_level --overwrite -o planet-admin.osm.pbf
osmconvert planet-admin.osm.pbf -o=planet-admin.osm
osmfilter planet-admin.osm --drop-tags="barrier= building= highway= landuse= office= place= waterway=" -o=planet-admin-noplace.osm
osmconvert planet-admin-noplace.osm -o=planet-admin-noplace.osm.pbf;
3 - Build cosmogony with command:
cargo run --release -- generate -i ../../planet-admin-noplace.osm.pbf
It's in this command that the process is getting "killed".
(Note: This happens with unfiltered planet.osm.pbf as well. I also don't know if this could be related to #118 ...)
I've been researching, and I found that my system might be killing the process after a possible OOM Panic. I've tried to set overcommit memory to 2, meaning it will try not to do it. I've also changed vm.oom-kill to 0. Let's see what'll happen now...
Edit: I was able to fully process and import unfiltered Poland-latest .osm.pbf abd run the cosmogony-explorer with it without the process being killed. I was also able to do the same but to asia osm.pbf file filtered by r/admin_level and without additional unnecessary tags(like in the command above), but when I tried to do the same with either planet or europe osm.pbf files it gets killed. I thought it could be due to the docker OOM Killswitch so I turned docker off, but now it still kills it(with RAM set to 8GB and CPU to 3/6 in the wslconfig file). I'll try with CPU 6/6 and RAM 14/16.