Merge multiple non overlapping cosmogony files

[antoine-de]

To generate a cosmogony of the world, you currently need a lot of ram since the whole planet.osm.pbf needs to be loaded.

We could read split non overlapping osm files, without a shared parent (eg. split by continent or country) and merge the generated cosmogony.

Since there is no contract on the indexes (id and parent) being consecutive, it seems a dumb offset by files, so each files will have ids in a separate range, will do the job.

I have a first shot at this, and need to test it on real cases but do you have any thoughts on this (I haven't worked on cosmogony for a while, maybe I'm completely off the mark)?

ping @prhod @amatissart

[prhod]

Thanks 🙏
I can try your POC on cosmogony.world (there is only europe for the moment). Only if live testing doesn't bother you :)