Using OSM GPX planet as a data-source

Question

Using OSM GPX planet as a data-source

Opened this issue 10 years ago · 2 comments

laurentg commented 10 years ago

Announcing the GPX Planet. 2.6 Trillion GPX points.

It could be nice to be able to use this (or part of it) as a GPS source.

Answer 1 · 2015-04-22T11:03:52.000Z

Let's ingest it! Just need to get the traces into a time sorted format with and we're good! We just need:

trace_id,time,lat,lon

2.6 trillion points will take a bit of crunching but we're currently getting about 15k points/sec/thread (and lots of room still for optimization) so that's 2000 days on a single thread. Scaling up is linear by throwing more CPU power at this.

Answer 2 · 2015-04-22T12:06:04.000Z

There exists regional extracts that can simplify things where only a small region is needed (GPX files in each extracts are stored by lat/lon bins). Also lots of those files are walk / bike / boat / planes etc... traces, a pre-filtering by mode would help reduce the amount of data to process (altough some heuristics are needed as the traces do not have mode information attached to them).