gosmparse uses a callback driven API, which is stable (Documentation).
It has been designed with performance and maximum usage convenience in mind; on an Intel Core i7-6820HQ with NVMe flash it is able to process ~75 MB/s, so a planet file can be processed in under 10 minutes. If you find possible speed-ups or other improvements, let me know.
- fast
- tested with different files from different sources/generators
- more than 85% test coverage and benchmarks for all hot spots
- one dependency only: protobuf package (a few more are used by tests and are included in the module)
- can read from any io.Reader (e.g. for parsing during download)
- supports history files
- Does not build geometries
- No element cache
go get -u github.com/thomersch/gosmparse
// Implement the gosmparser.OSMReader interface here.
// Streaming data will call those functions.
type dataHandler struct{}
func (d *dataHandler) ReadNode(n gosmparse.Node) {}
func (d *dataHandler) ReadWay(w gosmparse.Way) {}
func (d *dataHandler) ReadRelation(r gosmparse.Relation) {}
func ExampleNewDecoder() {
r, err := os.Open("filename.pbf")
if err != nil {
panic(err)
}
dec := gosmparse.NewDecoder(r)
// Parse will block until it is done or an error occurs.
err = dec.Parse(&dataHandler{})
if err != nil {
panic(err)
}
}It is possible to parse during download, so you don't have to wait for a download to finish to be able to start the parsing/processing. You can simply use the standard Go net/http package and pass resp.Body to the decoder.
resp, err := http.Get("http://download.geofabrik.de/europe/germany/bremen-latest.osm.pbf")
if err != nil {
panic(err)
}
defer resp.Body.Close()
dec := gosmparse.NewDecoder(resp.Body)
err = dec.Parse(&dataHandler{})
if err != nil {
panic(err)
}If you found a case, where gosmparse broke, please report it and provide the file that caused the failure.