gboeing/osmnx

Download multiple OSM way fields using "infrastructure" parameter in graph_from_* (e.g. "cycleway", "surface", "bicycle")?

Closed this issue · 4 comments

cgthigpen commented

Question description

I would like to be able to download multiple OSM way fields, including for example, "cycleway", "bicycle", "surface", etc. (a) Can this be done in a single ox.graph_from_* call in the infrastructure parameter? (b) If not, and you must make multiple ox.graph_from_* calls with different infrastructure parameters, are there methods to join/merge the multiple resulting graphs (i.e. by osm_id) to a single graph with all of the relevant infrastructure fields included?

Code that illustrates the question

# This attempt to return both the highway and the cycleway fields doesn't work. 
# Perhaps I'm using the wrong syntax? I've tried this in a couple different ways 
# and haven't had any success.
gilbert_multiple_fields = ox.graph_from_place(query = 'Gilbert, Arizona, USA', 
                                       network_type='bike',
                                       infrastructure = 'way["highway", "cycleway"]')

# Returns the graph (street network) contained within the polygon of Tempe and 
# neighboring cities' boundaries and the highway field
gilbert_highway = ox.graph_from_place(query = 'Gilbert, Arizona, USA', 
                                       network_type='bike',
                                       infrastructure = 'way["highway"]')

# Returns the graph (street network) contained within the polygon of Tempe and 
# neighboring cities' boundaries and the cycleway field
gilbert_cycleway = ox.graph_from_place(query = 'Gilbert, Arizona, USA', 
                                       network_type='bike',
                                       infrastructure = 'way["cycleway"]')

# How best to merge/join?
gboeing commented

@cgthigpen This is a useful question to unpack as others might also have it. OSMnx's network_type paradigm operates by filtering things out. That is, if you request network_type='bike', OSMnx returns all the ways with a highway tag, filtering out all the ways that you cannot bike on:

osmnx/osmnx/core.py

Line 555 in 3befcf6

filters['bike'] = ('["area"!~"yes"]["highway"!~"footway|corridor|motor|proposed|construction|abandoned|platform|raceway"]'

But remember that highways and cycleways aren't different infrastructure types on OSM. All paths, streets, cycleways, etc are types of highways: https://wiki.openstreetmap.org/wiki/Highways

The latest release of OSMnx includes the ability to use custom query filters, which was added in this PR: #139 For example, see the custom filter documentation for graph_from_place: https://osmnx.readthedocs.io/en/stable/osmnx.html#osmnx.core.graph_from_place Specifically, what you pass as infrastructure and custom_filter will get inserted into an Overpass query here:

osmnx/osmnx/core.py

Line 670 in 3befcf6

query_template = '[out:json][timeout:{timeout}]{maxsize};({infrastructure}{filters}({south:.6f},{west:.6f},{north:.6f},{east:.6f});>;);out;'

So you can customize it however you like. That said, part of the value-add of OSMnx's network_types is that you can avoid the pain of working with the Overpass Query Language... but for complicated tasks or mixed infrastructure types, the custom_filter argument offers the most power. Language resources:

Now, to get cycleways with OSMnx...

Something like this should get you everything where highway=cycleway:

G = ox.graph_from_place(query = 'Gilbert, Arizona, USA', 
                        infrastructure = 'way["highway"]',
                        custom_filter='["highway"~"cycleway"]')

Whereas something like this gets you everything with a cycleway key, regardless of its value:

G = ox.graph_from_place(query = 'Gilbert, Arizona, USA', 
                        infrastructure = 'way["cycleway"]')

And this gets you everything you "can" bike on, even if it's not dedicated bike infrastructure:

G = ox.graph_from_place(query = 'Gilbert, Arizona, USA', network_type='bike')

Finally, if you want a graph comprising everything with either a cycleway key or highway=cycleway:

import osmnx as ox
useful_tags = ox.settings.useful_tags_path + ['cycleway']
ox.config(use_cache=True, log_console=True, useful_tags_path=useful_tags)
G = ox.graph_from_place(query = 'Gilbert, Arizona, USA', network_type='bike', simplify=False)
non_cycleways = [(u, v, k) for u, v, k, d in G.edges(keys=True, data=True) if not ('cycleway' in d or d['highway']=='cycleway')]
G.remove_edges_from(non_cycleways)
G = ox.remove_isolated_nodes(G)
G = ox.simplify_graph(G)

That is, we permissively downloaded more than we needed using the query. Then we found all the non-cycleways and removed them from the graph. Then we remove any nodes left isolated by that action, before finally simplifying the graph's topology. Throwing useful_tags into the config makes sure OSMnx retains the values of that tag as graph edge attributes.

To summarize, the OSMnx paradigm is to query liberally then filter things out.

  • 👍9
cgthigpen commented

Thank you for the detailed and clear explanation, @gboeing.

I ended up just using the useful_tags = ox.settings.useful_tags_path + ['cycleway'] functionality you pointed out, which served my needs of downloading additional OSM fields for use in my analysis.

  • ❤️1
gboeing commented

In case anyone else comes along here looking for information, please note that the old infrastructure parameter was deprecated and replaced by the custom_filter parameter and that the old useful_tags_path setting was replaced by the useful_tags_way setting in recent releases.

  • 👍3
CubicrootXYZ commented

My first time with OSM Data, I am trying to compare bike infrastructure to car infrastructure. The example from above does not nearly provide all bike lanes in my hometown. It reports a total of 170 km edge length.

I added a third filter on the bicycle=designated tag, that gave me more realistic results, a total edge length of 455 km. But still not every bike infrastructure. I can see a lot more on https://www.opencyclemap.org/. A quick look in the OSM source data told me that many streets are not tagged for cycling altough they carry official bike routes. Yeah, kind of difficult to decide here what is bike infrastructure and what is just a normal street/forest path with some signs for bikers.

My quick and dirty working example:

import osmnx as ox
useful_tags = ox.settings.useful_tags_way + ['cycleway'] + ['bicycle']
ox.utils.config(use_cache=True, log_console=True, useful_tags_way=useful_tags)
G = ox.graph_from_place(query = 'Berlin, Deutschland', network_type='bike', simplify=False,  retain_all=True)

non_cyc = []
for u,v,k,d in G.edges(keys=True, data=True):
    bi = False
    if "bicycle" in d: #kind of uggly but not every edge has the bicycle tag
        if d['bicycle']=='designated':
            bi = True
            
    if d['highway']=='cycleway':
        pass
    elif 'cycleway' in d:
        pass
    elif bi:
        pass
    else:
        non_cyc.append((u,v,k))

G.remove_edges_from(non_cyc)
G = ox.utils_graph.remove_isolated_nodes(G)
G = ox.simplify_graph(G)
stats = ox.stats.basic_stats(G)
print(stats["edge_length_total"])
fig, ax = ox.plot_graph(G)
```

Yeah, just my 5 Cents for somebody with the same "problem" watching this issue here.
  • 👍2