pysal/libpysal

allow quick pass through the Graph constructor

Closed this issue · 1 comments

Graph constructor currently always ensures sorting.

ids = adjacency.index.get_level_values(0).unique().values
adjacency = adjacency.reindex(ids, level=0).reindex(ids, level=1)

While needed, it is relatively costly operation, esp. on larger graphs (I am now working on one with 120mil edges and it quite noticeable). When doing stuff like transformation, we know that the order does not change, only the weight. We should implement a fast path that skips sorting for adjacencies that are known to be sorted.

I think that this is also affecting read_parquet as it is surprisingly slow on large graph.