Kemsekov/GraphSharp

Lazy load data

Opened this issue · 1 comments

emdadgar2 commented

Hi,
I have a +300M nodes with +1G edges, loading all data from database to process with your great project,
needs too resource and take long time

Is there any way to Lazy load data from data source as needed?
Thanks

Kemsekov commented

This is interesting question.

I honestly don't even know what it means to implement such lazy loading in the first place. 😅

You mean to create some graph implementation that known where to take nodes and edges (aka knows how to load them from DB to memory), but does it only once particular node or edge is actually accessed, meanwhile keeping illusion of all data being loaded at once?

There is currently no such implementation.

But i believe this is pretty simple to do.

You may look into interfaces INodeSource and IEdgeSource, implement them with such lazy techniques and then use this sources with graph G.SetSources.

Maybe I will work on this when I will have a bit of free time 😅

  • 👍1