uclasystem/dorylus

Question about the loading of Friendster

TracyRixner opened this issue · 1 comments

TracyRixner commented

@ivanium @kevalvora @josehu07
Hi authors,
Good work!
I use the c5n.4xlarge instance. However, I find the command to prepare dataset

./prepare <PathToRawGraph> <Undirected? (0/1)> <NumVertices> <NumPartitions> <PathToRawFeatures> <DimFeatures> <PathToRawLabels> <LabelKinds>

There is an Out of Memory error.(42GB is not enough) To run friendster, do I need to use a large memory node to make the partition first? And then distribute those partition to each node, right? How large should the RAM be?

Thanks for the help!

ivanium commented

Hi Tracy,

Thank you for your interest in Dorylus.

Yes, you are right that partitioning large graphs can take a huge amount of memory. I didn't have concrete memory consumption numbers right now, but I would suggest to use AWS r-series instances, specifically I recommend at least 256GB memory for friendster. For example, r6i.8xlarge is a good choice.

Hope this helps, and good luck!