Reduce memory usage.
Closed this issue · 0 comments
Janekdererste commented
Currently each MPI-Process loads the network and the population file. If we have a lot of slots on a single machine, we might run out of RAM, also reading from disk becomes a bottle neck then. (Even on HLRN we run out of memory if we assign the full 192 slots available per machine)
First idea:
One process does the loading.
- Load Network
- Load Population
- Do partition stuff and id mapping
- Broadcast network and population partitions, as well as id-mappings to other processes.
Refinement:
Split up loading of Network and Population. Each of the loading processes can also do the id mapping and broadcast this to the other nodes.