Prune messages that are not needed
piotrm50 opened this issue · 2 comments
Currently all the messages are stored in memory for all simulated nodes, which implies huge memory consumption. This is problematic when the experiment needs to run for some time as the memory is filled up and the system kills the simulator.
In order to avoid this problem, it might be necessary to remove messages which are solid and not part of the tip set.
See if this is possible and under what conditions, and if it's possible then implement it.
Pruning messages by removing parent's of solid messages, or parents of fully processed messages was always causing errors.
We also tried using storage to reduce memory requirements, but that slowed down the simulation too much.
The cause of high memory consumption is the approval weight's weightSlice which is part of the metadata.
On branch pruning there is code to remove confirmed messages and their metadata which shows slight improvement in memory consumption which might be beneficial for memory usage. However that is not stable for lower TPS and for this reason I'm not creating a PR with those changes.