storj-archived/core

Allow farmer to set maximum total RAM usage by Storj Share (CLI and GUI)

Closed this issue · 1 comments

MeijeSibbel commented

Package Versions

Replace the values below using the output from npm list storj. Use npm list -g storj if installed globally.

storj-bridge@0.6.5 /home/gordon/Code/storj-bridge
└── storj@0.6.5  -> /home/gordon/Code/storj-core

Replace the values below using the output from node --version.

v4.2.3

Expected Behavior

Please describe the program's expected behavior. Include an example of your
usage code in the back ticks below if applicable.

Allow farmers to set MAX RAM usage at startup or as a configuration parameter

Actual Behavior

Please describe the program's actual behavior. Please include any stack traces
or log output in the back ticks below.

Ok so one might ask why one would want this, let me explain. 

We have been stress testing the network over the last few days which causes a increase of traffic to the nodes and a subsequent increase in RAM usage per node. Most people don't take into account that a Storj Share node can consume +-4GB or more during peak times. What we are starting to see more and more during stress tests is that farmers will just turn their nodes OFF because the excessive RAM usage is causing their system to either become slow and unresponsive or just crash. This is a problem because we need farmers to be reliable, more so during times of abundant demand for the Storage. As demand will increase farmers with for example 4x cores and 4GB of RAM will run out of RAM almost immediately. 

Now there are a few ways to solve this in my opinion.

1. When adding a node, Storj checks how much RAM it needs (and how much is already used by other nodes) and if that amount of RAM is available. If it is available (minus 20-30% for the OS) it will allow the user to add another node.

2. Farmers can set the max RAM usage they want the node to consume at node setup (or a parameter in the config for the CLI).

3. Reduce the RAM usage to always fall 30-40% of total capacity (and make it adjustable), this would probably mean ignoring bigger shards and download/upload less shards at a time.

Steps to Reproduce

Please include the steps the reproduce the issue, numbered below. Include as
much detail as possible.

  1. ...
  2. ...
  3. ...