WEHI-ResearchComputing/FileSenderCli

Configurable concurrency limits

Closed this issue · 1 comments

The async rewrite (#13) was designed to increase performance at some cost to memory usage. We could allow users to reduce the memory usage by implementing a semaphore that requires the semaphore to be open before the next chunk of data is read from any file being uploaded. Thus the user will be able to limit the amount of RAM used to n * chunk_size, where both n (the semaphore limit) and chunk_size are configurable.

Closed by #19.