felixrieseberg/slack-archive

Download files in parallel?

cheapsteak opened this issue · 1 comments

Sometimes there's a large file or a file on a slow server, clogs the downloads up quite a bit

Thoughts on going through all the channel data to create a queue, and going through the queue with some parallelism? (e.g. with p-limit)

I'm mostly worried about rate limits, which I'm already hitting quite a bit - but it's worth a try?