ikatson/rqbit

Too many open files error

Closed this issue · 6 comments

When I try to download a torrent with a lot files, I get this error.
image

Here is the torrent link: magnet:?xt=urn:btih:dc5cf0f1dc97fc19afb959210c022d53853868c1&dn=Top 100-ish Sci-Fi Books - Various&tr=udp://tracker.ccc.de:80/announce&tr=udp://explodie.org:6969/announce&tr=udp://tracker.opentrackr.org:1337/announce&tr=udp://tracker.tiny-vps.com:6969/announce&tr=udp://tracker.vanitycore.co:6969/announce&tr=http://tracker.baravik.org:6970/announce&tr=http://tracker2.wasabii.com.tw:6969/announce&tr=http://tracker.publicbt.com/announce&tr=udp://tracker.publicbt.com:80/announce&tr=http://tracker.istole.it/announce&tr=udp://tracker.istole.it:80/announce&tr=udp://tracker.leechers-paradise.org:6969&tr=udp://tracker.coppersurfer.tk:6969&tr=udp://tracker.desu.sh:6969.

Wow that's a lot of files in a torrent! Has 4185 files.
The easiest workaround is to run with ulimit e.g.

(ulimit -n unlimited && rqbit download -o /tmp/scratch 'magnet:?xt=urn:btih:dc5cf0f1dc97fc19afb959210c022d53853868c1')

Don't know how to do this on Windows though.

I see a few things we can implement inside rqbit:

  1. Increase rlimits in rqbit itself, so that you don't need to do it outside.
  2. Open/close files on every IO. This will probably slow things down.
  3. Mmap files instead of open/close. This might work, but not sure if it will.

Will keep the issue open for now, maybe will get to it someday

#65 and #64 should make it easier to work with such torrents

@ikatson could you make a new release too?

@ikatson could you make a new release too?

Done

@IgnisDa let me know if it works on your system. Seemed to have worked on my osx

Yep it works on my system too. Thanks for the prompt turnover!