vbuell/torram

feature: download large torrent files into pieces

Opened this issue · 2 comments

ehsanonline commented

Hi. I have been looking for a linux(ubuntu) torrent cli client, or python torrent/magnet downloader that could download large files and split them WHILE IT IS DOWNLOADING, for example: a large file like file.iso with 10GB size, gets downloaded as ten 1GB files like file.iso.part1, file.iso.part2 (without download the whole file and then split it).
it is because my server has limited storage capacity. then i can join them together using cat in my PC.
Is it possible to add this feature to this library? or do you know any client/library that can do this?
thank you.

vbuell commented

I'm not aware of torrent clients which do what you want but I'm pretty sure they have to exist.

As to the strategy of downloading and joining - it depends on the way how would the client store the chunks... If that's going to be files 1GB size each as you described, then the solution would be quite trivial - just joining all downloaded files as you described. No any tool needed.

Another option is if the client support specifying a data range but still stores the data in 10GB files files. If your file system supports sparce files then this option would work for you too, because the actual disk space usage would be only 1GB even though the file length in listed as 10GB:

Server 1 (downloading 0GB-1GB). file.iso: [XXXX-------------------------------]
Server 2 (downloading 1GB-2GB). file.iso: [----XXXX---------------------------]
...
Server 10 (downloading 9GB-10GB). file.iso: [---------------------------------XX]

Having 10 files you coudl either:

  • cut the downloaded part and then join chunks together
  • run torram pointing to the directory having all parts

The one thing you shoudl keep in mind if you decide to go with torram - your range should be aligned to the boundaries of torrent data chunks. Or, you can just give the client ranges which overlap each others (e.g: 1-108MB, 92MB-208MB, etc).

  • 👍1
cben commented

You can try using any torrent client, with the destination being on a virtual filesystem that's actually backed by separate chunks.

  • 👍2