tusharmath/Multi-threaded-downloader

Support chunked downloads (Failure Error: File size could not be determined)

Opened this issue · 8 comments

stukennedy commented

If I run the example:-

mtd --url http://www.sample-videos.com/video/mp4/720/big_buck_bunny_720p_1mb.mp4

I get

Starting...
Failure Error: File size could not be determined
    at MTDError (/usr/local/lib/node_modules/mt-downloader/.dist/Error.js:10:1)
    at MapObservable.size$.map.totalBytes (/usr/local/lib/node_modules/mt-downloader/.dist/Utils.js:135:38)
    at MapObservable.selector (/usr/local/lib/node_modules/mt-downloader/node_modules/rx/dist/rx.js:5386:51)
    at MapObservable.selector (/usr/local/lib/node_modules/mt-downloader/node_modules/rx/dist/rx.js:5386:67)
    at /usr/local/lib/node_modules/mt-downloader/node_modules/rx/dist/rx.js:5386:67
    at tryCatcher (/usr/local/lib/node_modules/mt-downloader/node_modules/rx/dist/rx.js:63:31)
    at InnerObserver.next (/usr/local/lib/node_modules/mt-downloader/node_modules/rx/dist/rx.js:5407:43)
    at InnerObserver.Rx.internals.AbstractObserver.AbstractObserver.onNext (/usr/local/lib/node_modules/mt-downloader/node_modules/rx/dist/rx.js:1762:31)
    at InnerObserver.tryCatcher (/usr/local/lib/node_modules/mt-downloader/node_modules/rx/dist/rx.js:63:31)
    at AutoDetachObserverPrototype.next (/usr/local/lib/node_modules/mt-downloader/node_modules/rx/dist/rx.js:5883:51)

I tried with another file and got the same error too (this is with the latest version)

tusharmath commented

What is the version?

stukennedy commented

v2.2.1

tusharmath commented

The response is chunked. Which is not supported as of now.

stukennedy commented

but this is the example Url given in the help file ... this worked on the previous version of mtd ... how easy would chunked responses be to support?

tusharmath commented

@continuata Its easily doable. We need to make sure only one thread is created and then make a range request with the header bytes=1000- (without an end).

stukennedy commented

oh cool ... is that in the code where we get the file size from the content and then destroy the connection? Will take a peek tonight

LeoMartinDev commented

Any update on that ?

cyqsimon commented

Any updates on this? This is a pretty big make-or-break feature for my use case 😞