fent/node-ytdl-core

Using audioonly filter slows down download tremendously

nikbrandt opened this issue · 12 comments

When downloading a video and converting to mp3 with ffmpeg, using filter: 'audioonly' in the options slows down speeds by magnitudes; ~72 seconds with filter, ~8 with no options, also ~8 with quality: 'highestaudio' (seems weird considering highestaudio requests just audio, like the filter).

This is tested using a 4:13 video on my gigabit ethernet.

code used:

const ytdl = require('ytdl-core');
const ffmpeg = require('fluent-ffmpeg');

let id = 'sDLsSQf3Hc0';
let start = Date.now();
let stream = ytdl(id, { quality: 'highestaudio'/*, filter: 'audioonly' */}); // mess around with quality, filter, and nothing at all being used

ffmpeg(stream)
	.on('end', () => {console.log('done, thanks - ' + (Date.now() - start) / 1000 + 's')})
	.audioBitrate(128)
	.save(`${__dirname}/${id}-${(Math.random() * 100000).toFixed(0)}.mp3`); // very efficient naming system here

This doesn't really affect me because I can just use quality: 'highestaudio', but it does seem like a weird effect.

i tried running your code it keeps giving me this issue
Could not extract signature deciphering actions

fent commented

This is a bit strange, some formats download way faster than others. Even when they're bigger in filesize. From the following list, the top 3 downloaded super fast. Then, it significantly slows down after that, but not slow enough that it would be slower than it plays.

itag container resolution video enc     audio bitrate audio enc
22   mp4       720p       H.264         192           aac
43   webm      360p       VP8           128           vorbis
18   mp4       360p       H.264         96            aac
36   3gp       240p       MPEG-4 Visual 32            aac
17   3gp       144p       MPEG-4 Visual 24            aac
299  mp4       1080p      H.264
303  webm      1080p HFR  VP9
137  mp4       1080p      H.264
248  webm      1080p      VP9
298  mp4       720p       H.264
302  webm      720p HFR   VP9
136  mp4       720p       H.264
247  webm      720p       VP9
135  mp4       480p       H.264
244  webm      480p       VP9
134  mp4       360p       H.264
243  webm      360p       VP9
133  mp4       240p       H.264
242  webm      240p       VP9
160  mp4       144p       H.264
278  webm      144p 15fps VP9
251  webm                               160           opus
140  m4a                                128           aac
171  webm                               128           vorbis
250  webm                               64            opus
249  webm                               48            opus

And I'm not sure there's much we can do about it. Since it still downloads faster than one can play a video, even on 2x, this rate limit could be intended. Although I'm not sure why it only happens for some formats.

There is the ratebypass param. There might be more params like it.

fent commented

Also, thank you for this code snippet. I'd say it's pretty common, and so I'll add it to the examples folder.

Alright, cool.
How would you find out about more parameters? Shuffle through all the info in .getInfo for the video? I'm willing to do that, lol.

@bolahanna44, I'm pretty sure you just need to update ytdl. (npm update ytdl-core or npm i ytdl-core@latest)

This might be obvious from the earlier testing you did, but looking at the formats;
those first three formats had ratebypass in the URL in two places.
They had it after the itag part of the link (... ?itag=18&ratebypass=yes ...) and towards the end of the deadly URL, before the requiressl part (... %2Cratebypass%2Crequiressl ...).

When I added ratebypass to those parts of a URL that did not have 'em, I was met with a lovely 403 Forbidden page.
Wooh

Probably is intended that only those formats allow ratebypass, it's really bizarre to wonder why though.

fent commented

Did you also add it to sparams?

Sorry for such a delayed reply

adding it to sparams => 403
not adding it to sparams => since ratebypass=yes is already in the query we're not changing anything

Yes, I also added it to sparams - ^ what they said
changing, say, a working url

https://r1---sn-nvopjoxu-nune.googlevideo.com/videoplayback?lmt=1517944841813447&key=yt6&itag=251&keepalive=yes&fvip=1&c=WEB&requiressl=yes&ip=184.23.255.99&mv=m&mt=1528232751&ms=au%2Crdu&mn=sn-nvopjoxu-nune%2Csn-n4v7sn7z&mm=31%2C29&pl=18&mime=audio%2Fwebm&id=o-AK1via8p26fKTNcSBEio9T87CCTx9TZBDGjVHhGbLA3j&ipbits=0&expire=1528254485&dur=253.101&initcwndbps=750000&ei=tfsWW9b_EPXI-APw0ZeQCg&clen=4486749&source=youtube&gir=yes&sparams=clen%2Cdur%2Cei%2Cgir%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Ckeepalive%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Crequiressl%2Csource%2Cexpire&signature=4A31B3796440A7D517654DCDE8CAE00F682CAC51.367CB30186ADD857BBC9DE8B36C3BAB56E75C3F6

screen shot 2018-06-05 at 2 34 54 pm

to include ratebypass as both a param and a sparam

https://r1---sn-nvopjoxu-nune.googlevideo.com/videoplayback?lmt=1517944841813447&key=yt6&itag=251&keepalive=yes&fvip=1&c=WEB&ratebypass=yes&requiressl=yes&ip=184.23.255.99&mv=m&mt=1528232751&ms=au%2Crdu&mn=sn-nvopjoxu-nune%2Csn-n4v7sn7z&mm=31%2C29&pl=18&mime=audio%2Fwebm&id=o-AK1via8p26fKTNcSBEio9T87CCTx9TZBDGjVHhGbLA3j&ipbits=0&expire=1528254485&dur=253.101&initcwndbps=750000&ei=tfsWW9b_EPXI-APw0ZeQCg&clen=4486749&source=youtube&gir=yes&sparams=clen%2Cdur%2Cei%2Cgir%2Cid%2Cinitcwndbps%2Cip%2Cipbits%2Citag%2Ckeepalive%2Clmt%2Cmime%2Cmm%2Cmn%2Cms%2Cmv%2Cpl%2Cratebypass%2Crequiressl%2Csource%2Cexpire&signature=4A31B3796440A7D517654DCDE8CAE00F682CAC51.367CB30186ADD857BBC9DE8B36C3BAB56E75C3F6

still leads you to the 403
screen shot 2018-06-05 at 2 36 05 pm

I'm not entirely sure there's anything we can do about this; it seems this is in place for a reason ¯\(ツ)

fent commented

Yeah. I'll close this and add a note to the readme.

Hi a question, does anyone tested it with multi requests?

Hi, I suppose you investigated this, but youtube-dl doesn't seem to suffer from rate limiting. I think you can copy its behaviour in ytdl-core.

I looked into it a little bit more, and it seems that adding a Range header, e.g.

fetch(u, {headers: {Range: 'bytes=0-10380331'}})

(here, using node-fetch) improves download speed dramatically.