[FIXED] YouTube is sometimes unreachable because of A/B testing of forced log in
wukko opened this issue · 48 comments
innertube returns "Sign in to confirm you’re not a bot. This helps protect our community. Learn more" instead of video info or regular errors.
cobalt may sometimes not be able to get the video, especially on larger instances (such as main one). if that happens, it'll show you an appropriate error that links to this issue:
related issue threads in repos of other projects:
FIXED:
@dumbmoron implemented support for OAuth2! main instance (cobalt.tools) is back to being able to download youtube videos (so far).
if you host a cobalt instance, you can generate your own token by running npm run token:youtube in local cobalt repo folder. please don't use your personal google account. it might get banned, restricted, or rate limited.
My YouTube alt account got blocked when I used the cookie.
When I play a video, it always shows "Video unavailable. This content isn’t available." Changing the IP doesn't help.
My YouTube alt account got blocked when I used the cookie.
this is exactly what i mean! using user cookies WILL cause issues. this is why using dummy accounts is not a good option, they will quickly run out.
support for youtube cookies was added to cobalt in f6632e2, but please don't use your main account cookies
main instance seems to be fine for now, but it's unknown how long it'll last. we shall see and find out
the dummy account cookie already got nuked omfg
update: you need to refresh the cookie every few minutes. it's best to not use it at all and instead use a different authentication method
From what I remember,
- Google needs a phone number for each account
- You can't make a YouTube account without a Google account
- A single phone number can only be used six times to verify Google accounts
Dummy/throwaway accounts are a stupid idea unless you happen to have unlimited cash to burn on burner phone numbers.
(Just letting you know in case you don't already)
in case you don't
i do know like no other lol
YouTube really hates its users and I personally saw this coming from miles away. They just bricked every YouTube downloader known to mankind with one simple change and are now handing out IP and account bans.* I guess this might be the end of YouTube download utilities until they either get their act together and stop (which they won't), or a workaround is discovered.
This also cripples legitimate uses for downloader utilities, including data preservation.
*Well, somewhat. They don't appear to be 100% bricked yet, but they are getting there rather fast. I hope that this causes significant controversy (much like the adblock crisis) and YouTube loses the battle. But it looks like we lost right out of the gates.
legitimate uses for downloader utilities, including data preservation
that's what cobalt is for, along with content creation and teaching
that's what cobalt is for, along with content creation and teaching
And yet, here we are. YouTube is going through the effort and potentially spending thousands of dollars to brick downloader utilities even though most uses are protected under fair use. Lets hope this generates controversy and that we win the war against YouTube harming their own users.
My Cobalt fork (the only change I made is using YTSTUDIO_ANDROID instead of WEB) still works. The number of users using Cobalt might be causing YouTube to be upset, but I don't know. It's worth a try, I guess.
Would using a Premium-enabled YouTube dummy account have the same limitations?
The only way I see a public-facing production instance continuing to work at scale for YouTube video downloading is if Cobalt asks the user to sign in with a Google account prior to downloading. It can then use that account to download the content. Definitely not ideal though, especially for the API..
Thanks YouTube, you guys really know what you're doing.
Wukko, will you be able to get this fixed? 🥺
I'm actually thankful this broke Discord music bots, as it'll surely bring a lot of mainstream attention to this issue. Fingers crossed.
The only way I see a public-facing production instance continuing to work at scale for > YouTube video downloading is if Cobalt asks the user to sign in with a Google > account prior to downloading. It can then use that account to download the content. > Definitely not ideal though, especially for the API..
Doing this is very risky, as YouTube is a part of Google. They can easily revoke API access or in the worst case scenario; outright terminate your account. Bans can also apply to users signing in to the API to use cobalt, too. The only reason why YouTube is doing this is not to prevent scraping; but to protect their bottom line. They tried going after adblockers and failed. We can only hope history repeats itself and this generates enough controversy that YouTube gives up trying. This will also fuck over Discord bots, so that might end up being the spark that starts the riot along with breaking YouTube downloaders.Can confirm this is fucking over Discord bots as well. I own a music bot on 400k+ servers and YouTube is currently completely broken right now.
Same, I own some YouTube downloader Sites, 4-5 million monthly users and I am losing my mind rn!!
The Maki Bot is working Idk how but it is
The Maki Bot is working Idk how but it is
Probably just haven't been banned (yet). Luck of the draw?
We should also get Louis Rossman on the line. He could bring additional attention to how fucked up YouTube's practices are. Combine that with the riot that's starting with Discord music bots and we got a firestorm that will most certainly secure our victory.
We should also get Louis Rossman on the line. He will likely bring additional attention to how fucked up YouTube's practices are.
That is a great idea. Someone email him the situation
9 hours ago? Wow, that's recent! I wonder how long this has been an issue before that...
And ihatespawn on #555 just mentioned this issue 15 minutes ago...
We should also get Louis Rossman on the line
pretty sure he funded an alternative youtube frontend app for android, it affects him directly
OH FOR CRYING OUT LOUD AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAa
I hope this doesn't make it out of A/B lmfao
Surprisingly, the custom self-hosted music bot that is used in a server I am active in works just fine at the moment, as well as YT-DLP on my machine, it seems neither I or the bot server have been shoved into that A/B testing shit.
9 hours ago? Wow, that's recent! I wonder how long this has been an issue before that...
(Replying to Mazedotexe) YT-DLP has the oldest issue, at 3 days ago. I however only noticed this issue since yesterday, so maybe YouTube made the IP limit stricter or something sometime in the middle.
Oh and here's every other issue I've found in case you're interested: Piped (1 day), YT-DLP (3 days), Newpipe (17 hours), Invidious (1 day). This issue and YT-DLP's seems to be the most active out of the bunch.
I propose a solution:
A client that intercepts incoming data from youtube's servers to the device and uses the client's own encoder to make its own video file
idk if this is possible though
edit: this proposal is a client that utilizes the device's own account to access the youtube servers, so not a web based thing
intercepts incoming data from youtube's servers to the device and uses the client's own encoder to make its own video file
Doesn't YouTube send video files though (in the googlevideo.com domain IIRC)? So I'm not sure what you can intercept and encode.
looking at the stats for nerds, you can see all the different codec and rate information that the video has, maybe a client that instead of decoding and playing the youtube video data received, it turns it into a file
looking at the stats for nerds, you can see all the different codec and rate information that the video has
Just like ffprobe...? You can easily get that data from a file too.
the youtube video data received
I'm pretty sure the YouTube data received is in the form of a URL to a video. BRB while I grab an example link.
the YouTube data received is in the form of a URL to a video
well a browser doesn't play a video only using a URL, the URL is an address to a point of data access (you guys know this which makes me look stupid) which the browser accesses to get the video data needed just like opening a regular file, and my proposal is taking said video data, not codec information or URL
my proposal is taking said video data, not codec information or URL
Interesting idea, but how can you access the video data if YouTube won't return it? I don't know how the block works but I'd assume that YouTube simply doesn't return the video, or a URL to the video, or anything like that. Perhaps it's possible though.
OH you mean grabbing the result from the browser instead of requesting the data itself? That sounds like a really smart idea actually.
have yall heard of cors
much like recording the screen playing youtube video, but more direct
Actually I just remembered, someone over in the YT-DLP issue says this happens for the YouTube website too, so I don't think that would work. (Edit: I think it was saying that it only happened on the YouTube website after you had been IP banned due to using something else... not sure since I can't find the comment again (perhaps it wasn't on the yt-dlp issue))
have yall heard of cors
Nope, BRB
the only weakness of my idea is: trackers. Someone at youtube who saw this issue could implement a tracker, that tracks client activity, stopping data sending when detecting my proposal client thing. and since youtube video data aren't sent in one big packet like downloads, trackers are a BIG no no
Someone at youtube who saw this issue
I don't think that's likely
stopping data sending when detecting my proposal client thing
How might they be able to detect something running on your local device yoinking packets from your browser or something? I don't think it would be possible to detect it, although I might be wrong.
How might they be able to detect something running on your local device
i thought maybe something that is a part of the browser's code, but i realised that would be damn stupid
i thought maybe something that is a part of the browser's code, but i realised that would be damn stupid
I'm not sure if you mean code in Chromium that detects this behaviour made by YouTube, or if you mean an extension that would intercept packets and do what your idea proposes.
If the first: absurd idea and that would effectively require banning Firefox users LOL
If the second: would be kind of easy to block on Chromium based browsers, since they report every extension to the website, however as long as the extension doesn't do something stupid like modifying the DOM, it might work great on Firefox.
Or just something like Wireshark running locally...
remember when you install an ad blocker and get a notification about how it can access site data? i thought my thing would be similar
BTW is this the right place to discuss that? A browser extension or something is so completely different from this project...
@dumbmoron implemented support for OAuth2! main instance (cobalt.tools) is back to being able to download youtube videos (so far).
if you host a cobalt instance, you can generate your own token by running npm run token:youtube in local cobalt repo folder. please don't use your personal google account. it might get banned, restricted, or rate limited.
i will close this issue if the sign in problem doesn't come back within next 2 hours :D
@dumbmoron implemented support for OAuth2! main instance (cobalt.tools) is back to being able to download youtube videos (so far).
if you host a cobalt instance, you can generate your own token by running
npm run token:youtubein local cobalt repo folder. please don't use your personal google account. it might get banned, restricted, or rate limited.i will close this issue if the sign in problem doesn't come back within next 2 hours :D
SUCCESS. We have cracked YouTube's downloader block. With any effort we will be able to crack it without the use of OAUTH or cookies. I wonder, is there a CLI version of Cobalt available much like yt-dlp? And we should obfuscate and encrypt our keys; this is in case Google is actively looking though the repo to find keys to revoke. This is likely as they are money hungry and greedy and will do ANYTHING for their investors and to protect their bottom line.
yep, seems to be all good
FIXED: @dumbmoron implemented support for OAuth2! main instance (cobalt.tools) is back to being able to download youtube videos (so far.)
I'm on Docker, which container should I run this on?