Default API key is disabled
trizen opened this issue ยท 41 comments
I recently got the following message from the YouTube API Services team:
Dear YouTube API Developer,
For your use of YouTube API Services, we have sent multiple reminders, which required you to submit a completed Audit Form with (i) fully-functional demo accounts, (ii) screenshots and (iii) documents in relation to your API Client. As a result we have reduced your quota access to YouTube Data API.
You have three (3) days to respond to this message with the requested information, failing which access to YouTube Data API would be affected.
Thanks,
YouTube API Services team
They reduced the quota limit from 25M to 12.5M queries per day:
Everyone is requested to create their own API key and client ID/SECRET values, otherwise youtube-viewer will no longer work.
Check the instructions in README.md for how to create your own API key.
Edit: the default API key of youtube-viewer is now disabled.
This would also be a good time to get in touch with distribution package maintainers. Distributions that have a nonprofit API account with Google may be able to include distribution-specific keys with their packaged versions.
I'm the Arch maintainer, and we're not yet set up for that, but I believe someone's working on it for unrelated reasons.
If a distribution wants to package youtube-viewer with an API key, I can modify the program to read the default values from /etc/youtube-viewer/api.json (if this file exists).
Then, if an user chooses to use their own API key, they can specify it in ~/.config/youtube-viewer/api.json.
Edit 1: as an alternative to using an YouTube API key, I created a fork of youtube-viewer, called straw-viewer, which uses the API of invidio.us to search for YouTube videos. The project is in its early stages of development and it may lack some features, but, for basic stuff, like searching for videos and playlists, it should work without many issues.
Edit 2: another alternative is pipe-viewer, which parses the YouTube website directly in a similar way like NewPipe does.
Then, if an user chooses to use their own API key, they can specify it in
~/.config/youtube-viewer/api.json.
How should that .json file look? I have one from kodi's youtube addon named api_keys.json. It looks like this and it contains the api keys I had made for myself a few days ago when google started all that bs
{
"keys": {
"developer": {},
"personal": {
"api_key": "xxx",
"client_id": "xxx",
"client_secret": "xxx"
}
}
}
Will youtube-viewer work if I just copy it over to the forementioned folder and rename it?
---edit
Nope. I just tried it and it did not work. It pops the exact same error I mentioned earluer today on #303
The format of ~/.config/youtube-viewer/api.json is:
{
"key": "API_KEY",
"client_id": "CLIENT_ID",
"client_secret": "CLIENT_SECRET"
}Still no go. This is how it looks now
{
"key": "xxx",
"client_id": "xxx",
"client_secret": "xxx"
}
And the error remains the same. I have to leave for work now, I will look it up when I get home.
Getting the api keys from google and inserting them properly on the addon was a real pita, that is why I copied the file over as I did for all my devices that run kodi.
it needs 5/10 mins to propogate to the gogle servers
If you're logged in using the previous credentials, try youtube-viewer --logout.
It's nothing much that I can do about it...
If a distribution wants to package
youtube-viewerwith an API key, I can modify the program to read the default values from/etc/youtube-viewer/api.json(if this file exists).Then, if an user chooses to use their own API key, they can specify it in
~/.config/youtube-viewer/api.json.By the way, mps-youtube is also affected: mps-youtube/mps-youtube#1063
Additionally, something somewhat funny: https://stackoverflow.com/a/60264537Edit: as an alternative to using an YouTube API key, I created a fork of
youtube-viewer, called straw-viewer, which uses the API of invidio.us to search for YouTube videos. The project is in its early stages of development and it may lack some features, but, for basic stuff, like searching for videos and playlists, it should work without many issues.
doesn't youtube-viewer already have invidious support, with the "--invidious!" command? Also the title itself is misleading: unless Google restricts people making API keys, we can just create our own API key for youtube-viewer/mpsyt and use that.
Do I also need to tell youtube-viewer about my youtube account, or is creating the api.json file sufficient?
@HawaiinPizza: doesn't youtube-viewer already have invidious support, with the "--invidious!" command?
Not really. The API of invidio.us is used only in extracting the streaming URLs for some videos.
@rien333: Do I also need to tell youtube-viewer about my youtube account, or is creating the api.json file sufficient?
Specifying the API key in api.json should be sufficient. The client ID/SECRET values are optional and they are used only for logging in.
EDIT: This is my fault. I was signed and forgot, but once I signed out I fixed the problem.
In light of having to use personal API key, I changed my API to the one I use for mpsyt, and for mpsyt it works. However, for youtube-viewer, it doesn't. I use the same key for mpsyt, I remove the client_id and client_secret thinking maybe they caused it, they didn't change anything. I made my own client_id and client_secret and put it in, and it didn't work.
The error message I'm getting is
** GET https://www.googleapis.com/youtube/v3/search?key=(KEY)&maxResults=20&part=snippet&prettyPrint=false&q=wow&type=video ==> 500 Internal Server Error (1s)
** GET https://www.googleapis.com/youtube/v3/search?key=(KEY)&maxResults=20&part=snippet&prettyPrint=false&q=wow&type=video ==> 500 Internal Server Error
** GET https://www.googleapis.com/youtube/v3/search?key=(KEY)&maxResults=20&part=snippet&prettyPrint=false&q=wow&type=video ==> 500 Internal Server Error
** GET https://www.googleapis.com/youtube/v3/search?key=(KEY)&maxResults=20&part=snippet&prettyPrint=false&q=wow&type=video ==> 500 Internal Server Error
** GET https://www.googleapis.com/youtube/v3/search?key=(KEY)&maxResults=20&part=snippet&prettyPrint=false&q=wow&type=video ==> 500 Internal Server Error
[500 Internal Server Error] Error occurred on URL: https://www.googleapis.com/youtube/v3/search?key=[...]&maxResults=20&part=snippet&prettyPrint=false&q=wow&type=video
Use of uninitialized value in numeric gt (>) at /usr/share/perl5/vendor_perl/WWW/YoutubeViewer/Utils.pm line 224.
I copied the url it mention [the (KEY) part was my API key] and it works.
I tried looking into Utils.pm, and at line 224, it's this code
sub has_entries {
my ($self, $result) = @_;
ref($result) eq 'HASH' and ($result->{results}{pageInfo}{totalResults} > 0);
}
I removed the and statment, but it didn't do anything, only removed the "Use of uninitialized value in numeric gt" error message
I am using YouTube Viewer 3.7.4 on Arch Linux, using the latest youtube-viewer package.
it needs 5/10 mins to propogate to the gogle servers
I logged in on my pc via ssh and tried it again 1+ hour after making the post above, but the error was the same.
If you're logged in using the previous credentials, try youtube-viewer --logout.
I do not use credentials on any app that plays content from youtube. I tried it though, but the result was the same.
Just to be sure, the api.json function is availabe to youtube viewer 3.7.4 only, right? Because that is what I have. If not, I will have to clone the git repo and try the git version, obviously because it is newer.
Bottom line, I am really fed up with google's bs, because I had similar issues while I was importing my keys to kodi's addon too. I may give straw-viewer a shot later on.
---edit
Just as I asked for the kodi addon, can someone inform me about the form of the keys? For me they are as follows
- api key = a string of 39 characters, containing numbers and letters, upper and lowercase
- cliend id = a string of 45 charactes as above, that starts with 12 characters which are numbers only, then a dash and then 32 characters as above
- client key = a string of 24 characters as above, that starts with 12 characters, then a dash and then 11 characters
Are yours similar?
Just to be sure, the api.json function is available to youtube viewer 3.7.4 only, right?
Yes. That is correct.
- The API key starts with
AIza...(39 characters long). - The client ID has the form:
xxx.apps.googleusercontent.com - The client SECRET is a string of 24 random characters.
The API key starts with AIza... (39 characters long).
Check!
The client ID has the form: xxx.apps.googleusercontent.com
Do I also need the ".apps.googleusercontent.com" part? The kodi addon automatically chops it off, so it was not in the file when it was copied.
The client SECRET is a string of 24 random characters.
Check!
Thank you :)
Try:
youtube-viewer --debug test
It should print an URL with the API key included. Check to see if it actually uses the correct key.
Optionally, copy-paste the URL into a browser to see if if that works and if there are any error messages.
I am an idiot :(
I had left an extra bracket ( "{" ) inside the file and it was messing things up. I fixed it now and it is working as it should. Sorry for all the hussle guys.
Thank you for pointing out the --debug switch. I was able to check the api key with it when youtube-viewer runs and found out that it used AIza..22Q which is youtube-viewer's default, instead of mine which is AIza...GR4.
so paste some keys...
Not sure whether that's gonna help.
I used yt-viewer w/ a very old gentoo key which worked for about a day (and maybe half a dozen searches), but 403's since yesterday evening CET.
I then tried the key arch uses for chromium. 403's as well (w/ and w/o ID and secret)
The little paranoid guy in the back of my head thinks they're shutting down API access to enforce using the browser which is, given of the awesome HW acceleration support in linux browsers, nice /sarcasm
yt-dl still works, so you can play a URL w/ eg. mpv.
Update: the key works again.
Either google is fiddling around or this is due to severly capped quotas in which case you'll need a private key.
@clort81
The keys are pesonal and linked to my google account, so I won't share them.
@luebking
I can't confirm that. Removing api.json makes the 403 error come up again. And the same happens on kodi's youtube addon.
And how did you get the keys from those distros? Debian had something similar for its chromium in /etc/chromium.d/apikeys. I tried them in kodi's addon, but they did not work.
Chromium PKGBUILD and the gentoo key was in some post, 5 years ago or so.
As mentioned, it meanwhile works again, so I guess this is a quota thing (and was exceeded last night)
Can't make it work because youtube-viewer keeps using the default API key. I have generated API keys and put them to ~/.config/youtube-viewer/api.json
~$ cat ~/.config/youtube-viewer/api.json
{ "key": "***ftQ", "client_id": "****", "client_secret": "****" }
Here is the debug output:
> youtube-viewer --debug test
** GET https://www.googleapis.com/youtube/v3/search?key=AIzaSyA2PQakgMYbX3U3imqplLMhpfM8mzEn22Q&maxResults=20&part=snippet&prettyPrint=false&q=youtube-viewer%20test&type=video ==> 403 Forbidden (1s)
[403 Forbidden] Error occured on URL: https://www.googleapis.com/youtube/v3/search?key=AIzaSyA2PQakgMYbX3U3imqplLMhpfM8mzEn22Q&maxResults=20&part=snippet&prettyPrint=false&q=youtube-viewer%20test&type=video
Notice it keeps using 22Q instead of my key ftQ. I've tried youtube-viewer --logout and launching youtube-viewer in a new terminal window. My distro is Elementary OS Juno (5.0)
Make sure to have the latest version of youtube-viewer installed (3.7.4).
I have my Youtube Data API key specified in .config/youtube-viewer/api.json, but I'm still getting a 403 errorr. youtube-viewer --debug test throws
** GET https://www.googleapis.com/youtube/v3/search?key=AIzaSyMyKeySRM&maxResults=20&part=snippet&prettyPrint=false&q=test&type=video ==> 403 Forbidden (1s) [403 Forbidden] Error occurred on URL: https://www.googleapis.com/youtube/v3/search?key=[...]&maxResults=20&part=snippet&prettyPrint=false&q=test&type=video Use of uninitialized value in numeric gt (>) at /usr/share/perl5/vendor_perl/WWW/YoutubeViewer/Utils.pm line 224.
When I copy that URL into Firefox, I get
"message": "Access Not Configured. YouTube Data API has not been used in project 171421846572 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/youtube.googleapis.com/overview?project=MyProjectName then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry."
The API is enabled and has been for a few months. Afaik it worked with mps-youtube or a similar youtube program when I made the key.
Try to create a new project with a new API key. Apparently, Google automatically disables the API keys that have not been used for 90 consecutive days.
So all that means, if i want to use it i will be tracked by google again ?
@asteriosos: Sadly, yes. As an alternative, I created straw-viewer, which uses the API of invidio.us, and thus it does not require an YouTube API key.
I have the following issue with my keys and I would like to ask if anyone else has it too.
I have made my personal keys and I use them on youtube-viewer and on kodi's youtube addon. That is... 4 devices in total, youtube-viewer on my pc and the kodi addon on the pc, an old laptop and an rpi3b+. In general, I have zero issues during the day and every search works as it should with no 403 errors.
But sometimes, that 403 error comes up when I least expect it too and it happens only on mornings (6.30am to 8.30am). The api resets the counter daily at 12pm pacific time, which is 10am for me here on gmt+2.
I make a search on the kodi addon and I get a 403. Then I make another one on youtube viewer and it works. Once the playback is over, I make another one and I get the 403.
What is even more weird, sometimes I search for something, e.g. song A, and I get the 403 error. I search again the very next second for e.g. song B and there is no error, just results that play! And it happens on both youtube-viewer and kodi's addon.
Does it only happen to me, or is the new api somewhat garbage? I have checked the "stats" table on my google account and the bars are merely visible.
Irrelevant to the above, but I have access to a google account that can not be deleted (info upon request). Do you want me to use it and make a set of keys for youtube-viewer?
It may be promised the "Incognito" mode feature implementation. Of course, it's a piece of work, you can't use API tokens and needs reverse engineering of browser flow. But it'll be player with keeping privacy.
One more thing that proves, at least for me, that the new api is garbage.
I am following the usual routine I described above, like on every other morning. Earlier, I tried to play this list here
youtube-viewer -s --pp=PLPl862cpmLY68gNTNA8irOPNNo9jf8wAL
And from all 12 songs, only 5, 6, 7 and 10 played and all the others were just skipped with a 403 error.
Everything starts as it should, with the descrtiption at the top
=> Description
------------------------------------------------------------------------------------------------------------------
blablabla
then the actual url of the video and its info
------------------------------------------------------------------------------------------------------------------
=> URL: https://www.youtube.com/watch?v=Jy3bsT2kH-k
------------------------------------------------------------------------------------------------------------------
=>> Anser x Eversor - ฮฮฮง <<=
-> Channel : Anser
-> ChannelID : UCtlJcPWvqTQjgAzkmaUykCg
-> VideoID : Jy3bsT2kH-k
blablabla
------------------------------------------------------------------------------------------------------------------
And yet nothing plays! Doing the same with the --debug option gives similar results It plays a different set of videos but it pops the usual 403 error on the ones that do not play, so I now definitely know the reason why they do not play.
What is even worse is that if I try to open a video on mpv, which simply uses youtube-dl to parse the urls and play them, the 403 error comes up again! I do have my own api keys, but how the f am supposed to make mpv use them?
Youtube-viewer 3.7.5, mpv 0.32, youtube-dl 2020.03.24. Every video that got skipped with the 403 error plays at it should in the web browser.
Whats about https://github.com/TeamNewPipe/NewPipe the use a way to analyze the website so they don't need a google account.... maybe this way can be ported here also ?
Can I somehow disable the video description from being printed in the terminal? It will help me a lot with debugging what plays and why.
In other news, one more proof that the new api is garbage. When you search something in kodi, the results are listed as
Channels >
Playlists >
Live >
result 1
result 2
...
When all the api requests are used, the user gets a notification and the addon returns to search. Yesterday, and for the very first time ever in the 6+ years that I use it, the addon returned a blank page! Even the 3 main categories at the top were missing!
All that at 4pm local time, 6 hours after the api resets its counters.
Another first today, for youtube-viewer this time.
I tried playing the list I mentioned above, and for the first time ever ALL the songs were skipped due to 403 error! All that on 3.20pm local time (gmt +2 +summer time), 5 hours and 20 minutes after the api does the reset and with no usage inbetween!
---edit
How can I try debian's api keys for chromium? If anyone is interested, this is what file /etc/chromium.d/apikeys looks like
$ cat /etc/chromium.d/apikeys
# API keys assigned to Debian by Google for access to their services like sync and gmail.
export GOOGLE_API_KEY="AIzaSyCkfPOPZXDKNn8hhgu3JrA62wIgC93d44k"
export GOOGLE_DEFAULT_CLIENT_ID="811574891467.apps.googleusercontent.com"
export GOOGLE_DEFAULT_CLIENT_SECRET="kdloedMFGdGla2P1zacGjAQh"
I made a big discovery about my "all playlist entries go to 403" issue!
I found out that youtube-dl stores some cache inside ~/.cache/youtube-dl and that removing it with
youtube-dl --rm-cache-dir
made all the playlist entries play as they should! There is also the option of --no-cache-dirwhich totally disables caching.
How about adding a variable or a parameter on youtube-viewer that clears the forementioned cache folder? Or make it run on every major update?
Other than that, I tried debian's keys I mention above and they do not seem to work. Comparing them to my keys, the api key and the client secret seem to have a proper "size", (39 and 24 characters respectively), but the client id is much smaller (45 vs 12 characters). Why?
Ok so guys i just dowloaded the program and it doesn't work.
(wasn't logged in)
if i understand i need to log in and ad my acount api?
Plz update the description to include a step by step guide.
Do you mean these?
https://github.com/trizen/youtube-viewer#logging-in
The new api is garbage, episode 10.
As of today, I get the 403 error all day long on both youtube-viewer and kodi. The output of youtube-viewer says nothing more than the 403 forbidden error, but the kodi addon says these in its log
2020-06-11 20:03:21.176 T:140451493947136 ERROR: GetDirectory - Error getting plugin://plugin.video.youtube/special/live/
2020-06-11 20:03:21.192 T:140453077533056 ERROR: CGUIMediaWindow::GetDirectory(plugin://plugin.video.youtube/special/live/) failed
2020-06-11 20:03:29.611 T:140451502339840 ERROR: [plugin.video.youtube] Error reason: |accessNotConfigured| with message: |Access Not Configured. YouTube Data API has not been used in project 498788153161 before or it is disabled. Enable it by visiting https://console.developers.google.com/apis/api/youtube.googleapis.com/overview?project=498788153161 then retry. If you enabled this API recently, wait a few minutes for the action to propagate to our systems and retry.|
Does it mean I have to make new keys now?
I am done with this garbage api, final episode.
As of today, I get this error in kodi
2020-06-19 07:07:03.998 T:140161903998720 ERROR: [plugin.video.youtube] Error reason: |badRequest| with message: |API key not valid. Please pass a valid API key.|
and this in youtube-viewer
[400 Bad Request] Error occurred on URL: https://www.googleapis.com/youtube/v3/playlistItems?key=[...]&maxResults=20&part=contentDetails,snippet&playlistId=PLPl862cpmLY68gNTNA8irOPNNo9jf8wAL&prettyPrint=false
After a full WEEK of pointless error messages in 2 different apps, that I had to... decrypt in order to find the real issue, it turns out it was the keys. How about getting an email to inform me in plain english, since I do use their own f*cking mail service?
I am done with google's stupid devs and their idiotic api. I won't make any new keys, because something else will break sooner or later, so this is the end for me and the kodi addon and youtube-viewer after 5+ years.
@trizen
Thank you for this great app! It has served me perfectly all those years.
Freetube seems to be a wrapper for the youtube web page. It is an electron based app after all. That means it works in a completely different way compared to youtube-viewer or kodi's addon, which access youtube's api directly, so no wonder it always works. So, as far as I am concerned, sorry but I will pass.
@asteriosos Something similar to NewPipe's approach for parsing the YouTube website directly: https://github.com/trizen/pipe-viewer
the instructions here on this github to set up an API Key seem to no longer to match up with the menus of google's cloud console.
It is confusing to say the least for a casual user