Stopped working
Closed this issue · 113 comments
I think IG changed the policy. Now to see a user page, you have to be logged in. So when you run the script, the GET returns the IG Login Page content.
My IP is now banned, so I can't do a clear check. But yes, something has changed for me too. I will check later if this new IG behaviour is permanent or only for banned IPs.
I'm getting the same issue too after a couple of page views Instagram isn't returning any images and in the console I get
Instagram Feed: It looks like YOUR network has been temporary banned because of too many requests. See https://github.com/jsanahuja/jquery.instagramFeed/issues/25
I'm also seeing the temporary banned message
@benbracedigital
Were you able to see images today before ban? What cache_time had you had?
Yes I was and I could see the images on a different server this morning but after a couple of refreshes, I get the error message. I've set the cache_time to 120.
It seems like 120min is not enough now. The new default and recommended value is 6 hours (360)
Sure, happy to increase to 6 hours but I would expect it to be cached for 2 hours previously and still work rather than error and not show images within minutes?
What version of the script do you have? I can see cached images even in the ban.
And true, with an error in the console
version 1.3.6
Caching is only available since 2.0.0
Ah, thank you. I'll download the latest version and see what results I get.
But only after the end of the ban.
I mean "will see" :)
I can at least see cached images now. I'm going to deployed to a different (non banned) server to see if that all works.
Your (browser) IP is banned, not the IP of the server of your page. Personally, I am currently banned on my PC, but I can flawlessly see the script results on my phone with mobile internet.
Looks to be working. Thanks for your help
I think IG changed the policy. Now to see a user page, you have to be logged in. So when you run the script, the GET returns the IG Login Page content.
I am finally able to check your issue after my last ban ended. The redirect to IG Login Page is only related to the ban. Now, I am able to open and preview any IG page with only offer to register at the bottom.
Yeah, it looks like these are tough times. IG has banned me right after the first successful page update. And ban will last 2-3 days, I expect.
Really tough.
@mahotilo how are you? Is to possible to avoid this issue by using cache_time setting? Thanks!
Is to possible to avoid this issue by using cache_time setting? Thanks!
The cache_time is the only cure we have, It works. But it is very easy to catch a ban if you simultaneously use the script on different sites and make experiments :)
I am thinking of increasing the cache_time up to 24 h for myself to avoid future bans.
I'm kind of on an "eternal ban", because sometimes the feed appears and sometimes not, the same happens with this error, sometimes it appears on the console and sometimes it doesn't, I really don't know what to do anymore :(
...I really don't know what to do anymore :(
First of all, you should update your script to the latest version 2.0.1
This is how your console should look like in case of a ban.
Having the same issue, updated to ver 2.0.1 and console reads:
jquery.instagramFeed.min.js:98 Instagram Feed: Your network has been temporary banned by Instagram because of too many requests. Consider increasing your 'cache_time'. See https://github.com/jsanahuja/jquery.instagramFeed/issues/25 and https://github.com/jsanahuja/jquery.instagramFeed/issues/101 4
Increased cache_time: 360 4 days ago and no change.
Any suggestions to proceed?
Any suggestions to proceed?
Wait until IG unbans you. Last time, 3 days were enough for me.
Well, what I have found about the ban from IG.
- The script and the caching approach work fine. You can update script output on 1-2 pages without being banned.
- But, @jsanahuja... It seems to me, the demo page that has 4 calls of script in a raw with different username + 1 call with a tag might be the point where the visitors get banned.
- But, @jsanahuja... It seems to me, the demo page that has 4 calls of script in a raw with different username + 1 call with a tag might be the point where the visitors get banned.
Maybe we should change it to use the same username on all the examples. The tag call doesn't matter because it never gets banned as far as I know
The tag call doesn't matter because it never gets banned as far as I know
I also think so.
Maybe we should change it to use the same username on all the examples.
Or maybe the function should be called only when the corresponding tab is opened? Barcelona should be kept:)
Or maybe the function should be called only when the corresponding tab is opened?
Mmm, way better
Barcelona should be kept:)
Hahaha, for sure 😄
Just have a question for clarification, if i get banned on one page, will i be banned on all pages that use this script, or is the ban on a per page basis? or is it on a per instagram page basis?
I believe, it is a ban on your IP. Any nonlogged-in requests from your IP are banned by IG during the ban period.
One last time (hopefully)
My clear experiment
- No ban, more than 6 h after the last update
- My first test page (1 "name" script instance)
- Demo page (4 "name" script instances)
- My second test page (1 "name" script instance)
- Ban
Magic number is 6.
I think this issue should be closed. It has nothing to do with script and caching. It is about the number of script calls.
#104 Feature/proxy this may solve the problem.
But it needs testing. This is still a draft.
I can post a backend heroku, everything is easy there.
Hm, it doesn't work for me
https://instagram-feed-proxy.herokuapp.com/?url=https://www.instagram.com/instagram
Hm, it doesn't work for me
https://instagram-feed-proxy.herokuapp.com/?url=https://www.instagram.com/instagram
This is normal. You only need to get _sharedData from html response.
This proxy is only needed to execute the script when requested via ajax get
You can test this without the caching feature that was added earlier.
@mahotilo I hope you understand what I'm talking about. I'm not very good at translating into English, sorry.
I hope you understand what I'm talking about.
Not a problem.
This is normal. You only need to get _sharedData from html response.
Yes, I can reproduce your example with _sharedData. Thx.
Well, I have just tested it by changing the host attribute to proxy.
'host': 'https://instagram-feed-proxy.herokuapp.com/?url=https://www.instagram.com/',
It works well. Hurrah!
But why do we need to introduce two new attributes just to change the value of the host?
Wouldn't it be easier to add the proxy address to the documentation as a recommended alternative to the host value?
P.S.
Funny. Today I had to open over 20 IG pages to get banned :)
As I can see it, this solution has previously been described here https://lopodofoodservice.com/assets/js/plugins/instagramfeed/
but with another proxy
EDIT
I mean, the proxy may or may not work. We cannot rely too much on third-party solutions. When the users themselves change the host, it is their decision and a headache to check the proxy health.
But why do we need to introduce two new attributes just to change the value of the
host?
Yeah, this is the best solution
Funny. Today I had to open over 20 IG pages to get banned :)
The whole problem is session related.
You can't create more than two (or three, haven't tested it) sessions from one IP. If you clear your cookies, you will lose the session ID, thereby losing access to the instagram view, for example through incognito mode.
But by saving your session, you can view without restrictions. On heroku there is no request proxying through third-party servers, there is just a request with the same session ID in the cookie.
But by saving your session, you can view without restrictions.
Hm, I guess this is the number of usernames views per day that is limited by IG for one IP. Today to get a ban I clicked on near 20 profiles on https://www.instagram.com/explore/tags/paradise. It was in incognito mode in one session.
Hm, I guess this is the number of username views per day that is limited by IG for one IP. Today to get a ban I clicked on near 20 profiles on https://www.instagram.com/explore/tags/paradise. It was in incognito mode in one session.
My mistake. It turns out I was using my account session. 🤦♂️
Heroku sends requests from my session and returns the result.
How many views will last before blocking or captcha?
I think I need to install and test the logger. But at least this solves the problem, especially if you are using caching.
How many views will last before blocking or captcha?
So far, I have run the script with the proxy in the host over 20 times in different incognito sessions. It still works like a charm!
But at least this solves the problem, especially if you are using caching.
It looks like a complete victory.
I cross my fingers.
The result of the discussion: all you need to do is specify 'host': 'https://instagram-feed-proxy.herokuapp.com/?url=https://www.instagram.com/' (or another proxy of your choice) in instagramFeed settings.
I will continue to explore caching, proxying and process optimization on heroku app, after which I will post it to the public.
@m0nclous, @jsanahuja
Would you make a PR for the documentation?
Maybe here?
@mahotilo I figured out how to do without a proxy.
To do this, you need to know the user id, from where the pictures will be taken.
https://www.instagram.com/web/search/topsearch/?query=<username>
Then we make a request with the found id
https://instagram.com/graphql/query/?query_id=17888483320059182&variables={"id":"<user_id>","first":20,"after":null}
Thus, you can keep the ability to insert instagram via username.
The ID can be cached in order not to receive it all the time.
I don't see any blocking from instagram.
I should be able to implement this in 1-2 days, including the vanilla plugin.
The question remains how to get data from the tag without blocking.
https://stackoverflow.com/questions/17373886/how-can-i-get-a-users-media-from-instagram-without-authenticating-as-a-user/47243409#47243409
https://stackoverflow.com/questions/11796349/instagram-how-to-get-my-user-id-from-username/52808289#52808289
jsanahuja/InstagramFeed#49 (comment)
It seems that the tag is never blocked.
Then there is no problem. I will try to find time tomorrow and implement what I wrote above for testing.
It looks like the issue will be resolved.
@mahotilo, any news about fix?
Currently the fix with proxy host: "https://instagram-feed-proxy.herokuapp.com/?url=https://www.instagram.com/'" doesn't work.
Currently the fix with proxy host: "https://instagram-feed-proxy.herokuapp.com/?url=https://www.instagram.com/'" doesn't work.
Unfortunately, that is to be expected. IG is hard to cheat.
But caching still does the trick. It is not perfect but quite acceptable.
@m0nclous, do you have any new findings?
Currently the fix with proxy host: "https://instagram-feed-proxy.herokuapp.com/?url=https://www.instagram.com/'" doesn't work.
Unfortunately, that is to be expected. IG is hard to cheat.
But caching still does the trick. It is not perfect but quite acceptable.@m0nclous, do you have any new findings?
If i will use a value "1440" for the "cache_time" then single IP will not be banned? If i understood right then this option allows to decrease a quantity of requests from each IP. Is this right?
As I understand it.
IG wants you to subscribe and log in. So, it limits the number of interactions of an unregistered user with images and profiles (usernames). It doesn't matter if it is several images in one profile or just one image in several profiles.
The script acts as unregistered user, so your IP is the subject of ban. This is not the IP address of the server hosting the page with the script. You should notice that other users are still able to see script output despite you are banned.
Thus, if you use a big cache time, you protect yourself from being banned when interacting with the same profile in the same page. But, if you run multiple scrips with different profiles (username) within a short period of time, you will still be banned, no mater how big the cache time is. Or not :) The question is how many IG data updates (get()'s) will you make.
@mahotilo, any news about fix?
Currently the fix with proxy host: "https://instagram-feed-proxy.herokuapp.com/?url=https://www.instagram.com/'" doesn't work.
Yeah, they still blocked the session after a while
@m0nclous, do you have any new findings?
I implemented (locally) what I said above, but I got the same problem - Instagram blocks requests to
https://www.instagram.com/web/search/topsearch/?query=<username>
after about 6 requests
BUT! Instagram doesn't block it
https://instagram.com/graphql/query/?query_id=17888483320059182&variables={"id":"<user_id>","first":20,"after":null}
Thus, we can avoid errors if we set the target profile using ID, but to completely solve the issue, we need to find a way to get ID based on username without blocking.
We can receive data through https://www.instagram.com/<username>/?__a=1 but this path is blocked after about 4 requests
but this path is blocked after about 4 requests
At least we can create an instruction on how to get this ID manually.
@mahotilo
https://instagram.com/graphql/query/?query_id=17888483320059182&variables={"id":"<user_id>","first":20,"after":null}
Doesn't return profile info, such as profile_pic_url/username/biography
What to do with it?
@m0nclous
Hmm. Very intriguing. While I was looking for an answer to your question I found this article
- author describes the same problem and the same approach to solving
- he gets missed user data from request https://www.instagram.com/username/?__a=1
- he gets more than 12 images with offset in request :)
His demo is at the end of
https://carlos-henreis.github.io/
@mahotilo Yes, I read that too. The request is blocking just like the others
We can cache this information for a long time, but this is not a solution to the problem, since caching is not a solution for many users, if you look at this issue
I believe that hiding user account data is a part of IG's policy. So there is little chance of finding a long-lasting trick.
But for many users, this script is just a handy way to get an easy-to-update picture gallery on the site. I have even hidden a user info in some projects.
So this might be an option:
- use cached full data by username with delay and possible pause for ban
- use instantly updated images by ID without username info. (If you can find the ID in JSON, you can insert the profile image and the name yourself)
Personally, I think caching is a great solution. The only real alternative is to use IG API.
Hello,
The plugin stopped work.
It looks like they have added a CORS restriction again but only in some of their servers. That is why some requests are success and some are CORS blocked. We will test and implement the graphql solution soon
@jsanahuja many thanks! Can you say as soon new solution will be ready? (approximate time of course). Sorry for the indiscreet question.
Unfortunately GraphQL API is also affected by the arbitrary CORS issues.
Release 2.1.1 implementing GraphQL (with user_id parameter) is ready. We have also implemented request retries (limited by max_tries parameter) to avoid arbitrary CORS errors.
@jsanahuja It is should updated plugin version to 2.1.1 on my site? Is this fix the CORS issue?
@jsanahuja It is should updated plugin version to 2.1.1 on my site? Is this fix the CORS issue?
CORS issue is not something we can fix client-side. As it is arbitrary (only some of their cluster servers have a CORS restriction) we have implemented a policy of retries.
You should definitely update to the 2.1.1 but the real solution would be using a proxy and the graphQL (user_id parameter)
It isn’t working anymore.
So since the idiots at IG introduced CORS, there is simply no way to reliably load the last 12 IG posts based on the username any longer?
The only solution is to use graphQL with user_id, which we.. can not get programatically and have to use manual workarounds to get it first?
They want to know personally everyone who opens their site :)
Actually, I've just found a (very tricky, but working) way to bypass the IG CORS policy – use this as the host:
'host': "https://images" + ~~(Math.random() * 3333) + "-focus-opensocial.googleusercontent.com/gadgets/proxy?container=none&url=https://www.instagram.com/"
Actually, I've just found a (very tricky, but working) way to bypass the IG CORS policy – use this as the host:
'host': "https://images" + ~~(Math.random() * 3333) + "-focus-opensocial.googleusercontent.com/gadgets/proxy?container=none&url=https://www.instagram.com/"
Thanks, it works!
Actually, I've just found a (very tricky, but working) way to bypass the IG CORS policy – use this as the host:
'host': "https://images" + ~~(Math.random() * 3333) + "-focus-opensocial.googleusercontent.com/gadgets/proxy?container=none&url=https://www.instagram.com/"
This works for me too! Thank you! <3
'host': "https://images" + ~~(Math.random() * 3333) + "-focus-opensocial.googleusercontent.com/gadgets/proxy?container=none&url=https://www.instagram.com/"
Indeed, magic works. But, if I try to open the generated URL (host + username) in the browser, I get the message We're sorry... ... but your computer or network may be sending automated queries. To protect our users, we can't process your request right now.
So the question is, should I just not open this URL directly in the browser, or will we eventually get banned by Google this time?
I would say it’s unlikely to get banned by Google, I‘ve seen other use this at a fairly high scale. I just hope that this doesn’t get banned by IG.
Actually, I've just found a (very tricky, but working) way to bypass the IG CORS policy – use this as the host:
'host': "https://images" + ~~(Math.random() * 3333) + "-focus-opensocial.googleusercontent.com/gadgets/proxy?container=none&url=https://www.instagram.com/"
Works like a charm! Cheers!
But, if I try to open the generated URL (host + username) in the browser, I get the message
We're sorry... ... but your computer or network may be sending automated queries. To protect our users, we can't process your request right now.
Google checks the referer header
@m0nclous
So as of now, this is kind of a permanent solution! Can we call it the Google proxy? :)
The proxy works thanks but I’m finding extra margin now between image rows - can I fix this?
Actually, I've just found a (very tricky, but working) way to bypass the IG CORS policy – use this as the host:
'host': "https://images" + ~~(Math.random() * 3333) + "-focus-opensocial.googleusercontent.com/gadgets/proxy?container=none&url=https://www.instagram.com/"No longer appears to be working.
@introzen This works if you make an ajax request or add a referer header manually #103 (comment)
https://images" + ~~(Math.random() * 3333) + "-focus-opensocial.googleusercontent.com/gadgets/proxy?container=none&url=
This is not working for following link
https://www.instagram.com/graphql/query/?query_hash=" + queryHash + "&variables=" + variables;
Upgrade to 3.0.0 and use this #103 (comment)
https://images" + ~~(Math.random() * 3333) + "-focus-opensocial.googleusercontent.com/gadgets/proxy?container=none&url=
This is not working for following link
https://www.instagram.com/graphql/query/?query_hash=" + queryHash + "&variables=" + variables;
Right. We are not able to use GraphQL anymore unless we use a real proxy.
This looks almost like we are spoofing/tricking IG by manufacturing the header - I would not be surprised if the solution goes down again, no?
@jsanahuja i noticed this error now. The instagram feed doesn't displayed. Browser firefox. I use the latest version 3.0.0.
@mahotilo Sometimes i see the error on real hosting too.
Also i noticed the next error when i change username in plugin settings:
Sometimes i see the error on real hosting too.
I do not have it here. Could you describe a condition to reproduce?
Also i noticed the next error when i change username in plugin settings:
It can be if parse_caption returns false. It should not be an isolated error, just a consequence of an early problem.
@mahotilo I noticed this error only for some instagram accounts. Please try the next user name:
modern_presta
Update: The error occur when instagram account doesn't have bio.
@jsanahuja i noticed this error now. The instagram feed doesn't displayed. Browser firefox. I use the latest version 3.0.0.
This happens when Google responds with an error, because in this case it does not send the header "Access-Control-Allow-Origin"
To understand the reason - you need to reproduce the problem and see the answer from Google manually.
I think this solution is still crude and more logging needs to be added.
@mahotilo I noticed this error only for some instagram accounts. Please try the next user name:
modern_prestaUpdate: The error occur when instagram account doesn't have bio.
I have checked, there is the same error here.
But, it has happened to me before too. It is related to the first VIDEO in the row.
@mahotilo I noticed this error only for some instagram accounts. Please try the next user name:
modern_presta
Update: The error occur when instagram account doesn't have bio.I have checked, there is the same error here.
But, it has happened to me before too. It is related to the first VIDEO in the row.
This happens when parse_caption() dont finds a caption and returns false in escape_string where the string is expected.
I will fix it.
We should definitely have created a separate new issue for this :)
We should definitely have created a separate new issue for this :)
Can you do it please?
@jsanahuja i noticed this error now. The instagram feed doesn't displayed. Browser firefox. I use the latest version 3.0.0.
This happens when Google responds with an error, because in this case it does not send the header "Access-Control-Allow-Origin"
To understand the reason - you need to reproduce the problem and see the answer from Google manually.I think this solution is still crude and more logging needs to be added.
@m0nclous this is because you are running the script without a web server. The solution is setting the 'Referer' header but web browsers won't allow us to do that with Javascript for security reasons
ALso gettings this. Using 3.0.0 whit the "fix" host
@thomashdk, I can only quote for you
this is because you are running the script without a web server
On my site I get this ?
@thomashdk, can you see the script output for the demo page https://www.sowecms.com/demos/jquery.instagramFeed/index.html?
hmm no ? @mahotilo
@thomashdk, can you see the script output for the demo page https://www.sowecms.com/demos/jquery.instagramFeed/index.html?
@thomashdk
Hmm, it works for me here
