Some sites throw a redirection error
shenders13 opened this issue · 1 comments
shenders13 commented
Some sites throw a redirection error
- http://www.dailytelegraph.com.au/technology/sydney-wild-weather-homes-at-risk-as-collaroy-narrabeen-battered-by-storm/news-story/45e63ecdca3e9678f51ac51270996781
- http://www.couriermail.com.au/news/queensland/police-declare-emergency-situation-at-elizabeth-street-woodend/news-story/914f6a0ab1ccc23aa8136280a938f551
- http://www.foxsports.com.au/football/premier-league/jamie-vardy-to-arsenal-striker-will-decide-leicester-future-before-leaving-for-france-with-england/news-story/90b9072963f2b2cde6cbeea2ce3c07e5
Thanks for your consideration in advance!
Sam
gottfrois commented
It seems to be an issue with cookies. The original URL returns a 302 with a Set-Cookie http header. The gem handle this http header correctly but even after the cookie is set, the site keeps returning 302 http responses.
I would need more time to investigate this. Also, I had plan to remove all that redirection home made logic and use another http client library such as faraday and typhoeus.
No quick fix i can think of right now unfortunately other than monkey patching on your side but with the effect of having some other website that will throw a redirect limit error as well because of cookie not set.