Unable to scrape yahoo finance
Closed this issue · 1 comments
ashishchow22 commented
@Ge0rg3 I am trying to scrape the yahoo finance website but getting the 404 error.
I am able to access the response url from the session object , so I believe it is not the problem with the url.
gateway = ApiGateway("https://finance.yahoo.com",
access_key_id=access_key_id, access_key_secret=access_key_secret,
regions=regions)
gateway.start()
session = requests.Session()
session.mount("https://finance.yahoo.com",gateway)
r=session.get("https://finance.yahoo.com/calendar/earnings",params={'symbol':'AAPL'})
print (r.url)
print (r.status_code)
gateway.shutdown()
Ge0rg3 commented
Hi, this is because Yahoo finance blocks requests from the AWS ip range, I believe.