302 Redirect after Login
Opened this issue · 2 comments
imukai commented
I can successfully populate and submit a login form with Scrapy. As can be common with logins the server sends back a 302 redirect to a main page that Scrapy sees as an error (a WebException).
I tried enabling .AllowAutoRedirect as well as AllowMetaRedirect on the ScrapingBrowser object with no change in behavior.
Please advise.
imukai commented
My current workaround is to trap for the exception and manually call the next page, but this seems less than ideal as relying on exceptions for normal flow is usually bad.
catch (AggregateException ex) {
var inner = ex.InnerExceptions.First() as WebException;
redoUrl = inner?.Response?.Headers["Location"];
}