rflechner/ScrapySharp

Error while dockerize ScrapySharp

Opened this issue · 3 comments

Hello, I have a problem when I try to dockerize an application that uses ScrapySharp, because in docker it throws me the following error "Unable to cast object of type 'System.Net.FileWebRequest' to type 'System.Net.HttpWebRequest'" in the line where "Click" function on hyperlink
obraz_2023-02-28_092022505

This is related to this issue: dotnet/runtime#22718

On linux Uri treats "/user/signin" as a valid UriKind.Absolute. I fixed this issue by using if (Action.StartsWith("/") == false && Uri.TryCreate(Action, UriKind.Absolute, out url)).

In my case, the issue was the same: A link in a form was not fully qualified and I adjusted it with something like:

loginForm.Action = (baseUrl.EndsWith("/") || loginForm.Action.StartsWith("/"))
    ? $"{baseUrl}{loginForm.Action}"
    : $"{baseUrl}/{loginForm.Action}";

For me, this is a dotnet bug on Linux / Mac... Might still be useful to state this as "Known issues" in the docs somewhere.