Not able to submit a form

Question

Not able to submit a form

Closed this issue 6 years ago · 7 comments

Hello :)

Thank you for this amazing project. I'm trying to log in on a Central Authentication Service (i think drupal...) so I have to fill a form with my username and password and then submit the form. I tried to submit the form or click on the button submit (or even both) but erik appears to be stuck on the login page.
I tried to connect with HtmlUnit (Java) and it worked, but I need to do it on an iOS App..
Could someone help me? Here are the snippets :

SWIFT :

url = URL(string:"http://edt.insa-cvl.fr")!
        let username = "username"
        let password = "password"
        // visit
        var future: Future<Document, NSError> = Erik.visitFuture(url: url)
        // fill input field
        future = future.flatMap { doc -> Future<Document, NSError> in
            if let inputUsername = doc.querySelector("input[name=\"username\"]") {
                inputUsername["value"] = username
            }
            
            if let inputPassword = doc.querySelector("input[name=\"password\"]") {
                inputPassword["value"] = password
            }
            
            
            
            if let form = doc.querySelector("form[id=\"fm1\"]") as? Form {
                form.submit()
            }
            
            if let buttonSubmit = doc.querySelector("input[name=\"submit\"]") {
                buttonSubmit.click()
            }

        
            return Erik.currentContentFuture()
        }
        // finally get final result as success or error
        future.onSuccess { doc in
            print(String(describing: doc))
        }
        future.onFailure { error in
            print("\(error)")
        }

JAVA :

try (final WebClient webClient = new WebClient(BrowserVersion.FIREFOX_52)) {
		    final HtmlPage page = webClient.getPage("http://edt.insa-cvl.fr");
		   final HtmlTextInput usernameField = (HtmlTextInput) page.getElementById("username");
		    final HtmlPasswordInput passwordField = (HtmlPasswordInput) page.getElementById("password");
		    
		    usernameField.setValueAttribute("username");
		    passwordField.setValueAttribute("password");
		    
		    final HtmlSubmitInput buttonSubmit = page.getElementByName("submit");
		    final HtmlPage page2 = buttonSubmit.click();
		     
		     System.out.println(page2.asXml());

		}

Thanks in advance!

Answer 1 · 2018-01-17T09:03:43.000Z

The javascript used (could be seen using compilation flag TEST)

var erik = document.querySelector('[id="username"]');
erik.setAttribute('value', 'username');

var erik = document.querySelector('[id="password"]');
erik.setAttribute('value', 'password');

var erik = document.querySelector('[id="fm1"]');
erik.submit(); // 
https://developer.mozilla.org/fr/docs/Web/API/HTMLFormElement/submit

VM263:1 Uncaught TypeError: erik.submit is not a function
at :1:6

Maybe some overlapping with the input button with name "submit", so you try with input button

var erik = document;
erik = erik.querySelector('input[name="submit"]');
erik.click(); // https://www.w3schools.com/jsref/met_html_click.asp

Answer 2 · 2018-01-17T09:09:14.000Z

I commit some new methods in Future.swift, you can update or copy it
https://github.com/phimage/Erik/blob/master/Sources/Future/Future.swift
in your code you can use clickFuture

            if let buttonSubmit = doc.querySelector("input[name=\"submit\"]") {
                return buttonSubmit.clickFuture().flatMap { o in
                    return Erik.currentContentFuture()
                }
            }

            return Erik.currentContentFuture()

I have in html result
<div id="msg" class="errors">Mauvais identifiant / mot de passe.</div>

remove the form.submit() from your code

Answer 3 · 2018-01-17T10:11:09.000Z

Thank you for your quick answer.
I tried to replace

if let form = doc.querySelector("form[id=\"fm1\"]") as? Form {
                form.submit()
            }
            
            if let buttonSubmit = doc.querySelector("input[name=\"submit\"]") {
                buttonSubmit.click()
            }

by

 if let buttonSubmit = doc.querySelector("input[name=\"submit\"]") {
                return buttonSubmit.clickFuture().flatMap { o in
                    return Erik.currentContentFuture()
                }
            }

I did synchronize the new version of Erik/Future but I still have the same result. I tried to print the html result in the future.onSuccess function but it appears to be stuck on the login page again.
Even with wrong login/pw, I'm not able to get the same result as yours.

For your information, my swift project is in Swift 3.2 and so are the Pods. Moreover I allowed Arbitrary Loads in App transport Security Settings in info.plist.
Did you make any change in your project to get your result?

Thank you
Arnaud

Answer 4 · 2018-01-17T10:43:08.000Z

I use unit testing to execute your code.
latest code from branch master so swift 3.2

I do not make any configuration on dispatchQueue used in Erik, and that could change the behaviour
Sometimes I need to do it to specify where I want to receive the callback, for instance:

 if let engine = Erik.sharedInstance.layoutEngine as? WebKitLayoutEngine {
         engine.javaScriptQueue = .main
         engine.callBackQueue = .main
}

Answer 5 · 2018-01-17T11:02:18.000Z

I do not quite understand why you are getting a result in your Unit tests while I am not.
However, I sent you a tweet to figure out this problem! :)
Will post here the solution of this issue (I hope)
Arnaud

Answer 6 · 2018-01-22T16:57:43.000Z

Hello everyone,

We finally managed to solve the problem. Actually it was more a website problem than an Erik problem.
To summarize, the website I wanted to browse is in fact a javascript web app I think, and every call to the website is encrypted. This website is pretty slow.
First I had to submit the login form, Eric kindly added a new function to make it easier, so here is the full process :

    let url = URL(string:"http://your-url.fr")!
    var future: Future<Document, NSError> = Erik.visitFuture(url: url)
    // fill input field
    future = future.flatMap { doc -> Future<Document, NSError> in
        
        if let inputUsername = doc.querySelector("input[name=\"username\"]") {
            inputUsername["value"] = username
        }
        
        if let inputPassword = doc.querySelector("input[name=\"password\"]") {
            inputPassword["value"] = password
        }
        
        if let buttonSubmit = doc.querySelector("input[name=\"submit\"]") {
            return buttonSubmit.clickFuture().flatMap { o in
                return Erik.currentContentFuture()
            }
        }     
        return Erik.currentContentFuture()
    }  
    future.onSuccess { doc in
        //do what you have to do, html inspection...
    }
    
    future.onFailure { error in
        print("\(error)")
    }

Then, as the website is pretty slow, I hadn't all the html in the returned doc. I added a delay in Eriks' source code (LayoutEngine.swift) so it can parse after the website is fully loaded (javascript is long to process due to encrypted stuff) :

fileprivate func handleHTML(_ completionHandler: CompletionHandler?) {
        javaScriptQueue.async { [unowned self] in
          DispatchQueue.main.asyncAfter(deadline: .now() + 2.5) {
                self.webView.evaluateJavaScript(self.javascriptToGetContent.javascript) { [unowned self] (obj, error) -> Void in
                    self.callBackQueue.async {
                        completionHandler?(obj, error)
                    }
                }
            }            
        }
    }

And for each click interaction on the page, I used Erik.currentContent to refresh the html.
Now it works like a charm.
I hope this will help someone one day.

One last thing, I changed the user-agent of Erik too by adding this to the beginning of the code:

let userAgent = "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.12; rv:57.0) Gecko/20100101 Firefox/57.0"
        
        if let engine = Erik.sharedInstance.layoutEngine as? WebKitLayoutEngine {
            engine.webView.customUserAgent = userAgent
        }

Thank you a lot for your help Eric!
Arnaud

Answer 7 · 2018-01-23T08:13:38.000Z

Thank you for this report (a lot of people do not take time to explain when the issue is fixed for them)

Wait end of page loading and maybe javascript execution is a subject that could be enhanced in Erik. I do not study other libraries to see how they achieve this. I make some implementation on my own and I think it's not very viable

I close