Support for authenticate prompts

[chanderlud]

The ability to enter text into authentication prompts like this one would be a great addition to thirtyfour.

Puppeteer supports this like so (code from here)

const puppeteer = require('puppeteer');

const proxy = 'http://my.proxy.com:3001';
const username = 'jimmy49';
const password = 'password123';

(async () => {
    // Pass proxy URL into the --proxy-server arg
    const browser = await puppeteer.launch({
        args: [`--proxy-server=${proxy}`],
    });

    const page = await browser.newPage()

    // Authenticate our proxy with username and password defined above
    await page.authenticate({ username, password });

    await page.goto('https://www.google.com');

    await browser.close();
})();

Something as simple as this would be awesome.

let mut caps = DesiredCapabilities::chrome();

let driver = WebDriver::new("http://localhost:9515", caps).await?;

driver.goto("https://www.google.com").await?;

driver.authenticate("user", "pass").await?;

[stevepryde]

I agree this would be useful. I haven't used puppeteer so I don't know how it figures out which field is which.

How do you expect it to find which element is the username field and which is the password? I assume we would match on a range of CSS attributes. It's probably also similar to what password managers do to automatically populate the fields.

Are you interested in contributing this feature? Even if we don't handle every case for now it's probably still worth it.

[chanderlud]

I dont think that this prompt is part of the DOM. I think it is a GUI element of Chrome. As far as figuring out which field is which, I think these prompts always have just a username and password field.

Chrome shows this popup when the WWW-Authenticate or Proxy-Authenticate header is set on a response, the response body on the request is empty and chrome is rendering the default empty html document. In this screenshot the auth type is Basic so chrome shows the username and password prompt.

I took a look at the puppeteer source code and I am pretty sure they are using request interception and rewriting to add the authentication header to each request instead of actually interacting with the chrome GUI. So I do not think it is possible to implement it the same way in thirtyfour.

Currently I am using a chrome plugin to input the values automatically, something like this. However I am having issues in my ubuntu server environment as chrome plugins are only supported by non-headless chrome.

[stevepryde]

Oh, if it's a prompt you may be able to handle it by using the send_alert_text() and accept_alert().

Otherwise we may need to support it the same way puppeteer does (which might work on chromium browsers only) using CDP.

There are a few different options mentioned here
https://stackoverflow.com/questions/11522434/how-to-handle-login-pop-up-window-using-selenium-webdriver

[stevepryde]

Fyi as far as I know puppeteer and others are just using CDP (chrome devtools protocol) to interact with the browser, which you can do via selenium. Thus anything puppeteer can do should in theory be possible using any selenium library (and yet selenium also supports webdriver so it is actually more capable than puppeteer etc).

Thirtyfour has some basic CDP features already implemented but it needs a lot more work to create high level wrappers for all CDP features. That said, it is there if anyone needs to use it.

[chanderlud]

Calling send_alert_text() while the authentication dialog is open results in NoSuchAlert("no alert is currently visible: no such alert\n (Session info: chrome=107.0.5304.107)")
I am going to take a look at using CDP to add an Authorization header globally.

[stevepryde]

Ok yeah it's probably an os dialog so no browser API can interact with it.

You can specify username:password in the url manually. That's probably a lot easier than trying to intercept network requests.

But if you do want to intercept network requests that is an official feature of selenium 4 (I believe selenium 4 just uses CDP to do it anyway). You can probably get some clues from other selenium libraries (java, python etc)

[chanderlud]

Specifying the credentials in the url does work great for the login to website prompts but unfortunately for login to proxy prompts it does nothing. I think I can use this method on chromiumoxide to globally set a Proxy-Authorization header on all requests. This should get rid of the pesky prompts...

[stevepryde]

Nice. I have mixed feelings about supporting CDP fully in thirtyfour just because (a) it's (mostly) chromium only and (b) it's not an official spec

It would be nice if the various crates could all talk to the webdriver via a unified backend though. That way you could create a connection to the webdriver and then use any other crate on top. But it's a lot of work to get there and requires other crate owners to buy into it.

[chanderlud]

I am going to close this issue now. For anyone else dealing with something similar to me, just use a chrome extension to enter the authentication details. It is easier to get non-headless chrome working than it is to set the authentication header using CDP.

[stevepryde]

I did think we could leave it open in case someone had a good fix for it. If it's possible to do it with CDP then it might be worth someone looking into it.

Happy to close it for now and someone can open a new issue if they want to