Vrtgs/thirtyfour

How to capture the Chromium's requests and responses traffic using thirtyfour

Opened this issue · 5 comments

Codekies commented

I want to use Rust with thirtyfour to open chromium and capture my HTTPS web app traffic. It is for the security tools development usage.

stevepryde commented

It's technically possible using CDP but you'll need to wire it up yourself.

CDP example here
https://github.com/stevepryde/thirtyfour/blob/main/thirtyfour/examples/chrome_devtools.rs

Chrome devtools protocol docs here
https://chromedevtools.github.io/devtools-protocol/

Codekies commented

Hi Steve,
Thanks a lot for the help. I have written some code to access google.com and capture the HTTP requests, response

use thirtyfour::extensions::cdp::ChromeDevTools;
use thirtyfour::prelude::*;

#[tokio::main]
async fn main() -> WebDriverResult<()> {
    let caps = DesiredCapabilities::chrome();
    let driver = WebDriver::new("http://localhost:9515", caps).await?;

    // Use Chrome Devtools Protocol (CDP).
    let dev_tools = ChromeDevTools::new(driver.handle.clone());

    // Enable network tracking.
    dev_tools.execute_cdp("Network.enable").await?;

    // Navigate to https://www.google.com.
    driver.goto("https://www.google.com").await?;

    // Wait for a short duration to allow requests and responses to be captured.
    tokio::time::sleep(std::time::Duration::from_secs(5)).await;

    // Get the captured network events.
    let request_events = dev_tools.execute_cdp("Network.Request").await?;
    let response_events = dev_tools.execute_cdp("Network.Response").await?;

    // Print the captured network events.
    println!("Requests:");
    println!("{:?}", request_events);

    println!("Responses:");
    println!("{:?}", response_events);

    // Disable network tracking.
    dev_tools.execute_cdp("Network.disable").await?;

    // Always explicitly close the browser. There are no async destructors.
    driver.quit().await?;

    Ok(())
}

but when I ran the executable, it showed me this.
Error: UnknownCommand(WebDriverErrorInfo { status: 404, error: "", value: WebDriverErrorValue { message: "unknown command: 'Network.Request' wasn't found\n (Session info: chrome=123.0.6312.123)", error: Some("unknown command"), stacktrace: Some("\tGetHandleVerifier [0x00007FF7F97A7072+63090]\n\t(No symbol) [0x00007FF7F9712CC2]\n\t(No symbol) [0x00007FF7F95AEC65]\n\t(No symbol) [0x00007FF7F959C96F]\n\t(No symbol) [0x00007FF7F959B100]\n\t(No symbol) [0x00007FF7F959BA8F]\n\t(No symbol) [0x00007FF7F959B9C0]\n\t(No symbol) [0x00007FF7F95B1973]\n\t(No symbol) [0x00007FF7F963FEBA]\n\t(No symbol) [0x00007FF7F9616FDA]\n\t(No symbol) [0x00007FF7F9633412]\n\t(No symbol) [0x00007FF7F9616D83]\n\t(No symbol) [0x00007FF7F95E83A8]\n\t(No symbol) [0x00007FF7F95E9441]\n\tGetHandleVerifier [0x00007FF7F9BA25CD+4238285]\n\tGetHandleVerifier [0x00007FF7F9BDF72D+4488493]\n\tGetHandleVerifier [0x00007FF7F9BD7A0F+4456463]\n\tGetHandleVerifier [0x00007FF7F98805B6+953270]\n\t(No symbol) [0x00007FF7F971E58F]\n\t(No symbol) [0x00007FF7F9719264]\n\t(No symbol) [0x00007FF7F971939B]\n\t(No symbol) [0x00007FF7F9709BD4]\n\tBaseThreadInitThunk [0x00007FFD482A257D+29]\n\tRtlUserThreadStart [0x00007FFD48EAAA48+40]\n"), data: None } })
Would you mind helping me to write an example code if you have time? Thanks a lot.

stevepryde commented

Hmm, looking at the docs here: https://chromedevtools.github.io/devtools-protocol/tot/Network/

It looks like Network.Request is a type, not a command.

I've seen some docs pointing at setRequestInterception: https://chromedevtools.github.io/devtools-protocol/tot/Network/#method-setRequestInterception but that shows as deprecated. It says to try the Fetch domain but I couldn't see anything obvious there.

There is some info here that might be useful: https://www.selenium.dev/documentation/webdriver/bidirectional/chrome_devtools/

I'm going to take a look at the selenium bindings and see how they do it there.

  • 👍1
stevepryde commented

There's more info in #55 that might be helpful. Also check out https://github.com/mattsse/chromiumoxide which uses CDP directly without the webdriver side (more like puppeteer).

  • 👍1