Error creating webdriver using docker-compose (Selenium-standalone: chrome)

Question

Error creating webdriver using docker-compose (Selenium-standalone: chrome)

Najidnadri opened this issue 2 years ago · 1 comments

Hello,

I'm having some trouble trying to create a webserver using selenium-standalone:chrome with docker-compose.

These are the necessary codes:

main.rs

#[get("/search/{keyword}")]
async fn search(path: web::Path<String>) -> impl Responder {
    let mut caps = DesiredCapabilities::chrome();
    caps.add_chrome_option(
        "prefs",
        serde_json::json!({
            "profile.default_content_settings": {
                "images": 2
            },
            "profile.managed_default_content_settings": {
                "images": 2
            }
        }),
    ).map_err(|_e| AppError::ChromeOptionErr).unwrap();
    let driver = WebDriver::new("http://localhost:4444/wd/hub", &caps)
    .await
    //.map_err(|_e| AppError::CreateWebDriverErr(4442))
    .unwrap();  //error comes up here

    let keyword = path.into_inner();
    let response = search_keyword(keyword, driver).await.unwrap();
    let server_response = ServerResponse::AnimeSearch(response);

    HttpResponse::Ok().json(server_response)
}

docker-compose file

services:
  web:
    build: .
    ports:
     - "8080:8080"

  chrome:
    image: selenium/node-chrome:4.1.2-20220217
    shm_size: 2gb
    depends_on:
      - selenium-hub
    environment:
      - SE_EVENT_BUS_HOST=selenium-hub
      - SE_EVENT_BUS_PUBLISH_PORT=4442
      - SE_EVENT_BUS_SUBSCRIBE_PORT=4443
  
  selenium-hub:
    image: selenium/hub:4.1.2-20220217
    container_name: selenium-hub
    ports:
      - "4442:4442"
      - "4443:4443"
      - "4444:4444"

Error Message

Network gogoanime-scrapper_default  Creating
Network gogoanime-scrapper_default  Created
Container gogoanime-scrapper-web-1  Creating
Container selenium-hub  Creating
Container gogoanime-scrapper-web-1  Created
Container selenium-hub  Created
Container gogoanime-scrapper-chrome-1  Creating
Container gogoanime-scrapper-chrome-1  Created
Attaching to gogoanime-scrapper-chrome-1, gogoanime-scrapper-web-1, selenium-hub
gogoanime-scrapper-web-1     | actix web go!
selenium-hub                 | 2022-03-16 14:10:59,276 INFO Included extra file "/etc/supervisor/conf.d/selenium-grid-hub.conf" during parsing
selenium-hub                 | 2022-03-16 14:10:59,280 INFO supervisord started with pid 8
selenium-hub                 | 2022-03-16 14:11:00,282 INFO spawned: 'selenium-grid-hub' with pid 10
selenium-hub                 | 14:11:00.682 INFO [LoggingOptions.configureLogEncoding] - Using the system default encoding
selenium-hub                 | 2022-03-16 14:11:00,685 INFO success: selenium-grid-hub entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
selenium-hub                 | 14:11:00.689 INFO [OpenTelemetryTracer.createTracer] - Using OpenTelemetry for tracing
selenium-hub                 | 14:11:00.863 INFO [BoundZmqEventBus.<init>] - XPUB binding to [binding to tcp://*:4442, advertising as tcp://172.21.0.3:4442], XSUB binding to [binding to tcp://*:4443, advertising as tcp://172.21.0.3:4443]
selenium-hub                 | 14:11:00.949 INFO [UnboundZmqEventBus.<init>] - Connecting to tcp://172.21.0.3:4442 and tcp://172.21.0.3:4443
selenium-hub                 | 14:11:00.981 INFO [UnboundZmqEventBus.<init>] - Sockets created
gogoanime-scrapper-chrome-1  | 2022-03-16 14:11:00,989 INFO Included extra file "/etc/supervisor/conf.d/selenium.conf" during parsing
gogoanime-scrapper-chrome-1  | 2022-03-16 14:11:00,992 INFO supervisord started with pid 9
selenium-hub                 | 14:11:01.984 INFO [UnboundZmqEventBus.<init>] - Event bus ready
gogoanime-scrapper-chrome-1  | 2022-03-16 14:11:01,994 INFO spawned: 'xvfb' with pid 11
gogoanime-scrapper-chrome-1  | 2022-03-16 14:11:01,997 INFO spawned: 'vnc' with pid 12
gogoanime-scrapper-chrome-1  | 2022-03-16 14:11:01,999 INFO spawned: 'novnc' with pid 13
gogoanime-scrapper-chrome-1  | 2022-03-16 14:11:02,002 INFO spawned: 'selenium-node' with pid 14
gogoanime-scrapper-chrome-1  | 2022-03-16 14:11:02,018 INFO success: xvfb entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
gogoanime-scrapper-chrome-1  | 2022-03-16 14:11:02,018 INFO success: vnc entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
gogoanime-scrapper-chrome-1  | 2022-03-16 14:11:02,018 INFO success: novnc entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
gogoanime-scrapper-chrome-1  | 2022-03-16 14:11:02,018 INFO success: selenium-node entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
gogoanime-scrapper-chrome-1  | Generating Selenium Config
gogoanime-scrapper-chrome-1  | Configuring server...
gogoanime-scrapper-chrome-1  | Setting up SE_NODE_HOST...
gogoanime-scrapper-chrome-1  | Setting up SE_NODE_PORT...
gogoanime-scrapper-chrome-1  | Setting up SE_NODE_GRID_URL...
gogoanime-scrapper-chrome-1  | Selenium Grid Node configuration:
gogoanime-scrapper-chrome-1  | [events]
gogoanime-scrapper-chrome-1  | publish = "tcp://selenium-hub:4442"
gogoanime-scrapper-chrome-1  | subscribe = "tcp://selenium-hub:4443"
gogoanime-scrapper-chrome-1  |
gogoanime-scrapper-chrome-1  | [node]
gogoanime-scrapper-chrome-1  | session-timeout = "300"
gogoanime-scrapper-chrome-1  | override-max-sessions = false
gogoanime-scrapper-chrome-1  | detect-drivers = false
gogoanime-scrapper-chrome-1  | max-sessions = 1
gogoanime-scrapper-chrome-1  |
gogoanime-scrapper-chrome-1  | [[node.driver-configuration]]
gogoanime-scrapper-chrome-1  | display-name = "chrome"
gogoanime-scrapper-chrome-1  | stereotype = '{"browserName": "chrome", "browserVersion": "98.0", "platformName": "Linux"}'
gogoanime-scrapper-chrome-1  | max-sessions = 1
gogoanime-scrapper-chrome-1  |
gogoanime-scrapper-chrome-1  | Starting Selenium Grid Node...
selenium-hub                 | 14:11:02.914 INFO [Hub.execute] - Started Selenium Hub 4.1.2 (revision 9a5a329c5a): http://172.21.0.3:4444
gogoanime-scrapper-chrome-1  | 14:11:02.953 INFO [LoggingOptions.configureLogEncoding] - Using the system default encoding
gogoanime-scrapper-chrome-1  | 14:11:02.962 INFO [OpenTelemetryTracer.createTracer] - Using OpenTelemetry for tracing
gogoanime-scrapper-chrome-1  | 14:11:03.106 INFO [UnboundZmqEventBus.<init>] - Connecting to tcp://selenium-hub:4442 and tcp://selenium-hub:4443
gogoanime-scrapper-chrome-1  | 14:11:03.166 INFO [UnboundZmqEventBus.<init>] - Sockets created
gogoanime-scrapper-chrome-1  | 14:11:04.170 INFO [UnboundZmqEventBus.<init>] - Event bus ready
gogoanime-scrapper-chrome-1  | 14:11:04.316 INFO [NodeServer.createHandlers] - Reporting self as: http://172.21.0.4:5555
gogoanime-scrapper-chrome-1  | 14:11:04.337 INFO [NodeOptions.getSessionFactories] - Detected 8 available processors
gogoanime-scrapper-chrome-1  | 14:11:04.401 INFO [NodeOptions.report] - Adding chrome for {"browserVersion": "98.0","browserName": "chrome","platformName": "Linux","se:vncEnabled": true} 1 times
gogoanime-scrapper-chrome-1  | 14:11:04.415 INFO [Node.<init>] - Binding additional locator mechanisms: id, relative, name
selenium-hub                 | 14:11:04.534 INFO [Node.<init>] - Binding additional locator mechanisms: id, relative, name
gogoanime-scrapper-chrome-1  | 14:11:04.682 INFO [NodeServer$1.start] - Starting registration process for node id 42ff5519-19ff-44c7-8940-a8f3541d719a
gogoanime-scrapper-chrome-1  | 14:11:04.684 INFO [NodeServer.execute] - Started Selenium node 4.1.2 (revision 9a5a329c5a): http://172.21.0.4:5555
gogoanime-scrapper-chrome-1  | 14:11:04.691 INFO [NodeServer$1.lambda$start$1] - Sending registration event...
selenium-hub                 | 14:11:05.039 INFO [LocalDistributor.add] - Added node 42ff5519-19ff-44c7-8940-a8f3541d719a at http://172.21.0.4:5555. Health check every 120s
gogoanime-scrapper-chrome-1  | 14:11:05.051 INFO [NodeServer.lambda$createHandlers$2] - Node has been added
selenium-hub                 | 14:11:05.064 INFO [GridModel.setAvailability] - Switching node 42ff5519-19ff-44c7-8940-a8f3541d719a (uri: http://172.21.0.4:5555) from DOWN to UP
gogoanime-scrapper-web-1     | updated!
gogoanime-scrapper-web-1     | thread 'actix-rt|system:0|arbiter:0' panicked at 'called `Result::unwrap()` on an `Err` value: HttpError(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Domain("localhost")), port: Some(4444), path: "/wd/hub/session", query: None, fragment: None }, source: hyper::Error(Connect, ConnectError("tcp connect error", Os { code: 99, kind: AddrNotAvailable, message: "Cannot assign requested address" })) })', src/main.rs:139:6
gogoanime-scrapper-web-1     | note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I am very sorry for the long issue. I hope to hear from your side as soon as possible. Please do leave a message if there's any questions.

For my side note, I think it has something to do with localhost link as it cannot assign to the port. I tested the selenium server using a normal test script (without dockering) and it works just fine.

Thank You Very Much :)

Answer 1 · 2022-03-17T02:27:50.000Z

After some reading and watching videos, the solution is very simple. change the localhost to the name of the container that we trying to connect.

    let driver = WebDriver::new("http://selenium-hub:4444/wd/hub", &caps)
    .await
    .unwrap();