openwpm/OpenWPM

Browser Manager, Spawn unsuccessful on Azure ubuntu 20.04 VM

Closed this issue · 4 comments

tim-stephenson commented

The way I am setting up openwpm is summarized by this script:

sudo apt update
sudo apt upgrade
sudo apt install build-essential
sudo apt install git
sudo apt install manpages-dev
sudo apt install wget
sudo apt install make
sudo apt install apt-utils
sudo apt install curl
sudo apt install libx11-xcb1 
sudo apt install packagekit-gtk3-module

wget https://get.docker.com -O - | sudo sh -

wget https://github.com/openwpm/OpenWPM/archive/refs/tags/v0.21.1.tar.gz -O - | tar -xzf -
mv OpenWPM-0.21.1 OpenWPM

cd OpenWPM
sudo docker build -t openwpm .

After changing display_mode in BrowserParams to display_mode="headless" in demo.py, I try running the demo.py via:

sudo docker run openwpm python demo.py

Which gets the following console output:

Executing <Task pending name='Task-1' coro=<StorageController._run() running at /opt/OpenWPM/openwpm/storage/storage_controller.py:329> wait_for=<_GatheringFuture pending cb=[Task.task_wakeup()] created at /opt/miniconda/envs/openwpm/lib/python3.10/asyncio/tasks.py:665> cb=[_run_until_complete_cb() at /opt/miniconda/envs/openwpm/lib/python3.10/asyncio/base_events.py:184] created at /opt/miniconda/envs/openwpm/lib/python3.10/asyncio/tasks.py:636> took 0.192 seconds
browser_manager      - INFO     - BROWSER 2184141122: Launching browser...
browser_manager      - ERROR    - BROWSER 2184141122: Crash in driver, restarting browser manager
Traceback (most recent call last):
  File "/opt/OpenWPM/openwpm/browser_manager.py", line 716, in run
    driver, browser_profile_path, display = deploy_firefox.deploy_firefox(
  File "/opt/OpenWPM/openwpm/deploy_browsers/deploy_firefox.py", line 140, in deploy_firefox
    driver = webdriver.Firefox(
  File "/opt/miniconda/envs/openwpm/lib/python3.10/site-packages/selenium/webdriver/firefox/webdriver.py", line 177, in __init__
    super().__init__(
  File "/opt/miniconda/envs/openwpm/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 272, in __init__
    self.start_session(capabilities, browser_profile)
  File "/opt/miniconda/envs/openwpm/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 364, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/opt/miniconda/envs/openwpm/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 429, in execute
    self.error_handler.check_response(response)
  File "/opt/miniconda/envs/openwpm/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py", line 243, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Process unexpectedly closed with status 1

browser_manager      - ERROR    - BROWSER 2184141122: Spawn unsuccessful  | Profile Created: True  | Profile Tar: True  | Display: True  | Launch Attempted: True  | Browser Launched: False  | Browser Ready: False
browser_manager      - ERROR    - BROWSER 2184141122: Crash in driver, restarting browser manager
Traceback (most recent call last):
  File "/opt/OpenWPM/openwpm/browser_manager.py", line 716, in run
    driver, browser_profile_path, display = deploy_firefox.deploy_firefox(
  File "/opt/OpenWPM/openwpm/deploy_browsers/deploy_firefox.py", line 140, in deploy_firefox
    driver = webdriver.Firefox(
  File "/opt/miniconda/envs/openwpm/lib/python3.10/site-packages/selenium/webdriver/firefox/webdriver.py", line 177, in __init__
    super().__init__(
  File "/opt/miniconda/envs/openwpm/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 272, in __init__
    self.start_session(capabilities, browser_profile)
  File "/opt/miniconda/envs/openwpm/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 364, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/opt/miniconda/envs/openwpm/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 429, in execute
    self.error_handler.check_response(response)
  File "/opt/miniconda/envs/openwpm/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py", line 243, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Process unexpectedly closed with status 1

browser_manager      - ERROR    - BROWSER 2184141122: Spawn unsuccessful  | Profile Created: True  | Profile Tar: True  | Display: True  | Launch Attempted: True  | Browser Launched: False  | Browser Ready: False
browser_manager      - ERROR    - BROWSER 2184141122: Crash in driver, restarting browser manager
Traceback (most recent call last):
  File "/opt/OpenWPM/openwpm/browser_manager.py", line 716, in run
    driver, browser_profile_path, display = deploy_firefox.deploy_firefox(
  File "/opt/OpenWPM/openwpm/deploy_browsers/deploy_firefox.py", line 140, in deploy_firefox
    driver = webdriver.Firefox(
  File "/opt/miniconda/envs/openwpm/lib/python3.10/site-packages/selenium/webdriver/firefox/webdriver.py", line 177, in __init__
    super().__init__(
  File "/opt/miniconda/envs/openwpm/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 272, in __init__
    self.start_session(capabilities, browser_profile)
  File "/opt/miniconda/envs/openwpm/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 364, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/opt/miniconda/envs/openwpm/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 429, in execute
    self.error_handler.check_response(response)
  File "/opt/miniconda/envs/openwpm/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py", line 243, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Process unexpectedly closed with status 1

browser_manager      - ERROR    - BROWSER 2184141122: Spawn unsuccessful  | Profile Created: True  | Profile Tar: True  | Display: True  | Launch Attempted: True  | Browser Launched: False  | Browser Ready: False
browser_manager      - ERROR    - BROWSER 2184141122: Crash in driver, restarting browser manager
Traceback (most recent call last):
  File "/opt/OpenWPM/openwpm/browser_manager.py", line 716, in run
    driver, browser_profile_path, display = deploy_firefox.deploy_firefox(
  File "/opt/OpenWPM/openwpm/deploy_browsers/deploy_firefox.py", line 140, in deploy_firefox
    driver = webdriver.Firefox(
  File "/opt/miniconda/envs/openwpm/lib/python3.10/site-packages/selenium/webdriver/firefox/webdriver.py", line 177, in __init__
    super().__init__(
  File "/opt/miniconda/envs/openwpm/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 272, in __init__
    self.start_session(capabilities, browser_profile)
  File "/opt/miniconda/envs/openwpm/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 364, in start_session
    response = self.execute(Command.NEW_SESSION, parameters)
  File "/opt/miniconda/envs/openwpm/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 429, in execute
    self.error_handler.check_response(response)
  File "/opt/miniconda/envs/openwpm/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py", line 243, in check_response
    raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Process unexpectedly closed with status 1

browser_manager      - ERROR    - BROWSER 2184141122: Spawn unsuccessful  | Profile Created: True  | Profile Tar: True  | Display: True  | Launch Attempted: True  | Browser Launched: False  | Browser Ready: False
task_manager         - CRITICAL - Browser spawn failure during TaskManager initialization, exiting...
storage_controller   - INFO     - Terminating handler, because the underlying socket closed
storage_controller   - INFO     - Received shutdown signal!
fatal: not a git repository (or any of the parent directories): .git
Traceback (most recent call last):
  File "/opt/OpenWPM/demo.py", line 50, in <module>
    with TaskManager(
  File "/opt/OpenWPM/openwpm/task_manager.py", line 133, in __init__
Shutdown took 35.58046865463257 seconds
    self.storage_controller_handle.save_configuration(
  File "/opt/OpenWPM/openwpm/storage/storage_controller.py", line 448, in save_configuration
    sock = DataSocket(self.listener_address)
  File "/opt/OpenWPM/openwpm/storage/storage_controller.py", line 365, in __init__
    self.socket.connect(*listener_address)
  File "/opt/OpenWPM/openwpm/socket_interface.py", line 127, in connect
    self.sock.connect((host, port))
ConnectionRefusedError: [Errno 111] Connection refused

One of the concerns about what might be the issue is that in the browser_manager error message, it says Display: True:

browser_manager      - ERROR    - BROWSER 2184141122: Spawn unsuccessful  | Profile Created: True  | Profile Tar: True  | Display: True  | Launch Attempted: True  | Browser Launched: False  | Browser Ready: False

Whereas I am working in a VM without display capabilities.

englehardt commented

The error you're receiving means the browser is crashing on launch. This generally has to do with your environment setup and is often related to the display. Are you sure you've updated the browser configuration within the docker image, rather than local to the VM? I.e., did you make that config edit before building the docker image, or did you rebuild the image after making the change?

tim-stephenson commented

Yes, this was this issue. I am new to docker, I did not fully understand how the docker images worked

soumboussaha commented

How did you solve this issue ?

soumboussaha commented

I am facing the same issue on my end