webfp/tor-browser-selenium

Constant high CPU usage

Opened this issue · 7 comments

TIgNoCkA commented

Imminently after initiating a TorBrowserDriver instance and without doing anything it consumes ~40% of my CPU. It doesn't happen with vanilla selenium with Geckodriver or when I start Tor-browser with --marionate from the shell. Any Idea what could potentially cause it or how to figure this out?

driver = TorBrowserDriver(...)
sleep(...) # CPU > 40%
driver.quit() # back to normal
gunesacar commented

Thanks for the report, @TIgNoCkA . Do you know which specific process is consuming the CPU? Is it the firefox process, geckodriver, or tor?

TIgNoCkA commented

It's firefox.real.

I attached strace to it and got this:

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ------------------
 39.12   12.415468           4   3004544   3004469 recvmsg
 14.10    4.474290           2   1522566           poll
 13.95    4.426782           2   1519226           ioctl
 10.09    3.203883           4    760547           write
  8.14    2.584054           3    760401           read
  7.59    2.409362           4    512918       707 futex
  6.96    2.208893           2    759613           recvfrom
  • 👍1
gunesacar commented

Thanks, that's super useful. Over 3 million recvmsg errors. That's curious.

Let me try to reproduce this locally. Do you have a command at hand to launch Tor browser with strace attached, or did you attach it after launching Tor Browser?

TIgNoCkA commented

I did strace -cp pid after lunching with TorBrowserDriver(). I also ran firefox.real directly and did the same:

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ------------------
 45.71    0.355209          99      3567           write
 23.72    0.184319          25      7180         1 poll
 10.57    0.082136           6     13383      1071 futex
  5.97    0.046428       23214         2           wait4
  5.25    0.040767           4      8825      7951 recvmsg

Lower yet still weirdly high rate of errors/calls for recvmsg. Anyway, it doesn't call it as often.

gunesacar commented

Thanks for the info @TIgNoCkA. I wonder if excessive logging by geckodriver could be the culprit here. I'll come back to this after updating the library to work with Selenium 4.

  • 👍1
t0m4s commented

Hi,

I've got the same issue.
From what I've seen, a lot of users seem to experiment this problem using tbselenium, selenium or even firefox.

It seems to be related with the firefox multi-threading system called Electrolysis or e10s.

You can disable it by doing so :
os.environ['MOZ_FORCE_DISABLE_E10S'] = '1'

I'm running my scripts this way since a few hours and it really seems to solve the problem. I'll come back here in a day or 2 to confirm if it's ok.

t0m4s commented

After some more testing :
I found no solution to deactivate multiprocessing.
browser.tabs.remote.autostart = False has been deactivated by mozilla ( https://www.ghacks.net/2019/05/17/going-forward-multi-process-cant-be-turned-off-anymore-in-firefox/ )
MOZ_FORCE_DISABLE_E10S=1 seems to have no effect.

Sometimes CPU consumption is ok, most of the time it takes 20~30% of the CPU.