program crashes somewhere when the browser is launched and chromium starts to consume very high cpu
Closed this issue · 6 comments
orphan chromium processes still remain active and consume cpu as if the browser isn't closed.
workaround for now:
- restart the container
- cron schedule to pkill all chromium after the bot runs
update: when this happens, the bot doesn't post anything on discord.
possible source of error could be these lines since:
codezinger-due-date-scraper/scrap_data.py
Lines 173 to 176 in 7401be2
since they're not under try/except
relevant log files:
red is the last log and since when the chromium high cpu usage is started
green is when i manually restarted the container
possible workarounds:
- if opening chrome or that process takes more than 90seconds, kill it and restart
(and/or set a cpu limit in compose) - add the
--disable-gpu
param - downgrade to python 3.9
- install all puppeteer's dependencies
exact LOC where the freezing happens (confirmed via logs):
codezinger-due-date-scraper/scrap_data.py
Line 176 in 7401be2
Logs when the program freezes:
crond: USER root pid 9 cmd /entrypoint.sh
codezinger-due-date-scraper Copyright (C) 2022 RoguedBear, Ya-s-h
This program comes with ABSOLUTELY NO WARRANTY; see COPYING
This is free software, and you are welcome to redistribute it
under certain conditions; see COPYING
loaded short names
Starting browser...
browser started
Logs when the program runs normally
crond: USER root pid 9 cmd /entrypoint.sh
codezinger-due-date-scraper Copyright (C) 2022 RoguedBear, Ya-s-h
This program comes with ABSOLUTELY NO WARRANTY; see COPYING
This is free software, and you are welcome to redistribute it
under certain conditions; see COPYING
loaded short names
Starting browser...
browser started
logging in.
Logged in. allegedly
sorting
loading more
Processing stuff
[<pyppeteer.element_handle.ElementHandle object at 0x7f1d26000640>, <pyppeteer.element_handle.ElementHandle object at 0x7f1d26001030>]
2
Scraped 2 questions(50%)
awaiting browser close
Browser closed
Processing new events...
Sent 0 new webhook messages
Finished, exiting...
running the program for 2 weeks, commit a64681c does seem to have fixed the issue