multiprocessing with dryscrape
Closed this issue · 0 comments
deter3 commented
This is not an issue , but sharing how to deal with xvfb .
I am using celery to run multiple sessions of webkit-server . I need to remove ' dryscrape.start_xvfb() ' in the code and put xvfb-run before celery command , like this
xvfb-run -a celery -A googlescrape worker --loglevel=INFO --concurrency=10 -Q googlescrape_task
I spent couple of hours to figure out and hope can help somebody to save the time .