Javascript not executed on Facebook
Opened this issue · 0 comments
1453064 commented
Hi, i am currently use ghost.py to write a mini crawler on facebook.
The problem is that the page doesn't execute any of it's script
- Python 2.7
- Ghost.py latest
This is my cmd log: I have tried install TimeSlice and BigPipe, no change.
The code:
from ghost import Ghost, Session
from bs4 import BeautifulSoup
ghost = Ghost()
with ghost.start():
session = Session(ghost, user_agent='Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2',
wait_timeout=8, wait_callback=None, display=True, viewport_size=(800, 600),
ignore_ssl_errors=True, plugins_enabled=True, java_enabled=True, javascript_enabled=True,
download_images=True, show_scrollbars=True)
session.wait_timeout = 999
# This a a cookie i generated from my login_facebook script
session.load_cookies('fbookie')
# Some random facebook id
page, extra_resources = session.open("https://www.facebook.com/100017894403983")
# Just some test with beautifulsoup
result, extra = session.evaluate('document.title;')
print('The title is: {}'.format(result))
if page.http_status == 200:
soup = BeautifulSoup(session.content, 'lxml')
soup.find_all("p", "title")
for a in soup.findAll('a'):
print a['href']
I am still unsure whether this is Ghost.py's fault or something else.