jeanphix/Ghost.py

Javascript not executed on Facebook

Opened this issue · 0 comments

Hi, i am currently use ghost.py to write a mini crawler on facebook.

The problem is that the page doesn't execute any of it's script

  • Python 2.7
  • Ghost.py latest

Homepage:
image

Friend:
image

This is my cmd log: I have tried install TimeSlice and BigPipe, no change.
image

The code:

from ghost import Ghost, Session
from bs4 import BeautifulSoup
ghost = Ghost()

with ghost.start():
    session = Session(ghost, user_agent='Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.121 Safari/535.2',
    wait_timeout=8, wait_callback=None, display=True, viewport_size=(800, 600),
    ignore_ssl_errors=True, plugins_enabled=True, java_enabled=True, javascript_enabled=True,
    download_images=True, show_scrollbars=True)
    session.wait_timeout = 999

    # This a a cookie i generated from my login_facebook script
    session.load_cookies('fbookie')

    # Some random facebook id
    page, extra_resources = session.open("https://www.facebook.com/100017894403983")
    
    # Just some test with beautifulsoup
    result, extra = session.evaluate('document.title;')
    
    print('The title is: {}'.format(result))
    if page.http_status == 200:
        soup = BeautifulSoup(session.content, 'lxml')

    soup.find_all("p", "title")

    for a in soup.findAll('a'):
        print a['href']

I am still unsure whether this is Ghost.py's fault or something else.