Local SERP

Google Search Engine Results Pages (SERP) in locally, no API key, no signup required

Make sure the chromedriver and required package are installed

Required packages

selenium
selenium-wire
beautifulsoup4
lxml

Then run to ensure everything is working

python -m serp.google

Example code

User must provide the search url as google have different forms of search parameters for different devices and portals.

By default it will force google to render results in english, if google decides to assign other languages some parsing will fail. This also affects the language which the knowledge graph is rendered, so for now this library try to normalize everything to english.

from serp.google import extract
import urllib

query = 'What is GDPR'

encode_query = urllib.parse.urlencode({'q': query, 'oq': query })
url = 'https://www.google.com/search?{}&aqs=chrome.0.69i59j0l8.940j0j9&sourceid=chrome&ie=UTF-8'.format(encode_query)
result = extract(query, url)

Supported features

Featured Snippet
Knowledge Graph extraction
- Accordion, Snippet, Attributes, Alternative search query, Images, Knowledge Graph Source
Spelling fix
Search stats : total results, time taken
Organic results
- title, link, snippet
Questions and Answers
- People also asked card
Related searches

Not supported features

Video, News, Images
Location spoofing : Chrome location mocking doesn't work for google search, maybe try mozilla driver instead?

Usage

Useful for OSINT, knowledge graph extraction, co-occurance query research, data mining, machine readable google results.

Other

This is part of the cause to build a better search engine