Usage:
Create virtual python environment:
python3 -m venv myenv
source myenv/bin/activate
Run the crawler to fetch JSON-LD data. Edit crawl.py to place the URL.
pip3 install -m requirements.txt
python3 crawl.py
Run deactivate to stop the python virtual environment.
- JSON-LD scraper based on ScraperHub: https://github.com/hackersandslackers/jsonld-scraper-tutorial
- Google tool for crawling and exposing structured data: https://search.google.com/test/rich-results
- Scraping using Selenium: https://www.toptal.com/python/web-scraping-with-python
- ScrapingHub project: https://github.com/scrapinghub