/auma_scraper

Primary LanguagePython

Steps to run the crawler locally:

1. Dependencies:

To be able to run crawler locally you have to meet following dependencies:

  • python3, pip3 and scrapy framework

1. Install App Requirements:

  • Switch to the project root directory.
  • run $ pip3 install -r requirements.txt

4. Run the crawler

  • Switch to project root directory.
  • run command to run the script python3 run_scraper.py -u <category url>
  • example run
    • python3 run_scraper.py -u https://apps.apple.com/us/genre/ios-books/id6018