/Web-Scraper

Build a web scraper using Python's requests and BeautifulSoup libraries. The scraper can extract data from a website of your choice, such as headlines, product information, or weather data, and save it to a local file or database. This project will give you hands-on experience with web requests, HTML parsing, and data extraction.

Primary LanguagePythonMIT LicenseMIT

ShutterScrape

ShutterScrape is a web scrapper for bulk downloading images and videos from Shutterstock with speed. ⚡ It implements Selenium for browser automation and Beautiful Soup for parsing.

Setting up

Configure shutterscrape.py to your Python version.

Install requirements from Terminal:

pip install beautifulsoup4 pip install selenium pip install lxml Install ChromeDriver.

(Optional) Configure environment variables paths for python.exe and chromedriver.exe.

Running

Open terminal in the directory of shutterscrape.py and enter:

python shutterscrape.py Go grab a cup of coffee while waiting... oh wait, it's already done!

Definitions

Search mode: Enter i for scraping images and v for scraping videos . Number of search terms: For example, if you want to search for drone single person, enter 3. Search term: Keyword(s) for searching on Shutterstock. Number of pages to scrape: Higher number of pages means greater quantity of content with lower keyword precision.

Updates

10/1/2020 Updated for new shutterstock page layout as of 10/1/2020.

4/26/2019 Updated for new shutterstock page layout as of 4/26/2019.

10/1/2018 Added GUI for save directory selection.

07/31/2018 More stability fixes.

07/25/2018 Added gettyscrape.py for scraping videos from Getty Images.

07/23/2018 Stability fixes.

Web-Scraper

Build a web scraper using Python's requests and BeautifulSoup libraries. The scraper can extract data from a website of your choice, such as headlines, product information, or weather data, and save it to a local file or database. This project will give you hands-on experience with web requests, HTML parsing, and data extraction.