This project is a collection of web scrapers designed to extract job postings from popular job search websites such as Indeed, SimplyHired, and Google. The scrapers use Selenium for web page automation and Beautiful Soup for parsing HTML content.
- Halleluyah Darasimi Oludele
To set up this project on your local machine, follow these steps:
-
Clone the repository to your local machine.
-
Install the required dependencies using pipenv:
-
Run the scrapers using the provided scripts.
To use the scrapers, execute the following commands:
-
For Indeed scraper:
python scrape_indeed.py -
For SimplyHired scraper:
python scrape_simplyhired.py -
There are jupyter notebooks available for the scraping and you can run your code directly in them
- Selenium: Python library for web automation.
- Beautiful Soup: Python library for parsing HTML and XML documents.
This project is licensed under the MIT License - see the LICENSE file for details.