This repository contains the task for the secret scraping dojo. The main objective was to ensure the following:
-
Modularity and Object-Oriented Code: The code has been written in a modular and object-oriented manner, with clearly defined objects, their properties, and methods.
-
Clean Code: Efforts have been made to write clean, understandable code with appropriate function and object method names.
-
Proxy Usage: The provided proxy has been used for scraping in the correct manner, as specified in the .env file. If the proxy does not work, there is an option to proceed without it.
-
Reliability: The scraping solution has been designed to closely imitate browser behavior to ensure reliability. Different methods are available for scraping, and the ones that mimic browser behavior are generally preferred.
-
Output File: The output file generated contains the correct results. Note that there are several pages of quotes, and all of them have been scraped.
The task started off easy but required some refinement over time. Enjoy exploring!