/web-scraping

The process includes steps from data collection (web scraping), data processing with PySpark, to process management with Apache Airflow. You can expand this project by adding more complex data processing tasks or deploying the process on different schedules through Airflow.

Primary LanguagePython

Stargazers