Repository for all the scripts about a data engineering project involving an ELT process extracting data from the Twitter API (querying for tweets referencing @AluraOnline) using Apache Airflow (v2.3.2), loading to a datalake and transforming the data to make it structured using Apache Spark (PySpark).
rd-coutinho/ELT_Twitter_API
Data engineering project about an ELT process using Apache Airflow and Apache Spark.
Python