/common-etl

This repo used for keep my common pythonic way on ETL especially to automate on Airflow and BigQuery as target storage

Primary LanguagePython

Common ETL Python

This repo represent the way i use python for ETL purpose. I'm getting used on BigQuery and Postgresql tech. But it's open for other tech stack as well. Reference on Apache Airflow to check any feature that can be used. The transorm section usually us pandas.

Prerequisites

  1. Python
  2. SQL
  3. Pandas
  4. Apache Airflow