/data-engineering-boilerplate

Boilerplate about data-engineering

Primary LanguageJupyter Notebook

data-engineering-boilerplate

Boilerplate about data-engineering

  • Spark DataFrame Manipulation
  • Airflow
    • DAG with some techniques
    • CustomOperator
  • Delta

TODO:

  1. Complete boto3.client('glue') to collect job runs