/assessment-test-de

Primary LanguagePythonMIT LicenseMIT

Assessment

Setup Airflow

By following these instructions. Using Airflow 2.4.1

Or you can simply run docker-compose up airflow-init then docker-compose up -d after cloning the project and create .env that mentioned in the docs.

Restore Chinook Tables and HR Tables

Chinook (PostgreSQL)

psql -h <hostname> -U <username> -d <database-name> -f ~/path/to/chinook_pg.sql

Human Resource (MySQL)

mysql -u <username> --host=<hostname> --port=<port> <dbname> -p < ~/path/to/human_resources_mysql.sql

Task 1

Data Modelling

data-modelling-task-1 data-modelling-task-1.2

DAG

dag-task-1 dag-task-1.2

Task 2

DAG

data-modelling-task-2

Tables

tables-task-2

Task 3

Setup Great Expectations here.

The project is stored at include/great_expectations.

DAG

To copy/upload local files stored at data/* to GCS and create table from those GCS files. dag-task-3

Expectations

Inventory

inventory

Products

products

Sales

sales

Stores

stores

Checkpoint Result Samples

checkpoint-1 checkpoint-2 checkppint-3