By following these instructions. Using Airflow 2.4.1
Or you can simply run docker-compose up airflow-init
then docker-compose up -d
after cloning the project and create .env
that mentioned in the docs.
Chinook (PostgreSQL)
psql -h <hostname> -U <username> -d <database-name> -f ~/path/to/chinook_pg.sql
Human Resource (MySQL)
mysql -u <username> --host=<hostname> --port=<port> <dbname> -p < ~/path/to/human_resources_mysql.sql
Setup Great Expectations here.
The project is stored at include/great_expectations
.
To copy/upload local files stored at data/*
to GCS and create table from those GCS files.