If you want to set up your own pipeline in BigQuery.
- Create a new google cloud project
- Create a new bigquery dataset
- Amend the script
create-db.sql
to point to your datasets and project - Run the script
create-db.sql
to create the tables and views required - Run the other scripts to generate the stored procedures to do the transformations
- To process a year of data (say 2021) run
CALL rw-algotrader.equity_statarb.load_features(2021);