Sub-Seasonal Climate Forecasting via Machine Learning and Transfer Learning
Code is adapted from He et al. Sub-Seasonal Climate Forecasting via Machine Learning: Challenges, Analysis, and Advances. Modified to support the CESM2 Forecast and Reforecast dataset and Transfer Learning. Work is part of my 2022-2023 Senior Thesis.
Requirements
The code is compatible with Python 3.6 and the following packages:
- numpy: 1.19.0
- pandas: 0.24.2
- joblib: 0.15.1
- pickle: 4.0
- scipy: 1.5.0
- pytorch: 1.2.0
- sklearn: 0.23.1
- xgboost: 1.0.2
- pytables: 3.5.2
Getting started
- Clone the repo
- Create virtual environments and install the necessary python packages listed above
- Set up raw datasets
- SSF: Load raw data from the google drive folder (https://drive.google.com/drive/folders/1qeDOWDULW-F8HnLYPEzWe8tCdtF4xQYT?usp=sharing) and save it to the folder named "data/SSF"
- CESM: Copy data from {} and save it to the folder name "data/CESM"
- Revise configutation files to adapt to the required settings
- cfg_target.py
- cfg_SSF.py
- cfg_CESM.py
- Data loading and preprocessing:
- Execute load_data.py to load the subset of data needed in generating forecasts
- Execute run_preprocess.py to preprocess covariates and target variable separately
- Alternatively, one can execute run_preprocess_map.py to zscore covariates and convert them to squared maps
- Execute create_covariates_pca.py to concatenate data
- Execute create_datasets.py to create training-validation sets and training-test sets
- Hyperparameter tuning: execute run_random_search.py to find the best parameter by random search
- Generate forecasts: execute main_experiments.py to train all the models and generate forecasts on test sets
- Evaluate forecasts: execute run_evaluation.py to evaluate the forecasting performance on both training and test sets
Project Structure
Scripts for generating forecasts
- cfg_target.py: configure file with all parameters from users
- load_data: script for a subset of data required by configure file (cfg_target.py)
- run_preprocess.py: script for data preprocessing
- run_preprocess_map.py: script for preprocessing covariates and converting them to squared maps
- create_covariates_pca.py: script for concatenating PCs from all climate variables (covariates)
- create_datasets.py: script for creating training-validation sets and training-test sets
- run_random_search.py: script for hyperparameter tuning via random search
- main_experiments: script for running experiments for all models on training and test sets
- run_evaluation.py: script for evaluate the performance of forecasting models on training and test sets
Functions
- data_load: folder contains functions for data loading
- preprocess: folder contains functions for data preprocessing
- hyperparameter_tuning: folder contains functions for random search
- forecasting: folder contains scripts for training forecasting models and evaluating on test sets
- evaluation: folder contains functions for evaluation
- model: collection of models implemented in experiments
- utils:utility functions