Subseasonal Forecasting Toolkit
The subseasonal_toolkit package provides implementations of the subseasonal forecasting ABC model of
Adaptive Bias Correction for Subseasonal Forecasting
Soukayna Mouatadid, Paulo Orenstein, Genevieve Flaspohler, Judah Cohen, Miruna Oprescu, Ernest Fraenkel, and Lester Mackey. Nature Communications. June 2023.
@article{mouatadid2023adaptive,
author = {Mouatadid, Soukayna and Orenstein, Paulo and Flaspohler, Genevieve and Cohen, Judah and Oprescu, Miruna and Fraenkel, Ernest and Mackey, Lester},
journal = {Nature Communications},
number = {1},
pages = {3482},
title = {Adaptive bias correction for improved subseasonal forecasting},
volume = {14},
year = {2023}}
and the machine learning models and meteorological baselines of
SubseasonalClimateUSA: A Dataset for Subseasonal Forecasting and Benchmarking
Soukayna Mouatadid, Paulo Orenstein, Genevieve Flaspohler, Miruna Oprescu, Judah Cohen, Franklyn Wang, Sean Knight, Maria Geogdzhayeva, Sam Levang, Ernest Fraenkel, and Lester Mackey. Advances in Neural Information Processing Systems (NeurIPS). Dec. 2023.
@inproceedings{mouatadid2023toolkit,
author = {Soukayna Mouatadid, Paulo Orenstein, Genevieve Flaspohler, Miruna Oprescu, Judah Cohen, Franklyn Wang, Sean Knight, Maria Geogdzhayeva, Sam Levang, Ernest Fraenkel, and Lester Mackey},
booktitle = {Advances in Neural Information Processing Systems},
editor = {A. Globerson and K. Saenko and M. Hardt and S. Levine},
publisher = {Curran Associates, Inc.},
title = {SubseasonalClimateUSA: A Dataset for Subseasonal Forecasting and Benchmarking},
volume = {36},
year = {2023}
}
System Requirements
This package has been tested with the following operating system and Python pairings:
- macOS Monterey 12.6.3 with Python 3.9.12
- Linux CentOS 7 with Python 3.7.9
A complete list of Python dependencies can be found in setup.cfg; these dependencies are required upon installation.
Getting Started
- Install the subseasonal toolkit package:
pip install subseasonal-toolkit- Installation completed in under 1 minute with pip 22.2.2 on a 2021 MacBook Pro with 16 GB of RAM running macOS Monterey version 12.6.3.
- Define the environment variable
$SUBSEASONALDATA_PATHto point to your desired data directory; any data files needed by a model will be read from, saved to, or synced with this directory - Run the following demo which generates and evaluates Raw CFSv2 precipitation forecasts across the contiguous U.S. for the 2018-2021
std_paper_forecastevaluation period of "Adaptive Bias Correction for Subseasonal Forecasting":python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -u -e -m raw_cfsv2 --task us_tmp2m_1.5x1.5_34w- This demo ran to completion in 7 minutes with Python 3.9.12 on a 2021 MacBook Pro with 16 GB of RAM running macOS Monterey version 12.6.3.
- Expected outputs
- A forecast folder
models/cfsv2pp/submodel_forecasts/cfsv2pp-debiasFalse_years12_margin0_days1-1_leads15-15_lossmse/us_tmp2m_1.5x1.5_34w/containing daily forecast files from 20180101 through 20211231 - A metrics folder
eval/metrics/raw_cfsv2/submodel_forecasts/cfsv2pp-debiasFalse_years12_margin0_days1-1_leads15-15_lossmse/us_tmp2m_1.5x1.5_34w/containing 6 evaluation metrics:lat_lon_error-us_tmp2m_1.5x1.5_34w-std_paper_forecast.h5lat_lon_rmse-us_tmp2m_1.5x1.5_34w-std_paper_forecast.h5lat_lon_skill-us_tmp2m_1.5x1.5_34w-std_paper_forecast.h5rmse-us_tmp2m_1.5x1.5_34w-std_paper_forecast.h5score-us_tmp2m_1.5x1.5_34w-std_paper_forecast.h5skill-us_tmp2m_1.5x1.5_34w-std_paper_forecast.h5
- A forecast folder
Generating Model Forecasts
The following examples demonstrate how to generate contiguous US forecasts for the target dates evaluated in "Adaptive Bias Correction for Subseasonal Forecasting" or "SubseasonalClimateUSA: A Dataset for Subseasonal Forecasting and Benchmarking" using each implemented model.
- ABC-CCSM4:
- Generate predictions for each Climatology++ model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -e -u -b -m climpp - Generate predictions for each CCSM4++ model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -e -u -b -m ccsm4pp - Run ABC
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -a -m ccsm4
- Generate predictions for each Climatology++ model configuration
- ABC-CFSv2:
- Generate predictions for each Climatology++ model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -e -u -b -m climpp - Generate predictions for each CFSv2++ model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -e -u -b -m cfsv2pp - Run ABC
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -a -m cfsv2
- Generate predictions for each Climatology++ model configuration
- ABC-ECMWF:
- Generate predictions for each Climatology++ model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -e -u -b -m climpp - Generate predictions for each ECMWF++ model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -e -u -b -m ecmwfpp - Run ABC
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -a -m ecmwf
- Generate predictions for each Climatology++ model configuration
- ABC-FIMr1p1:
- Generate predictions for each Climatology++ model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -e -u -b -m climpp - Generate predictions for each FIMr1p1++ model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -e -u -b -m fimr1p1pp - Run ABC
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -a -m fimr1p1
- Generate predictions for each Climatology++ model configuration
- ABC-GEFS:
- Generate predictions for each Climatology++ model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -e -u -b -m climpp - Generate predictions for each GEFS++ model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -e -u -b -m gefspp - Run ABC
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -a -m gefs
- Generate predictions for each Climatology++ model configuration
- ABC-GEMS:
- Generate predictions for each Climatology++ model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -e -u -b -m climpp - Generate predictions for each GEMS++ model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -e -u -b -m gemspp - Run ABC
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -a -m gems
- Generate predictions for each Climatology++ model configuration
- ABC-GEOS_v2p1:
- Generate predictions for each Climatology++ model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -e -u -b -m climpp - Generate predictions for each GEOS_v2p1++ model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -e -u -b -m geos_v2p1pp - Run ABC
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -a -m geos_v2p1
- Generate predictions for each Climatology++ model configuration
- ABC-NESM:
- Generate predictions for each Climatology++ model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -e -u -b -m climpp - Generate predictions for each NESM++ model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -e -u -b -m nesmpp - Run ABC
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -a -m nesm
- Generate predictions for each Climatology++ model configuration
- ABC-SubX:
- Generate predictions for each Climatology++ model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -e -u -b -m climpp - Generate predictions for each SubX++ model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -e -u -b -m subx_meanpp - Run ABC
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -a -m subx_mean
- Generate predictions for each Climatology++ model configuration
- AutoKNN:
python -m subseasonal_toolkit.generate_predictions -t std_paper -u -m autoknn - CCSM4++:
- First generate predictions for each model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -e -u -b -m ccsm4pp - Then select a model configuration using the tuner
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -tu -m ccsm4pp
- First generate predictions for each model configuration
- CFSv2++:
- First generate predictions for each model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -e -u -b -m cfsv2pp - Then select a model configuration using the tuner
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -tu -m cfsv2pp
- First generate predictions for each model configuration
- Climatology:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -m climatology - Climatology++:
- First generate predictions for each model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -e -u -b -m climpp - Then select a model configuration using the tuner
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -tu -m climpp
- First generate predictions for each model configuration
- Debiased CFSv2:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -m deb_cfsv2 - Debiased ECMWF Control and Ensemble:
- First, select your desired source (control or ensemble) for debiasing and forecasting in
subseasonal_toolkit/models/deb_ecmwf/selected_submodel.jsonby setting theforecast_withanddebias_withkeys as described indeb_ecmwf.ipynb. - Then, run the selected model:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -m deb_ecmwf
- First, select your desired source (control or ensemble) for debiasing and forecasting in
- Debiased LOESS CFSv2:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -m deb_loess_cfsv2 - Debiased LOESS ECMWF:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -m deb_loess_ecmwf - Debiased Quantile Mapping CFSv2:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -m deb_quantile_cfsv2 - Debiased Quantile Mapping ECMWF:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -m deb_quantile_ecmwf - Debiased SubX:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -m deb_subx_mean - ECMWF++:
- First generate predictions for each model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -e -u -b -m ecmwfpp - Then select a model configuration using the tuner
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -tu -m ecmwfpp
- First generate predictions for each model configuration
- FIMr1p1++:
- First generate predictions for each model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -e -u -b -m fimr1p1pp - Then select a model configuration using the tuner
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -tu -m fimr1p1pp
- First generate predictions for each model configuration
- GEFS++:
- First generate predictions for each model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -e -u -b -m gefspp - Then select a model configuration using the tuner
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -tu -m gefspp
- First generate predictions for each model configuration
- GEM++:
- First generate predictions for each model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -e -u -b -m gempp - Then select a model configuration using the tuner
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -tu -m gempp
- First generate predictions for each model configuration
- GEOS++:
- First generate predictions for each model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -e -u -b -m geos_v2p1pp - Then select a model configuration using the tuner
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -tu -m geos_v2p1pp
- First generate predictions for each model configuration
- Informer:
python -m subseasonal_toolkit.generate_predictions -t std_paper -u -m informer - LocalBoosting:
- First generate predictions for each model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -u -b -m localboosting - Then select a model configuration using the tuner
python -m subseasonal_toolkit.generate_predictions -t std_paper -u -tu -m localboosting
- First generate predictions for each model configuration
- MultiLLR:
python -m subseasonal_toolkit.generate_predictions -t std_paper -u -m multillr - N-BEATS:
python -m subseasonal_toolkit.generate_predictions -t std_paper -u -m nbeats - NN-A:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -m nn-a - Online Ensemble:
python -m subseasonal_toolkit.generate_predictions -t std_paper -u -m online_learning - Persistence:
python -m subseasonal_toolkit.generate_predictions -t std_paper -u -m persistence - Persistence++ CCSM4:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -u -e -m perpp_ccsm4 - Persistence++ CFSv2:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -u -e -m perpp_cfsv2 - Persistence++ ECMWF:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -u -e -m perpp_ecmwf - Persistence++ FIMr1p1:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -m perpp_fimr1p1 -u -e - Persistence++ GEFS:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -m perpp_gefs -u -e - Persistence++ GEM:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -m perpp_gem -u -e - Persistence++ GEOS_v2p1:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -m perpp_geos_v2p1 -u -e - Persistence++ NESM:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -m perpp_nesm -u -e - Persistence++ SubX:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -m perpp_subx_mean -u -e - Prophet:
python -m subseasonal_toolkit.generate_predictions -t std_paper -u -m prophet - Raw CCSM4:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -u -e -m raw_ccsm4 - Raw CFSv2:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -u -e -m raw_cfsv2 - Raw ECMWF:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -u -e -m raw_ecmwf - Raw FIMr1p1:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -m raw_fimr1p1 -u -e - Raw GEFS:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -m raw_gefs -u -e - Raw GEM:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -m raw_gem -u -e - Raw GEOS_v2p1:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -m raw_geos_v2p1 -u -e - Raw NESM:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -m raw_nesm -u -e - Raw SubX:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -m raw_subx_mean -u -e - Salient2:
- First generate predictions for each model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -u -b -m salient2 - Then select a model configuration using the tuner
python -m subseasonal_toolkit.generate_predictions -t std_paper -u -tu -m salient2
- First generate predictions for each model configuration
- SubX++:
- First generate predictions for each model configuration
python -m subseasonal_toolkit.generate_predictions -t std_paper_eval -e -u -b -m subx_meanpp - Then select a model configuration using the tuner
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -tu -m subx_meanpp
- First generate predictions for each model configuration
- Uniform Ensemble:
python -m subseasonal_toolkit.generate_predictions -t std_paper_forecast -e -u -m linear_ensemble
For Developers
Installation
After cloning this repository, install from source in editable mode using pip install -e . in this directory or pip install -e path/to/directory from another directory.
Contributing
This project welcomes contributions and suggestions. Most contributions require you to agree to a Contributor License Agreement (CLA) declaring that you have the right to, and actually do, grant us the rights to use your contribution. For details, visit https://cla.opensource.microsoft.com.
When you submit a pull request, a CLA bot will automatically determine whether you need to provide a CLA and decorate the PR appropriately (e.g., status check, comment). Simply follow the instructions provided by the bot. You will only need to do this once across all repos using our CLA.
This project has adopted the Microsoft Open Source Code of Conduct. For more information see the Code of Conduct FAQ or contact opencode@microsoft.com with any additional questions or comments.
Trademarks
This project may contain trademarks or logos for projects, products, or services. Authorized use of Microsoft trademarks or logos is subject to and must follow Microsoft's Trademark & Brand Guidelines. Use of Microsoft trademarks or logos in modified versions of this project must not cause confusion or imply Microsoft sponsorship. Any use of third-party trademarks or logos are subject to those third-party's policies.