/TSF-Skeleton

Primary LanguagePython

TSF Skeleton

This project provides a highly customizable time-series forecasting (TSF) pipeline. It also includes a large collection of forecasting baseline models ranging from basic MLP, RNN to novel Transformer-based models and DLinear.

Why?

During our research on the Time-series forecasting topic, we find that the open-source repositories often adopts totally different pipeline, which takes a lot of time to read and debug just in order to run a baseline experiment on it. Recently, we find that a series of work prefer to develop on our repository LSTF-Linear. Thus, we make the TSF Skeleton based on it as a baseline repository and a easy to use pipeline.

Features

  • Baseline Results: We validate every models in this repository to make sure the results is similiar to the original implementation. You can also directly run these model with costomized experiment settings.
  • **Simple Pipeline: ** We keep the pipeline as concise as possible. So that you can make modification and realize your idea without too much learning cost.
  • Forecasting Models: A large collection of forecasting models; from transformer models (such as Informer) to deep learning models (such as RNN). See table of models below.
  • Multivariate Support: Datasets contain multiple time-varying dimensions instead of a single scalar value. Many models can consume and produce multivariate series.
  • Regression Models: It is possible to plug-in any scikit-learn compatible model to obtain forecasts as functions of lagged values of the target series and covariates.
  • Data processing: This repository has its own dataloader to preprocess the datasets.
  • Metrics: A variety of metrics for evaluating time series' goodness of fit; from RMSE to Mean Absolute Scaled Error.
  • Datasets In the dataset folder, we offer some popular time series datasets for rapid experimentation (such as ETT).
  • cfgs sysyem In this repository, we create a unique configuration file system. we have provided paremeters of all available datasets, models and expression files. All parameters initially are default. You can also change it according to your needs.
  • **Documentation (Working on): ** We provide detailed documentation about not only how to use, but also how to develop based on TSF Skeleton.

Forecasting Models

Here's a breakdown of the forecasting models in our current repository . We are constantly working on bringing more models and features.

Model Univariate Multivariate UseTimeFeature Reference
RNN ✅ ✅ DeepAR paper
SCINet ✅ ✅ SCINet paper
MTGNN ✅ ✅ ✅ MTGNN paper
InceptionTime ✅ ✅ InceptionTime paper
STGCN ✅ ✅ ✅ STGCN paper
FNN ✅ ✅ ✅ FNN paper
DCRNN ✅ ✅ ✅ DCRNN paper
Autoformer ✅ ✅ ✅ Autoformer paper
MLP ✅ ✅
gMLP ✅ ✅
ResNet ✅ ✅ ResNet paper
TCN ✅ ✅ TCN paper

Dataset

We conduct the experiments on 7 popular time-series datasets, namely Electricity Transformer Temperature (ETTh1, ETTh2 , ETTm1 and ETTm2) , and Solar-Energy, Electricity and Exchange Rate, from power, energy, finance and traffic domains.

Overall information of the 7 datasets

Datasets Variants Timesteps Granularity Start time Task Type
ETTh1 7 17,420 1hour 7/1/2016 Multi-step
ETTh2 7 17,420 1hour 7/1/2016 Multi-step
ETTm1 7 69,680 15min 7/1/2016 Multi-step
ETTm2 7 69,680 15min 7/1/2016 Multi-step
Solar-Energy 137 52,560 1hour 1/1/2006 Single-step
Electricity 321 26,304 1hour 1/1/2012 Single-step
Exchange-Rate 8 7,588 1hour 1/1/1990 Single-step

Parameter highlights

Parameter Name Description Default
path The root path of datasets N/A
dataset_name dataset N/A
horizon Horizon 24
lookback Look-back window 48
channel features of dataset 7
train_ratio part of dataset used for trainning 0.6
valid_ratio part of dataset used for validation 0.2
test_ratio part of dataset used for test 0.2
multivariate support multivariate true
freq Freq for time features encoding (defaults to h). This can be set to s,t,h,d,b,w,m (s:secondly, t:minutely, h:hourly, d:daily, b:business days, w:weekly, m:monthly).You can also use more detailed freq like 15min or 3h h
target Target feature in S or MS task (defaults to OT) OT
scalar scalar of the dataset StandardScalar
normalize Normalize 2

Get started

Requirements

Install the required package first:

cd REPO_skeleton
conda create -n REPO_skeleton python=3.8
conda activate REPO_skeleton
pip install -r requirements.txt

Dataset preparation

All datasets can be downloaded here. ett financial

The data directory structure is shown as follows.

./
└── datasets/
    ├── ETT-data
    │   ├── ETTh1.csv
    │   ├── ETTh2.csv
    │   ├── ETTm1.csv
    |   └── ETTm2.csv
    ├── financial
    │   ├── electricity.txt
    │   ├── exchange_rate.txt
    │   ├── solar_AL.txt

Quick start

This project provides highly customizable time-series model training backbone. If you want to configure your model manually, please refer to our example json config files under the path cfgs/, which will give you some insight about how to write a json config file. To use a custom dataset, you can put the dataset file under the datasets/ directory for better file management, then don't forget to specify the path in the config file. Remember to also install the dependencies, follow the steps here. After that, you just have to tell the program where to find your config file. Here is an example:

python run.py --cfg_file 'path_to_your_json_file.json'

However, if you want to try out with our provided examples, you can follow the steps below:

  1. Download our example dataset file from here.
    • In this demo, we use the ETTh1.csv dataset file under the path dataset/ETT-data/ETT/ in our provided link.
  2. Put the downloaded dataset file under datasets/ directory.
  3. Follow the steps in the requirements section to install the dependencies.
  4. Run the program with the following commands:
python run.py --cfg_file 'cfgs/exp/SCINet/SCINet_ETTh1_mult_s48h24.json'

The above demo trains with a SCINet model and a Electricity Transformer Temperature dataset.

  • Details can be found in the config file cfgs/exp/SCINet/SCINet_ETTh1_mult_s48h24.json.

Results

We have updated the experiment results of all models on 7 datasets.

SCINET RNN InceptionTime TransformerModel STGCN FNN AutoEncoder MLP gMLP FCN ResNet TCN OmniScaleCNN
Dataset Metric
ETTh1 MAE 0.387 0.390 0.433 0.413 0.470 0.374 0.498 0.530 0.405 0.490 0.459 0.416 0.398
MSE 0.358 0.353 0.401 0.383 0.500 0.345 0.520 0.568 0.379 0.497 0.451 0.398 0.363
RSE 0.534 0.530 0.564 0.552 0.631 0.524 0.643 0.672 0.615 0.629 0.599 0.563 0.537
CORR 0.796 0.797 0.774 0.775 0.701 0.801 0.693 0.654 0.779 0.707 0.733 0.753 0.789
ETTh2 MAE 0.279 0.271 0.278 0.269 0.313 0.252 0.349 0.419 0.260 0.338 0.303 0.269 0.267
MSE 0.177 0.165 0.169 0.163 0.228 0.150 0.251 0.388 0.158 0.246 0.206 0.166 0.161
RSE 0.446 0.430 0.436 0.428 0.505 0.411 0.531 0.660 0.422 0.526 0.480 0.432 0.426
CORR 0.738 0.737 0.730 0.739 0.666 0.750 0.665 0.610 0.757 0.650 0.672 0.744 0.744
ETTm1 MAE 0.223 0.238 0.217 0.232 0.256 0.216 0.255 0.304 0.209 0.232 0.219 0.217 0.219
MSE 0.119 0.124 0.113 0.124 0.150 0.114 0.134 0.184 0.106 0.117 0.108 0.116 0.118
RSE 0.363 0.372 0.354 0.371 0.409 0.351 0.385 0.452 0.343 0.360 0.346 0.146 0.362
CORR 0.828 0.804 0.825 0.803 0.779 0.815 0.798 0.740 0.835 0.805 0.829 0.822 0.809
ETTm2 MAE 0.397 0.408 0.386 0.418 0.382 0.378 0.367 0.448 0.378 0.402 0.390 0.214 0.402
MSE 0.412 0.448 0.400 0.455 0.326 0.383 0.337 0.479 0.391 0.410 0.405 0.118 0.436
RSE 0.572 0.596 0.564 0.601 0.574 0.551 0.517 0.617 0.557 0.571 0.567 0.352 0.589
CORR 0.788 0.771 0.797 0.755 0.774 0.791 0.825 0.765 0.805 0.786 0.792 0.825 0.769