`MinTrace` fails for zero-inflated Time Series with `res_methods` in `['wls_var', 'mint_cov', 'mint_shrink']`
baniasbaabe opened this issue · 2 comments
What happened + What you expected to happen
Applying MinTrace
with a res_method
will fail due to the following error:
ValueError: Insample residuals close to 0, zero_residual_prc=1.0. Check
Y_df``
How can you overcome this? It's probably because some time series has many zeros.
Versions / Dependencies
0.3.0
Reproduction script
# !pip install -U numba statsforecast datasetsforecast
import pandas as pd
# compute base forecast no coherent
from statsforecast.core import StatsForecast
from statsforecast.models import AutoARIMA, Naive
#obtain hierarchical reconciliation methods and evaluation
from hierarchicalforecast.core import HierarchicalReconciliation
from hierarchicalforecast.methods import BottomUp, TopDown, MiddleOut, MinTrace
#obtain hierarchical datasets
from datasetsforecast.hierarchical import HierarchicalData
# Load TourismSmall dataset
Y_df, S, tags = HierarchicalData.load('./data', 'TourismSmall')
Y_df['ds'] = pd.to_datetime(Y_df['ds'])
Y_df['y'] = 0.0
# Compute base level predictions
sf = StatsForecast(df=Y_df,
models=[AutoARIMA(season_length=12), Naive()],
freq='M', n_jobs=-1)
forecasts_df = sf.forecast(h=12, fitted=True)
# Reconcile the base predictions
reconcilers = [
MinTrace(method="mint_shrink")
]
hrec = HierarchicalReconciliation(reconcilers=reconcilers)
reconciled_forecasts = hrec.reconcile(Y_df=sf.forecast_fitted_values(), Y_hat_df=forecasts_df, S=S, tags=tags)
Issue Severity
Medium: It is a significant difficulty but I can work around it.
Hey @baniasbaabe,
You might want to force it using a bit of Gaussian noise data augmentation.
See the FavoritaHierarchicalDataset.load_item_data example:
https://github.com/Nixtla/hierarchicalforecast/blob/main/experiments/hierarchical_baselines/nbs/run_favorita_baselines.ipynb
We have specialized hierarchical methods on zero inflated processes here:
https://nixtla.github.io/neuralforecast/examples/hierarchicalnetworks.html
The paper of a HINT-related method recently got accepted in the International Journal of Forecasting.
Kin G. Olivares, O. Nganba Meetei, Ruijun Ma, Rohan Reddy, Mengfei Cao, Lee Dicker (2023).”Probabilistic Hierarchical Forecasting with Deep Poisson Mixtures”. International Journal Forecasting, accepted paper. URL https://arxiv.org/pdf/2110.13179.pdf.
Many thanks for the fast reply, I will look into it!