business-science/modeltime

extract_nested_future_forecast() doesn't forecast Prophet w/ Regressors Properly

CharleyDublin opened this issue · 1 comments

I use a variety of models in a hierarchical time series forecast and the "best" model for a grouping variable doesn't show up in the future forecast. I am using XGBoost and Prophet and the Prophet models, when they are deemed "best", do not work.

Here are the steps:

  1. Run error test - No errors
    nested_modeltime_tbl %>%
    extract_nested_error_report()

  2. Select best model:

best_nested_modeltime_tbl <- nested_modeltime_tbl %>%
modeltime_nested_select_best(
metric = "mape",
minimize = TRUE,
filter_test_forecasts = TRUE
)

Here is the output:

Brno_1 2 XGBOOST Test 427.09197 4.808718
Budapest_1 3 XGBOOST Test 200.44970 2.969605
Frankfurt_1 1 XGBOOST Test 78.13787 4.246560
Munich_1 7 PROPHET W REGRESSORS Test 570.40167 9.995363
Prague_1 3 XGBOOST Test 284.27875 2.774745
Prague_2 2 XGBOOST Test 214.87820 3.509766
Prague_3 3 XGBOOST Test 166.21495 3.043598

  1. I fit the best models to the future data

nested_modeltime_refit_tbl <- best_nested_modeltime_tbl %>%
modeltime_nested_refit(
control =
control_nested_refit(verbose = TRUE)
)

  1. Future model

nested_modeltime_refit_tbl %>%
extract_nested_future_forecast() %>%
group_by(id) %>%
filter(id=="Munich_1")

0 Rows

The "Munich_1" data set doesn't show if the "Best" model is Prophet. It does, however, work if I only have XGBoost.