Nixtla/hierarchicalforecast

[FEAT] Explore AnyDataframe (spark, ray, and dask) integration

Opened this issue · 1 comments

Currently, hierarchicalforecast only supports a pandas dataframe as input. For the library to scale horizontally, we need to explore different alternatives on how to integrate frameworks such as spark, ray, and dask. This issue is intended to discuss possible approaches.

The main problem right now is that hierarchicalforecast takes a dataframe and then converts it to numpy arrays that are fed to the reconciliation methods. Because of the characteristic of the hierarchical reconciliation problem, those numpy arrays must be used at once in each reconciliation method.

Any ideas?