Issue with frequency since pandas 2.2.0 in the DeepAREstimator
vm3538 opened this issue · 4 comments
vm3538 commented
Description
After updating pandas from 2.1.4 to 2.2.0, I get exceptions because of the frequency.
To Reproduce
Train a DeepAREstimator
import pandas as pd
import torch
from gluonts.dataset.pandas import PandasDataset
from gluonts.dataset.split import split
from gluonts.torch import DeepAREstimator
df = pd.read_csv(
'AirPassengers.csv',
index_col=0,
parse_dates=True,
)
dataset = PandasDataset(df, target="#Passengers")
training_data, _ = split(dataset, offset=-36)
model = DeepAREstimator(
prediction_length=12, freq="M", trainer_kwargs={"max_epochs": 5}
).train(training_data)Error message or code output
//trainer/minimal_code.py:1: DeprecationWarning:
Pyarrow will become a required dependency of pandas in the next major release of pandas (pandas 3.0),
(to allow more performant data types, such as the Arrow string type, and better interoperability with other libraries)
but was not found to be installed on your system.
If this would cause problems for you,
please provide us feedback at https://github.com/pandas-dev/pandas/issues/54466
import pandas as pd
/usr/local/lib/python3.10/site-packages/gluonts/time_feature/_base.py:243: FutureWarning: 'M' is deprecated and will be removed in a future version, please use 'ME' instead.
offset = to_offset(freq_str)
/usr/local/lib/python3.10/site-packages/gluonts/time_feature/lag.py:104: FutureWarning: 'M' is deprecated and will be removed in a future version, please use 'ME' instead.
offset = to_offset(freq_str)
Traceback (most recent call last):
File "//trainer/minimal_code.py", line 20, in <module>
).train(training_data)
File "/usr/local/lib/python3.10/site-packages/gluonts/torch/model/estimator.py", line 246, in train
return self.train_model(
File "/usr/local/lib/python3.10/site-packages/gluonts/torch/model/estimator.py", line 165, in train_model
training_network = self.create_lightning_module()
File "/usr/local/lib/python3.10/site-packages/gluonts/torch/model/deepar/estimator.py", line 382, in create_lightning_module
return DeepARLightningModule(
File "/usr/local/lib/python3.10/site-packages/gluonts/core/component.py", line 364, in init_wrapper
return init(self, **all_args)
File "/usr/local/lib/python3.10/site-packages/gluonts/torch/model/deepar/lightning_module.py", line 59, in __init__
self.model = DeepARModel(**model_kwargs)
File "/usr/local/lib/python3.10/site-packages/gluonts/core/component.py", line 364, in init_wrapper
return init(self, **all_args)
File "/usr/local/lib/python3.10/site-packages/gluonts/torch/model/deepar/module.py", line 140, in __init__
self.lags_seq = lags_seq or get_lags_for_frequency(freq_str=freq)
File "/usr/local/lib/python3.10/site-packages/gluonts/time_feature/lag.py", line 149, in get_lags_for_frequency
raise Exception("invalid frequency")
Exception: invalid frequency
Changing frequency to ME won't solve the issue, since it is not supported in gluonTS.
//trainer/minimal_code.py:1: DeprecationWarning:
Pyarrow will become a required dependency of pandas in the next major release of pandas (pandas 3.0),
(to allow more performant data types, such as the Arrow string type, and better interoperability with other libraries)
but was not found to be installed on your system.
If this would cause problems for you,
please provide us feedback at https://github.com/pandas-dev/pandas/issues/54466
import pandas as pd
Traceback (most recent call last):
File "//trainer/minimal_code.py", line 20, in <module>
).train(training_data)
File "/usr/local/lib/python3.10/site-packages/gluonts/torch/model/estimator.py", line 246, in train
return self.train_model(
File "/usr/local/lib/python3.10/site-packages/gluonts/torch/model/estimator.py", line 165, in train_model
training_network = self.create_lightning_module()
File "/usr/local/lib/python3.10/site-packages/gluonts/torch/model/deepar/estimator.py", line 382, in create_lightning_module
return DeepARLightningModule(
File "/usr/local/lib/python3.10/site-packages/gluonts/core/component.py", line 364, in init_wrapper
return init(self, **all_args)
File "/usr/local/lib/python3.10/site-packages/gluonts/torch/model/deepar/lightning_module.py", line 59, in __init__
self.model = DeepARModel(**model_kwargs)
File "/usr/local/lib/python3.10/site-packages/gluonts/core/component.py", line 364, in init_wrapper
return init(self, **all_args)
File "/usr/local/lib/python3.10/site-packages/gluonts/torch/model/deepar/module.py", line 140, in __init__
self.lags_seq = lags_seq or get_lags_for_frequency(freq_str=freq)
File "/usr/local/lib/python3.10/site-packages/gluonts/time_feature/lag.py", line 149, in get_lags_for_frequency
raise Exception("invalid frequency")
Exception: invalid frequency
Environment
- Operating system: debian
- Python version: 3.10
- GluonTS version: 0.14.3
- MXNet version: -
- Pandas version: 2.2.0
lostella commented
@vm3538 thanks for filing this! After a quick look, I think we may need to fix this utility function that is used internally to normalize frequency strings. The case ending in E probably needs to be added, and we should check that tests cover this.
pantanurag555 commented
Aliases deprecated in pandas v2.2: https://pandas.pydata.org/docs/dev/whatsnew/v2.2.0.html
Will need to add a version check in the utility function before normalizing.
lostella commented