Strange predictions.

[asmodaay]

Hi! Do you have any ideas about strange shift in predictions? I mean the shape of prediction is pretty well and looks like target, but the distance between context and prediction is too big.

[qiyanjun]

@jakegrigsby can you check this?

[jakegrigsby]

Hmm, there's not a lot to go on here but possibly this is a normalization issue? Is that one of our datasets or are you trying your own? It doesn't seem normalized... or maybe it is normalized but is really non-stationary so that the validation data from the end of the timeseries is much larger than what was seen during training (common problem in domains where things generally go up over time... finance for example). That might lead to the correct shape but with predictions that are too low like you're seeing here. We do have an extra trick you can try to deal with this:

spacetimeformer/spacetimeformer/spacetimeformer_model/spacetimeformer_model.py

Line 48 in 1ee8c81

linear_window: int = 0,

We have this "linear_window" concept that we don't talk about in the paper draft because we don't use it on those datasets. Basically it uses a linear combination of linear_window points to create an offset for the rest of the predictions. This lets the Transformer learn the shape while the linear model learns the starting point.

spacetimeformer/spacetimeformer/forecaster.py

Lines 108 to 125 in 1ee8c81

	def forward(
	self,
	x_c: torch.Tensor,
	y_c: torch.Tensor,
	x_t: torch.Tensor,
	y_t: torch.Tensor,
	**forward_kwargs,
	) -> Tuple[torch.Tensor]:
	preds, *extra = self.forward_model_pass(x_c, y_c, x_t, y_t, **forward_kwargs)
	baseline = self.linear_model(y_c)
	if isinstance(preds, Normal):
	preds.loc = preds.loc + baseline
	output = preds
	else:
	output = preds + baseline
	if extra:
	return (output,) + tuple(extra)
	return (output,)

I put this in because LSTNet does something a lot like this and I wanted to see how much it mattered. The answer was not much, at least on the paper datasets. But it did help a lot on an exchange rates dataset that is non-stationary.

To summarize: make sure you are normalizing the data based on training set statistics, and then try --linear_window 12 or 24.

[asmodaay]

Thank a lot for your advice, i will try it and return with feedback. This is really non-stationary financial data with a high growth rate, but I had same issue with your exchange dataset but in a less degree.

[ebuss81]

Hey everyone,
first, many thanks to all your work. Its great!
I probably have a comparable question. My data is highly non-stationary and noisy sucht that the predictions are only a straight line. Can we assume that the high non-stationarity and noise of the data is probably the main cause?

Best
Eddy

[jakegrigsby]

Hey @ebuss81 , that's possible. That's also much noisier data than I think I've ever tested on, but I would've hoped for a better result. Any chance you can email me this dataset so I can look at this?

[meerahshivn]

Hello, i'm a beginner on this field and this project seems very interesting to explore for my research. Well, using asos dataset, I obtained result as shown in the image without the error computation MAPE, is it something hard-coded and required to change or is it related any other issues dependency of version...please do advise me...