Quantifying uncertainity in forecasts
Closed this issue · 3 comments
UCSB-Parker commented
Hi I'm new to EDM. Is there statistical uncertainty associated with generating the parameters and/or making forecasts with EDM? If so, is it possible to quantify?
SoftwareLiteracy commented
This is a deeper question than I'm equipped to satisfactorily answer. I
think it has several facets. This is what I can muster at the moment:
One attempt is the `Pred_Variance` reported in the prediction output. In
the case of Simplex:
S = Simplex( dataFrame = sampleData["block_3sp"], lib = "1 99",
pred = "105 190", E = 3, columns = "x_t", target = "x_t")
S.head(5)
time Observations Predictions Pred_Variance
0 107 -0.212900 NaN NaN
1 108 1.140320 0.829999 0.071708
2 109 -1.152276 -1.690308 0.169022
3 110 0.585776 0.703370 0.091298
4 111 0.022673 0.208247 0.107307
Pred_Variance represents a variance estimate of the simplex at each
prediction point. Ethan Deyle expresses this as:
Σ [ w ( X_t - X_(t_n) ) ]^2 / Σ w^2
Here, w are the simplex weights, and X_t the observed and X_t_n predicted state
vectors [1].
[1] Sugihara, G., May, R. Nonlinear forecasting as a way of
distinguishing chaos from measurement error in time series. Nature 344,
734–741 (1990). https://doi.org/10.1038/344734a0
On 1/8/21 1:30 AM, UCSB-Parker wrote:
Hi I'm new to EDM. Is there statistical uncertainty associated with
generating the parameters and/or making forecasts with EDM? If so, is
it possible to quantify?
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#11>, or unsubscribe
<https://github.com/notifications/unsubscribe-auth/ABQGO5THPR7NZ7S37LHQLQLSY2RBTANCNFSM4V2AOISA>.
--
UCSB-Parker commented
Thank you! Does the same column exist for an sMap forecast?
SoftwareLiteracy commented
Yes. The same metric as simplex is used. It would perhaps be better to have one derived from the SMap linear system residuals, but this is not formulated/implemented yet.
>>> SM = SMap( dataFrame = sampleData["block_3sp"], lib = "1 99", pred = "105 190", E = 3, columns = "x_t", target = "x_t", theta = 3 )
>>> SM.keys()
dict_keys(['predictions', 'coefficients'])
>>> SM['predictions'].head(5)
time Observations Predictions Pred_Variance
0 107 -0.212900 NaN NaN
1 108 1.140320 0.840418 0.999999
2 109 -1.152276 -1.472882 2.209543
3 110 0.585776 0.600260 0.478918
4 111 0.022673 0.155858 0.645198